Elin and I looked at R041 and found no-green light at its connection point to i17.bfb, and an orange light on the other end of the connection to B13. We reseated the end connected to B13 with no change, then reseated the connection to i17.bfb, and that light came on green, and the light on the B13 end went from orange to green.
Will be watching the connection to make sure the cable is still working over the next few months.
Changing to hardware.
Called Chris G and asked about heat dissipation rates.
Switching from bug to feature request
I believe this used to be gharp9a and gharp9b, so I have formatted and remounted this drive as 2 1.4TB partitions as /gharp9a and /gharp9b and created files on each partition to identify them as gharp9a/gharp9b.
I've also re-instated a cronjob on tumulus that will keep the disk "awake" by doing some quick file operations on each partition once every 2-3 minutes.
The directories are owned by obs:obs
In reply to M Huff's test report (on SRI's bug tracking system?)
Using a few high watt resistors (0.2 Ohms/ea), we were able to load the power
supply unit at various currents. The power supply failed or shutdown at 25A
and 12.5A loads, far below the unit's spec. At 8A, the supply seemed to
be pretty stable, delivering 4.93V to the load.
I don't recall another BEE2 having this exact same problem but I do recall a BEE2
that had very poor crimping at the end of the ~8 or 10 AWG copper cable from the
PSU to the connector that mates to the PCB. There was so much resistance that
the voltage delivered to the PCB was much less than that output by the PSU that
the PCB would crash and then reboot itself or hang while the PSU output stayed at 5.0 VDC.
This doesn't seem to match the behavior with BEE2 2.2.26 especially if the load resistors
were directly connected to the PSU's output posts but maybe useful for future debug efforts...
Has the PSU's internal fan died ?
A different BEE2 had a bulging cap C5 on the middle board called
"BEE2 Power Supply HUB Board V2"
That cap is labeled : NIPPON CHEMI_CON 200V 1000uF 680068 SMQ
The BOM calls out :
Digikey 565-2738-ND ERSMQ201VSN102MR30S
Now Digikey 565-2738-ND maps to ESMQ201VSN102MR30S
Newark had them in stock back in Feb 2011and called them 16M8462.
That 1 cap, C5, was replaced with a new CAP (new part from Newark) and the PSU and BEE2
has been AOK since.
Even if the ripple reducing caps haven't died maybe the inverter itself has died ?
this would be a first that I've heard of.
Watch out : 300 V internal voltages - probe w/ care!
sorry for these rather basic or generic thoughts.
OK by me to delay trip to HCRO until Feb 21-24 to allow time to plan for a more extensive
BEE2 debug session.
Or even the following week.
goal of travel this week was to get FB heatsink rework/experiment off my todo list.
Someone else needs to set priorities ...
Right now, what would make the most sense would be to come to HCRO at the same time that Matt Dexter is here. He is planning on making a trip up Feb 15-17th. Confirming with him now. Understandably short notice. Or, a visit between Matt and SRI in the Bay Area.
I'd be interested in assisting in the testing an repair with Matt Dexter if schedules permit. I think there is a lot that I could learn from troubleshooting this component him. I'm sure Dennis would like to join as well.
Added SRI/SETI/et al to notify list.
Conferred with Mike Huff today about further testing of b14 and recommended he keep b14 until I have a CX4 cable sent down so that he can run the exhaustive self tests. Dan C points out further testing can also happen using laptops to allow b14 to boot.
Open question on ata-staff list is should b14 stay at Menlo Park for further testing and repair or return to HCRO ASAP for Matt Dexter to take over testing and repair.
Please open new tickets for instead of appending to older tickets opened for other devices/antennas.
2m seemed functional the same day (dec 5th) and was drivable. 3d appears to also be fine as of 1337 PST.
Where are these offline message coming from?
Ant 3d went offline on Tuesday, 12-13.
2011-12-05 4 antennas were offline when SonATA started up. 2a,2e,2m,3d.
Issued a recover and all came back except 2m.
2011-12-08 2 antennas were offline when SonATA started up 2a and 2e.
Both came back when recover was issued.
Note, those are warnings. Whatever failed wasn't because of the warnings... What extension was on the filename?
I'm unable to see how to attach a couple of JPGs to this case but if I could I would reference 2 JPGs Colby took for me showing how 1 of the PCBs internal to the 1.1.17 PSU which had been powered BEE2 2.2.17 aka B10 show marks that an overly long screw was used to hold the chassis lid to the BEE2 chassis and ran into the PSU. This can cause trouble.
during the afternoon of 2011nov30 after a bunch of bench testing BEE2 2.2.19 aka B16 with the internal PSU 1.1.17 was re-installed back into the BF system. And as far as I know it has been running AOK since then.
See the ata-staff emails with subject "BEE2 2.2.19 (aka B16) with internal PSU 1.1.17" for more information
Notes on modified software to work with new ibob arrangement:
Please do this as soon as possible, I may not be able to get to replacing the fan before tomorrow if the test isn't run soon.
Hmm, if you can tell me which board it was last night, I could go ahead and switch out the fan on it. Quite possibly if it is not hot today, it's because the correlator isn't running.
Please try taking some data with the correlator to force it to run hotter so that I can replace this today before I go on vacation tomorrow.
You are correct. I've been swapping those numbers in my mind all day. Thanks for correcting me. I will now attempt to edit my original post.
Your last post has me confused. It seems to make perfect sense if "b10" is swapped with "b16" --- since b10 was the flakier of the two, and I think we decided that b10 was the one more important to get running again (and b10 seems to be up while b16 is down).
Removed b10 and b16 from rack.
Ran both on the bench. Each started up with no problem. After about 5 minutes, b10's chip fans (5volt) all stopped and all the LEDs on the board went out. The chassis fans (12volt) remained running. I cycled the power switch and the unit appeared to start up and run, although the LEDs appeared to dim a bit. I measured the 5v output of the PS and it was a poorly regulated voltage of about 4.7v to 4.9v.
I then measured the 5v output of the power supply in b16 and it was a rock solid 5.002v.
I removed the PS from b10 and opened it up to check for obvious problems. The capacitors all looked good visibly and measuring with a ohm meter they all showed no shorts. There were no other obvious visual abnormalities.
After discussing this with Colby, it was determined that it was more important to get b10 up than b16. It was decided that our best option at this time was to remove the PS from b16 and install it in b10 even though it also was somewhat suspect.
To document the swap:
Power supply s/n PS1.1.19 removed from b16 (s/n 2.2.19).
Power supply s/n PS1.1.19 installed in b10 (s/n 2.2.17).
Power supply s/n PS1.1.17 removed from b10 (s/n 2.2.17) and labeled as suspect.
As I write this, b10 has been in the rack and running for about 1.5 hours.
As far as I know the maximum current measured for a heavily loaded BEE2
was 5 VDC @ 65 amps. Thus, the 80 amp internal PSU is conservatively sized.
The ATA Beamformer applications may be well under the 65 Amp figure.
The maximum value of my true power measures
are about 150 watts and 272 volt-amps at the VAC input supply.
If all of this were on the 5VDC supply, and ignoring startup inrush sorts of issues and so on,
then a supply that could output 5VDC @ 40amp may be OK.
2011nov17 notes on BEE2 power supplies
FYI one can get access to the entire BEE2 community
including the actual engineer that designed the chassis
and PSU via the email list at email@example.com
more info at https://casper.berkeley.edu/wiki/BEE2
Internal PSU description from the unit's designer
The unit provides two primary voltages: 80A at 5V, and 10A
at 12V. A secondary section provides aux/standby power of
3A at 5V and 3A at 12V, but is NOT intended for functional
usage aside from low wattage control, etc. Total output
power is 550W all driven, with an nominal efficiency ranging from
86% down to 82% (worst case). Thus, the input power requirement
is 670W (we use 750W). The mains are fully isolated,
filtered, and transient protected in two separate stages
(detailed data later). Independent soft control for both 5V and 12V
is available remotely and optically isolated to 1KV.
The unit has an input ranges of 90-130VAC and 180-260VAC
(automatic switching) at freq of 47-63 Hz; max inrush 30A.
We do not anticipate to be UL or FCC certified,
however the design parameters exceed all requirements
for the above, especially with respect to grounding and
isolation industrial equipment (UL,CSA,IEC Class X1/Y1
Bench testing of the prototype was preformed primarily
to confirm initial design goals, and limited to lower
current ops mostly to characterize turn-on behavior and
Regulation under load was confirmed to less than spec'ed
0.2% (no attempt to measure at higher precision);
5.00V at no load, 5.00V at 30A
Vrms ripple was 24mV, 14mV, and 5.8mV at 10A,
20A, and 30A respectively. Vpp was in the 100mV
range max and less than 1% duty cycle (by eye). Vpp
declines proportionally with Vrms under load. The
fundamental was about 60kHz, and no 60 Hz residual
was evident in the trace. This exceeds the performance
of all of our commercial laboratory supplies by about
a factor of two, including the premium Lambda and Xantrex
The power supply had about a 10A load start-up limitation
based upon automatic shutdown of the rectifier
and cycling of the restart at a few Hz. (This
condition we will not be able to duplicate with
the BEE2 since it requires already-running FPGAs.)
Essentially, this is how much you can draw as the
regulation is coming up, which is not a normal scenario
for a power supply. However, I probably will adjust
the turn-on time constant to not pick up load for
another 500ms or so.
Once stabilized, application of a transient load, i.e.
loading up to 60A, caused no detectable effect.
Thermally, the unit is fully self-sufficient (most
built-in supplies sink to the frame) and is essentially
over-cooled at 12-14 CFM with little detectable delta T
while shedding 40W.
Form-factor wise, the unit is 2U tall, about 15" long,
requires 15CFM, and provides high quality smoothed power
at about 2.75W per cubic inch at and average 85% efficiency.
For 120 VAC operation a 7amp fast blow fuse is typically
For 240 VAC operation
again a quote from the PSU designer
I have been telling people 3.5A, but 4A is fine as well,
perhaps even better. I don't have any means to measure
inrush at 240VAC, but I suspect we are fine with either.
I have been specifying fast blow fuses for normal operation,
so is suggest trying that first. Let me know if any issues
arise and we can consider larger/slower fuses.
Vendor: SAE Materials, Inc.
340 Martin Avenue
Santa Clara, CA 95050
Phone: 408 492 1784
Fax: 408 492 1505
VP of Operations
Cell: 408 828 6961
my last order was in late 2007 for
1) Internal power supply rev1 and labor 8 $1,084.91 $ 8,679.28
At the end of 2007 and early 2008 Whizz Systems was suposed to
take over the fabrication and assembly of BEE2s, including
chassis and internal PSUs:
3140 Alfred Street
Santa Clara Ca 95054
They never replied to my request for quotes. I think this was
because they were only interested in orders of 10 or more complete
My last order for an external PSU was 2009apr14
14 High Bridge Road
Sandy Hook, CT USA 06482
Tel: 203 270-4700
Direct: 203 270-4736
Fax: 203 270-4799
Digikey MP6-3E-1L-1L-00 $1.6K (higher end part)
Newark MP6-3E-1L-00 34C9141 61 day leadtime: $870.72
description P/N Quantity price price
Both power supplies:
universal VAC input (120/240, 43-60Hz)
5VDC @ 120 amp, 12VDC @ 17 amp
regulation 0.4% or 20mVworst case.
ripple 0.1% or 10mV rms
1% or 50mV ptop
efficiency 70-80% at full load;
fuse internal. MP6 15A
New power supply Astec MP6-3E-1L-00 1 $650.00 $650.00
used power supply Astec MP6-3E-1L-00 1 $300.00 $300.00
PSU to PCB (see remote sense lines below)
As originally specified
Vendor McMaster Carr
12 VDC, 8 AWG stranded, 9697T6 N $1.80 $N*1.80
buy by the foot, 2 conductor,
copper, PCV insulation, black and red
10mm Mini-Fit Sr. 538-42816-0212 200 $0.71 $142.00
10mm Mini-Fit Sr. 538-42815-0031 100 $0.43 $43.00
An alternative power cable can, and has, been made with
Vendor HSC Electronic Supply
500ft reel BLACK 10AWG BLACK JSC Wire 1660 2 $194.00 $388.00
413/36 stranded cu wire
500ft reel RED 10AWG RED JSC Wire 1660 2 $194.00 $388.00
413/36 stranded cu wire
10mm Mini-Fit Sr. 538-42815-0011 400 $0.33 $132.00
PSU to fan and PCB remote sense to PSU
12 VDC, 24 awg pair of wires 9697T1 N $ 0.19 $ N*.19
buy by the foot
Vendor HSC Electronic Supply
100ft BLACK 20AWG Consolidated Wire 820-0-100 1 $ 9.49 $ 9.49
10/30 stranded cu wire
100ft RED 20AWG Consolidated Wire 820-2-100 1 $ 9.49 $ 9.49
10/30 stranded cu wire
PSU +12VDC side
1x4 5.08mm pitch Mouser: 538-15-24-4048 10 $0.72 $7.20
socket conn housing
Socket terminal for Mouser: 538-02-08-1202 100 $0.15 $15.00
above conn housing
chassis fan side
1x4 5.08mm pitch Newark: 82F9868 10 $0.385 $3.85
plug conn housing
Bag of 100 plug terminals Newark: 82F9911 1 $14.08 $14.08
for above conn.
14-20 AWG wires.
Please ship them up to HCRO. Thx!