MRO Site Trips and Outages
This table has been updated so the most current/recent changes are at the top.
Any issues or queries please send to operations@lists.mwatelescope.org
Forecast Outages
| Status | Date and Time (AWST) of Outage | Date and Time (AWST) of Latest Update | Description | Affected Projects | Reason | Notes |
|---|---|---|---|---|---|---|
CLOSED | SUNDAY 0900-1600h AWST 16 April 2023 | 28 Feb 2023 | MRO Community Open Day | All MRO instruments and equipment | Community engagement | Members of the Murchison community will be invited and escorted onto the observatory. RFI from photography, radio communications, vehicles and other equipment will be produced. |
CLOSED | 0800h 27 Mar - 1200h 31 Mar 2023 | 0530h 07 Feb 2023 | HV electrical maintenance MRO | All equipment and telescopes on the MRO | Electrical maintenance of HV equipment | |
CLOSED | 0700-1700h AWST 8-9th March 2023 | 28 Feb 2023 | Drainage repair works on the MRO control building | All equipment and instruments on the MRO | Remove water build up in the control room sub floor spaces | This activity is further maintenance work following the burst water pipe event Xmas 2021. |
CLOSED | 0700-1700h AWST 07-20 Mar 2023 | 27 Feb 2023 | UAV LiDAR surveys of SKAO locations | All projects within the Curtin Site Licence Area | Possible RFI from UAV | Surveyors will be conducting UAV LiDAR surveys of the northern spiral arm during these dates. The UAV will be flying below 200m and operating in the 2.4GHz frequency. The UAV is long range, battery operated remote piloted aircraft. RFI is possible from the operation of the platform, vehicles onsite and pilot radio communications in the VHF band. |
CLOSED | 0700-1700h AWST 27th March - 02 April 7-14th April | 28 Feb 2023 | EDGES ground plane welding activity | All projects within the MRO | Possible RFI form welding activities | EDGES personnel will be conducting welding activities on the EDGES ground plane. Activity will be conducted across weekends and public holidays. |
CLOSED | Unknown day timings. Flights will occur any day between 16-20th Jan | 1710h 13 Jan 2023 | LIDAR survey of SKA arms | All projects ont he Curtin site licence area | Low flying helicopter will produce significant RFI. | The flight crew for the survey will advise when the flight route for the northern arm spiral will approach the Curtin Site Licence Area. The flight crew will provide two hours notice. In that two hours - all telescope will be powered down. Expected flight time is less than one hour. Exact timings will not be known as the flight is significantly weather dependent. Flights will only occur in daylight hours. |
CLOSED | 0800h 13 Jan - 2000h 16th Jan | 0820h 13 Jan | HV maintenance works at Pawsey. | All MRO instruments and equipment | HV maintenance | All comms links between Pawsey and MRO will be offline for the duration of the outage. |
CLOSED | 31 Oct - Nov 04 | 26 Oct | No outage. EDGES team conducting groundplane repairs and improvements that will involve using welding eqpt | All MRO instruments and equipment | ||
CLOSED | 02 Aug | 0730-1600h | RFI emissions testing of a shielded room at the MRO power station | All MRO telescopes | Critical emissions testing of a container in order to support ongoing SKA construction planning. | Discussions between MWA, AAVS and ASKAP have identified - at short notice - that 02 Au was a suitable day for all projects to support this activity |
CLOSED | 2-4 May 2022 | 23rd Mar 2022 | Kalli Cattle Muster | All MRO telescopes | Helicopters and / or spotter planes, vehicles, motor bikes and UHF radios will be in use during daylight hours 2-4 May inclusive | |
CLOSED | 26-29th April 2022 POSTPONED - FUTURE DATES TBC UPDATE: New dates:13-23 June | 01 June | Ongoing repair work to the Control Building | All MRO teelscopes | These works will require the RFI doors to be open between the hours of 0730h – 1700h daily. Equipment used will be water pumps and grinding / cutting equipment | |
CLOSED | 19-29th April 2022 POSTPONED - FUTURE DATES TBC UPDATE: New dates:13-23 June | 01 June 2022 | MRO Access road maintenance | All MRO telescopes | Heavy machinery, UHF radios and a heavy duty diesel water pump will create EMI onsite during daylight hours 19-29th April inclusive | |
CLOSED | RFI effects only 01-03 Feb 2022 | 1700h 27 Jan 2022 | C-building maintenance and repairs | All MRO telescopes. | Repair works are being undertaken on the c-room sub floor water leak. The floor will be lfited in sections to remove the water and ventilate the sub-floor. Doors will be open in the c-building 0700h-1700h daily | Over the Xmas break a significant subfloor water leak occurred in the c-building. A significant amount of water is trapped in the c-building sub floor and needs to be removed by pumps. The sub floor space needs to be ventilated and dried. |
CLOSED | 28 Feb - 11 Mar | 04 Jan 2022 | Reconfiguration of MWA telescope from compact to extended observations. | MWA | ||
CLOSED | 10-14, 17-21 Jan | 04 Jan 2022 | Upgrades to MWA telescope | All MRO instruments | To reduce the effects of the extreme heat common to the Murchison at this time of the year, the upgrade program will start 0500h Tues - Fri. This may impact on early morning observations . | |
CLOSED | 28th June - 16 July | 24 Mat 2021 | Routine site trip | Nil | Nil | |
CLOSED | Tues 29th - Wed 30th June | 24 May 2021 | HVTX maintenance | All instruments and equipment within the Curtin Site Licence area. Any equipment within the corrleator room will not be effected. | Annual safety maintenance | During the dates specified, all power will be shutdown to the Curtin site licence area. The High Voltage Transformer (HVTX) will undergo annual safety inspection. |
CLOSED | 31 May - 13 July | 21 Jun 2021 | Documentary film production | AAVS2, EDA2 and MWA | During the dates specified, three separate film crews will be undertaking filming of different aspects of instruments and equipment within the Curtin site licence area and ASKAP. Filming within the AAVS / EDA2 instruments will be daytime only. Filming within the MWA area will be at night. During daylight hours, AAVS2 and EDA2 arrays will be powered down in order to prevent any possible damage to instruments and equipment during filming. UPDATE 31 May Over the last three days, significant levels of rainfall have been received within the Murchison area. All access roads between Geraldton and the MRO are now closed. It is unknown when the roads will be open again. Consequently, the filming activities have not been able to commence. At this stage, CSIRO and film crews are waiting for road access conditions to change before making decisions about when / if filming will take place UPDATE 04 Jun The film crews were able to make it to site effective 04 Jun. Filming will commence between 04 - 13 Jun. AAVS2 , EDA2 and MWA will remain powered down whilst filming and drone flights take place. Filming will be night and day. UPDATE 21 Jun 2021 Due to poor weather, the majority of night filming activities did not occur. The filming activities have been rescheduled for the following dates:
| |
CLOSED | 23rd April | 1400h 29 April | Power outage in C-room | All MRO instruments | The 23rd May correlator room outage was triggered by a release of fire suppression gas. The gas release triggered a power shutdown. The sudden loss of load on the power generation facility resulted in the power generation facility shutting down for approximately one second. The power generation facility came back online via diesel generators. CIRA operations staff have not identified any damage to equipment or instruments as a result of the rapid power cycle. However, all equipment in the correlator room within the Curtin racks is not protected by a gaseous fire suppression system in the event of an actual fire. Therefore, as a safety measure ALL equipment and instruments within the correlator room will be powered down by 1600h WST 29 April. It is expected that all equipment will remain offline to at least the 10 May. UPDATE MWA correlator will remain offline pending the outcome of the causes of equipment failure. All other instruments and equipment can continue to operate as normal | |
CLOSED | 05 Jul - 13 Aug | 21 Jun 2021 | Maintenance to Boolardy Station Boundary fencing | All MRO instruments | repair / replace degraded fence line in order to control animal movements on Boolardy station |
The yellow line on the above map indicates the section of fencing involved in the maintenance activity. The red line indicates the boundary of the MRO within Boolardy Station. It is a considerable length of boundary fence line (probably close to 80km) with Meerberrie station and will take 2 months to complete. Works will occur in in daylight hours only. Works will involve the use of petrol driven equipment such as chainsaws, jackhammers and pole drivers. Tractors, trucks and utility vehicles will also be used to move personnel and materiel. All instruments may observe spurious RF emissions during daytime observations. UPDATE 24 May 2021 Originally these works were due to commence mid May and be concluded by Late Jun. the works have yet to commence and have been rescheduled for dates post June. New dates will be forwarded as soon as possible. UPDATE 21 Jun 2021 The fencing works will commence 05 Jul and are expected to be completed on or about 13 Aug. |
CLOSED | Mon 19 - Wed 21 Apr | 29 Mar 2021 | Inspection of vermin damaged FOBOT at Curtin fibre aggregation point | All instruments except AAVS2 and EDA2 | Inspection and assessment | Vermin have entered FOBOT#2 of the main trunk cable between the Curtin fibre aggregation point and the correlator building. The vermin have damaged several fibre connections within the FOBOT. In order to remove and inspect the FOBOT, all fibre optic cables within the cabinet need to be disconnected and removed. It is assessed that the outage will take 2-3 days. |
CLOSED | Mon 19 Apr - Fri 23 Apr | 29 Mar 21 | MRO Maintenance site trip | All Curtin Site area instruments | General maintenance | Routine maintenance activities are planned for Apr 19-23. All activities will occur during daylight hours only. Spurious RFI may be observed by instruments. |
CLOSED | Fri PM Mar 19- Thurs PM Mar 25 | 15 Mar 21 | MRO Power outage and cooling system shutdown | All MWA, SKA and external instruments | Repairs to chiller#3 in correlator room damaged during power outage Dec 22; repairs to SESKA power supply system | Mar 22-26 maintenance works will be undertaken on the MRO correlator cooling system and the MRO power station. During this period all instruments and equipment must be powered down and not used. In order to protect equipment from any power surges or temperature fluctuations, all site equipment will be powered down PM Fri 19 Mar. All site circuit breakers will be flicked and rack equipment will be isolated at distribution boards. Equipment will be powered back up Thurs PM 25th Mar. |
CLOSED | 1100h 21 Dec 2020 | 1600h 22 Dec 2020 | MRO site wide power outage | All MWA, SKA and external instruments | Unknown | Update 22 Dec 2020 The source of the power outage was a blown EM filter on a LV supply that provided power to the correlator room. The correlator room lost power and some HV breakers tripped across the site. Fortunately, the Curtin site area did not lose power. Only equipment within the Curtin racks of the correlator room lost power. All MWA, external instruments, AAVS and EDA2 field equipment did not lose power and a quick remote check has identified no obvious damage. It is recommended all instrument managers check their field instruments and advise of any concerns to operations@lists.mwatelescope.org Correlator room equipment and machines are now being brought online. Should anyone have any concerns or issues with their equipment in the correlator room please email operations@lists.mwatelescope.org. There will be no further updates regarding this power outage. Update 21 Dec 2020 As at Australian WST1100h 21 Dec 2020, power was lost to all instruments, telescopes and equipment at the MRO. On site CSIRO staff were unable to identify the cause of the fault. Tomorrow 22 Dec 2020, electrical contractors and other CSIRO staff will travel to the MRO to support the repair effort. Next update: WST 1800h 22 Dec 2020. |
CLOSED | 07-11 Dec 2020 | 03 Nov 2020 | Site wide HV electrical maintenance and Cooling system maintenance in the correlator room | All MWA, SKA and external instruments | Scheduled maintenance and fault repairs | Dec 7-11, 2020 CSIRO will be conducting HV electrical maintenance works and cooling system maintenance in the correlator room. Power supply to all instruments and equipment within the Curtin site area will be cut approximately midday Monday 7th Dec and restored ~midday 11 Dec, 2020. All equipment and instrument managers are to safely shutdown their equipment prior to midday Monday 07 Dec 2020. CIRA Operations staff will be onsite MRO to assist in the local management of any equipment. |
CLOSED | TBA by separate email on occurrence | 0730h 22 Sept 2020 | MWA maintenance operations on site | All MWA, SKA and external instruments | MWA staff will be onsite MRO 21 Sept - 03 Oct inclusive. Operations staff will be conducting routine maintenance tasks on all telescopes within the Curtin site licence area. Impact on equipment and instruments will be minimal. RFI will be experienced during daytime hours as a result of the activities of staff on site. Individual telescopes will be without power during small periods of time as a safety precaution for technicians only | |
CLOSED | Nil | 1300h 28 aug 2020 | MWA operations site trip | All MWA, SKA and external instruments | MWA staff will be onsite MRO 30 Aug - 10 Sept inclusive. Operations staff will be conducting routine maintenance tasks on all telescopes within the Curtin site licence area. Impact on equipment and instruments will be minimal. RFI will be experienced during daytime hours as a result of the activities of staff on site. | |
| CLOSED | Ongoing from 23 July 2020, 1500h | 26 Aug 2020, 1145h | Loss of power in correlator room | All MWA, SKA and external instruments and equipment within the MRO correlator room | Unknown - under investigation | The MRO Correlator room will remain without power for an unknown period of time. The cause and significance of the power outage remains under investigation. The next information update will occur before COB Wed 29th July. Should you have any questions or concerns, please contact operations@lists.mwatelescope.org. Please do not contact CSIRO / MRO staff directly. The staff members are very busy and are cognisant of the implications of the power outage. UPDATE: 30 Jul 2020 The power outage in the correlator room was caused by the activation of fire suppression systems. The fire suppression was activated 23 July in response to a small component failure on a PCB that emitted particulate and smoke into the correlator room environment. The cause for the activation of the fire systems has been investigated and identified. CSIRO are currently focussed on reinstalling and re-certifying all the components necessary to bring the fire suppression systems back online. Power was quickly restored to the c-room but without fire control systems, equipment and instruments cannot be returned to operational status. CSIRO have advised this process is likely to be complete by the 14th August. UPDATE: 14 Aug 2020 CSIRO have advised that works continue on recertifying the correlator room fire suppression systems but it is unlikely that instruments and equipment will be back online 17 Aug 2020. The next update will be provided 21 Aug 2020 unless further information from CSIRO is provided in the interim. CSIRO are aware of the impact this is having on all MRO users and are working with contractors to bring the MRO back online as quickly as possible. UPDATE: 25 Aug 2020 CSIRO have advised that the fire suppression systems are being repaired. However, during the repairs one of the gas canisters containing fire suppression gasses activated. This has prevented the correlator room returning to operational status. CSIRO are currently conducting a risk assessment to determine the level of risk associated with a reduced capacity to suppress a fire in the correlator room. The outcome of this risk assessment will determine when and if the correlator room is returned to operational status this week. UPDATE: 26 Aug 2020 CSIRO have advised they are willing to accept the reduced fire suppression system risk. The equipment in the correlator room is in the process of being brought online. |
CLOSED | 03 Sept 2020, 0700-1200h | 25 Aug 2020 | Loss of connectivity to MRO | All MWA, AAVS and external instruments | Cable installation works on the fibre cable between Mullewa and the MRO | A new gold mine is to be established at Yuin station, 150km south of the MRO. The gold mine is proposed for an area where the current fibre communication cable between Boolardy and Mullewa travels. A new cable is currently being laid around the proposed mine site. 03 Sept 0700-1200 (WST) the fibre link between Mullewa and the MRO on the old cable will be cut and replaced by the new cable. This process is expected to take 6 hours. |
CLOSED | 27 Jul - 10 August 2020 | Livestock mustering, whole of Boolardy station | All MWA, AAVS and external instruments | Mustering of livestock | During this period, up to four light planes and helicopters supported by as many as ten vehicles will be conducting a livestock muster on Boolardy station. The muster will occur during daylight hours and will start at the northern boundary of the station moving south. Increased RF activity onsite will occur due to UHF radio use, vehicles and light aircraft. | |
CLOSED | 9-10 July 2020 | 15 July 2020 | Unscheduled correlator room outage | All MWA, SKA and external instruments and equipment within the MRO correlator room | Cooling system failure | Cooling systems failures in the MRO Correlator room continued overnight. Automated shutdowns of equipment and instruments also occurred. Cooling has been re-established to racks and equipment. Equipment and instruments can be brought back online. The source of the fault is known, the cause of the fault remains unknown. Updates will be distributed as information becomes available. UPDATE PM 10 Jul 2020 The cooling system issue remains unresolved. ASKAP is powering down and will not operate over the weekend. The repair contractors are based in Adelaide and are attempting to remote in but not experiencing any positive outcomes for their efforts to identify the fault and repair the fault. CSIRO will depart site at 1400h WAST. They will not return to site till ~1200h Mon 13 Jul. Based on this timeline and the complexity of this fault, all instrument managers are unlikely to have equipment and instruments online before AM Wed 15 Jul. MWA Ops will be provide further updates PM Mon and PM Tues on any new developments. Unless the updates provide a different timeline, all instruments managers will NOT have access to their instruments or equipment before AM Wed 15 July 2020. As always, any issues or queries please send to operations@lists.mwateelscope.org UPDATE 15 JUL 2020 The cooling system issue remains unresolved. ASKAP remains powered down and is only operating intermittently. CSIRO and the equipment contractor are continuing to work on a solution to the fault. At this stage, access to all instruments and equipment on site and in the correlator room remains restricted. access is only provided on an as required basis. This will prevent equipment and instruments experiencing damage whilst operating in circumstances of high temperature. At this stage, instrument managers should not expect access to their equipment or instruments before Wed 22 Jul 2020. |
CLOSED | 8-9 July 2020 | 9 July 2020, 1000h | Unscheduled correlator room outage | All MWA, SKA and external instruments and equipment within the MRO correlator room | Cooling system failure | Overnight, the cooling systems for the correlator room failed at the MRO. This caused significant overheating of equipment and triggered automated shutdowns of equipment and instruments. As at 09 Jul 2020, 1000h (WAST) the cooling system has been repaired and cooling re-established to all correlator room equipment. |
CLOSED | 24-26 June 2020 | MWA Site area power outage | All MWA, SKA and external instruments on the Curtin site area of the MRO | Annual servicing works on the site area HVTX | During this period, power will be switched off to all site instruments and equipment within the Curtin site licence area. | |
CLOSED | 22-26 June 2020 | Correlator room power outage | All MWA, SKA and external instruments on the Curtin site area of the MRO | Maintenance works on LV cabling within the correlator room | During this period, power will be switched off in the correlator room in order to replace some LV cables. It is expected that this work will take five days. | |
CLOSED | 24-25 May 2020 | Forecasted weather event | Unknown | To the west of the MRO, a late season tropical low from the north is expected to merge with a cold front coming form the south producing strong winds, heavy rains and areas of localised high rainfalls. The impact and intensity of the event on the MRO is unknown. | ||
CLOSED | 17-21 February 2020 | Repair works on Correlator room cooling systems | Possibly all ASKAP, MWA and Curtin external instruments | Repairs to damaged / faulty equipment | Early Feb 2020 the correlator cooling equipment failed due to damaged / faulty equipment. Of the three cooling systems only two have been successfully repaired and back on line. Feb 17-21 attempts will be made to repair all three systems to return to normal operations. Repairs are not expected to impact correlator room function but all MRO users should be aware this work is underway. | |
CLOSED | Jan-Feb 2020 | Proposed upgrades to EDGES | All ASKAP and MWA / Curtin external instruments | Installation of a new EDGES instrument | The proposal forecasts the installation of a new EDGES instrument. The installation will involve establishment of a new ground plane. Installation of the new ground plane will involve a period of welding conducted during daytime hours. | |
CLOSED | 25-29 November 2019 | MRO Electrical safety testing | All ASKAP and MWA / Curtin external instruments | HV RCD testing and LV RCD testing | Site wide power outage including the correlator room. All instruments should expect to be without power from Mon 25 - Fri 29 Nov | |
CLOSED | 20-22 November 2019 | AAVS2 / EDA2 site trip | AAVS2 / EDA2 | Installation and commissioning of new servers to support SKA signal chain | AAVS 2 and EDA2 arrays will be powered down whilst new servers and network architecture is installed. | |
CLOSED | 12-17 November 2019 | Lightning strikes | All MWA, Curtin site area external instruments and ASKAP | Weather | Numerous lightning round strikes were recorded on the MRO during this period. Nil damage identified in MWA / Curtin site area instruments. | |
CLOSED | 4-10 November 2019 | SMART box installation | AAVS2 / EDA2 | Installation of 16 SMART boxes on EDA2. Final array commissioning. | Increased RF activity onsite due to UHF radio use, vehicles and handheld equipment. Overnight RFI due to filming activity 6-8 Nov inclusive. | |
CLOSED | 30 Sept - 8 Oct 2019 | AAVS2 antenna installation | AAVS2 / EDA2 | Installation of 256 INAF SKALA 4.1 antennas | Increased RF activity onsite due to UHF radio use, vehicles and handheld equipment. Overnight RFI due to filming activity 2-4 Oct. | |
CLOSED | 5-13 Sept 2019 | MILSTD 461F testing on site | All MWA, Curtin site area external instruments and ASKAP | Testing of the new SKA Field IT processing facility in order to confirm its suitability for operations on the MRO. Testing will also occur of the T-Hut and a random receiver. | Frequency ranges and tests conducted will be posted to this page at the conclusion of tests. | |
CLOSED | 19-31 Aug 2019 | MWA Reconfiguration | All MWA and external instruments | Reconfiguration of the telescope from extended to compact array | During week Aug 19-23 there will be no power to the telescope due to the HV transformer power upgrade. During week Aug 26-31 fault finding and maintenance of the compact array will be occurring. During this week some tiles and receivers will be powered down for maintenance. | |
CLOSED | 19-31 Aug 2019 | Power outage site-wide | All MWA and external instruments | Upgrade of HV transformer | Outage does not include the correlator room. Outage will effect all instruments within the MWA site area | |
CLOSED | 29 July - 4 Aug 2019 | Fibre rollout | All MWA and external instruments | Laying of new fibre cables to support SKA array trials | New 288 core fibre cables will be laid from the site office area to the EDA2 and INAF SKALA 4.1 arrays. This will involve the use of heavy machinery (telehandler). | |
CLOSED | 13-27 June 2019 | MRO access road repairs | All MWA and external instruments | Road maintenance | The contractor has extended the dates for the completion of roadworks on access roads leading to the MRO. The previous finish date was 17 Jun 19. During this period heavy earth moving equipment and water trucks will be used to rebuild and repair the access road to the MRO. Works will during daylight hours only. A generator will be used to provide power to a water pump based at the intersection of Beringarra rd and the MRO access road. The works crew will be using UHF channel 40 during daylight hours. These tests may cause spurious RFI emissions in observations taken during this time period. | |
CLOSED | 3-7 June 2019, 0700-1700h daily | RFI activities on site | All MWA and external instruments | Testing of EDA 2 and AAVS 1.5 antennas using drone based transmitters | A drone testing campaign will be run on behalf of SKA. Details of the drone campaign can be reviewed here. These tests may cause spurious RFI emissions in observations taken during this time period. |
MRO Site Trips Calendar
This calendar of current and future (proposed) site trips is maintained by the fieldwork coordinator (FWC). There may be intermittent outages or interruptions to observations during scheduled site trips due to array maintenance.
Fieldwork Coordinator (FWC)
For any questions regarding the content on this page, please don't hesitate to contact:
Andrew (Andy) McPhail
(+61 8) 9266 9175, or 0432 603 948
External Instrument Contacts
The following people are on a distribution list for rapid communication regarding MRO events (e.g. unexpected outages). Please contact the FWC (above) to request changes.
| Project | Name |
|---|---|
| Breakthrough Listen | Steve Croft, David MacMahon, Matt Lebofsky, Danny Price |
| SSA | Steven Tingay |
| EDA | Randall Wayth |
| AAVS | Mark Waterson, Jader Monari |
| BIGHORNS | Marcin Sokolowski |
| CRAM | Cath Trott |
| Cosmic Ray Detector | Alexander Williamson |
| ORBCOMM | Jack Line |
| SKA | Andre Van Es, David Minchin, Andrea Mattana, Federico Perini, Angela Teale, Mark Bennett, Antony Schinckel, Lachlan Mann |
| SITARA | Jishnu Thekkeppattu |
This table has been updated so the most current/recent changes are at the top.
Any issues or queries please send to operations@lists.mwatelescope.org
Forecast Outages
| Status | Date and Time (AWST) of Outage | Date and Time (AWST) of Latest Update | Description | Affected Projects | Reason | Notes |
|---|---|---|---|---|---|---|
OPEN | 28th June - 16 July | 24 Mat 2021 | Routine site trip | Nil | Nil | |
OPEN | Tues 29th - Wed 30th June | 24 May 2021 | HVTX maintenance | All instruments and equipment within the Curtin Site Licence area. Any equipment within the corrleator room will not be effected. | Annual safety maintenance | During the dates specified, all power will be shutdown to the Curtin site licence area. The High Voltage Transformer (HVTX) will undergo annual safety inspection. |
OPEN | 31 May - 13 July | 21 Jun 2021 | Documentary film production | AAVS2, EDA2 and MWA | During the dates specified, three separate film crews will be undertaking filming of different aspects of instruments and equipment within the Curtin site licence area and ASKAP. Filming within the AAVS / EDA2 instruments will be daytime only. Filming within the MWA area will be at night. During daylight hours, AAVS2 and EDA2 arrays will be powered down in order to prevent any possible damage to instruments and equipment during filming. UPDATE 31 May Over the last three days, significant levels of rainfall have been received within the Murchison area. All access roads between Geraldton and the MRO are now closed. It is unknown when the roads will be open again. Consequently, the filming activities have not been able to commence. At this stage, CSIRO and film crews are waiting for road access conditions to change before making decisions about when / if filming will take place UPDATE 04 Jun The film crews were able to make it to site effective 04 Jun. Filming will commence between 04 - 13 Jun. AAVS2 , EDA2 and MWA will remain powered down whilst filming and drone flights take place. Filming will be night and day. UPDATE 21 Jun 2021 Due to poor weather, the majority of night filming activities did not occur. The filming activities have been rescheduled for the following dates:
| |
CLOSED | 23rd April | 1400h 29 April | Power outage in C-room | All MRO instruments | The 23rd May correlator room outage was triggered by a release of fire suppression gas. The gas release triggered a power shutdown. The sudden loss of load on the power generation facility resulted in the power generation facility shutting down for approximately one second. The power generation facility came back online via diesel generators. CIRA operations staff have not identified any damage to equipment or instruments as a result of the rapid power cycle. However, all equipment in the correlator room within the Curtin racks is not protected by a gaseous fire suppression system in the event of an actual fire. Therefore, as a safety measure ALL equipment and instruments within the correlator room will be powered down by 1600h WST 29 April. It is expected that all equipment will remain offline to at least the 10 May. UPDATE MWA correlator will remain offline pending the outcome of the causes of equipment failure. All other instruments and equipment can continue to operate as normal | |
OPEN | 05 Jul - 13 Aug | 21 Jun 2021 | Maintenance to Boolardy Station Boundary fencing | All MRO instruments | repair / replace degraded fence line in order to control animal movements on Boolardy station |
The yellow line on the above map indicates the section of fencing involved in the maintenance activity. The red line indicates the boundary of the MRO within Boolardy Station. It is a considerable length of boundary fence line (probably close to 80km) with Meerberrie station and will take 2 months to complete. Works will occur in in daylight hours only. Works will involve the use of petrol driven equipment such as chainsaws, jackhammers and pole drivers. Tractors, trucks and utility vehicles will also be used to move personnel and materiel. All instruments may observe spurious RF emissions during daytime observations. UPDATE 24 May 2021 Originally these works were due to commence mid May and be concluded by Late Jun. the works have yet to commence and have been rescheduled for dates post June. New dates will be forwarded as soon as possible. UPDATE 21 Jun 2021 The fencing works will commence 05 Jul and are expected to be completed on or about 13 Aug. |
CLOSED | Mon 19 - Wed 21 Apr | 29 Mar 2021 | Inspection of vermin damaged FOBOT at Curtin fibre aggregation point | All instruments except AAVS2 and EDA2 | Inspection and assessment | Vermin have entered FOBOT#2 of the main trunk cable between the Curtin fibre aggregation point and the correlator building. The vermin have damaged several fibre connections within the FOBOT. In order to remove and inspect the FOBOT, all fibre optic cables within the cabinet need to be disconnected and removed. It is assessed that the outage will take 2-3 days. |
CLOSED | Mon 19 Apr - Fri 23 Apr | 29 Mar 21 | MRO Maintenance site trip | All Curtin Site area instruments | General maintenance | Routine maintenance activities are planned for Apr 19-23. All activities will occur during daylight hours only. Spurious RFI may be observed by instruments. |
CLOSED | Fri PM Mar 19- Thurs PM Mar 25 | 15 Mar 21 | MRO Power outage and cooling system shutdown | All MWA, SKA and external instruments | Repairs to chiller#3 in correlator room damaged during power outage Dec 22; repairs to SESKA power supply system | Mar 22-26 maintenance works will be undertaken on the MRO correlator cooling system and the MRO power station. During this period all instruments and equipment must be powered down and not used. In order to protect equipment from any power surges or temperature fluctuations, all site equipment will be powered down PM Fri 19 Mar. All site circuit breakers will be flicked and rack equipment will be isolated at distribution boards. Equipment will be powered back up Thurs PM 25th Mar. |
CLOSED | 1100h 21 Dec 2020 | 1600h 22 Dec 2020 | MRO site wide power outage | All MWA, SKA and external instruments | Unknown | Update 22 Dec 2020 The source of the power outage was a blown EM filter on a LV supply that provided power to the correlator room. The correlator room lost power and some HV breakers tripped across the site. Fortunately, the Curtin site area did not lose power. Only equipment within the Curtin racks of the correlator room lost power. All MWA, external instruments, AAVS and EDA2 field equipment did not lose power and a quick remote check has identified no obvious damage. It is recommended all instrument managers check their field instruments and advise of any concerns to operations@lists.mwatelescope.org Correlator room equipment and machines are now being brought online. Should anyone have any concerns or issues with their equipment in the correlator room please email operations@lists.mwatelescope.org. There will be no further updates regarding this power outage. Update 21 Dec 2020 As at Australian WST1100h 21 Dec 2020, power was lost to all instruments, telescopes and equipment at the MRO. On site CSIRO staff were unable to identify the cause of the fault. Tomorrow 22 Dec 2020, electrical contractors and other CSIRO staff will travel to the MRO to support the repair effort. Next update: WST 1800h 22 Dec 2020. |
CLOSED | 07-11 Dec 2020 | 03 Nov 2020 | Site wide HV electrical maintenance and Cooling system maintenance in the correlator room | All MWA, SKA and external instruments | Scheduled maintenance and fault repairs | Dec 7-11, 2020 CSIRO will be conducting HV electrical maintenance works and cooling system maintenance in the correlator room. Power supply to all instruments and equipment within the Curtin site area will be cut approximately midday Monday 7th Dec and restored ~midday 11 Dec, 2020. All equipment and instrument managers are to safely shutdown their equipment prior to midday Monday 07 Dec 2020. CIRA Operations staff will be onsite MRO to assist in the local management of any equipment. |
CLOSED | TBA by separate email on occurrence | 0730h 22 Sept 2020 | MWA maintenance operations on site | All MWA, SKA and external instruments | MWA staff will be onsite MRO 21 Sept - 03 Oct inclusive. Operations staff will be conducting routine maintenance tasks on all telescopes within the Curtin site licence area. Impact on equipment and instruments will be minimal. RFI will be experienced during daytime hours as a result of the activities of staff on site. Individual telescopes will be without power during small periods of time as a safety precaution for technicians only | |
CLOSED | Nil | 1300h 28 aug 2020 | MWA operations site trip | All MWA, SKA and external instruments | MWA staff will be onsite MRO 30 Aug - 10 Sept inclusive. Operations staff will be conducting routine maintenance tasks on all telescopes within the Curtin site licence area. Impact on equipment and instruments will be minimal. RFI will be experienced during daytime hours as a result of the activities of staff on site. | |
| CLOSED | Ongoing from 23 July 2020, 1500h | 26 Aug 2020, 1145h | Loss of power in correlator room | All MWA, SKA and external instruments and equipment within the MRO correlator room | Unknown - under investigation | The MRO Correlator room will remain without power for an unknown period of time. The cause and significance of the power outage remains under investigation. The next information update will occur before COB Wed 29th July. Should you have any questions or concerns, please contact operations@lists.mwatelescope.org. Please do not contact CSIRO / MRO staff directly. The staff members are very busy and are cognisant of the implications of the power outage. UPDATE: 30 Jul 2020 The power outage in the correlator room was caused by the activation of fire suppression systems. The fire suppression was activated 23 July in response to a small component failure on a PCB that emitted particulate and smoke into the correlator room environment. The cause for the activation of the fire systems has been investigated and identified. CSIRO are currently focussed on reinstalling and re-certifying all the components necessary to bring the fire suppression systems back online. Power was quickly restored to the c-room but without fire control systems, equipment and instruments cannot be returned to operational status. CSIRO have advised this process is likely to be complete by the 14th August. UPDATE: 14 Aug 2020 CSIRO have advised that works continue on recertifying the correlator room fire suppression systems but it is unlikely that instruments and equipment will be back online 17 Aug 2020. The next update will be provided 21 Aug 2020 unless further information from CSIRO is provided in the interim. CSIRO are aware of the impact this is having on all MRO users and are working with contractors to bring the MRO back online as quickly as possible. UPDATE: 25 Aug 2020 CSIRO have advised that the fire suppression systems are being repaired. However, during the repairs one of the gas canisters containing fire suppression gasses activated. This has prevented the correlator room returning to operational status. CSIRO are currently conducting a risk assessment to determine the level of risk associated with a reduced capacity to suppress a fire in the correlator room. The outcome of this risk assessment will determine when and if the correlator room is returned to operational status this week. UPDATE: 26 Aug 2020 CSIRO have advised they are willing to accept the reduced fire suppression system risk. The equipment in the correlator room is in the process of being brought online. |
CLOSED | 03 Sept 2020, 0700-1200h | 25 Aug 2020 | Loss of connectivity to MRO | All MWA, AAVS and external instruments | Cable installation works on the fibre cable between Mullewa and the MRO | A new gold mine is to be established at Yuin station, 150km south of the MRO. The gold mine is proposed for an area where the current fibre communication cable between Boolardy and Mullewa travels. A new cable is currently being laid around the proposed mine site. 03 Sept 0700-1200 (WST) the fibre link between Mullewa and the MRO on the old cable will be cut and replaced by the new cable. This process is expected to take 6 hours. |
CLOSED | 27 Jul - 10 August 2020 | Livestock mustering, whole of Boolardy station | All MWA, AAVS and external instruments | Mustering of livestock | During this period, up to four light planes and helicopters supported by as many as ten vehicles will be conducting a livestock muster on Boolardy station. The muster will occur during daylight hours and will start at the northern boundary of the station moving south. Increased RF activity onsite will occur due to UHF radio use, vehicles and light aircraft. | |
CLOSED | 9-10 July 2020 | 15 July 2020 | Unscheduled correlator room outage | All MWA, SKA and external instruments and equipment within the MRO correlator room | Cooling system failure | Cooling systems failures in the MRO Correlator room continued overnight. Automated shutdowns of equipment and instruments also occurred. Cooling has been re-established to racks and equipment. Equipment and instruments can be brought back online. The source of the fault is known, the cause of the fault remains unknown. Updates will be distributed as information becomes available. UPDATE PM 10 Jul 2020 The cooling system issue remains unresolved. ASKAP is powering down and will not operate over the weekend. The repair contractors are based in Adelaide and are attempting to remote in but not experiencing any positive outcomes for their efforts to identify the fault and repair the fault. CSIRO will depart site at 1400h WAST. They will not return to site till ~1200h Mon 13 Jul. Based on this timeline and the complexity of this fault, all instrument managers are unlikely to have equipment and instruments online before AM Wed 15 Jul. MWA Ops will be provide further updates PM Mon and PM Tues on any new developments. Unless the updates provide a different timeline, all instruments managers will NOT have access to their instruments or equipment before AM Wed 15 July 2020. As always, any issues or queries please send to operations@lists.mwateelscope.org UPDATE 15 JUL 2020 The cooling system issue remains unresolved. ASKAP remains powered down and is only operating intermittently. CSIRO and the equipment contractor are continuing to work on a solution to the fault. At this stage, access to all instruments and equipment on site and in the correlator room remains restricted. access is only provided on an as required basis. This will prevent equipment and instruments experiencing damage whilst operating in circumstances of high temperature. At this stage, instrument managers should not expect access to their equipment or instruments before Wed 22 Jul 2020. |
CLOSED | 8-9 July 2020 | 9 July 2020, 1000h | Unscheduled correlator room outage | All MWA, SKA and external instruments and equipment within the MRO correlator room | Cooling system failure | Overnight, the cooling systems for the correlator room failed at the MRO. This caused significant overheating of equipment and triggered automated shutdowns of equipment and instruments. As at 09 Jul 2020, 1000h (WAST) the cooling system has been repaired and cooling re-established to all correlator room equipment. |
CLOSED | 24-26 June 2020 | MWA Site area power outage | All MWA, SKA and external instruments on the Curtin site area of the MRO | Annual servicing works on the site area HVTX | During this period, power will be switched off to all site instruments and equipment within the Curtin site licence area. | |
CLOSED | 22-26 June 2020 | Correlator room power outage | All MWA, SKA and external instruments on the Curtin site area of the MRO | Maintenance works on LV cabling within the correlator room | During this period, power will be switched off in the correlator room in order to replace some LV cables. It is expected that this work will take five days. | |
CLOSED | 24-25 May 2020 | Forecasted weather event | Unknown | To the west of the MRO, a late season tropical low from the north is expected to merge with a cold front coming form the south producing strong winds, heavy rains and areas of localised high rainfalls. The impact and intensity of the event on the MRO is unknown. | ||
CLOSED | 17-21 February 2020 | Repair works on Correlator room cooling systems | Possibly all ASKAP, MWA and Curtin external instruments | Repairs to damaged / faulty equipment | Early Feb 2020 the correlator cooling equipment failed due to damaged / faulty equipment. Of the three cooling systems only two have been successfully repaired and back on line. Feb 17-21 attempts will be made to repair all three systems to return to normal operations. Repairs are not expected to impact correlator room function but all MRO users should be aware this work is underway. | |
CLOSED | Jan-Feb 2020 | Proposed upgrades to EDGES | All ASKAP and MWA / Curtin external instruments | Installation of a new EDGES instrument | The proposal forecasts the installation of a new EDGES instrument. The installation will involve establishment of a new ground plane. Installation of the new ground plane will involve a period of welding conducted during daytime hours. | |
CLOSED | 25-29 November 2019 | MRO Electrical safety testing | All ASKAP and MWA / Curtin external instruments | HV RCD testing and LV RCD testing | Site wide power outage including the correlator room. All instruments should expect to be without power from Mon 25 - Fri 29 Nov | |
CLOSED | 20-22 November 2019 | AAVS2 / EDA2 site trip | AAVS2 / EDA2 | Installation and commissioning of new servers to support SKA signal chain | AAVS 2 and EDA2 arrays will be powered down whilst new servers and network architecture is installed. | |
CLOSED | 12-17 November 2019 | Lightning strikes | All MWA, Curtin site area external instruments and ASKAP | Weather | Numerous lightning round strikes were recorded on the MRO during this period. Nil damage identified in MWA / Curtin site area instruments. | |
CLOSED | 4-10 November 2019 | SMART box installation | AAVS2 / EDA2 | Installation of 16 SMART boxes on EDA2. Final array commissioning. | Increased RF activity onsite due to UHF radio use, vehicles and handheld equipment. Overnight RFI due to filming activity 6-8 Nov inclusive. | |
CLOSED | 30 Sept - 8 Oct 2019 | AAVS2 antenna installation | AAVS2 / EDA2 | Installation of 256 INAF SKALA 4.1 antennas | Increased RF activity onsite due to UHF radio use, vehicles and handheld equipment. Overnight RFI due to filming activity 2-4 Oct. | |
CLOSED | 5-13 Sept 2019 | MILSTD 461F testing on site | All MWA, Curtin site area external instruments and ASKAP | Testing of the new SKA Field IT processing facility in order to confirm its suitability for operations on the MRO. Testing will also occur of the T-Hut and a random receiver. | Frequency ranges and tests conducted will be posted to this page at the conclusion of tests. | |
CLOSED | 19-31 Aug 2019 | MWA Reconfiguration | All MWA and external instruments | Reconfiguration of the telescope from extended to compact array | During week Aug 19-23 there will be no power to the telescope due to the HV transformer power upgrade. During week Aug 26-31 fault finding and maintenance of the compact array will be occurring. During this week some tiles and receivers will be powered down for maintenance. | |
CLOSED | 19-31 Aug 2019 | Power outage site-wide | All MWA and external instruments | Upgrade of HV transformer | Outage does not include the correlator room. Outage will effect all instruments within the MWA site area | |
CLOSED | 29 July - 4 Aug 2019 | Fibre rollout | All MWA and external instruments | Laying of new fibre cables to support SKA array trials | New 288 core fibre cables will be laid from the site office area to the EDA2 and INAF SKALA 4.1 arrays. This will involve the use of heavy machinery (telehandler). | |
CLOSED | 13-27 June 2019 | MRO access road repairs | All MWA and external instruments | Road maintenance | The contractor has extended the dates for the completion of roadworks on access roads leading to the MRO. The previous finish date was 17 Jun 19. During this period heavy earth moving equipment and water trucks will be used to rebuild and repair the access road to the MRO. Works will during daylight hours only. A generator will be used to provide power to a water pump based at the intersection of Beringarra rd and the MRO access road. The works crew will be using UHF channel 40 during daylight hours. These tests may cause spurious RFI emissions in observations taken during this time period. | |
CLOSED | 3-7 June 2019, 0700-1700h daily | RFI activities on site | All MWA and external instruments | Testing of EDA 2 and AAVS 1.5 antennas using drone based transmitters | A drone testing campaign will be run on behalf of SKA. Details of the drone campaign can be reviewed here. These tests may cause spurious RFI emissions in observations taken during this time period. |
MRO Site Trips Calendar
This calendar of current and future (proposed) site trips is maintained by the fieldwork coordinator (FWC). There may be intermittent outages or interruptions to observations during scheduled site trips due to array maintenance.
Fieldwork Coordinator (FWC)
For any questions regarding the content on this page, please don't hesitate to contact:
Andrew (Andy) McPhail
(+61 8) 9266 9175, or 0432 603 948
External Instrument Contacts
The following people are on a distribution list for rapid communication regarding MRO events (e.g. unexpected outages). Please contact the FWC (above) to request changes.
| Project | Name |
|---|---|
| Breakthrough Listen | Steve Croft, David MacMahon, Matt Lebofsky |
| SSA | Steven Tingay |
| EDA | Randall Wayth |
| AAVS | Mark Waterson, Jader Monari |
| BIGHORNS | Marcin Sokolowski |
| CRAM | Cath Trott |
| Cosmic Ray Detector | Alexander Williamson |
| ORBCOMM | Jack Line |
| SKA | Andre Van Es, Raunaq Bhushan, David Minchin, Andrea Mattana, Federico Perini |
| SITARA | Ben McKinley, Jishnu Thekkeppattu |
This table has been updated so the most current/recent changes are at the top.
Any issues or queries please send to operations@lists.mwatelescope.org
Forecast Outages
| Status | Date and Time (AWST) of Outage | Date and Time (AWST) of Latest Update | Description | Affected Projects | Reason | Notes |
|---|---|---|---|---|---|---|
OPEN | 28th June - 16 July | 24 Mat 2021 | Routine site trip | Nil | Nil | |
OPEN | Tues 29th - Wed 30th June | 24 May 2021 | HVTX maintenance | All instruments and equipment within the Curtin Site Licence area. Any equipment within the corrleator room will not be effected. | Annual safety maintenance | During the dates specified, all power will be shutdown to the Curtin site licence area. The High Voltage Transformer (HVTX) will undergo annual safety inspection. |
OPEN | 31 May - 13 July | 21 Jun 2021 | Documentary film production | AAVS2, EDA2 and MWA | During the dates specified, three separate film crews will be undertaking filming of different aspects of instruments and equipment within the Curtin site licence area and ASKAP. Filming within the AAVS / EDA2 instruments will be daytime only. Filming within the MWA area will be at night. During daylight hours, AAVS2 and EDA2 arrays will be powered down in order to prevent any possible damage to instruments and equipment during filming. UPDATE 31 May Over the last three days, significant levels of rainfall have been received within the Murchison area. All access roads between Geraldton and the MRO are now closed. It is unknown when the roads will be open again. Consequently, the filming activities have not been able to commence. At this stage, CSIRO and film crews are waiting for road access conditions to change before making decisions about when / if filming will take place UPDATE 04 Jun The film crews were able to make it to site effective 04 Jun. Filming will commence between 04 - 13 Jun. AAVS2 , EDA2 and MWA will remain powered down whilst filming and drone flights take place. Filming will be night and day. UPDATE 21 Jun 2021 Due to poor weather, the majority of night filming activities did not occur. The filming activities have been rescheduled for the following dates:
| |
CLOSED | 23rd April | 1400h 29 April | Power outage in C-room | All MRO instruments | The 23rd May correlator room outage was triggered by a release of fire suppression gas. The gas release triggered a power shutdown. The sudden loss of load on the power generation facility resulted in the power generation facility shutting down for approximately one second. The power generation facility came back online via diesel generators. CIRA operations staff have not identified any damage to equipment or instruments as a result of the rapid power cycle. However, all equipment in the correlator room within the Curtin racks is not protected by a gaseous fire suppression system in the event of an actual fire. Therefore, as a safety measure ALL equipment and instruments within the correlator room will be powered down by 1600h WST 29 April. It is expected that all equipment will remain offline to at least the 10 May. UPDATE MWA correlator will remain offline pending the outcome of the causes of equipment failure. All other instruments and equipment can continue to operate as normal | |
OPEN | 05 Jul - 13 Aug | 21 Jun 2021 | Maintenance to Boolardy Station Boundary fencing | All MRO instruments | repair / replace degraded fence line in order to control animal movements on Boolardy station |
The yellow line on the above map indicates the section of fencing involved in the maintenance activity. The red line indicates the boundary of the MRO within Boolardy Station. It is a considerable length of boundary fence line (probably close to 80km) with Meerberrie station and will take 2 months to complete. Works will occur in in daylight hours only. Works will involve the use of petrol driven equipment such as chainsaws, jackhammers and pole drivers. Tractors, trucks and utility vehicles will also be used to move personnel and materiel. All instruments may observe spurious RF emissions during daytime observations. UPDATE 24 May 2021 Originally these works were due to commence mid May and be concluded by Late Jun. the works have yet to commence and have been rescheduled for dates post June. New dates will be forwarded as soon as possible. UPDATE 21 Jun 2021 The fencing works will commence 05 Jul and are expected to be completed on or about 13 Aug. |
CLOSED | Mon 19 - Wed 21 Apr | 29 Mar 2021 | Inspection of vermin damaged FOBOT at Curtin fibre aggregation point | All instruments except AAVS2 and EDA2 | Inspection and assessment | Vermin have entered FOBOT#2 of the main trunk cable between the Curtin fibre aggregation point and the correlator building. The vermin have damaged several fibre connections within the FOBOT. In order to remove and inspect the FOBOT, all fibre optic cables within the cabinet need to be disconnected and removed. It is assessed that the outage will take 2-3 days. |
CLOSED | Mon 19 Apr - Fri 23 Apr | 29 Mar 21 | MRO Maintenance site trip | All Curtin Site area instruments | General maintenance | Routine maintenance activities are planned for Apr 19-23. All activities will occur during daylight hours only. Spurious RFI may be observed by instruments. |
CLOSED | Fri PM Mar 19- Thurs PM Mar 25 | 15 Mar 21 | MRO Power outage and cooling system shutdown | All MWA, SKA and external instruments | Repairs to chiller#3 in correlator room damaged during power outage Dec 22; repairs to SESKA power supply system | Mar 22-26 maintenance works will be undertaken on the MRO correlator cooling system and the MRO power station. During this period all instruments and equipment must be powered down and not used. In order to protect equipment from any power surges or temperature fluctuations, all site equipment will be powered down PM Fri 19 Mar. All site circuit breakers will be flicked and rack equipment will be isolated at distribution boards. Equipment will be powered back up Thurs PM 25th Mar. |
CLOSED | 1100h 21 Dec 2020 | 1600h 22 Dec 2020 | MRO site wide power outage | All MWA, SKA and external instruments | Unknown | Update 22 Dec 2020 The source of the power outage was a blown EM filter on a LV supply that provided power to the correlator room. The correlator room lost power and some HV breakers tripped across the site. Fortunately, the Curtin site area did not lose power. Only equipment within the Curtin racks of the correlator room lost power. All MWA, external instruments, AAVS and EDA2 field equipment did not lose power and a quick remote check has identified no obvious damage. It is recommended all instrument managers check their field instruments and advise of any concerns to operations@lists.mwatelescope.org Correlator room equipment and machines are now being brought online. Should anyone have any concerns or issues with their equipment in the correlator room please email operations@lists.mwatelescope.org. There will be no further updates regarding this power outage. Update 21 Dec 2020 As at Australian WST1100h 21 Dec 2020, power was lost to all instruments, telescopes and equipment at the MRO. On site CSIRO staff were unable to identify the cause of the fault. Tomorrow 22 Dec 2020, electrical contractors and other CSIRO staff will travel to the MRO to support the repair effort. Next update: WST 1800h 22 Dec 2020. |
CLOSED | 07-11 Dec 2020 | 03 Nov 2020 | Site wide HV electrical maintenance and Cooling system maintenance in the correlator room | All MWA, SKA and external instruments | Scheduled maintenance and fault repairs | Dec 7-11, 2020 CSIRO will be conducting HV electrical maintenance works and cooling system maintenance in the correlator room. Power supply to all instruments and equipment within the Curtin site area will be cut approximately midday Monday 7th Dec and restored ~midday 11 Dec, 2020. All equipment and instrument managers are to safely shutdown their equipment prior to midday Monday 07 Dec 2020. CIRA Operations staff will be onsite MRO to assist in the local management of any equipment. |
CLOSED | TBA by separate email on occurrence | 0730h 22 Sept 2020 | MWA maintenance operations on site | All MWA, SKA and external instruments | MWA staff will be onsite MRO 21 Sept - 03 Oct inclusive. Operations staff will be conducting routine maintenance tasks on all telescopes within the Curtin site licence area. Impact on equipment and instruments will be minimal. RFI will be experienced during daytime hours as a result of the activities of staff on site. Individual telescopes will be without power during small periods of time as a safety precaution for technicians only | |
CLOSED | Nil | 1300h 28 aug 2020 | MWA operations site trip | All MWA, SKA and external instruments | MWA staff will be onsite MRO 30 Aug - 10 Sept inclusive. Operations staff will be conducting routine maintenance tasks on all telescopes within the Curtin site licence area. Impact on equipment and instruments will be minimal. RFI will be experienced during daytime hours as a result of the activities of staff on site. | |
| CLOSED | Ongoing from 23 July 2020, 1500h | 26 Aug 2020, 1145h | Loss of power in correlator room | All MWA, SKA and external instruments and equipment within the MRO correlator room | Unknown - under investigation | The MRO Correlator room will remain without power for an unknown period of time. The cause and significance of the power outage remains under investigation. The next information update will occur before COB Wed 29th July. Should you have any questions or concerns, please contact operations@lists.mwatelescope.org. Please do not contact CSIRO / MRO staff directly. The staff members are very busy and are cognisant of the implications of the power outage. UPDATE: 30 Jul 2020 The power outage in the correlator room was caused by the activation of fire suppression systems. The fire suppression was activated 23 July in response to a small component failure on a PCB that emitted particulate and smoke into the correlator room environment. The cause for the activation of the fire systems has been investigated and identified. CSIRO are currently focussed on reinstalling and re-certifying all the components necessary to bring the fire suppression systems back online. Power was quickly restored to the c-room but without fire control systems, equipment and instruments cannot be returned to operational status. CSIRO have advised this process is likely to be complete by the 14th August. UPDATE: 14 Aug 2020 CSIRO have advised that works continue on recertifying the correlator room fire suppression systems but it is unlikely that instruments and equipment will be back online 17 Aug 2020. The next update will be provided 21 Aug 2020 unless further information from CSIRO is provided in the interim. CSIRO are aware of the impact this is having on all MRO users and are working with contractors to bring the MRO back online as quickly as possible. UPDATE: 25 Aug 2020 CSIRO have advised that the fire suppression systems are being repaired. However, during the repairs one of the gas canisters containing fire suppression gasses activated. This has prevented the correlator room returning to operational status. CSIRO are currently conducting a risk assessment to determine the level of risk associated with a reduced capacity to suppress a fire in the correlator room. The outcome of this risk assessment will determine when and if the correlator room is returned to operational status this week. UPDATE: 26 Aug 2020 CSIRO have advised they are willing to accept the reduced fire suppression system risk. The equipment in the correlator room is in the process of being brought online. |
CLOSED | 03 Sept 2020, 0700-1200h | 25 Aug 2020 | Loss of connectivity to MRO | All MWA, AAVS and external instruments | Cable installation works on the fibre cable between Mullewa and the MRO | A new gold mine is to be established at Yuin station, 150km south of the MRO. The gold mine is proposed for an area where the current fibre communication cable between Boolardy and Mullewa travels. A new cable is currently being laid around the proposed mine site. 03 Sept 0700-1200 (WST) the fibre link between Mullewa and the MRO on the old cable will be cut and replaced by the new cable. This process is expected to take 6 hours. |
CLOSED | 27 Jul - 10 August 2020 | Livestock mustering, whole of Boolardy station | All MWA, AAVS and external instruments | Mustering of livestock | During this period, up to four light planes and helicopters supported by as many as ten vehicles will be conducting a livestock muster on Boolardy station. The muster will occur during daylight hours and will start at the northern boundary of the station moving south. Increased RF activity onsite will occur due to UHF radio use, vehicles and light aircraft. | |
CLOSED | 9-10 July 2020 | 15 July 2020 | Unscheduled correlator room outage | All MWA, SKA and external instruments and equipment within the MRO correlator room | Cooling system failure | Cooling systems failures in the MRO Correlator room continued overnight. Automated shutdowns of equipment and instruments also occurred. Cooling has been re-established to racks and equipment. Equipment and instruments can be brought back online. The source of the fault is known, the cause of the fault remains unknown. Updates will be distributed as information becomes available. UPDATE PM 10 Jul 2020 The cooling system issue remains unresolved. ASKAP is powering down and will not operate over the weekend. The repair contractors are based in Adelaide and are attempting to remote in but not experiencing any positive outcomes for their efforts to identify the fault and repair the fault. CSIRO will depart site at 1400h WAST. They will not return to site till ~1200h Mon 13 Jul. Based on this timeline and the complexity of this fault, all instrument managers are unlikely to have equipment and instruments online before AM Wed 15 Jul. MWA Ops will be provide further updates PM Mon and PM Tues on any new developments. Unless the updates provide a different timeline, all instruments managers will NOT have access to their instruments or equipment before AM Wed 15 July 2020. As always, any issues or queries please send to operations@lists.mwateelscope.org UPDATE 15 JUL 2020 The cooling system issue remains unresolved. ASKAP remains powered down and is only operating intermittently. CSIRO and the equipment contractor are continuing to work on a solution to the fault. At this stage, access to all instruments and equipment on site and in the correlator room remains restricted. access is only provided on an as required basis. This will prevent equipment and instruments experiencing damage whilst operating in circumstances of high temperature. At this stage, instrument managers should not expect access to their equipment or instruments before Wed 22 Jul 2020. |
CLOSED | 8-9 July 2020 | 9 July 2020, 1000h | Unscheduled correlator room outage | All MWA, SKA and external instruments and equipment within the MRO correlator room | Cooling system failure | Overnight, the cooling systems for the correlator room failed at the MRO. This caused significant overheating of equipment and triggered automated shutdowns of equipment and instruments. As at 09 Jul 2020, 1000h (WAST) the cooling system has been repaired and cooling re-established to all correlator room equipment. |
CLOSED | 24-26 June 2020 | MWA Site area power outage | All MWA, SKA and external instruments on the Curtin site area of the MRO | Annual servicing works on the site area HVTX | During this period, power will be switched off to all site instruments and equipment within the Curtin site licence area. | |
CLOSED | 22-26 June 2020 | Correlator room power outage | All MWA, SKA and external instruments on the Curtin site area of the MRO | Maintenance works on LV cabling within the correlator room | During this period, power will be switched off in the correlator room in order to replace some LV cables. It is expected that this work will take five days. | |
CLOSED | 24-25 May 2020 | Forecasted weather event | Unknown | To the west of the MRO, a late season tropical low from the north is expected to merge with a cold front coming form the south producing strong winds, heavy rains and areas of localised high rainfalls. The impact and intensity of the event on the MRO is unknown. | ||
CLOSED | 17-21 February 2020 | Repair works on Correlator room cooling systems | Possibly all ASKAP, MWA and Curtin external instruments | Repairs to damaged / faulty equipment | Early Feb 2020 the correlator cooling equipment failed due to damaged / faulty equipment. Of the three cooling systems only two have been successfully repaired and back on line. Feb 17-21 attempts will be made to repair all three systems to return to normal operations. Repairs are not expected to impact correlator room function but all MRO users should be aware this work is underway. | |
CLOSED | Jan-Feb 2020 | Proposed upgrades to EDGES | All ASKAP and MWA / Curtin external instruments | Installation of a new EDGES instrument | The proposal forecasts the installation of a new EDGES instrument. The installation will involve establishment of a new ground plane. Installation of the new ground plane will involve a period of welding conducted during daytime hours. | |
CLOSED | 25-29 November 2019 | MRO Electrical safety testing | All ASKAP and MWA / Curtin external instruments | HV RCD testing and LV RCD testing | Site wide power outage including the correlator room. All instruments should expect to be without power from Mon 25 - Fri 29 Nov | |
CLOSED | 20-22 November 2019 | AAVS2 / EDA2 site trip | AAVS2 / EDA2 | Installation and commissioning of new servers to support SKA signal chain | AAVS 2 and EDA2 arrays will be powered down whilst new servers and network architecture is installed. | |
CLOSED | 12-17 November 2019 | Lightning strikes | All MWA, Curtin site area external instruments and ASKAP | Weather | Numerous lightning round strikes were recorded on the MRO during this period. Nil damage identified in MWA / Curtin site area instruments. | |
CLOSED | 4-10 November 2019 | SMART box installation | AAVS2 / EDA2 | Installation of 16 SMART boxes on EDA2. Final array commissioning. | Increased RF activity onsite due to UHF radio use, vehicles and handheld equipment. Overnight RFI due to filming activity 6-8 Nov inclusive. | |
CLOSED | 30 Sept - 8 Oct 2019 | AAVS2 antenna installation | AAVS2 / EDA2 | Installation of 256 INAF SKALA 4.1 antennas | Increased RF activity onsite due to UHF radio use, vehicles and handheld equipment. Overnight RFI due to filming activity 2-4 Oct. | |
CLOSED | 5-13 Sept 2019 | MILSTD 461F testing on site | All MWA, Curtin site area external instruments and ASKAP | Testing of the new SKA Field IT processing facility in order to confirm its suitability for operations on the MRO. Testing will also occur of the T-Hut and a random receiver. | Frequency ranges and tests conducted will be posted to this page at the conclusion of tests. | |
CLOSED | 19-31 Aug 2019 | MWA Reconfiguration | All MWA and external instruments | Reconfiguration of the telescope from extended to compact array | During week Aug 19-23 there will be no power to the telescope due to the HV transformer power upgrade. During week Aug 26-31 fault finding and maintenance of the compact array will be occurring. During this week some tiles and receivers will be powered down for maintenance. | |
CLOSED | 19-31 Aug 2019 | Power outage site-wide | All MWA and external instruments | Upgrade of HV transformer | Outage does not include the correlator room. Outage will effect all instruments within the MWA site area | |
CLOSED | 29 July - 4 Aug 2019 | Fibre rollout | All MWA and external instruments | Laying of new fibre cables to support SKA array trials | New 288 core fibre cables will be laid from the site office area to the EDA2 and INAF SKALA 4.1 arrays. This will involve the use of heavy machinery (telehandler). | |
CLOSED | 13-27 June 2019 | MRO access road repairs | All MWA and external instruments | Road maintenance | The contractor has extended the dates for the completion of roadworks on access roads leading to the MRO. The previous finish date was 17 Jun 19. During this period heavy earth moving equipment and water trucks will be used to rebuild and repair the access road to the MRO. Works will during daylight hours only. A generator will be used to provide power to a water pump based at the intersection of Beringarra rd and the MRO access road. The works crew will be using UHF channel 40 during daylight hours. These tests may cause spurious RFI emissions in observations taken during this time period. | |
CLOSED | 3-7 June 2019, 0700-1700h daily | RFI activities on site | All MWA and external instruments | Testing of EDA 2 and AAVS 1.5 antennas using drone based transmitters | A drone testing campaign will be run on behalf of SKA. Details of the drone campaign can be reviewed here. These tests may cause spurious RFI emissions in observations taken during this time period. |
MRO Site Trips Calendar
This calendar of current and future (proposed) site trips is maintained by the fieldwork coordinator (FWC). There may be intermittent outages or interruptions to observations during scheduled site trips due to array maintenance.
Fieldwork Coordinator (FWC)
For any questions regarding the content on this page, please don't hesitate to contact:
Andrew (Andy) McPhail
(+61 8) 9266 9175, or 0432 603 948
External Instrument Contacts
The following people are on a distribution list for rapid communication regarding MRO events (e.g. unexpected outages). Please contact the FWC (above) to request changes.
| Project | Name |
|---|---|
| Breakthrough Listen | Steve Croft, David MacMahon, Matt Lebofsky |
| SSA | Steven Tingay |
| EDA | Randall Wayth |
| AAVS | Mark Waterson, Jader Monari |
| BIGHORNS | Marcin Sokolowski |
| CRAM | Cath Trott |
| Cosmic Ray Detector | Alexander Williamson |
| ORBCOMM | Jack Line |
| SKA | Andre Van Es, Raunaq Bhushan, David Minchin, Andrea Mattana, Federico Perini |
| SITARA | Ben McKinley, Jishnu Thekkeppattu |