| Table of Contents |
|---|
Top-Level Architecture
...
In this section we describe the flow of signals from the MWA receivers to the correlator and then into long term storage at the MWA Archive at the Pawsey Supercomputing Centre, illustrated in the diagram below.
MWAX UDP Capture + Voltage Capture To Disk
...
The mwax_mover instance running on each MWAcache Server at Curtin University will be configured to automatically forward all visibility and voltage files to the Pawsey Front-End (FE) Servers, where the data will be stored in Pawsey’s hierarchical storage management system (HSM). Once data is successfully transferred to Pawsey the data will be moved to a cache which will also be managed by mwax_mover. MWAX mover will cache up to a configurable high-water mark (e.g. 85% of volume capacity) and will remove the oldest data to remain under that threshold.The NGAS servers running at Pawsey will execute the commands needed to write the visibilities and voltages to Pawsey’s tape subsystem. The NGAS software then mwax_mover updates the MWA metadata database to tag the files as being safely archived at Pawsey.
...
In the event that there is an issue with the systems at Pawsey (e.g. monthly maintenance or any other system error), or the 100 Gbps link between the Curtin 206 data centre and Pawsey is down, mwax_mover will requeue the transfer and try again at a later time. Each MWAcache Server has approximately 250 TB of usable storage space to store visibilities and voltage data temporarily until it can be successfully archived.
Researchers will access correlated data be able to access visibilities and voltages using the existing MWA All-Sky Virtual Observatory data portal (MWA ASVO), where users will either process the raw visibilities themselves or they may opt to have the MWA ASVO convert the data to CASA measurement set or uvfits format (with or without applying calibration solutions). Science teams wishing to utilize the high time resolution voltage data will use the existing “voltdownload” utility to obtain the raw voltage files. In a future release, the MWA ASVO will be upgraded to make use of any data cached on the MWAcache servers in B206, bypassing the need to stage files from the Pawsey tape subsystem and allowing the data to be delivered to users with less delay.
More information
| Child pages (Children Display) |
|---|
...