Go to the website

Register

You will need to register to create an account. There are two ways to register (and then log in) with the MWA ASVO:

Choose How to Login / Register

Option 1: Log in / Register with a AAO Data Central Account

The MWA ASVO supports logging in via our partner ASVO node: Data Central:

Open

For more information about Data Central, please visit their website.

• Click on "Register with your AAO data central account"

• This will take you to their sign in page. Log in to Data Central as normal.

• You will then be automatically logged in to the MWA ASVO!

Option 2: Log in / Register by creating an MWA ASVO username and password

• Click on "Register for an MWA ASVO account"

• Fill in and submit the registration form.

• You will be sent an email to verify your email address.

• Confirm your email address by clicking on the link in the email.

• An administrator will review the registration and then activate your account..

• You will receive an email confirming your account is active. Until then you will not be able to log in to the system.

Logging In

Once your account is activated, you will now be able to login to the system. Click "Login" in the top right of the home page.

If you chose Option 1 above, then click on "Login through an AAO Data Central account", otherwise click "Login with MWA ASVO account".

Searching for Observations

The first thing you'll want to do is to search for observations via the Search for Observations page. Click on "Search".

• You can specify parameters on the left-hand panel (MWA ASVO Cone Search Query)

• OR, if you are an advanced user, write your own ADQL query against the obscore schema.

Review the list of results in the table at the right side of the page. You may do any of the following:

• Create a conversion job or download job for a single observation through this web interface (see Create a Data Job section below)

• Select any number of rows in the table (or use the "Select All/None" buttons), then click:

• Submit Conversion: this will allow you to submit / generate a CSV file based on the conversion job options and observations you have selected, which can be used as input to the manta-ray-client (the command-line download tool)

• Submit Download: this will do the same as "Submit 44Conversion", except for visibility download job.

Create a Data Job

Start a new data job

While in the Query Results panel, clicking on an observation ID (obsid) will take you to the New Data Job page where you can specify how you want your data.
Alternatively, if you already know the obsid you want to download, you can go My Jobs. Then, click on New Data Job button. Next, you must decide if you want to run a Conversion job or a Visibility Download job or a voltage Download Job.

Conversion Job

This will retrieve raw visibilities from the MWA Archive and convert the MWA-specific file format into a CASA measurement set or uvfits file format using the cotter pre-processing utility. Optionally you may opt for an appropriate calibration solution to be found and applied to your data. You must also specify the frequency & time averaging options, as well as the edge-width and other settings. (See below)

• Time Resolution (s): Average N seconds of time steps together before writing output.

• Freq. Resolution (kHz): Average N kHz bandwidth of channels together before writing output. When averaging: flagging, collecting statistics and cable length fixes are done at the highest resolution. UVW positions are recalculated for new time steps.

• Edge Width (kHz): Flag the given width of edge channels of each coarse channel (default: 80 kHz)

• Calibrate: The system will try to find an appropriate calibration solution for this observation. The solution is applied before averaging. If you select this and it cannot find a calibration solution, the job will fail, however most observations will have a calibration solution available by Q2 2019. For more information on how the "calibrate" feature works, please see MWA ASVO: Calibration Option.

• Delivery: By default, ASVO delivers the data to Acacia, which is an object storage. Alternatively, users who have a Pawsey account set up in their profile will be able to select /scratch, which is a file system provided by Pawsey.

Now you submit your data job. If there are any problems with your parameters you will see an appropriate message on the page.

Visibility Download Job

This will retrieve either just the metadata files associated with the observation and/or the raw visibilities (still in their MWA-specific format). This option is intended for advanced users who are familiar with MWA tools such as cotter, the MWA RTS, etc and want more fine-grained control. Now you submit your data job. If there are any problems with your parameters you will see an appropriate message on the page.

Voltage Download Job

This type of job allows you to download raw MWA voltage data in raw format.  This option is only for advanced users as the data is in an MWA-specific raw voltage format. This can only be delivered to /scratch as the data will be usually huge. Now you submit your data job. If there are any problems with your parameters you will see an appropriate message on the page.

Beamformer Download Job

This type of job allows you to download beamformer data products from MWAX_BEAMFORMER or MWAX_CORR_BF observations. A beamformer observation is a high-time-resolution recording created by electronically steering the telescope tiles toward a specific sky coordinate to capture the rapid signals of objects like pulsars and fast radio bursts. A beamformer observation might comprise of data captured by an arbitrary number of coherent or incoherent beams.

Imaging Job

This type of job enables users to generate sky images from visibility data using WSClean, as a fast wideband imager for radio astronomy. Radio astronomy imaging is the process of converting visibility data (raw measurements from telescope arrays) into sky images (2D representations of the radio sky). This allows users to quickly identify data of interest and filter observations according to their research goals.

The wsclean software performs this transformation through:

1. Fourier Transformation - converting visibility (UV-plane) data to image (sky) plane

2. deconvolution - Removing telescope response (dirty beam) via CLEAN algorithm

3. wide-band imaging - Handling frequency-dependent effects across MWA’s bandwidth

4. W-projection - Correcting for wide field-of-view effects

Imaging jobs can only be processed into images from measurement sets. The workflow includes:

1. staging: Raw visibility files are retrieved from the archive

2. pre-processing: Calibration and averaging using Birli

3. imaging: WSClean generates sky images with CLEAN deconvolution

4. delivery: FITS image (and optionally all intermediate products) is delivered

Parameters

• output_mode: Output Content. Determines which products are returned after processing.

  • fitsOnly the final restored image.

  • all_fitsAll FITS images (e.g. residual and model images)

  • all_filesThe full output package, including logs and intermediate products (like .ms files)

• avg_time_res: Time Resolution (seconds). The temporal binning of the visibility data. Increasing this reduces file size and processing time but can lead to “time-average smearing” at large distances from the phase center.

• avg_freq_res: Frequency Resolution (kHz). The spectral binning of the data. Similar to time averaging, coarser resolution speeds up imaging but can cause “bandwidth smearing”

• flag_edge_width: Flag Edge Width (kHz). The amount of bandwidth to discard at the edges of each coarse channel to remove “aliasing” or filter roll-off artifacts.

• apply_di_cal: Apply DI Calibration. If enabled, applies standard gain solutions (amplitude and phase) to the data before imaging to correct for instrumental and atmospheric effects.

• image_size: Image Dimensions (pixels). The with and height of the output image in pixels (e.g. 4096 x 4096)

• pixel_scale: Pixel Scale (arcsec/deg). The angular size of a single pixel. This must be small enough to “sample” the synthesized beam (usually 3-5 pixels across the beam)

• weighting: Weighting Scheme. Controls the trade-off between sensitivity and resolution

  • naturalMaximum sensitivity but lower resolution

  • uniformBetter resolution but higher noise

  • briggsA popular compromise between the two

• robust: Briggs Robustness. Used only when weighting is set to ‘briggs’. Values range from -2.0 (closer to uniform) to 2.0 (closer to natural).

• clean_iterations: Max Iterations (-niter). The maximum number of components the CLEAN algorithm will attempt to find and subtract. Set to 0 for a “dirty” image.

• clean_threshold: Absolute Threshold (mJy). Deconvolution stops when the peak residual brightness falls below this level.

• auto_threshold: Automated Threshold (σ). Tells WSClean to stop cleaning when the residuals reach a multiple of the standard deviation (noise floor) of the image.

• phase_center: Phase Center. The celestial coordinates (RA/Dec) that will be at the center of the image

• nwlayers: W-Stacking Layers. Sets the number of layers used to account for the non-coplanar nature of the MWA array. More layers increase accuracy for wide-field imaging but require more memory.

• apply_primary_beam: Apply Primary Beam Correction. Corrects for the MWA tile response pattern. Since the tiles are more sensitive at the center of the “beam” than at the edges, this ensures flux densities are physically accurate across the entire field of view.

• multiscale: Multi-scale CLEAN. An algorithm extension that identifies sources of varying sizes. Highly recommended for MWA data to properly model extended galactic emission and large radio galaxies

Monitor Jobs

After submitting your data job, or if you want to come back and check on the status of your job(s), go to the Data Jobs page. You will see all of your current jobs and you can take different actions depending on their status. MWA ASVO data jobs go through the following statuses:

• Queued: Your data job is submitted and is awaiting processing. You may cancel your data job during this phase using the Cancel button.

• Processing: Your data job has been picked up by a worker node and the data job is running. The worker process will be downloading the data from the MWA Archive which is located at the Pawsey Supercomputing Centre, processing a data conversion job using cotter (if you selected a data conversion job) and then placing the resultant data product in a data retrieval area on our servers. You may cancel your data job during this phase using the Cancel button.

• Complete: Your data job completed successfully and the data product is ready to download (see below).

• Error: Something went wrong and the system could not produce the output you requested. The error message may provide you with details of what went wrong. If you are having difficulty processing an observation please contact MWA ASVO Technical Support and provide the obsid and job_id you are having trouble with.

Download Data

There are two ways we deliver the data. 

• Acacia: This is an object storage. Once your data job is successfully completed, ASVO uploads the data into Acacia. There will be a hyperlink to your data product in the "Download" column of the table on the Job Results page. MWA ASVO downloads can be very large, so we recommend copying the link URL and using tools such as wget to download your data. Your download will be available for 7 days from completion, after which it will be removed from the MWA ASVO servers.

• Scratch: This is a file system provided by Pawsey. You need to have a Pawsey account to access this storage, where ASVO directly delivers the data. It is up to the user to manage and clean up the data under their allocation.

Conversion Job Contents

After a conversion job (obsid_ms.zip or obsid_uvfits.zip) and you unzip the contents you will have the following files:

Output type: CASA Measurement Set

Output type: UVFITS

Download Job Contents

Beamformer Job Contents

Output Type: Voltage interleaved data

Output Type: Filterbank

Imaging Job Contents

Output Type: FITS image

Bulk Downloads

Once you have submitted and downloaded a few jobs via the web interface you might be interested in a more efficient method to submit jobs and download data so you can do it in bulk. It is recommended that you use the manta-ray-client command line tool and API which makes submitting and downloading batches of jobs easier. Please see: MWA ASVO: Command Line Clients for more information.