...

• Status
  colour Green title web

   This item relates to the front end web site of the MWA ASVO and the associated VO TAP service.

• Status
  colour Blue title client

   This item relates to the python API and command line client of the MWA ASVO (MWA ASVO: Command Line Client (manta-ray-client)).

• Status
  colour Yellow title backend

   This item relates to the server back-end of the MWA ASVO system, which includes the dispatch service, processor service and cotter.

• Status
  subtle true colour Red title Breaking

   This represents a breaking change to existing functionality

27th January 2022

• Jobs will now be processed on Garrawarla
• Pawsey users will have the option to leave data directly on /astro. These will be left in /astro/[group]/asvo/[job_id].
• If you do not choose this option, you will be given a URL to access your data, much in the same way as the new system. This will be served out from AWS S3, and will eventually be replaced by Pawsey’s new Acacia system once it comes online.
• MWA members can now download data that was processed with the shiny new MWAX correlator.
• You can also process this data using Birli, a Cotter replacement that works with both the legacy and MWAX correlator. However, you can choose whether to use Cotter or Birli when working with legacy data.
• VCS downloads are now supported. However you’ll need to get in touch with an admin or the VCS team if you’re interested in getting access to VCS data.
• You can now batch submit jobs straight from the web frontend, instead of having to download a CSV and submit using the CLI.

Regarding the new /astro data delivery option:

• This option only applies to people who have an account with Pawsey and are in one of the existing MWA groups (mwasci, mwaeor, mwavcs, mwaops)
• Those groups are still responsible for managing their own quota on /astro. If you choose the /astro option and your group doesn’t have enough space, the job will fail. So make sure that you have sufficient quota on /astro before submitting the job.
• To use this option, your ASVO account will need to be configured with your Pawsey group. This needs to be done by an admin, so you’ll need to let me know which group you would like to be attached to this account.
• It might be an idea for each group PI to compile a list of users that should have access to this feature.

Regarding the new Birli preprocessing software:

• Birli is not quite at feature parity with Cotter, so you may be a little limited in terms of your options for processing MWAX data.
• While Birli does support writing CASA Measurement Sets, this functionality is still experimental and slow. It is therefore disabled in ASVO for now, but you can still download the raw files and run Birli yourself if needed.
• There are a number of other features that are still in development. I suggest that you take a look at the repository here to familiarise yourself with the feature set and interface for using Birli https://github.com/mwatelescope/birli
• Otherwise, Birli is an incredible effort by @dev and is a stable and performant preprocessor for MWAX and legacy correlator data.

Regarding the deployment itself:

• The manta ray client requires a few changes to make it compatible with the new system. After the deployment, your client will no longer work until it’s updated.
• I have tried to make things backwards compatible as much as possible. However, You may need to make changes to your own scripts and programs to make them compatible with the new client.
• However, we will be updating the client and Docker image on Garrawarla so you don’t have to.
• In order to perform the upgrade, I need to drain the old system of jobs. This means that I will have to clear out the database table on deploy day. Any jobs that are queued up before then will be lost. Any jobs that have been completed before then will still work, but the download link will not be accessible from ASVO itself. If that data is critical for your work, please save a copy of your download links yourself.

26th November 2020

• Enhancements:
  

  • Status
    colour Blue title client

     When submitting jobs from the command line client, it will no longer fail subsequent jobs if one of them has no available files.

...