/
giant-squid (ASVO downloading code)

giant-squid (ASVO downloading code)

giant-squid is an alternative ASVO client (the official client being manta-ray).


Reasons you would want to use giant-squid over manta-ray:

  • It's twice as efficient at downloading data.
    • Technical details: It doesn't download a zip by default; it transparently unzips the data from ASVO, resulting in half the amount of IO required to actually get at the data.
  • It's written in Rust, not Python
    • No strange Python dependency issues!
  • It allows you to control how long your jobs live. The default is 7 days, but it might be useful to do a large batch of data with a 1 day lifetime.
    • (I don't think manta-ray allows you to control this, but I could be wrong)


Usage on Pawsey's garrawarla cluster

Get access to non-Pawsey-built software:

module use /pawsey/mwa/software/python3/modulefiles

Load the default module:

module load giant-squid

Look at the help text to help you decide what to do next:

giant-squid -h

More information

https://github.com/MWATelescope/giant-squid

Related content

MWA ASVO: Command Line Client (giant-squid)
MWA ASVO: Command Line Client (giant-squid)
More like this
MWA ASVO: Command Line Client (manta-ray-client)
MWA ASVO: Command Line Client (manta-ray-client)
More like this
MWA ASVO Use with HPC Systems
MWA ASVO Use with HPC Systems
More like this
MWA ASVO: Release Notes
MWA ASVO: Release Notes
More like this
MWA ASVO Command Line Clients
MWA ASVO Command Line Clients
More like this
Using the new MWA ASVO
Using the new MWA ASVO
More like this