giant-squid (ASVO downloading code)
giant-squid is an alternative ASVO client (the official client being manta-ray).
Reasons you would want to use giant-squid over manta-ray:
It's twice as efficient at downloading data.
Technical details: It doesn't download a zip by default; it transparently unzips the data from ASVO, resulting in half the amount of IO required to actually get at the data.
It's written in Rust, not Python
No strange Python dependency issues!
It allows you to control how long your jobs live. The default is 7 days, but it might be useful to do a large batch of data with a 1 day lifetime.
(I don't think
manta-rayallows you to control this, but I could be wrong)
Usage on Pawsey's garrawarla cluster
Get access to non-Pawsey-built software:
module use /pawsey/mwa/software/python3/modulefilesLoad the default module:
module load giant-squidLook at the help text to help you decide what to do next:
giant-squid -h