/
giant-squid (ASVO downloading code)
giant-squid (ASVO downloading code)
giant-squid
is an alternative ASVO client (the official client being manta-ray
).
Reasons you would want to use giant-squid
over manta-ray
:
- It's twice as efficient at downloading data.
- Technical details: It doesn't download a zip by default; it transparently unzips the data from ASVO, resulting in half the amount of IO required to actually get at the data.
- It's written in Rust, not Python
- No strange Python dependency issues!
- It allows you to control how long your jobs live. The default is 7 days, but it might be useful to do a large batch of data with a 1 day lifetime.
- (I don't think
manta-ray
allows you to control this, but I could be wrong)
- (I don't think
Usage on Pawsey's garrawarla cluster
Get access to non-Pawsey-built software:
module use /pawsey/mwa/software/python3/modulefiles
Load the default module:
module load giant-squid
Look at the help text to help you decide what to do next:
giant-squid -h
More information
, multiple selections available,
Related content
MWA ASVO: Getting Started
MWA ASVO: Getting Started
More like this
MWA ASVO: Release Notes
MWA ASVO: Release Notes
More like this
MWAX Offline Correlator
MWAX Offline Correlator
More like this
Web Services
Web Services
More like this
Data Access
Data Access
More like this
MWA Services Outage Notices
MWA Services Outage Notices
More like this