MWA Data Retention
- Greg Sleap
As the MWA Archive capacity is finite, the MWA Board approved the MWA Data Retention Policy in order to manage the necessary deletion of data in order to make room for new observations .Where possible, the oldest, least valuable data (assessed by the Principal Scientists) or data nominated by PIs will be considered first for any required deletions.
Principal Investigators (PI) of MWA Projects may nominate data for deletion (in the next round) via:
Logging into MWA ASVO, and assuming your MWA ASVO User record is associated with the MWA Project you are PI of, you can select one or many observations and click Request Deletion OR
Email to the Principal Scientists (scientist@mwatelescope.org), especially if the request is too large or complex to be managed submitted via the MWA ASVO
If you are a PI, but when you log into MWA ASVO you do not see a "Delete Requests" option under your "Profile" menu, or the "Request Deletion" button in the Observations page, please contact support.
In the following wiki pages:
No calibrators, nor observations in Collections are be eligible for deletion.
The dataquality column defaults to "Good" for all observations unless it is updated by the Operations team based on feedback from users/PIs. Therefore "Good" may not necessarily mean the data is good quality.
The downloads column is derived from parsing many GB of logs dating back to 2013. Some logs were not recoverable, so the downloads figure, prior to MWA ASVO going online (where records are kept in a database) may not include all downloads for a particular observation. See table below for more details
Observation Type | Dates | Notes |
|---|---|---|
Correlator | <2018 | NGAS Log files were incomplete, so a "0" does not necessarily mean it has never been downloaded, but N (where N>0) means it has definitely been downloaded at least N times. |
Correlator | >=2018 | All downloads are tracked by MWA ASVO database- numbers are accurate. |
VCS | <2022 | NGAS Log files were incomplete, so a "0" does not necessarily mean it has never been downloaded, but N (where N>0) means it has definitely been downloaded at least N times. |
VCS | >=2022 | All downloads are tracked by MWA ASVO database- numbers are accurate. |
As required there will be a round of deletions to free up space for future observations (usually before a new observing semester). These are listed below: