Metafits files
Each MWA observation is described by an associated 'metafits file' - a FITS format file containing header information, plus one or more FITS binary tables, that describe the complete state of the MWA telescope during that observation. These metafits were created in one of three ways:
- By a daemon running on-site, creating metafits files for observations a few hours after they were taken. These metafits files have names that look like NNNNNNNNNN_metafits_ppds.fits, where NNNNNNNNNN is the observation ID. They contain an extra two HDUs ('DigGains' and 'PPDs') which have digital gain data, and full-spectrum (0-655.36 MHz) average power data taken from all tiles during the observation, useful for making sure the analog attenuation was set correctly for that observation, to avoid clipping or undersampling. These metafits files are archived at Pawsey, and will be downloaded with your data files. They may have been generated years ago, if you are reducing old data, so may not contain some of the FITS header cards described below. Use these NNNNNNNNNN_metafits_ppds.fits only for the PPD data that they contain, not for reducing the data.
- By an MWA web service, either directly, or as part of the ASVO data download process. These files have names that look like NNNNNNNNNN.metafits, and were created when the data was downloaded to your local computer. If that was years ago, you can use the MWA web services to download a new copy. It's possible to pass 'include_ppds=1' to the web service, in which case two extra HDU's with digital gain and PPD data (described above) will be included.
- By a daemon running on-site as part of the new MWAX correlator. These files are generated in real time, at the start of each observation, and used to pass the observation data through to the MWAX processing software. These files have names that look like NNNNNNNNNN_metafits.fits but it's not clear yet whether they will be archived and distributed with the data files, or only exist inside MWAX. These files have an additional HDU with a binary table containing Altitude/Azimuth data for the target position, calculated every 4 seconds for the duration of the observation.
Primary HDU header cards
These include:
- GPSTIME - the observation ID, which is the START time of the observation in GPS seconds. This is always an integer multiple of 8 seconds.
- TELESCOP - The string 'MWA'.
- INSTRUME - The string '128T' for observations with the original (Ord) calibrator, or 'MWAX' for observations with the new correlator (from Sep 6th, 2021).
- VERSION - the version number of this metafits file format.
- EXPOSURE - the length of the observation, in seconds. This is also always an integer multiple of 8 seconds.
- FILENAME - despite the name of this card, this is the (arbitrary text) name of the observation, not of any file associated with it.
- MJD - Modified Julian Day (Julian Day minus 2400000.5) at the START of the observation.
- DATE-OBS - An ISOT format string with the UTC date and time at the START of the observation.
- LST - Local sidereal time (in degrees) at the MIDPOINT TIME of the observation, not the start time.
- DUT1 - Value of UT1-UTC from astropy.time.Time.delta_ut1_utc at the time of the observation.
- HA - hour angle of primary beam centre, at the START TIME of the observation, in hours.
- AZIMUTH - primary beam centre azimuth, in degrees.
- ALTITUDE - primary beam centre elevation above the horizon, in degrees.
- RA and DEC - coordinates of primary beam pointing centre (typically a 'sweet spot' near the target of interest), in degrees, calculated for the MIDPOINT TIME of the observation.
- RAPHASE and DECPHASE - the coordinates of the desired target, in degrees. These cards will not be present if the observation was scheduled for a fixed Alt/Az, or using raw beamformer dipole delays.
- DELAYMOD - the name of the current MWAX delay mode (not applicable to older observations).
- CABLEDEL - If this is 0, MWAX has NOT applied cable delays, if 1, MWAX has already corrected for the cable and receiver clock cable lengths. FUTURE FEATURE - if 2, MWAX has corrected for cable lengths AND the pointing-dependent delays due to beamformer dipole delays.
- GEODEL - Which geometric delays MWAX has already applied: 0=nothing, 1=zenith, 2=tile-pointing (constant during an observation), 3=az/el table tracking.
- SIGCHDEL - if this is 1, MWAX has applied the signal chain phase corrections needed to fully calibrate inputs on different analogue signal chain types - eg the RRI receiver ASC module, versus an NI receiver.
- CALIBDEL - If this is 1, MWAX has applied calibration delays and gains. If 0, no calibration correction was applied by MWAX.
- DELDESC - Human readable string describing the current MWAX delay mode.
- ATTEN_DB - Overall (observation-wide) analogue attenuation applied before digitisation inside the receiver, in dB. Note that the MWAX correlator has applied a correction factor for this during processing, while the old correlator does not.
- SUN-DIST, MOONDIST, JUP-DIST - distance in degrees between the primary beam pointing centre and the Sun, Moon and Jupiter, respectively, at the MIDPOINT TIME of the observation.
- SUN-ALT - Elevation of the Sun above the horizon, at the MIDPOINT TIME of the observation.
- CREATOR - String containing the name of the person or daemon that scheduled the observation.
- PROJECT - Project ID, eg C001
- MODE - Correlator mode name, eg HW_LFILES, VOLTAGE_START, NO_CAPTURE, etc.
- RECVRS - String containing a comma separated list of receiver numbers active in this observation. Almost always '1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16'.
- GRIDNAME - The name of the sweet-spot grid being used, almost always 'sweet'.
- GRIDNUM - The grid-point number, within the given grid, used for the primary beam pointing.
- DELAYS - A string containing a list of delay values for the primary beam pointing, eg '0,0,0,0,6,6,6,6,12,12,12,12,18,18,18,18'.
- CALIBRAT - A Boolean indicating whether this observation was intended to be used for calibration
- CALIBSRC - IF CALIBRAT is True, this card contains the name of the calibrator source, eg 'HerA'.
- CENTCHAN - The centre coarse channel number - technically, the 13th coarse channel out of the 24 in the observation.
- CHANNELS - A string containing a comma separated list of all 24 coarse channel numbers in the observation, in ascending order.
- CHANSELS - A string containing a list of numbers between 0 and 23, defining the subset of channels to be analysed. Almost always '0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23'.
- FINECHAN - Fine channel width, in kHz. For the old correlator, this will be one of 10, 20 or 40. For MWAX, this can be 1 to 640 kHz
- INTTIME - Correlator output time averaging, in seconds. For the old correlator, this will be one of 0.5, 1.0 or 2.0. For MWAX, this can be 0.25 to 8.0 seconds
- NAV_FREQ - Number of native fine channels (each 10 kHz for legacy correlator, or 0.2 kHz for MWAX) averaged to make one output fine channel - so for FINECHAN=20, NAV_FREQ will be 2 for the legacy correlator, or 100 for MWAX.
- NSCANS - Number of correlator dumps (each for INTTIME seconds) contained in the whole observation.
- NINPUTS - Number of correlator inputs, which is twice the number of tiles (each has X and Y. For most observations before 2022, the value is 256 (128 tiles time two polarisations).
- NCHANS - Total number of fine channels (for the old correlator, always one of 768, 1536 or 3072 for FINECHAN values of 40, 20 and 10 respectively).
- BANDWDTH - Bandwidth in MHz, almost always 30.72.
- DERIPPLE - Whether the observation was in de-ripple mode, to flatten the coarse channel shape. Value is 0 or 1.
- OVERSAMP - Whether the data was taken in oversampling mode, or critically sampled. Value is 0 or 1.
- MWAXSTAT - Human readable MWAX correlator status string.
- DR_FLAG - Deprecated, will soon be deleted.
- DR_PARAM - If present, the current de-ripple parameter set name. Will be added to the metafits file if/when a new set of deripple parameters is adopted (to be read from mwax heartbeat status packets).
- MWB_ENAB - A list of all the MWAX correlator boxes that should have been running.
- MWB_LIVE - A list of all the MWAX correlator boxes detected as running during this observation.
- FREQCENT - Centre frequency in MHz assuming contiguous channels. For the legacy correlator, this is (1.28 * CENTCHAN) - 0.64 + (0.005 * (NAV_FREQ - 1)), for MWAX it is (1.28 * CENTCHAN) - 0.64. See here for details.
- DATESTRT - An ISOT format string with the UTC date and time when the data started being recorded, Should always be the same as DATE-OBS.
- DATE - An ISOT format string with the UTC date and time when this metafits file was written.
- UNIXTIME - The Unix 'seconds since epoch' value at the start time of the observation (the obsid as a unix timetamp). Not present in all versions.
- QUACKTIM - The number of seconds of time, after the nominal start of the observation (the obsid) that should be discarded from any reduction because the telescope hardware state has not yet settled.
- GOODTIM - The (unix) timestamp of the first good data in the observation - equal to GPSTIME + QUACKTIME, converted to a unix timestamp.
- RAWSCALE - Raw visibilities were multiplied by this value in the correlator before being saved, to put them in a more useful scale.
- CALFITID - The ID of the calibration fitting pipeline run that created this fit data.
- CALCDVER - Version number of the calibration fit code.
- CALDTIME - Date/Time when the calibration fit was run.
- CALCRTR - Who created the calibration fit.
- CALITERS - Number of iterations run in the calibration fit.
- CALITLIM - Maximum number of iterations for the calibration fit.
The main data table - TILEDATA
This table is present in all metafits files, and has 256 rows, one for each of the correlator inputs (more rows are possible in the future with the MWAX correlator). Each row has these columns:
- Input - old correlator input number (usually 0-255 in order of the rows in the table)
- Antenna - A number from 0-127 (duplicated for X and Y on the same tile) giving some arbitrary (CASA? Miriad?) index to each tile.
- Tile - The MWA tile ID number - 11-18, 21-28, ... 161-168 for old 128T tiles, 1001-1072 for new Hex tiles, 2001-2056 for long-baseline tiles.
- TileName - The MWA tile name, eg 'Tile072' or 'HexE17' or 'LBE7'.
- Pol - A string, either 'X' or 'Y'.
- Rx - Receiver ID, from 1-16.
- Slot - Receiver slot, from 1-8.
- Flag - 0 if the input is not flagged for this observation, 1 if it is flagged because of some error.
- Length - A string containing 'EL_' followed by the corrected electrical length of the cable to that tile, in metres at 'c'. Note that these values are all negative, because the lengths of the clock cables TO each receiver have been subtracted.
- North - Tile coordinate, in metres North of the array centre.
- East - Tile coordinate, in metres East of the array centre.
- Height - Tile coordinate, in mtres above sea level (NOT the array centre).
- Gains - Digital gain multipliers applied in the receiver for each of the 24 coarse channels in the observation, in the same order as the CHANNELS primary header card. Cotter and birli uses this data to correct these digital gains when it loads the raw visibility files.
- BFTemps - Beamformer temperature in degrees at the start of the observation (contents maybe be NaN in some metafits files).
- Delays - Actual delays used for that tile during the observation. Values of 32 for a delay mean that dipole was disabled (not summed into the final output) because of a known fault.
- VCSOrder - Index for that correlator input in VCS data from the old correlator.
- Flavors - The cable flavor (eg 'RG6_150' or 'LMR400_320' for that input. The first part of the name is the cable type (RG6, LMR400, RFOF), the second is the nominal cable length in metres.
- Receiver_Types - The type string for the receiver responsible for digitising that tile. Currently, the only possible values are 'RRI' (the original 16 receivers), 'NI' (receivers 17 and 18), and 'PSEUDO' (receiver 99, a fictitious receiver logically connected to CRAM and EDA2, both of which have data injected via UDP multicast).
Note that the calibration fit data described here has only been applied to the data if it was produced by the MWA correlator, and if the CALIBDEL card value is True. The calibration data is provided here so that it can be undone, if the user wants to apply a different calibration fit. For example, the calibration gains applied by MWAX are constant across each coarse channel, while full calibration fits (downloaded from the calibration web service, for example) fit a linear gain function across all the fine channels in each coarse channel.
The calibration data table - CALIBDATA
This table is present in some metafits files, and if present, contains data from the calibration_solutions table with a calibration fit from the most recent fitted calibration observation with the same frequency channel setttings as this observation. Each row has these columns:
- Antenna - A number from 0-127 (duplicated for X and Y on the same tile) giving some arbitrary (CASA? Miriad?) index to each tile.
- Tile - The MWA tile ID number - 11-18, 21-28, ... 161-168 for old 128T tiles, 1001-1072 for new Hex tiles, 2001-2056 for long-baseline tiles.
- TileName - The MWA tile name, eg 'Tile072' or 'HexE17' or 'LBE7'.
- Pol - A string, either 'X' or 'Y'.
- Calib_Delay - The calibration offset, in metres, for that input, derived from the most recently processed calibrator observation with the same coarse channels. May be missing or all zeros in some metafits files.
- Calib_Gains - The calibration gain multiplier (not in dB) for each of the 24 channels in this observation, for this input. Derived from the most recently processed calibrator observation with the same coarse channels. May be missing or all ones in some metafits files.
The header for this HDU also contains copies of the calibration fit details from the primary HDU header:
- CALFITID - The ID of the calibration fitting pipeline run that created this fit data.
- CALOBSID - The obsid of the calibration observation used for this fit
- CALCDVER - Version number of the calibration fit code.
- CALDTIME - Date/Time when the calibration fit was run.
- CALCRTR - Who created the calibration fit.
- CALITERS - Number of iterations run in the calibration fit.
- CALITLIM - Maximum number of iterations for the calibration fit.
The digital gains data table - DigGains
This table is only found in _metafits_ppds.fits files, or ones created by the web service with the 'include_ppds=1' parameter, and contains digital gains multipliers (applied in the receiver) for all 256 channels, for all of the cable flavors (the TILEDATA table only contains digital gains for the 24 channels used in that observation). The columns are:
- Flavors - The cable flavor (eg 'RG6_150' or 'LMR400_320' for that input. The first part of the name is the cable type (RG6, LMR400, RFOF), the second is the nominal cable length in metres.
- Gains - An array of digital gain multipliers applied in the receiver for all 256 coarse channels.
The PPD data table - PPDs
This table is only found in _metafits_ppds.fits files, or ones created by the web service with the 'include_ppds=1' parameter, and contains the PPD data - averaged power measurements in each channel for every tile/polarisation, for the duration of the observation. Each row represents one measurement (power averaged over 1 second, for one tile, in each of the 256 possible coarse channels). The receiver downloads this power data from one of its 8 slots every 8 seconds,, so each tile is measured once every 64 seconds The columns are:
- Time - time in GPS seconds that this row was downloaded from the receiver. The data is averaged over one UTC second prior to this timetamp.
- TileID - The MWA tile ID number - 11-18, 21-28, ... 161-168 for old 128T tiles, 1001-1072 for new Hex tiles, 2001-2056 for long-baseline tiles.
- Power - a 2 x 256 array of floats, where Power[0] is 256 Y polarisation values, and Power[1] is 256 X polarisation values.
The pointing table - AltAz
This table is only found in MWAX correlator internal metafits files - NNNNNNNNNN_metafits.fits. It contains Altitude/Azimuth/Distance values for the source of interest, calculated every 4 seconds for the duration of the observation. This is used by the MWAX correlator to calculate geometric delays depending on the source position relative to each tile position. The columns are:
- gpstime - the time in GPS seconds for this alt/az value.
- Alt - Altitude above the horizon, in degrees.
- Az - Azimuth in degrees.
- Dist_km - Source distance from the MWA, in kilometres. This column will always contain zeroes unless the source is an Earth-orbiting satellite, in which case MWAX will perform near-field geometric corrections.