MWAX PSRDADA header
- Greg Sleap
- Christopher Phillips
- Luke Williams (Unlicensed)
- Ian Morrison (Unlicensed)
The PSRDADA format is used internally within the MWAX correlator and beamformer components. It is not user-facing.
Pre Correlator
This is the header that is created by mwax_u2s when creating a subfile from the multicast UDP. It therefore describes headers that MWAX VCS users would see in the VCS subfiles.
Keyword | Valid values or Example | Used By | Description |
|---|---|---|---|
HDR_SIZE | 4096 | PSRDADA | Required by PSRDADA - size of header in bytes (should always be 4096) |
POPULATED | 0 | 1 | mwax_db2correlate2db | Used to ensure the correlator does not read an incomplete file. 0 means the file is not yet complete, 1 means the file is ready for consumption |
OBS_ID | 1244973688 | mwax-db2fits | Obs_id of this observation |
SUBOBS_ID | 1244973688 | mwax-db2fits | GPS second of this part of the observation. Increments in 8 sec intervals |
MODE | MWAX_CORRELATOR MWAX_VCS NO_CAPTURE | mwax_db2correlate2db mwax-db2fits | This matches the "mode" from the existing metadata database/schedule. |
UTC_START | 2018-10-11-05:26:14 | PSRDADA mwax-db2fits | Required by PSRDADA. This is the UTC start date/time of the subobservation(?). Used by mwax-db2fits to append a human readable date/time to the filename produced. |
OBS_OFFSET | 0 (n seconds since start of observation) | PSRDADA mwax-db2fits | Required by PSRDADA. I believe this is the number of seconds offset from the start of the observation. First subobservation will have this set to 0. Second subobservation will be 8, etc. |
NBIT | 8 | mwax_db2correlate2db | Bits per sample (should always be 8) |
NPOL | 2 | mwax_db2correlate2db mwax-db2fits | Polarisations per tile (i.e. rf_chains) (should always be 2) |
NTIMESAMPLES | 64000 | mwax_db2correlate2db | Number of timesamples in each input sub-block of a subobservation (nominally sub-blocks are 50 ms) |
NINPUTS | 256 | n rfinputs | mwax_db2correlate2db | Number of signal paths present in the input subobservation file, to be correlated (must be <= NINPUTS_XGPU) |
NINPUTS_XGPU | 256 | n rfinputs in multiples of 16 | mwax_db2correlate2db | Number of signal paths actually correlated by xGPU (must be a multiple of 16; must match the number of paths that xGPU has been configured for) |
APPLY_PATH_WEIGHTS | 0 | 1 | mwax_db2correlate2db | Should scalar path weights be applied prior to correlation (for gain normalisation and/or sub-arraying) |
APPLY_PATH_DELAYS | 0 | 1 | mwax_db2correlate2db | Should fractional path delays be applied prior to correlation (for phasing up to a correlation pointing centre) |
APPLY_PATH_PHASE_OFFSETS | 0 | 1 | mwax_db2correlate2db | Should path phase offsets be applied prior to correlation (for fringe-stopping, in conjunction with applying path delays) |
APPLY_COARSE_DERIPPLE | 0 | 1 | mwax_db2correlate2db | Should coarse PFB channeliser de-ripple be applied within the F-stage, prior to cross-correlation (key/value pair not currently implemented) |
DR_PARAM | NULL | deripplev1 | mwax_db2correlate2db | If APPLY_COARSE_DERIPPLE==1 then this value tells db2correlate2db which deripple settings to use (key/value pair not currently implemented) |
INT_TIME_MSEC | 200 - 8000 | mwax_db2correlate2db mwax-db2fits | Correlator integration time (milliseconds) |
FSCRUNCH_FACTOR | 40 (1 - 6400) | mwax_db2correlate2db mwax-db2fits | Correlator frequency scrunch factor: number of ultrafine 200 Hz fine channels to average together |
APPLY_VIS_WEIGHTS | 0 | 1 | mwax_db2correlate2db | Should data occupancy weights be applied to normalise the output visibilities |
TRANSFER_SIZE | 5275652096 | PSRDADA mwax_db2correlate2db | Total number of bytes of input data for one subobservation (of nominally 8 seconds) |
PROJ_ID | G0008 | mwax-db2fits | Project ID of observation |
EXPOSURE_SECS | 8 (in incremements of 8 seconds) | mwax-db2fits | Duration of observation in seconds |
COARSE_CHANNEL | 76 (0 - max receiver channel number == 255) | mwax-db2fits | Receiver channel number |
CORR_COARSE_CHANNEL | 9 | (1 - coarse channel max) | mwax-db2fits | Correlator channel number - which correlator channel "index" is this? e.g. first channel would be 1. In a 24 channel system, last channel would be 24. This would map to gpubox01-24 in the legacy Correlator |
SECS_PER_SUBOBS | 8 | mwax_db2correlate2db mwax-db2fits | Number of seconds per subobservation |
UNIXTIME | 1539235574 | mwax-db2fits | Unix time of the start of this observation. This field is used by mwax-db2fits to provide a UNIX timestep for each integration throughout the observation. |
UNIXTIME_MSEC | 0 | mwax-db2fits | Unix time milliseconds offset of the start of this observation. This will always be 0 with the current setup, but is here for completeness. |
FINE_CHAN_WIDTH_HZ | 10000 | mwax-db2fits | Correlator output fine channel width in Hz |
NFINE_CHAN | 128 | mwax_db2correlate2db mwax-db2fits | Correlator output number of fine channels |
BANDWIDTH_HZ | 1280000 | mwax-db2fits | Coarse channel passband width in Hz, i.e. the channel spacing |
SAMPLE_RATE | 1280000 | mwax_db2correlate2db | Actual sample rate of input data in samples/s (whether critically-sampled or oversampled). |
MC_IP | 0.0.0.0 | mwax-db2fits | Multicast IP for this coarse channel (for debug purposes) |
MC_PORT | 0 | mwax-db2fits | Multicast port for this coarse channel (for debug purposes) |
MWAX_U2S_VER | X.Y.Z | mwax_db2fits | Version number of mwax_u2s used to capture the UDP packets for this subfile |
MWAX_SUB_VER | 1 | mwax_u2s | Version number of subfile format. A value of 2 indicates the presence of the following fields: |
IDX_PACKET_MAP | (OFFSET)+(SIZE)¹ | mwax_u2s | Location and size of packet map in block 0 metadata - note that the packetmap has been present in subfiles going back to some time in 2023, but the offsets have only been correct since u2s version 2.10 (rolled out ~ may 7 2024) |
IDX_METAFITS | 0+0 |
| deprecated/never implemented. Ignore if present. Location and size of copy of metafits file at time of the observation. This was dropped before we ever implemented the embedding, older subfiles may contain spurious entries. |
IDX_DELAY_TABLE | (OFFSET)+(SIZE)¹ | mwax_u2s | Location and size of delay table in block 0 metadata |
IDX_MARGIN_DATA | (OFFSET)+(SIZE)¹ | mwax_u2s | Location and size of margin data section in block 0 metadata |
IDX_OCCUPANCY_TABLE | (OFFSET)+(SIZE)¹ | mwax_u2s | Location and size of data occupancy table in block 0 metadata (this key/value pair is currently not implemented) |
¹ These fields constitute an index into the block 0 metadata. For example a value of 1234+567 denotes a section beginning at byte 1234 which is 567 bytes long. This is to avoid the need for downstream consumers to calculate these locations based on other fields, and allows for future restructuring without breaking applications that use these index fields to locate the information they are interested in. Currently inserted by mwax_u2s and supported by subtool, but not yet used by mwax_db2correlate2db or mwalib.
Post Correlator changes
The below changes are made during the mwax_db2correlate2db process. The only recipient of this updated header is mwax_db2fits, which uses it to produce the output visibility FITS files.
Keyword | Valid values or Example | Used By | Description |
|---|---|---|---|
NBIT | 32 | mwax-db2fits | Bits per complex visibility component (real | imaginary) |
NTIMESAMPLES | 1 - 40 (for 8 s subobservations) | ?t | Number of integration times (visibility sets produced) per subobservation |
TRANSFER_SIZE | n bytes | mwax-db2fits | Total number of bytes of output data for this 8 second subobservation. Calculated by: bytes = visibilities + weights visibilities = integrations * baselines * fine channels * polarisations * bytes_real * bytes_imaginary weights = integrations * baselines * polarisations * bytes_real |
MWAX_DB2CORR2DB_VER | X.Y.Z | mwa-db2fits | Version number of the correlator executable used to perform the F and X stages of correlation |
Block 0 Metadata
Voltage data samples are stored in blocks 1-160, but the PSRDADA block numbering scheme is 0-indexed, so the contents of block 0 are user-defined. In our case, a number of data tables are stored in this area. They are:
The delay table, storing whole- and fractional-sample delay values, used for cable delay corrections and fringe stopping.
The margin data table, storing the complete payload of the packet immediately before and after the subobservation, as well as the first and last packet within it. This permits whole-sample delays applied by u2s to be undone, or a different set of delays to be applied.
The UDP packet map. This is a bitmap recording precisely which of the expected packets for the subobservation were seen, and which were missing.
The occupancy table. A coarser analogue of the UDP packet map, storing a count of packets received per 5ms time slice, which can be used directly by the correlator for calculating weights.
(Not currently implemented) A copy of the metafits file at the time the subfile was created.
Block 0: Delay table
The delay table records whole- and fractional-sample delays for cable delays and geometric delays. It is stored in a densely-packed binary format, with each row corresponding to an RF source, in the same order as the voltage data. There are 10 fixed columns per row, followed by a column for each fractional delay according to a granularity specified externally (currently 1600, for a total of 1610 columns) but also calculable from data in the table (the num_pointings column). All numbers are little-endian. In version 1 subfiles, the delay table immediately follows the 4096 byte PSRDADA header and its size is not explicitly defined. In version 2 subfiles, the delay table offset and size is defined by the IDX_DELAY_TABLE field in the PSRDADA header (in practice, it remains in its original position, but this is not guaranteed and version 2 consumers should refer to IDX_DELAY_TABLE).
The table schema is as follows:
# | Name | Type | Byte offset | Size (bytes) | Description |
|---|---|---|---|---|---|
0 | rf_input | uint16 | 0 | 2 | RF input ID. This is the tile ID right-shifted by 1,with the lower bit indicating polarity: 0 for X,1 for Y. |
1 | ws_delay | int16 | 2 | 2 | Whole sample delay. This indicates how many samples that u2s has shifted the data stream forward (negative values indicate a backwards shift). |
2 | initial_delay | float64 | 4 | 8 | The calculated delay at the start of the subobservation, in milliseconds. |
3 | delta_delay | float64 | 12 | 8 | The rate of change of the delay, in milliseconds per FFT block (currently 5ms). |
4 | delta_delta_delay | float64 | 20 | 8 | The rate of change of the delta_delay, in milliseconds per FFT block (currently 5ms). |
5 | start_total_delay | float64 | 28 | 8 | Equivalent to initial_delay, first point for quadratic interpolation of fractional delay values. |
6 | middle_total_delay | float64 | 36 | 8 | Second point for quadratic interpolation of fractional delay values, delay at middle of subobservation. |
7 | end_total_delay | float64 | 44 | 8 | Third point for quadratic interpolation of fractional delay values, delay at end of subobservation. |
8 | num_pointings | uint16 | 52 | 2 | The number of fractional delay values in the row, N. |
9 | _reserved | uint16 | 54 | 2 | Reserved for future use. Must always be 0. Used by subtool for automatic detection of delay table format in exported delay tables. |
10 .. N | frac_delay[1..N] | float32 | 56 | 4N | Fractional delay values, in microsamples. |
Block 0: Margin data
To facilitate the removal or editing of whole-sample delays applied by u2s, which would ordinarily be a lossy operation, the full contents of the packets at either end of the subobservation are recorded for each RF source. These "margins" consist of the first packet within the subobservation and the packet immediately preceding it, and the last packet within the subobservation and the packet that follows it, for a total of 4 packets worth of data per RF source. The table consists of rows for each RF source, in the same order as the voltage sample data. This table is not formally specified for version 1 subfiles (though it has been present for a long time, it is left as an exercise for the reader to find it in the unlikely event there is a need to). In version 2 subfiles, the delay table offset and size is defined by the IDX_MARGIN_DATA field in the PSRDADA header.
The table schema is as follows. Note that for the purpose of this table, we use a single 16-bit unsigned integer to represent 1 "sample". More properly speaking, these represent two 8-bit values, forming the real and imaginary parts of a complex integer.
# | Name | Type | Offset | Length | Description |
|---|---|---|---|---|---|
0 - 2047 | Head | uint16 | 0 | 4096 | Voltage data from the packet immediately before the start of the subobservation. |
2048 - 4095 | First | uint16 | 4096 | 4096 | Voltage data from the first packet of the subobservation. |
4096 - 6143 | Last | uint16 | 8192 | 4096 | Voltage data from the last packet of the subobservation. |
6144 - 8191 | Tail | uint16 | 12288 | 4096 | Voltage data from the packet immediately following the end of the subobservation. |
Block 0: UDP Packet Map
The UDP packet map is a bitmap structure representing which of the expected packets for the subobservation were received, and which were missing, with a value of 1 indicating a received packet. It may be considered to be organised into rows representing each RF source, in the same order as the voltage data samples, and 1-bit columns, ordered by time. Thus the byte length of a row is the number of packets per second, multiplied by the number of seconds per subobservation, divided by the number of bits per byte. Both of these values happen to be 8, so the byte length of the row is simply the packets-per-second value - currently either 625 for critically-sampled observations, or 800 for oversampling mode. This table is not formally specified for version 1 subfiles (though it has been present for a long time, it is left as an exercise for the reader to find it in the unlikely event there is a need to). In version 2 subfiles, the delay table offset and size is defined by the IDX_PACKET_MAP field in the PSRDADA header.