The PSRDADA format is used internally within the MWAX correlator and beamformer components. It is not user-facing.
...
| Keyword | Valid values or Example | Used By | Description |
|---|---|---|---|
| HDR_SIZE | 4096 | PSRDADA | Required by PSRDADA - size of header in bytes (should always be 4096) |
| POPULATED | 0 | 1 | mwax_db2correlate2db | Used to ensure the correlator does not read an incomplete file. 0 means the file is not yet complete, 1 means the file is ready for consumption |
| OBS_ID | 1244973688 | mwax-db2fits | Obs_id of this observation |
| SUBOBS_ID | 1244973688 | mwax-db2fits | GPS second of this part of the observation. Increments in 8 sec intervals |
| MODE | MWAX_CORRELATOR MWAX_VCS NO_CAPTURE | mwax_db2correlate2db mwax-db2fits | This matches the "mode" from the existing metadata database/schedule. |
| UTC_START | 2018-10-11-05:26:14 | PSRDADA mwax-db2fits | Required by PSRDADA. This is the UTC start date/time of the subobservation(?). Used by mwax-db2fits to append a human readable date/time to the filename produced. |
| OBS_OFFSET | 0 (n seconds since start of observation) | PSRDADA mwax-db2fits | Required by PSRDADA. I believe this is the number of seconds offset from the start of the observation. First subobservation will have this set to 0. Second subobservation will be 8, etc. |
NBIT | 8 | mwax_db2correlate2db | Bits per sample (should always be 8) |
| NPOL | 2 | mwax_db2correlate2db mwax-db2fits | Polarisations per tile (i.e. rf_chains) (should always be 2) |
| NTIMESAMPLES | 64000 | mwax_db2correlate2db | Number of timesamples in each input sub-block of a subobservation (nominally sub-blocks are 50 ms) |
| NINPUTS | 256 | n rfinputs | mwax_db2correlate2db | Number of signal paths present in the input subobservation file, to be correlated (must be <= NINPUTS_XGPU) |
| NINPUTS_XGPU | 256 | n rfinputs in multiples of 16 | mwax_db2correlate2db | Number of signal paths actually correlated by xGPU (must be a multiple of 16; must match the number of paths that xGPU has been configured for) |
APPLY_PATH_WEIGHTS | 0 | 1 | mwax_db2correlate2db | Should scalar path weights be applied prior to correlation (for gain normalisation and/or sub-arraying) |
| APPLY_PATH_DELAYS | 0 | 1 | mwax_db2correlate2db | Should fractional path delays be applied prior to correlation (for phasing up to a correlation pointing centre) |
| APPLY_PATH_PHASE_OFFSETS | 0 | 1 | mwax_db2correlate2db | Should path phase offsets be applied prior to correlation (for fringe-stopping, in conjunction with applying path delays) |
| APPLY_DE-RIPPLE | 0 | 1 | mwax_db2correlate2db | Should coarse PFB channeliser de-ripple be applied within the F-stage, prior to cross-correlation |
| INT_TIME_MSEC | 200 - 8000 | mwax_db2correlate2db mwax-db2fits | Correlator integration time (milliseconds) |
| FSCRUNCH_FACTOR | 40 (1 - 6400) | mwax_db2correlate2db mwax-db2fits | Correlator frequency scrunch factor: number of ultrafine 200 Hz fine channels to average together |
| APPLY_VIS_WEIGHTS | 0 | 1 | mwax_db2correlate2db | Should data occupancy weights be applied to normalise the output visibilities |
TRANSFER_SIZE | 5275652096 | PSRDADA mwax_db2correlate2db | Total number of bytes of input data for one subobservation (of nominally 8 seconds) |
PROJ_ID | G0008 | mwax-db2fits | Project ID of observation |
| EXPOSURE_SECS | 8 (in incremements of 8 seconds) | mwax-db2fits | Duration of observation in seconds |
| COARSE_CHANNEL | 76 (0 - max receiver channel number == 255) | mwax-db2fits | Receiver channel number |
| CORR_COARSE_CHANNEL | 9 | (1 - coarse channel max) | mwax-db2fits | Correlator channel number - which correlator channel "index" is this? e.g. first channel would be 1. In a 24 channel system, last channel would be 24. This would map to gpubox01-24 in the legacy Correlator |
| SECS_PER_SUBOBS | 8 | mwax_db2correlate2db mwax-db2fits | Number of seconds per subobservation |
UNIXTIME | 1539235574 | mwax-db2fits | Unix time of the start of this observation. This field is used by mwax-db2fits to provide a UNIX timestep for each integration throughout the observation. |
| UNIXTIME_MSEC | 0 | mwax-db2fits | Unix time milliseconds offset of the start of this observation. This will always be 0 with the current setup, but is here for completeness. |
FINE_CHAN_WIDTH_HZ | 10000 | mwax-db2fits | Correlator output fine channel width in Hz |
| NFINE_CHAN | 128 | mwax_db2correlate2db mwax-db2fits | Correlator output number of fine channels |
| BANDWIDTH_HZ | 1280000 | mwax-db2fits | Coarse channel passband width in Hz, i.e. the channel spacing |
SAMPLE_RATE | 1280000 | mwax_db2correlate2db | Actual sample rate of input data in samples/s (whether critically-sampled or oversampled). |
| MC_IP | 0.0.0.0 | mwax-db2fits | Multicast IP for this coarse channel (for debug purposes) |
| MC_PORT | 0 | mwax-db2fits | Multicast port for this coarse channel (for debug purposes) |
| MWAX_U2S_VER | X.Y.Z | mwax_db2fits | Version number of mwax_u2s used to capture the UDP packets for this subfile |
| MWAX_SUB_VER | 1 | mwax_u2s | Version number of subfile format. A value of 2 indicates the presence of the following fields: |
| IDX_PACKET_MAP | (OFFSET)+(SIZE)¹ | mwax_u2s | Location and size of packet map in block 0 metadata |
| IDX_DELAY_TABLE | (OFFSET)+(SIZE)¹ | mwax_u2s | Location and size of delay table in block 0 metadata |
| IDX_MARGIN_DATA | (OFFSET)+(SIZE)¹ | mwax_u2s | Location and size of margin data section in block 0 metadata |
| IDX_OCCUPANCY_TABLE | (OFFSET)+(SIZE)¹ | mwax_u2s | Location and size of data occupancy table in block 0 metadata |
...
The table schema is as follows:
| # | Name | Type | Byte offset | Size (bytes) | Description |
|---|---|---|---|---|---|
| 0 | rf_input | uint16 | 0 | 2 | RF input ID. This is the tile ID right-shifted by 1,with the lower bit indicating polarity: 0 for X,1 for Y. |
| 1 | ws_delay | int16 | 2 | 2 | Whole sample delay. This indicates how many samples that u2s has shifted the data stream forward (negative values indicate a backwards shift). |
| 2 | initial_delay | float64 | 4 | 8 | The calculated delay at the start of the subobservation, in milliseconds. |
| 3 | delta_delay | float64 | 12 | 8 | The rate of change of the delay, in milliseconds per FFT block (currently 5ms). |
| 4 | delta_delta_delay | float64 | 20 | 8 | The rate of change of the delta_delay, in milliseconds per FFT block (currently 5ms). |
| 5 | start_total_delay | float64 | 28 | 8 | Equivalent to initial_delay, first point for quadratic interpolation of fractional delay values. |
| 6 | middle_total_delay | float64 | 36 | 8 | Second point for quadratic interpolation of fractional delay values, delay at middle of subobservation. |
| 7 | end_total_delay | float64 | 44 | 8 | Third point for quadratic interpolation of fractional delay values, delay at end of subobservation. |
| 8 | num_pointings | uint16 | 52 | 2 | The number of fractional delay values in the row, N. |
| 9 | _reserved | uint16 | 54 | 2 | Reserved for future use. Must always be 0. Used by subtool for automatic detection of delay table format in exported delay tables. |
| 10 .. N | frac_delay[1..N] | float32 | 56 | 4N | Fractional delay values, in microsamples. |
Block 0: Margin data
To facilitate the removal or editing of whole-sample delays applied by u2s, which would ordinarily be a lossy operation, the full contents of the packets at either end of the subobservation are recorded for each RF source. These "margins" consist of the first packet within the subobservation and the packet immediately preceding it, and the last packet within the subobservation and the packet that follows it, for a total of 4 packets worth of data per RF source. The table consists of rows for each RF source, in the same order as the voltage sample data. This table is not formally specified for version 1 subfiles (though it has been present for a long time, it is left as an exercise for the reader to find it in the unlikely event there is a need to). In version 2 subfiles, the delay table offset and size is defined by the IDX_MARGIN_DATA field in the PSRDADA header.
The table schema is as follows. Note that for the purpose of this table, we use a single 16-bit unsigned integer to represent 1 "sample". More properly speaking, these represent two 8-bit values, forming the real and imaginary parts of a complex integer.
| # | Name | Type | Offset | Length | Description |
|---|---|---|---|---|---|
| 0 - 2047 | Head | uint16 | 0 | 4096 | Voltage data from the packet immediately before the start of the subobservation. |
| 2048 - 4095 | First | uint16 | 4096 | 4096 | Voltage data from the first packet of the subobservation. |
| 4096 - 6143 | Last | uint16 | 8192 | 4096 | Voltage data from the last packet of the subobservation. |
| 6144 - 8191 | Tail | uint16 | 12288 | 4096 | Voltage data from the packet immediately following the end of the subobservation. |
Block 0: UDP Packet Map
The UDP packet map is a bitmap structure representing which of the expected packets for the subobservation were received, and which were missing, with a value of 1 indicating a received packet. It may be considered to be organised into rows representing each RF source, in the same order as the voltage data samples, and 1-bit columns, ordered by time. Thus the byte length of a row is the number of packets per second, multiplied by the number of seconds per subobservation, divided by the number of bits per byte. Both of these values happen to be 8, so the byte length of the row is simply the packets-per-second value - currently either 625 for critically-sampled observations, or 800 for oversampling mode. This table is not formally specified for version 1 subfiles (though it has been present for a long time, it is left as an exercise for the reader to find it in the unlikely event there is a need to). In version 2 subfiles, the delay table offset and size is defined by the IDX_PACKET_MAP field in the PSRDADA header.