VCSBeam: Offline Correlator

*** WARNING: Multiple versions of the offline correlator exist, and the difference between them, and how they are accessed, is currently not well documented! This page describes the most recent version available through VCSBeam. ***

For the standard use case of creating correlated visibilities from legacy data with a time integration of at least 100 ms, the version of the offline correlator that can be accessed via the instructions given on the legacy page should be used. The version that ships with VCSBeam is intended for fast imaging, capable of producing visibilities with a minimum of 2 ms integrations. The choice of 2 ms was set partly by the constraints set by the underlying (3rd party) correlator engine, xGPU, which requires the number of time steps per integration to be a multiple of 4, but at the same time must divide the number of timesteps in a second evenly. Since there are 10000 time steps per second (for the legacy data), the smallest number of time steps that fits both criteria is 4 (= 0.4 ms), but 20 time steps (= 2 ms) was chosen as a "rounder" number, considered adequate for science cases involving rapid transients, such as searching for FRBs.

The output of the Offline Correlator is a GPUBox file which conforms to the FITS standard, and which can be read in and interpreted using the mwalib library.

Currently, each call to offline_correlator processes only a single legacy .dat file (i.e. a single second, a single coarse channel). Processing of multiple seconds and coarse channels requires running offline_correlator in batches.

Command line options

offline_correlator: a light-weight correlator for the MWA. Takes VCS data files and correlates as per the parameters of the linked xGPU library
usage: offline_correlator -c <coarse_channel> -d <infile> [options]
Options:
   -e EDGES
        Set EDGES channels at both top and bottom of the band to 0 [default: 0]
   -h
        Display this help and exit
   -n CHAN_AVERAGE
        Average CHAN_AVERAGE adjacent channels in the final output [default: 4]
   -o OBSID
        The observation ID of the input VCS data [required]
   -r DUMPS_PER_SECOND
        The number of correlator dumps per second to write to file [default: 1]
   -s STARTTIME
        The time (in Unix seconds) corresponding to the input file[required]

Example of use on Garrawarla

offline_correlator_example.sh

#!/bin/bash -l

#SBATCH --nodes=1
#SBATCH --mem=370gb
#SBATCH --partition=gpuq
#SBATCH --gres=gpu:1
#SBATCH --time=00:10:00
#SBATCH --account=mwavcs
#SBATCH --export=NONE

module use /pawsey/mwa/software/python3/modulefiles
module load vcsbeam/correlator

INPUT_DATA_FILE=/path/to/recombined/data/file/1313388760_1313388762_ch144.dat
START_SECOND=1313388762
DUMPS_PER_SECOND=20 # This sets the output time resolution
                    # (e.g. 20 --> 1/20 = 0.05s = 50 ms)
                    # Minimum allowed resolution is 2 ms
CHANS_TO_AVERAGE=4 # This sets the output frequency resolution
                   # (e.g. 4 --> 4x10 kHz = 40 kHz)
GPUBOX_CHANNEL_NUMBER=20 # This should be chosen to "match" the input channel
                         # This is not easy! (mwalib handles this, but at the
                         # moment, offline_correlator is not using mwalib)
OUTPUT_PREFIX=1313388760 # Output files begin with this

srun -N 1 -n 1 offline_correlator \
    -d ${INPUT_DATA_FILE} \
    -s ${START_SECOND} \
    -r ${DUMPS_PER_SECOND} \
    -n ${CHANS_TO_AVERAGE} \
    -c ${GPUBOX_CHANNEL_NUMBER} \
    -o ${OUTPUT_PREFIX}