Skip to end of metadata
Go to start of metadata

You are viewing an old version of this content. View the current version.

Compare with Current View Version History

« Previous Version 14 Current »

***THIS PAGE (and the code it describes) IS UNDER ACTIVE DEVELOPMENT***

The beamforming pipeline described here is applicable to fine-channelised (10 kHz) MWA data that was produced up to September 2021 (hereafter called legacy data), when the legacy Voltage Capture System was decommissioned. The upgraded system, MWAX (documented elsewhere on this wiki), produces coarse-channelised (1.28 MHz) high-time resolution (HTR) data (hereafter called MWAX data; see the format description here). Apart from the obvious requirement to be able to process the new data format, a number of other factors have motivated the development of VCSBeam, which operates sufficiently differently from the legacy beamformer that it has now branched off into its own repository, housed at this GitHub page.


Processing Pipeline Overview

Individual processing steps

  1. Download (not part of VCSBeam): This step is currently identical to the downloading step described on the legacy page. For legacy data, the existing pipeline performs the "recombine step" automatically, so that the data which are made available to the user are the "Recombined voltages", or ".dat" files. As of this writing (25 Sep 2021), it is not yet clear whether the current downloading instructions will correctly download MWAX data (i.e. after September 2021), or whether that functionality still needs to be implemented. However, it is intended that the data that will be downloaded are the "Coarse channel voltages", or ".sub" files.
  2. Offline PFB: This step mimics the FPGA-based polyphase filter bank that was implemented in the legacy system, but which is not supplied by the MWAX system. It is currently (as of September 2021), the only way to process MWAX data, although there are many outstanding issues with this that are described below.
  3. Offline correlator: Used for producing correlated visibilities in the form of GPUBox files, e.g., for in-beam calibration, or making fast images.
  4. Beamformer: This produces the maximum sensitivity towards a desired look-direction. It incorporates the multi-pixel functionality described in Swainston et al. (in prep). The outputs of the beamformer are either full Stokes PSRFITS files at 100 μs resolution, or dual polarisation (XY) VDIF files at 0.78 μs resolution.

Download

Refer to the legacy page for instructions.

Offline PFB

Documentation for the Offline PFB is found here.

Offline Correlator

For the standard use case of creating correlated visibilities from legacy data with a time integration of at least 100 ms, the version of the offline correlator that can be accessed via the instructions given on the legacy page should be used. The version that ships with VCSBeam is intended for fast imaging, capable of producing visibilities with a minimum of 2 ms integrations. The choice of 2 ms was set partly by the constraints set by the underlying (3rd party) correlator engine, xGPU, which requires the number of time steps per integration to be a multiple of 4, but at the same time must divide the number of timesteps in a second evenly. Since there are 10000 time steps per second (for the legacy data), the smallest number of time steps that fits both criteria is 4 (= 0.4 ms), but 20 time steps (= 2 ms) was chosen as a "rounder" number, considered adequate for science cases involving rapid transients, such as searching for FRBs.

The output of the Offline Correlator is a GPUBox file which conforms to the FITS standard, and which can be read in and interpreted using the mwalib library.

Currently, each call to offline_correlator processes only a single legacy .dat file (i.e. a single second, a single coarse channel). Processing of multiple seconds and coarse channels requires running offline_correlator in batches.

Example of use on Garrawarla

offline_correlator_example.sh 
#!/bin/bash -l

#SBATCH --nodes=1
#SBATCH --mem=370gb
#SBATCH --partition=gpuq
#SBATCH --gres=gpu:1
#SBATCH --time=00:10:00
#SBATCH --account=mwavcs
#SBATCH --export=NONE

module use /pawsey/mwa/software/python3/modulefiles
module load vcsbeam/correlator

INPUT_DATA_FILE=/path/to/recombined/data/file/1313388760_1313388762_ch144.dat
START_SECOND=1313388762
DUMPS_PER_SECOND=20 # This sets the output time resolution
                    # (e.g. 20 --> 1/20 = 0.05s = 50 ms)
                    # Minimum allowed resolution is 2 ms
CHANS_TO_AVERAGE=4 # This sets the output frequency resolution
                   # (e.g. 4 --> 4x10 kHz = 40 kHz)
GPUBOX_CHANNEL_NUMBER=20 # This should be chosen to "match" the input channel
                         # This is not easy! (mwalib handles this, but at the
                         # moment, offline_correlator is not using mwalib)
OUTPUT_PREFIX=1313388760 # Output files begin with this

srun -N 1 -n 1 offline_correlator \
    -d ${INPUT_DATA_FILE} \
    -s ${START_SECOND} \
    -r ${DUMPS_PER_SECOND} \
    -n ${CHANS_TO_AVERAGE} \
    -c ${GPUBOX_CHANNEL_NUMBER} \
    -o ${OUTPUT_PREFIX}


Beamformer

VCSBeam provides two binaries for beamforming:

  1. make_mwa_incoh_beam
  2. make_mwa_tied_array_beam

Incoherent beam

Example of use on Garrawarla

Tied-array beam

Example of use on Garrawarla
Example of use on Garrawarla 
#!/bin/bash -l

#SBATCH --nodes=1
#SBATCH --mem=370gb
#SBATCH --partition=gpuq
#SBATCH --gres=gpu:1
#SBATCH --time=01:00:00
#SBATCH --account=mwavcs
#SBATCH --export=NONE

module use /pawsey/mwa/software/python3/modulefiles
module load vcsbeam

srun -N 24 -n 24 make_mwa_tied_array_beam \
        -m PATH/TO/1240826896_metafits_ppds.fits \
        -b 1240826897 \
        -T 295 \
        -f 133 \
        -d PATH/TO/VCS/DATA \
        -P pointings.txt \
        -c PATH/TO/CAL/1240827912.metafits \
        -C PATH/TO/RTS/SOLUTION \
        -p



  • No labels