Versions Compared

Old Version 19

changes.mady.by.user Samuel James McSweeney

Saved on

compared with

New Version Current

changes.mady.by.user Samuel James McSweeney

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

ObsidDuration (sec)FilesPulsar(s) / Calibrator(s)RA/DecPC RA/DecAz/ElGoalProcessingDownload folderVCSBeam version*Beamforming time
132369032029624In-beam calibration (for 1323690616)11.8 / -5.88.5 / -7.4341.6 / 68.01, 5/astro/mwavcs/asvo/253001v3.3.31_c279695
132369061612003600 VCSB0031-07
J0036-1033
J0034-0534		8.0 / -5.28.5 / -7.4326.3 / 64.71, 5/astro/mwavcs/asvo/253008v3.3.31_c2796951 channel: 1600s
24 channels: 4200-5000s
132370861629624In-beam calibration (for 1323709032)95.1 / -26.797.7 / -28.60.0 / 90.01, 4/astro/mwavcs/asvo/253000

132370891212024PicA (for 1323709032)75.9 / -46.480.0 / -45.8213.7 / 64.71, 4/astro/mwavcs/asvo/252998

13237090326001800 VCSB0628-2897.5 / -26.797.7 / -28.60.0 / 90.01, 4/astro/mwavcs/asvo/253009

132377672012024PKS2356-61 (for 1323776840)4.3 / -62.7359.8 / -60.9191.3 / 52.81, 2/astro/mwavcs/asvo/252999

132377684012003600 VCSJ2241-5236346.9 / -51.9340.4 / -52.6216.9 / 53.61, 2/astro/mwavcs/asvo/253010

*VCSBeam version is the earliest version that was successfully used for the processing. For calibration observations, this means using mwa_plot_calibration, and for the VCS observations, make_mwa_tied_array_beam.

Processing details

...

Run Hyperdrive

...

Setting up Garrawarla environment

#!/bin/bash -l #SBATCH --job-name=hyp-$1 #SBATCH --output=hyperdrive.out #SBATCH --nodes=1 #SBATCH --ntasks-per-node=40 #SBATCH --time=01:00:00 #SBATCH --clusters=garrawarla #SBATCH --partition=gpuq #SBATCH --account=mwavcs #SBATCH --export=NONE #SBATCH --gres=tmp:50g #SBATCH --gres=gpu:1 # Load the Hyperdrive module module use /pawsey/mwa/software/python3/modulefiles module load hyperdrive set -eux which hyperdrive #
Code Block
languagebash
themeMidnight
title/astro/mwavcs/vcs/1323690320/hyperdrive/hyperdrive.sbatch
collapsetrue
Setting up Garrawarla environment
module use /pawsey/mwa/software/python3/modulefiles
module load singularity
module load vcsbeam
module load psrcat
module load hyperdrive

1323690320
Anchor
1323690320
1323690320

Run Hyperdrive

Create Hyperdrive batch script:

Code Block
languagebash
themeMidnight
title/astro/mwavcs/vcs/1323690320/hyperdrive/hyperdrive.sbatch
collapsetrue
#!/bin/bash -l
#SBATCH --job-name=hyp-$1
#SBATCH --output=hyperdrive.out
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=40
#SBATCH --time=01:00:00
#SBATCH --clusters=garrawarla
#SBATCH --partition=gpuq
#SBATCH --account=mwavcs
#SBATCH --export=NONE
#SBATCH --gres=tmp:50g
#SBATCH --gres=gpu:1

# Load the Hyperdrive module
module use /pawsey/mwa/software/python3/modulefiles
module load hyperdrive

set -eux
which hyperdrive

# The location of the correlated visibility FITS files:
DATADIR=/astro/mwavcs/asvo/253001

# The path to the associated metafits file
CAL_METAFITS=${DATADIR}/1323690320.metafits

# The base source list (you shouldn't need to change this)
SRCLIST=/pawsey/mwa/software/python3/srclists/master/srclist_pumav3_EoR0aegean_fixedEoR1pietro+ForA_phase1+2.txt

# If it hasn't been done already, run Hyperdrive in "srclist-by-beam" to get just the subset of sources in the beam
if [[ ! -r srclist_1000.yaml ]]; then
   hyperdrive srclist-by-beam \
       -n 1000 \
       -m ${CAL_METAFITS} \
       /pawsey/mwa/software/python3/srclists/master/srclist_pumav3_EoR0aegean_fixedEoR1pietro+ForA_phase1+2.txt \
       srclist_1000.yaml
fi

# Run Hyperdrive to generate a calibration solution
hyperdrive di-calibrate \
    -s srclist_1000.yaml \
    -d ${DATADIR}/*.fits \
    ${CAL_METAFITS}

...

Code Block
languagebash
themeMidnight
titleRun Hyperdrive on Garrawarla
cd /astro/mwavcs/vcs/1323690320/hyperdrive
sbatch hyperdrive.sbatch

Run VCSBeam plotting

Create VCSBeam : v3.3.30_edafd20

Create VCSBeam plotting script:

...

plotting script:

Code Block
languagebash
themeMidnight
title/astro/mwavcs/vcs/1323690320/hyperdrive/plot.sbatch
collapsetrue
#!/bin/bash -l
#SBATCH --job-name=plot_calsol
#SBATCH --output=plot_calsol.out
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=2
#SBATCH --time=01:00:00
#SBATCH --clusters=garrawarla
#SBATCH --partition=gpuq
#SBATCH --account=mwavcs
#SBATCH --export=NONE
#SBATCH --gres=tmp:50g
#SBATCH --gres=gpu:1

# Load the VCSBeam module
module use /pawsey/mwa/software/python3/modulefiles
module load vcsbeam

# The location of the correlated visibility FITS files:
DATADIR=/astro/mwavcs/asvo/253001

# The paths to the associated metafits files
CAL_METAFITS=${DATADIR}/1323690320.metafits
OBS_METAFITS=/astro/mwavcs/asvo/253008/1323690616.metafits # <-- This is the observation for which the calibration solution is needed

# Use VCSBeam to plot the solutions
srun -N 1 -n 1 mwa_plot_calibration -m ${OBS_METAFITS} -c ${CAL_METAFITS} -C hyperdrive_solutions.bin -R NONE -U 0,0 -O -X -z nan > plot_data.txt

# The following will have to be done on your own computer (not on Garrawarla)
# Download
#    (1) plot_calibration.py (use "which plot_calibration.py" to find out where it is)
#    (2) plot_data.txt
# and then run the following
#plot_calibration.py  --phases_png phases.png  --amps_png amps.png  plot_data.txt

...

It looks like the solutions are probably fine for most, if not all the antennas (except possibly HexE23, since there's a lot of power in the off-diagonal terms, Dpq and Dqp), but the curviness of the phases makes it hard to tell. Therefore, I will choose a reference antenna to plot all the phases relative to that. Since there's a bunch of tiles whose phases are strongly sloped (in near the middle of the plot), I'll choose a reference tile with a negative slope as well, but whose gradient is not as steep... HexE25 will do. To do this, I need to re-run the plotting script with the following change:

...

Code Block
themeMidnight
-R NONE -U 0,0
# (and without -X)

...

 NONE -U 0,0
# (and without -X)

1323690616
Anchor
1323690616
1323690616

Make sure you have a calibration solution ready. this example uses the solution derived from observation 1323690320.

B0031-07
Anchor
1323690616-B0031-07
1323690616-B0031-07

Create a VCSBeam beamforming script:

Code Block
languagebash
themeMidnight
title/astro/mwavcs/vcs/1323690616/vcsbeam/vcsbeam_0031.sbatch
collapsetrue
#!/bin/bash -l

#SBATCH --job-name=1323690616-0031
#SBATCH --output=vcsbeam_0031.out
#SBATCH --nodes=24
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=1
#SBATCH --mem=370gb
#SBATCH --partition=gpuq
#SBATCH --gres=gpu:1
#SBATCH --time=03:00:00
#SBATCH --account=mwavcs
#SBATCH --nice=0
#SBATCH --export=NONE

# Load the VCSBeam module 
module use /pawsey/mwa/software/python3/modulefiles
module load vcsbeam

# Define the directories where the data are
DATADIR=/astro/mwavcs/asvo/253008
CALSOL=/astro/mwavcs/vcs/1323690320/hyperdrive/hyperdrive_solutions.bin

OBS_METAFITS=${DATADIR}/1323690616.metafits
CAL_METAFITS=/astro/mwavcs/asvo/253001/1323690320.metafits

# Run VCSBeam for all 24 coarse channels
# Start 8 seconds in, just to avoid quacktime
# -p -v = Output both PSRFITS and VDIF
srun -N 24 -n 24 make_mwa_tied_array_beam \
    -m ${OBS_METAFITS} \
    -b 1323690624 \
    -T 1192 \
    -f 109 \
    -d ${DATADIR} \
    -P pointings_0031.txt \
    -c ${CAL_METAFITS} \
    -C ${CALSOL} \
    -O \
    -R NONE -U 0,0 \
    -p \
    -v


Prepare a pointings file (PSR B0031-07):

Code Block
languagebash
themeMidnight
title/astro/mwavcs/vcs/1323690616/vcsbeam/pointings_0031.txt
collapsetrue
00:34:08.8703 -07:21:53.409

Run the VCSBeam beamforming script:

Code Block
languagebash
themeMidnight
titleRun VCSBeam beamforming on Garrawarla
cd /astro/mwavcs/vcs/1323690616/vcsbeam
sbatch vcsbeam_0031.sbatch

Generate an ephemeris:

Code Block
languagebash
themeMidnight
titleRun psrcat on Garrawarla
psrcat -e J0034-0534 > 0034.par

Use PRESTO's prepfold utility to fold on the pulsar's period from the FITS files:

Code Block
languagebash
themeMidnight
titleRun PRESTO prepfold on Garrawarla
cd /astro/mwavcs/vcs/1323690616/vcsbeam
/pawsey/mwa/singularity/presto/presto.sif prepfold -psr J0034-0721 -nosearch -nodmsearch G0057_1323690616_00:34:08.87_-07:21:53.41_ch109-132_00*.fits


Image Added

Use DSPSR and PSRCHIVE to fold on the pulsar's period from the VDIF files:

Code Block
languagebash
themeMidnight
titleRun PRESTO prepfold on Garrawarla
cd /astro/mwavcs/vcs/1323690616/vcsbeam
for freq in {109..132}; do /pawsey/mwa/singularity/dspsr/dspsr.sif dspsr -E 0031.par -b 512 -L 10 -A -O B0031-07_ch${freq}.ar G0057_1323690616_00:34:08.87_-07:21:53.41_ch${freq}.hdr; done
/pawsey/mwa/singularity/dspsr/dspsr.sif psradd -o B0031-07.ar -R B0031-07_ch1*
# (Did some pazi flagging: there was something weird in the first subint. This generated B0031-07.ar.pazi)
pav -g B0031-07.png/png -DFT B0031-07.ar.pazi

Image Added

J0036-1033
Anchor
1323690616-J0036-1033
1323690616-J0036-1033

Create a VCSBeam beamforming script:

Code Block
languagebash
themeMidnight
title/astro/mwavcs/vcs/1323690616/vcsbeam/vcsbeam_0036.sbatch
collapsetrue
#!/bin/bash -l

#SBATCH --job-name=1323690616-0036
#SBATCH --output=vcsbeam_0036.out
#SBATCH --nodes=24
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=1
#SBATCH --mem=370gb
#SBATCH --partition=gpuq
#SBATCH --gres=gpu:1
#SBATCH --time=03:00:00
#SBATCH --account=mwavcs
#SBATCH --nice=0
#SBATCH --export=NONE

# Load the VCSBeam module 
module use /pawsey/mwa/software/python3/modulefiles
module load vcsbeam

# Define the directories where the data are
DATADIR=/astro/mwavcs/asvo/253008
CALSOL=/astro/mwavcs/vcs/1323690320/hyperdrive/hyperdrive_solutions.bin

OBS_METAFITS=${DATADIR}/1323690616.metafits
CAL_METAFITS=/astro/mwavcs/asvo/253001/1323690320.metafits

# Run VCSBeam for all 24 coarse channels
# Start 8 seconds in, just to avoid quacktime
# -p -v = Output both PSRFITS and VDIF
srun -N 24 -n 24 make_mwa_tied_array_beam \
    -m ${OBS_METAFITS} \
    -b 1323690624 \
    -T 1192 \
    -f 109 \
    -d ${DATADIR} \
    -P pointings_0036.txt \
    -c ${CAL_METAFITS} \
    -C ${CALSOL} \
    -O \
    -R NONE -U 0,0 \
    -p \
    -v

Prepare a pointings file (PSR J0036-1033):

Code Block
languagebash
themeMidnight
title/astro/mwavcs/vcs/1323690616/vcsbeam/pointings_0036.txt
collapsetrue
00:36:15.1 -10:33:14

Run the VCSBeam beamforming script:

Code Block
languagebash
themeMidnight
titleRun VCSBeam beamforming on Garrawarla
cd /astro/mwavcs/vcs/1323690616/vcsbeam
sbatch vcsbeam_0036.sbatch

Get an ephemeris from Nick:

Code Block
languagebash
themeMidnight
title/astro/mwavcs/vcs/1323690616/vcsbeam/0036.par
collapsetrue
PSRJ           J0036-1033
RAJ            00:36:14.70                 
DECJ           -10:33:37.18                
F0             1.1111412561028151282     1  0.00000000152553465775   
F1             -3.9179999999999999999e-14    2.264e-14                
PEPOCH         46635                       
POSEPOCH       46635                       
DMEPOCH        46635                       
DM             23.123                      
EPHEM          DE405

Use PRESTO's prepfold utility to fold on the pulsar's period from the FITS files:

Code Block
languagebash
themeMidnight
titleRun PRESTO prepfold on Garrawarla
cd /astro/mwavcs/vcs/1323690616/vcsbeam
/pawsey/mwa/singularity/presto/presto.sif prepfold -par 0036.par -nosearch -nodmsearch G0057_1323690616_00:36:15.10_-10:33:14.00_ch109-132_00*.fits

Image Added

Use DSPSR and PSRCHIVE to fold on the pulsar's period from the VDIF files:

Code Block
languagebash
themeMidnight
titleRun PRESTO prepfold on Garrawarla
cd /astro/mwavcs/vcs/1323690616/vcsbeam
for freq in {109..132}; do /pawsey/mwa/singularity/dspsr/dspsr.sif dspsr -E 0036.par -b 512 -L 10 -A -O J0036-1033_ch${freq}.ar G0057_1323690616_00:36:15.10_-10:33:14.00_ch${freq}.hdr; done

J0034-0534
Anchor
1323690616-J0034-0534
1323690616-J0034-0534

Create a VCSBeam beamforming script:

Code Block
languagebash
themeMidnight
title/astro/mwavcs/vcs/1323690616/vcsbeam/vcsbeam_0036.sbatch
collapsetrue
#!/bin/bash -l

#SBATCH --job-name=1323690616-0034
#SBATCH --output=vcsbeam_0034.out
#SBATCH --nodes=24
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=1
#SBATCH --mem=370gb
#SBATCH --partition=gpuq
#SBATCH --gres=gpu:1
#SBATCH --time=03:00:00
#SBATCH --account=mwavcs
#SBATCH --nice=0
#SBATCH --export=NONE

# Load the VCSBeam module 
module use /pawsey/mwa/software/python3/modulefiles
module load vcsbeam

# Define the directories where the data are
DATADIR=/astro/mwavcs/asvo/253008
CALSOL=/astro/mwavcs/vcs/1323690320/hyperdrive/hyperdrive_solutions.bin

OBS_METAFITS=${DATADIR}/1323690616.metafits
CAL_METAFITS=/astro/mwavcs/asvo/253001/1323690320.metafits

# Run VCSBeam for all 24 coarse channels
# Start 8 seconds in, just to avoid quacktime
# -p -v = Output both PSRFITS and VDIF
srun -N 24 -n 24 make_mwa_tied_array_beam \
    -m ${OBS_METAFITS} \
    -b 1323690624 \
    -T 1192 \
    -f 109 \
    -d ${DATADIR} \
    -P pointings_0034.txt \
    -c ${CAL_METAFITS} \
    -C ${CALSOL} \
    -O \
    -R NONE -U 0,0 \
    -p \
    -v

Prepare a pointings file (PSR J0034-0534):

Code Block
languagebash
themeMidnight
title/astro/mwavcs/vcs/1323690616/vcsbeam/pointings_0036.txt
collapsetrue
00:34:21.83422 -05:34:36.722

Run the VCSBeam beamforming script:

Code Block
languagebash
themeMidnight
titleRun VCSBeam beamforming on Garrawarla
cd /astro/mwavcs/vcs/1323690616/vcsbeam
sbatch vcsbeam_0034.sbatch

Use PRESTO's prepfold utility to fold on the pulsar's period from the FITS files:

Code Block
languagebash
themeMidnight
titleRun PRESTO prepfold on Garrawarla
cd /astro/mwavcs/vcs/1323690616/vcsbeam
/pawsey/mwa/singularity/presto/presto.sif prepfold -psr J0034-0534 -nosearch -nodmsearch G0057_1323690616_00:34:21.83_-05:34:36.72_ch109-132_00*.fits

Image Added

FAIL!!

Generate an ephemeris:

Code Block
languagebash
themeMidnight
titleRun psrcat on Garrawarla
psrcat -e J0034-0534 > 0034.par

Use DSPSR and PSRCHIVE to fold on the pulsar's period from the VDIF files:

Code Block
languagebash
themeMidnight
titleRun PRESTO prepfold on Garrawarla
cd /astro/mwavcs/vcs/1323690616/vcsbeam
psrcat -e J0034-0534 > 0034.par
for freq in {109..132}; do /pawsey/mwa/singularity/dspsr/dspsr.sif dspsr -E 0034.par -b 512 -L 10 -A -O J0034-0534_ch${freq}.ar G0057_1323690616_00:34:21.83_-05:34:36.72_ch${freq}.hdr; done

1323708616
Anchor
1323708616
1323708616

...