hyperdrive-di-calibrate 0.2.0-alpha9
Perform direction-independent calibration on the input MWA data. See for more info:
https://github.com/MWATelescope/mwa_hyperdrive/wiki/Calibration-usage

USAGE:
    hyperdrive di-calibrate [OPTIONS] [--dont-convert-lists ] [ARGUMENTS_FILE]

ARGS:
Don't attempt to convert "list" flux densities to power law flux densities. See for more
           <ARGUMENTS_FILE>    All of the arguments to di-calibrate may be specified in a toml or json file. Any CLI arguments
                     info: https://github.com/MWATelescope/mwa_hyperdrive/wiki/Source-lists  override parameters set in the file

--filter-pointsOPTIONS:
    -v, --verbosity   Don't includeThe pointverbosity componentsof from the inputprogram. skyIncrease modelby specifying multiple       --filter-gaussians      Don't include Gaussian components from the input sky modeltimes (e.g. -vv). The default is to print
                 --filter-shapelets      Don't include shapelet components from the input sky modelonly high-level information
          -v, -dry-verbosityrun      Don't actually do calibration; just verify that Thearguments verbositywere ofcorrectly theingested program.and Theprint default is to print out high-level
information         --dry-run              level Don'tinformation
actually do any work; just verify that the input arguments were correctly ingested
   -h, --help         Print help information
    -V, --version      Print version information

INPUT FILES:
    -d, --data <DATA>...     and print out high-level information         --cpu       Paths to input data files to be calibrated. These can include a Usemetafits thefile,
CPU for visibility generation. This is deliberately made non-default because                                 using a GPU is much faster  gpubox files, mwaf -hfiles, --helpa measurement set and/or uvfits files
    -s,   --source-list <SOURCE_LIST>     Prints help information     -V, --version Path to the sky-model source list file
       Prints version information

OPTIONS:
    -s, --source-list <source-list>type <SOURCE_LIST_TYPE>    The type of sky-model source list. Valid types are: hyperdrive, rts, woden,
 Path to the sky-model source list used for simulation     -m, --metafits <metafits>                      Path to the metafits file     -o, --output-model-file <output-model-file>    Path to the output visibilities file [default: model.uvfits]
    -r, --ra <ra> ao. If not specified, all types are attempted

OUTPUT FILES:
    -o, --outputs <OUTPUTS>...
            ThePaths phaseto centrethe rightcalibration ascensionoutput [degrees]files. Supported If this is not specified, then the metafits phase/pointingcalibrated visibility outputs: uvfits. Supported calibration
            solution formats: fits, bin. centre is usedDefault: hyperdrive_solutions.bin

    -dm, --dec <dec>model-filename <MODEL_FILENAME>
            The phase centre declination [degrees] path to the file where the generated sky-model visibilities are written. If this isargument notisn't specifiedsupplied, then
the metafits phase/pointing centre is        no file is written.  usedSupported formats: uvfits

       -c, --numignore-fine-channels <num-fine-channels>autos
            When Thewriting totalout numbercalibrated of fine channels in the observation [default: 384]visibilities, don't include auto-correlations

        -f, -output-freqvis-res <freqtime-res> average <OUTPUT_VIS_TIME_AVERAGE>
            When writing out calibrated visibilities, average this many The fine-channel resolution [kHz] [default: 80]
timesteps together. Also supports a target time
       --middle-freq <middle-freq>    resolution (e.g. 8s). The value must be a multiple Theof middlethe frequencyinput ofdata's the simulation [MHz]time resolution. IfThe thisdefault is notto specified,preserve
then the middle frequency specified        the input data's time  inresolution. e.g. If the input metafitsdata is used
    -t, --num-timesteps <num-timesteps> in 0.5s resolution and this variable is 4, then we
            average The2s numberworth of calibrated timedata stepstogether usedbefore fromwriting the metafitsdata epoch [default: 14]

    out. If the variable is instead 4s, then 8
   --time-res <time-res>        calibrated timesteps are averaged together before writing the data out

   The time resolution [seconds] [default: 8]
 --output-vis-freq-average <OUTPUT_VIS_FREQ_AVERAGE>
       --beam-file <beam-file>    When writing out calibrated visibilities, average this many fine Thefreq. path to the HDF5 MWA FEE beam file. If not specified, thischannels together. Also supports a target
            freq. resolution (e.g. 80kHz). The value must be provideda multiple byof the MWA_BEAM_FILE
      input data's freq. resolution. The default is to
     environment variable      preserve the  --dipole-delays <dipole-delays>...
      input data's freq. resolution. e.g. If the input data is in 40kHz resolution and this variable is 4,
     Specify the MWA dipoles delays, ignoring whatever is in the metafits file

#!/bin/bash -l
#SBATCH --job-name=hyp-$1
#SBATCH --output=hyperdrive.out
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=40
#SBATCH --time=01:00:00
#SBATCH --clusters=garrawarla
#SBATCH --partition=gpuq
#SBATCH --account=mwaeor
#SBATCH --export=NONE
#SBATCH --gres=tmp:50g
#SBATCH --gres=gpu:1

module use /pawsey/mwa/software/python3/modulefiles
module load hyperdrive

set -eux
which hyperdrive

cd /astro/mwaeor/MWA/data/1090008640

# Make a source list if it isn't already there
if [[ ! -r srclist_1000.yaml ]]; then
   hyperdrive srclist-by-beam -n 1000 -m *.metafits \
              /pawsey/mwa/software/python3/srclists/master/srclist_pumav3_EoR0aegean_fixedEoR1pietro+ForA_phase1+2.txt \
              srclist_1000.yaml
fi

hyperdrive di-calibrate -s srclist_1000.yaml -d *gpubox*.fits *.metafits

# Write solutions to "hyp_sols.fits" and calibrated vis to "hyp_cal.uvfits"
hyperdrive di-calibrate -s srclist_1000.yaml \
						-d *gpubox*.fits *.metafits \
						-o hyp_sols.fits hyp_cal.uvfits \
						--output-vis-time-average 2 \
						--output-vis-freq-average 80kHz

hyperdrive solutions-plot -m *.metafits hyp_sols.fits

#!/usr/bin/env python

import sys
import numpy as np
from astropy.io import fits
import matplotlib.pyplot as plt

if len(sys.argv) == 1:
    filename = "hyp_sols.fits"
else:
    filename = sys.argv[1]

f = fits.open(filename)
data = f[1].data
# Only looking at the first timeblock.
i_timeblock = 0
data = data[i_timeblock, :, :, ::2] + data[i_timeblock, :, :, 1::2] * 1j

# Uncomment if you want to divide by a reference.
# i_tile_ref = -1
# refs = []
# for ref in data[i_tile_ref].reshape((-1, 2, 2)):
#     refs.append(np.linalg.inv(ref))
# refs = np.array(refs)
# j_div_ref = []
# for tile_j in data:
#     for (j, ref) in zip(tile_j, refs):
#         j_div_ref.append(j.reshape((2, 2)).dot(ref))
# data = np.array(j_div_ref).reshape(data.shape)

# Amps
amps = np.abs(data)

_, ax = plt.subplots(8, 16, sharex=True, sharey=True)
# Uncomment if you want to manually set the y-limit
# ax[0, 0].set_ylim(0, 2)
for i in range(128):
    ax[i // 16, i % 16].plot(amps[i, :, 0].flatten())  # XX
    ax[i // 16, i % 16].plot(amps[i, :, 3].flatten())  # YY
plt.show()

# Phases
phases = np.rad2deg(np.angle(data))

_, ax = plt.subplots(8, 16, sharex=True, sharey=True)
ax[0, 0].set_ylim(-180, 180)
for i in range(128):
    ax[i // 16, i % 16].plot(phases[i, :, 0].flatten())  # XX
    ax[i // 16, i % 16].plot(phases[i, :, 3].flatten())  # YY
plt.show()