Page Comparison

...

Flow

Image Added

Chips grids a uvfits file, producing multiple .dat that can be used for crosspower and 1d power spectrum analysis.

The naming convention for .dat files usually contains the polarisation (xx or yy) and unique extension (ext) to identify that file.

.dat files from multiple uvfits files can be combined together.

arguments:

• ext - unique name for the grid
• eorband - either 0 (low, 139-170MHz), 1 (high, 167-198MHz)
• eorfield - either 0 (radec=0,-27 deg), 1 (radec=60,-30 deg)
• nchans - number of channels to analyse
• freq_idx_start - first channel number to analyse
• pol - polarization to analyse (xx or yy)
• period - integration time in seconds (e.g. 8.0)
• chanwidth - frequency resolution in Hz (e.g. 80000)
• lowfreq - first frequency in the data in Hz
• umax - max uvw (default 300)
• addsub - add (0) or subtract (1)
• nbins - number of power spectra bins (e.g. 80)
• bias_mode - one of 0/10/11/12 e.g. 0

environment variables:

• DATADIR
• INPUTDIR - where to look for input files
• OUTPUTDIR - where resulting files are written to
• OBSDIR
• OMP_NUM_THREADS - number of threads to use

binaries

• gridvisdiff - calculate the beam weights for an MWA observation
• prepare_diff - combine data over frequency
• combine_data - combine data over multiple gridded sets
• lssa_fg_simple - compute the LS spectral power (no kriging) using diff, tot and weights uvf binary files

Creating grids

gridvisdiff $uvfits $obsid $ext $eorband -f $eorfield
# for pol in xx yy; do
prepare_diff $ext $nchans $freq_idx_start $pol $ext $eorband -p $period -c $chanwidth -n $lowfreq -u umax
# this produces {vis_tot,vis_diff,noise_tot,noise_diff,weights}_${pol}_${ext}.dat in $OUTPUTDIR

Combining grids

combine grids with extensions ext1, ext2 together into a grid with extension $group_ext

# for pol in xx yy; do
# create a file containing all the exts to combine together prefixed with "${pol}."
export combinelist=combine_${pol}.${group_ext}.txt 
for ext in \
    "ext1" \
    "ext2" \
; do
    echo ${pol}.${ext} | tee -a ${combinelist}
done
# ensure {vis_tot,vis_diff,noise_tot,noise_diff,weights}_${pol}.${ext}.dat all exist in $OUTPUTDIR

combine_data ${combinelist} $nchans "${pol}.${group_ext}" $addsub
# this produces {crosspower,residpower,residpowerimag,totpower,flagpower,fg_num,outputweights}_${pol}_${bias_mode}.${ext}.dat in $OUTPUTDIR

Compute Power Spectra

# for pol in xx yy; do
lssa_fg_simple $ext $nchans $nbins $pol $maxu $ext $bias_mode $eorband

Plotting power spectra

see: https://github.com/JLBLine/plot_CHIPS

Garrawarla (new)

# load the chips module
module use /astro/mwaeor/software/modulefiles
module load chips/cmt

# optional optimization: do everything in nvmetmp, need to request enough --tmp from slurm
mkdir /nvmetmp/deleteme
cd /nvmetmp/deleteme

export DATADIR="$PWD"
export INPUTDIR="$PWD/"
export OUTPUTDIR="$PWD/"
export OBSDIR="$PWD/"

Garrawarla (old)

CHIPS is available on Garrawarla

/astro/mwaeor/MWA/chips

...

which will generate the following scripts, located in a new directory called logs_test_ozstar_chips :

$ ls logs_test_ozstar_chips
run_clean_test_8dec_test_ozstar_chips.sh ##Cleans up intermediate data products
run_grid_1093641624_test_8dec_test_ozstar_chips.sh ##Grids the first observation
run_grid_1093641864_test_8dec_test_ozstar_chips.sh ##Grids the second observation 
run_grid_1093642232_test_8dec_test_ozstar_chips.sh ##Grids the third observation 
run_lssa_test_8dec_test_ozstar_chips_xx.sh ##Makes the XX power spectra
run_lssa_test_8dec_test_ozstar_chips_yy.sh ##Makes the YY power spectra 

if there are any errors in the input arguments, you should get some form of error message telling you what to do next. If this command runs find. To actually launch your jobs, remove the --no_run command. I usually put this all in a bash script, so you'd do something like:

$ more run_CHIPS.sh
source /fred/oz048/jline/software/chips/module_load_chips.sh

run_CHIPS.py \
     --cluster=ozstar \
     --obs_list=/fred/oz048/jline/test_CHIPS_scripts/obs_list.txt \
     --uvfits_dir=test_8dec \
     --output_tag=test_ozstar_chips \
     --output_dir=/fred/oz048/jline/test_CHIPS_scripts/CHIPS_outputs/ \
     --band=high --obs_range=0,3

$ source run_CHIPS.sh
Command run: sbatch --parsable run_grid_1093641624_test_8dec_test_ozstar_chips.sh
Job ID: 27145363
Command run: sbatch --parsable --dependency=afterok:27145363 run_grid_1093641864_test_8dec_test_ozstar_chips.sh
Job ID: 27145364
Command run: sbatch --parsable --dependency=afterok:27145363:27145364 run_grid_1093642232_test_8dec_test_ozstar_chips.sh
Job ID: 27145365
Command run: sbatch --parsable --dependency=afterok:27145363:27145364:27145365 run_lssa_test_8dec_test_ozstar_chips_xx.sh
Job ID: 27145366
Command run: sbatch --parsable --dependency=afterok:27145363:27145364:27145365 run_lssa_test_8dec_test_ozstar_chips_yy.sh
Job ID: 27145367
Command run: sbatch --parsable --dependency=afterok:27145366:27145366 run_clean_test_8dec_test_ozstar_chips.sh

...

Debugging actual CHIPS runs: all the log outputs and error messages for this run will be output into logs_test_ozstar_chips , so if you jobs error, try checking the output logs and error messages located there. If everything runs fine, you can just delete that entire logs_test_ozstar_chips  directory (saves on the number of files quota).

Plotting the outputs

See (page to be written soon) for a full overview of the plotting commands available. But a quick example of how to make a 2D cross-power spectra using outputs of the OzStar example is below. Note the output tag for the files is a combination of the --output_tag and --uvfits_dir options that were fed to run_CHIPS.py :

source /fred/oz048/jline/software/chips/module_load_chips.sh

plotchips_all.py \
    --basedir=/fred/oz048/jline/test_CHIPS_scripts/CHIPS_outputs/ \
    --polarisation='yy' \
    --chips_tag=test_8dec_test_ozstar_chips \
    --min_power=1e3 --max_power=1e15

this will produce a plot called chips2D_yy_test_8dec_test_ozstar_chips_crosspower.png which looks like this:

Image Added

This is some random EoR1 data I found on OzStar so I have no idea of the quality or what peeling was done, but hey, it's a power spectrum!