Introduction

mwalib is a programming library developed in Rust (with an FFI / C / Python compatible API) for low-level preprocessing tools to seamlessly interact with MWA metadata, voltages and visibilities, including legacy and MWAX formats.

• Full Rust documentation can be found here: mwalib documentation on crates.io

NOTE: The pymwalib project is now deprecated as python bindings are now published from the mwalib repository and thus a separate repository/wrapper is no longer required.

mwalib supports reading the following data products:

• MWA Legacy Correlator observations

• MWA Legacy VCS observations *

• MWAX Correlator observations

• MWAX VCS observations

* = only MWA VCS legacy observations which have been recombined (data rearranged into per coarse channel files)

What mwalib Does

• Provides access to most of an observation’s metadata via the MetafitsContext struct/class.

• When data files are also passed to mwalib, the entire metadata for an observation is available.

• Provides functions to read visibility data as a block of floats.

• Provides functions to read voltage data as a block of bytes.

What mwalib Does Not Do

• mwalib does not apply any corrections to the data. That is the job of pre-processing tools, such as Birli.

• mwalib does not modify the metadata or data.

Python Installation

You can install mwalib via Pypi:

$ pip install mwalib

Example Python scripts can be found in the mwalib repository in the examples directory.

Python Alternate Install

Download and manually install the appropriate release tarball (see: mwalib Releases, below) and install the included wheel using pip:

$ pip install NAME_OF_WHEEL.whl

Python Build From Source

You can build from source using maturin.

• Install Rust

• Clone the mwalib repository

• Within your Python environment, install maturin:

$ pip install maturin[patchelf]

• Now build a local python package for mwalib (this will trigger a build of the Rust mwalib library too:

$ maturin develop --features=python,cfitsio-static

Rust Installation

You can add mwalib as a library to your existing Rust project:

• Add mwalib to your Cargo.toml file in your project.

...
[dependencies]
mwalib = { version = "1.5.0", features = ["cfitsio-static"]}
...

• as seen above, you may want to include the feature "cfitsio-static" if you do not have the CFITSIO library installed.

Rust Build From Source

• Install Rust

• Clone the mwalib repository

• Install cfitsio

  • If using a package manager install the cfitsio-dev package. If building from source ensure you build it with with --enable-reentrant option. See: FITSIO Homepage Alternatively get the fitsio-sys crate to compile cfitsio for you by adding --features cfitsio-static to your cargo build command (below).

• Compile the source

  cargo build --release

• Statically-linking cfitsio

  • If any of the MWALIB_LINK_STATIC_CFITSIO, STATIC_CFITSIO or PKG_CONFIG_ALL_STATIC environment variables exist and are set to a non-zero value, rustc will statically link libcfitsio. The default is to dynamically link libcfitsio. This is an attempt to give developers the choice of having a static or dynamic link of the fits library to ease the build and deploy process.

C and CPP Installation

To install on most Linux x86/64 distributions, do the following:

1. Download release from mwalib github releases. (Where X.Y.Z is the current release version). See “mwalib Releases“ further down this page.

$ wget "https://github.com/MWATelescope/mwalib/releases/download/vX.Y.Z/mwalib-vX.Y.Z-linux_x86_64.tar.gz" -O mwalib.tar.gz

2. Untar the tarball

$ mkdir mwalib
$ tar xvf mwalib.tar.gz -C mwalib

3. Install the libraries and include files (the below example uses system directories- but you can install them where ever you want, and thus may not need sudo permissions)

$ sudo cp mwalib/libmwalib.* /usr/local/lib
$ sudo cp mwalib/mwalib.h /usr/local/include

4. Register the library with ldconfig

$ sudo ldconfig

or, alternatively, if you have installed the libraries in a directory other than the standard Linux system library directories, you can set your LD_LIBRARY_PATH to the directory where the mwalib dynamic library is installed:

$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/mwalib

5. You can then use mwalib by adding the include directive in your C/CPP source file(s):

#include "mwalib.h"

6. And then compile with something like (C compilation shown below):

gcc sample_program.c -o sample_program -I path/to/include -lm -lpthread -ldl -L/path/to/mwalib.so -lmwalib

mwalib Releases

mwalib releases are available in the mwalib repository. Each release will provide the following files.

Within the Linux and Mac non-python tarballs you will find the following files:

• lib/libmwalib.a (Statically compiled library for Linux or MacOS)

• lib/libmwalib.so (Dynamic library for Linux)

• lib/libmwalib.dylib (Dynamic library for MacOS)

• include/mwalib.h (C Header file)

• CHANGELOG.md (Change log for this and previous releases)

• LICENSE (License for using mwalib in your projects)

• LICENSE-cfitsio (Since libcfitsio is statically compiled into our static and dynamic libraries, we also include it's license)

Usage

Usage via C

• In the examples directory, see build_c_examples.sh and *.c for examples of usage.

Usage via Python

The primary Python interface to mwalib is the mwalib python package which has a virtually identical API to the Rust library.

There are also a couple of simple examples Python scripts here that use the mwalib python interface- see: examples/*.py.

Usage in Rust

In the examples directory, see mwalib-print-context.rs and mwalib-print-corr-context.rs for examples. Also run cargo doc --open to see the rendered documentation (or view them here).

Usage approach

• Populate a Context struct (MetafitsContext, CorrelatorContext or VoltageContext)

  • CorrelatorContext - Use this when you have a metafits file and one or more gpubox files. This struct provides information about the correlator observation and provides access to a MetafitsContext for metadata, as well as methods for reading raw visibility data.

  • MetafitsContext - an efficient way to get access to most of the observation metadata. Only requires that you pass a valid MWA metafits file.

  • VoltageContext - Use this when you have a metafits file and one or more recombined (or MWAX) voltage files. This struct provides information about the VCS observation and provides access to a MetafitsContext for metadata, and in an upcoming release- methods for reading raw voltage data.

  • During creation of a CorrelatorContext or MetafitsContext, a number of checks are performed on the metadata and gpubox/voltage files to ensure consistency. Once created, the context is primed for reading data.

• Read raw data

  • The read_by_baseline function associated with CorrelatorContext takes in a timestep index (see: CorrelatorContext.timesteps vector) and a coarse channel index (see: CorrelatorContext.coarse_channels vector) and will return a vector of 32bit floats.

  • The data is organised as [baseline][fine_channel][polarisation][r][i].

  • The read_by_frequency function is similar to read_by_baseline but outputs data in [fine_channel][baseline][polarisation][r][i] order.