Introduction

mwalib is a programming library developed in Rust (with an FFI / C / Python compatible API) for low-level preprocessing tools to seamlessly interact with MWA metadata, voltages and visibilities, including legacy and MWAX formats.

• Full Rust documentation can be found here: mwalib documentation on crates.io

NOTE: The pymwalib project is now deprecated as python bindings are now published from the mwalib repository and thus a separate repository/wrapper is no longer required.

mwalib supports reading the following data products:

• MWA Legacy Correlator observations

• MWA Legacy VCS observations *

• MWAX Correlator observations

• MWAX VCS observations

* = only MWA VCS legacy observations which have been recombined (data rearranged into per coarse channel files)

What mwalib Does

• Provides access to most of an observation’s metadata via the MetafitsContext struct/class.

• When data files are also passed to mwalib, the entire metadata for an observation is available.

• Provides functions to read visibility data as a block of floats.

• Provides functions to read voltage data as a block of bytes.

What mwalib Does Not Do

• mwalib does not apply any corrections to the data. That is the job of pre-processing tools, such as Birli.

• mwalib does not modify the metadata or data.

Python Installation

You can install mwalib via Pypi:

Example Python scripts can be found in the mwalib repository in the examples directory.

Python Alternate Install

Download and manually install the appropriate release tarball (see: mwalib Releases, below) and install the included wheel using pip:

Python Build From Source

You can build from source using maturin.

• Install Rust

• Clone the mwalib repository

• Within your Python environment, install maturin:

• Now build a local python package for mwalib (this will trigger a build of the Rust mwalib library too:

Rust Installation

You can add mwalib as a library to your existing Rust project:

• Add mwalib to your Cargo.toml file in your project.

• as seen above, you may want to include the feature "cfitsio-static" if you do not have the CFITSIO library installed.

Rust Build From Source

• Install Rust

• Clone the mwalib repository

• Install cfitsio

  • If using a package manager install the cfitsio-dev package.

  • If building from source:

    • For CFitsio <=3.49 ensure you build it with with --enable-reentrant option

    • For CFitsio >3.49 using cmake ensure you call cmake with: -DUSE_PTHREADS=ON. E.g. cmake -S . -B build -DCMAKE_INSTALL_PREFIX=/usr/local -DUSE_PTHREADS=ON -DUSE_SSE2=ON -DUSE_SSSE3=ON -DUSE_CURL=ON

  • See: FITSIO Homepage. Alternatively get the fitsio-sys crate to compile cfitsio for you by adding --features cfitsio-static to your cargo build command (below).

• Compile the source

  cargo build --release

• Statically-linking cfitsio

  • If any of the MWALIB_LINK_STATIC_CFITSIO, STATIC_CFITSIO or PKG_CONFIG_ALL_STATIC environment variables exist and are set to a non-zero value, rustc will statically link libcfitsio. The default is to dynamically link libcfitsio. This is an attempt to give developers the choice of having a static or dynamic link of the fits library to ease the build and deploy process.

C and CPP Installation

To install on most Linux x86/64 distributions, do the following:

1. Download release from mwalib github releases. (Where X.Y.Z is the current release version). See “mwalib Releases“ further down this page.

2. Untar the tarball

3. Install the libraries and include files (the below example uses system directories- but you can install them where ever you want, and thus may not need sudo permissions)

4. Register the library with ldconfig

or, alternatively, if you have installed the libraries in a directory other than the standard Linux system library directories, you can set your LD_LIBRARY_PATH to the directory where the mwalib dynamic library is installed:

5. You can then use mwalib by adding the include directive in your C/CPP source file(s):

6. And then compile with something like (C compilation shown below):

mwalib Releases

mwalib releases are available in the mwalib repository. Each release will provide the following files.

Within the Linux and Mac non-python tarballs you will find the following files:

• lib/libmwalib.a (Statically compiled library for Linux or MacOS)

• lib/libmwalib.so (Dynamic library for Linux)

• lib/libmwalib.dylib (Dynamic library for MacOS)

• include/mwalib.h (C Header file)

• CHANGELOG.md (Change log for this and previous releases)

• LICENSE (License for using mwalib in your projects)

• LICENSE-cfitsio (Since libcfitsio is statically compiled into our static and dynamic libraries, we also include it's license)

Usage

Usage via C

• In the examples directory, see build_c_examples.sh and *.c for examples of usage.

Usage via Python

The primary Python interface to mwalib is the mwalib python package which has a virtually identical API to the Rust library.

There are also a couple of simple examples Python scripts here that use the mwalib python interface- see: examples/*.py.

Usage in Rust

In the examples directory, see mwalib-print-context.rs and mwalib-print-corr-context.rs for examples. Also run cargo doc --open to see the rendered documentation (or view them here).

Usage approach

• Populate a Context struct (MetafitsContext, CorrelatorContext or VoltageContext)

  • CorrelatorContext - Use this when you have a metafits file and one or more gpubox files. This struct provides information about the correlator observation and provides access to a MetafitsContext for metadata, as well as methods for reading raw visibility data.

  • MetafitsContext - an efficient way to get access to most of the observation metadata. Only requires that you pass a valid MWA metafits file.

  • VoltageContext - Use this when you have a metafits file and one or more recombined (or MWAX) voltage files. This struct provides information about the VCS observation and provides access to a MetafitsContext for metadata, and in an upcoming release- methods for reading raw voltage data.

  • During creation of a CorrelatorContext or MetafitsContext, a number of checks are performed on the metadata and gpubox/voltage files to ensure consistency. Once created, the context is primed for reading data.

• Read raw data

  • The read_by_baseline function associated with CorrelatorContext takes in a timestep index (see: CorrelatorContext.timesteps vector) and a coarse channel index (see: CorrelatorContext.coarse_channels vector) and will return a vector of 32bit floats.

  • The data is organised as [baseline][fine_channel][polarisation][r][i].

  • The read_by_frequency function is similar to read_by_baseline but outputs data in [fine_channel][baseline][polarisation][r][i] order.