A disk-based system powered by Dell, named Acacia after Australia’s national floral emblem the Golden Wattle – Acacia pycnantha, providing 60PB of high-speed object storage for hosting research data online. This multi-tiered cluster separates different types of data to improve data availability.
Detailed information on how to upload to acacia is given in the pawsey help/support page!
Setting up ACACIA
Use the link: https://portal.pawsey.org.au/origin to go to the pawsey page. Login is using the Pawsey username and password. Under the ACACIA tab is the relevant page where the upload happens.
Once you are in the Acacia tab, Storage name shows all the relevant storage that is assigned under the specific user. For example, USERNAME, mwavcs etc
- Click on the storage name where you want to upload the data.
- Press on the View Keys button. This shows the existing keys for the user.
- As first time user, click on the Create New Key button.
- On doing so, it will show an Access ID and a Secret Key that is very very important to set up acacia on garrawarla etc.
NOTE: Make sure that you copy the Access ID and the Secret Key before closing the pop up window.
- Once you have the ID and Key, login to Garrawarla/magnus etc. Then type:
mc -h
mc is MinIO Client for cloud storage and filesystems. -h option shows all the functions of mc.
- The next step is to setup an alias for the storage:
mc alias set <ALIAS NAME> https://acacia.pawsey.org.au ACCESS ID Secret Key
The alias name that you set will be used from now on to upload and access data on acacia.
Uploading the data
- The next step is to create a bucket where all your data files will be uploaded. This can be thought of like a directory in our system. It can be done by:
set +o history
mc mb <ALIAS_NAME>/<BUCKET_NAME>
set -o history
This will create the bucket according to the name you have given. Once the bucket is created we can start uploading our data there.
- To upload the data:
mc cp <PATH_TO_FILE> <ALIAS_NAME>/<BUCKET_NAME>/<NAME_OF_FILE>
Other essentials
- To download an object from Acacia:
mc cp <ALIAS_NAME/BUCKET_NAME/NAME_OF_FILE> <PATH_TO_DOWNLOAD_DIRECTORY>
- To list the buckets in your account using MC, use the mc ls command:
mc ls <ALIAS_NAME>
- To list the objects in a bucket:
mc ls <ALIAS_NAME>/<BUCKET_NAME>
Essential links:
- To create buckets:
https://support.pawsey.org.au/documentation/pages/viewpage.action?pageId=104302738#
- To upload and download data:
https://support.pawsey.org.au/documentation/pages/viewpage.action?pageId=104302741
- To list buckets and data in buckets:
https://support.pawsey.org.au/documentation/display/US/List+buckets+and+objects#