PROGRAM: S-6
Title:
THE DATA AND METADATA MANAGEMENT IN AUSTRALIAN BUREAU OF METEOROLOGY’S SPACE WEATHER SERVICESAbstract:
With explosive growth in the amount of space weather research data, the development, management and use of
data and metadata is becoming increasingly important for data exploration, extraction, and inter-operations
between various data centres. As a member of the ICSU World Data System, the Australian Bureau of
Meteorology’s Space Weather Services (SWS) has developed metadata records based upon Space Physics
Archive Search and Extract (SPASE) data model. SPASE was chosen after comparison with Australian and New
Zealand metadata standard AS/NZS ISO 19115.1:2015, the Dublin Core Metadata Initiative (DCMI) and
Australian Government Locator Service (AGLS). The SWS metadata records have been available from the SWS
website, http://www.sws.bom.gov.au/World_Data_Centre/3/1, since 2016.
Due to historical and practical reasons, space weather related data are archived in different locations and
institutions across many countries. In order to integrate distributed space research data, the United States and
Japan have developed their metadata network. In USA,The SPASE effort is a Heliophysics community-based
project with the goals of facilitating data search and retrieval across the Space and Solar Physics data
environment with a common metadata language. It is a united space research metadata portal that collects and
archives space research related metadata from 20 virtual observatories and repositories over the world.
http://spase-group.org/registry/explorer/
ASWS (Australian Space Weather Services) has been registered as a unique Naming Authority with SPASE. 67
XML Metadata files of eight datasets, 20 instruments and 18 observatories of ASWS have been published with
SPASE registry explorer via GitHub, which is a web-based Git or version control repository and Internet hosting
service. https://github.com/hpde/ASWS. Figure 1 shows an ASWS Learmonth Spectrograph Metadata file.
The Bureau of Meteorology’s Space Weather Services (SWS) has accumulated and archived more than 2130GB
space weather data recorded at stations maintained by the Bureau of Meteorology’s Space Weather Network
(SWN). All real time data files are recorded locally and transferred to SWS head office in Sydney. SWSresearcher
use this real time data to issue space weather reports and make forecasts. The majority of this data
will be automatically archived into the SWS World Data Centre (WDC), and then synced to SWS FTP server for
public download. A copy of the files also replicated to the Bureau of Meteorology’s Data Centre located in
Melbourne. Most of the SWS space weather datasets are stored in a text, binary or image file formats instead of
in a relational database.
In addition to SWS storing a copy of its archived data into the Bureau’s Data Centre, the Bureau’s Data
Catalogue also requires SWS to provide space weather metadata following the metadata standard adopted by the
Bureau of Meteorology. The standard is AS/NZS ISO 19115.1:2015 - geographic information. The Bureau’s
Data Catalogue has developed an online metadata editor and template. Currently, nine SWS space weather
datasets have had metadata created online within the Bureau’s Data Catalogue website.
There are many space research related government agencies and universities in the world and they hold a wide
range of space weather related data. Developing and utilizing metadata files enables researchers to cross-search
between databases distributed over these institutions and overcomes the difficulty of collecting all original data
into a single global data centre. Rather than setting up a global WDS metadata portal to hold as many metadata
files as possible, we suggest sharing metadata among discipline related organisations by exchange and
publishing metadata files in every organisation’s website. The metadata standards adopted by each organization
or website may be different. However, it should not be a burden to give a link in the metadata files to the original
data archived in each different organisation.
Metadata exchanges can be conducted simultaneously with the development of a global WDS metadata portal.
The portal ought to contain as many as possible metadata files related to space weather and other disciplines
research data, including; metadata about researchers, instruments, ground and space based observatories and
services owned by all WDS members.
Acknowledgements:
The author would like to thank Mr Todd King of the Institute of Geophysics & Planetary Physics, University of
California, Los Angeles (UCLA) and Dr. Leonard Garcia of NASA, USA for their kind help in the development
and publication of the Australian Space Weather Services (ASWS) metadata files with SPASE.
The author would like to thank Ms Kate Roberts of the Information Modelling and Data Catalogue, Bureau of
Meteorology, Australia for her kind help in the development and publication of the Australi
References:
The Space Physics Archive Search and Extract (SPASE) website,
http://www.spase-group.org/
The Australian Bureau of Meteorology Data Catalogue website,
http://www.bom.gov.au/metadata/catalogue/search.shtml?query=%22space+weather+services%22