Australian Bureau of Statistics Connector

info@ismena.com

Connector Details

Type

Virtual machines, Single VM , BYOL

Runs on

Google Compute Engine

Last Update

24 October, 2024

Category

Overview

The Australian Bureau of Statistics (ABS) Connector is a custom integration tool for Google Cloud Platform (GCP) that provides seamless access to the ABS Data API. This connector enables users to retrieve economic, social, and Census data from ABS datasets, leveraging the Statistical Data and Metadata Exchange (SDMX) standard. It supports querying data and metadata in JSON, XML, and CSV formats, with flexible filtering and structural exploration capabilities.

Integration Overview

This document provides a detailed guide for each integration point, its purpose, configuration, and workflow support using the ABS Data Connector.
Supported Integration Action Points

getData: Retrieve data from a specific dataflow with filtering options.
getStructuresByAgencyId: List all structures of a specific type for an agency.
getLatestStructure: Retrieve the latest version of a specific structure.
getStructures: Retrieve a specific version of a structure.

Detailed Integration Documentation

2.1 Get Data Retrieval

Action	getData
Purpose	Retrieves data from a specified ABS dataflow, with options to filter by dimensions, time periods, and output formats (JSON, XML, CSV). This enables access to detailed statistical data for analysis or application integration.
Configuration	Ensure the connector is configured with the base URL via the `CONNECTOR_ENV_ABSDATAAPI_BASE_URL` environment variable.
Parameters	Required: dataflowIdentifier: Dataflow identifier in {agencyId},{dataflowId},{version} format (string, e.g., ABS,CPI,1.1.0). dataKey: Key to query data, using dimension codes or "all" (string, e.g., 1.115486.10.50.Q). Optional: startPeriod: Start period for time filtering (string, e.g., 2023-Q1). endPeriod: End period for time filtering (string, e.g., 2024-Q4). format: Output format (string, enum: [csvfilewithlabels, csvfile, jsondata, genericdata, structurespecificdata]). detail: Level of detail (string, enum: [full, dataonly, serieskeysonly, nodata], default: full). dimensionAtObservation: Data organization (string, e.g., TIME_PERIOD, AllDimensions).
Output	Successful: Returns data in the specified format: JSON: SDMX-JSON with header, datasets, and structure details. XML: SDMX-ML generic or structure-specific format. CSV: Dimension codes or codes with labels. Failure: Returns error details (e.g., 404: No records found).
Workflow Example	Configure the connector with the appropriate base URL. Execute the `getData` action with `dataflowIdentifier=ABS,CPI,1.1.0` and `dataKey=1.115486.10.50.Q`. Process the response to analyze Consumer Price Index data.

2.2 Get Structures By Agency ID Retrieval

Action	getStructuresByAgencyId
Purpose	Lists all structures of a specific type (e.g., dataflow, datastructure) maintained by an agency, such as ABS. This supports discovery of available datasets and their metadata structures.
Configuration	Ensure the connector is configured with the base URL via the `CONNECTOR_ENV_ABSDATAAPI_BASE_URL` environment variable.
Parameters	Required: structureType: Type of structure (string, enum: [dataflow, datastructure, codelist, conceptscheme, categoryscheme, contentconstraint, actualconstraint, agencyscheme, categorisation, hierarchicalcodelist]). agencyId: Agency identifier (string, e.g., ABS). Optional: detail: Level of detail (string, enum: [full, allstubs, referencestubs, referencepartial, allcompletestubs, referencecompletestubs], default: full).
Output	Successful: Returns a JSON or XML object with: header: Message metadata (id, prepared, sender). structures: Array of structure objects (e.g., dataflows, datastructures) with id, agencyID, version, name, and description. Failure: Returns error details (e.g., 404: No records found).
Workflow Example	Execute the `getStructuresByAgencyId` action with `structureType=dataflow` and `agencyId=ABS`. Review the response to identify available dataflows. Use the dataflow identifiers for subsequent data queries.

2.3 Get Latest Structure Retrieval

Action	getLatestStructure
Purpose	Retrieves the latest version of a specific structure (e.g., dataflow, codelist) for a given agency and structure ID. This enables access to metadata for dataset integration or analysis.
Configuration	Ensure the connector is configured with the base URL via the `CONNECTOR_ENV_ABSDATAAPI_BASE_URL` environment variable.
Parameters	Required: structureType: Type of structure (string, enum: [dataflow, datastructure, codelist, conceptscheme, categoryscheme, contentconstraint, actualconstraint, agencyscheme, categorisation, hierarchicalcodelist]). agencyId: Agency identifier (string, e.g., ABS). structureId: Structure identifier (string, e.g., CPI). Optional: references: Related artefacts to include (string, enum: [none, parents, parentsandsiblings, children, descendants, all, specific types], default: none). detail: Level of detail (string, enum: [full, allstubs, referencestubs, referencepartial, allcompletestubs, referencecompletestubs], default: full).
Output	Successful: Returns a JSON or XML object with: header: Message metadata (id, prepared, sender). structures: Structure details (id, agencyID, name, description, dimensions, or codes). Failure: Returns error details (e.g., 404: No records found).
Workflow Example	Execute the `getLatestStructure` action with `structureType=datastructure`, `agencyId=ABS`, and `structureId=CPI`. Process the response to understand the dataset’s structure. Use the metadata to construct data queries.

2.4 Get Structures Retrieval

Action	getStructures
Purpose	Retrieves a specific version of a structure (e.g., dataflow, codelist) for a given agency, structure ID, and version. This supports precise metadata access for integration or validation.
Configuration	Ensure the connector is configured with the base URL via the `CONNECTOR_ENV_ABSDATAAPI_BASE_URL` environment variable.
Parameters	Required: structureType: Type of structure (string, enum: [dataflow, datastructure, codelist, conceptscheme, categoryscheme, contentconstraint, actualconstraint, agencyscheme, categorisation, hierarchicalcodelist]). agencyId: Agency identifier (string, e.g., ABS). structureId: Structure identifier (string, e.g., CPI). structureVersion: Structure version (string, e.g., 1.1.0). Optional: references: Related artefacts to include (string, enum: [none, parents, parentsandsiblings, children, descendants, all, specific types], default: none). detail: Level of detail (string, enum: [full, allstubs, referencestubs, referencepartial, allcompletestubs, referencecompletestubs], default: full).
Output	Successful: Returns a JSON or XML object with: header: Message metadata (id, prepared, sender). structures: Structure details (id, agencyID, version, name, description, dimensions, or codes). Failure: Returns error details (e.g., 404: No records found).
Workflow Example	Execute the `getStructures` action with `structureType=datastructure`, `agencyId=ABS`, `structureId=CPI`, and `structureVersion=1.1.0`. Save the structure details for reference. Use the metadata to validate or query specific dataset versions.

Workflow Creation with the Connector

Example Workflow: Economic Data Analysis

Retrieve Available Dataflows	Use the `getStructuresByAgencyId` action with `structureType=dataflow` and `agencyId=ABS` to fetch available dataflows. Identify target dataflows (e.g., ABS,CPI,1.1.0).
Query Dataset Structure	Execute the `getLatestStructure` action with `structureType=datastructure`, `agencyId=ABS`, and `structureId=CPI` to fetch metadata. Process the response to understand dimensions and codes.
Fetch Data	Use the `getData` action with `dataflowIdentifier=ABS,CPI,1.1.0` and `dataKey=1.115486.10.50.Q` to retrieve Consumer Price Index data. Integrate the data into an analytics dashboard.
Validate Specific Version	Use the `getStructures` action with `structureType=datastructure`, `agencyId=ABS`, `structureId=CPI`, and `structureVersion=1.1.0` to confirm metadata. Ensure data consistency for historical analysis.

This workflow enables applications to provide users with accurate statistical data and metadata, enhancing economic analysis, policy development, and data-driven decision-making.

Pricing

bill-and-calculator-2023-11-27-05-25-06-utc 1

Request a Quote

Support

For Technical support please contact us on

custom-connectors-support@isolutions.sa

A tech solution company dedicated to providing innovation thus empowering businesses to thrive in the digital age.

Register to Wired for Banking