Custom Connector
KAPSARC Data Portal
KSA
The KAPSARC Data Portal API enables seamless integration with the KAPSARC Data Portal, providing access to energy-economics and related datasets for analytics, research, and visualization. It supports dataset enumeration, record queries, data exports, and facet-based navigation — organized around REST (HTTP GET), returning JSON, and using ODSQL for querying.
Overview
Integration Overview
This document outlines each integration point, its purpose, configuration, and supported workflows using the KAPSARC Data Portal API. Free registration is required for full access, including advanced filters, downloadable data, and API-key authentication.
- getDatasets. Retrieves a list of all datasets in the catalog.
- listExportFormats / exportDatasets / exportCatalogCSV / exportCatalogDCAT. List and run catalog exports in a chosen format, CSV, or DCAT-AP.
- getDatasetsFacets. Retrieves global catalog-level facets.
- getDataset / getRecords / getRecord. Retrieve dataset metadata, its records, or a single record.
- listDatasetExportFormats / exportRecords / CSV / Parquet / GPX. List and run per-dataset exports in a chosen format.
- getRecordsFacets. Retrieves available facets for a dataset.
- getDatasetAttachments. Retrieves files or attachments related to a dataset.
Documentation
Detailed Integration Documentation
Catalog Datasets Retrieval
| Action | getDatasets |
|---|---|
| Purpose | Retrieves a comprehensive list of all datasets in the catalog — the entry point for exploring data (e.g. saudi-arabia-oil-database). |
| Configuration | Requires registration and API-key authentication. Ensure environment variables are configured for connectivity. |
| Parameters | Optional: select; where (ODSQL, e.g. publisher=“KAPSARC”); order_by; limit (default 10, max 100); offset; refine; exclude; lang; timezone (e.g. Asia/Riyadh); group_by; include_links; include_app_metas. |
| Output | Successful: JSON with total_count, _links, and a results array of dataset objects. Failure: unauthorized access or invalid ODSQL query. |
| Workflow example | Configure the API key, then execute getDatasets to identify datasets (e.g. saudi-arabia-oil-database). |
Catalog Exports (list / format / CSV / DCAT)
| Action | listExportFormats / exportDatasets / exportCatalogCSV / exportCatalogDCAT |
|---|---|
| Purpose | Lists catalog export formats and exports the catalog in a chosen format, CSV, or RDF/XML (DCAT-AP). |
| Configuration | Requires registration and API-key authentication. Ensure environment variables are configured for connectivity. |
| Parameters | exportDatasets required: format (csv, json, xlsx, rdf). CSV optional: delimiter; list_separator; quote_all; with_bom. DCAT required: dcat_ap_format; optional include_exports, use_labels_in_exports. |
| Output | Successful: a JSON list of export links or a file. Failure: unsupported format / authentication error. |
| Workflow example | Call listExportFormats, then exportCatalogCSV with delimiter=, for analysis. |
Catalog Facets Retrieval
| Action | getDatasetsFacets |
|---|---|
| Purpose | Retrieves catalog-level facet values (e.g. theme, publisher) for navigation and filtering. |
| Configuration | Requires registration and API-key authentication. Ensure environment variables are configured for connectivity. |
| Parameters | Optional: facet (e.g. theme); refine; exclude; where; timezone. |
| Output | Successful: JSON with links and facets arrays. Failure: invalid facet. |
| Workflow example | Execute getDatasetsFacets with facet=theme to refine dataset queries. |
Dataset Metadata & Records
| Action | getDataset / getRecords / getRecord |
|---|---|
| Purpose | Retrieves metadata for a specific dataset; queries its records; and reads a single record by ID. |
| Configuration | Requires registration and API-key authentication. Ensure environment variables are configured for connectivity. |
| Parameters | Required: dataset_id (getRecord also needs record_id). Optional: select; where (ODSQL); group_by; order_by; limit (max 100); offset; refine; exclude; lang; timezone; include_links; include_app_metas. |
| Output | Successful: dataset metadata, a results array of records, or a single record. Failure: invalid dataset_id/record_id. |
| Workflow example | Use getDataset to select saudi-arabia-oil-database, getRecords with where=year:2020, then getRecord with a record_id. |
Dataset Exports (formats / CSV / Parquet / GPX)
| Action | listDatasetExportFormats / exportRecords / exportRecordsCSV / exportRecordsParquet / exportRecordsGPX |
|---|---|
| Purpose | Lists a dataset’s export formats and exports its records in a chosen format, CSV, Parquet (snappy/zstd), or GPX. |
| Configuration | Requires registration and API-key authentication. Ensure environment variables are configured for connectivity. |
| Parameters | Required: dataset_id (and format for exportRecords). Optional: select, where, order_by, group_by, limit, refine, exclude, lang, timezone, use_labels, epsg; CSV delimiter options; parquet_compression; GPX name_field/description_field_list/use_extension. |
| Output | Successful: a file in the requested format. Failure: invalid format. |
| Workflow example | Execute exportRecordsParquet with dataset_id=saudi-arabia-oil-database, then ingest into BigQuery. |
Record Facets & Attachments
| Action | getRecordsFacets / getDatasetAttachments |
|---|---|
| Purpose | Retrieves facet values for a dataset’s records (guided navigation) and lists a dataset’s files/attachments. |
| Configuration | Requires registration and API-key authentication. Ensure environment variables are configured for connectivity. |
| Parameters | Required: dataset_id. Optional (facets): facet; where; refine; exclude; lang; timezone. |
| Output | Successful: facet enumerations, or an attachments array (href, mime-type, title). Failure: invalid dataset_id/facet. |
| Workflow example | Use getRecordsFacets with facet=year, and getDatasetAttachments to download supplementary files. |
Example Workflow: Exploring & Exporting Oil Production Data
| Retrieve datasets | Use getDatasets to identify a target (e.g. saudi-arabia-oil-database). |
|---|---|
| Refine & query | Use getRecordsFacets with facet=year, then getRecords with where=year:2020. |
| Export | Use exportRecordsCSV / exportRecordsParquet / exportRecordsGPX (name_field=region). |
| Metadata & catalog | Fetch metadata via getDataset and attachments via getDatasetAttachments; export the catalog via exportCatalogCSV or exportCatalogDCAT. |
Use Case
The KAPSARC Data Portal API enables seamless integration with the KAPSARC Data Portal, providing access to energy-economics and related datasets for analytics, research, and visualization. Supporting dataset enumeration, record queries, data exports, and facet-based navigation, it advances energy-economics research with datasets like oil databases, consumer price indices, and natural-gas data — organized around REST (HTTP GET), returning JSON, and using ODSQL for querying.
Support
For technical support, contact custom-connectors-support@isolutions.sa.