Custom Connector
Royal Commission for Riyadh City (RCRC) Open Data Portal
KSA
The RCRC Open Data Portal connector v2.1 is a RESTful interface for accessing datasets related to the Riyadh Region, encompassing environmental, administrative, urban planning, and demographic data. Built on the Opendatasoft platform, it supports querying, filtering, and exporting datasets using ODSQL, delivering data in JSON and multiple export formats. Most endpoints do not require authentication, but an optional API key may be needed for restricted endpoints.
Overview
Integration Overview
This document details each integration point of the RCRC Open Data Portal API v2.1, including purpose, configuration, and supported workflows for a GCP custom connector — querying datasets, retrieving records, exporting data, and exploring facets. Authentication via an API key is optional but may be required for specific endpoints.
- getDatasets. Retrieves a list of datasets in the Riyadh Region catalog.
- listExportFormats. Enumerates available export formats for the catalog or datasets.
- exportDatasets. Exports the catalog in a specified format (CSV, JSON, DCAT).
- exportCatalogCSV. Exports the catalog in CSV format with customizable parameters.
- exportCatalogDCAT. Exports the catalog in RDF/XML (DCAT) format with regional variants.
- getDatasetsFacets. Retrieves facet values for datasets to enable guided navigation.
- getRecords. Queries records within a specific dataset.
- listDatasetExportFormats. Lists export formats for a specific dataset.
- exportRecords. Exports dataset records in a specified format (CSV, GeoJSON, Parquet).
- exportRecordsCSV. Exports dataset records in CSV format with customizable delimiters.
- exportRecordsParquet. Exports dataset records in Parquet format with compression options.
- exportRecordsGPX. Exports dataset records in GPX format for geospatial data.
- getDataset. Retrieves metadata and endpoints for a specific dataset.
- getRecordsFacets. Enumerates facet values for dataset records.
- getDatasetAttachments. Lists attachments associated with a dataset.
- getRecord. Retrieves a single record by its identifier.
Documentation
Detailed Integration Documentation
Datasets Retrieval
| Action | getDatasets |
|---|---|
| Purpose | Fetches a list of datasets in the Riyadh Region catalog, including metadata such as dataset identifiers, fields, and features. Serves as the primary entry point for dataset discovery. |
| Parameters | Optional: apiKey, select, where (ODSQL filter), order_by, limit (default 10, max 100), offset, refine, exclude, lang, timezone, group_by, include_links, include_app_metas. |
| Configuration | Configure the GCP connector with the API endpoint. Include apiKey only for restricted endpoints. Handle JSON responses and ODSQL query parameters. |
| Output | Successful: JSON with total_count, links (if enabled), and a results array of dataset objects (dataset_id, fields, metas). Failure: error_code / message (e.g. ODSQLError). |
| Workflow example | Execute getDatasets with refine=theme:Environment to filter environmental datasets, then parse the response in a GCP Dataflow pipeline. |
Catalog Export Formats
| Action | listExportFormats |
|---|---|
| Purpose | Enumerates supported export formats for the catalog (CSV, JSON, DCAT), enabling selection of appropriate formats for data extraction. |
| Parameters | Optional: apiKey (for restricted formats). |
| Output | Successful: JSON with links to export endpoints. Failure: error details (401 Unauthorized). |
| Workflow example | Execute listExportFormats, parse supported formats, and store options in Cloud Storage for subsequent exports. |
Catalog Export
| Action | exportDatasets |
|---|---|
| Purpose | Exports the entire catalog in a specified format, supporting bulk data retrieval for analysis or archival within GCP workflows. |
| Parameters | Required: format (csv, json, dcat). Optional: apiKey, select, where, order_by, limit, offset, refine, exclude, lang, timezone, group_by. |
| Output | Successful: file in the specified format. Failure: error details (400 Bad Request). |
| Workflow example | Execute exportDatasets with format=csv, store the file in Cloud Storage, and process in BigQuery. |
Catalog CSV Export
| Action | exportCatalogCSV |
|---|---|
| Purpose | Exports the catalog in CSV format with customizable delimiter and encoding options, optimized for integration with GCP tools like BigQuery. |
| Parameters | Optional: apiKey, delimiter (default ;), list_separator, quote_all, with_bom (default true). |
| Output | Successful: CSV file with catalog data. Failure: error details (429 Too Many Requests). |
| Workflow example | Execute exportCatalogCSV with delimiter=; and with_bom=true, upload to BigQuery for analysis. |
Catalog DCAT Export
| Action | exportCatalogDCAT |
|---|---|
| Purpose | Exports the catalog in RDF/XML (DCAT) format with regional variants, suitable for semantic web applications within GCP. |
| Parameters | Required: dcat_ap_format (e.g. _ap_ch, _ap_de). Optional: apiKey, include_exports, use_labels_in_exports (default true). |
| Output | Successful: RDF/XML file. Failure: error details (403 Forbidden). |
| Workflow example | Verify subscription for DCAT, execute exportCatalogDCAT with dcat_ap_format=_ap_ch, process the RDF/XML in a semantic pipeline. |
Datasets Facets Retrieval
| Action | getDatasetsFacets |
|---|---|
| Purpose | Retrieves facet values for datasets to enable guided navigation across the catalog. |
| Parameters | Optional: apiKey, facet, refine, exclude, where, timezone. |
| Output | Successful: JSON with links and facets array (e.g. publisher with counts). Failure: error details (400). |
| Workflow example | Execute getDatasetsFacets with facet=theme, then build a navigation interface for environmental datasets. |
Records Retrieval
| Action | getRecords |
|---|---|
| Purpose | Queries records within a specific dataset, enabling detailed exploration of Riyadh-specific data (environmental or demographic records). |
| Parameters | Required: dataset_id. Optional: apiKey, select, where (ODSQL), group_by, order_by, limit (default 10, max 100), offset, refine, exclude, lang, timezone, include_links, include_app_metas. |
| Output | Successful: JSON with total_count, links, and a results array of record objects. Failure: error details (400). |
| Workflow example | Execute getRecords with dataset_id=geonames-all-cities, refine=cou_name_en:Saudi Arabia, then process in a Dataflow pipeline and store in BigQuery. |
Dataset Export
| Action | exportRecords |
|---|---|
| Purpose | Exports dataset records in a specified format, supporting large-scale data retrieval for GCP-based analysis or visualization. |
| Parameters | Required: dataset_id, format (csv, geojson). Optional: apiKey, select, where, order_by, group_by, limit, refine, exclude, lang, timezone, use_labels, compressed, epsg (default 4326). |
| Output | Successful: file in the specified format. Failure: error details (429). |
| Workflow example | Execute exportRecords with format=geojson, store in Cloud Storage, and visualize in Data Studio. |
Dataset CSV Export
| Action | exportRecordsCSV |
|---|---|
| Purpose | Exports dataset records in CSV format with customizable options, optimized for BigQuery or Dataflow. |
| Parameters | Required: dataset_id. Optional: apiKey, delimiter (default ;), list_separator, quote_all, with_bom (default true). |
| Output | Successful: CSV file with dataset records. Failure: error details (400). |
| Workflow example | Execute exportRecordsCSV with delimiter=;, upload to BigQuery, and generate reports. |
Dataset Parquet Export
| Action | exportRecordsParquet |
|---|---|
| Purpose | Exports dataset records in Parquet format, optimized for big data processing in GCP environments like Dataproc. |
| Parameters | Required: dataset_id. Optional: apiKey, parquet_compression (default snappy). |
| Output | Successful: Parquet file. Failure: error details (400). |
| Workflow example | Execute exportRecordsParquet with parquet_compression=zstd, process in Dataproc, store results in BigQuery. |
Dataset GPX Export
| Action | exportRecordsGPX |
|---|---|
| Purpose | Exports dataset records in GPX format, suitable for geospatial applications in GCP such as mapping Riyadh’s environmental features. |
| Parameters | Required: dataset_id. Optional: apiKey, name_field, description_field_list, use_extension (default true). |
| Output | Successful: GPX file. Failure: error details (400). |
| Workflow example | Execute exportRecordsGPX with name_field=name, store in Cloud Storage, and import into a GIS tool. |
Dataset Information Retrieval
| Action | getDataset |
|---|---|
| Purpose | Retrieves metadata and endpoints for a specific dataset, enabling structural understanding for GCP workflows. |
| Parameters | Required: dataset_id. Optional: apiKey, select, lang, timezone, include_links, include_app_metas. |
| Output | Successful: JSON with dataset metadata (dataset_id, fields, metas, attachments). Failure: error details (400). |
| Workflow example | Execute getDataset with dataset_id=geonames-all-cities, parse metadata to identify fields, then plan record queries. |
Records Facets Retrieval
| Action | getRecordsFacets |
|---|---|
| Purpose | Enumerates facet values for dataset records (country, timezone), enabling guided navigation in GCP applications. |
| Parameters | Required: dataset_id. Optional: apiKey, facet, where, refine, exclude, lang, timezone. |
| Output | Successful: JSON with links and facets array. Failure: error details (400). |
| Workflow example | Execute getRecordsFacets with facet=cou_name_en to filter records for Saudi Arabia and build a navigation interface. |
Dataset Attachments Retrieval
| Action | getDatasetAttachments |
|---|---|
| Purpose | Lists attachments (e.g. ZIP files) associated with a dataset, providing supplementary data for GCP workflows. |
| Parameters | Required: dataset_id. Optional: apiKey. |
| Output | Successful: JSON with links and attachments array (href, mime-type). Failure: error details (400). |
| Workflow example | Execute getDatasetAttachments, download attachments to Cloud Storage, and process supplementary data. |
Single Record Retrieval
| Action | getRecord |
|---|---|
| Purpose | Retrieves a single record by its identifier, enabling detailed inspection of specific data points in GCP applications. |
| Parameters | Required: dataset_id, record_id. Optional: apiKey, select, lang, timezone. |
| Output | Successful: JSON with record data (_id, name, coordinates). Failure: error details (404 Not Found). |
| Workflow example | Execute getRecord with a specific record_id, parse details for a Riyadh location, and visualize in Data Studio. |
Support
For technical support, contact custom-connectors-support@isolutions.sa.