REST, OData and Public Data

When a system isn’t in the catalog, connecting it usually means reading its API docs, working out the authentication and paging, and testing calls until the data looks right. The Connector Agent does that work with you. Give it the API’s docs or a feed URL, and it builds a connector you can preview and approve before anything loads.

If you’ve used Postman, think of this as the agentic version: you point it at an API, and it works out the auth, the paging and the shape of the response with you.

Before you start

• You need to be an organization admin. Members can browse the Public data tab, but only admins can build a connector or start one from a template.

What you can connect

REST APIs. Any JSON REST API on a public https address. Authentication can be none, a bearer token, an API key (in a header, the query string or a cookie), a username and password, or OAuth2 client credentials (a token URL and client ID). Paging is detected from the docs or the response: next links, cursors, offset and limit, or page numbers.

OData feeds. OData v4, and v2 or v3. Paste a feed URL, an entity set or a service root; a service root lists its entity sets as candidate tables. Open-data portals that offer an OData link work too, such as Socrata portals (“Access this Dataset via OData”).

Not supported: GraphQL, SOAP, XML-only REST APIs, private or internal addresses, and APIs that require each person to sign in with their own account.

Build a connector from an API

Open the Connector Agent on the Connectors page and ask Build a connector for a REST API, or name the API you want, for example “Connect to the GitHub API for our org’s repos”.

1. Give it the docs. Paste the API’s OpenAPI or Swagger spec URL if there is one (JSON or YAML); the agent takes paths, paging and keys straight from it. A docs web page works too: the agent follows its spec link, or reads the page.
2. Approve the draft. The agent creates a REST API connector draft and opens it on the page: Settings at the top, then a Preview with one tab per table.
3. Add a key if it needs one. The agent opens a secure prompt, such as Enter the API Key. The value is saved encrypted; the agent only learns that it is set.
4. Review each change. Every change the agent makes to the draft shows up as a Suggested change card with Accept and Reject. Nothing in the draft changes until it’s accepted.
5. Check the preview. After each change the agent previews one page of real data in the Preview panel, and checks that the columns look right. You can also click Run preview yourself.
6. Choose the slice. After the first good preview the agent tells you how many rows the load will be. Over about 100,000 rows, it suggests two or three filters with the row count for each, and previews each filter before recommending it.
7. Save and load. When you’re happy, approve the agent’s save card, or click Save and Load on the page. The connector is published and starts loading into your Library.

The agent also offers incremental loading when records have a field that only grows, such as updated_at. Later loads then fetch only new and changed records.

One connector, several tables

A connector can hold up to 50 tables, and one table can depend on another. For example, posts/{resources.posts.id}/comments fetches the comments for each post.

OData notes

• The agent detects OData from the URL or the response, and uses the version the server actually speaks.
• Filters and column choices go into the feed’s own $filter and $select, so the server sends less.
• On a Socrata portal, prefer the v4 link: the v2 endpoint can take most of a minute per page.
• If a Socrata portal answers “Too many requests” (HTTP 429), the agent asks for an app token. Get a free one from the portal and enter it in the secure prompt.

Start from a public data template

The Public data tab on the Connectors page lists tested templates for public datasets. There are 40, from 31 publishers, across finance, economy, health, government, energy, weather and more. For example:

• BLS Consumer Price Index and Labor Force
• U.S. Census ACS 5-Year County Profile and County Business Patterns
• U.S. Treasury Daily Par Yield Curve and Debt to the Penny
• EIA Energy Spot Prices and Electricity Retail Sales by State
• New York Fed Reference Rates
• SEC EDGAR XBRL Financial Data
• World Bank Development Indicators
• USGS Earthquake Catalog and NOAA Daily Weather Observations
• openFDA Drug Recalls and NHTSA Vehicle Recalls

1. Search by topic, place or publisher, or filter by category.
2. Open a template to see its tables, licence, the credit line to keep, and links to the dataset page, API docs and terms.
3. Check its key badge:
   • No key: works as is.
   • Key optional: works without one. A free key from the publisher gives higher limits for regular loads.
   • Your own key: sign up with the publisher for a free key. Querri never supplies one.
4. Click Use this data. A REST API connector draft opens, already configured. You own it.
5. Ask the agent to change what you need, such as the series, the years or a filter, then save and load as above.

Every template is tested against the publisher’s service, and shows when it was last checked. Data comes from the publisher itself.

If nothing matches, ask the Connector Agent. When you ask for public or government data, it checks the templates first, and builds from the publisher’s API if none fits.

Licences

Each template runs under the publisher’s licence. Keep its credit line with anything you publish, and check the terms before commercial use when the template says to.

Security

• The agent never sees credentials. Keys, tokens, passwords and client secrets go into the secure prompt, never into the chat or the draft. A change that tries to put one in the settings is refused.
• Public hosts only. A connector calls only the public https host it was built for, plus its OAuth token host. Private, internal and localhost addresses are refused.
• Docs are reference only. The agent treats API docs and responses as information, not instructions.
• It fetches what you approved. A connector sends only the requests in its settings, and loads what comes back into your Library.
• You approve the load. Previews read one page and change nothing. Loading data waits for your approval.

Next Steps

• Connector Agent: everything else the agent does
• Data Connectors Overview: the connectors already in the catalog
• Managing Connections: change, refresh and remove connectors