Datasette LLM: Configure a Local OpenAI-Compatible Endpoint

Datasette LLM 0.1a11 can connect directly to OpenAI-compatible model servers without an additional provider plugin. For Datasette users running a local model, that means the endpoint can live in the application configuration. It does not mean every server or model can execute an agent's tools correctly.

The November 14 prerelease requires LLM 0.37.1 or newer. Treat this as an alpha integration to evaluate against a small database copy, not a reason to move a working production analysis workflow immediately. The steps below are based on the released documentation; I have not run this model integration.

Give the local endpoint its own name

In an existing Datasette installation with the endpoint-capable plugin, add a model reference to datasette.yaml. This example assumes a model server is already running on the same machine at port 1234:

plugins:
  datasette-llm:
    default_model:
      model: local-analysis
      model_name: YOUR_SERVER_MODEL_ID
      api_base: http://localhost:1234/v1

Replace YOUR_SERVER_MODEL_ID with the identifier exposed by your server. local-analysis is a separate, local name that Datasette uses for selection and filtering. Keeping these names distinct lets you change the hosted model without making every application setting depend on a vendor's naming scheme.

The tagged configuration guide explains that Chat Completions is the default protocol. Add responses: true only when you intend to use a server's Responses endpoint. In this example, the base URL includes /v1; use the path prefix your server requires. The plugin appends the selected protocol's route.

A server answering a health check is not enough. Confirm that it implements the protocol you selected, then check the actual model identifier. If Datasette runs in a container or on another machine, localhost refers to that environment, not automatically to the laptop hosting your model.

Keep credentials out of extra headers

For an authenticated endpoint, use the key field to name a stored credential. The release resolves that name through Datasette Secrets first and then LLM's saved keys. A missing explicit name raises NeedsKeyException; it does not silently use a different provider credential.

An endpoint with no key is treated as unauthenticated and does not receive a default OpenAI key. This is useful for a deliberately local, unauthenticated server, but it is not a way to disable authentication on a remote service.

Do not put a bearer token into the endpoint's headers dictionary. The release warns that extra headers can appear in Datasette's /-/config.json. A configuration that makes a request successfully can still be the wrong place to store its secret.

Test one known SQL result before a real question

Datasette Agent's CLI accepts a configuration file and a single prompt. With that plugin installed, use a disposable database containing three synthetic invoices: 40 and 60 marked paid, and 25 marked unpaid. Load your configuration with -c and ask the agent to calculate paid revenue using SQL.

The expected answer is 100. More importantly, inspect whether the agent actually queried the table, filtered the paid rows and returned the result. An answer of 100 without a tool call is not evidence that the integration works; the model could have inferred it from your prompt or guessed.

Repeat with an empty paid-invoice set and with a row containing a missing amount. Decide how your application should handle those cases before judging the answer. A natural-language explanation can sound convincing while using the wrong aggregation rule.

This is a proposed acceptance test, not a reported benchmark. Record the model, protocol, query and observed result together. Keep the test fixture unchanged when comparing another endpoint so you are comparing model behavior rather than different data.

Diagnose tool failures separately from model selection

The endpoint configuration defaults to advertising tool and streaming support, but the server must actually provide those capabilities. A model that produces fluent text may still fail the SQL-tool round trip. If streaming is unsupported, the configuration guide provides can_stream: false; that does not repair missing tool support.

Also check filtering before changing credentials. In this release, allowlists and blocklists apply when selecting a model, not just when displaying available models. An excluded model can raise ModelNotFoundError even though its server is running. The endpoints list registers choices without itself becoming an allowlist; the models setting restricts choices.

For the first test, keep ordinary permissions and confirmation prompts. Datasette Agent documents separate switches for root access and automatic approval, but neither is an endpoint troubleshooting fix. Moving inference onto your machine should not require broadening what the agent can change.

The useful improvement is a shorter path from an existing model server to a database workflow. The acceptance criterion remains the same: a correct query, an inspectable result and no unnecessary access to data or credentials.

I’m Paras. I work at the intersection of AI, engineering and distribution.

I help developer-tool teams turn configuration changes and practical workflows into technical guides people can use.

Book a 30-minute call