LibreChat

Get going in 30 seconds

Add this block to librechat.yaml:

endpoints:
  custom:
    - name: "Northern Inference"
      apiKey: "${NI_API_KEY}"
      baseURL: "https://northerninference.ca/v1"
      models:
        default:
          - "mistral-large-2402-t3"
          - "llama3-70b-instruct-t3"
        fetch: false
      titleConvo: true
      titleModel: "mistral-large-2402-t3"
      modelDisplayLabel: "NI"

Then:

export NI_API_KEY=ni_live_...
docker compose up -d

Open LibreChat at http://localhost:3080, pick Northern Inference from the endpoint dropdown, and send a message.

LibreChat is a self-hostable multi-provider chat UI. Point it at Northern Inference and your team gets a familiar chat interface with fail-closed Canadian routing on eligible Tier 3 routes and a per-request record of the route Northern Inference selected up to the provider boundary. You keep your own data store and any local MCP servers; NI is only the inference layer.

Picking a model: Canadian residency AND data retention

The example above uses two routes that are both Canadian-resident and zero-retention. They run on AWS Bedrock in ca-central-1, which does not retain or train on your prompts and completions, so custody reads NI-CA -> AWS Bedrock-CA.

Two things to know when you choose other routes:

1. Residency comes from the route suffix. A -ca route stays in Canada. A -global or -us route does not. Copy exact route IDs from /models.
2. Retention is separate from residency. Some Canadian-resident routes still let the provider retain data (a Vertex/Gemini route keeps data in-region; an Azure route may store prompts flagged for abuse review, with no maximum retention period published by the provider). Northern Inference is zero-retention by default and will pause a request on a retaining route until you confirm, so you never silently send data to a provider that keeps it. For strict sovereignty, prefer the Bedrock -ca routes.

librechat.yaml

version: 1.2.1

endpoints:
  custom:
    - name: "Northern Inference"
      apiKey: "${NI_API_KEY}"
      baseURL: "https://northerninference.ca/v1"
      models:
        default:
          - "mistral-large-2402-t3"
          - "llama3-70b-instruct-t3"
        fetch: false
      titleConvo: true
      titleModel: "mistral-large-2402-t3"
      summarize: false
      summaryModel: "mistral-large-2402-t3"
      forcePrompt: false
      modelDisplayLabel: "NI"

Then:

export NI_API_KEY=ni_live_...
docker compose up -d

Open LibreChat at http://localhost:3080, pick Northern Inference from the endpoint dropdown.

Route keys and per-key authorization

The model value is the route contract. Copy exact model IDs from /models. If you want some users limited to Canadian routes and others allowed to use global or provider routes, create separate NI API keys with different allowed route tiers.