LibreChat | Northern Inference docs

Use Northern Inference in LibreChat with exact Canadian-resident routes and per-request custody records for government and regulated teams.

Northern Inference is a sovereign Canadian AI API for government, defence, health and regulated teams, including organizations under the Controlled Goods Program. Every route carries PIPEDA-aligned Canadian data residency, an exact path to the latest frontier and open-weight models, and a per-request chain-of-custody record.

Before you start

Create an API key. In the portal, open Keys and click Create Key. Your key starts with ni_live_.
Add credit. New accounts start with a balance of $0, and every request is refused until you add credit. Open Billing and click Buy credits.
Install Docker and Git. Install Docker Desktop and start it, and install Git.

LibreChat is a self-hosted chat interface. You keep your own data store and any local MCP servers; Northern Inference is only the model layer.

Set up LibreChat with Northern Inference

The commands below work in Terminal on Mac and Linux and in PowerShell on Windows.

1. Download LibreChat.

git clone https://github.com/LibreChat-AI/LibreChat.git
cd LibreChat
cp .env.example .env
cp docker-compose.override.yml.example docker-compose.override.yml

2. Add your key. Open the .env file in the LibreChat folder and add this line at the end:

NI_API_KEY=ni_live_YOUR_KEY_HERE

3. Let LibreChat read its config file. Open docker-compose.override.yml and remove the # in front of the lines that mount librechat.yaml into the api service (the line ends in /app/librechat.yaml, together with the services:, api: and volumes: lines above it). Without this, LibreChat ignores the file.

4. Create the config file. In the LibreChat folder, create a file named librechat.yaml with:

version: 1.3.17

endpoints:
  custom:
    - name: "Northern Inference"
      apiKey: "${NI_API_KEY}"
      baseURL: "https://northerninference.ca/v1"
      models:
        default:
          - "mistral-large-2402-t3"
          - "llama3-70b-instruct-t3"
        fetch: false
      titleConvo: true
      titleModel: "mistral-large-2402-t3"
      modelDisplayLabel: "Northern Inference"

5. Start LibreChat.

docker compose up -d

Open http://localhost:3080, create your account, choose Northern Inference in the endpoint list and send a message. It appears in the portal under Usage within a few seconds.

After any later change to .env or librechat.yaml, restart with docker compose down and then docker compose up -d.

Choosing models

Add more names under default, exactly as they appear on the Models page. The two models above run in Canada (names ending in -t3) on providers that do not retain your prompts.

Residency comes from the tier tag. A -t3 model runs only in a Canadian provider region; if none can serve the request, it is refused. A -t4 model may run outside Canada.
Retention is separate from residency. Some providers keep data, for example for abuse review. Northern Inference pauses a request to such a model until you confirm, so data is never sent to a retaining provider without your say.

To give some people Canadian-only access, create a separate key for them in the portal with only Tier 3 allowed.

If a model name is not recognized

Northern Inference answers with HTTP 404 and Model 'X' is not a route this key can call. It never swaps in a different model. Use a name from the Models page, or type ni-models as a message to list the models your key can call.

Troubleshooting

What you see	What to do
Northern Inference is not in the endpoint list	LibreChat is not reading `librechat.yaml`. Check step 3, then restart. `docker compose logs api` shows config errors.
"Missing API Key"	The `NI_API_KEY` line is missing from `.env`. Add it and restart.
402 `Insufficient credits. Add credits in the portal under Billing (https://northerninference.ca/portal/?page=billing) to continue.`	Add credit in the portal under Billing.
404 `Model 'X' is not a route this key can call.`	Check the names under `default` against the Models page, or type `ni-models`.