LibreChat

Get going in 30 seconds

export NI_API_KEY=ni_live_...
docker compose up -d

Add this block to librechat.yaml first:

endpoints:
  custom:
    - name: "Northern Inference"
      apiKey: "${NI_API_KEY}"
      baseURL: "https://northerninference.ca/v1"
      models:
        default:
          - "vertex_ai/gemini-2.5-pro-ca"
          - "bedrock/global.anthropic.claude-haiku-4-5-20251001-v1:0-global"
        fetch: false
      titleConvo: true
      titleModel: "bedrock/global.anthropic.claude-haiku-4-5-20251001-v1:0-global"
      modelDisplayLabel: "NI"

Open LibreChat at http://localhost:3080, pick Northern Inference from the endpoint dropdown, and send a message.

LibreChat is a self-hostable multi-provider chat UI.

librechat.yaml

version: 1.2.1

endpoints:
  custom:
    - name: "Northern Inference"
      apiKey: "${NI_API_KEY}"
      baseURL: "https://northerninference.ca/v1"
      models:
        default:
          - "vertex_ai/gemini-2.5-pro-ca"
          - "bedrock/global.anthropic.claude-haiku-4-5-20251001-v1:0-global"
        fetch: false
      titleConvo: true
      titleModel: "bedrock/global.anthropic.claude-haiku-4-5-20251001-v1:0-global"
      summarize: false
      summaryModel: "bedrock/global.anthropic.claude-haiku-4-5-20251001-v1:0-global"
      forcePrompt: false
      modelDisplayLabel: "NI"

Then:

export NI_API_KEY=ni_live_...
docker compose up -d

Open LibreChat at http://localhost:3080, pick Northern Inference from the endpoint dropdown.

Route keys and per-key authorization

The model value is the route contract. Copy exact model IDs from /models. If you want some users limited to Canadian routes and others allowed to use global or provider routes, create separate NI API keys with different allowed route tiers.