Vision & capabilities | Northern Inference docs

Send images and detect per-model capabilities such as vision and context window on Northern Inference's sovereign Canadian routes for government and regulated teams.

Northern Inference is a sovereign Canadian AI API for government, defence, health and regulated teams, including organizations under the Controlled Goods Program. Every route carries PIPEDA-aligned Canadian data residency, an exact path to the latest frontier and open-weight models, and a per-request chain-of-custody record.

Northern Inference passes images straight through to vision-capable models (Gemini, Claude, Gemini 2.5 Pro-class, and others). Two things are worth knowing: how to send an image, and how a capability-aware client can detect that a route supports images.

How NI advertises capabilities

Every model route NI serves carries capability metadata on both /v1/models and /api/billing/models. The OpenAI-style /v1/models response adds NI fields on each model object:

curl -sS https://northerninference.ca/v1/models \
  -H "Authorization: Bearer $NI_API_KEY" | jq '.data[] | {id, supports_vision, capabilities}'

You will see, per route:

{
  "id": "claude-sonnet-4-6-t4",
  "supports_vision": true,
  "capabilities": {
    "supports_vision": true,
    "modalities": ["text", "image"],
    "context_window": 1048576,
    "max_output_tokens": 65535,
    "supports_function_calling": true
  }
}

supports_vision is mirrored at the top level so a client that reads only flat fields can still detect it. The same capabilities object is on each entry of /api/billing/models.

These facts are data-driven: NI resolves them from the model capability store (the upstream model catalog), then a curated base-model family match, then a conservative default. They are not a hand-maintained list, so newly discovered models carry capabilities automatically.

Send an image (OpenAI image_url content block)

Send the image as a content block on the user message. A base64 data URL works without any upload step:

B64=$(base64 -i screenshot.png | tr -d '\n')
curl -sS https://northerninference.ca/v1/chat/completions \
  -H "Authorization: Bearer $NI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-6-t4",
    "messages": [
      {"role": "user", "content": [
        {"type": "text", "text": "What does this image say? Quote the text exactly."},
        {"type": "image_url", "image_url": {"url": "data:image/png;base64,'"$B64"'"}}
      ]}
    ],
    "max_tokens": 300
  }'

A public https:// image URL works too (the provider fetches it):

{"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}}

Pick any route whose supports_vision is true. Gemini 2.5 pro/flash, Claude 4.x, and Gemini 2.5 Pro-class routes all qualify.

Anthropic-native /v1/messages

If you call /v1/messages instead of /v1/chat/completions, use the Anthropic image block shape:

{"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "<BASE64>"}}

If your client refuses to attach images

Some clients (Cursor, Continue, and other capability-aware tools) keep a built-in table of which model names support images, keyed on standard names like gpt-4o or gemini-2.5-pro. NI route IDs are custom (for example claude-sonnet-4-6-t4), so a client with a hardcoded table will not match the route and will refuse to attach the image, stripping it before it ever reaches NI. The request then arrives as text only and the model never sees the picture.

Two knobs fix this:

Use a client that reads capabilities from /v1/models. It will pick up

supports_vision and allow images on NI routes automatically.

For a client with a hardcoded table, set a per-model override so the

client treats the NI route as vision-capable. Most such clients expose a per-model "supports images" / "vision" toggle in their model settings. Turn it on for the NI route you added. (See the client-specific guide, for example cursor.md, for where that toggle lives.)

If images still do not arrive, confirm the client is actually embedding the image as an image_url block and not sending a file path as plain text. You can verify NI received an image by checking the request in portal → Usage: a vision request shows image content parts.

Reasoning and thinking effort

Models that think before answering (Claude extended thinking, OpenAI o-series and gpt-5+ reasoning) expose two different controls. Pick the portable one unless you are certain your route is Anthropic.

Use reasoning_effort for cross-provider portability

reasoning_effort takes low, medium, or high and is the portable control:

On an OpenAI reasoning route (o-series, gpt-5+) it maps to that model's native reasoning effort.
On a Claude route, NI maps it to the control that route uses. A route on adaptive thinking gets adaptive thinking at that effort. A route on budget thinking gets an Anthropic thinking budget (low about 8k, medium about 25k, high about 64k thinking tokens).

{
  "model": "claude-sonnet-4-6-t4",
  "messages": [{"role": "user", "content": "Plan a refactor of this module."}],
  "reasoning_effort": "high",
  "max_tokens": 70000
}

Reasoning works on chat completions and on Responses

Send reasoning_effort on /v1/chat/completions, or reasoning.effort on /v1/responses. Either way the model reasons, and usage reports the reasoning tokens.

Reasoning text is opt-in, as in OpenAI's Responses API. It comes back only when a /v1/responses request sets reasoning.summary to "auto" or "detailed". On /v1/chat/completions, a reasoning_summary field is refused with a 400.
Without reasoning.summary, the model still reasons and usage still counts it, but the reply's reasoning item has an empty summary and no readable text. What that item carries on Claude routes is explained under "Keep Claude's reasoning between tool calls on /v1/responses" below.
Some models refuse a sampling setting such as temperature or top_p. NI leaves a refused setting out, so the request runs with the model's default instead of failing. The x-ni-params-dropped response header names what was left out, and a /v1/responses reply shows those fields as null.

thinking budget_tokens is Anthropic-native, auto-translated elsewhere

The Anthropic-native thinking block (and Claude Code's effort flag, which sends it) is native on Anthropic routes. On a non-Anthropic route, NI automatically converts the thinking budget into the portable reasoning_effort (bucketed to low, medium, or high) so your reasoning request still runs on GPT, Gemini, DeepSeek, and other reasoning models, instead of being dropped. That conversion is coarse (an exact token budget becomes a three-level effort), so for precise cross-provider control prefer reasoning_effort directly. A route with no reasoning mode at all (for example Llama or Mistral) returns a normal answer with no reasoning.

max_tokens must exceed the thinking budget

The thinking budget is spent out of max_tokens. If your budget is 32000, max_tokens must be larger (for example 40000), or the request returns a 400 because there is no room left for the visible answer. Rule of thumb: set max_tokens to at least the thinking budget plus the longest answer you expect.

Where the reasoning text goes

Anthropic routes return the thinking text in a thinking content block (or reasoning_content on the OpenAI-compatible path), so you can show or log it.
OpenAI o-series routes hide the raw reasoning; you get only the final answer plus a reasoning-token count in usage.

NI bills thinking tokens explicitly using each route's thinking rate, with no double-billing against output tokens.

Keep Claude's reasoning between tool calls on /v1/responses

When a Claude route thinks before it calls a tool, the /v1/responses reply includes a reasoning item in output, just before the item it led to. Its encrypted_content field carries Claude's thinking blocks exactly as the model returned them, with Claude's signature.

Send these items back so Claude can pick up its own reasoning after your tool runs:

Keep every item from output, in the order you received it, including each reasoning item.
On the next request, put those items in input, followed by your function_call_output items.
Do not change, reorder or rebuild the reasoning items.

If you leave them out, the request still works, but Claude answers the tool result without its earlier thinking.

You get encrypted_content on every reply from these routes. You do not need to send include: ["reasoning.encrypted_content"], and sending it changes nothing.
While Northern Inference has this turned off, replies carry no encrypted_content, x-ni-reasoning-state says skipped=disabled, and reasoning items you send back are left out. Your requests still work.
NI does not store responses, so previous_response_id cannot bring the reasoning back for you. Keep the items yourself.
A reply can contain more than one reasoning item, for example when Claude writes a short progress note before a later tool call. Keep all of them.
summary contains readable thinking text only when the route returns it, for example when you ask for reasoning.summary on a route that supports it. Otherwise summary is empty and the reasoning travels only in encrypted_content.
NI signs encrypted_content so it can detect changes. NI does not encrypt it, and it can contain the same text you see in summary. Store it with the same care as the rest of the conversation.
NI passes your reasoning on only when the same provider, on the same account, serves your next request. Otherwise that request runs without the earlier reasoning, and x-ni-reasoning-state says skipped=domain_mismatch.
NI checks each item it receives. If an item was changed, came from a different route or a different account, or is not directly before the item it belongs to, NI leaves out every reasoning item from that reply, not just that one. The request then runs as if you had not sent that reasoning. The x-ni-reasoning-state response header tells you whether your reasoning was sent on.
Reasoning items from other providers are ignored on Claude routes, and NI removes its own items before a request goes to a route that cannot use them.
Claude reads the reasoning you send back as input, billed as input tokens or cached input tokens exactly as the provider reports them.