LLM Providers

Usejarvis supports multiple LLM providers and routes requests between them by tier.

Since 0.9, LLM configuration lives in the dashboard: Settings → LLM at http://localhost:3142. Non-secret settings are stored in the database; API keys go to an encrypted keychain. config.yaml and environment variables have no authority over LLM settings: any llm: block in the file is ignored on load, and env vars like JARVIS_OPENAI_KEY are no longer read.

Saving provider changes applies them immediately on the running daemon. No restart needed.

Supported Providers

• Anthropic
• OpenAI
• Groq
• Gemini
• Ollama (local, base_url-based)
• OpenRouter
• NVIDIA NIM
• OpenAI-compatible (any proxy or server speaking the OpenAI API, base_url-based)
• LiteLLM (base_url-based)

Providers are named instances: each entry has a name you choose and a kind that selects the provider class. Two Ollama instances (say ollama-local and ollama-remote) can coexist, and models are referenced as name:model-id, for example anthropic:claude-sonnet-4-6 or ollama-local:qwen2.5:3b.

Single Model or Multi-Tier

The dashboard offers two modes:

• Single: one default model handles everything. The right starting point.
• Multi-tier: requests are routed by tier. Each tier maps to one provider and model:
  • conversation: fast model for chat; its presence switches on router-first mode
  • high: hardest tasks
  • medium: standard tasks
  • low: cheap background work

An unassigned tier falls up (low → medium → high); conversation never falls up. At least one of medium or high must be set in multi-tier mode.

Retry Behavior

There is no cross-provider fallback list. A failing provider is retried up to 3 times with a 90 second timeout per call; requests do not silently switch providers. Reliability comes from tier fall-up, not provider chains.

Provider Notes

Anthropic

Best default for many users: strong instruction following and tool use, good balance for complex autonomous tasks.

OpenAI

A strong alternative with a broad model family and good tool use.

Groq

Useful when you care about latency.

Gemini

A strong additional cloud option.

Ollama

Best when you want local inference or reduced cloud dependence: no per-token billing, local hosting.

The dashboard lists exactly the models you have pulled, with their tags (for example qwen2.5:3b), by querying your Ollama install through the daemon. Untagged names like llama3 resolve to :latest and fail if you have not pulled that tag.

Important operational note:

• base_url is resolved from the daemon’s network point of view, not your browser’s.
• If the daemon runs on a VPS and Ollama runs on your laptop, http://localhost:11434 will not work unless the daemon and Ollama are on the same host.

This is one of the most common setup mistakes. See Troubleshooting.

OpenRouter

Access to many models through one API key.

NVIDIA NIM

Key-based access to NVIDIA-hosted models; the dashboard can list available models live.

OpenAI-compatible and LiteLLM

Point Usejarvis at any endpoint that speaks the OpenAI API (a proxy, a gateway, a self-hosted server). Model id is free text.

Prompt Caching

Prompt caching is on by default for providers that support it, reducing cost and latency on long agent loops. You can disable it per provider in the LLM settings. Cache hit rates are visible in the Usage room.

Good Setups

• Cloud-first: single mode with one Anthropic or OpenAI model. Add tiers later.
• Hybrid: multi-tier with a cloud model on high/medium and a local Ollama model on low.
• Local-first: single mode with an Ollama model. Expect weaker tool use than frontier cloud models.