Kode AI Provider Configuration: Gateway, Direct Providers

Configure AI providers in Kode: the Kode Gateway for unified access, or direct connections to OpenAI, Anthropic, Google, Groq, and more.

Kode supports 25+ AI providers and ships with its own gateway at api.trykode.xyz for unified, zero-friction access to the best models across OpenAI, Anthropic, DeepSeek, Google, and OpenRouter. You configure your provider once — in kode.json or via environment variables — and Kode handles routing, streaming, and key management automatically.

The Kode Gateway

The Kode Gateway is the default provider. It is a routing layer that sits in front of multiple upstream providers and exposes a single OpenAI-compatible endpoint. You authenticate with one key (KODE_LLM_API_KEY) and the gateway handles dispatch to the right upstream model.

The gateway is pre-configured in every kode init output:

.kode/kode.json

{
  "providers": {
    "primary": "kode",
    "gateway_url": "https://api.trykode.xyz"
  }
}

To authenticate, set your API key:

export KODE_LLM_API_KEY=your-kode-api-key

The gateway routes to OpenAI, Anthropic, DeepSeek, Google, and OpenRouter based on the model you select in kode.json. You do not need individual provider keys when using the gateway.

Gateway tiers

Tier	How to activate	Rate limit	Models
Lite (free)	No key required — use "public"	20 requests / day / IP	deepseek-v4-flash, nvidia/nemotron-3-nano-30b-a3b:free
Pro	Set KODE_PRO_API_KEY	Unlimited	deepseek-v4-pro, deepseek-r1, and all Lite models

The Lite tier is rate-limited at the gateway level. When you exceed 20 requests per day, the gateway returns a rate limit error explaining that your daily Lite quota has been reached. Set KODE_PRO_API_KEY or upgrade at trykode.xyz/pricing to remove the limit.

Using a direct provider

If you prefer to call a provider’s API directly — bypassing the Kode Gateway — set model in kode.json to the provider/model string for your chosen model and export the corresponding API key environment variable.

1. Set the model in kode.json

   Edit .kode/kode.json and set the model field to the provider/model string. For example, to use Claude Sonnet 4.6 directly via Anthropic:

   .kode/kode.json

   {
     "model": "anthropic/claude-sonnet-4-6",
     "small_model": "anthropic/claude-haiku-4-5"
   }
   
2. Export the provider API key

   Set the environment variable corresponding to your chosen provider:

   Anthropic

   export ANTHROPIC_API_KEY=sk-ant-...
   
3. Run Kode

   Kode picks up the key and routes directly to the provider:

   kode run "add input validation to the login handler"
   

Supported model strings

Use these strings in the model and small_model fields of your kode.json:

Provider	Model string	Notes
Anthropic	anthropic/claude-sonnet-4-6	Default — recommended for most tasks
Anthropic	anthropic/claude-haiku-4-5	Fast and cheap; default small_model
OpenAI	openai/gpt-4o	Powerful, widely supported
OpenAI	openai/o3-mini	Fast reasoning model
Google	google/gemini-2.0-flash	Fast and cost-effective
Groq	groq/llama-3.3-70b-versatile	Very fast inference via Groq hardware
DeepSeek	deepseek/deepseek-v4-flash	Fast reasoning, strong coding
DeepSeek	deepseek/deepseek-r1	Deep chain-of-thought reasoning

Run kode models to see every model string available for your configured providers, including metadata such as input and output cost per million tokens.

Provider management CLI

The TypeScript CLI layer exposes providers and models subcommands for managing credentials and browsing available models. These commands are forwarded automatically from the kode binary:

Credentials

# Show all configured providers and their auth status
kode providers list

# Interactively log in to a provider (stores credential in ~/.kode/auth.json)
kode providers login

# Log in to a specific provider directly, skipping the selection prompt
kode providers login --provider anthropic

# Remove a stored credential
kode providers logout

Models

# List all available models across configured providers
kode models

# Filter to a single provider
kode models --provider anthropic

# Show verbose output including cost metadata
kode models --verbose

# Refresh the model catalog cache from models.dev
kode models --refresh

Credentials stored via kode providers login are saved to ~/.kode/auth.json. Environment variables (e.g. ANTHROPIC_API_KEY) are detected automatically by kode providers list and shown alongside stored credentials. Environment variables always take precedence over stored credentials at runtime.

Pointing Kode at a local model

You can run Kode against any OpenAI-compatible local inference server — Ollama, LM Studio, llama.cpp, vLLM, and others all work. Override the endpoint and model via environment variables:

# Ollama with Llama 3.1 8B
export KODE_LLM_ENDPOINT=http://localhost:11434/v1
export KODE_LLM_MODEL=llama3.1:8b
export KODE_LLM_API_KEY=ollama   # Ollama accepts any non-empty key

kode run "refactor the auth middleware"

# vLLM serving Mistral
export KODE_LLM_ENDPOINT=http://localhost:8000/v1
export KODE_LLM_MODEL=mistralai/Mistral-7B-Instruct-v0.3
export KODE_LLM_API_KEY=token-abc123

kode run "write unit tests for the payment service"

Local models vary widely in instruction-following quality. Kode’s verification pipeline (blast-radius checks, TDD gating, SAST scanning) still runs against the output, so bad patches are caught before reaching disk — but expect more retries and self-correction cycles with smaller models.

Related

kode.json reference

Full reference for all model, engine, and permission settings.

Environment Variables

Complete list of environment variables including all provider key names.