Language Model Configuration

elizaOS uses a plugin-based architecture for integrating different Language Model providers. This guide explains how to configure and use LLM plugins, including fallback mechanisms for embeddings and model registration.

Key Concepts

Model Types

elizaOS supports many types of AI operations. Here are the most common ones:

TEXT_GENERATION (TEXT_SMALL, TEXT_LARGE) - Having conversations and generating responses
TEXT_EMBEDDING - Converting text into numbers for memory and search
OBJECT_GENERATION (OBJECT_SMALL, OBJECT_LARGE) - Creating structured data like JSON

Think of it like different tools in a toolbox:

Text Generation = Having a conversation
Embeddings = Creating a “fingerprint” of text for finding similar things later
Object Generation = Filling out forms with specific information

Plugin Capabilities

Not all LLM plugins support all model types. Here’s what each can do:

Plugin	Text Chat	Embeddings	Structured Output	Runs Offline
OpenAI	✅	✅	✅	❌
Anthropic	✅	❌	✅	❌
Google GenAI	✅	✅	✅	❌
Ollama	✅	✅	✅	✅
OpenRouter	✅	❌	✅	❌

Key Points:

🌟 OpenAI & Google GenAI = Do everything (jack of all trades)
💬 Anthropic & OpenRouter = Amazing at chat, need help with embeddings
🏠 Ollama = Your local hero - does almost everything, no internet needed!

Plugin Loading Order

The order in which plugins are loaded matters significantly. From the default character configuration:

plugins: [
  // Core plugins first
  '@elizaos/plugin-sql',

  // Text-only plugins (no embedding support)
  ...(process.env.ANTHROPIC_API_KEY?.trim() ? ['@elizaos/plugin-anthropic'] : []),
  ...(process.env.OPENROUTER_API_KEY?.trim() ? ['@elizaos/plugin-openrouter'] : []),

  // Embedding-capable plugins (optional, based on available credentials)
  ...(process.env.OPENAI_API_KEY?.trim() ? ['@elizaos/plugin-openai'] : []),
  ...(process.env.GOOGLE_GENERATIVE_AI_API_KEY?.trim() ? ['@elizaos/plugin-google-genai'] : []),

  // Ollama as fallback (only if no main LLM providers are configured)
  ...(process.env.OLLAMA_API_ENDPOINT?.trim() ? ['@elizaos/plugin-ollama'] : []),
]

Understanding the Order

Think of it like choosing team players - you pick specialists first, then all-rounders:

Anthropic & OpenRouter go first - They’re specialists! They’re great at text generation but can’t do embeddings. By loading them first, they get priority for text tasks.
OpenAI & Google GenAI come next - These are the all-rounders! They can do everything: text generation, embeddings, and structured output. They act as fallbacks for what the specialists can’t do.
Ollama comes last - This is your local backup player! It supports almost everything (text, embeddings, objects) and runs on your computer. Perfect when cloud services aren’t available.

Why This Order Matters

When you ask elizaOS to do something, it looks for the best model in order:

Generate text? → Anthropic gets first shot (if loaded)
Create embeddings? → Anthropic can’t, so OpenAI steps in
No cloud API keys? → Ollama handles everything locally

This smart ordering means:

You get the best specialized models for each task
You always have fallbacks for missing capabilities
You can run fully offline with Ollama if needed

Real Example: How It Works

Let’s say you have Anthropic + OpenAI configured:

Task: "Generate a response"
1. Anthropic: "I got this!" ✅ (Priority 100 for text)
2. OpenAI: "I'm here if needed" (Priority 50)

Task: "Create embeddings for memory"
1. Anthropic: "Sorry, can't do that" ❌
2. OpenAI: "No problem, I'll handle it!" ✅

Task: "Generate structured JSON"
1. Anthropic: "I can do this!" ✅ (Priority 100 for objects)
2. OpenAI: "Standing by" (Priority 50)

Model Registration

When plugins load, they “register” what they can do. It’s like signing up for different jobs:

// Each plugin says "I can do this!"
runtime.registerModel(
  ModelType.TEXT_LARGE,        // What type of work
  generateText,                // How to do it
  'anthropic',                 // Who's doing it
  100                         // Priority (higher = goes first)
);

How elizaOS Picks the Right Model

When you ask elizaOS to do something, it:

Checks what type of work it is (text? embeddings? objects?)
Looks at who signed up for that job
Picks based on priority (higher number goes first)
If tied, first registered wins

Example: You ask for text generation

Anthropic registered with priority 100 ✅ (wins!)
OpenAI registered with priority 50
Ollama registered with priority 10

But for embeddings:

Anthropic didn’t register ❌ (can’t do it)
OpenAI registered with priority 50 ✅ (wins!)
Ollama registered with priority 10

Embedding Fallback Strategy

Remember: Not all plugins can create embeddings! Here’s how elizaOS handles this: The Problem:

You’re using Anthropic (great at chat, can’t do embeddings)
But elizaOS needs embeddings for memory and search

The Solution: elizaOS automatically finds another plugin that CAN do embeddings!

// What happens behind the scenes:
// 1. "I need embeddings!"
// 2. "Can Anthropic do it?" → No ❌
// 3. "Can OpenAI do it?" → Yes ✅
// 4. "OpenAI, you're up!"

Common Patterns

Anthropic + OpenAI Fallback

{
  "plugins": [
    "@elizaos/plugin-anthropic",  // Primary for text
    "@elizaos/plugin-openai"       // Fallback for embeddings
  ]
}

OpenRouter + Local Embeddings

{
  "plugins": [
    "@elizaos/plugin-openrouter",  // Cloud text generation
    "@elizaos/plugin-ollama"        // Local embeddings
  ]
}

Configuration

Environment Variables

Each plugin requires specific environment variables:

# .env file

# OpenAI
OPENAI_API_KEY=sk-...
OPENAI_SMALL_MODEL=gpt-4o-mini          # Optional: any available model
OPENAI_LARGE_MODEL=gpt-4o               # Optional: any available model

# Anthropic  
ANTHROPIC_API_KEY=sk-ant-...
ANTHROPIC_SMALL_MODEL=claude-3-haiku-20240307    # Optional: any Claude model
ANTHROPIC_LARGE_MODEL=claude-3-5-sonnet-latest   # Optional: any Claude model

# Google GenAI
GOOGLE_GENERATIVE_AI_API_KEY=...
GOOGLE_SMALL_MODEL=gemini-2.0-flash-001  # Optional: any Gemini model
GOOGLE_LARGE_MODEL=gemini-2.5-pro-preview-03-25  # Optional: any Gemini model

# Ollama
OLLAMA_API_ENDPOINT=http://localhost:11434/api
OLLAMA_SMALL_MODEL=llama3.2              # Optional: any local model
OLLAMA_LARGE_MODEL=llama3.1:70b          # Optional: any local model
OLLAMA_EMBEDDING_MODEL=nomic-embed-text  # Optional: any embedding model

# OpenRouter
OPENROUTER_API_KEY=sk-or-...
OPENROUTER_SMALL_MODEL=google/gemini-2.0-flash-001  # Optional: any available model
OPENROUTER_LARGE_MODEL=anthropic/claude-3-opus      # Optional: any available model

**Important**: The model names shown are examples. You can use any model available from each provider.

### Character-Specific Secrets

You can also configure API keys per character:

```json
{
  "name": "MyAgent",
  "settings": {
    "secrets": {
      "OPENAI_API_KEY": "sk-...",
      "ANTHROPIC_API_KEY": "sk-ant-..."
    }
  }
}

Available Plugins

Cloud Providers

OpenAI Plugin - Full-featured with all model types
Anthropic Plugin - Claude models for text generation
Google GenAI Plugin - Gemini models
OpenRouter Plugin - Access to multiple providers

Local/Self-Hosted

Ollama Plugin - Run models locally with Ollama

Best Practices

1. Always Configure Embeddings

Even if your primary model doesn’t support embeddings, always include a fallback:

{
  "plugins": [
    "@elizaos/plugin-anthropic",
    "@elizaos/plugin-openai"  // For embeddings
  ]
}

2. Order Matters

Place your preferred providers first, but ensure embedding capability somewhere in the chain.

3. Test Your Configuration

Verify all model types work:

// The runtime will log which provider is used for each operation
[AgentRuntime][MyAgent] Using model TEXT_GENERATION from provider anthropic
[AgentRuntime][MyAgent] Using model EMBEDDING from provider openai

4. Monitor Costs

Different providers have different pricing. Consider:

Using local models (Ollama) for development
Mixing providers (e.g., OpenRouter for text, local for embeddings)
Setting up usage alerts with your providers

Troubleshooting

”No model found for type EMBEDDING”

Your configured plugins don’t support embeddings. Add an embedding-capable plugin:

{
  "plugins": [
    "@elizaos/plugin-anthropic",
    "@elizaos/plugin-openai"  // Add this
  ]
}

“Missing API Key”

Ensure your environment variables are set:

# Check current environment
echo $OPENAI_API_KEY

# Or use the CLI
elizaos env edit-local

Models Not Loading

Check plugin initialization in logs:

Success: Plugin @elizaos/plugin-openai initialized successfully

Migration from v0.x

In elizaOS v0.x, models were configured directly in character files:

// ❌ OLD (v0.x) - No longer works
{
  "modelProvider": "openai",
  "model": "gpt-4"
}

// ✅ NEW (v1.x) - Use plugins
{
  "plugins": ["@elizaos/plugin-openai"]
}

The modelProvider field is now ignored. All model configuration happens through plugins.

Getting Started

Core Plugins

LLM Providers

Platform Integrations

DeFi Plugins

Key Concepts

Model Types

Plugin Capabilities

Plugin Loading Order

Understanding the Order

Why This Order Matters

Real Example: How It Works

Model Registration

How elizaOS Picks the Right Model

Embedding Fallback Strategy

Common Patterns

Anthropic + OpenAI Fallback

OpenRouter + Local Embeddings

Configuration

Environment Variables

Available Plugins

Cloud Providers

Local/Self-Hosted

Best Practices

1. Always Configure Embeddings

2. Order Matters

3. Test Your Configuration

4. Monitor Costs

Troubleshooting

”No model found for type EMBEDDING”

“Missing API Key”

Models Not Loading

Migration from v0.x

Getting Started

Core Plugins

LLM Providers

Platform Integrations

DeFi Plugins

​Key Concepts

​Model Types

​Plugin Capabilities

​Plugin Loading Order

​Understanding the Order

​Why This Order Matters

​Real Example: How It Works

​Model Registration

​How elizaOS Picks the Right Model

​Embedding Fallback Strategy

​Common Patterns

​Anthropic + OpenAI Fallback

​OpenRouter + Local Embeddings

​Configuration

​Environment Variables

​Available Plugins

​Cloud Providers

​Local/Self-Hosted

​Best Practices

​1. Always Configure Embeddings

​2. Order Matters

​3. Test Your Configuration

​4. Monitor Costs

​Troubleshooting

​”No model found for type EMBEDDING”

​“Missing API Key”

​Models Not Loading

​Migration from v0.x

Key Concepts

Model Types

Plugin Capabilities

Plugin Loading Order

Understanding the Order

Why This Order Matters

Real Example: How It Works

Model Registration

How elizaOS Picks the Right Model

Embedding Fallback Strategy

Common Patterns

Anthropic + OpenAI Fallback

OpenRouter + Local Embeddings

Configuration

Environment Variables

Available Plugins

Cloud Providers

Local/Self-Hosted

Best Practices

1. Always Configure Embeddings

2. Order Matters

3. Test Your Configuration

4. Monitor Costs

Troubleshooting

”No model found for type EMBEDDING”

“Missing API Key”

Models Not Loading

Migration from v0.x