Routing

DriftOS routes messages to semantic branches using one of two strategies: LLM-based (Core) or embedding-based (Embed).

DriftOS Core (LLM Routing)

How It Works

1. Analyze - LLM examines message content, conversation history, and existing branches
2. Decide - Returns STAY, BRANCH, or ROUTE with reasoning
3. Execute - Creates branch if needed, assigns message, updates state

The Three Actions

STAY - Continue in current branch

Current: "dog-adoption"
Message: "What's the adoption fee?"
→ STAY (same topic)

BRANCH - Create new branch

Current: "dog-adoption"
Message: "Tell me about black holes"
→ BRANCH (new topic detected)

ROUTE - Switch to existing branch

Current: "black-holes"
Message: "Back to dogs—what was the fee?"
→ ROUTE to "dog-adoption" (returning to old topic)

Performance

• Routing latency: 500-1000ms (Groq inference)
• Accuracy: High (LLM reasoning)
• Cost: ~$0.001 per routing decision

When to Use

Use DriftOS Core when:

• Accuracy is more important than latency
• You need nuanced understanding of topic shifts
• Your conversations have subtle context that embeddings might miss
• Users frequently switch back to old topics

DriftOS Embed (Embedding Routing)

How It Works

1. Embed - Message is embedded using paraphrase-MiniLM-L6-v2
2. Compare - Cosine similarity against current branch centroid
3. Decide - Based on thresholds: STAY or BRANCH
4. Update - Branch centroid updated with running average

The Two Actions

STAY - Continue in current branch

Current: "dog-adoption"
Message: "What's the adoption fee?"
Similarity: 0.87 (> 0.70 threshold)
→ STAY

BRANCH - Create new branch

Current: "dog-adoption"
Message: "Tell me about black holes"
Similarity: 0.23 (<0.70 threshold)
→ BRANCH

Note: DriftOS Embed does NOT have a ROUTE action. It creates new branches instead of switching to old ones.

Threshold Logic

similarity > 0.70  → STAY (same topic)
similarity <0.70  → BRANCH (new topic)

Performance

• Routing latency: <200ms
• Embedding generation: ~30ms
• Zero LLM costs for routing decisions
• LLM used only for fact extraction (optional)

When to Use

Use DriftOS Embed when:

• Latency is critical (<200ms)
• You want zero LLM API costs for routing
• Your topic shifts are clear/obvious
• Conversations are generally sequential (no jumping back to old topics)

Comparison

Next Steps

• Choosing a Routing Engine - Detailed comparison
• Branches - How branches work
• Troubleshooting - Fix routing issues