Introduction
LLM-powered semantic conversation routing engine with high-accuracy drift detection
DriftOS Core
LLM-powered semantic conversation routing engine. High-accuracy drift detection using Groq's fast inference for real-time routing decisions.
Overview
DriftOS Core uses LLM reasoning to detect topic shifts and route messages:
- STAY - Same topic, continue in current branch
- BRANCH - Topic drift detected, create new branch
- ROUTE - Return to a previous topic
Result: Focused context windows. 20 relevant messages instead of 1000.
Why LLM-Based?
| Approach | Latency | Cost | Accuracy |
|---|---|---|---|
| LLM-based routing | 500-1000ms | ~$0.001/call | High |
| Embedding-based | <200ms | $0 | Good |
DriftOS Core uses Groq's fast inference (Llama 3.1) for nuanced understanding of topic shifts, context, and conversational intent. Best for applications where accuracy matters more than latency.
How It Works
- Analyze - LLM examines message content, conversation history, and existing branches
- Decide - Returns STAY, BRANCH, or ROUTE with reasoning
- Execute - Creates branch if needed, assigns message, updates state
- Extract - When leaving a branch, facts are automatically extracted
Performance
- Routing latency: 500-1000ms (Groq inference)
- Accuracy: High (LLM reasoning)
- Cost: ~$0.001 per routing decision
When to Use
Use DriftOS Core when:
- Accuracy is more important than latency
- You need nuanced understanding of topic shifts
- Your conversations have subtle context that embeddings might miss
For faster routing with zero LLM costs, see DriftOS Embed.