For months, our enrichment pipeline relied on Groq-powered LLM calls to extract sentiment, emotion, topics, and signals from social media posts. It worked well, but came with tradeoffs we couldn't ignore.
The Problem with LLM Enrichment
LLMs are powerful, but for structured data enrichment at scale, they introduce several challenges:
- **Cost per post** — Each API call adds up when you're processing millions of posts daily
- **Latency** — LLM calls take 500ms-2s each, creating bottlenecks
- **Non-determinism** — The same post can get different results on different runs
- **Token limits** — Long posts get truncated, losing context
Building the Heuristic Engine
We replaced the LLM pipeline with a deterministic rule engine built on pattern matching, lexicon analysis, and statistical models. The engine uses:
- **VADER sentiment** — Optimized for social media text with custom augmentations
- **Keyword-based topic detection** — 200+ topic categories with hierarchical matching
- **Entity extraction** — Regex and dictionary-based recognition for people, orgs, locations, products
- **Engagement scoring** — Statistical models tuned per platform
- **Bot detection signals** — Behavioral pattern analysis
Results
The numbers speak for themselves:
- Latency dropped from ~800ms to <1ms per post
- API costs went to zero
- Results are perfectly reproducible
- Quality scores remained within 2% of the LLM baseline
Lessons Learned
Not every ML problem needs a neural network. For structured classification tasks with clear patterns, deterministic approaches often win. We still use ML for certain signals, but the heuristic engine handles 90% of our enrichment workload.
— Heshan Sanjuka, Founder