Scaling Data Quality Assurance Across 100+ Social Media Platforms

Data quality is the silent killer of AI projects. Models trained on noisy, incomplete, or biased data produce unreliable outputs — and the problem compounds silently through downstream systems. When your data spans 100+ platforms with wildly different formats, quality assurance becomes an engineering discipline in itself.

The Quality Challenge

Consider what "a social media post" means across platforms:

• **Reddit** — Title, body, comments, votes, flair, nested threading
• **YouTube** — Title, description, comments, view counts, channel metadata
• **Mastodon** — Content, boosts, favorites, instance metadata, content warnings
• **4chan** — Anonymous posts, image attachments, board-specific norms
• **GitHub** — Issues, pull requests, code comments, structured metadata

Each platform produces different fields, at different quality levels, with different noise characteristics.

Our Quality Scoring Framework

Every record in our system receives a composite quality score from 0-100, computed from several dimensions:

Completeness Score (0-25)

Measures how many expected fields are present and populated. A Reddit post with title, body, score, and comments scores higher than one with only a title. Missing fields reduce the score proportionally.

Reliability Score (0-25)

Evaluates source trustworthiness based on:

• **Account age and activity history** — Established accounts are more reliable
• **Bot probability** — Higher bot scores reduce reliability
• **Content quality signals** — Grammar, coherence, effort level
• **Engagement authenticity** — Flagged engagement patterns reduce score

Freshness Score (0-25)

Temporal relevance matters. Recent data scores higher, with decay curves tuned per use case. Real-time applications need fresh data; historical analysis values temporal depth.

Signal Strength Score (0-25)

Measures how much useful information the record contains. High-engagement posts, detailed opinions, and content-rich posts score higher than low-effort reactions or single-word replies.

Platform-Specific Calibrations

We maintain platform-specific quality thresholds because quality distributions vary:

• **GitHub** — High baseline quality (structured, technical content)
• **Reddit** — Wide variance; top-subreddit posts are high quality, some subreddits produce mostly noise
• **4chan** — Low average quality; useful signals require aggressive filtering
• **YouTube comments** — Bimodal distribution; insightful reviews alongside emoji-only reactions

Automated Quality Monitoring

We run continuous quality checks:

Impact on Downstream Systems

Quality scores directly impact:

• **Dataset exports** — Customers can filter by minimum quality score
• **Enrichment accuracy** — Higher-quality input produces higher-quality enrichment
• **Training data curation** — Quality-weighted sampling improves model performance
• **Research validity** — Quality-aware analysis produces more reliable insights

— Heshan Sanjuka, Founder