Skip to content

Labeeb Platform

Welcome to the Labeeb documentation hub — your guide to understanding, building with, and operating the AI-powered news analysis platform. Whether you're exploring how the platform works, integrating with the API, or responding to an incident, start here.

How It Works API Reference Architecture

Business Value

Labeeb transforms raw news into verified, evidence-backed intelligence — reducing manual fact-checking costs by 90% while delivering sub-second results across Arabic and English. These docs enable teams to integrate, operate, and scale the platform with confidence.

Platform Status: Operational

Labeeb automates the entire content verification lifecycle — from ingestion and enrichment to evidence retrieval and credibility scoring — across Arabic and English. These docs cover platform concepts, service APIs, operational runbooks, and deployment guides.

On-Call: Platform Team · Escalation: #platform-oncall

1. Platform Mission & Scope

Labeeb's mission is to automate the entire content analysis lifecycle—from ingestion and enrichment to delivering evidence-backed insights via a REST API. We aim to combat misinformation by providing trustworthy, verified news analysis at scale.

Scope of Responsibilities

Labeeb Platform is responsible for: - ✅ Automated news ingestion from RSS, HTML, and API sources - ✅ AI-powered fact-checking and information extraction - ✅ Hybrid search (keyword + semantic) with evidence retrieval - ✅ Multilingual support (Arabic and English) - ✅ Real-time indexing and ranking of content

NOT responsible for: - ❌ Manual content curation - ❌ Opinion generation - ❌ Content distribution to third parties

2. Why Labeeb

  • Evidence you can trust Every result is backed by citations and diagnostics — see the why, not just the what.

  • Fast where it counts Optimized retrieval and lean workflows keep latency low and UIs responsive.

  • Arabic-first, globally accessible Full Arabic and English support from ingestion to UI — RTL/LTR, localized copy, and language-aware processing.

  • Built for reliability Queue-based processing, circuit breakers, graceful degradation, and comprehensive observability baked in.

3. Platform Architecture Overview

flowchart LR
    A[External Sources] --> B[Scraper]
    B --> C[API Service]
    C --> D[Queue Workers]
    D --> E[AI-Box]
    E --> F[SinaTools NLP]
    D --> G[(PostgreSQL)]
    D --> H[(OpenSearch)]
    E --> H
    C --> I[Frontend]
    I --> J[Cloudflare Edge]

Platform Components

Key Services: - API Service () — Ingestion gateway and orchestration layer - AI-Box Service () — Evidence retrieval and AI analysis - Scraper Service () — Content extraction from various sources - SinaTools (:material-language:) — Arabic NLP processing - Search Service () — Hybrid BM25+kNN retrieval - Frontend () — Bilingual user interface

4. Choose Your Path

  • Understand how it works A non-technical walkthrough of the platform's components, processing stages, and infrastructure. How It Works

  • :material-http: Make your first request Send a health check, then an authenticated API call. API Reference

  • See retrieval in action Hybrid search with language filters and ranked citations. Search Service

  • Ingest your first article Learn the ingestion contract, idempotency, and validation. /v1/ingest/articles

  • Shape search results Hybrid BM25+vector, filters, and ranking modes. Search Overview

  • Ship a clean analyst UX Bilingual (AR/EN), RTL/LTR, and streaming responses. Frontend Guide

  • Deploy with confidence Environments, Cloudflare integration, and rollout playbooks. Deploy & Infra

  • SLOs & observability Latency, error rates, and queue depth—what to watch and where. Observability

  • Runbooks Incident response: backlog, timeouts, ingestion errors. Runbooks

System Overview

Scraper → API Service → Database ↘︎ Search Service ↔︎ AI-Box ↘︎ Frontend

Think of it as a digital newsroom: the Scraper is your field reporter gathering stories; the API Service is the editor organizing and filing them into the Database; the Search Service is the library that finds the right clippings; the AI-Box is the fact-checker analyzing and comparing; and the Frontend is the polished front page where people read and interact.

  • Scraper Service Profile-driven ingestion from RSS, HTML, and partner APIs; normalization, scheduling, and deduplication. Overview

  • API Service Central orchestrator — ingestion gateway, pipeline dispatch, search proxy, and single writer to all datastores. Overview · Endpoints

  • Search Service Hybrid retrieval (BM25 + kNN) with RRF fusion, Arabic analyzers, and ranking controls. Overview

  • AI-Box Service Evidence retrieval, stance detection, check-worthiness scoring, and verdict generation. Overview · Endpoints

  • SinaTools NLP Arabic-specific morphology, named entity recognition, and linguistic analysis for the AI pipeline. Overview

  • Frontend Next.js bilingual interface (AR/EN) with RTL/LTR support, streaming results, and evidence exploration. Overview

  • Infrastructure Terraform-managed cloud resources and Cloudflare edge layer for CDN, WAF, and global delivery. Terraform · Cloudflare

Operational Metrics to Monitor

Key Performance Indicators: - API Response Time: < 500ms (p95) - Ingestion Queue Depth: < 100 jobs - Search Latency: < 300ms (p95) - AI-Box Processing Time: < 2s per article - Database Connection Pool: > 20% available

Health Endpoints: - Platform: http://localhost:8080/health - API Service: http://localhost:8080/api/health - AI-Box: http://localhost:8081/health - Search: http://localhost:9200/_cluster/health

Data model at a glance

Labeeb's core entities make the system predictable, extensible, and easy to query.

  • Article — normalized content with source metadata
  • Claim — checkworthy statements extracted from articles
  • Entity — people, orgs, and places detected across content
  • Source — eTLD+1 attribution for consistent provenance
  • Annotation — machine/analyst labels, judgments, and notes

Explore the schema

5. API Specs & Playground

  • API Service (OpenAPI 3.0) Contracts, auth, errors, pagination. Spec

  • AI-Box Service (OpenAPI 3.0) Retrieval, ranking, explainability. Spec

  • Scraper Service (OpenAPI 3.0) Profiles, schedules, selectors. Spec

Use the interactive Swagger UI to explore and test endpoints. API Service Spec

6. Operational Excellence

  • Runbooks Step-by-step playbooks for common incidents—before you restart anything. Runbooks

  • SLOs & Observability What we measure (latency, errors, queue depth) and how we alert. Observability

  • GitHub Management Branching, PR templates, releases, and versioning. Guide

  • Authentication & Security Bearer tokens, environment secrets, and origin protection. Security

7. Multilingual by Design

Arabic-First, Globally Accessible

Labeeb natively supports Arabic and English from ingestion to UI.

  • APIs accept lang (ar or en) and preserve context across the pipeline.
  • UI supports full RTL/LTR with mirrored layouts and localized copy.
  • Attribution uses domain normalization (eTLD+1) for consistent source names.
  • Switch language anytime from the footer or by using a lang query parameter.

FAQs

  • TL;DR: Local dev uses sample keys; production uses environment secrets & service policies.
  • Do next: Create a service account, set env vars, test /health then an authenticated call.
  • API Auth
  • TL;DR: Full AR/EN support; UI is RTL/LTR; APIs accept lang (ar/en).
  • Do next: Pass lang in requests; enable locale toggle in the UI.
  • Frontend i18n
  • TL;DR: Check health → dashboards → use degrade modes if needed.
  • Do next: Inspect query diagnostics; verify index health; apply fallback strategy.
  • API Runbook
  • TL;DR: Supported via Cloudflare Workers for global low-latency.
  • Do next: Configure environment, bind secrets, deploy Worker, smoke-test endpoints.
  • Cloudflare & Infra
  • TL;DR: Bearer tokens, scoped secrets, origin protection (AOP).
  • Do next: Rotate keys, restrict roles, enable mTLS/AOP where applicable.
  • Security
  • TL;DR: Core entities: Article, Claim, Entity, Source, Annotation.
  • Do next: Review schema, map IDs, align queries to relationships.
  • Schema

8. Development & Operations

Area What you'll find Key links
Contributing Coding standards, branching strategy, PR flow GitHub Guide · Coding Guidelines
Local development Platform overview, env configuration, smoke tests How It Works · API Runbook
Testing Unit, contract, and end-to-end strategy with examples API Service
Data model Entities, relationships, and versioning conventions Schema
Search & AI-Box Retrieval, ranking, evaluation, explainability Search · AI-Box
Deploy & Infra Environments, Terraform, Cloudflare Workers and Vector services Terraform · Cloudflare
Observability Dashboards, metrics, health endpoints, alert routing Observability · Runbooks
Security AuthN/Z patterns, secret management, origin protections Security

9. Need Help?

Quick Links