Skip to content

Labeeb SRE Playbook

Mission Control

This playbook is the single source of truth for operating the Labeeb platform. It provides actionable guidance for deployment, monitoring, and troubleshooting to ensure the system runs reliably and at scale. All content is written from a Site Reliability Engineering (SRE) perspective.

The Labeeb platform is a modular, AI-powered news analysis system. It automates the entire lifecycle of content analysis, from ingestion and enrichment to providing evidence-backed insights via a GraphQL API. Our primary design principle is operational excellence.

Introduction: What is the Goal of Labeeb?

In a world overflowing with information, finding reliable and verified news has become a major challenge. The "Labeeb" platform was created to address this problem. It is an intelligent system that automatically gathers news from multiple sources, analyzes it using AI to verify facts and extract key information, and then presents these findings in a clear, evidence-backed manner. The ultimate goal is to empower users (like journalists, researchers, and the public) to quickly understand complex topics and combat misinformation.

Platform Overview

What you can do from here

This platform overview sits at the top of the doc tree. It links directly to the on-call actions and the data contracts you will need in an incident.

sequenceDiagram
    autonumber
    participant SCR as Scraper
    participant API as API
    participant OS as OpenSearch
    participant AI as AI-Box
    SCR->>API: POST /ingest (batch)
    API->>OS: index
    API->>AI: rerank/enrich (async)
    API-->>Client: Search results

Degraded path

If AI-Box is slow, serve lexical now and hedge rerank: see API ▸ Runbook.

Core SRE Principles

Our operational philosophy is guided by these core principles:

  • Automation First: All routine operational tasks—deployment, testing, and recovery—should be automated. Human intervention should be reserved for novel problems.
  • Observe Everything: Every service is instrumented with detailed metrics, logging, and tracing to provide deep visibility into system health and performance.
  • Assume Failure: The system is designed with the expectation that components will fail. We build for resilience, with health checks, automated restarts, and graceful degradation.
  • Actionable Alerting: Alerts must be urgent, important, and provide a clear path to resolution. Every alert links directly to a runbook for that scenario.

Core Services

The platform is composed of four primary microservices, each with a distinct operational responsibility.

  • API Service

    The central nervous system. A Laravel application that handles all data persistence, orchestration of AI jobs, and client-facing API requests.

    Go to Service Details

  • Scraper Service

    The data ingestion engine. A profile-driven FastAPI service that fetches and normalizes articles from a wide variety of sources.

    Go to Service Details

  • AI-Box Service

    The intelligence core. A FastAPI service that hosts the AI models for check-worthiness, NER, and semantic search retrieval.

    Go to Service Details

  • Search Service

    The retrieval backbone. An OpenSearch cluster that provides fast, scalable full-text and vector search capabilities.

    Go to Service Details

Glossary: The Services Explained Simply

To understand how the platform works, here’s a simple, non-technical breakdown of the core components:

Term (Service) Simple Meaning Its Role in the Platform
AI-Box "The Brain" This is the intelligent part of the system. It analyzes articles, verifies their claims, and extracts important information like people and places.
API Service "The Conductor" This is the control center that manages all operations. It receives data from the Scraper, sends it to "The Brain" for analysis, and stores it in an organized way.
Scraper Service "The Data Collector" Its job is to browse the internet and various news sources to collect articles and raw content, sending it to the platform for processing.
Search Service "The Librarian" This service is responsible for organizing all the collected and analyzed information, making it possible to search for and find anything incredibly fast.

System Architecture at a Glance

Data flows through the system in a well-defined, observable pipeline. The Scraper ingests raw content, the API orchestrates its processing and storage, the AI-Box enriches it with insights, and OpenSearch makes it instantly searchable.

flowchart LR
    classDef ext fill:#e0f2fe,stroke:#0ea5e9,stroke-width:1px;
    classDef svc fill:#f8fafc,stroke:#64748b,stroke-width:1px;
    classDef store fill:#f0fdf4,stroke:#22c55e,stroke-width:1px;

    subgraph "Data Sources"
        direction LR
        NEWS[External News Websites & Feeds]:::ext
    end

    subgraph "Labeeb Platform"
        SCR[(Scraper)]:::svc
        API[(API)]:::svc
        AIB[(AI-Box)]:::svc
        OS[(OpenSearch)]:::store
        PG[(PostgreSQL)]:::store
    end

    NEWS --> SCR
    SCR -- "Batch Ingest" --> API
    API -- "Store Metadata" --> PG
    API -- "Index for Search" --> OS
    API -- "Dispatch Enrichment Jobs" --> AIB
    AIB -- "Fetch Data for Analysis" --> OS
    AIB -- "Write Enrichments" --> API

See Detailed Architecture Diagrams

Operator's Quickstart

This guide provides the essential commands for running the Labeeb stack locally for development and testing.

Objective: To build and initialize the entire platform from a fresh checkout.

  1. Configure Environment: Create your local .env file from the template. You only need to do this once. 

    cp .env.example .env

  2. Build & Start Services: Build the Docker images and start all services in the background. 

    docker compose up -d --build

  3. Initialize Database: Run the Laravel database migrations to create the required tables. 

    docker compose exec api php artisan migrate

  4. Bootstrap Search Index: Set up the required OpenSearch indices, pipelines, and templates. 

    bash tools/search/install.sh

  5. Verify Operation: Run the end-to-end smoke test to confirm all services are working correctly. 

    bash tools/search/smoke.sh

Objective: To work on the platform day-to-day.

  1. Pull Latest Changes: 

    git pull origin main

  2. Start Services: Start all services in the background. A rebuild is only needed if Dockerfile or dependencies change. 

    docker compose up -d

  3. Run Local Tests: Before committing, run the local test suites for the service you are working on. 

    # Example for the scraper service
cd scraper && pytest

  4. View Logs: Tail the logs for a specific service to debug issues. 

    docker compose logs -f api

Key Playbooks

This documentation is organized into actionable guides to help you operate the system effectively.