Skip to content

Pipelines

Ingest and search pipelines are defined under api/resources/search/pipelines and installed via tools/search/install.sh.

Pipelines-as-Code & Relevance

Query Pipeline (concept)

  • Collectors: BM25 (must), kNN (optional)
  • Combiner: RRF with k=60, rank_constant=10
  • Reranker: optional AI-Box step, 1.5s budget

Request example

{
  "query": {
    "multi_match": {
      "query": "سوريا الهدنة",
      "fields": ["title^3","content"],
      "type": "best_fields"
    }
  },
  "knn": {
    "field": "vector",
    "query_vector": "EMBEDDING_HERE",
    "k": 100,
    "num_candidates": 1000
  }
}

RRF Combiner (pseudo)

def rrf(bm25_rank, knn_rank, k=60, c=10):
    return 1.0/(k + bm25_rank + c) + 1.0/(k + knn_rank + c)

Response shape

{
  "took": 120,
  "hits": [{"_id":"...","_score":12.3,"_source":{}}],
  "degraded": false,
  "reranker": {"enabled": true, "model": "reranker-x", "latency_ms": 900}
}

Synonyms & Dictionaries

  • Maintain per-language synonyms files; reload via _reload_search_analyzers during low-traffic windows.

AB/Routing toggles

  • Feature flags: RERANKER_ENABLED, KNN_ENABLED