أفضل ممارسات RAG للمحتوى العربي في المؤسسات

Pipeline كامل

PDF/Word → OCR (if scanned) → Clean → Normalize → Chunk → Embed
                                              ↓
User query → Normalize → Hybrid retrieve → Rerank → LLM + cite

1. OCR & ingestion

Arabic OCR errors propagate forever in vector space.

Must-do:

• 300 DPI minimum scans
• deskew, denoise
• human review sample 5% of pages
• store ocr_confidence per page — block low-confidence from auto-index

Deep dive: /learn/arabic-nlp/ocr-arabic-documents-best-practices

2. Text normalization

Step	Example
Unicode NFC	consistent encoding
Remove tatweel (ـ)	search consistency
Normalize alef (أ إ آ → ا)	optional, document both
Normalize ya/alif maqsura	ى → ي optional
Strip diacritics for index	َ ُ ِ
Preserve for display layer	original chunk text

Rule: same normalization function for index and query.

3. Chunking strategies

Fixed token (baseline)

• 400 tokens, overlap 80
• Simple, works for uniform docs

Structure-aware (preferred)

• Split on ##, numbered articles (المادة 5), tables as atomic units
• Attach metadata: section_title, article_id

Parent-child

• Small chunks for retrieval
• Parent doc section for LLM context
• LangChain / LlamaIndex patterns

4. Retrieval: hybrid

score = α * BM25(q, doc) + (1-α) * cosine(embed(q), embed(doc))

Why BM25 for Arabic? exact match on rare terms, product names, legal citations.

Start α=0.3–0.5; tune on golden set.

5. Reranking

Retrieve top-20 → cross-encoder (e.g. bge-reranker) → top-5 to LLM

Cost: +50–100ms latency — worth it for accuracy-critical (legal, gov).

6. Generation guardrails

Prompt template:

Answer ONLY from context below in Arabic MSA.
Cite [doc_id:section] after each claim.
If insufficient context, say: لا توجد معلومة كافية في المصادر.
Context:
{chunks}
Question: {query}

7. Evaluation loop

Metric	Target
Recall@5	> 0.85
Faithfulness (LLM-judge or human)	> 0.90
Citation accuracy	> 0.95
P95 latency	< 5s

Weekly: add failed queries to golden set → iterate.

Common failure modes

Symptom	Cause	Fix
wrong law version	no version metadata	effective_date filter
English answer	weak prompt	enforce Arabic
duplicate chunks	bad chunking	dedupe hash
missed dialect query	MSA index only	query expansion / dialect synonyms

Tooling

• LlamaIndex / LangChain: orchestration
• Elasticsearch / OpenSearch: hybrid native
• Qdrant / Milvus / pgvector: vector store
• Unstructured.io: parsing (test Arabic)

Pilot timeline

Week	Deliverable
1	100 docs ingested + normalized
2	baseline BM25 + embed A/B
3	hybrid + rerank + 30 golden Q
4	UAT 20 users

المصادر

• Lewis et al., RAG (2020)
• Thakur et al., «BEIR» benchmark — retrieval evaluation
• Arabic NLP survey papers (ACL Anthology)
• InjazAI OCR + RAG field experience (MENA enterprise)