BM25
BM25 is a lexical ranking function that scores documents based on keyword frequency and term distribution. It is the standard keyword search algorithm in most RAG systems and search engines.
Definition
BM25 assigns higher scores to documents where query keywords appear frequently but not excessively, adjusting for document length to avoid penalizing longer content. It models the probability that a document is relevant given the query terms. Despite its age, BM25 remains the baseline for keyword retrieval in production systems due to its robustness and efficiency.
In hybrid search, BM25 runs alongside semantic search. Documents with exact keyword matches score high in BM25, even if their embeddings are less semantically similar. This ensures that technical specifications, product names, and specific terminology still retrieve relevant results.
Why it matters for AI visibility
BM25 rewards clear use of industry terms and product names in your content. While semantic search matters for concept matching, BM25 ensures that when someone specifically asks about your category or uses your terminology, your content is found. Brands that own clear keyword territory in their space rank well on BM25 signals.
Related terms
Hybrid Search
Hybrid Search combines vector search with keyword search, leveraging both semantic similarity and exact term matching to retrieve relevant documents. It provides more robust results than either method alone.
AIVector Search
Vector Search is the technique of finding semantically similar documents by comparing their embeddings. It powers retrieval in AI search engines and RAG systems.
AIRetrieval-Augmented Generation (RAG)
RAG is the technique of retrieving relevant documents at answer time, then using them to ground an LLM's response. It enables AI search engines to cite sources while generating answers.
SEOSemantic Search
Semantic Search is the search engine capability to understand the meaning, context, and intent of search queries and content, going far beyond simple keyword matching. It interprets synonyms, recognizes relationships between concepts, disambiguates ambiguous terms, and matches user intent to relevant pages. Semantic understanding powered by natural language processing and knowledge graphs is fundamental to modern search ranking.