keepcalculator

06 Numbers and Powers

Python Search Highlighting: Snippets & Matches

Card No. 06·31477 · Updated · Standard formula, shown below

The calculator on this card is switched off while it is checked. The formula is below, and the worked example shows how to do the sum by hand.

Python search highlighting lets you see exactly where your search terms appear in a document or dataset by visually marking matched text. You provide the search query and the text to search, and the code returns the same text with matches wrapped in highlight markers (like tags) plus optional snippet extracts around each match. This article explains the key facts about snippets and matches, how highlighting algorithms work, and how to implement it in Python.

Formula

The core of search highlighting is not a single formula but a set of operations. For a simple case, the formula is:

highlighted_text = text.replace(query, f"{query}", count)

Where:

  • text – the original string you are searching.
  • query – the term or phrase you want to highlight.
  • count – optional; maximum number of replacements (default is all).

For snippets, the goal is to extract a short window of text around the first (or best) match. A simple snippet formula is:

snippet = text[max(0, pos - radius) : pos + len(query) + radius]

Where:

  • pos – the index of the first character of the match.
  • radius – number of characters to include before and after the match.

These are the building blocks. Real-world libraries use more complex scoring and tokenization, but this is the essence.

Worked Example

Using the default values from the calculator block:

Text: “The quick brown fox jumps over the lazy dog. The dog sleeps.”
Query: “dog”

  1. Find the first occurrence of “dog”: at index 40 (0-based).
  2. Create a snippet with a radius of 15 characters: snippet = text[25:55] → “lazy dog. The dog sl”
  3. Highlight the matches in the snippet: replace “dog” with “dog” → “lazy dog. The dog sl”

The calculator block above simply returns the full text with no highlighting because the expression is just t. In a real implementation, you would use a function to perform the replacement. The default output here is the original text, which is not helpful. For a correct example, you would need a custom function. The block is a placeholder to illustrate the concept.

How Search Highlighting Works

Search highlighting is a two-step process: matching and rendering.

  • Matching – find where the query appears in the text. This can be as simple as a substring search (str.find()) or as advanced as token-based matching with stemming, synonyms, and fuzzy matching.
  • Rendering – wrap the matched text with a marker. In HTML, you use tags. In plain text, you might use asterisks or brackets.

Libraries like whoosh and elasticsearch have built-in highlighters that return fragments with or tags. The key is that the original text is not modified; you only insert markers around matches.

Snippets vs. Full Text

A snippet is a short extract around a match, usually 100–200 characters, shown in search results. Full-text highlighting marks every occurrence in the entire document.

When to use snippets:

  • Search result pages – show a few lines around the match so the user can see context.
  • Long documents – avoid overwhelming the user with the whole text.

When to use full text:

  • Editing tools – highlight all occurrences so the user can jump to each.
  • Code review – show all matches in a file.

The choice depends on the use case. Most search engines use snippets for performance and readability.

Common Pitfalls and Misconceptions

One common mistake is using str.replace() on a case-insensitive basis. Python’s replace() is case-sensitive, so you must use re.sub() with the IGNORECASE flag for case-insensitive highlighting.

Another misconception is that highlighting is just about wrapping text. In reality, you must handle overlapping matches, special characters in the query (like regex metacharacters), and HTML escaping to avoid breaking the page.

For example, if the query is “C++”, a naive replace() will work, but if the query is “a.b”, the dot is a regex wildcard, so you must escape it. Use re.escape(query) to avoid this.

How to Choose a Highlighting Library

You can implement highlighting from scratch, but libraries save time. Consider these factors:

  • Performance – for large corpora, a library like whoosh is optimized.
  • Features – stemming, fuzzy matching, and language support.
  • Output format – HTML, plain text, or custom.

For simple tasks, a few lines of Python are enough. For production search, use a dedicated engine like Elasticsearch, which has built-in highlighting.

Performance Considerations

Highlighting can be slow if you scan the entire text for each query. Use efficient algorithms:

  • Pre-index the text with a search engine to find matches faster.
  • Limit the number of matches highlighted (e.g., first 10).
  • Cache the highlighted result if the text is static.

For a single document, str.find() is fast enough. For millions of documents, you need an inverted index.

Alternatives to Custom Code

If you don’t want to write your own highlighter, use:

  • Whoosh – a pure-Python search engine with highlighting support.
  • Elasticsearch – a distributed search engine with advanced highlighting.
  • Django Haystack – if you use Django, it integrates with search backends.

These tools handle tokenization, stemming, and snippet generation out of the box. They are more robust than a custom script.

Frequently Asked Questions

How do I highlight multiple search terms in Python?

Loop over each term and apply the highlighting sequentially. Be careful with overlapping terms; you can use a regex alternation pattern to highlight all at once.

Can I highlight matches in a PDF or Word document?

Yes, but you need to extract the text first using libraries like PyPDF2 or python-docx. Then apply highlighting to the extracted text. The original formatting may be lost.

What is the difference between a snippet and a match?

A match is the exact occurrence of the query in the text. A snippet is a short window of text around the match, shown to give context.

How do I make highlighting case-insensitive?

Use re.sub() with the re.IGNORECASE flag, or convert both the text and query to lowercase before searching, but then you lose the original case for display.

Is it safe to use str.replace() for highlighting?

Only if the query has no special regex characters and you don’t need case-insensitivity. For robust highlighting, use re.sub() with escaped query.

Related

Formula used
in t | Text to search | - | The quick brown fox jumps over the lazy dog. The dog sleeps. | 0
in q | Search term | - | dog | 0
out snippet | Snippet with highlight | - | t | 0

Each line is one input (in, sel, list) or one result (out: label, unit, formula, decimals). How the calculators work

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.

More cards in this drawer

The whole Numbers and Powers drawer →