0.80 Similarity Floor: Why It Silenced a Matching Channel

This article explains why setting a 0.80 similarity floor (a minimum cosine or Jaccard similarity score of 0.80) can effectively silence a semantic matching channel, meaning it stops returning almost any matches. The short answer: a 0.80 threshold is extremely high for most real-world semantic similarity systems, which typically operate with scores between 0.60 and 0.75 for genuinely related texts, so nearly all candidate pairs fall below the cutoff and get rejected.

Formula

The first output, Z-score, measures how many standard deviations the threshold (t) sits above the mean similarity of true matches (s). The formula is:

Z = (t - s) / d

where:

• t = threshold (0.80 in the example)
• s = mean similarity score for pairs that are actually semantically related (e.g., 0.70)
• d = standard deviation of those scores (e.g., 0.05)

The second output, percentage of true matches above threshold, assumes the scores of true matches follow a normal (bell-shaped) distribution. It uses the cumulative distribution function (CDF) of the normal distribution:

Percentage = 100 × (1 - Φ(Z))

where Φ(Z) is the cumulative probability up to Z. The calculator implements this using the error function (erf): Φ(Z) = 0.5 × (1 + erf(Z / √2)).

Worked Example

Using the default values from the calculator (threshold = 0.80, mean = 0.70, std dev = 0.05):

1. Compute the Z-score: Z = (0.80 – 0.70) / 0.05 = 0.10 / 0.05 = 2.0
2. Compute Φ(2.0) = 0.5 × (1 + erf(2.0 / √2)) = 0.5 × (1 + erf(1.414)) ≈ 0.5 × (1 + 0.9545) = 0.97725
3. Percentage above threshold = 100 × (1 – 0.97725) = 2.28%

So with a 0.80 floor, only about 2.3% of true matches would pass, effectively silencing the channel. Even if the mean similarity were 0.75, a 0.80 threshold would still reject about 84% of true matches (Z = 1.0, Φ = 0.8413).

Why 0.80 Is So High in Practice

Semantic similarity scores depend heavily on the embedding model and the text type. For modern sentence transformers (e.g., models trained on natural language inference), typical cosine similarity scores for paraphrases range from 0.60 to 0.75. Scores above 0.80 usually indicate near-duplicate text, not merely similar meaning. For example:

• “The cat sat on the matâ€� vs. “The cat is sitting on the matâ€� might score 0.82 – near duplicates.
• “The cat sat on the matâ€� vs. “A feline rested on the rugâ€� might score 0.65 – semantically similar but different wording.
• “The stock market fell todayâ€� vs. “Equities declined in tradingâ€� might score 0.58 – related but not close enough for a 0.80 floor.

If your channel is meant to match paraphrases or topically related content, a 0.80 floor will keep only the near-duplicates, which are often already caught by simpler lexical methods. The channel becomes “silent� because it returns almost nothing.

How to Choose a Realistic Threshold

Instead of picking an arbitrary number, calibrate the threshold using a labeled sample:

1. Collect 100–200 pairs of texts that you consider true matches (e.g., user queries that should map to the same product).
2. Compute similarity scores for those pairs using your embedding model.
3. Plot the distribution – note the mean and standard deviation.
4. Set the threshold to the 10th or 20th percentile of the true-match distribution, so you retain 80–90% of true matches.
5. Also collect a sample of non-matches and check that your threshold rejects most of them (false positive rate below 5%).

If the mean similarity of your true matches is 0.70 with a standard deviation of 0.05, then the 10th percentile is approximately 0.64 (mean − 1.28 × std). A threshold of 0.64 would retain ~90% of true matches. A threshold of 0.80 would retain only ~2%.

Common Problems with High Thresholds

Silencing is the most obvious symptom, but other issues arise:

• Zero recall: The channel returns nothing, so downstream tasks fail silently – no matches, no errors.
• Bias toward short texts: Short texts often have higher similarity scores because there are fewer tokens to differ; a 0.80 floor may let through only short, generic phrases.
• Model sensitivity: Different embedding models produce different score scales. A threshold that works for one model may be meaningless for another.
• Domain drift: If your corpus changes (e.g., new product categories), the score distribution shifts, and a fixed threshold becomes even more restrictive.

To debug, log the scores of the top 100 candidate pairs. If the highest score is 0.79, you know the floor is above the model’s natural ceiling for your data.

Comparison with Lexical Matching

Lexical matching (e.g., TF-IDF, BM25) uses exact word overlap. A 0.80 lexical similarity (e.g., Jaccard) is also high but more interpretable – it means 80% of words are shared. Semantic similarity is not interpretable the same way; 0.80 does not mean “80% similar meaning.� It’s a relative score. Therefore, applying a lexical-style threshold to semantic scores is a category error.

In practice, many production systems use a dual approach: lexical matching for precision (e.g., BM25 top-100), then semantic re-ranking with a threshold around 0.60–0.70, not 0.80. The 0.80 floor is more suitable for deduplication (finding near-identical records) than for semantic matching.

Alternatives to a Fixed Floor

If you need to filter low-confidence matches, consider:

• Relative threshold: Keep the top K matches per query, regardless of absolute score.
• Percentile-based threshold: Compute the score distribution over a validation set and set the threshold at the 10th percentile of true matches.
• Score normalization: Apply min-max scaling to the scores to make them more interpretable (e.g., map the observed range to 0–1).
• Two-stage filtering: Use a low threshold (e.g., 0.50) to get candidates, then apply a classifier or manual review for final decisions.

These methods avoid the brittleness of an absolute floor and adapt to your data.

Frequently Asked Questions

Why did my channel return zero results after I set the threshold to 0.80?

Because your embedding model likely produces scores below 0.80 for most semantically related pairs. Check the maximum score in your logs – if it’s below 0.80, the floor is above the model’s ceiling. Lower the threshold to the 10th percentile of your true-match scores.

What is a good similarity threshold for semantic search?

There is no universal number. It depends on your model, text type, and task. As a starting point, try 0.65 for paraphrase detection and 0.70 for duplicate detection. Always calibrate with a labeled sample rather than guessing.

Does a higher threshold always mean better precision?

Not necessarily. A very high threshold may only return near-duplicates, which are easy to match but may not be what you need. Precision can be high because you return few results, but recall drops to near zero, making the channel useless. Balance both using a validation set.

How do I know if my similarity scores are normally distributed?

Plot a histogram. If the scores cluster around a peak and taper off on both sides, a normal approximation is reasonable. If the distribution is skewed (e.g., many scores near 0 and a few near 1), use the empirical percentile method instead of the Z-score formula.

Can I use the same threshold for different embedding models?

No. Different models have different score scales. For example, a model trained on contrastive learning may produce scores skewed toward 0.9 for duplicates, while another model may produce 0.7. Always re-calibrate when you change models.

Related

• Water Heater Load Calculation: Estimate Energy Use
• Water Heater ROI Calculator: Estimate Payback and Savings
• Resistance Voltage Calculator: Find Voltage with Ohm’s Law

in t | Threshold (0 to 1) | - | 0.80 | 0 | 1
in s | Mean similarity of true matches | - | 0.70 | 0 | 1
in d | Std dev of true match scores | - | 0.05 | 0 | 1
out z | Z-score of threshold | - | (t - s) / d | 2
out pct | % of true matches above threshold | % | (1 - 0.5 * (1 + erf((t - s) / (d * sqrt(2))))) * 100 | 1