keepcalculator

06 Numbers and Powers

The Check That Never Failed: Proving It Can Catch a Fault

Card No. 06·81091 · Updated · Standard formula, shown below

Confidence check catches a fault 98.0 %
Expected failures in next n runs
0.0196
Result
Times check passed
50
You entered
You enter
Times check passed
You get
Confidence check catches a fault, Expected failures in next n runs
Rounding
1 dp · 2 dp

This calculator works out the probability that a single check—one that has never failed in the past—will actually catch a fault when it is next used. You enter the number of times the check has been performed without a miss, and the calculator returns the statistical confidence that the check is reliable, based on the rule of succession.

Formula

The core formula is the rule of succession, attributed to Laplace. It estimates the probability that the next trial will succeed, given that all previous n trials have succeeded:

P(next success) = (n + 1) / (n + 2)

For the probability of catching a fault (the next trial being a failure of the check, which is what we want), it is the complement:

P(catches a fault) = 1 − (n + 1) / (n + 2) = 1 / (n + 2)

Where:

  • n = number of times the check has been performed without a miss (a positive integer).
  • P(catches a fault) = estimated probability that the check will fail on the next use, expressed as a percentage.

The calculator above uses a slightly simplified version, n / (n + 1), which gives a very close approximation for large n and is easier to explain. The difference between the two formulas shrinks as n grows; for n=50, the simplified version gives 98.0% while the full version gives 98.1%. The calculator uses the simplified version to keep the arithmetic transparent.

Worked Example

Suppose a quality-control check on a production line has been run 50 times and has never missed a fault. Using the calculator’s default value of n = 50:

  1. Apply the formula: P(catches a fault) = n / (n + 1) = 50 / 51.
  2. Calculate the fraction: 50 ÷ 51 = 0.98039.
  3. Convert to a percentage: 0.98039 × 100 = 98.04%.

The calculator rounds this to 98.0%. The second output, expected failures in the next 50 runs, is 1 / (n + 1) = 1 / 51 = 0.02, meaning statistically you would expect about 0.02 failures—i.e., it is more likely than not that the check will still pass all 50 times.

Why a Perfect Record Does Not Mean Certainty

The intuitive mistake is to think that 50 out of 50 successes means the check is 100% reliable. The rule of succession corrects this by acknowledging that you are estimating an unknown probability from finite data. With zero failures, the best estimate is not 100% but something slightly less, because a single success (or 50 successes) could still be luck.

This matters in real-world testing. For example, if a safety interlock on a machine has never failed in 100 tests, the rule of succession says the probability it fails on the next test is about 1/102, or roughly 1%. That is not zero. If the consequence of failure is severe, a 1% risk may be unacceptable, and more testing or a different design is needed.

The formula assumes the underlying failure rate is constant and that each test is independent. If the check has been improved, or the conditions have changed, the past record is less predictive.

How to Choose the Number of Tests

To use this calculator meaningfully, you need a realistic count of how many times the check has actually been performed. This is not the number of times it could have been run, but the number of times it was run under the same conditions you care about.

  • Count only tests where the check was genuinely applied to a real or simulated fault scenario.
  • Exclude tests that were skipped, aborted, or where the check was not the deciding factor.
  • If the check is run automatically, use the log count.
  • If the check is manual, use a conservative estimate—people often overcount.

If you have a mix of conditions (different operators, different environments), the rule of succession still applies but the estimate is less precise. The formula gives a point estimate; it does not give a confidence interval.

Common Misconceptions and Mistakes

The most common error is treating the output as a guarantee. A 98% confidence is not a promise. The second common error is using the formula when the check has been changed. If you modified the check after the 30th test, the remaining 20 tests are a different check, and you should only count those 20.

Another mistake is ignoring the base rate. If faults are extremely rare, then even a check that catches 99% of faults will have a low positive predictive value. This calculator only estimates the check’s sensitivity (its ability to catch a fault when one is present), not the overall probability that a fault exists.

Finally, the rule of succession assumes the prior probability of failure is uniform. If you have strong prior evidence that the check is either very good or very bad, the estimate will be off. For most practical purposes, however, the rule of succession is a reasonable default.

Alternatives and Related Methods

For small sample sizes, the rule of succession is the standard Bayesian answer. For larger samples, you might use the Wilson score interval to get a confidence interval around the failure rate. With zero failures in n trials, the Wilson upper bound is approximately 3 / n, which is a common rule of thumb (the “rule of three”).

The rule of three states that if you have seen zero failures in n trials, you can be 95% confident that the true failure rate is less than 3/n. For n=50, that upper bound is 6%, which is much wider than the point estimate of 2% from the rule of succession. The rule of three is more conservative and is often used in safety engineering.

If you have observed some failures, you would use a different formula, such as the Laplace point estimate (s+1)/(n+2) where s is the number of successes, or the more common (s)/(n) for the observed rate.

Limitations of the Formula

The rule of succession is a mathematical estimate, not a physical law. It does not account for:

  • Wear and tear: A check that passes 1000 times may fail on the 1001st because a component has worn out. The formula assumes a constant failure rate.
  • Human factors: If the check is performed by a person, fatigue or complacency can change the effective failure rate over time.
  • Changing conditions: If the system being checked is modified, the old test results may no longer apply.

For safety-critical systems, you should not rely solely on this estimate. Combine it with failure mode analysis, redundancy, and periodic re-validation. The formula tells you what the data says, not what you should do.

Frequently Asked Questions

Does a perfect record prove the check is infallible?

No. A perfect record is strong evidence but not proof. The rule of succession shows that with n successes, the estimated failure probability is 1/(n+1). To reach a 99.9% confidence of catching a fault, you would need about 999 consecutive successes. Even then, it is an estimate, not a guarantee.

What is the difference between this and the “rule of three”?

The rule of three is a conservative upper bound: with zero failures in n trials, you can be 95% confident the true failure rate is below 3/n. The rule of succession gives a point estimate of 1/(n+1). For n=100, the rule of three says the failure rate is likely below 3%, while the rule of succession estimates it at about 1%. The rule of three is safer for high-stakes decisions.

How many successful tests do I need to be 99% sure?

Using the rule of succession, you need 1/(n+1) ≤ 0.01, which means n ≥ 99. So 99 consecutive successes give you a 99% estimated confidence. For 99.9% confidence, you need n ≥ 999.

Can I use this for software testing?

Yes, with caution. If a unit test has passed 200 times without failing, you can estimate it will catch a regression on the next run with about 99.5% confidence. However, software changes constantly, so past success on old code does not predict success on new code. The formula works best for stable, repeatable checks.

What if the check has failed once in the past?

This calculator only handles zero failures. If you have one failure in n trials, the standard estimate is (n−1)/n for the success rate, and the failure rate is 1/n. You would use a different formula, not the rule of succession.

Is this the same as the “birthday paradox” or other probability puzzles?

No. The birthday paradox deals with collisions in a finite set. The rule of succession is about estimating an unknown probability from observed successes. They are unrelated.

Related

Formula used
in n | Times check passed | - | 50 | 1 | 100000
out confidence | Confidence check catches a fault | % | (n) / (n + 1) * 100 | 1
out failures | Expected failures in next n runs | - | 1 / (n + 1) | 2

Each line is one input (in, sel, list) or one result (out: label, unit, formula, decimals). How the calculators work

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.

More cards in this drawer

The whole Numbers and Powers drawer →