keepcalculator

06 Numbers and Powers

Bug Report Metrics: What to Measure

Card No. 06·80912 · Updated · Standard formula, shown below

When you file a bug report, vague phrases like “it’s slow” or “it crashes a lot” don’t give developers enough to act on. This guide shows you how to document exactly what your bug report measured, so the person reading it can reproduce the issue, verify the fix, and understand the impact. You’ll learn what data to collect, how to record it, and how to present it clearly.

What You’ll Need

  • The software or app where the bug occurs (version number, if available)
  • A way to measure time (stopwatch, phone timer, or the app’s own performance tools)
  • A way to capture screenshots or screen recordings (built-in tools are fine)
  • Access to system information (OS version, hardware specs, available memory)
  • A text editor or note-taking app to draft the report

Step-by-Step

  1. Identify the exact behavior you’re measuring. Instead of “the app is slow,” decide what specific action is slow. For example, “time from clicking ‘Save’ to seeing the confirmation message” or “time from opening the app to the login screen appearing.” Write this as a clear, single sentence.
  2. Set up a repeatable test scenario. Define the starting state (e.g., “fresh install,” “logged in, no projects open”) and the exact steps to trigger the behavior. Write them down in order. The goal is that anyone can follow your steps and get the same result.
  3. Measure the metric multiple times. Run the scenario at least three times to get a range, not just one number. Note the environment each time: what else was running, whether you were on Wi-Fi or ethernet, and the time of day if relevant.
  4. Record the raw values. Write down each measurement with its unit (seconds, MB, frames per second, etc.). Do not average yet—keep the raw data visible. For example: “Run 1: 4.2s, Run 2: 4.5s, Run 3: 4.1s.”
  5. Calculate a simple summary. Give the mean (average) and the range (min–max). For the example above: “Average 4.27s, range 4.1–4.5s.” This gives developers a sense of typical performance and variability.
  6. Capture supporting evidence. Take a screenshot or screen recording that shows the behavior. If it’s a performance issue, include a screenshot of the task manager or performance monitor if possible. For crashes, capture the error message or crash log.
  7. Note the environment in detail. Include your operating system version, device model, available RAM, storage space, and the app version. If the bug is network-related, note your connection type and speed test results.
  8. Compare to a baseline if possible. If you have a previous version of the app or a different device, measure the same scenario there and include those numbers. This helps identify whether the issue is a regression or hardware-specific.
  9. Write the report with the metric first. Start with a one-line summary like: “Saving a project takes 4.1–4.5 seconds (average 4.3s) on Windows 11 with 8GB RAM, versus 1.8s on the previous version.” Then add the steps and evidence.
  10. State what you did not measure. Be explicit about limits: “I did not test on a low-end device” or “I did not measure CPU usage.” This prevents developers from assuming your data covers more than it does.

Common Mistakes

  • Mixing units. Saying “it took 2 seconds” when you actually measured 2 minutes. Always write the unit (s, ms, min) next to every number.
  • Reporting only the best or worst run. This misrepresents typical behavior. Always give the range or all runs.
  • Ignoring the environment. A bug that appears only on Wi-Fi vs. ethernet, or only with low battery, will be missed if you don’t record those details.
  • Measuring the wrong thing. For example, measuring page load time but including the time you spent reading the page. Define the start and end points precisely.
  • Not stating the software version. A bug might be fixed in a newer build; without the version, your measurement is useless.

Tips

  • Use a screen recording with a visible timer to make your measurements verifiable.
  • If the app has built-in performance metrics (e.g., FPS counter, network tab), use those instead of guessing.
  • Keep a template for your reports so you don’t forget fields. A simple checklist works.
  • When possible, measure on the same hardware and network to reduce variables.
  • If the bug is intermittent, note how many times it occurred out of how many attempts (e.g., “crashed 2 out of 5 tries”).

Troubleshooting

If your measurements are wildly inconsistent: Check for background processes or network fluctuations. Run the test a few more times and note any anomalies. It may be that the bug is not a simple performance issue but something that depends on state.

If you can’t reproduce the bug: Try different starting states (fresh login, after using the app for 10 minutes, with a large file open). Document what you tried and what you saw. Sometimes the bug only appears under specific conditions.

If you don’t have access to a second device for baseline: Use the same device but measure before and after a change (e.g., clearing cache, updating the app). That still gives useful comparative data.

If the developer asks for more data: Be prepared to run additional tests. Ask for specific instructions on what to measure and how, so you collect the right data the first time.

Frequently Asked Questions

Q: How many times should I repeat a measurement?
A: At least three, but five is better if the bug is intermittent. More runs give a clearer picture of variability.

Q: What if I don’t have a stopwatch?
A: Use your phone’s timer, or a free online stopwatch. Screen recording with a timestamp is even better because it’s verifiable.

Q: Should I include the raw data or just the average?
A: Include both. The raw data lets developers see the distribution; the average gives a quick summary.

Q: What if the bug is a crash, not a performance issue?
A: Measure the frequency (e.g., “crashes every time I click X”) and the time from action to crash. Also capture the crash log and any error codes.

Q: Can I measure memory usage or CPU usage without special tools?
A: Yes, most operating systems have built-in task managers or activity monitors. You can screenshot those.

Q: What if I don’t know the app version?
A: Look in the app’s settings or about page. If you can’t find it, say “version unknown” and describe the build date if visible. That’s better than leaving it blank.

Related

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.

More cards in this drawer

The whole Numbers and Powers drawer →