Playwright Reporter: Track Test Runs and Brief Your Assistant

A custom Playwright reporter can record what happened in each test run and create a compact, repeatable summary for your coding assistant. This guide shows how to capture useful results without sharing secrets or flooding the assistant with irrelevant output. You’ll need an existing Playwright project and a Node.js environment that can run its tests.

What You’ll Need

• A working Playwright Test project, including its existing configuration and test scripts.
• Node.js and the project’s package manager, with dependencies installed.
• A coding assistant that can read a text or Markdown report you provide. Keep reports local unless you have checked your organization’s data-sharing rules.
• A small amount of time to run the tests and confirm the report matches the actual results.

Step-by-Step

1. Decide what the report should answer. Start with the practical questions your assistant needs to help: which tests passed or failed, where failures occurred, how long the run took, and what error messages or attachments can help explain a failure. Avoid collecting everything by default. Test output can contain usernames, URLs, tokens, customer data, or other sensitive information.
2. Check the project’s Playwright setup. Find the existing playwright.config.ts, playwright.config.js, or equivalent configuration file. Review its current reporters before changing anything. Playwright supports built-in reporters, and a custom reporter can be used alongside them. Preserve any reporter your team already relies on, such as one that produces a CI artifact.
3. Create a reporter file. Add a file such as reporters/run-summary.ts at the project root. Implement Playwright Test’s reporter interface: the reporter receives lifecycle callbacks, including run start, test end, and run end. In TypeScript, import the relevant reporter types from playwright and use the types provided by the installed version. A reporter is not an ordinary test; Playwright loads it as part of the test runner.
4. Collect a minimal, useful record. In the test-end callback, capture the test title, outcome, duration, and—when relevant—a sanitized error message. The test result includes status and timing information; use the run-end callback to summarize totals and the overall outcome. Do not assume every test has an error, retry, attachment, or project name. Check that a value exists before reading it, and account for retries rather than treating every attempt as a separate final test.
5. Write a readable summary when the run ends. Use the run-end callback to produce a Markdown or plain-text file, for example test-results/assistant-summary.md. Include the run’s final status, counts, failed test names, and short error excerpts. Make file-writing asynchronous where appropriate and handle errors so a report-writing problem is visible rather than silently ignored. Create the output directory if needed, and avoid overwriting a report that another concurrent run is using; consider a timestamped filename for parallel runs.
6. Register the reporter in the configuration. Add the custom reporter to the existing reporter setting rather than accidentally replacing the project’s other reporters. A typical configuration uses an array of reporter entries; a custom entry can point to the reporter file and pass options. Follow the syntax supported by the installed Playwright version and the project’s TypeScript or JavaScript setup. If you are unsure, compare with Playwright’s reporter documentation for that version.
7. Run a small verification first. Run one known test or a focused test file using the project’s usual command. Confirm that Playwright loads the reporter, the summary is created, and its totals and statuses agree with the runner output. Then run a test that fails intentionally in a temporary or controlled test file, verify that the summary records the failure, and remove that test. Do not commit a deliberately failing test by mistake.
8. Make the report easy to hand off. Tell your coding assistant what you want it to do, then paste the summary or point it to the file if the assistant can access local project files. Include the relevant test file or code diff when useful. Ask for diagnosis and a proposed change, not an automatic broad rewrite. For example: “Review this Playwright run summary. Identify the likely cause of each failure, distinguish product defects from test instability, and suggest the smallest change. Do not modify files yet.”
9. Review the assistant’s reasoning and verify changes. Check its claims against the source code, full error output, and test behavior. A summary can omit context, and an assistant may infer causes that the evidence does not establish. After any change, rerun the affected tests and then the wider relevant suite. Keep the summary as a debugging aid, not proof that the application is correct.

Common Mistakes

Replacing built-in reporters: Setting a custom reporter can unintentionally remove the existing list, HTML, or CI reporter. Add it to the existing configuration and confirm every required output still appears.

Recording only the final count: A total such as “three failures” gives an assistant little to investigate. Include test titles and concise error details, but link or direct the reader to full output when the summary is not enough.

Ignoring retries and skipped tests: A retried test may fail once and then pass. Report the final outcome and, when helpful, note that retries occurred. Distinguish skipped or timed-out tests from ordinary failures rather than grouping all non-passing results together.

Exposing sensitive data: Error text, page URLs, screenshots, traces, and attachments may contain private information. Inspect what your reporter writes, redact where possible, and follow project policy before sending anything to an external service.

Assuming the report itself is infallible: A reporter can fail to load, write to the wrong directory, or omit data. Compare it with Playwright’s own output and treat discrepancies as a reporting bug to fix.

Tips

• Keep the summary brief: counts, failed test names, a small error excerpt, and the report location are usually enough to start.
• Include the command, browser project, and environment when they affect reproduction, but do not include secrets or unnecessary machine details.
• Use structured output such as JSON if a script will consume the report; use Markdown when a person or coding assistant will read it directly.
• Store generated summaries in an appropriate ignored or artifact directory. Decide explicitly whether CI should retain them and for how long.
• Give the assistant the test output plus the relevant implementation or test code. Ask it to separate observed facts from hypotheses.

Troubleshooting

The reporter does not load: Check the file path, export style, and configuration syntax. Look at the first Playwright startup error; a TypeScript module mismatch or import error may be the real cause. Confirm that the reporter file is compatible with how the project runs tests.

No summary appears: Verify that the run reaches the callback that writes the file, that the destination directory exists or is created, and that the process can write there. Check whether the configured output path is relative to a different working directory than expected.

Counts do not match: Determine whether the report counts test cases, retries, or final outcomes. Compare it with the runner’s summary and adjust the aggregation rules to match the question the report is meant to answer.

The assistant gives a vague or incorrect diagnosis: Provide the full relevant error, the failing test, and nearby application code. Ask it to cite the exact evidence behind each suggestion. If the summary clipped important details, improve the excerpt or share the full Playwright output securely.

Reports collide in CI: Parallel workers or jobs may write to the same filename. Use a unique path per run or job, and have CI collect the resulting artifacts after the run instead of letting separate processes overwrite one shared file.

Frequently Asked Questions

Does a custom reporter change how tests run?
A reporter is intended to observe and report the run. It should not change test behavior; avoid adding application actions or assertions to reporter callbacks.

Can I use a custom reporter with the HTML reporter?
Yes. Configure both if you need both outputs, and preserve the project’s existing reporter entries when adding the custom one.

Should I send the full Playwright report to my coding assistant?
Not automatically. Start with a sanitized summary and provide full logs or attachments only when needed and permitted by your data-handling rules.

What should the summary include?
At minimum, include the final run result, useful counts, failed test names, and concise error details. Add duration, project, or retry information when it helps reproduce or interpret the run.

Can the reporter fix failing tests automatically?
No. A reporter records information. An assistant may suggest changes, but a person should review them and rerun the relevant tests.

How do I know the reporter works?
Compare its output with Playwright’s normal run summary, and verify both a passing run and a controlled failure. Also check that your other configured reporters still produce their expected output.

Related

• Can Satellites Spot Ocean Plastic? Reality Check
• Logical vs Primary: Key Differences Explained
• OnePlus 13s vs Oppo Reno14 Pro: Which Fits Your Priorities?