CI Green But Tests Failed: Optional Gate Explained

When a CI pipeline shows green while six tests failed on the main branch, it usually means the failed tests are in a separate, non-blocking job or stage — an “optional gate” that runs but does not affect the overall pipeline status. This happens when the pipeline is configured so that certain checks are informational only, or when the failure occurs in a job that is explicitly marked as allowed to fail.

What an Optional Gate Is

In continuous integration, a “gate” is any automated check that must pass before a change is merged or deployed. A required gate blocks the pipeline if it fails; an optional gate does not. Optional gates are common in large codebases where some tests are slow, flaky, or cover non-critical features. The pipeline reports the overall status as green because the required jobs passed, even though optional jobs failed.

For example, a project might have a required job that runs unit tests in under 10 minutes, and an optional job that runs end-to-end tests taking 40 minutes. If the end-to-end job fails, the pipeline may still show green because the team decided those tests are not blocking for every commit.

How CI Systems Represent Optional Jobs

Most CI platforms (like Jenkins, GitLab CI, GitHub Actions, and CircleCI) have a way to mark a job as non-blocking:

• GitLab CI: Use allow_failure: true on a job. The pipeline stays green, but the job is marked with a warning icon.
• GitHub Actions: A job can be marked as continue-on-error: true. The overall check result is still “success” even if that job fails.
• Jenkins: Use the catchError step or set the job as “unstable” rather than “failed”.
• CircleCI: Use the when: always condition and ignore the exit code, or use the API to set the job status to “success” regardless.

In all these cases, the pipeline status is green because the failure is explicitly excluded from the final verdict.

Why Teams Use Optional Gates

Optional gates are a pragmatic choice, but they come with trade-offs. Common reasons include:

• Flaky tests: Tests that intermittently fail due to timing, network, or environment issues. Marking them optional prevents blocking development while still collecting data.
• Slow tests: Long-running tests (e.g., full browser suites) that would slow down every merge if required.
• Non-critical coverage: Tests for features that are not part of the core product, or that are known to be incomplete.
• Gradual adoption: When introducing a new test suite, teams may start with optional to see if it is stable enough to later make required.

However, optional gates can hide real regressions. If a test is consistently failing, it should be fixed or removed, not left as a permanent “yellow” status.

How to Check Which Tests Failed

If you see a green pipeline but suspect failures, look at the job list in the CI interface. Failed optional jobs are usually shown with a yellow or orange icon, or with a note like “allowed to fail”. Click into the job to see the exact test names and failure logs. In GitLab, the pipeline page shows a warning triangle next to the job. In GitHub Actions, the check run will show as “neutral” or “success with annotations”.

You can also check the pipeline’s YAML or configuration file to see which jobs have allow_failure or continue-on-error set. That tells you immediately which tests are non-blocking.

Common Problems and Misconceptions

A frequent misconception is that “green” means “all tests passed”. In reality, green only means “all required checks passed”. Optional failures are not shown in the summary status, so they are easy to miss. Another issue is that optional gates can accumulate: if a test is flaky, it may be marked optional, but then it never gets fixed because it doesn’t block anyone. Over time, the optional suite becomes a graveyard of broken tests.

Another problem is that some developers rely on the green status and merge code that actually breaks an optional test, which later causes issues when the test is promoted to required. It is important to review optional failures regularly, even if they don’t block the merge.

How to Decide If a Gate Should Be Optional

Use optional gates sparingly. A good rule of thumb is:

• If a test is flaky, fix the flakiness first (e.g., add retries, fix race conditions) rather than marking it optional.
• If a test is slow, consider splitting it into a separate pipeline that runs on a schedule or on merge requests, but still make it required before merging to main.
• If a test is for a non-critical feature, ask whether the test should exist at all. If it is not important enough to block, it may be better to delete it.

When you do use optional gates, set a clear policy: they must be reviewed weekly, and any test that fails for more than a few days must be fixed or removed.

Alternatives to Optional Gates

Instead of marking tests as optional, consider these alternatives:

• Retry logic: Automatically re-run flaky tests once or twice before marking them as failed.
• Separate pipelines: Run slow tests in a nightly pipeline that is not part of the merge check, but still report results to the team.
• Quarantine: Move flaky tests to a separate “quarantine” suite that is not run in CI, but is tracked in an issue tracker.
• Required with warnings: Make the test required but allow the pipeline to show “unstable” instead of “failed”. This still blocks merge, but gives more nuance.

Frequently Asked Questions

Why does the pipeline show green if a test failed?

Because the failed test is in a job that is configured as optional. The CI system only reflects the status of required jobs in the overall pipeline result. Optional jobs run but do not affect the final status.

How do I find which tests failed if the pipeline is green?

Open the pipeline details page and look for jobs with a warning icon or a status like “allowed to fail”. Click on that job to see the test output. You can also search the configuration file for allow_failure or continue-on-error.

Should I merge code if an optional test fails?

Technically you can, but it is risky. The failure may indicate a real bug that is not covered by required tests. Investigate the failure first. If it is flaky, re-run the job. If it is a real regression, fix it before merging.

What is the difference between “unstable” and “failed” in CI?

“Unstable” usually means that some tests failed but the build itself succeeded. “Failed” means the build could not complete or a critical step failed. Optional gates often result in a status of “unstable” or “neutral” rather than “failed”.

Can I make an optional gate required later?

Yes. Remove the allow_failure or continue-on-error flag from the job. But first ensure the tests are stable and fast enough to be part of the required set. Otherwise, you will start blocking merges with flaky failures.

Related

• Project Zomboid: How to Host a Private MP Server
• Jailbreak Events Schedule & Rotation Guide
• Survive the Killer Codes? What They Are