Feature flag cleanup bots should stop automatically deleting flags the moment they detect they are no longer referenced in code, because that action can break running systems, destroy audit trails, and remove flags that are still toggled in production. A bot must pause and hand off to a human whenever a flag is still referenced in a non-code location, when its removal could affect an active release, or when the flag’s history is needed for compliance or debugging. This article explains the specific conditions under which automated cleanup should halt and what a safe handoff process looks like.
Why Cleanup Bots Exist
Feature flags are a powerful tool for gradually rolling out features, running A/B tests, and performing canary releases. But every flag that is left in the codebase adds complexity, cognitive load, and technical debt. Over time, stale flags accumulate, making code harder to read and maintain. Cleanup bots automate the removal of flags that are no longer needed, freeing developers from manual housekeeping.
However, automation is only safe when the bot has complete and accurate information about every place a flag might be used. In practice, that is rarely true. Flags often appear in configuration files, feature flag management systems, third-party services, or even in documentation. A bot that only scans source code will miss these references and may delete a flag that is still actively controlling behavior in production.
When the Bot Must Stop: Critical Scenarios
There are several clear situations where a cleanup bot must halt and escalate to a human. The most important are:
- Non-code references exist: If the flag appears in configuration files (e.g., YAML, JSON, environment variables), databases, or external feature flag services, the bot cannot safely delete it. These references may be used by other services that the bot cannot see.
- Active toggles in the last 7 days: If anyone has changed the flag’s state recently, it is likely still in use. A flag that was toggled yesterday might be part of an ongoing experiment or a temporary emergency switch.
- The flag is part of a release in progress: If the flag is referenced in a release branch or a deployment pipeline, deleting it could break the release process. The bot must wait until the release is complete and stable.
- Compliance or audit requirements: Some industries require a full history of feature flag changes for regulatory reasons. Deleting a flag removes its audit trail, which could be a compliance violation.
- Uncertainty about the flag’s purpose: If the bot cannot determine why the flag was created or what it controls, it should not delete it. A human can check the flag’s description, linked issues, or the original author.
How to Choose Safe Cleanup Criteria
Setting up a cleanup bot requires careful thought about what constitutes “safe to delete.” A common starting point is to require that a flag has:
- Zero references in the source code (after a thorough search).
- Zero toggles in the last 30–90 days (the exact window depends on your release cadence).
- Zero references in configuration files or external systems.
- No linked issues or pull requests that are still open.
- An age older than a minimum threshold, e.g., 30 days, to avoid deleting newly created flags that are not yet referenced.
These criteria should be configurable per project, because a small startup and a large enterprise will have very different risk tolerances. The bot should also allow a “dry run” mode that lists what it would delete without actually deleting anything, so developers can review the list.
Common Problems and Misconceptions
One common misconception is that a flag with zero code references is automatically safe to delete. That is false. The flag might still be referenced in a database, a mobile app that hasn’t been updated, or a third-party API. Another misconception is that cleanup bots are “set and forget” – in reality, they require ongoing monitoring and occasional manual intervention.
A frequent problem is that bots generate false positives because they don’t understand dynamic references. For example, a flag might be passed as a string variable from one function to another, and the bot’s simple grep-like search may miss it. More advanced bots use abstract syntax tree (AST) analysis to find all references, but even that can miss reflection or dynamic code generation.
Another issue is that cleanup bots can delete flags that are still used in feature flag management platforms (like LaunchDarkly, Split, or Flagsmith). The bot removes the code reference, but the flag remains in the platform, causing confusion and potential accidental re-enablement later. The bot should also remove the flag from the platform, but only after confirming it is truly unused.
Costs of Over-Automation
Deleting a flag that is still in use can have serious consequences. At best, it causes a minor bug that is quickly caught. At worst, it can disable a critical feature for all users, trigger a cascading failure, or cause a security vulnerability if the flag was a kill switch for a dangerous feature.
The cost of a false positive is far higher than the cost of a false negative. Leaving a flag in place for a few extra weeks is harmless; deleting a flag that is still needed can lead to emergency rollbacks, incident response, and lost revenue. Therefore, cleanup bots should be biased toward caution, and any borderline case should be escalated to a human.
Alternatives to Full Automation
Instead of fully automated deletion, many teams use a “propose and confirm” workflow. The bot identifies flags that meet the safe criteria and creates a pull request with the removal changes. A human reviews the PR, checks the flag’s history, and merges it. This gives the benefit of automation (no manual searching) while keeping a human in the loop for final approval.
Another alternative is to use a “flag expiration date” feature, where each flag has a mandatory expiration date set at creation time. When the date passes, the bot automatically opens a PR to remove the flag, but only after notifying the owning team. This ensures that flags are reviewed on a regular schedule, rather than waiting for them to become stale.
Some teams also use a “flag health score” that combines multiple signals (age, references, toggles, owner activity) to rank flags by risk. The bot then only auto-deletes flags with a very low risk score, while flagging others for review. This approach balances automation with safety.
Frequently Asked Questions
What is the safest way to configure a cleanup bot?
The safest configuration requires multiple conditions to be met before deletion: zero code references, zero active toggles for at least 30 days, zero configuration references, and no open issues or PRs linked to the flag. Always run in dry-run mode first and have a human review the list.
Can a cleanup bot delete flags that are used in a mobile app?
No, not reliably. Mobile apps may have older versions still in the wild that reference the flag. The bot cannot know if a user is running an outdated version. Therefore, flags used in mobile apps should be kept for a long time after the code reference is removed, or handled manually.
How long should a flag be inactive before it is safe to delete?
There is no universal answer. It depends on your release frequency and how long you keep old versions. A common rule of thumb is 30–90 days of zero toggles and zero references. For high-risk features, longer is better.
What should I do if a bot deletes a flag that was still needed?
Immediately check your version control history to restore the flag. Most cleanup bots work by creating a commit, so you can revert that commit. Then, add the flag back to your feature flag platform and investigate why the bot missed the reference.
Do cleanup bots work with all feature flag platforms?
Most bots are platform-agnostic because they operate on the codebase, not the platform. However, they may not integrate with the platform’s API to remove the flag from the service. You may need to manually delete the flag from the platform after the code removal.
Is it better to use a bot or do manual cleanup?
For large codebases, bots save significant time and catch flags that humans might miss. But for small projects, manual cleanup is often simpler and avoids the risk of automation errors. The best approach is a hybrid: use a bot to identify candidates, but have a human approve each deletion.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.
More cards in this drawer
- Skip Ryobi and Milwaukee? Harbor Freight Tools to Buy
- Funny Subreddits That Make You Laugh
- Why Retro Gaming Is Still Worth Playing Today
- Neptune Facts That Will Blow Your Mind
- Survival Crafting Game Fatigue: Why & How to Fix It
- Sell Your EV Now Before Prices Plummet?