Node.js Kill Switch: Handle Flag Failures

This guide explains how a Node.js kill switch should behave when a feature flag fails, ensuring your application fails safely without crashing or exposing incomplete features. You’ll learn the core principles, step-by-step implementation, common pitfalls, and troubleshooting techniques.

What You’ll Need

• Node.js installed (version 14 or later recommended)
• A basic understanding of async/await and error handling in Node.js
• Access to your feature flag service (e.g., LaunchDarkly, Flagsmith, or a custom flag server)
• Your application’s logging system (e.g., Winston, Pino, or console)

Step-by-Step

1. Define the kill switch behavior. Decide what should happen when a flag fails. Common options: use a default value, return a 503, or throw an error. For most cases, fail closed (disable the feature) is safest, but for non-critical features, fail open may be acceptable. Document this policy clearly.
2. Create a flag evaluation wrapper. Write a function that wraps your flag service call. It should accept a flag key, a default value, and an optional fallback behavior. This wrapper centralizes error handling and makes it easy to change policy later.
3. Implement timeout handling. Add a timeout (e.g., 100ms) to the flag request. If the service doesn’t respond in time, treat it as a failure. Use Promise.race or an AbortController to enforce the timeout.
4. Handle network errors. Catch network exceptions (ECONNREFUSED, ETIMEDOUT, etc.) and log them. Then apply your fail policy. For example, return the default value and log a warning.
5. Handle malformed responses. Validate the response shape. If the flag value is not a boolean or string as expected, treat it as a failure. Use a simple schema check (e.g., typeof value === 'boolean').
6. Implement a circuit breaker. If the flag service fails repeatedly (e.g., 5 times in 30 seconds), open the circuit and stop calling it for a cooldown period (e.g., 60 seconds). This prevents hammering a down service. Use a library like opossum or implement a simple counter.
7. Log everything. Log every flag failure with context: flag key, error message, fallback used, and timestamp. This is crucial for debugging and monitoring.
8. Test the kill switch. Write unit tests that simulate flag service failures (network errors, timeouts, malformed data). Verify that your wrapper returns the correct fallback and logs appropriately.
9. Monitor and alert. Set up alerts on the failure logs (e.g., using Sentry or a log-based alert). This ensures you know when the kill switch is active.
10. Document the behavior. Add comments in code and a README section explaining what happens when a flag fails, so future developers don’t accidentally change the policy.

Common Mistakes

• Ignoring the error and throwing. If you let the error propagate, your entire request may fail. Always catch and apply a fallback.
• Using a synchronous call. Flag services are async. Blocking the event loop with a sync call defeats the purpose and can crash the app.
• Not setting a timeout. Without a timeout, a hung flag service can stall your request indefinitely.
• Failing open for security-critical features. If the flag gates a payment or admin feature, failing open could expose sensitive functionality. Always fail closed for such flags.
• Not testing failure scenarios. If you only test the happy path, you’ll discover bugs in production during an outage.

Tips

• Use a default value that is safe. For a new feature, the default should be the old behavior (off). For a migration, it might be the new behavior.
• Cache flag values. If a flag is used frequently, cache the result for a short TTL (e.g., 30 seconds) to reduce load on the flag service and improve resilience.
• Make the kill switch configurable. Allow operators to override the fail policy via environment variables (e.g., KILL_SWITCH_MODE=strict).
• Use a structured logger. Include a flagKey and fallback field in logs to make searching easier.
• Consider using a library. Libraries like flagsmith-nodejs or launchdarkly-node-server-sdk often have built-in error handling. Read their docs and use their recommended patterns.

Troubleshooting

Flag service is down and your app is crashing. Check that your wrapper catches all errors. If you’re using a library, ensure you’re not throwing inside the callback. Add a global process.on('unhandledRejection') handler as a last resort, but fix the root cause.

Fallback value is not being used. Verify that your wrapper is actually catching the error. Add a console.log in the catch block to confirm. Also check if the error is happening before the wrapper (e.g., in the flag service initialization).

Circuit breaker is not opening. Ensure your failure counter increments on every failure and resets on success. Check your cooldown logic—maybe it’s resetting too early.

Timeout is not firing. If using Promise.race, make sure the timeout promise rejects. Use AbortController to actually cancel the request, otherwise the timeout may be ignored.

Logs are flooding. Implement log throttling or only log the first failure per minute per flag. Use a simple rate limiter in your logging wrapper.

Frequently Asked Questions

Should I fail open or fail closed?

It depends on the risk. For non-critical UI tweaks, fail open (show new feature) is acceptable. For security or billing features, fail closed (disable) is mandatory.

What if the flag service returns a wrong type?

Treat it as a failure. Validate the type and use the default if it doesn’t match.

How long should the timeout be?

Typically 50-200ms. If your flag service is slow, increase it, but never exceed 1 second to keep your app responsive.

Can I use a kill switch for non-flag features?

Yes, the same pattern applies to any external dependency (database, API). The key is to have a fallback and a circuit breaker.

How do I test the kill switch locally?

Mock your flag service to throw errors or time out. Write unit tests that simulate these conditions and assert the fallback behavior.

What if my flag service has a built-in cache?

That’s fine. The kill switch should handle cases where the cache is empty and the service is unreachable.

Related

• Resistance Voltage Calculator: Find Voltage with Ohm’s Law
• US Water Heater Calculation: Estimate Tank Size
• PPM Error Calculator for Mass Spectrometry