Integration Testing for APIs, Databases & External Services

Integration testing verifies that your API, database, and external services work correctly when combined. This guide walks you through setting up and running integration tests that catch real-world issues like broken SQL queries, mismatched API contracts, and timeouts. You’ll learn how to structure tests, manage test data, and mock external dependencies without losing confidence in the system.

What You’ll Need

• A running instance of your application (or a test environment that mirrors production)
• Access to a test database (separate from production, ideally a fresh copy or fixture)
• Credentials for any external services you’ll test against (or a mock server)
• A testing framework appropriate for your language (e.g., JUnit, pytest, Jest, Go’s testing package)
• A way to run your API locally or in a container (Docker is common)
• Version control to store test code and configuration

Step-by-Step

1. Identify the integration points. List the critical paths where your API talks to the database or external services. Prioritize the ones that are most likely to break: complex queries, data transformations, third-party API calls, and event processing.
2. Set up a dedicated test environment. Create a separate database instance (e.g., app_test). Use migrations or schema scripts to create the same structure as production. Configure your API to point to this environment via environment variables or a config file.
3. Write a test that hits the real API endpoint. Start with a simple happy-path test: call an endpoint that reads from the database. Assert the response status code, content type, and that the JSON body contains the expected fields.
4. Seed test data. Insert known records into the test database before each test (or before the test suite). Use a fixture loader or raw SQL inserts. Ensure the data covers edge cases: empty tables, duplicate keys, null values, and large payloads.
5. Test database write operations. Call an endpoint that creates or updates data. After the call, query the database directly to verify the change was persisted correctly. Clean up afterward to avoid interference.
6. Test database constraints and transactions. Try to insert invalid data (e.g., a duplicate primary key) and verify the API returns a proper error. Test that a failed operation rolls back correctly—for example, if a multi-step transaction fails midway, the database should be unchanged.
7. Mock external services for most tests. Use a mocking library (e.g., WireMock, Mockito, unittest.mock) to simulate external APIs. This keeps tests fast and deterministic. Configure the mock to return realistic responses and also simulate error conditions (timeouts, 500s).
8. Write a contract test for the external service. For each external API you depend on, create a test that verifies your request format (headers, body, query params) matches what the service expects. Use the service’s published schema or a recorded sample.
9. Add a test that hits the real external service (optional). If you have a sandbox environment, run a few tests against it to verify your assumptions. Mark these as integration or smoke tests so they can be run separately from the fast suite.
10. Test error handling and retries. Simulate an external service failure and verify your API returns a friendly error and handles retries correctly (e.g., exponential backoff).
11. Automate the tests in CI. Configure your CI pipeline to spin up the test database, run migrations, and execute the integration tests. Use a service container for the database and a mock server for external dependencies.
12. Run tests in parallel where possible. If your framework supports parallel execution, isolate test data per test (e.g., unique IDs) to avoid conflicts. This speeds up the suite.

Common Mistakes

• Testing against production data. Never run integration tests on your production database—it can corrupt data and violate compliance.
• Not cleaning up test data. Leftover records cause false failures and make tests depend on each other.
• Mocking too much. If you mock the database, you’re not testing integration. Keep the database real; mock only external services.
• Ignoring test order. Tests should be independent. If test B depends on test A running first, you’ll have flaky tests.
• Hardcoding connection strings. Use environment variables or config files so tests can run in different environments.
• Forgetting to test failure paths. Happy path tests are not enough—test timeouts, invalid input, and service outages.

Tips

• Use a transaction rollback strategy: wrap each test in a transaction and roll it back after the test, so the database stays clean without manual cleanup.
• Use testcontainers (Docker) to spin up a disposable database for each test run—this ensures a clean state.
• Name tests clearly: test_create_user_persists_to_db tells you exactly what’s being verified.
• Keep integration tests separate from unit tests in your build so you can run them on demand or in a slower CI stage.
• Log the exact request and response when a test fails—this will save hours of debugging.

Troubleshooting

Tests pass locally but fail in CI. Check environment differences: database version, timezone, locale, and environment variables. Ensure your test setup runs migrations before tests.

Database connection refused. Verify the database service is running and the connection string is correct. In Docker Compose, ensure the database container is healthy before running tests.

Flaky tests due to timing. External services may be slow. Increase timeouts or use retry logic. For mocks, ensure responses are immediate.

Data conflicts between tests. Use unique identifiers (e.g., UUIDs) for test records, and clean up after each test. If using parallel execution, isolate databases or schemas per worker.

API returns 500 but no logs. Enable debug logging for your API. Check the application logs for the full stack trace. Also, verify the test database schema matches the code’s expectations.

Frequently Asked Questions

Q: Should I test against a real external service or a mock?
A: Use mocks for most tests to keep them fast and reliable. Add a small set of tests against a sandbox service to verify your assumptions, but run them separately.

Q: How do I handle authentication in integration tests?
A: Create a test user and obtain a token (or use a test-only authentication bypass). Never use production credentials.

Q: How often should I run integration tests?
A: Run them on every commit if they’re fast (under a few minutes). If they’re slow, run them on pull requests and before releases.

Q: Can I use an in-memory database like SQLite for testing?
A: Yes, but be aware of differences from your production database (e.g., MySQL vs SQLite). Use the same database engine as production for maximum fidelity.

Q: What if my external service has rate limits?
A: Use mocks for most tests. For the few real-service tests, limit them to a single run per day and use a test account with higher limits.

Q: How do I test database migrations?
A: Write a test that runs all migrations from scratch on a fresh database and then performs a basic query. This catches migration errors early.

Related

• Recover Deleted Safari History on Mac: Full Guide
• Hulu Buried Smartest Sci-Fi Thrillers in Lab?
• Restore PostgreSQL Database: Full Guide