Test Environments and Test Data
You wrote a perfect test. It passes on your machine. It fails in staging. It works again in production. What happened? Almost always, the answer is the environment or the data.
What's a Test Environment?
A test environment is a copy of the system you're testing — but not the real one. Typical environments:
| Environment | Purpose | Data |
|---|---|---|
| Local | Developer's machine | Seed data, usually small |
| Dev/Integration | Shared dev testing | Test data, frequently reset |
| Staging/Pre-prod | Near-production testing | Production-like data (sanitized) |
| Production | The real thing | Real user data |
Each environment might have different configurations, different database versions, different third-party service connections. Those differences cause bugs that are invisible in other environments.
Environment Differences That Cause Bugs
- Database content: Your test works with 10 users. Production has 100,000. Performance issues hide in small datasets.
- Configuration: Dev has debug mode on, production has it off. Error messages display differently.
- Third-party services: Dev uses a mock payment gateway. Production uses the real one. They behave differently.
- Infrastructure: Dev is one server. Production is load-balanced across three. Session handling bugs appear only at scale.
Test Data Strategies
Good test data is essential. Bad test data leads to false confidence.
Realistic data: Use data that resembles production. Names with special characters (O'Brien, José), international addresses, maximum-length fields. The bugs live in the edges.
Edge case data: Empty strings, null values, Unicode characters, extremely long inputs, negative numbers, dates far in the future or past. These are the inputs your automated happy-path tests won't cover.
State-specific data: Need to test a "returning customer" flow? You need a user who has already placed an order. Need to test "account locked after 3 failed attempts"? You need an account with 2 failed attempts. Test data setup is often the hardest part of testing.
The Golden Rule
Never test on production data without sanitizing it first. Real names, real emails, real credit card numbers — using these in test environments is a security and privacy risk. Sanitize by replacing sensitive fields with fake but realistic data.
The key takeaway: Your test is only as reliable as the environment it runs in. Know your environment, know your data, and document the differences.
Your test passes on your local machine but fails in staging. Which is the MOST likely cause?
Environment differences are the most common cause of tests that pass locally but fail elsewhere. Different database content, configurations (debug mode vs. production), third-party service connections, and infrastructure (single server vs. load-balanced) all cause environment-specific failures.
Why should you never test with unsanitized production data?
Click to reveal answer
Production data contains real names, emails, credit card numbers, and other sensitive information. Using it in test environments is a security and privacy risk that could violate regulations (GDPR, HIPAA). Always sanitize by replacing sensitive fields with fake but realistic data.
Click to flip back
Put these test environments in order from least production-like to most production-like.