Oracles: How Do You Know It's a Bug?

You're testing a feature and something looks... off. But how do you know it's a bug and not just a design choice you don't understand? This is the oracle problem, and every tester faces it daily.

What's an Oracle?

An oracle is any source of information you use to determine whether the software is behaving correctly. It's the reference you compare actual behavior against.

Think of it like a teacher marking an exam — they need an answer key (oracle) to know which answers are wrong.

The HICCUPPS Heuristic

Michael Bolton created the HICCUPPS mnemonic for common oracle categories. Each letter is a different way to recognize a bug:

H — History: "It didn't do this yesterday." If behavior has changed and nobody changed the code, something broke. Compare current behavior to previous versions.

I — Image: "This doesn't match our brand/standards." The company's image, style guides, and design system are oracles. A neon green button on a corporate banking site is probably a bug even if it functions correctly.

C — Comparable Products: "Other similar apps don't work this way." If every other calendar app lets you click a date to select it, and yours requires a double-click, that's arguably a bug — it violates user expectations.

C — Claims: "The spec says it should do X." Requirements, user stories, API documentation, and marketing materials are all oracles. If the feature page says "supports 10,000 concurrent users" and it crashes at 500, that's a bug.

U — Users' Expectations: "No user would expect this." Even without a spec, you can reason about what users expect. A checkout flow that clears your cart when you click "Back" violates basic user expectations.

P — Product: "This contradicts another part of the same product." If the settings page says your timezone is UTC-5 but the dashboard shows timestamps in UTC, the product contradicts itself.

P — Purpose: "This defeats the purpose of the feature." A search function that returns results but takes 30 seconds defeats its purpose, even if the results are correct.

S — Standards: "This violates an industry standard." WCAG accessibility guidelines, RFC specifications, OWASP security standards — these are external oracles with authority.

Using Oracles in Practice

When you find something suspicious:
1. Identify which oracle you're using (HICCUPPS)
2. State the expected behavior based on that oracle
3. Describe the actual behavior
4. Consider whether the oracle itself might be wrong

Step 4 is crucial. Specs can be outdated. User expectations can be misguided. The oracle is a reference, not gospel. Your job is to flag the discrepancy and let the team decide.

Exercise

You're testing a ride-sharing app and you notice these behaviors. For each one, identify which HICCUPPS oracle(s) would flag it as a potential bug:

1. The estimated arrival time is shown in 24-hour format, but the rest of the app uses 12-hour format
2. The fare after the ride is $2 higher than the estimate shown before booking
3. Tapping "Cancel ride" doesn't ask for confirmation — it just cancels immediately

The key takeaway: A bug is only a bug relative to an oracle. Know your oracles, name them, and use them deliberately.