What a smoke test in software actually verifies
A smoke test answers one question about a build: is it worth testing properly? You are not confirming that the product is correct. You are confirming that the paths nobody can ship without still function at a coarse level, so the deeper suites have something worth running against.
The name comes from hardware testing: you power the board up and see whether it smokes. Software kept the instinct and lost none of the bluntness. If the sign-in page throws, the checkout total comes back empty, or the service refuses to start against its own database, a smoke suite says so in minutes and the build stops there.
Coverage is deliberately shallow and wide. A smoke check asserts that a route renders with its expected heading, that a form submits and shows a confirmation, that an endpoint answers with a known field. It does not assert every field, every permission combination, every rounding rule, or every visual state. That depth belongs to the suites that run after it.
Smoke test, sanity test, regression test: what is the difference
Check | Question it answers | Scope | Usual trigger | What a failure tells you |
|---|---|---|---|---|
Smoke | Is this build worth testing at all? | A handful of critical paths | Every candidate build, right after it is deployed | The build is unusable, so deeper testing would be wasted effort |
Sanity | Does this specific change behave as intended? | One changed area, checked more closely | After a fix or a small change | The change is not ready, even if the rest of the product is fine |
Regression | Did behavior that used to work break? | Broad and deep, feature by feature | Nightly runs and release candidates | A specific behavior that previously worked has broken |
Unit and integration | Do components and their wiring behave? | Code level, per function or service | Every commit | A defect located close to the code that failed |
The boundaries are practical rather than academic. Many teams say "sanity" when they mean "smoke" and the reverse, and the label matters far less than keeping the purpose distinct. The test for where a check belongs is the question it asks. If a check is asking whether one particular change is right, it is sanity checking. If it is asking whether the build is worth anyone's afternoon, it is a smoke check.
What belongs in a good smoke suite
- Chosen by business cost, not code convenience. Start from the paths whose failure would make you cancel a release: sign-in, the primary transaction, the one report the sales team opens every morning, the export a customer runs weekly.
- Small on purpose. A suite that grows with every bug report has quietly become a regression suite. Add a check only when a real incident proves the path is critical.
- Asserted on outcomes a person can see. A rendered heading, a saved record, a confirmation message, an expected response field. Assertions on internal identifiers, DOM paths, or auto-generated class names break when the design changes and tell you nothing about the product.
- Independent and safe to repeat. No check should depend on another check's leftovers, and running the suite twice in a row should produce the same result. Chained steps hide which link in the chain actually failed.
- Fed with deterministic data. Create or reset the data the check needs inside the run, or point it at a dedicated account. Shared test data is the most common reason a smoke suite starts failing on a Monday morning.
- Fast enough that running it is never a decision. In a process that runs on every deploy, minutes are affordable and hours are not.
- Readable on failure. Which check failed, which step it was on, what was expected, and what happened instead.
You can verify a suite in one sitting: deliberately break a core path, run the suite, and watch what it reports. A smoke suite that still passes after you have broken the product is decoration, not a gate. That deliberate break is the completion standard, and it is worth repeating every time someone adds a check.
A starter smoke checklist you can adapt
Area | Example check | Pass condition |
|---|---|---|
App shell | Open the entry page | The page renders with its heading and the expected primary action |
Authentication | Sign in with a known test account | The account lands on its home view with the account name visible |
Core transaction | Create one record with fixed test data | The new record appears in the list with the values that were submitted |
Data read | Open the most-used report or list view | Rows render and no error state is shown |
Data update | Edit the record created above and reload | The edited value persists after the reload |
Integration | Trigger the outbound webhook or email in test mode | The receiving endpoint acknowledges the event |
Access control | Open a restricted route with a standard user | Access is refused and no restricted data is displayed |
The wording matters more than the tooling. Write each row as a sentence a colleague could follow without opening the test code, and keep the expected result on the same line as the step. A checklist that reads clearly is easier to keep honest than a clever abstraction.
Where the smoke check belongs in a CI/CD flow
Put the smoke check after the build is deployed and before the expensive suites. Commit, build, deploy to an environment that behaves like production, smoke, deeper tests, release. That order buys you the cheapest early warning available and the clearest possible stop signal.
- Run it on every deploy candidate, not only on merge. Code that compiles and passes unit tests can still fail to start, and only a running system reveals that.
- Point it at production-like configuration. The same configuration path, the same authentication provider, seeded test accounts, and a database reachable the way production reaches it. A suite that only ever runs against a developer's machine measures nothing about deployment risk.
- Treat a failure as a stop, not a retry. Hold the promotion, tell the owner which check failed, and keep the build out of release until it is green or explicitly waived. A gate that retries itself teaches everyone to ignore it.
- Keep the gate short enough to be tolerated. A gate that takes an hour gets bypassed within a month, and the bypass outlives the reason it was introduced.
- Separate "will not start" from "one feature is broken". A build that cannot boot or cannot authenticate should always block. A broken secondary path may be a decision rather than a hard stop, so agree in advance on which checks hold the pipeline and which only raise a ticket.
- After a production deploy, smoke the live environment read-only. A short pass with dedicated accounts confirms the release is actually serving. Keep it non-destructive, and never let it write to real customer data.
Where smoke suites quietly stop working
- Scope creep. Every incident adds a check until the five-minute gate becomes a forty-minute nightly run that nobody watches.
- Assertions on implementation detail. A renamed button or a new wrapper element turns a healthy product red, and the team learns to re-run instead of reading.
- Coupled checks. One broken login makes twenty results red, and the real failure disappears in the noise.
- Flakiness covered by retries. Retrying until green converts a real intermittent defect into a permanent blind spot.
- Blind spots by design. A green smoke suite never proves the product is correct. It proves the build is worth deeper testing, and it is only as good as the paths someone remembered to include.
The trade-off is the point of the exercise. Smoke testing buys breadth and speed by giving up depth and defect localization: you learn quickly that something essential is broken, not precisely where or why. Every check you add buys coverage at the cost of the speed that made the suite useful, which is why the list deserves the same scrutiny as the code it guards.
Frequently asked questions
How many checks should a smoke suite have?
Enough to cover every path that would make you postpone a release, few enough that one person can read the whole result in a minute. Treat any number you hear as a starting point rather than a standard. The rule that matters is that the list changes only when the business changes.
Should smoke tests run on every commit or every deploy?
Every deploy candidate is the minimum, because that is the moment the build becomes a running system. Running a fast subset on commits is a reasonable extra, but it does not replace the check against a deployed build.
Does a passing smoke test mean the release is safe?
No. It means the build is worth deeper testing. Treat a green result as permission to keep going, never as evidence that the product is correct.
Can you smoke test an API or a mobile app?
Yes. The shape changes, the purpose does not. For an API, the critical paths are authentication, the primary endpoint, and one representative write. For a mobile app, they are install or launch, sign-in, and the main flow.
Who should own the smoke suite?
Someone who can decide what critical means and is accountable when the gate goes red, usually the QA lead or the on-call engineer working with the product owner. Ownership matters more than the tool you pick.
How long should a smoke suite take?
Short enough that nobody negotiates about running it. If people start scheduling the gate instead of triggering it, the suite has drifted into a regression suite and needs splitting.
Start with three checks
Pick the three paths that would stop you shipping today. Write each as a step and an expected result in plain language. Run them against a production-like environment on your next deploy, then break one of them on purpose and confirm the suite catches it. Keep the list at three until a real incident argues for a fourth, and let that argument be the only reason the suite grows.
