# Your Tests Are the Spec Now. > Your tests are the spec now. When AI writes the code, a planning document goes stale the moment the first agent ships. Executable tests do not. Orbyt's 11,372-test suite is the real specification, the part of vibe coding nobody can vibe, and the only reason a vibe-built product survives production. - Canonical URL: https://justinbartak.ai/blog/tests-are-the-spec-vibe-coding - Author: Justin Bartak, Chief AI Officer (https://justinbartak.ai) - Author entity: https://justinbartak.ai/#person - Author profiles: https://www.wikidata.org/wiki/Q139551829, https://orcid.org/0009-0005-2615-3624, https://linkedin.com/in/justinbartak, https://x.com/JustinBartak, https://www.crunchbase.com/person/justin-bartak, https://medium.com/@justinbartak, https://github.com/justinbartak, https://about.me/justinbartak - Published: 2026-08-06 - Updated: 2026-08-12 - Category: Engineering - Keywords: vibe coding tests, test-driven AI development, AI generated code testing, tests as specification, vibe coding to production, AI-native engineering, executable spec, production-ready AI code, agentic coding discipline - Reading time: 8 min read --- When AI writes the code, your tests are the spec. A planning document goes stale the moment the first agent ships. The tests do not. They run. They fail loudly the second reality drifts from what you wrote down. You stop writing a spec and start writing checks. The checks are the only specification the codebase actually honors. **The discipline moved from the spec to the test. That move is the part of vibe coding nobody can vibe.** ## Are your tests the spec now? Yes. You do not write a spec anymore. You write tests, and the tests are the spec the codebase actually obeys. A prose spec is a description of intent. A test is a description of behavior that fails the instant the system stops matching it. Only one of those is enforceable. The expired card gets rejected because a test says it does, not because a document hoped it would. I built [Orbyt](https://www.orbytlabs.ai) solo in 32 days. It is over 425,000 lines and 11,372 tests now. No prose document governs that system. The suite does. I do not own a doc that describes what Orbyt is supposed to do. I own a test suite that proves what it does, on every commit, with no one watching. That is the shift. The spec used to be the thing you wrote before you built. Now the spec is the thing that runs after every change and tells you whether you still have a product. ## Why does the planning spec die the moment AI starts shipping? Because AI changes the codebase faster than any human can update a document. A spec written Monday describes a system that no longer exists by Thursday. The map stops matching the territory, and everyone keeps trusting the map. That is how a team ships against a fiction for a week without noticing. Pre-AI, code was the bottleneck. A spec stayed roughly current because shipping was slow enough that a person could keep the document in step. AI removed the bottleneck. The document is now the slowest artifact in the loop, and the slowest artifact is the one that lies first. Here is the rule. If an artifact cannot run, it cannot verify. If it cannot verify, it rots. Prose cannot run. Picture an agent refactoring a payment path at 2am. The spec doc still describes the old flow, confidently, in full paragraphs. The test either passes against the new behavior or it does not. Only the test caught the drift, because only the test was awake. This is not a claim that planning is dead. It is the form of the plan that changed. The durable form of a plan is executable, not narrative. Spec-driven development assumes a stable, human-paced write cycle. Vibe coding breaks that assumption by construction, so the narrative spec was never going to survive it. ## What does it mean to write the test as the spec? It means encoding correctness as a check before or alongside the feature, so the requirement and its proof live in the same place. You stop describing what should be true and start asserting it. On every commit. With no human in the loop to confirm. The practice is simple. Write the assertion first, in plain language, as the test name. The test name is the requirement sentence. Then let the agent build until that assertion passes. On Orbyt the agent writes the test alongside the feature, and the test is the artifact I trust. Not the diff. The diff is what the agent did. The test is what the agent was allowed to do. A spec-grade test has three properties prose does not. It is executable, so it can verify itself. It fails closed, so a break denies rather than permits. And it is unambiguous about what done means, because a passing assertion is a binary, not an opinion. Prose has none of the three. Fail-closed is the property that matters most. On Orbyt, subscription gating denies access when the check breaks rather than granting it. The test encodes the safe default, so the spec carries its own failure behavior. A broken document tells you nothing. A broken fail-closed test locks the door. **If correctness is not encoded as a check, it is not specified. It is hoped for.** ## What makes a vibe-coded app actually production ready? A test suite dense enough that no change, human or agent, reaches production without proving the system still does what it must. Vibe coding gets you to a demo. The suite is the only thing that gets you to production and keeps you there. The demo-to-production gap is entirely a verification gap. The vibe gives you velocity. The tests give you the right to ship it. A vibe-built app with no suite is not a product. It is a prototype that happens to be deployed, and the first agent edit that drifts silently breaks it while the demo still looks great. Coverage that matters is not a percentage. A high line-coverage number on the wrong paths is theater. The real question is whether every path you cannot afford to break has a check that fails closed. Count those paths, not lines. On Orbyt the 11,372 tests sit inside a 35-dimension audit harness. One command checks build, types, lint, fifteen locales, the iOS app, the Safari extension, a security battery, and accessibility. Production readiness is a binary that harness answers, not a feeling the model hands me. No feature is done when the model says done. It is done when the harness says done. The model is confident by default. The harness is correct by default. I ship on the second one. ## How do you test AI-generated code you cannot read? You replace human reading with machine proof. I cannot review 425,000 lines by hand, and neither can your team. So you build a suite and a harness that review them for you on every commit, and you act on the verdict. I argue the verdict-over-diff point in full in [Verification Is the New Literacy](/blog/verification-is-the-new-literacy), so I will not re-run it here. For the lived account of shipping into a codebase no person can hold in their head, see [I Cannot Read My Own Codebase. I Ship to It Daily.](/blog/cannot-read-my-own-codebase) The mechanics for tests-as-spec come down to four steps. 1. Encode every non-negotiable as a test before you let the agent loose on that area. The non-negotiables are your real spec. 2. Make the agent write a test with every feature, and reject any change that ships behavior without a corresponding check. 3. Wrap the tests in a single-command harness, so verification is one action and not a checklist a tired human skips. 4. Make critical paths fail closed, so an unverified state denies rather than permits. This holds even when the engine underneath you changes. I switched frontier models mid-session twice in June 2026. Once by choice, once when export controls forced it. Both swaps were uneventful, because the suite held the contract, not the model. When the spec is executable, the model is a replaceable part. When the spec lives in someone's head, every model swap is a rewrite. ## What can a CTO do with this on Monday? Stop measuring AI output by lines of code. Start measuring it by checks that pass without a human in the loop. Pick your three most dangerous production paths and make each one fail-closed and test-covered this week. That list is your real spec, encoded. List the paths where being wrong costs you a customer, a fine, or trust. That list is the specification, whether or not anyone wrote it down. Encode each path as a fail-closed test, and you have turned the document you were never going to maintain into a contract that maintains itself. Add one rule to every AI initiative. Done means the harness passes, not the model claims it is finished. That single line ends most of the demo-to-production arguments before they start. Make verification coverage a tracked metric, the same as revenue or churn. Count checks. Time the harness. Track which critical paths fail closed and which still fail open. What you measure is what survives the next refactor. Stop maintaining the prose spec as the source of truth. Let it be the onboarding narrative a new hire reads to understand the shape of the thing. Let the tests be the contract. One honest limit. A 35-dimension harness covers 35 dimensions. Name what is not covered out loud, because the gaps are exactly where the next incident lives. The discipline is in choosing those dimensions and writing those checks. The discipline is the part nobody can vibe. **The vibe writes the code. The tests decide whether it was ever a product.** Related reading: - [Verification Is the New Literacy](/blog/verification-is-the-new-literacy) why verifying, not authoring, is the scarce leadership skill - [I Cannot Read My Own Codebase. I Ship to It Daily.](/blog/cannot-read-my-own-codebase) shipping into 425,000 lines no human can read - [I Built a Production SaaS in 32 Days. Claude Wrote Our Story.](/blog/built-production-saas-32-days) the solo 32-day Orbyt build whose suite became the spec - [You Can Vibe a Product.](/blog/you-can-vibe-a-real-product) the outcome this method makes safe: shipping a real product, not just a demo - [84 Ways to Tell Me I'm Wrong.](/blog/ai-code-audit-harness) the layer above the suite: what a deterministic harness catches that tests structurally cannot - [Heal What You Can Prove.](/blog/self-healing-ai-code) the healers that restore artifacts toward the spec, and what they may never touch --- ## Frequently Asked Questions ### Do you need tests for vibe coding? Yes, if you want it in production. Vibe coding gets you to a working demo fast, but the demo-to-production gap is a verification gap. Without a dense, fail-closed test suite, the first agent edit that drifts silently breaks the product. The tests are what make vibe-built software survive real users. ### How do you test AI-generated code you cannot read? You replace human reading with machine proof. You cannot review 425,000 lines by hand, so the agent writes a test alongside every feature, and a single-command harness runs all of its checks on every commit. You act on the pass-fail verdict, and critical paths fail closed so an unverified state denies rather than permits. ### Why are tests the spec instead of a planning document? Because AI changes the code faster than anyone can update a document. A prose spec written Monday describes a system that no longer exists by Thursday. A test runs, fails loudly when behavior drifts, and stays true to what the code actually does. Only the executable artifact stays current. ### What makes a vibe-coded app production ready? A test suite dense enough that no change, human or agent, ships without proving the system still does what it must. On Orbyt that is 11,372 tests inside a 35-dimension audit harness. Production readiness is a binary the harness answers, not a feeling the model gives you. --- Source: https://justinbartak.ai/blog/tests-are-the-spec-vibe-coding Cite as: Justin Bartak, "Your Tests Are the Spec Now.", https://justinbartak.ai, 2026-08-12. https://justinbartak.ai/blog/tests-are-the-spec-vibe-coding