I just tried an experiment using Spec-Kit from GitHub to build a CLI tool. Perhaps the scope of the tool doesn't align itself with Spec-Driven Development, but I found the many many hours—tweaking, asking, correcting, analyzing, adapting, refining, reshaping, etc—before getting to see any code challenging. As would be the case with Waterfall today, the lack of iterative end-to-end feedback is foreign and frustrating to me.
After Claude finally produced a significant amount of code, and after realizing it hadn't built the right thing, I was back to the drawing board to find out what language in the spec had led it astray. Never mind digging through the code at this point; it would be just as good to start again than to try to onboard myself to the 1000s of lines of code it had built... and I suppose the point is to ignore the code as "implementation detail" anyway.
Just to make clear: I love writing code with an LLM, be it for brainstorming, research, or implementation. I often write—and have it output—small markdown notes and plans for it to ground itself. I think I just found this experience with SDD quite heavy-handed and the workflow unwieldy.
This article if for those who already made up their mind that "spec-based-development" isn't for them.
I believe (and practice) that spec-based development is one of the future methodologis for developing projects with LLMs. At least it will be one of the niches.
Author thinks about specs as waterfalls. I think about them as a context entrypoint for LLMs. Giving enough info about the project (including user stories, tech design requirements, filesystem structure and meaning, core interfaces/models, functions, etc) LLM will be able to build sufficient initial context for the solution to expand it by reading files and grepping text. And the most interesting is that you can make LLM to keep the context/spec/projetc file updated each time LLM updates the project. Viola: now you are in agile again: just keep iterating on the context/spec/project
And this is absolutely fine, because the problem with waterfall wasn't the detailed spec, it was
a) the mulyi-year lead time from starting the spec to getting a finished product
b) no (cheap) way to iterate or deliver outside the spec
Neither of these are a problem with SDD.
> Coding assistants are intimidating: instead of an IDE full of familiar menus and buttons, developers are left with a simple chat input. How can we ensure that the code is correct with so little guidance?
Isn't this was Kiro IDE is about? Spec-driven dev?
I have similar feelings. I’m willing to believe there are scenarios where this kind of thing makes sense, maybe – a colleague has had great success on a small, predictable greenfield project with it. I don’t work on many of those. My main objections are that I’ve had plenty of success with LLMs without intermediate detailed specs (and I don’t think my failures would have been helped by them), and I just don’t like the idea of primarily reviewing specs. Some sort of plan or research document is a different matter - that’s fine. But the kind of code-like formalised spec thing? I want to look at code, it’s just easier. Plus, I’m going to be reviewing the code too (not doing so is irresponsible in my opinion), so having spec AND code is now double the text to read.
The part of the process the actually needs improving, in my experience in larger codebases, is the research phase, not the implementation. With good, even quite terse research, it’s easy to iterate on a good implementation and then probably take over to finish it off.
I really think LLMs and their agent systems should be kept in their place as tools, first and foremost. We’re still quite early in their development, and they’re still fundamentally unreliable, that I don’t think we should be re-working over-arching work practices around them.
> Agile methodologies killed the specification document long ago. Do we really need to bring it back from the dead?
it didn't really kill it - it just made the spec massively disjoint, split across hundreds to thousands of randomly filled Jira tickets.
Bit of a tangent, but this reminds me of a video[1] I watched a bit ago where there was someone who interviewed 20 or so people who were both engineers and programmers, and asked them what the two fields could learn from each other. One of the things it mentioned from the perspective of a physically-based engineer is that a little more up-front planning can make a big difference, and that's stuck with me ever since.
[1] (pretty sure this is the right one): https://youtu.be/CmIGPGPdxTI
This is a weird article. How many times in your career have you been handed a grossly under-specified feature and had to muddle your way through, asking relevant people along the way and still being told at the end that it’s wrong?
This is exactly the same thing but for AIs. The user might think that the AI got it wrong, except the spec was under-specified and it had to make choices to fill in the gaps, just like a human would.
It’s all well and good if you don’t actually know what you want and you’re using the AI to explore possibilities, but if you already have a firm idea of what you want, just tell it in detail.
Maybe the article is actually about bad specs? It does seem to venture into that territory, but that isn’t the main thrust.
Overall I think this is just a part of the cottage industry that’s sprung up around agile, and an argument for that industry to stay relevant in the age of AI coding, without being well supported by anything.
Developers spend most of their time reading long Markdown files, hunting for basic mistakes hidden in overly verbose, expert-sounding prose. It’s exhausting.
Those requirements exist regardless of whether you write them down in Markdown files or not. Spec-driven Development is just making what needs to be built explicit rather than assuming the whole team know what the code should do.
Practically every company already uses 'spec-driven development', just with incredibly vague specs in the form of poorly written Jira tickets. Developers like it because it gives them freedom to be creative in how they interpret what needs to be done, plus they don't need to plan things and their estimates can be total nonsense if they want, and Product Owners and BAs like it because it means they can blame shift to the dev team if something is missed by saying "We thought that was obvious!"
Every team should be capturing requirements at a level of detail that means they know how the code should work. That doesn't need to be done up front. You can iterate. Requirements are a thing that grow with a project. All that spec-driven development is doing is pushing teams to actually write them down.
Thank you for writing this. It was also my first impression after seeing not only spec driven dev, but agentic systems that try to mimic human roles and processes 1:1. It feels a bit like putting a saddle on an automobile so that it feels more familiar.
There’s nothing keeping you from scoping the spec to an agile package of work. To the contrary: even if you start with a full spec for a multi-day-AI-coding-session, you are free to instruct it to follow agile principles. Just ask it to add checkpoints at which you want to be able to let users test a prototype, or where you want to revisit and discuss the plan.
Replace the product people with LLMs and keep the engineers. You'll get better results that way.
It's a nice observation that Spec-Driven Development essentially implements the waterfall model.
Personally, I tried SDD, consciously trying to like it, but gave up. I find writing specs much harder than writing code, especially when trying to express the finer points of a project. And of course, there is also that personal preference: I like writing code, much more than text. Yes, there are times where I shout "Do What I Mean, not what I say!", but these are mostly learning opportunities.
Don't really think the thesis is fair.
SDD as it's presented is a bit heavy weight, if you experimented with a bit, there is a lighter version that can work.
For some mini modules, we keep a single page spec as 'source of truth' instead of the code.
It's nice but has it's caveats but they are less of a concern over time.
A point I like to make in discussions like this is that software and hardware specifications are very different. We think of software as the thing we're building. But it's really just a spec that gets turned into the thing we actually run. It's just that the building process is fully automated. What we do when we create software is creating a specification in source code form.
Compared to what an architect does when they create a blueprint for a building, creating blueprints for software source code is not a thing.
What in waterfall is considered the design phase is the equivalent of an architect doing sketches, prototypes, and other stuff very early in the project. It's not creating the actual blue print. The building blue print is the equivalent of source code here. It's a complete plan for actually constructing the building down to every nut and bolt.
The big difference here is that building construction is not automated, costly, and risky. So architects try to get their blueprint to a level where they can minimize all of that cost and risk. And you only build the bridge once. So iterating is not really a thing either.
Software is very different; compiling and deploying is relatively cheap and risk free. And typically fully automated. All the effort and risk is contained in the specification process itself. Which is why iteration works.
Architects abandon their sketches and drafts after they've served their purpose. The same is true in waterfall development. The early designs (whiteboard, napking, UML, brainfart on a wiki, etc.) don't matter once the development kicks off. As iterations happen, they fall behind and they just don't matter. Many projects don't have a design phase at all.
The fallacy that software is imperfect as an engineering discipline because we are sloppy with our designs doesn't hold up once you realize that essentially all the effort goes into creating hyper detailed specifications, i.e. the source code.
Having design specifications for your specifications just isn't a thing. Not for buildings, not for software.
I think I've seen enough of a trend: all these LLM ideas eventually get absorbed by the LLM provider and integrated. The OSS projects or companies with products eventually become irrelevant.
So they're more like 3rd party innovations to lobby LLM providers to integrate functionalities.
X prompting method/coding behaviors? Integrated. Media? Integrated. RAG? Integrated. Coding environment? Integrated. Agents? Integrated. Spec-driven development? It's definitely present, perhaps not as formal yet.
Most software developers are doomed to rediscover time and time again that 4 weeks of frantic developments save two hours of calmly thinking about the task.
But specs are per feature, it’s just an up front discussion first like you’d have on may things rather than questions-> immediate code writing from a model.
>> You can see my instructions in the coding session logs
such a rare (but valued!) occurrence in these posts. Thanks for sharing
I'm still not convinced there's anything wrong with waterfall for some projects.
That's Event-B (minus the formal side) with LLM
What LLM tools are folks seeing that do the most or the best to integrate specs?
Amazon's Kiro is incredibly spec driven. Haven't tried it but interested. Amplifier has a strong document-driven-development loop also built-in. https://github.com/microsoft/amplifier?tab=readme-ov-file#-d...
For myself, I found that having a work methodology similar to spec-driven development is much better than vibe coding. The agent makes less mistakes, it stays on the path and I have less issues to fix.
And while at it, I found out that using TDD also helps.
Much of the hype around SDD is really about developers never having experienced a real waterfall project.
Of course SDD/Waterfall helps the LLM/Outsourced labor to implement software in a predictable way. Waterfall was always a method to please Managers and in the case of SDD the manager is the user promoting the coding agent.
The problem with SDD/Waterfall is not the first part of the project. The problems come when you are deep into the project, your spec is a total mess and the tiniest feature you want to add requires extremely complex manipulation of the spec.
The success people are experiencing is the success managers have experienced at the beginning of their software projects. SDD will fail for the same reason Waterfall has failed. The constant increasing of complexity in the project, required to keep code and spec consistent can not be managed by LLM or human.
I, for one, welcome the fact that agile/scrum/daily standup/etc. rituals will be outdated. While they might be somehow useful in some software development projects, in the past 10 years it turned out to be a cult of lunatics who want to apply it to any engineering work, not just software, and think any other approach than that will result in bad outcomes and less productivity. Can't wait for the "open office" BS to die next too, literally a boomer mindset that came from government offices back in the day, and they think it's more productive that way.
You can't code without specifications, period. Specifications can have various forms but in ultimately define how your program should work.
The problem with what people call "Waterfall" is that there is an assumption that at some point you have a complete and correct spec and you code off of that.
A spec is never complete. Any methodology applied in a way that does not allow you to go back to revise and/or clarify specs will cause trouble. This was possible with waterfall and is more explicitly encouraged with various agile processes. How much it actually happens in practice differs regardless of how you name the methodology that you use.
I vibe coded for months but switched to spec driven development in the last 6 months
I'm also old enough to have started my career learning the rational unified process and then progressed through XP, agile, scrum etc
My process is I spend 2-3 hours writing a "spec" focusing on acceptance criteria and then by the end of the day I have a working, tested next version of a feature that I push to production.
I don't see how using a spec has made me less agile. My iteration takes 8 hours.
However, I see tons of useless specs. A spec is not a prompt. It's an actual definition of how to tell if something is behaving as intended or not.
People are notoriously bad at thinking about correctness in each scenario which is why vibe coding is so big.
People defer thinking about what correct and incorrect actually looks like for a whole wide scope of scenarios and instead choose to discover through trial and error.
I get 20x ROI on well defined, comprehensive, end to end acceptance tests that the AI can run. They fix everything from big picture functionality to minor logic errors.