The Prototype Loop From Hell
How to validate fast, avoid spaghetti code, and move from prototype to production
As a machine learning engineer, or more broadly an AI and ML engineer, I am responsible for the entire lifecycle of the tools I build, whether they use AI or not.
Most ideas start with upper management or a business analyst. These people are closest to clients and customers, so they understand real pain points and where our processes could be improved. They come to me with an idea they want to test, along with expectations around what the tool should do and what results should look like. My role is to decide whether AI is actually needed and whether the idea is worth building at all.
The prototype phase
This is where I get to work and we enter the prototype phase. The prototype phase is intentionally fast and messy. It is not about clean architecture or perfect code. It is about getting something working quickly.
I usually create a rough script or a simple tool that produces the kind of output we expect, just enough so the business analyst and sometimes customers can test it and give feedback.
How prototypes turn into a trap
This is where things often become problematic.
At first, it is fun. People try it out and come back with small requests. One extra feature. A widget. A slight change. I think, sure, why not. I add it, ship an updated version, and wait for feedback. Then another request comes in. Then another. Suddenly two or three months have passed, and we still have not made a clear decision on whether the tool is worth moving forward with.
Meanwhile, others start asking when it is going into production or when we can start using it seriously. And you do not have a good answer because the prototype never officially ended.
The code also evolves into a nightmare. What was acceptable as a quick prototype becomes messy, scattered, and hard to maintain. You cannot easily find what certain functions do. It turns into spaghetti code. At that point you are often forced to step back and refactor everything from scratch, wasting time and momentum.
The advice
My advice is simple. Do not fall into the prototype loop where you keep adding features but never move forward.
The prototype phase should be short and focused. Its job is to validate whether the idea works and whether it delivers value.
A simple process that prevents the loop
Set a clear goal at the start. Define what success looks like and what output is required to validate the idea. This can be a simple metric or clear acceptance criteria. What must be true for us to say, yes, this is worth building.
Build the smallest prototype that can validate the goal. Keep it minimal. Do not over engineer it. Do not treat it like a production system.
Collect feedback only against the success criteria. If the prototype meets the requirements, move forward. If it does not, either iterate once with purpose or scrap it.
Moving to version 1.0
If the prototype passes, that is when you transition into version 1.0. You go back while everything is still fresh and relatively small, and you reorganize the code properly, using good engineering principles, a clear structure, and maintainable design.
Once version 1.0 exists, then you can start accepting new feature requests and pulling items from the backlog in a controlled way.
There is one exception during prototyping. If a requested change is absolutely required to validate the results, meaning the prototype cannot be fairly evaluated without it, then you may add it once or twice. Otherwise, feature requests go into the backlog, not into the prototype.
In summary
Prototype fast. Validate clearly. Promote to version 1.0 early. Add features later, properly.
P.S.
I changed the name of the newsletter. It will no longer be called the AI Learning Hub Newsletter.
It will now be Dan’s AI Notes.
So, don’t be alarmed that it’s something different.
And see you in the next issue!