Most 'build vs buy' frameworks ignore a third option — port a proven open-source implementation. We score vendors on switching cost, model-portability, data-gravity, and eval-portability, ruling out 80% before procurement gets involved. Five categories with the call for each at SMB scale.
Almost every AI infrastructure decision comes down to a build-vs-buy slide. The slide is missing a third option: port a proven open-source implementation and own the result. Buy is fastest, build is most flexible, port is the path most engineering teams should default to and rarely consider.
A four-axis scoring framework that rules out 80% of vendors before procurement gets involved, plus the call for each of five common categories at SMB scale. The framework is not a substitute for talking to vendors — it is the screen you run before booking the meeting.
The four axes
1. Switching cost
How much work to migrate off this vendor in 18 months when the landscape has moved? High-switching-cost vendors include anything that owns your data store, your prompt library, or your eval harness. Low-switching-cost vendors are the ones whose API can be replaced with a competitor in a week.
2. Model portability
Can you swap the underlying model? Vendors that abstract away the model give you optionality; vendors that hard-code their preferred model lock you in to whatever pricing and capability they negotiate.
3. Data gravity
The more of your data lives in the vendor, the higher the switching cost. Embeddings stores, conversation logs, eval datasets — these accumulate gravity that becomes prohibitive to move.
4. Eval portability
When you decide to switch, can your eval pipeline transfer? Or do you have to rebuild it inside the next vendor? Vendors that lock you into proprietary eval formats are the slow-burn lock-in nobody talks about.
Most vendors that look great on the demo lock in your evaluation pipeline so you cannot prove a competitor is better.
Five categories, five calls
- Orchestration / agent framework. Port. The open-source landscape moves faster than any vendor; commercial agent frameworks become legacy in 18 months. Use a permissive-licence framework you can fork.
- Eval tooling. Port. The eval space is moving fast; a year-old commercial eval product is already missing capabilities your team will need. OpenAI Evals, Promptfoo, and similar are good defaults.
- Observability / tracing. Buy. Observability is cross-cutting and you do not want to operate it. Pick a vendor with OpenTelemetry compatibility so the switching cost is bounded.
- Vector store. Port or buy depending on volume. Below ~10M vectors, an open-source store on a single VPS is fine. Above that, the operational overhead of self-hosting beats most teams.
- Foundation models. Buy via API. Self-hosting frontier models is uneconomical for almost every SMB; open-weight models are useful for narrow tasks but not as the daily driver.
The procurement question that ends the meeting
When the vendor demo finishes, ask: "If we want to switch off you in 18 months, what is the migration plan?" If the answer is vague, polite, or hostile, score them down. The vendors who have a real answer are the ones worth buying — they have thought about the relationship past the close.
The procurement teams that ask this question end up with a stack that lasts. The ones that do not end up with a stack their successor has to rip out.