Apptivity gets hired to clean up these situations so often we had to say something. From vendors who skip problem definition to those who can't explain your costs — here are the red flags.
I don't love doing "red flag" posts but Apptivity gets hired to fix these situations so often I have to say something.
01🚩 1. They Talk About Models Before They Talk About Your Problem
Good consultants ask 50 questions before writing a line of code. If they jumped to "we'll use GPT-4" in the first meeting, run.
The model is an implementation detail. The problem is the product. Any vendor who leads with technology instead of understanding is optimizing for their comfort, not your outcome.
02🚩 2. No Evaluation Framework in Sight
"It looks good to me" is not a testing strategy. If they can't show you automated evals, they're guessing.
Ask them: "How do you measure whether the AI is getting better or worse over time?" If the answer involves a human reading outputs, you don't have a production system. You have an expensive intern.
03🚩 3. They Want to Fine-Tune on Day One
If you've read my other posts, you know where I stand on this. Fine-tuning is step 10. Most teams try to start there because it sounds impressive and justifies a large budget.
04🚩 4. The Prototype Was Fast, But "Production-Ready" Keeps Getting Pushed Back
Demos are easy. Production is architecture. These are different skills.
A prototype that works on 10 examples is not the same as a system that handles 10,000 concurrent users with 99.9% uptime, proper error handling, cost controls, and security. If your vendor shipped a demo in two weeks but has been "almost production-ready" for three months, the architecture isn't there.
05🚩 5. They Can't Explain the Costs
If your AI bill is a surprise every month, the architecture is wrong. At Apptivity we build cost controls INTO the system, not around it.
Every API call should be tracked, every model choice should be justified, and every cost spike should be explainable. If your vendor can't give you a cost-per-query breakdown, they're not managing your infrastructure — they're hoping for the best.
If any of these sound familiar, it might be time for a second opinion. No pitch, just an honest 15-minute diagnostic. The catch is that 60% of free calls turn into clients because the problems are that fixable.