A colleague of mine, Horacio Fernandez, and I were sharing a familiar frustration one day. Sellers were bringing us into client calls that hadn’t been qualified. We were walking into conversations cold, and it was happening often enough that it stopped feeling like an accident.
That frustration turned into curiosity. We decided to stop complaining about it and actually look at the data.
Defining “qualified”
The first thing we had to do was decide what “qualified” meant in concrete terms. We landed on BANT: Budget, Authority, Need, Timeline. Not because it’s sophisticated. Because it’s the floor. BANT is the bare minimum you’d expect from a seller who’s doing their job. It’s also well-defined enough that you can identify whether those things are being gathered during a conversation. You don’t need a subjective judgment call. Either the seller asked about budget or they didn’t. Either they identified the decision maker or they didn’t.
That clarity mattered, because we weren’t interested in grading calls on vibes. We wanted to know, with data, whether our theory was correct: that sellers were skipping qualification entirely.
The Tooling
All sales calls were already being recorded and transcribed. The data was sitting there. Nobody was using it for this purpose, but it existed. A wealth of recorded conversations, timestamped, transcribed, just waiting for someone to ask a question of it.
So we went to work. We built some basic tooling. Nothing fancy. The workflow was straightforward: batch the calls that were identified as new customer conversations, run each transcript through an LLM with a prompt that defined what to look for and how to identify it, and collect the results. BANT criteria mapped against what actually happened on each call.
It worked really well. We were able to analyze months of call data in a very short period of time and determine whether the pattern we’d been seeing anecdotally held up when you looked at it at scale.
When you define the problem well, the technology part gets simple fast.
What If It Ran Live
Once you’ve proven that this kind of analysis works in batch, the obvious next question is: what if it ran live? What if every sales call got graded automatically, and that data flowed back to sales leadership in something close to real time? Not as a gotcha. As a signal. A way to know, with actual evidence, whether qualification is happening or not.
That’s the part that got us thinking bigger. Sales is one of the hardest functions to hold accountable with real data, because so much of the data depends on the seller’s participation and honesty. CRM entries, pipeline updates, deal stages, all of it requires the seller to accurately report what’s happening. And the role itself demands autonomy. Sellers need room to operate. They need to run their own process, build their own relationships, close on their own terms. You can’t micromanage someone into closing a deal.
The Boring Work First
What Horacio and I stumbled into wasn’t really a sales project. It was a proof point for something larger. Your organization is generating data all the time, in conversations, in workflows, in the normal course of doing business. Most of that data is sitting unused, or it’s being used in the most surface-level way possible. AI gives you the ability to actually interrogate that data. To ask questions of it that would have taken a team of analysts weeks to answer manually, and get answers in hours.
When you define the problem well, the technology part gets simple fast.
The reason it came together so quickly is that we’d done the boring work first. We defined what we were looking for before we built anything. BANT gave us a concrete framework to measure against, not a vague direction like “assess call quality.” That specificity is what made the tooling trivial to build. The LLM didn’t need to make judgment calls. It needed to check whether specific things happened on a call. When you define the problem well, the technology part gets simple fast.
Other Places This Lives
The sales call example is concrete and specific, but the pattern is bigger than sales and bigger than accountability. It’s about finding novel ways to use the data your organization is already generating to solve problems that were previously too expensive or too manual to touch.
Most people are still thinking about AI as a way to do existing work faster. The more interesting use is pointing it at problems you couldn’t practically solve before.
Think about customer success. Every support call, every ticket, every chat interaction is generating signal about where your product is confusing, where your documentation is failing, where customers are getting stuck in the same spot over and over. Or think about engineering. Every PR, every code review, every incident retro has signal in it about where the team is strong and where it’s struggling. You don’t need a consultant to run a six-month assessment. The work product is already telling you if you know what to ask.
In a client engagement, my team used a similar approach to grade the quality of stories in an engineering backlog. The results were fascinating and uncomfortable. The overall quality of the work definition was so poor that AI tooling wouldn’t have actually accelerated anything, because the entire system depended on a handful of people who carried enough institutional context in their heads to translate vague requirements into real work. The tooling didn’t fix that problem, but it surfaced it in a way that months of standups and retros hadn’t.
No matter what your industry, there’s probably a version of this sitting in your organization. A problem everyone can feel but nobody has the data to prove. A process generating information that nobody’s asking questions of. Most people are still thinking about AI as a way to do existing work faster. The more interesting use is pointing it at problems you couldn’t practically solve before.