At a recent roundtable discussion with SaaS marketing and revenue leaders, one participant described an AI initiative that had consumed months of effort and produced little in return.
Like many organizations, her company had been looking for ways to apply artificial intelligence more aggressively throughout its go-to-market operation. The goal was straightforward: determine whether software could take over outbound prospecting work that had traditionally been handled by sales development representatives (SDRs).
Instead of having SDRs place calls, management wanted to see whether an AI-powered caller could initiate conversations, respond to common objections, and perform enough of the prospecting process to reduce the need for human involvement.
To make that possible, the company purchased contact data, worked with a specialized vendor, built scripts, mapped responses to common objections, and trained the system to carry on conversations that sounded convincingly human.
For roughly four to five months, the company worked to make the system effective.
After months of development, testing, and expense, the organization concluded that the effort was not producing enough value to justify continuing it.
The experiment highlights a question that sits beneath much of the current conversation around artificial intelligence. New capabilities appear almost weekly, and predictions about how work will change have become commonplace. Organizations are investing in tools, redesigning workflows, and revisiting hiring plans based on assumptions about future productivity gains.
The difficulty is that many of those decisions are being made before companies have agreed on how success should be measured.
Every week brings new claims about how artificial intelligence will transform sales, marketing, software development, customer service, and countless other functions.
Yet much less attention is given to a simpler question:
How should an organization determine whether the technology is actually creating value?
A recent interview examining enterprise AI adoption revealed a challenge that is becoming increasingly common. Companies can tell you how many employees are using AI, how much content has been generated, how often the tools are being used, and how much money is being spent. What they often struggle to explain is whether any of those activities are improving the business.
Most organizations have no shortage of data. They can tell you how many employees logged into a system, how many documents were generated, how many customer interactions involved AI, and how many internal processes were touched by automation. Those figures are relatively easy to collect, compare, and report.
The outcomes companies actually care about are far more difficult to evaluate. Beneath many AI initiatives sits the same set of questions: can artificial intelligence reduce the amount of work that requires human involvement, can teams operate more efficiently, can decisions be made faster, can revenue be generated more effectively, and can future hiring be reduced or avoided altogether?
Those are the assumptions driving much of the current race to automate work. Measuring activity is relatively straightforward. Determining whether the technology is producing meaningful business value is considerably more difficult.
The challenge becomes even greater when organizations begin making decisions based on expectations about future capabilities. A company may believe that a particular AI system will eventually reduce headcount, accelerate growth, or eliminate the need for additional hiring. Those outcomes exist in the future. The costs of implementation, experimentation, and ongoing operation exist in the present.
In the same interview, Uber's Chief Operating Officer, Andrew MacDonald, observed that it is easy to be impressed by AI when someone else is paying the bill, but eventually someone has to pay it. His observation reflects a reality that many companies are beginning to encounter. The technology itself may be impressive, but the relevant question is whether the expected benefits justify the cost required to achieve them.
The advertising executive Rory Sutherland offers a useful framework for understanding why so many organizations struggle to evaluate the success of artificial intelligence initiatives.
Sutherland often describes what he calls the Doorman Fallacy, a tendency for organizations to focus on whatever happens to be easiest to measure and then assume that the measurement reflects the outcome that actually matters.
His best-known example involves a hotel receiving complaints about elevator wait times. Management naturally assumes the problem is the speed of the elevators and begins exploring engineering solutions that would make them move faster.
Yet in many cases the complaints disappear after something much simpler is introduced: mirrors placed near the elevators. Guests become occupied looking at themselves, the wait feels shorter, and satisfaction improves even though the elevators are moving at exactly the same speed.
The lesson is not really about elevators. It is about the tendency to confuse what can be measured with what should be measured.
Businesses encounter versions of this problem constantly.
A company improves a metric, reports progress, and assumes the business has improved as well. Sometimes that assumption is correct. Often it is not.
The current race to automate work appears particularly vulnerable to this mistake. Organizations can measure software adoption, AI usage, content generation, automated interactions, and countless other indicators of activity.
What those measurements do not necessarily reveal is whether customers are happier, employees are more productive, decisions are improving, sales opportunities are being identified earlier, or the business is creating more value than it was before.
This distinction may explain why some AI projects generate excitement without producing meaningful results. The organization can point to impressive activity metrics yet remain unable to demonstrate that the underlying business problem has been solved.
The failed SDR experiment described earlier in this article illustrates the challenge.
The company could point to the software it had purchased, the scripts it had built, the conversations the system was capable of conducting, and the months invested in development.
What ultimately mattered, however, was whether the technology could perform enough of the prospecting function to justify reducing dependence on human SDRs. The project succeeded or failed on that measure alone, regardless of how sophisticated the technology appeared or how much activity it generated.
The same principle applies far beyond sales.
Artificial intelligence may be capable of generating content, writing code, answering questions, summarizing information, and completing a growing list of tasks that once required human effort.
Those capabilities are impressive, but they do not automatically translate into business value. The relevant question is not whether the technology can perform a task. The relevant question is whether performing that task changes an outcome that matters.
Many discussions about artificial intelligence begin with tasks because tasks are easy to identify. Salespeople make calls. Marketers create content. Customer success managers answer questions. Analysts conduct research.
Organizations, however, do not hire people simply to perform tasks. They hire them to produce outcomes.
A sales development representative is not valuable because a call was placed. The value comes from identifying a qualified opportunity. A marketer is not valuable because content was published. The value comes from influencing buyer behavior. A customer success manager is not valuable because questions were answered. The value comes from retaining and growing customer relationships.
Artificial intelligence can often perform parts of the task. Determining whether it can produce the desired outcome is a different question entirely.
That distinction helps explain why replacing people is proving more difficult than many executives expected. Completing an activity and achieving a result are not necessarily the same thing.
One of the more interesting moments in the roundtable discussion came when the conversation shifted away from replacing people and toward helping them make better decisions.
Several participants described using artificial intelligence to identify opportunities that would have been difficult to uncover through traditional methods.
They discussed analyzing hiring patterns, competitive relationships, organizational changes, market developments, and other indicators that might suggest a company was more likely to engage in a buying conversation.
Several also emphasized the importance of finding signals that their competitors had not.
This approach addresses a problem that has existed long before artificial intelligence entered the conversation. Most sales organizations are not suffering from a shortage of activity. They are struggling to determine where activity should be directed, which accounts deserve attention, and how to identify meaningful opportunities before competitors do.
Artificial intelligence is particularly well suited to processing large amounts of information and identifying patterns that would otherwise remain hidden.
When used in this way, the technology is not replacing salespeople, marketers, or account executives. It is helping them focus their time and attention more effectively by surfacing information that would have been difficult, or in some cases impossible, to assemble manually.
This distinction matters because it shifts the conversation away from tasks and toward outcomes. The objective is not simply to determine whether software can perform work that was previously assigned to a person.
The more important question is whether better information can improve decisions, increase productivity, and strengthen business performance.
Once those improvements become visible, leaders are in a much better position to determine what talent is needed, where additional hiring makes sense, and where certain responsibilities may no longer require the same level of human involvement.
That may not be the vision of artificial intelligence that dominates headlines or investor presentations. It is, however, much closer to the reality many companies are experiencing today.
The most successful applications of artificial intelligence are often found not in replacing people, but in helping them understand where opportunities exist, where risks are emerging, and where their effort is most likely to produce results.
For go-to-market teams, this may prove to be one of the technology's most valuable contributions.
A salesperson who knows which accounts are entering a buying window, who the key stakeholders are, what issues they are trying to solve, and which prospects are exhibiting signals that competitors have overlooked is in a stronger position than one who simply has a larger list of names to call.
In that sense, the value of artificial intelligence may have less to do with automation than with awareness.
The lesson emerging from the past several years is not that artificial intelligence lacks value. Many organizations are already benefiting from it. The challenge is that companies often begin with the technology before they understand the problem they are trying to solve.
Executives ask which tools they should deploy, which workflows they should automate, and which costs they should reduce. Those questions are understandable, particularly given the pressure many leaders feel to demonstrate progress.
The organizations seeing the greatest benefit from AI often begin somewhere else. They start by examining where decisions are breaking down, where information is missing, where opportunities are being overlooked, and where employees spend time on activities that contribute little value.
Only after those questions have been explored does the technology become relevant.
For many go-to-market teams, that process begins with signals.
The most valuable opportunities frequently emerge from information that competitors have not noticed, connections that have not been made, or changes in the market that have not yet become obvious. Helping teams uncover those signals may prove far more valuable than automating another task.
The purpose is not to generate another dashboard or add another piece of software to an already crowded technology stack. It is to better understand where information can create leverage, where judgment can be improved, and where artificial intelligence can contribute to meaningful business outcomes.
The race to automate work has encouraged many organizations to begin with headcount and work backward.
Yet, the companies seeing the greatest success with AI appear to be doing the opposite. They begin by asking how the business can perform better, how decisions can improve, and how opportunities can be identified earlier.
The staffing implications, if they come at all, tend to follow from those improvements rather than lead them.