The Practical Limits of Generative AI in Enterprises

Generative AI creates text, images, code, and other content from a user’s request and available information. At work, it helps most with bounded tasks: the inputs are clear, and a person can check the results. It is less dependable when a task requires accurate facts across many steps or draws on information scattered across company systems; it is also less dependable for high-stakes decisions.

Business results depend on more than the model: data, software connections, human review, governance, and ongoing costs all matter. A successful demonstration does not show that a deployment will save time or money.

Where generative AI helps workers

Generative AI has its clearest enterprise value when it helps a person handle recurring work faster. Examples include drafting a response, summarizing a document, finding relevant information, or producing a first version of code. These tasks have recognizable outputs, and a worker can compare them with the request or reference information before using them.

A field study of 5,179 customer-support agents at one large software company found that an AI assistant increased the number of issues resolved per hour by about 14% on average. The assistant suggested answers for agents to accept or ignore; it did not serve customers on its own. Gains were largest among less-experienced and lower-performing agents, at about 34% in the working paper’s abstract. Experienced, highly skilled agents saw little effect. This suggests AI can help less-experienced workers find and use knowledge that more experienced colleagues have built up.

The same tools do not help every worker in the same way. A 2025 randomized trial of experienced open-source developers working in mature projects they already knew found that they took longer when AI tools were available. The study reflects a specific group, set of projects, and tools from early 2025, so it is not a general estimate for software development. It does show why leaders should test a tool with the people and work it is meant to support rather than assume that assistance always speeds up skilled work.

For enterprise use, the practical test is whether AI reduces effort after a person checks its work. A draft that needs light editing can save time. A draft that requires line-by-line fact checking may simply move effort from writing to review.

Why plausible AI answers fail

A language model predicts what text is likely to come next. A fluent answer can still contain false claims. It can give a confident answer despite missing or incorrect information. KPMG’s 2025 board survey found that respondents identified inaccurate underlying data and inaccurate outputs, including hallucinations, among their leading concerns. Those figures report concerns. They do not measure error rates across companies using AI.

The risk depends on what happens after the model answers. An employee reviewing a suggested customer-service response can catch an error before sending it. When a system sends messages, changes records, approves transactions, or triggers actions in other software, there is less chance to catch an error before it takes effect. The more consequential the action, the more the deployment needs clear limits, a reliable path for uncertain cases, and a human who can intervene.

Testing should therefore include ordinary requests and the cases most likely to expose weaknesses: missing information, conflicting records, unusual requests, and questions the system cannot answer from approved material. For each case, decide whether the system should answer, ask for clarification, or pass the work to a person. A confident but unsupported answer should count as a failure even if it sounds polished.

How company data limits AI

A model’s usefulness depends on whether it can access the right company information and interpret it in context. A search tool that finds an outdated policy or a duplicate record can supply the wrong basis for an otherwise well-written answer. Documents also need appropriate permissions: an employee should not receive information simply because a model can retrieve it.

The data needs to be current and accurate, access controls must be in place, and the information must connect to the work at hand. If a support assistant needs current product guidance, for example, the organization must make that guidance easy to find and keep older instructions from competing with it. If a sales assistant needs account details, the relevant records must be accessible under the same permissions that govern employees’ access.

Company data quality is separate from questions about a model’s original training data. The U.S. Government Accountability Office reported that developers it interviewed generally provided only high-level descriptions of training data, and many said it included publicly available internet material. That finding concerns transparency about commercial models; it does not establish the quality of any particular company’s internal records.

Poor data and weak connections between software systems can make a capable model ineffective. The 2025 MIT Project NANDA report describes a gap between general-purpose tools employees can try and custom tools that fit a company’s actual work. It reports that projects stall before production when tools do not fit company workflows or connect well to other software. The report labels its findings preliminary, so they describe reported patterns rather than a universal failure rate. Check whether the tool can find the right information and fit the actual work before judging the model.

AI governance needs operating controls

A written policy does not control what a deployed AI system can access or do. Someone must be responsible for each AI system in use. That person decides what data it can use and sets review requirements. They also respond when it produces an unsafe or incorrect result. Those responsibilities also need to cover changes to the model, connected data, and surrounding software.

KPMG’s 2025 board survey found that 70% of respondents said their company was developing responsible-use policies, while 40% said it was implementing a recognized AI risk-governance framework. This suggests respondents were further along in writing policies than in putting formal risk management in place. It does not show how well either one worked in practice.

The National Institute of Standards and Technology’s Generative AI Profile gives organizations a voluntary framework for managing risks throughout design, development, use, and evaluation. A framework can help structure decisions, but it does not certify a particular deployment as reliable or safe. The U.S. Government Accountability Office also identifies risks such as hidden instructions that try to change a model’s behavior (prompt injection), attempts to bypass its safeguards (jailbreaks), and corrupted training data (data poisoning). These threats need testing in the deployed workflow, especially when a model reads outside content or can use tools connected to company records.

Match human review to the consequences of an error. A suggestion used internally may need a different level of control from a response sent to a customer or an automated change to a financial record. Define who approves each action, what the system must log, and how a worker can report and correct failures.

The business case includes more than model fees

Paying for each model request or an employee license covers only part of the cost of putting AI to work. Preparing data and building connections require effort, as do securing access and reviewing outputs. Monitoring performance, training workers, and maintaining the service also take effort. Those costs vary with the task and the way an organization deploys AI, so a low unit price does not establish a positive return.

The cost of running leading models to generate output has fallen sharply since 2023, according to a Google Research survey of the economics of generative AI. Lower inference costs make some uses more affordable, but they do not remove integration and oversight costs. They can also make it easier to expand usage, increasing total spending even as each request becomes cheaper. There is no neutral, cross-industry comparison of the total cost per successful enterprise task.

Assess the full task by counting the time employees spend reviewing and correcting results, along with the engineering and operating costs of keeping the tool connected to current data. Compare that with the value of work completed, such as time saved or a measurable change in service performance. If the tool raises output volume but also creates more correction work, the net benefit may be small.

Evaluate AI in production conditions

A demo can show that a model produces an attractive answer for a prepared prompt. It cannot show whether the system performs well on an organization’s real requests, incomplete records, exceptions, or permissions. General benchmarks have the same limitation: a score on a broad test does not establish quality in a company’s specific process.

Before deploying, collect representative examples of the task and decide what counts as an acceptable result. Test factual accuracy and completeness, but also measure how often people need to edit the answer, how long review takes, and whether the system handles missing or unsupported information appropriately. Where a mistake carries significant consequences, test those cases directly and set a lower tolerance for errors.

Measure results in the workflow after launch as well. Compare the time and quality of work with the earlier method, and check whether people are using the tool as intended. Retest after changes to the model, data, or software connections because those changes can alter results. NIST’s risk-management guidance recommends evaluating an AI system throughout its life. The evidence does not show that any one measure proves AI delivers value in every job.

Judge a completed task, not just the generated response. A response counts as useful only when it meets the required standard, reaches the right person, and does not create more work or risk than it removes.

AI adoption takes workflow changes

Widespread access does not mean widespread impact. Stanford’s 2025 AI Index reported that 71% of survey respondents said their organizations used generative AI in at least one business function in 2024. In McKinsey’s 2025 survey, 88% reported regular AI use in at least one business function, but about one-third said their organizations had begun scaling AI programs. The surveys covered different dates and used different wording. Both show that a business can use AI in some areas while making limited progress toward wider use.

Reported financial effects were generally modest. Stanford found that most reported cost savings were below 10%, and the most common reported revenue increases were below 5%. McKinsey reported that some respondents attributed an enterprise earnings impact to AI, with most of that group reporting an impact below 5%. These are survey responses, not audited results or causal estimates.

The 2025 MIT Project NANDA report said 95% of organizations in its study had no measurable profit-and-loss return. That headline needs particular care: the report labels itself preliminary and draws on 52 organizational interviews, 153 senior-leader survey responses, and a review of publicly disclosed initiatives. Its sample and definition of measurable return differ from Stanford’s and McKinsey’s. Treat the figure as a finding from that study, not a universal rate of enterprise failure.

Adoption also depends on people and management. In KPMG’s survey, 47% of respondents cited new skills and talent needs, and 42% cited workforce transformation, including retraining or workforce reductions, as significant disruptions. Workers need training to check outputs and understand when to reject them. Managers may need to change how work moves between people and software. If existing steps remain unchanged, AI can add another interface and another review burden without improving the outcome.

Choose bounded AI tasks first

A sensible first use has a frequent task, accessible source information, a clear quality standard, and an employee who can review the result. A system that drafts routine responses from current guidance is easier to evaluate than one expected to manage a long-running customer relationship or make decisions across several business systems.

Before expanding, name the outcome the deployment should improve and establish how it will be measured. Test the tool against current performance, include the time people spend checking results, and set a clear point at which the system must hand work to a person. Expand only when the measured benefit persists under real operating conditions and the organization can maintain the data, controls, and oversight.