It’s an increasingly common tale within corporations today: The AI project performs admirably in testing during the pilot phase, gets the green light for a broader rollout…and then stops working properly; Or it fails to deliver the expected business results.
Finger pointing, recriminations, and embarrassment ensue.
The problem is not always the technology. In fact, the fault is often in the planning, processes, and expectations that companies have established—or not established—around their AI projects, according to business leaders who spoke at a roundtable discussion at Fortune Brainstorm Tech this month.
For starters, not every AI project deserves to be rolled out widely, said Amgen Chief Technology Officer Sean Bruich.
“It’s so easy with a pilot to let a thousand flowers bloom,” he said. That’s not a bad thing, since it encourages experimentation. But, he said, “the key to making pilots scale successfully is actually having a wide number of ideas, but a very tight governance on which pilots are actually greenlit.”
A key criteria before taking the next step, said Salesforce Chief Customer and Commercial Officer Lashonda Anderson-Williams, is understanding the intended outcome of the project. Too many companies are focused on the successful implementation of AI features—the technological bells of whistles—instead of the business outcome, she says.
That mentality is a recipe for disappointment: The AI features work great, but the new technology isn’t driving meaningful business results.
Agents needs a map
When it comes to agentic AI, Anderson-Williams noted, a detailed understanding of the workflow—which individuals, groups, or touch points are necessary to complete a task— is critical. What a lot of companies are finding, she said, is that documentation of the workflow either doesn’t exist or is poorly documented: “When you put AI on top of that, the expectation is you’re going to see some magic, and there’s no magic there.”
Access to data is a particularly common stumbling block that AI projects encounter in the transition from the pilot phase to full deployment. With data often scattered in different silos throughout an organization, and with all that data governed by different access privileges and by varying privacy and security considerations, things can get complicate fast. It’s important to map out the contours of the AI project and all the potential data that will be required ahead of time, the panelists stressed. “The earlier we can uncover that in discovery, the better we’ll be set up for success,” Thomson Reuters Chief Data Officer Caitlin Halferty said.
That also means getting buy-in from the right groups and stakeholders within the organization. “Is there some element of PII (personally identifiable information) or confidential data that’s going to trigger privacy?” Halfery said. If the answer is yes, then the right people need to be part of the project. “Is there a cyber element? Let’s get security on board,” she said.
Amgen’s Bruich echoed the importance of broad buy-in, noting that an AI project that is transformational to the company will by necessity involve leaders in finance, technology, HR, and other groups across the organization. A truly impactful AI project, he said, needs to do more than just make work processes more efficient for a small group of employees. It needs to deliver “an outcome that matters to the enterprise.”
The narrative surrounding AI projects in corporations often follows a discouraging pattern: successful pilot testing is followed by a problematic rollout, leading to finger-pointing and accountability crises. According to discussions at Fortune Brainstorm Tech, the issue frequently lies not in the technology itself but in the planning, processes, and expectations that govern AI initiatives.
Amgen’s Chief Technology Officer, Sean Bruich, emphasized that not all AI projects are suited for widespread deployment. While pilot programs encourage innovation, successful scaling requires stringent governance to determine which initiatives should proceed. Bruich advocates for a balance between fostering creativity in pilot projects and maintaining tight control over which projects receive a green light for expansion.
Salesforce’s Chief Customer and Commercial Officer, Lashonda Anderson-Williams, articulated the necessity of having a clear understanding of the intended outcomes of AI projects. Many companies mistakenly prioritize the implementation of AI features—focusing on the technological aspects—over achieving tangible business results. This misalignment can lead to unsuccessful deployments where the technology functions well but fails to drive meaningful business impact.
A significant factor in the success of AI projects is the understanding of workflows. Anderson-Williams pointed out that many organizations lack comprehensive documentation of the processes involved in their operations. Without this clarity, the introduction of AI can lead to unrealistic expectations of “magic” results, when in reality, foundational issues remain unaddressed.
Data accessibility is another common hurdle encountered when transitioning from pilot to full-scale deployment. Organizations often store data across various silos, each governed by different access permissions and privacy regulations. To mitigate these challenges, panelists stressed the importance of mapping out the necessary data landscape before embarking on an AI project. Caitlin Halferty, Chief Data Officer at Thomson Reuters, highlighted the importance of early discovery in identifying potential data requirements, which can significantly bolster the chances of success in AI initiatives.
Involving the right stakeholders is crucial for the success of AI projects. Halferty pointed out the need to engage individuals responsible for privacy, security, and data governance if the project involves personally identifiable information (PII) or other confidential data. This proactive approach ensures that all potential issues are addressed before they become impediments.
Bruich reiterated the importance of broad organizational buy-in for transformative AI projects. He noted that any project with the potential to significantly impact a company will inherently involve collaboration across multiple departments, including finance, technology, and human resources. Successful AI initiatives should aim not just to enhance efficiency for a select group but to create outcomes that have substantial value for the entire enterprise.
In summary, the roundtable discussions at Fortune Brainstorm Tech illuminated several critical aspects of successfully implementing AI in organizations. These include the need for rigorous governance in pilot projects, a focus on desired business outcomes rather than just technological features, a solid understanding of workflows, ensuring data accessibility, and obtaining broad stakeholder engagement. By addressing these elements, companies can enhance their chances of successfully scaling AI initiatives and achieving meaningful results.
