Fawad Butt doesn’t mince words when talking about the healthcare AI boom.
As CEO of Penguin Ai, he has watched the market become flooded with startups promising transformative technology — but he believes much of the industry’s messaging is more hype than substance.
Butt sits at the helm of a Palo Alto-based startup selling an AI platform aimed at automating costly administrative tasks like prior authorization and medical coding. Before launching Penguin Ai in 2024, he had held C-suite data leadership roles at both Kaiser Permanente and UnitedHealthcare, as well as served as an advisor and board member for several health tech startups.
Drawing on these years of experience building technology for better healthcare workflows, Butt has developed a candid perspective on what’s actually driving progress in the sector. Below are four hot takes he shared during an interview earlier this month at the HIMSS conference in Las Vegas.
Healthcare AI startups lack real technical differentiation right now.
The market for healthcare AI products is getting more and more crowded, and rapid advances in cloud tools and large language models make it easy for new startups to quickly replicate what others build, Butt pointed out.
“For customers, they’re seeing 30 companies come in and pitch the exact same thing, so they’re confused as hell. I think, if the question is, what’s the differentiation that can sustain it, then I think the answer is, I have no freaking idea. Honestly speaking, I think whoever is telling you that they have differentiation right now is making it up,” he declared.
In this day and age, sustainable advantages are more likely to come from go-to-market strategy, branding, trust, distribution and customer support, not technology itself, Butt said.
Data leakage may be accelerating AI model improvement.
Knowledge and performance among healthcare AI models had hit somewhat of a plateau a few years ago — but they have gotten a lot smarter since, Butt noted. Models are getting smarter because they’re accessing more data, which he suspects might partly come from healthcare employees unintentionally feeding sensitive enterprise data into tools like ChatGPT and Claude.
“The only way you can make these models smarter is to give them more data. And my assumption is that all the public data had been consumed, and all the rest of the data was in private places like United and Humana and Elevance and Cleveland Clinic and so forth. But somehow these models keep popping up and getting smarter and smarter, and there’s only one answer. They’re getting access to some of this data somehow,” he remarked.
Agentic AI has some utility — but it struggles with complex workflows
Butt believes AI agents do a swell job of handling small, discrete tasks but struggle to coordinate more complicated workflows that require various parts working together, such as managing the full medical coding-to-claims submission process.
“I think of agents as small bits of code that can do independent things. But enterprises don’t work on small bits of code that do independent things — they work on small bits of code that collectively do large things. And I think agents can’t do that,” he explained.
Providers will have to remove the human from the loop in some cases.
As AI models continue to improve, keeping humans in oversight roles will likely become inefficient and ineffective because people start automatically approving outputs, Butt said.
“After so many of these, your brain just goes on autopilot — accept, accept, accept. So you’re not in the loop anymore,” he stated. “The more you put the human in the loop, I think the more out of the loop they become, because they get automated. Their brain is also a big computer that starts to automate the task that they’re doing, which is pretty much accepting the computer’s output.”
In the next five years or so, he expects automation to take over many administrative functions, which would allow staff to focus on clinical work where shortages persist. For instance, tasks like documentation review or insurance verification might be completed independently by AI.
Photo: Peter Dazeley, Getty Images