‘Black Box’ AI Puts Health Plans at Risk. Here’s What You Can Do About It

Most health plans don’t realize they’ve outsourced their reputation until a denial makes the news. In an environment where every decision can trigger an audit, an appeal, or a grievance, relying on black-box AI for utilization management (UM) is a risk few health plans can afford.

What is ‘black-box AI’?

Black-box AI refers to artificial intelligence systems whose internal workings are not transparent or easily explainable to users. These systems generate outputs without revealing the decision-making process; health plans cannot audit, question, or match decisioning to medical policies. Black-box AI is like the student who turns in an assignment without showing his work.

In contrast, transparent AI provides clear, traceable reasoning behind each output, ensuring recommendations align with clinical evidence and the appropriate medical policy. Transparent AI is necessary in UM to cultivate provider trust and ensure operational oversight.

What are the risks behind black-box AI in delegated UM?

Because clinicians and administrators cannot see how recommendations or decisions are made, black-box AI poses significant challenges to health plans.

• Health plans cannot validate or audit the logic against codified medical policy
• Providers receive no rationale for automated denials
• Regulators may flag decisions that can’t be traced or explained

The risk multiplies when delegated UM vendors deploy proprietary black-box AI. Delegation may add scale, but it subtracts visibility and control. When care is delayed or denied inappropriately, it’s the health plan’s name on the denial letter, not the delegated vendor’s.

How are regulations increasing the pressure for transparency?

The Centers for Medicare & Medicaid Services (CMS) is raising the bar on transparency and accountability. The Interoperability and Prior Authorization Final Rule (CMS-0057-F) requires FHIR-based APIs, faster turnaround times, and documented decision rationales. Similarly, the June 2025 pledge authored by the Blue Cross Blue Shield Association and AHIP (and signed by more than 50 health plans) reinforces this direction with industry-wide commitments to real-time authorization.

Delegated vendors using opaque systems are quickly becoming a liability. Surface-level compliance won’t protect health plans; adding an API to an opaque process does nothing to reduce audit risk, reputational exposure, or provider abrasion.

Why is transparent, policy-aligned AI a better alternative?

Transparent AI using real-time clinical data and codified medical policy is the way to transform prior authorization. This approach provides:

• Traceable logic for every decision
• Clinical-first recommendations
• Readiness for compliance audits

Predictive or regression-based models score and make decisions based on historical claims patterns; but clinical-first AI relies on real time clinical data and codified medical policy. Predictive models can essentially automate yesterday’s mistakes; transparent AI ensures decisions reflect today’s clinical evidence and policy guidelines.

Many delegated vendors blur the line between predictive models and true clinical decision support. Availity draws a clear line: our AI doesn’t guess—it interprets structured clinical data against your medical policy. That means no black-box scoring, no historical claims training sets; just intelligent automation that puts prior authorization in the hands of the health plan.

What does success with transparent AI look like?

It looks like fewer faxes, faster responses, and more control. One regional Blues plan implemented Availity Intelligent Utilization Management and quickly reduced incoming faxes, lowered call volume, and improved provider experiences. Most importantly, they gained real-time visibility into prior authorization activity; proving transparency and scale can coexist.

Why should health plans rethink fully delegated UM?

Many health plans are re-evaluating vendor relationships and moving toward hybrid models that preserve strategic partnerships while restoring internal oversight.

Availity enables this shift by supporting hybrid UM models where health plans can insource core functions while still leveraging delegated vendors where it makes sense.

How does Availity deliver on transparent AI?

Availity provides purpose-built infrastructure that restores control:

• Clinical-First Decisioning: Availity Intelligent Utilization Management evaluates each case using the health plan’s codified medical policy and real-time clinical data.
• Transparent Logic: Every decision is explainable and audit ready.
• Proven Results:
  • 76% of prior authorizations recommended for approval
  • Average response time under 20 seconds
  • 99% of submissions included necessary clinical documentation

Every decision is tied to codified policy, current patient data, and is fully auditable, so clinicians and compliance teams know exactly what the system is doing and why.

What’s the next step for health plans?

Black-box AI isn’t innovation—it’s a liability. Health plans need solutions that are built for better transparency, trust, and compliance at scale. Availity Intelligent Utilization Management does exactly that—helping health plans modernize prior authorization while reducing risk, abrasion, and delay.

➔ Learn how Availity’s Intelligent UM is redefining prior authorization

Matt Cunningham, EVP of Product at Availity, spent nine years in the Army in light and mechanized infantry units, including the 2nd Ranger Battalion. He brought his Army operations experience to the healthcare industry and has been focused on solving the problem of prior authorizations and utilization management for the past 15+ years. He helped scale a services company from $20M to the largest healthcare benefit services company. Matt has served as Head of Call Center Operations, Director of Product Operations, Chief Information Officer, and lead integration efforts for mergers and acquisitions.