The AI Passed the Test. But Is Your Organization Ready to Use It?

Team MDA Solutions LLC
6 hours ago
8 min read

Michele D. Alexander | Founder and CEO, MDA Solutions LLC

Healthcare organizations are investing heavily in artificial intelligence. Yet one of the most important questions about AI governance receives far too little attention:

Are we actually using the tools we already have?

Governance is often discussed as a way to control AI and prevent privacy violations, biased decisions, inaccurate results, and unauthorized applications. Those protections are essential.

But governance must also help organizations identify, implement, and responsibly use valuable technology. An approved AI tool that sits unused because no one understands it, trusts it, or knows where to find it represents another kind of governance failure.

The promise is real

AI can help clinicians recognize subtle patterns across medical images, laboratory results, clinical documentation, vital signs, medications, and patient records. In some cases, it may identify a concern before the individual clues become obvious.\

Potential applications include:

Identifying suspicious pulmonary nodules and supporting earlier cancer detection.
Prioritizing imaging that may indicate a stroke or brain hemorrhage.
Recognizing patterns associated with sepsis and clinical deterioration.
Detecting cardiovascular and heart failure risks.
Identifying chronic kidney or liver disease progression.
Finding patients who are overdue for cancer screenings or surveillance.
Closing gaps involving abnormal laboratory, imaging, and pathology results.
Recognizing patients at increased risk for maternal morbidity.
Detecting medication related harm and avoidable hospital complications.

Maternal morbidity is particularly personal to me. When I had my daughter, my physicians knew me, understood my medical history, and recognized that I was at higher risk. They had the information they needed to assess my condition and manage my care. However, that is not always what happens, and there are some risks even experienced clinicians may not be able to foresee.

Patients sometimes arrive at labor and delivery ready to deliver, and the healthcare team may have limited information about their prenatal care, medical history, previous results, or known risks. In those situations, appropriately designed AI tools could help clinicians bring together available laboratory results, vital signs, symptoms, medications, and other clinical information more quickly.

AI would not replace the judgment of the physicians, nurses, midwives, or other professionals caring for that patient. It could help them identify critical information and warning signs when time matters, particularly when the patient’s complete history is unavailable or staffing shortages affect the organization’s ability to respond.

These technologies are not hypothetical. The FDA maintains a growing list of AI enabled medical devices authorized for marketing in the United States. However, these devices remain heavily concentrated in imaging related specialties. A recent peer reviewed analysis found that approximately 76 percent of FDA authorized AI medical devices were in radiology.

FDA AI Enabled Medical Devices

JAMA Network Open analysis

Authorization, however, is not the same as successful implementation. It also does not establish that a product will work equally well in every population, organization, or clinical workflow.

The implementation gap

A recent article about the clinical AI deployment gap reported that fewer than 15 percent of purchased tools in its referenced audit were being used routinely. Because the underlying audit methodology is not publicly detailed, that figure should not be treated as a universal measurement of healthcare adoption.

Still, the operational problem it describes is very real. Healthcare organizations can acquire sophisticated technology without successfully embedding it into care delivery.

Buying AI is not the same as implementing AI.

Activating a feature is not the same as integrating it into a workflow.

Sending an announcement or tip sheet is not the same as developing a structured rollout and training clinicians, technicians, and staff to use the tool properly.

If physicians, residents, physician assistants, nurses, and other team members do not know that a capability exists, or do not understand when, why, and how to use it, the organization has not completed its implementation. The tool may technically be available, but operationally, it does not exist.

A good tool still has to fit the organization

Not every AI tool belongs in every healthcare setting.

A product may be effective for an individual provider’s office but inadequate for a medium-sized clinic. A tool designed for a clinic may not be able to support the complexity, patient volume, integrations, and governance requirements of a large hospital system. Another application may work best for an ambulatory network connected to a hospital, where clinical information can move across multiple specialties and levels of care.

A tool can be excellent and still be the wrong tool for a particular organization.

It may be too limited for the volume or complexity of the work. It may also be far more complicated and expensive than a smaller practice needs. An organization can end up paying for features it will never use while its staff struggle with a system that was not designed for their environment.

This is becoming increasingly difficult because the number of AI products is growing rapidly. Healthcare leaders are being asked to compare tools with different purposes, evidence, data requirements, integration needs, risk levels, and staffing demands.

Before selecting a product, organizations must ask:

What type of healthcare setting was this tool designed to support?
What patient population was used to develop and validate it?
Can it handle our patient volume and clinical complexity?
Does it integrate with our current systems and workflows?
Do we have the staff, infrastructure, and governance needed to support it?
Are we purchasing more technology than we need?
Are we expecting a small tool to solve a problem that requires a larger organizational solution?
Do we already own another product that performs the same function?
The goal is not simply to identify the most impressive technology. The goal is to select the technology that fits the organization, its workforce, its patients, and the problem it is trying to solve.

Public health should be part of the conversation

Public health professionals can contribute significantly to these decisions.

Public health has long examined how health interventions perform across populations, not just for one individual patient. Public health professionals also bring experience in prevention, surveillance, health equity, quality improvement, performance measurement, community needs, and population outcomes.

Those skills are directly relevant to AI selection and implementation.

Public health should not be responsible for making these decisions alone. Effective evaluation requires participation from frontline clinicians, clinical informatics, quality improvement, operations, information technology, compliance, privacy, finance, procurement, and patient representatives.

However, public health can help organizations ask questions that may otherwise be missed:

Will this tool improve outcomes across the population we serve?
Could it widen existing disparities?
Does it address a documented community or organizational need?
How will its effectiveness be measured?
Which groups might be missed or incorrectly identified?
What happens after the tool identifies a patient at risk?
Will it improve prevention and early intervention, or simply generate more alerts?
Does the organization have the resources to respond to what the tool finds?

That final question matters. There is little value in identifying hundreds of additional patients who need services if the organization has no process, capacity, or accountable owner to provide those services.

Why valuable tools remain unused

Several predictable problems contribute to underuse:

The organization does not maintain a complete inventory of its AI capabilities.
AI features are embedded inside existing systems and are not clearly identified.
The organization has purchased multiple tools that perform similar functions.
The selected product is not appropriate for the organization’s size or clinical setting.
The rollout focused on technical activation rather than clinical adoption.
Training occurred once, too early, or without scenarios specific to each role.
Clinicians do not understand how the tool fits into their existing workflow.
No one owns the response after the AI generates an alert or recommendation.
Staff members do not trust the results or understand the tool’s limitations.
The organization measures implementation completion but not actual utilization.
Alert fatigue causes clinicians to ignore potentially valuable information.
No process exists to monitor performance, overrides, disparities, or patient outcomes.

That last point is critical. An alert has no value if no one is responsible for acting on it.

Detecting an abnormal result does not improve care unless someone communicates it, arranges appropriate follow up, documents the action, and verifies that the loop was closed.

The most effective model is not autonomous diagnosis. It is AI-supported detection combined with accountable clinical workflow.

Before purchasing another tool, examine what you already own

Healthcare leaders should be able to answer several basic questions:

What AI capabilities are currently operating within our organization?
Which capabilities have been purchased but are rarely or never used?
Are multiple products performing the same or similar functions?
Is each tool appropriate for the size, setting, patient population, and complexity of our organization?
Who is authorized to use each tool?
Was the tool validated for our patients, setting, and workflow?
Were the appropriate clinical and operational teams trained before deployment?
Who owns the workflow after the tool identifies a possible risk?
Are we measuring utilization, false positives, overrides, completed follow up, outcomes, and performance across different patient populations?
Does the original implementation require a structured relaunch?

A relaunch is not necessarily an admission of failure.

Workflows change. Employees leave. New clinicians arrive. Systems are upgraded. Initial training is forgotten, especially when it takes place long before employees begin using the application.

Sometimes the responsible decision is to pause, reassess, retrain, and reintroduce the technology with clearer expectations.

Governance should make responsible adoption possible

This is where AI governance, implementation, and training must work together.

An organization should:

Base an inventory of its approved, embedded, experimental, and unauthorized AI tools.
Evaluate their clinical purpose, evidence, risks, organizational fit, and local applicability.
Authorize appropriate uses and establish clear boundaries.
Commission each tool through workflow design, testing, training for each role, and accountable ownership.
Oversee utilization, outcomes, safety signals, disparities, and unintended consequences.
Navigate changes in technology, regulation, staffing, and clinical practice.

Governance should not become a collection of policies that sits on a shelf. It should create a repeatable operating system for selecting, implementing, monitoring, and improving AI.

What is underuse costing you?

If your organization purchased an AI capability that clinicians cannot find or confidently use, what has that investment produced?

Consider both sides of the equation:

What are you paying in licensing, integration, support, and maintenance costs?
Are you paying for duplicate capabilities across different products?
Did you purchase a system that is too large, too limited, or too complicated for your organization?
How much staff time was spent implementing a tool that never became part of routine work?
Could that technology help close abnormal result loops?
Could it help identify a deteriorating patient earlier?
Could it reduce preventable emergency department visits or hospitalizations?
Could it help a patient receive screening, treatment, or follow up sooner?

The cost of an unsuccessful AI implementation is not limited to wasted technology spending. It may also include missed opportunities to improve patient care.

Healthcare organizations do not necessarily need more AI. Many first need a clearer understanding of the AI they already possess and a disciplined plan for using it responsibly.

Before purchasing the next promising application, ask: