By Aly Brown –
Registered nurses and members of the California Nurses Association on Monday rallied at the foot of Kaiser Permanente San Francisco Medical Center to highlight their concerns about artificial intelligence and its impact on patient safety.
“We’re protesting to make sure that the hospital corporations, one of them being Kaiser, and the regulators take a bigger pause in their big push to use AI-generative systems in patient care,” said Michelle Gutierrez Vo, CNA president and longtime registered nurse at Kaiser Permanente Fremont Medical Center.
Two of Kaiser’s systems in particular raised red flags for nurses, she said.
The first is Epic, an acuity system used in the inpatient hospital that Gutierrez Vo said was used for predicting care needs or when patients should be discharged, along with making other health assessments.
“When patients are acutely ill in the hospital, they need a nurse that’s highly trained, who has the competencies and experiences to detect the nuances and changes in the human condition,” Gutierrez Vo said. “No machine or robot can predict and accurately assess what’s going on with the human condition.”
The other is Kaiser’s Desktop Medicine program, which categorizes patient messages. Kristine Lee, a doctor and associate executive director of Kaiser’s virtual medicine and technology, said the system was designed to combat the challenge of handling the expanding inboxes of physicians who must evaluate and respond to messages along with their other duties of caring for patients.
“Physicians in a modern health care system are managing in-person patient care alongside virtual care, such as video visits and secure messaging,” Lee said. “While doctors welcome expanded ways to connect with their patients, they also need assistance with the changing workload. Our program found that AI tools may offer tools that could help when paired with robust workflows.”
The problem Gutierrez Vo highlighted was that not every patient knows how to frame messages. She gave the example of a heart attack patient who might be discharged with the advice to seek care if they have chest pains.
“They may say, ‘I need a refill on my nitroglycerin medication, because I’m having some type of chest discomfort.’ Well, because of the word ‘medication,’ this AI might now sort out and categorize it as a medication refill and send it to the pharmacy, which is nonurgent, as opposed to if it’s sent to a nurse that’s triaging this patient message and will see it right away,” she said.
In a statement responding to the protest, Kaiser Permanente said it was empowering nurses with state-of-the-art tools and technologies that support the organization’s mission of providing high-quality, affordable health care to best meet members’ and patients’ needs.
“We have consistently invested in and embraced technology that enables nurses to work more effectively, resulting in improved patient outcomes and nurse satisfaction, and we will continue to do so,” the statement continued. “At Kaiser Permanente, artificial intelligence tools don’t make medical decisions, our physicians and care teams are always at the center of decision making with our patients. We believe that AI may be able to help our physicians and employees and enhance our members’ experience. As an organization dedicated to inclusiveness and health equity, we ensure the results from AI tools are correct and unbiased; AI does not replace human assessment.”
Gutierrez Vo said that while such technologies as Epic weren’t directly replacing nurses, corners were being cut and decisions on nurse staffing were being made, and the technology was being implemented without input from nurses.
“Nurses are all for tech that enhances our skills and the patient care experience. But what we are witnessing in our hospitals is the degradation and devaluation of our nursing practice through the use of these untested technologies. As patient advocates, we are obligated to speak out,” she said. “Our licenses are in danger because we are caring for patients but also are being forced to use these systems that are untested, unvalidated and unregulated.”
Cathy Kennedy, a registered nurse at Kaiser Permanente Roseville Medical Center and a president of CNA, said union nurses will never stop fighting for a health care system that guarantees person-to-person, hands-on care for every patient.
“We know there is nothing inevitable about AI’s advancement into health care,” Kennedy added. “No patient should be a guinea pig, and no nurse should be replaced by a robot.”
Gutierrez Vo reported that members of CNA called for hospitals to stop implementing products to test in the field with actual patients and to give nurses a seat at the discussion table.
“We demand that workers and unions be involved at every step of the development of data-driven technologies and be empowered to decide whether and how AI is deployed in the workplace,” Gutierrez Vo said.
IMHO – AI should be a tool nurses would use at their prerogative as a help – not instead of personal nurse care
“…untested, unvalidated and unregulated” raises red flags. I would be less concerned if the phrase were being applied to a coffee marketing strategy involving AI – but here patients’ access to care is potentially involved. And the term “guinea pig,” when applied to patients and AI is also of concern.
If we don’t put a limit to what we allow AI to do, we are going to be in worse situations than all the homeless ppl.
Anyone seeing the glut of AI videos on the web? The faces are often distorted, the bodies deformed. If AI can’t figure out how to represent the image of a person, how could it understand something as nuanced as health.
Side note: The pop ups on this site are out of control.