Arthur Wallace, M.D., Ph.D., is Chief, Anesthesia Services, San Francisco Veterans Affairs Medical Center, Professor and Vice Chair of Anesthesiology and Perioperative Care, University of California, San Francisco.
Arthur Wallace, M.D., Ph.D., is Chief, Anesthesia Services, San Francisco Veterans Affairs Medical Center, Professor and Vice Chair of Anesthesiology and Perioperative Care, University of California, San Francisco.
As an anesthesia resident, I went to a code at 3 a.m. A 12-year-old girl had oral surgery the day before and someone thought she would be “more comfortable staying overnight in the hospital.” At 2 a.m., a nurse went to see her and administer morphine IM. She gave the injection without waking the child. At 3 a.m., the child’s mother, who had been sleeping in the same room, awoke to find her daughter dead. Despite our best efforts, our attempts to resuscitate the young girl were unsuccessful.
This event was devastating for everyone involved. I’ve thought about this tragedy many times when evaluating other quality improvement (QI) events. The 35-year-old father of three using a PCA who stops breathing. The 45-year-old after knee surgery who arrests. Opiate respiratory depression can lead to respiratory and cardiac arrest and monitoring every patient in the hospital is difficult for these “infrequent” events. In the VA system, 30-day mortality for in-patient surgery is around 0.5 percent or 1 in 200 patients. A standard commercial aircraft, the 737, carries 85 to 215 passengers. How would the airline industry react if one passenger died on every flight?
I’ve thought about how to monitor patients and how to develop monitors more effectively for many years. I spent a decade developing the ECOM (Endotracheal Cardiac Output Monitor) and gained experience in algorithm development, electronics, animal and clinical testing, user interface design, artificial intelligence, regulatory approval, and the complexities of funding medical device development. Medical devices require not only the engineering development of non-medical devices, they also require clinical testing to demonstrate safety and efficacy, regulatory approval from the FDA, and then marketing. There is a reason few truly new medical monitors have been developed over the years. When you review our standard monitors, most were developed prior to 1980, secondary to the cost, complexity and time requirements of development. Monitors have fancy new displays, but few have truly new classes of sensors.
Successful development of a new medical monitor takes an idea, research, time, money, engineering, intellectual property, a corporate structure, regulatory approval, marketing and some luck.
In November of 2010, Microsoft introduced Kinect, a new type of sensor that has a red, green, blue (RGB) camera, infrared (IR) camera, a depth Camera and a microphone array. The robotics and development community quickly provided interfaces to the Kinect device and Microsoft later provided the SDK interface software for this camera array. We realized that this device could provide a basis for a new class of monitors based on machine vision. Machine vision software can find the patient or subject. In 2011 we began testing the resolution of the depth camera and found that we could detect and measure respiration remotely.
Oxygen saturation (SaO2) data from 31 healthy volunteers during breath hold. Comparison between SaO2 from AVD-M monitor and finger pulse oximeter.
Oxygen saturation (SaO2) data from 31 healthy volunteers during breath hold. Comparison between SaO2 from AVD-M monitor and finger pulse oximeter.
Typical oxygen desaturation data from healthy volunteer during breath hold. AVD-M monitor, since it is measuring pulse oximetry from the face, responds more rapidly to desaturation than finger pulse oximeter.
Typical oxygen desaturation data from healthy volunteer during breath hold. AVD-M monitor, since it is measuring pulse oximetry from the face, responds more rapidly to desaturation than finger pulse oximeter.
In 2015, Adrienne Ng, a first-year medical student on a FAER Medical Student Anesthesia Research Fellowship (MSARF) working in the lab for two months, learned LabView and then wrote code to detect and measure respiratory rate in patients. In the school year of 2015, Talya Klinger, a 15-year-old high school student wrote code to extract RBG and IR video signals and detect heart rate using Eulerian Magnification algorithms. In the summer of 2016, Gopal Kodumudi, another first-year medical student on a FAER MSARF, learned LabView and wrote the code to do Fast Fourier Transforms (FFT) analysis of those video signals to extract the heart rate and respiratory rate. Shannon Haley worked in the lab in the summer of 2017 on a FAER MSARF and fixed problems with the FFT analysis getting accurate heart and respiratory rates. This work was seminal to Derek Tan’s 2018 FAER MSARF summer project to develop code to measure pulse oximetry remotely from the RGB and IR camera signals.
During their two-month FAER MSARF fellowship, each of these medical students learned computer programming, signal analysis, machine vision, and then were able to make fundamental advances in monitor development. This work gave these medical students an opportunity to contribute to the creation of a new class of monitors and to present their research at the ASA annual conference, ANESTHESIOLOGY®.
In 2018, we presented the AVD-M (AudioVisual Detection Monitor) at the Society for Technology in Anesthesia Annual Meeting in Turnberry Isle Miami, Florida, as part of FAER’s Swimming with Sharks – Technology Version. While AVD-M didn’t win the competition, we won the audience vote for best product and we gained valuable feedback on the technology. We again presented AVD-M at FAER’s Swimming with Sharks session at ANESTHESIOLOGY 2019, held in Orlando, Florida. Our “venture capitalist advisor,” Dr. Pamela Palmer, M.D., Ph.D., provided essential guidance on how to present AVD-M to venture capitalists. We won first place and made several contacts from this forum.
Successful development of a new medical monitor takes an idea, research, time, money, engineering, intellectual property, a corporate structure, regulatory approval, marketing and some luck. The process of taking a tragic QI event, identifying a possible method to prevent it, and then successfully providing that technology to hospitals is a long one. Without the work necessary to make them a reality, ideas like this are just interesting thoughts.
Dr. Wallace fields audience questions regarding the AVD-M at FAER’s Swimming with Sharks program, held at ANESTHESIOLOGY 2019.
Dr. Wallace fields audience questions regarding the AVD-M at FAER’s Swimming with Sharks program, held at ANESTHESIOLOGY 2019.
The Foundation for Anesthesia Education and Research (FAER) has provided tremendous support through MSARF student fellowships and the advice of FAER’s Swimming with Sharks program to the development of a new class of monitors using machine vision and artificial intelligence to avoid the morbidity and mortality associated with anesthesia and surgical care. The potential for this new class of remote patient monitors is enormous for early detection and prevention of desaturations, respiratory and cardiac arrest, strokes, septic shock, aspiration, sleep apnea and falls. AVD-M monitors will allow simple and inexpensive monitoring of all hospital and nursing home patients without sensors on the patient. If a patient is in the hospital, we should know if they are alive. Chronic breathing and beating of the heart are essential if we want to improve hospital safety and provide the best care possible to patients and their families.
