Steven Ringer, MD, Ph.D., Director of Newborn Services, Brigham and Women's Hospital, Harvard Medical School, Boston MA Richard Slavin, Technical Director of Neonatal Respiratory Therapy, Brigham and Women's Hospital, Harvard Medical School, Boston MA Marjorie L Icenogle, Ph.D., Assistant Professor of Management University of South Alabama, Mobile, Alabama Steven Zimmerman, Ph.D., VP Research, Biomedical Quality Control of America, Inc., Mobile, Alabama
ABSTRACT
Excessive false alarms are a clinical monitoring problem. In particular, SaO2 monitoring is subject to a high false alarm rate. The Nellcor (Symphony) 3000tm was developed as a potential solution to eliminate excessive false alarms. The Nellcor 200 tm is representative of the products available prior to the Nellcor 3000 tm. When Nellcor introduced the Nellcor Symphony ™ N-3000 they claimed the use of data filtering procedures to identify patient movement and other artifacts would reduce false alarms and help to stabilize the data. Statistical process control(SPC)analysis indicates that the Nellcor monitor accomplishes its objective of giving the clinical decision-maker a more stable data set.
All patient (process) control procedures have two types of errors the:
The procedures built into the Nellcor Symphony ™ N-3000 reduce the Type I error by filtering out the effects of movement and other artifacts on the data. The objective of this paper is to compare the data generated by the N-3000 and N-200 from the same individual and to examine the differences in data behavior. The SPC methods of process control charts for averages and standard deviations are used to study data behavior.
INTRODUCTION
Clinical data collection is often a matter of selecting what can be measured easily with minimum danger (non-invasive) and minimum discomfort to the patient. Pulse-oximeters such as the Nellcor N-3000 and N-200 are both non-invasive and cause minimum discomfort and danger to the patient. Current clinical practice requires the use of an alarm based on a single data point beyond the specification limits. When the number of false alarms is excessive, there is a possibility that some caregivers will ignore the alarms. The objective of current clinical monitoring research has been to reduce the number of false alarms. The N-3000 is a product designed to fulfill this need.
STATISTICAL PROCESS CONTROL
Statistical process control (SPC) is a standard of practice for controlling industrial processes. Based on the assumption that the human body is a process like any other process, SPC may be used to control a patient's vital signs. SPC identifies the changes in a process. In this study SPC is utilized to evaluate the differences in the behavior of the observations generated from the same individual using both the N-3000 and N-200 at exactly the same time.
One important feature of SPC is that the user can balance the probability of false positives and false negatives. A false positive is a Type I error, looking for a change when no change occurred and a false negative is a Type II error, not looking for a change when a change has occurred. Type I and Type II errors exist in all measuring control systems. A reduction in the Type I error results in an increase in the Type II error. The "improvements" made in the N-3000 relative to the N-200 reduces the Type I at the expense of increasing Type II errors; and when the N-3000 is used the caregiver has no control over the balance. Similar results to the N-3000 can be obtained by using SPC in combination with the N-200. The advantage of using SPC and the N-200 is that the caregiver retains control of the probability of Type I and Type II errors.
ANALYSIS PROCEDURE
A non-patient(a healthy male) was connected to an N-3000 and N-200. Each Nellcor monitor was connected to a microcomputer using a custom cable. Each microcomputer was loaded with custom software for data collection, recording, and analysis purposes. A "large" number of data files were collected using both devices connected to the non-patient. During one series of sessions SpO2 and heart rate were collected as the non-patient simply sat with no movement. During a second series of data collection sessions the non-patient was instructed to:
After the data were collected and recorded, the files for each data stream (SpO2 and heart rate) were displayed for the purpose of detailed examination and analysis. Each vital sign was uniquely different and will be examined individually. Figure 1 shows a single non-patient (hand) with two finger probes connected to the N-200 and the N-3000, which were both connected to a microcomputer collecting, analyzing, and graphing data in real-time.
Figure 1 A Two-finger Test
SPO2 N-3000 VERSUS N-200
For healthy individuals SpO2 data are relatively stable and are often highly autocorrelated because the patient is usually near a state of homeostasis. Data are autocorrelated when an observation is related to its prior value plus or minus a small random change. By definition, the more an observation is related to its prior reading, the higher the autocorrelation. A healthy patient is often in a steady state condition or status (hemostats). A balanced system such as a patient in homeostasis may be expected to produce autocorrelated data. In addition, the faster the data collection, the more autocorrelated the data.
Figure 2 illustrates SPC charts for a data collection session with no movement. The top two charts (a control chart for averages and a control chart for standard deviations) are for the N-200 and the bottom two charts (averages and standard deviations) are for the N-3000. The data collection rate of the N-200 was heart-beat to heart-beat. The data collection rate of the N-3000 was once per second, so there are fewer data observations in the N-3000 chart than in the N-200 chart. Generally the N-200 set at heart-beat to heart-beat data collection will generate more data then the N-3000 at one observation per second, however, the communication procedure also limits the data collection rate. The time period covered by the data on the screen was from 15:27:16 to 15:40:48 or approximately 13 minutes, the amount of time to fill the screen with data using the N-200 and a subgroup size of n=5. The beginning time: 15:27:16 is shown in Figure 2, however, the ending time is not because the bottom file, which shows the data collected by the N-3000 stopped recording data before the time expired for the N-200. All four charts shown in Figure 2 display the same time period.
Figure 2 SaO2-Oxygen Saturation N-3000 versus N-200
Stable System of Chance Causes
The control charts illustrated in Figure 2 did not indicate any changes in the non-patient's vital signs. The software changes the background color to yellow when a change in a vital sign occurs. The only yellow background on the chart was for the initial calculation time period. A total of 23 subgroups of size n=5 (in this example) or a total of 115 observations were used to calculate the patient's initial control chart statistics. The red background at the end of the bottom chart indicates that the end of the data set was reached. A red background also means that a one-digit number is being received when a two-digit number was expected. Since the subject was a healthy male, no changes in vital signs were expected.
Movement- An Unstable System
Movement affects SaO2 measurements. Figure 3 illustrates a screen for the non-patient subject when movement was introduced. The control chart pair displayed in the top of the screen is for the Nellcor N-200. The control chart pair displayed in the bottom of the screen is for the Nellcor N-3000. Looking at the very bottom of the screen, the 1|'s on the bottom of the screen indicate when movement started and stopped for the N-3000 data and the 2|'s indicate when the movement started and stopped for the N-200 data. In neither case did the control chart indicate a need to recalculate control limits. The drop in SaO2 can be more clearly observed from the data generated from the N-3000 than the N-200.
Figure 3 Movement SaO2
HEART RATE N-3000 VERSUS N-200
Heart rates tend to change faster and to react to the environment more than SaO2. Examples of the Type I and Type II errors due to heart rate are shown below.
Type I Error
Figure 4 illustrates process control chart pairs for heart rate data for a non-patient for the N-200 (top) and the N-3000 (bottom). The control chart for the N-200 illustrates a Type I error, a false positive. The control chart indicated a change, when no change had occurred. Although the researchers know that heart rate has not changed, the N-200 indicates more changes than the N-3000. This is due to in part to the fact that the N-200 generates more data and therefore, has a higher probability of observing a run with observations beyond the control limits. The N-200 also shows more changes than the N-3000 because the N-3000 has improved filtering of "bad" data.
Figure 4 Heart Rate No Changes
Type II Errors
A Type II error is not looking for the reason for a change when a change has occurred. Figure 5 illustrates two control chart pairs when movement is introduced. The graph clearly shows the motion on both the N-200 and N-3000, yet in no change is indicated on the control chart for the N-3000. The control chart missed the change because the change was not sustained an adequate length of time. This error occurs more often on the N-3000 because it collects fewer data observations and because it filters the data. We know of no measurement system where you can reduce the Type I error without increasing the Type II error.
Figure 5 II Type II Errors
CONCLUSION
When using the N-3000, there is a lower probability of making a Type I error, finding false positives, than when using the N-200. When using the N-3000, there is a higher probability of missing events, making a Type II error, than when using the N-200. The filtering procedures inside the N-3000 reduces Type I errors, but increases Type II errors. The control of the balance between Type I and II errors is taken over by Nellcor and the caregiver is given no options.
REFERENCES
1. Laffel, Glenn, PhD., M.D., Robert Luttman, and Steven M. Zimmerman, Ph.D. (1993), "Using Control Charts to Analyze Serial Patient?Related Data,", accepted for publication Quality Management in Health Care Fall 1994.
2. Al Pfadt and Donald J. Wheeler (1993), "Control Charts?Powerful Tools in a Clinical Setting," SPC Ink, 1?4.
3. P Plsek (1992), "Introduction to control charts," Quality Manage Health Case. 65?73.
4. Zimmerman, Steven M., Robert N. Zimmerman, Lonnie D. Brown, and Shannon S. Brown, (1992) "Using Moving Average Process Control Charts in Biomedical Applications," Proceedings? Ninth International Conference of the Israel Society of Quality Assurance, 1992, November 1992, 761-764.
5. Zimmerman, Steven M., Lonnie D. Brown, Shannon S. Brown, and Richard L Goldhamer, M.D. (1990), "Quality Control Charts for Patient Data." The 8th International Conference of Israel Society for Quality Assurance Transactions November 26-29, 1990 Jerusalem.
6. Zimmerman, Steven M., Lonnie Brown, Shannon Brown, and Robert N. Zimmerman (1992), "Using the Theory of Runs in a Biomedical Application," 46th Annual Quality Control Congress Transactions May 18-20, p903?908.
7. Zimmerman, Steven M., Lonnie Brown, Shannon Brown, and Leroy Alexander, M.D. (1990), "Human Body Function Control Charts for the Physician," 44th Annual Quality Congress Transactions May 14--16, p408-412. May 15, 1994 .
