TYPE I ERROR FOR PATIENT CONTROL CHARTS OF NEWBORN VITAL SIGN DATA
Marjorie L. Icenogle, Ph.D., Assistant Professor of Management, University of South
Alabama, Mobile AL
Steven M. Zimmerman, Ph.D. VP or Research, Biomedical Quality Control of America, Inc.,
Mobile, Al
Abstract
This study examines the rules for recalculating control limits when Shewhart-type patient
control charts with control limits adjusted for autocorrelation are used and
the data are collected with a heartbeat-to-heartbeat data collection
monitor. According to process control
guidelines, when a series of data points fall beyond the control limits, the
limits should be recalculated. In a
healthcare setting, the assumption is that the recalculation of control limits
provides a signal for the caregiver to examine the patient in order to identify
the cause of the change and to determine if a change in treatment is necessary. A type I error occurs when the caregiver
searches for a reason for a change, when no change has occurred, while a type
II error occurs when the caregiver does not search for a reason for a change
when a significant change has occurred.
From a clinical perspective, a pertinent research question is, “can
caregivers identify statistically significant changes in patients’ vital signs
that are also clinically significant so that important changes in vital signs
are not missed?” This study investigates the application of various rules for
recalculating control limits in a clinical setting so that clinically
significant changes are not missed. The
study shows how computer monitoring affects the timing of patient examinations
and aids caregivers in finding reasons for changes in a timely manner.
Introduction
Autocorrelation is an important issue in applying quality control charts to patient
monitoring. Autocorrelation occurs when
an observation is a function of the prior observation. Patient vital sign data go in and out of autocorrelation by
self-correcting (observations exceed the control limits and then come back
within the control limits without any caregiver intervention). Control
charts tracking patient vital sign data help caregivers identify changes that
are statistically significant
(observations that are beyond the control limits for a specified period). However, a clinically significant change is a change in a vital sign that
helps the caregiver identify the need to change the patient’s treatment.
In clinical monitoring a false indicator of a change (type I error) is an important issue. When a type I error occurs, the caregiver tries to identify the reason for the change in vital signs although no change has occurred. Another false indicator of concern is a type II error. When a type II error occurs, the caregiver does not look for a change when in fact the vital sign has changed. In clinical monitoring, the objective is to balance type I and type II errors. Therefore, the research question is:
How should vital sign data be collected so that clinically significant changes in vital signs are recognized without looking for changes when no changes have occurred?
This study examines a series of data files collected from premature babies during a period of time when there were no clinical actions taken by the caregiver. The study focuses on the rules that are used
to identify statistically and clinically significant changes in the data. A type I error is defined as the average
number of times the control chart system indicates that the caregiver should
examine the patient looking for the reason for a change when no change has
occurred. Procedures that require the
caregiver to examine the patient even when the change is a non-clinically
significant change once an hour or less is considered acceptable.
The monitor’s data collection method is critical to this study; therefore, the
researchers used a Nellcor 200 monitor, which collects data at the rate of one
observation per heartbeat. It is important to note that the results of this
study cannot be applied to another type of monitor that uses a different
sampling methodology. Sequential
observations were collected in samples of 20 heartbeats. As the 20 observations were collected, a
computer connected to the monitor calculated the average and the standard
deviation of each sample. The rate at which the control limits are reset is
determined by the caregiver. The caregiver is expected to examine the patient
whenever the control limits are reset. Therefore, the caregiver may choose to reset the control limits
when the average of one subgroup of observations is outside the control limits
or the caregiver may prefer to recalculate the limits after the averages of
several subgroups are beyond the control limits. This study examines the differences in the number of times the
caregiver would examine the patient if the following rules are applied: when a
run of 1, 3, 5, 10, 15, 20, 25, or 30 subgroups are outside the control limits.
Experimental Approach
The real-time patient control chart in this study is based on the Shewhart concept and includes an adjustment of the control limits for autocorrelation and data truncation. Autocorrelated data are data that are anchored in the past. For example, it is not possible to change from an overweight individual to an underweight individual over night. The change takes time because the weight of an individual is anchored to the past.
The data collection rate was one observation per heartbeat and the subgroup size was based on 20 observations. The rule that required control limits to be reset after a single subgroup beyond the control limits is not practical because:
When a reset rule of one point beyond the control limits was applied, the researchers detected about 5 observations beyond the control limits per hour, one every 12 minutes by chance alone, which created an unacceptable rate of type I errors. Initially, the control chart program allowed the user to select: 5, 10, 15, 20, 25, or 30 subgroups in a row as the indicator that the vital sign had changed beyond the control limits. When an outlier indicated occur the system was reset by recalculating the patient’s control charts control limits and center lines while at the same time changing the background color on the monitor from white to yellow.
Experiment
Fifty-eight (58) hours of patient vital sign data were selected from the database. The
number of resets was counted using the reset rules specified above and the
number of resets for each rule was compared. Figure 1 indicates a patient
control chart for data set: D:\z\med\h\r-pt15\14a21a0.bqc. The yellow background indicates that the
control limits and centerlines have just been recalculated (reset). The rule used was five (5) points beyond a
control limit and three (3) sigma limits.
Figure 1 Patient Control Charts for SaO2 (Oxygen saturation)
When a reset rule of one point beyond the control limits was used the screen looked like it was always being reset. Figure 2 illustrates the results using the same file as illustrated in Figure 1 with the reset rule of one observation beyond the control limits.
Figure 2 Reset rule - one point beyond control limit
Figure 3 illustrates the results in terms of resets per hour based on the number of
subgroups in a reset decision. Using a
run of five (5) subgroups beyond the control limits resulted in an average
resets rate of 1.07 per hour for the SaO2 (oxygen saturation data) and an
average resets rate of 1.01 per hour for the heart rate data. Using a run of 10 subgroups beyond the
control limits resulted in an average reset rate was 0.43 per hour for the SaO2
data and 0.53 resets per hour for the heart rate data. Using a reset rule of 15
observations beyond the control limits resulted in .29 and .38 resets per hour
for SaO2 and heart rate data. As the
number of subgroups was increased, the number of resets declined. As a result of this analysis the software
was revised to allow the user to set the reset rules using 1, 3, 5, 10, 15, or
20 subgroups in the reset decision. The
default value was set at 10 subgroups in a reset decision.
Figure 3 Resets per hour based on the number of observations beyond the control limits
When using patient control charts for more than a single parameter, the ten in a row rule is recommended because the estimate of the type I error is calculated based on the total number of control charts needed to monitor the parameters. Since the objective is to reduce the number of unnecessary trips to examine the patient based on the reset alarms, the ten in a row rule reduces the number of reset alarms, while still allowing for adequate patient monitoring.
Conclusions
Using the rule one subgroup beyond the patient control limits in this setting, results in
the caregiver examining the patient every 12 minutes or more, even though no
change in vital signs has occurred.
Using the rule of five subgroups in a row beyond the control limits, the
patient will be examined approximately once per hour. Using ten subgroups in a row beyond the control limits, the
patient will have to be examined approximately once every two hours. We
recommend the ten in a row rule, because of the current importance of the type
I error, looking for the reason for a change, when there was no change in
patient vital signs.
The final
proof of any rule is in the using.
Caregivers may try each of the rules to determine the most effective
rule for the particular monitoring situation.
The results of this study are applicable for newborns only. When monitoring adults, the analysis should
be replicated by varying the sample sizes and varying the reset rules. A method that may be used to convert these
findings to other types of patients or with newborns using a fixed time monitor
is to examine the amount of time the subgroups are out of limits rather than to
look at the number of subgroups.
Bibliography