Factors or conditions facilitating application of SPC*
We found more information on SPC benefits and facilitating factors than on limitations and barriers, and this may represent a form of publication bias, as indicated by the quote in the Introduction. 11 We did not find any study that reported failed SPC application. We can speculate that there have been situations when SPC application failed, just as there must be many cases of successful SPC application that have not been reported in the literature. Studies of failed SPC application efforts, as well as studies designed to identify successful ways to apply SPC to manage change, would help inform future SPC application efforts. On the basis of this review, we agree with the argument that “medical quality improvement will not reach its full potential unless accurate and transparent reports of improvement work are published frequently and widely (p 319),” 75 and also that the way forward is to strengthen QI research rather than to lower the bar for publication. 76
None of the studies we found was designed to evaluate the effectiveness quantitatively—that is, the magnitude of benefits—of SPC application. This would have required other study designs such as cluster randomised trials or quasi-experimental studies. 12 Although the “methods of evaluating complex interventions such as quality improvement interventions are less well described [than those to evaluate less complex interventions such as drugs]”, Eccles et al argue that the “general principle underlying the choice of evaluative design is . simple—those conducting such evaluations should use the most robust design possible to minimise bias and maximise generalisability. [The] design and conduct of quantitative evaluative studies should build upon the findings of other quality improvement research (p 47).” 77 This review can provide such a foundation for future evaluative studies.
An important distinction is warranted here: we believe that SPC rests on a solid theoretical, statistical foundation and is a highly robust method for analysing process performance. The designs of the studies included in this systematic review were, however, not particularly robust with regard to evaluating the effectiveness of SPC application, and that was not their objective. This does not mean that SPC is not a useful tool for QI in healthcare, only that the studies reviewed here were more vulnerable to bias than more robust study designs, even if they do indicate many clear benefits of SPC application (table 6). Despite the studies not being designed to evaluate the effectiveness of SPC, many used SPC to effectively show the impact of QI or other change initiatives. In this way, SPC analysis can be just as powerful and robust as study designs often deemed superior, such as randomised controlled trials (RCTs). 77 The key to this power is the statistical and practical ability to detect significant changes over time in process performance when applying SPC. 9 On the basis of a theoretical comparison between control charts and RCTs, Solodky et al 38 argue that control charts can complement RCTs, and sometimes even be preferable to RCTs, since they are so robust and enable replication—“the gold standard” for research quality—at much lower cost than do RCTs. These points have been further elaborated in subsequent work. 78, 79
A curious methodological wrinkle in our review is: can you evaluate the application of a method (eg, SPC) using that same method for the evaluation? Several of the included studies used SPC both as (part of) an intervention and as a method to evaluate the impact of that intervention. For example, Curran et al used annotated control charts to feed information on MRSA acquisition rates back to stakeholders and used these same control charts to show the effectiveness of the feedback programme. 61
When SPC is applied for monitoring, rather than for managing change, the aims are different—for example, to detect even small but clinically important deviations in performance—as are the methodological challenges. 80, 81 This review focused on the latter. Thus although studies on SPC application for monitoring healthcare performance were excluded from this review, we recognise the importance of such monitoring. The demarcation between monitoring and improvement is not absolute. Indeed, there are important connections between measurement, monitoring and improvement, even if improvement does not follow automatically from indications of dissatisfactory performance. “To improve performance, organizations and individuals need the capability to control, improve, and design processes, and then to monitor the effects of this improvement work on the results. Measurement alone will not suffice (pp 1–35).” 82
Monitoring performance by way of control charts has been suggested as a better approach to clinical governance in the British National Health Service. Through six case studies, Mohammed et al demonstrate how control chart monitoring of performance can distinguish normal performance from performance that is either substandard or better than usual care. “These case studies illustrate an important role for Shewhart’s approach to understanding and reducing variation. They demonstrate the simplicity and power of control charts at guiding their users towards appropriate action for improvement (p 466).” 83
No search strategy is perfect, and we may well have missed some studies where SPC was applied to healthcare QI. There are no SPC specific keywords (eg, Medical Subject Headings, MeSH) so we had to rely on text words. Studies not containing our search terms in the title or abstract could still be of potential interest although presumably we found most of the articles where SPC application was a central element. We believe the risk that we systematically missed relevant studies to be small. Therefore, our findings would probably not have changed much due to such studies that we might have missed.
The review draws on our reading, interpretation and selection of predominantly qualitative data—in the form of text and figures—in the included articles to answer the questions in our data abstraction form. The questions we addressed, the answers we derived from the studies, and the ways we synthesised the findings are not the only ways to approach this dataset. Furthermore, each member of the review team brought different knowledge and experiences of relevance to the review, potentially challenging the reliability of our analysis. An attempt was made to reduce that risk by having one investigator read all data abstraction forms, and obtain clarifications or additional data from the original articles when needed. That investigator also conducted the initial data synthesis, which was then reviewed by the entire team and the two outside experts. Although other interpretations and syntheses of these data are possible, we believe that ours are plausible and hope they are useful.
The methods for reviewing studies based primarily on qualitative data in healthcare are less well developed than the more established methods for quantitative systematic reviews, and they are in a phase of development and diversification. 13, 84, 85 Among the different methods for synthesising evidence, our approach is best characterised as an interpretive (rather than integrative) review applying thematic analysis—it “involves the identification of prominent or recurrent themes in the literature, and summarising the findings of different studies under thematic headings”. 86 There is no gold standard for how to conduct reviews of primarily qualitative studies. Our response to this uncertainty has been to use the best ideas we could find, and to be explicit about our approach to allow readers to assess the findings and their provenance.
The main limitation of this review is the uncertainty regarding the methodological quality of many of the primary studies. Assessment of quality of qualitative studies is still under debate, and there is no consensus on whether at all, or, if so, how to conduct such assessments. 84 We reviewed all the studies that satisfied our inclusion criteria and made no further quality assessment. Therefore our findings should be considered as tentative indications of benefits, limitations, etc to be corroborated, or rejected, by future research. The main strength of this review is our systematic and explicit approach to searching and including studies for review, and to data abstraction using a standardised form. It has helped generate an overview of how SPC has been applied to healthcare QI with both breadth and depth—similar to the benefits of thematic analysis reported by investigators reviewing young people’s views on health and health behaviour. 87
In conclusion, this review indicates how SPC has been applied to healthcare QI with substantial benefits to diverse stakeholders. Although there are important limitations and barriers regarding its application, when applied correctly SPC is a versatile tool which can enable stakeholders to manage change in healthcare and improve patients’ health.
Database search strategy
Web of Science (1986 – 11 June 2004)
TS [topic search] = ((statistical process control or statistical quality control or control chart* or (design of experiment and doe)) and (medical or nurs* or patient* or clinic* or healthcare or health care))
We limited the search to articles in English only which reduced the number of hits from 167 to 159. We saved these 159 titles with abstracts in an EndNote library. Using a similar strategy, we searched the following databases through Ovid:
This yielded 287 hits, including many duplicates, which we saved in the same EndNote library as above.
Centre for Reviews and Dissemination (CRD)
We searched all CRD databases and found two articles which we also added to our EndNote library.
We thank Ms Christine Wickman, Information Specialist at the Karolinska Institutet Library, for expert assistance in conducting the database searches. We also acknowledge the pilot work conducted by Ms Miia Maunuaho as a student project at Helsinki University, supervised by Professor Brommels, which provided a starting point for this study. We thank Professor Duncan Neuhauser, Case Western Reserve University, Cleveland, Ohio, USA, and Professor Bo Bergman, Chalmers University of Technology, Gothenburg, Sweden, for their helpful comments on an earlier version of this manuscript. We thank Dr Rebecca Popenoe for her editorial assistance.
Burnes B. Managing change: a strategic approach to organisational dynamics , 3rd edn. London: Financial Times/Prentice Hall, 2000 .
Wheeler D J, Chambers D S. Understanding statistical process control , 2nd edn. Knoxville, TN: SPC Press, 1992 .
Fisher L M, Humphries B L. Statistical quality control of rabbit brain thromboplastin for clinical use in the prothrombin time determination. Am J Clin Pathol 1966 ; 45 : 148 –52.
Carey R G. Improving healthcare with control charts: basic and advanced SPC methods and case studies . Milwaukee: ASQ Quality Press, 2003 .
Daniels S, Johnson K, Johnson C. Quality glossary. 2006 [cited 28 June 2006], http://www.asq.org/glossary/s.html.
Blumenthal D. Applying industrial quality management science to physicians’ clinical decisions. In: Blumenthal D, Scheck A, eds. Improving clinical practice: total quality management and the physician . San Francisco: Jossey-Bass Publishers, 1995 : 25 –49.
Berwick D M. A primer on leading the improvement of systems. BMJ 1996 ; 312 : 619 –22.Berwick D M. Controlling variation in health care: a consultation from Walter Shewhart. Med Care 1991 ; 29 : 1212 –25.
Benneyan J C, Lloyd R C, Plsek P E. Statistical process control as a tool for research and healthcare improvement. Qual Saf Health Care 2003 ; 12 : 458 –64.
Plsek P E. Quality improvement methods in clinical medicine. Pediatrics 1999 ; 103 (1 Suppl E) : 203 –14.
Wilcock P M, Thomson R G. Modern measurement for a modern health service. Qual Health Care 2000 ; 9 : 199 –200.
Grimshaw J, McAuley L M, Bero L A, et al. Systematic reviews of the effectiveness of quality improvement strategies and programmes. Qual Saf Health Care 2003 ; 12 : 298 –303.
Mays N, Roberts E, Popay J. Synthesising research evidence. In: Fulop N, Allen P, Clarke A, et al, eds. Studying the organisation and delivery of health services: research methods . London: Routledge, 2001 : 188 –220.
Fulop N, Allen P, Clarke A, et al. Issues in studying the organisation and delivery of health services. In: Fulop N, Allen P, Clarke A, et al, eds. Studying the organisation and delivery of health services: research methods . London: Routledge, 2001 : 1 –23.
Re R N, Krousel-Wood M A. How to use continuous quality improvement theory and statistical quality control tools in a multispecialty clinic [see comment]. Qrb Qual Rev Bull 1990 ; 16 : 391 –7.
Schnelle J F, Newman D R, Fogarty T E, et al. Assessment and quality-control of incontinence care in long-term nursing facilities. J Am Geriatr Soc 1991 ; 39 : 165 –71.
Bluth E I, Havrilla M, Blakeman C. Quality improvement techniques: value to improve the timeliness of preoperative chest radiographic reports. AJR Am J Roentgenol 1993 ; 160 : 995 –8.
McKenzie L. Process management: two control charts. Health Care Superv 1993 ; 12 : 70 –81.Rollo J L, Fauser B A. Computers in total quality management. Statistical process control to expedite stats. Arch Pathol Lab Med 1993 ; 117 : 900 –5.
Dey M L, Sluyter G V, Keating J E. Statistical process control and direct care staff performance. J Ment Health Adm 1994 ; 21 : 201 –9.
Guinane C S, Sikes J I, Wilson R K. Using the PDSA cycle to standardize a quality assurance program in a quality improvement-driven environment. Jt Comm J Qual Improv 1994 ; 20 : 696 –705.
Laffel G, Luttman R, Zimmerman S. Using control charts to analyze serial patient-related data. Qual Manag Health Care 1994 ; 3 : 70 –7.
Nelson F E, Hart M K, Hart R F. Application of control chart statistics to blood pressure measurement variability in the primary care setting. J Am Acad Nurse Pract 1994 ; 6 : 17 –28.
Carey R G, Teeters J L. CQI case-study—reducing medication errors. Jt Comm J Qual Improv 1995 ; 21 : 232 –7.
Oniki T A, Clemmer T P, Arthur L K, et al. Using statistical quality control techniques to monitor blood glucose levels. Proc Annu Symp Comput Appl Med Care 1995 : 586 –90.
Shaha S H. Acuity systems and control charting. Qual Manag Health Care 1995 ; 3 : 22 –30.Ziegenfuss J TJr, McKenna C K. Ten tools of continuous quality improvement: a review and case example of hospital discharge. Am J Med Qual 1995 ; 10 : 213 –20.
Johnson C C, Martin M. Effectiveness of a physician education program in reducing consumption of hospital resources in elective total hip replacement. South Med J 1996 ; 89 : 282 –9.
Piccirillo J F. The use of patient satisfaction data to assess the impact of continuous quality improvement efforts. Arch Otolaryngol Head Neck Surg 1996 ; 122 : 1045 –8.
Shahian D M, Williamson W A, Svensson L G, et al. Applications of statistical quality control to cardiac surgery. Ann Thorac Surg 1996 ; 62 : 1351 –8 discussion 89.
Tsacle E G, Aly N A. An expert system model for implementing statistical process control in the health care industry. Computers & Industrial Engineering 1996 ; 31 : 447 –50.
Cornelissen G, Halberg F, Hawkins D, et al. Individual assessment of antihypertensive response by self-starting cumulative sums. J Med Eng Technol 1997 ; 21 : 111 –20.
Linford L H, Clemmer T P, Oniki T A. Development of a blood glucose protocol using statistical quality control techniques. Nutr Clin Pract 1997 ; 12 : 38 –41.
Ornstein S M, Jenkins R G, Lee F W, et al. The computer-based patient record as a CQI tool in a family medicine center. Jt Comm J Qual Improv 1997 ; 23 : 347 –61.
Boggs P B, Wheeler D, Washburne W F, et al. Peak expiratory flow rate control chart in asthma care: chart construction and use in asthma care. Ann Allergy Asthma Immunol 1998 ; 81 : 552 –62.
Konrad C, Gerber H R, Schuepfer G, et al. Transurethral resection syndrome: effect of the introduction into clinical practice of a new method for monitoring fluid absorption. J Clin Anesth 1998 ; 10 : 360 –5.
Nelson E C, Splaine M E, Batalden P B, et al. Building measurement and data collection into medical practice. Ann Intern Med 1998 ; 128 : 460 –6.
Solodky C, Chen H G, Jones P K, et al. Patients as partners in clinical research—a proposal for applying quality improvement methods to patient care. Med Care 1998 ; 36 : AS13 –20.
Vitez T S, Macario A. Setting performance standards for an anesthesia department. J Clin Anesth 1998 ; 10 : 166 –75.
Boggs P B, Hayati F, Washburne W F, et al. Using statistical process control charts for the continual improvement of asthma care. Jt Comm J Qual Improv 1999 ; 25 : 163 –81.
Konrad C, Gerber H, Schupfer G, et al. Detection of fluid volume absorption by end-tidal alcohol monitoring in patients undergoing endoscopic renal pelvic surgery. J Clin Anesth 1999 ; 11 : 386 –90.
Levett J M, Carey R G. Measuring for improvement: from Toyota to thoracic surgery. Ann Thorac Surg 1999 ; 68 : 353 –8.
Pollard J B, Garnerin P. Outpatient preoperative evaluation clinic can lead to a rapid shift from inpatient to outpatient surgery: a retrospective review of perioperative setting and outcome. J Clin Anesth 1999 ; 11 : 39 –45.
Purba M. Use of a control chart to monitor diarrhoea admissions: a quality improvement exercise in West Kalimantan Provincial Hospital, Pontianak, Indonesia. J Qual Clin Pract 1999 ; 19 : 145 –7.
Schwab R A, DelSorbo S M, Cunningham M R, et al. Using statistical process control to demonstrate the effect of operational interventions on quality indicators in the emergency department. J Healthc Qual 1999 ; 21 : 38 –41.
Bonetti P O, Waeckerlin A, Schuepfer G, et al. Improving time-sensitive processes in the intensive care unit: the example of “door-to-needle time” in acute myocardial infarction. Int J Qual Health Care 2000 ; 12 : 311 –7.
Ratcliffe M B, Khan J H, Magee K M, et al. Collection of process data after cardiac surgery: Initial implementation with a Java-based intranet applet. Ann Thorac Surg 2000 ; 69 : 1817 –21.
Saturno P J, Felices F, Segura J, et al. Reducing time delay in the thrombolysis of myocardial infarction: an internal quality improvement project. Am J Med Qual 2000 ; 15 : 85 –93.
Sinanan M, Wicks K, Peccoud M, et al. Formula for surgical practice resuscitation in an academic medical center. Am J Surg 2000 ; 179 : 417 –21.
Staker L V. Changing clinical practice by improving systems: the pursuit of clinical excellence through practice-based measurement for learning and improvement. Qual Manag Health Care 2000 ; 9 : 1 –13.
Alemi F, Oliver D W. Tutorial on risk-adjusted P-charts. Qual Manag Health Care 2001 ; 10 : 1 –9.Alemi F, Sullivan T. Tutorial on risk adjusted X-bar charts: applications to measurement of diabetes control. Qual Manag Health Care 2001 ; 9 : 57 –65.
Aronsky D, Kendall D, Merkley K, et al. A comprehensive set of coded chief complaints for the emergency department. Acad Emerg Med 2001 ; 8 : 980 –9.
Caron A, Neuhauser D V. Health care organization improvement reports using control charts for key quality characteristics: ORYX measures as examples. Qual Manag Health Care 2001 ; 9 : 28 –39.
Quinn D C, Graber A L, Elasy T A, et al. Overcoming turf battles: developing a pragmatic, collaborative model to improve glycemic control in patients with diabetes. Jt Comm J Qual Improv 2001 ; 27 : 255 –64.
Roman S H, Chassin M R. Windows of opportunity to improve diabetes care when patients with diabetes are hospitalized for other conditions. Diabetes Care 2001 ; 24 : 1371 –6.
Boelle P Y, Bonnet F, Valleron A J. An integrated system for significant anaesthetic events monitoring. J Am Med Inform Assoc 2002 ; 9 : S23 –7.
Burnett L, Chesher D, Burnett J R. Optimizing the availability of “stat” laboratory tests using Shewhart “C” control charts. Ann Clin Biochem 2002 ; 39 : 140 –4.
Carey R G. Improving patient satisfaction: a control chart case study. J Ambul Care Manage 2002 ; 25 : 78 –83.
Chu J, Neuhauser D V, Schwartz I, et al. The efficacy of automated/electrical twitch obtaining intramuscular stimulation (atoims/etoims) for chronic pain control: evaluation with statistical process control methods. Electromyogr Clin Neurophysiol 2002 ; 42 : 393 –401.
Curran E T, Benneyan J C, Hood J. Controlling methicillin-resistant Staphylococcus aureus: A feedback approach using annotated statistical process control charts. Infect Control Hosp Epidemiol 2002 ; 23 : 13 –8.
Marshall T, Mohammed M A, Lim H T. Understanding variation for clinical governance: an illustration using the diagnosis and treatment of sore throat. Br J Gen Pract 2002 ; 52 : 277 –83.
Stewart L J, Greisler D. Measuring primary care practice performance within an integrated delivery system: a case study. J Healthc Manag 2002 ; 47 : 250 –61.
Fasting S, Gisvold S E. Statistical process control methods allow the analysis and improvement of anesthesia care. Can J Anaesth 2003 ; 50 : 767 –74.
Hyrkas K, Lehti K. Continuous quality improvement through team supervision supported by continuous self-monitoring of work and systematic patient feedback. J Nurs Manag 2003 ; 11 : 177 –88.
Marshall T, Mohammed M A. Understanding variation in quality improvement: the treatment of sore throats in primary care. Fam Pract 2003 ; 20 : 69 –73.
Mertens W C, Higby D J, Brown D, et al. Improving the care of patients with regard to chemotherapy-induced nausea and emesis: the effect of feedback to clinicians on adherence to antiemetic prescribing guidelines. J Clin Oncol 2003 ; 21 : 1373 –8.
Norberg A, Christopher N C, Ramundo M L, et al. Contamination rates of blood cultures obtained by dedicated phlebotomy vs intravenous catheter. JAMA 2003 ; 289 : 726 –9.
Rosow E, Adam J, Coulombe K, et al. Virtual instrumentation and real-time executive dashboards. Solutions for health care systems. Nurs Adm Q 2003 ; 27 : 58 –76.
Alemi F, Neuhauser D. Time-between control charts for monitoring asthma attacks. Jt Comm J Qual Saf 2004 ; 30 : 95 –102.
Hart M K, Robertson J W, Hart R F, et al. Application of variables control charts to risk-adjusted time-ordered healthcare data. [comment]. Qual Manag Health Care 2004 ; 13 : 99 –119.
Nelson E C, Batalden P B, Huber T P, et al. Microsystems in health care: Part 1. Learning from high-performing front-line clinical units. Jt Comm J Qual Improv 2002 ; 28 : 472 –93.
Olsson J, Elg M, Molfenter T. Reflections on transnational transferability of improvement technologies—a comparison of factors for successful change in the United States and northern Europe. Qual Manag Health Care 2003 ; 12 : 259 –69.
Thor J, Wittlov K, Herrlin B, et al. Learning helpers: how they facilitated improvement and improved facilitation—lessons from a hospital-wide quality improvement initiative. Qual Manag Health Care 2004 ; 13 : 60 –74.
Davidoff F, Batalden P. Toward stronger evidence on quality improvement. Draft publication guidelines: the beginning of a consensus project. Qual Saf Health Care 2005 ; 14 : 319 –25.
Pronovost P, Wachter R. Proposed standards for quality improvement research and publication: one step forward and two steps back. Qual Saf Health Care 2006 ; 15 : 152 –3.
Eccles M, Grimshaw J, Campbell M, et al. Research designs for studies evaluating the effectiveness of change and improvement strategies. Qual Saf Health Care 2003 ; 12 : 47 –52.
Diaz M, Neuhauser D. Pasteur and parachutes: when statistical process control is better than a randomized controlled trial. Qual Saf Health Care 2005 ; 14 : 140 –3.
Neuhauser D, Diaz M. Quality improvement research: are randomised trials necessary? Qual Saf Health Care 2007 ; 16 : 77 –80.
Benneyan J C, Borgman A D. Risk-adjusted sequential probability ratio tests and longitudinal surveillance methods [see comment] [comment]. Int J Qual Health Care 2003 ; 15 : 5 –6.
Lim T O. Statistical process control tools for monitoring clinical performance. Int J Qual Health Care 2003 ; 15 : 3 –4.
Berwick D M, James B, Coye M J. Connections between quality measurement and improvement. Med Care 2003 ; 41 (1 Suppl) : I30 –8.
Mohammed M A, Cheng K K, Rouse A, et al. Bristol, Shipman, and clinical governance: Shewhart’s forgotten lessons. Lancet 2001 ; 357 : 463 –7.
Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy 2005 ; 10 (Suppl 1) : 6 –20.
Pawson R, Greenhalgh T, Harvey G, et al. Realist review—a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy 2005 ; 10 (Suppl 1) : 21 –34.
Dixon-Woods M, Agarwal S, Jones D, et al. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy 2005 ; 10 : 45 –53.
Harden A, Garcia J, Oliver S, et al. Applying systematic review methods to studies of people’s views: an example from public health research. J Epidemiol Community Health 2004 ; 58 : 794 –800.