For much of modern history, physicians have made decisions about patient care by consulting their peers and by using rule-based approaches. Specialists have been consulted throughout history and became a necessity in the 1800s, as medicine became more complex. To reduce the need for consulting specialists for routine decision-making and to increase the likelihood that evidence-based medicine will be practiced, rule-based decision support systems have been used since the 1970s to help physicians make healthcare-related decisions.
To facilitate the practice of evidence-based medicine, multiple professional societies have established appropriateness criteria, which are rule-based algorithms for systematically assessing whether medical care is likely to be appropriate, inappropriate, or of uncertain appropriateness. The American College of Cardiology (ACC) established the ACC Foundation Appropriateness Criteria Working Group in 2005 to characterize the indications for which diagnostic studies are likely to affect a patient’s care in a positive way.1,2 Appropriateness criteria have been introduced by the ACC for many diagnostic modalities, including diagnostic catheterization, single-photon emission computed tomography (CT), cardiac magnetic resonance imaging, cardiac CT, cardiac radionuclide imaginghttp://www.ahdbonline.com/administrator/index.php?option=com_mams&view=article&layout=edit&art_id=2646#affiliations, echocardiography, and stress echocardiography.3-8 As a byproduct of these efforts, decision support tools have been established to foster evidence-based care.
The lack of definitive guidance regarding appropriate use criteria is acknowledged within the preface to the ACC appropriate use criteria for diagnostic catheterization.8 Research on the use of decision support to assist physicians in ordering noninvasive imaging for suspected coronary artery disease has revealed that a substantial portion of orders cannot be addressed by appropriate use criteria.9 Uncertainty in appropriate use has also been shown to be a concern in an evaluation of coronary CT angiography orders.10
Despite the existence of evidence-based guidelines, wide variation exists in actual clinical practice. Furthermore, substantial regional variation in diagnostic practices has been documented.11 To address this issue, prior authorization programs have been used to ensure that medical care is consistent with evidence-based practices. The prior authorization process requires ordering physicians to seek permission before delivering care and creates an opportunity for outside parties to deliver messages on evidence-based care.
The prior authorization program that is the subject of this study operates on a nondenial basis, as it modifies utilization by educating physicians rather than by restricting their activities. This educational approach has been shown to be more effective at reducing imaging utilization than a prior authorization program using denials.12 A study examining neuroimaging orders that were changed as a result of the program found that in more than 99% of the cases examined, the ordering physician did not request again the original order within 1 month after agreeing to change it.13 These findings suggest that the nondenial approach may be effective in managing utilization and that changes recommended by the program largely met the clinical needs.
Ensuring that patients are appropriately selected for cardiovascular diagnostics is one means for improving the quality of care.14 Our study examines the outcomes of a specialty benefit management company’s 2-stage approach toward ensuring appropriate selection through prior authorization. The company uses a rule-based decision support system to identify orders that are potentially appropriate and thus can be subjected to less-intensive review, and then has all the orders manually reviewed by its consult process. At minimum, every order is reviewed by a customer service representative. When needed, the consult process has nurses and physicians review the orders as well. When orders are not clearly appropriate after review by a customer service representative, nurse, and physician, the ordering physician is contacted and then engaged in a physician-led peer-to-peer consult.
This program evaluation is focused on orders for outpatient diagnostic catheterizations, because catheterization has the potential to cause harm. Diagnostic cardiac catheterizations have been reported to have a complication rate ranging from 1.35% to 1.5%.15,16 Between 1993 and 2001, cardiac catheterization utilization increased by nearly 70%, a substantial change that cannot be explained by shifts in disease prevalence.17 Although the 2012 ACC appropriate use criteria might have changed practice patterns since then, there remains a wide range of circumstances in which the criteria are not able to provide definitive guidance regarding whether catheterization is appropriate.8
Our study reports how the rule-based and consult-based components of a nondenial prior authorization program were used to evaluate the appropriateness of diagnostic catheterization orders. Previous research has found that the impact of decision support systems on the ordering of radiologic procedures differed by physician specialty.18 Therefore, the analysis was stratified by the specialty of the ordering physician. State-based variation was explored in our study, because evidence shows that state-level differences exist in practice patterns.11 Variation between health maintenance organization (HMO) and preferred provider organization (PPO) utilization was also explored in our study, because variances have been found in previous analyses of data from the health plan studied.19
Method
This program evaluation examined how orders for outpatient elective diagnostic cardiac catheterization were processed by a specialty benefit management company simultaneously using a rule-based and a consult-based system for evaluating orders. Prior authorization outcomes from the consult system were grouped according to the preliminary determination from the rule-based system. Additional subanalyses were performed to examine the impact of patient location and the ordering physician’s specialty on rule-based and consult-based system outcomes.
Program
The rule-based decision support system. To efficiently process orders, the prior authorization program uses a rule-based approach for making the preliminary determination of whether orders are potentially appropriate, potentially nonindicated, or inadequately justified. Ordering physicians provide supporting rationale for their orders to the nondenial prior authorization program by selecting a series of criteria for justifying the order. The criteria may be selected on the specialty benefit management company’s website or by speaking with a customer service representative.
The criteria selections are then compared with a set of rules that have been defined by the specialty benefit management company using the scientific literature and expert consensus on good clinical practice. The sequence of responses submitted with an order may trigger a rule suggesting that medical care is either potentially appropriate or potentially nonindicated. Conversely, orders may trigger neither type of rule, and may thus appear to be inadequately justified. Regardless of the outcome of the rule-based system, all orders are sent to the consult system.
The consult system. The prior authorization program operates a 3-tier model for performing consults on orders. All cardiac catheterization orders are initially reviewed by a customer service representative for information that might have been input improperly during the initial ordering process, which enhances the rate of immediate approvals. Although the consult system is used for every order, the vast majority of the orders deemed potentially appropriate by the rule-based system are approved by the customer service representative. If the customer service representative is unable to approve an order on the basis of the information initially provided, he or she refers the order to a nurse reviewer for further inspection. If the nurse reviewer is unable to obtain the necessary information to justify the order, or if a potential concern exists about the order, the order is passed to a consult physician for review.
If the consult physician cannot approve the order without further information, the consult physician engages in a peer-to-peer dialogue with the ordering physician that is centered on education, quality improvement, and understanding why the order was made. The discussion provides the consult physician with greater detail on the patient’s clinical situation and fills gaps that may exist in the information provided during the ordering process. The discussion is also intended to provide the ordering physician with the training necessary to make guidelines-based orders in the future and to work with the ordering physician through the process of interpreting the guidelines in the context of the current patient.
Unlike a system that provides immediate, automated feedback in cases of nonapproval, the discussion process enables prior authorization to consider the intricacies of the patient’s situation. Because the prior authorization program operates under a nondenial model, the only requirement that the ordering physician must fulfill to receive approval is to engage in an educational peer-to-peer discussion with a physician contracted with the specialty benefit manager.
The peer-to-peer discussion has 5 potential outcomes. If the ordering physician (1) does not respond to the request for the meeting within 2 business days, authorization is not issued (ie, a no call back). If the meeting occurs, the ordering physician and consult physician may (2) reach a consensus that diagnostic catheterization is appropriate, (3) agree that a different study is appropriate, (4) decide that the order for the catheterization should be withdrawn without a substitute study, or (5) reach no consensus. Authorization is always issued in the event of no consensus.
Sample Selection
HealthHelp, the specialty benefit manager operating the nondenial prior authorization program studied, obtained a sample of all orders for elective diagnostic catheterization made during 2015 that pertained to the Medicare Advantage membership of Humana Inc., a healthcare company that provides medical and pharmacy benefit plans nationally.20 Exclusions were made for orders that pertained to individuals with health plans that used risk arrangements other than traditional full insurance (such as administrative services only) and for orders that pertained to individuals with health plans that were not an HMO or a PPO plan (such as private fee for service).
Measurement and Analysis
Each order had 3 possible outcomes from the rule-based decision support system and 5 possible outcomes from the consult system. The number of orders receiving each of the 15 potential combinations of the rule and consult outcomes were counted. The consult system outcomes were then consolidated into 2 groups—“approved” and “other outcome.” The consults that resulted in no call back, a change in the order, withdrawal of the order, or no consensus were included in the “other outcome” category to indicate that the consult process did not lead to a straightforward approval.
The frequencies of outcomes were arranged hierarchically, with the frequency of each consult outcome being grouped according to the initial outcome from the rule-based decision support system. Outcomes were then categorized according to whether they pertained to individuals with HMO or PPO plans, whether they pertained to orders placed by cardiologists or by noncardiologists, and by the state in which the patient resided. Chi-square tests were conducted to examine whether the classification of the order by the rule-based and consult systems were independent of the plan type, physician specialty, or state of residence.
Results
As shown in the Figure, the prior authorization program processed 4102 orders for elective outpatient diagnostic catheterization pertaining to patients with Medicare Advantage insurance from the national health insurance company in 2015. After exclusions were made, the study sample consisted of 3808 orders. Among these orders, 1254 (32.9%) came from individuals with an HMO plan and 2554 (67.1%) came from individuals with a PPO plan. The majority (2918; 76.6%) of the orders were made by cardiologists. Although the patients were dispersed nationally, 41.8% of the orders were from patients residing in 5 states, including Texas (10.7%), Louisiana (9.4%), Ohio (7.7%), Kentucky (7.6%), and Florida (6.4%). The remaining 58.2% of the orders involved patients from 38 additional states.
As shown in Table 1, 72.6% of the orders overall were deemed potentially appropriate by the rule-based decision support system. Of the orders deemed potentially appropriate, less than 1% did not receive final approval by the consult system. Most of the orders originally deemed to be nonindicated or inadequately justified were eventually approved by the consult system, but a substantial proportion, especially among the potentially nonindicated orders, were not approved. Overall, only 261 (6.9%) of the 3808 orders were not approved by the consult system.
The rule and consult determinations were stratified by plan type (Table 1), physician specialty (Table 2), and the patient’s state of residence (Table 3). Chi-square tests did not show a significant association between plan type (P = .18) or physician’s specialty (P = .89) and whether an order was classified by the rule-based system as potentially appropriate, potentially nonindicated, or having inadequate initial justification. However, a significant association was found between the classification of the order by the rule-based system and the patient’s state of residence (P <.001). Chi-square tests also did not show a significant association between the determination of the consult and the plan type (P = .10), the physician’s specialty (P = .57), or the patient’s state of residence (P = .73).
Discussion
Overall, the orders that were deemed potentially appropriate by the rule-based system were the most likely to be approved by reviewers, and the orders that were deemed to have inadequate initial justification were the least likely to be approved by reviewers. Although 27.4% of the overall orders were not designated as potentially appropriate by the rule-based system, only 6.9% of the total orders were not explicitly approved by the consult system. These findings suggest that if the prior authorization program did not implement a consult system or a nondenial model, a substantial number of patients who needed diagnostic catheterizations might not have received them.
Because rule-based appropriateness determinations can be made with minimal cost, and consult-based appropriateness determinations can require expensive physician time, the combination approach enables peer consults to be used judiciously. The ambiguity in the guidelines for diagnostic catheterization forces purely rule-based prior authorization programs to take a laissez-faire approach, with a greater number of patients receiving unnecessary catheterizations, or a conservative approach, with a greater number of patients not receiving the necessary catheterizations. By implementing a combination approach, the prior authorization program examined is conservative, while enabling patients to receive needed care.
A total of 72.6% of orders were deemed potentially appropriate by the rule-based system in 2015, which differs from the finding of a previous study that 35.3% of orders in 2010 and 2011 within New York State’s Cardiac Diagnostic Catheterization Database were rated as appropriate.21 Several plausible reasons exist for this discrepancy, including that the New York study used data8 that preceded the publication of the ACC appropriate use criteria, and that the present study used a proprietary set of criteria that differed from those used by the New York study.8 Furthermore, because the consult process used by the prior authorization program has an educational focus, one would expect that physicians learn to order more appropriately over the years that the process has been in place.
The general directional finding that the prior authorization program was more likely to approve orders that were deemed potentially appropriate by rule-based criteria and less likely to approve other orders is consistent with the medical literature. A study examining the concurrence between the classification of transthoracic echocardiographs using the 2007 and 2010 ACC appropriate use criteria and the precertification decisions made by 2 radiology benefit managers found general concordance.22 The 2 radiology benefit managers preauthorized the majority of orders that met the 2007 or 2010 ACC appropriate use criteria, and they did not preauthorize the majority of the orders that were deemed inappropriate by those criteria.22
Nonetheless, multiple studies have shown that decision support tools are unable to ascertain the appropriateness of a substantial portion of orders for diagnostics.9,10 UnitedHealthcare ran a prospective multicenter cohort study in which physicians ordering noninvasive imaging for suspected coronary artery disease were required to go through a decision support tool instead of prior authorization.9 The researchers found that of 472 tests in the study, 20% of orders were of uncertain appropriateness and 11% were not addressed by the appropriate use criteria.9 In a second study, the appropriateness of 251 patients’ coronary CT angiography orders made by an academic medical center in 2007 were evaluated; 10% were deemed to have uncertain appropriateness and 46% were not classifiable.10
In contrast to these studies, the rule-based system in the present study determined that 9.0% of orders were potentially nonindicated and 18.4% of orders had inadequate initial justification, which is a slightly lower rate of inappropriateness and ambiguity (Table 2). Although some of the difference may be because the orders were for different indications in each of the studies, another possibility is that the educational consult process used by the prior authorization program increased the quality of the orders.
The lack of a significant difference in the appropriateness of orders between cardiologists and noncardiologists is consistent with the literature in the field of radiology. Previous research has found no evidence that the appropriateness of single-photon emission CT orders varies by physician specialty.23 Although one study has shown that a decision support system for radiology had a greater impact on the appropriateness of orders by some specialties immediately after implementation, the relative rate of appropriateness by specialty in the years after implementation was not reported by the study.18
The interstate variation in physician ordering behavior shown in the present study is consistent with the findings that the Dartmouth Atlas Project has been demonstrating for more than a decade.24 Because physicians learn practice behaviors during their residencies and tend to continue their practices near where they were trained, geographic practice pattern variations persist. Nondenial prior authorization has the potential to provide physicians with supplemental educational materials that may shift ordering behaviors. Nonetheless, in the case of diagnostic catheterization, some state variation has persisted.
Limitations
This study has several limitations that affect the degree to which the findings of this study can be compared with others. The findings of this study represent the program at a steady state, and thus conclusions cannot be drawn as to how the program might have enhanced appropriateness over time.
In addition, the rule-based system used a set of proprietary criteria that are based on published evidence but do not directly replicate professional society guidelines. Thus, the outcomes of the rule-based system might have been different if particular versions of professional society guidelines had been used.
Finally, orders for diagnostic catheterizations are made for a heterogeneous set of reasons. It is possible that there was variation among health plans, states, or medical specialties regarding the reasons for which diagnostic catheterizations were being ordered.
Conclusions
When physician reviewers evaluate orders as a part of a nondenial prior authorization program, they are able to obtain the information necessary to determine whether care of uncertain appropriateness should be approved. Because published guidelines suggest that there are many situations in which the need for diagnostic catheterization is ambiguous, the peer-to-peer consults that occur within nondenial prior authorization provide a valuable opportunity for 2 physicians to work together to explore what is best for each patient. The consult process plays an important role in ensuring that all patients receive the necessary and most appropriate diagnostic tests, whereas the rule-based decision support system enables orders to be triaged so that peer consults may be used in a judicious manner. In this study, the vast majority of orders sent to peer consultation, whether for inadequate initial justification or for being potentially nonindicated, were ultimately approved. The findings of this study suggest that there may be clinical benefits to supplementing rule-based review with peer review rather than treating them as substitutes.
Acknowledgment
The authors would like to thank Teresa Rogstad for her contributions to the manuscript.
Funding Source
Funding for the development of this manuscript was provided by HealthHelp and Humana.
Author Disclosure Statement
Dr Powell is an employee of HealthHelp and of Payer+ Provider Syndicate, owns stocks of many healthcare companies, and has received honoraria from the American College of Radiology and a research grant from the Max Institute of Healthcare Management. Dr Price and Mr Long are employees of Humana. Mr Nguyen was an employee of HealthHelp during this study. Mr Smith is an employee of HealthHelp. Dr Deshmukh is an employee of HealthHelp, was a consultant to Bayer and the Patient-Centered Outcomes Research Institute, and owns stocks of several healthcare companies.
Dr Powell is Director, Outcomes Research, HealthHelp, Houston, TX; Dr Price is Consultant, Clinical Vendor Oversight and Trend Management, Humana, Louisville, KY; Mr Nguyen is Analyst, Medical Economics & Informatics, HealthHelp; Mr Smith is Vice President, Medical Economics & Informatics, HealthHelp; Mr Long is Director, Clinical Vendor Oversight and Trend Management, Humana; Dr Deshmukh is Chief Medical Officer, HealthHelp.
References
1. Patel MR, Spertus JA, Brindis RG, et al. ACCF proposed method for evaluating the appropriateness of cardiovascular imaging. J Am Coll Cardiol. 2005;46:1606-1613.
2. Bonow RO. ASNC 2009 keynote lecture: measuring cost, cost-effectiveness, and quality in cardiovascular imaging. J Nucl Cardiol. 2010;17:362-369.
3. Brindis RG, Douglas PS, Hendel RC, et al. ACCF/ASNC appropriateness criteria for single-photon emission computed tomography myocardial perfusion imaging (SPECT MPI). J Am Coll Cardiol. 2005;46:1587-1605. Erratum in: J Am Coll Cardiol. 2005;46:2148-2150.
4. Hendel RC, Patel MR, Kramer CM, et al. ACCF/ACR/SCCT/SCMR/ASNC/NASCI/SCAI/SIR 2006 appropriateness criteria for cardiac computed tomography and cardiac magnetic resonance imaging. J Am Coll Cardiol. 2006;48:1475-1497.
5. Douglas PS, Khandheria B, Stainback RF, et al. ACCF/ASE/ACEP/ASNC/SCAI/SCCT/SCMR 2007 appropriateness criteria for transthoracic and transesophageal echocardiography. J Am Coll Cardiol. 2007;50:187-204.
6. Douglas PS, Khandheria B, Stainback RF, et al. ACCF/ASE/ACEP/AHA/ASNC/SCAI/SCCT/SCMR 2008 appropriateness criteria for stress echocardiography. J Am Coll Cardiol. 2008;51:1127-1147.
7. Hendel RC, Berman DS, Di Carli MF, et al. ACCF/ASNC/ACR/AHA/ASE/SCCT/SCMR/SNM 2009 appropriate use criteria for cardiac radionuclide imaging. J Am Coll Cardiol. 2009;53:2201-2229.
8. Patel MR, Bailey SR, Bonow RO, et al. ACCF/SCAI/AATS/AHA/ASE/ASNC/HFSA/HRS/SCCM/SCCT/SCMR/STS 2012 appropriate use criteria for diagnostic catheterization. J Am Coll Cardiol. 2012;59:1995-2027.
9. Lin FY, Dunning AM, Narula J, et al. Impact of an automated multimodality point-of-order decision support tool on rates of appropriate testing and clinical decision making for individuals with suspected coronary artery disease: a prospective multicenter study. J Am Coll Cardiol. 2013;62:308-316.
10. Miller JA, Raichlin E, Williamson EE, et al. Evaluation of coronary CTA Appropriateness Criteria in an academic medical center. J Am Coll Radiol. 2010;7:125-131.
11. Song Y, Skinner J, Bynum J, et al. Regional variations in diagnostic practices. N Engl J Med. 2010;363:45-53. Erratum in: N Engl J Med. 2010;363:198.
12. Robinson JD, Hippe DS, Hiatt MD. The effect of a no-denial policy on imaging utilization. J Am Coll Radiol. 2013;10:501-506.
13. Powell AC, Hippe DS, Long JW, Robinson JD. Reinitiation of withdrawn or modified neuroimaging requests after collaborative consultation. Acad Radiol. 2015;22:1433-1438.
14. Douglas P, Iskandrian AE, Krumholz HM, et al. Achieving quality in cardiovascular imaging: proceedings from the American College of Cardiology–Duke University Medical Center Think Tank on Quality in Cardiovascular Imaging. J Am Coll Cardiol. 2006;48:2141-2151.
15. Chandrasekar B, Doucet S, Bilodeau L, et al. Complications of cardiac catheterization in the current era: a single-center experience. Catheter Cardiovasc Interv. 2001;52:289-295.
16. Dehmer GJ, Weaver D, Roe MT, et al. A contemporary view of diagnostic cardiac catheterization and percutaneous coronary intervention in the United States: a report from the CathPCI Registry of the National Cardiovascular Data Registry, 2010 through June 2011. J Am Coll Cardiol. 2012;60:2017-2031. Erratum in: J Am Coll Cardiol. 2012;60:2605-2606.
17. Lucas FL, DeLorenzo MA, Siewers AE, Wennberg DE. Temporal trends in the utilization of diagnostic testing and treatments for cardiovascular disease in the United States, 1993–2001. Circulation. 2006;113:374-379.
18. Rosenthal DI, Weilburg JB, Schultz T, et al. Radiology order entry with decision support: initial clinical experience. J Am Coll Radiol. 2006;3:799-806.
19. Powell AC, Levin DC, Kren EM, et al. 2005 to 2014 CT and MRI utilization trends in the context of a nondenial prior authorization program. Health Serv Res Manag Epidemiol. 2017;4:1-6.
20. United States Securities and Exchange Commission. Form 10-K: annual report pursuant to section 13 or 15(d) of the Securities Exchange Act of 1934: for the fiscal year ended December 31, 2015. Humana Inc. February 18, 2016. www.sec.gov/Archives/edgar/data/49071/000004907116000117/hum-20151231x10k.htm. Accessed October 1, 2017.
21. Hannan EL, Samadashvili Z, Cozzens K, et al. Appropriateness of diagnostic catheterization for suspected coronary artery disease in New York State. Circ Cardiovasc Interv. 2014;7:19-27.
22. Willens HJ, Hendel RC, Inhaber FR, et al. Appropriateness use criteria for transthoracic echocardiography: relationship with radiology benefit managers preauthorization determination and comparison of the new (2010) criteria to the original (2007) criteria. Am Heart J. 2011;162:772-779.
23. Carryer DJ, Askew JW, Hodge D, et al. The impact of ordering provider specialty on appropriateness classification. J Nucl Cardiol. 2012;19:285-290.
24. Dartmouth Medical School Center for the Evaluative Clinical Sciences. The care of patients with severe chronic illness: an online report on the Medicare program by the Dartmouth Atlas Project. Lebanon, NH; 2006. www.dartmouthatlas.org/downloads/atlases/2006_Chronic_Care_Atlas.pdf. Accessed May 5, 2018.