The term “detailing” has conventionally referred to face-to-face promotional activities to physicians conducted by pharmaceutical sales representatives.1,2 Although detailing began in the nineteenth century, the rise of modern pharmaceutical detailing began in the 1940s and resulted from major expansions within the pharmaceutical industry.3 Major pharmaceutical companies currently devote substantial resources to detailing activities for their major products,4 with estimated annual spending of $7 billion to $20 billion.5 This type of detailing can be problematic because the focus on increasing sales can distort physicians’ perception of the safety and efficacy of the promoted drug and affect prescribing choices. Ultimately, this can affect patient outcomes and increase the costs of care.6-8 Conversely, academic detailing, also known as educational outreach, uses the one-on-one outreach approach of pharmaceutical detailing to provide clinicians with unbiased, evidence-based information that can be used to improve clinical practice.9-14
Originally developed in the 1980s in the context of prescribing decisions, the methods of academic detailing have since been adapted to other areas of healthcare, such as smoking cessation, cancer screening, and reducing hospital readmissions, to disseminate information and change behavior.15-18
A systematic review of the effect of academic detailing in 69 studies, representing more than 15,000 health professionals, showed that this intervention could be effective in changing physician prescribing behavior and in improving clinical practice.9 That review also showed that the effect of academic detailing in reducing inappropriate prescribing was modest but significant, with a median (adjusted) risk difference of 4.8% and an interquartile range of 3% to 6.5%.
In terms of behavior not related to prescribing, the effect range was greater (median adjusted risk difference, 6%; interquartile range, 3.6%-16%). Although studies generally have shown that academic detailing can be effective as a stand-alone intervention or as part of a multipronged approach, the context of the intervention and the environment in which it is delivered may influence its effectiveness.19
Many approaches to implementation of academic detailing have been described in the literature, reflecting a lack of agreement on what this intervention should entail among those who implement and evaluate it. Academic detailing programs exist in various settings. Government-supported programs of academic detailing have been implemented in many countries, including a nationwide program in Australia and multiple provincial programs in Canada.20,21
In the United States, some academic detailing programs are supported by local or state government (eg, New York State Medicaid Prescriber Education Program), some exist as part of the organizational structure of integrated health systems (eg, Kaiser Permanente system in California), and others are supported by insurance companies or nonprofit entities.9,22
The US Veteran Affairs health system has implemented several academic detailing programs to improve mental healthcare and prescribing practices for antihypertensive medications.23,24
Although the diversity of implementation settings and the lack of uniformity in the literature can reflect the adaptability of this educational approach, such variation poses a challenge for those who seek templates to guide the implementation of new academic detailing programs and for those who wish to evaluate the effectiveness of academic detailing.9,25-27
This article is the second of a 2-part series on academic detailing. Part 1 was descriptive findings from a systematic review about important characteristics of academic detailing, published in American Health & Drug Benefits in 2015.28 In this second article, we used the Delphi method to collect information from an international panel of experts to develop a consensus regarding the standards for academic detailing.
The Delphi method is a widely accepted technique of structured and systematic information-gathering from a group of experts (ie, the Delphi Panel) on a specific topic using a series of questionnaires.29-32 The Delphi method allows a panel of experts to provide insight and opinions, even when not located in the same geographic area. The questionnaires are administered, in iterative rounds, to all experts who agree to participate. The responses from each previous round are compiled and provided to the panel before the subsequent round begins. This structure allows each participant to reevaluate his or her initial answer after considering the responses of the other panel members.
Rounds are conducted until consensus is reached or until there is stability in the questionnaire answers; the process typically requires 3 or 4 rounds. A key advantage of the Delphi method is that it allows each participant to have a voice, in an anonymous manner. Participants do not interact or speak with each other during the process; they are merely given feedback about their own answer in the context of the group’s responses.
Methods
Recruitment of the Delphi Expert Panel
We used purposeful sampling of potential experts. We invited 26 experts to participate in our Delphi exercise, 20 of whom accepted the invitation. The experts were known to one of the study investigators (M.A.F.), who directs the National Resource Center for Academic Detailing (NaRCAD), an initiative supported by the Agency for Healthcare Research and Quality (AHRQ) to encourage and support academic detailing programs. Experts were selected based on experience in areas of clinical content, research, or administration related to academic detailing. All experts indicated their willingness to participate in the Delphi process and did not receive any monetary compensation for participation.
Survey Design
We previously identified 5 critical characteristics of academic detailing, including (1) content, (2) communication process, (3) clinicians targeted, (4) change agents delivering intervention, and (5) context for intervention.4 Based on these 5 critical characteristics, we developed 72 survey items for the Delphi exercise, covering 9 topics (see Appendix at www.AHDBonline.com). The experts were asked to rank each item on a scale of 1 to 4, from least to greatest importance (1 = not at all important, 2 = somewhat important, 3 = very important, 4 = essential).
The 9 general survey topics were:
- What information do you believe should be offered through academic detailing visits?
- What outcomes should studies using educational outreach be seeking to change and to measure?
- Should information provided in academic detailing visits be tailored?
- In what ways should academic detail interventions select providers for invitation to participate?
- Which facets of visits are most important to academic detailing?
- Which groups are critical to academic detailing visits?
- Which qualifications are critical for academic detailing outreach workers to have?
- What areas of training should academic detailing workers receive?
- Should the academic detailing outreach worker have the same or a different employer than the providers being visited?
Survey Administration
The experts participated in 3 rounds of the Delphi consensus exercise during a 2-month period, from January to February 2014. Each expert was e-mailed a link to the confidential online survey for each of the 3 rounds. The experts were given 2 weeks to complete each survey round, and 2 reminder e-mails were sent (1 week before and 1 day before the conclusion of each round). All 20 experts were sent round 1 of the online survey. Nonresponders to a given survey round were excluded from subsequent rounds.
Consensus Determination
We divided the consensus into “critical” and “not critical” features of academic detailing. “Critical” consensus comprised the ratings “essential” and “very important,” whereas “not critical” consensus comprised the ratings “somewhat important” and “not at all important.”
Consensus was reached via 1 of 2 ways. First, if an item received at least 70% of the votes in a single rating category (ie, 1 of the 4 options), then we assigned consensus based on the respective rating category. For items that did not achieve a single rating of 70% or more, consensus was reached if the sum of the rating percentages within one of the dichotomized consensus categories represented 80% or more of the votes.
For example, the survey item pertaining to whether “best practices” information should be delivered during academic detailing intervention achieved ratings of “essential” (28%) and “very important” (67%). Although this item did not receive at least 70% of responses in 1 rating category, “critical” consensus was achieved because the sum of “essential” and “very important” ratings was at least 80%.
Delphi Exercise
In round 1, the panel was asked about the 9 elements described above. Before round 2, the survey was modified to reflect the findings from round 1. At the start of round 2, each panel member was reminded of his or her original answers and was shown the aggregated group findings from round 1 to situate his or her position relative to that of the group. Each expert was then asked to consider his or her original responses in the context of the group findings using the same ordinal scale presented in round 1.
The survey for round 3 was modified to reflect the results from round 2 in 2 ways. First, the panel was shown the results of the consensus items and was given the opportunity to make open-ended comments. Second, the panel was presented with the list of nonconsensus items and their relative rating. Each expert was then given the final opportunity to modify his or her answers.
Results
Table 1 shows the demographics of the panel participants. The participants included physicians (N = 5), pharmacists (N = 10), and those with advanced training in statistics/epidemiology or a related healthcare field (N = 5). A total of 70% (N = 14) were based in the United States, 45% (N = 9) were primarily affiliated with an academic medical center, and 70% (N = 14) had authored published peer-reviewed research or review articles related to academic detailing.
In round 1, 19 of the 20 invited participants completed the online survey. In round 2, 18 of the 19 remaining participants completed the survey. In round 3, 17 of the 18 remaining participants completed the survey. Overall, we achieved a response rate of at least 94% in each of the 3 rounds of the Delphi exercise, and a cumulative overall response rate of 95% (54 of 57 surveys). We did not detect any particular features that would differentiate nonresponders from those who completed the survey.
At the end of round 1 of the Delphi exercise, 36% (N = 26) of the survey items achieved consensus using the predetermined metric. After receiving feedback from the original survey, including group responses, the panel achieved consensus on 58% (N = 42) of the survey items by the end of round 2. At the end of round 3, 75% (N = 24) of the 32 items that previously had not met consensus achieved it. The remaining 8 survey items attained response stability (Table 2).
Overall, 92% (N = 66) of the 72 survey items achieved consensus by completion of round 3 of the Delphi exercise. The list of consensus items was then presented to the panel for the final time, for informational purposes, in the form of a brief report (Table 3). The panel experts were given the opportunity to make open-ended responses to the list of questions.
Context and evaluation of academic detailing. Several themes emerged from the Delphi panel’s consensus responses. All experts clearly agreed that academic detailing should be targeted toward clinicians and not to patients (consensus rating, 100%), with targeting of clinicians focused on practice location, practice specialties, and/or patient characteristics (consensus ratings, 83%-94%). In contrast to practice facilitation, care coordination, or technical assistance, panel members defined the core of academic detailing as providing education to clinicians (consensus rating, 95%). They noted that the goal of this education is to improve clinical performance (consensus rating, 89%), recommend practice changes (consensus rating, 100%), and offer decision support (consensus rating, 82%).
The panel agreed that evaluation of academic detailing should focus on changes in clinician behavior or performance (consensus rating, 100%), patient outcomes (consensus, 78%), resource utilization (consensus rating, 90%), and clinician knowledge (consensus rating, 84%), rather than on changes in practice workflow or efficiency. The impact of academic detailing programs on changes in the prevalence of diseases was considered less appropriate as a target outcome. Panel members did not reach consensus on whether healthcare cost should be an outcome in the evaluation of academic detailing.
Delivering and tailoring academic detailing interventions. The panel identified physicians or pharmacists as the individuals best qualified to perform the frontline work of academic detailing (consensus rating, 83%); rankings were lower for nurses, those involved in quality-improvement work, and public health workers. There was strong agreement that academic detailing should be tailored based on the clinical context of the practice or provider (consensus rating, 94%) and on specific barriers faced by the practice or provider (consensus rating, 100%).
Characteristics of academic detailing visits. Although the panel assigned high importance to the frequency of academic detailing visits, they disagreed about the optimal length of the intervention period (eg, time between first and last intervention), the duration of each visit, and the overall number of visits. In terms of training for academic detailing programs, the experts would like detailers to receive instructions on communication and interpersonal skills (consensus rating, 100%) to overcome anticipated clinician barriers to behavior change (consensus ratings, 94%-100%). The experts also believe that detailers should be provided with specific strategies and solutions to give to the targeted practice or provider (consensus rating, 95%).
Discussion
The findings of this Delphi panel process further expand the principles identified in an early report defining academic detailing.11 Academic detailing provides clinicians with useful and relevant information to change behavior in ways that improve patient outcomes. To achieve behavior change through academic detailing, the interventions need to maintain credibility with frontline clinicians.
Our findings suggest that individuals with relevant clinical expertise, such as physicians and pharmacists,33 can be very effective in promoting change by targeted providers. To promote active learner involvement, academic detailing messages should relate to the needs and interests of the provider and should identify barriers to evidence-based clinical decisions.
Furthermore, detailing programs need to provide concise decision support material to further encourage evidence-based clinical decisions, and detailers should be equipped with the skills needed to identify providers’ baseline knowledge and motivation for prescribing patterns.
The inclusion of experts with varying institutional and operational experiences in different facets of academic detailing represents a particular strength of this process. We were able to achieve a high level of response throughout the entire Delphi process. Although we cannot find a published definition of what constitutes “consensus,” we chose relatively high thresholds for consensus: 70% (single Likert option) or 80% (combined Likert options).34
Despite this, our panel achieved agreement on nearly 90% of the survey items; the remaining 10% of items failed to achieve consensus after 3 rounds of the Delphi exercise. This is consistent with the literature indicating that most changes in survey responses occur within the first 2 rounds, and that little is gained in further iterations.35
Limitations
The limitations of our study should be kept in mind when considering its findings. Although our Delphi panel included experts from various countries and regions, many individuals involved in academic detailing have communicated or worked together at some point, and thus the experiences of the panel are unlikely to be completely independent of one another.
Furthermore, we only included participants from English-speaking countries, but there are academic detailing programs in Japan and Brazil and perhaps in other non–English-speaking countries.
In addition, because standard definitions are lacking for some terms used in our survey (eg, resource utilization, practice change, decision support), ambiguity may have existed about how to interpret these terms, and this may have influenced the choices made.
Conclusion
Our Delphi consensus exercise was undertaken to develop an accepted standard of what academic detailing should entail. Insights from our study, along with evidence from existing academic detailing literature and the experiences of those involved in academic detailing, should serve as the basis in formulating this framework. The results of the Delphi exercise can inform individuals (leaders and researchers) and organizations (institutions and agencies) involved in the development, implementation, research, or evaluation of academic detailing and provide a set of general principles for academic detailing initiatives.
Our Delphi panel of international experts reached consensus on several main themes related to academic detailing. The messages of academic detailing need to be tailored to provide strategies and solutions for clinicians to overcome barriers to evidence-based best practices. Evaluation of academic detailing programs should focus on changes in clinician knowledge and performance, improvements in patient outcomes, and utilization of resources.
Funding Source
The Society for Academic Continuing Medical Education (SACME) provided financial support for this study. The views expressed in this article are those of the authors and not necessarily the views of SACME.
Author Disclosure Statement
Dr Yeh and Dr Van Hoof reported no conflicts of interest. Dr Fischer is a clinical consultant to the Alosa Foundation, a nonprofit organization that provides academic detailing services.
Dr Yeh is a Research Fellow, Program On Regulation, Therapeutics, And Law (PORTAL), Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, and a Clinical Fellow, Harvard Medical School, Boston, MA; Dr Van Hoof is Associate Professor, University of Connecticut School of Nursing, Storrs, CT, and Associate Professor, Department of Community Medicine and Health Care, University of Connecticut School of Medicine, Farmington, CT; Dr Fischer is Director, National Resource Center for Academic Detailing, Division of Pharmacoepidemiology and Pharmacoeconomics, Brigham and Women’s Hospital, and Associate Professor, Harvard Medical School, Boston, MA.
References
1. Greene JA, Watkins ES, eds. Prescribed: Writing, Filling, Using, and Abusing the Prescription in Modern America. Baltimore, MD: Johns Hopkins University Press; 2012.
2. Smith MC, Kolassa EM, Perkins G, Siecker B. Pharmaceutical Marketing: Principles, Environment, and Practice. New York, NY: Informa Healthcare; 2002.
3. Greene JA. Attention to ‘details’: etiquette and the pharmaceutical salesman in postwar American. Soc Stud Sci. 2004;34:271-292.
4. Van Hoof TJ, Miller NE, Meehan TP. Do published studies of educational outreach provide documentation of potentially important characteristics? Am J Med Qual. 2013;28:480-484.
5. Gagnon M-A, Lexchin J. The cost of pushing pills: a new estimate of pharmaceutical promotion expenditures in the United States. PLoS Med. 2008;5:e1-e5.
6. Mintzes B, Lexchin J, Sutherland JM, et al. Pharmaceutical sales representatives and patient safety: a comparative prospective study of information quality in Canada, France and the United States. J Gen Intern Med. 2013;28:1368-1375.
7. Spurling GK, Mansfield PR, Montgomery BD, et al. Information from pharmaceutical companies and the quality, quantity, and cost of physicians’ prescribing: a systematic review. PLoS Med. 2010;7:e1000352.
8. Larkin I, Ang D, Avorn J, Kesselheim AS. Restrictions on pharmaceutical detailing reduced off-label prescribing of antidepressants and antipsychotics in children. Health Aff (Millwood). 2014;33:1014-1023.
9. O’Brien MA, Rogers S, Jamtvedt G, et al. Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2007:CD000409.
10. Avorn J, Soumerai SB. Improving drug-therapy decisions through educational outreach: a randomized controlled trial of academically based “detailing.” N Engl J Med. 1983;308:1457-1463.
11. Soumerai SB, Avorn J. Principles of educational outreach (‘academic detailing’) to improve clinical decision making. JAMA. 1990;263:549-556.
12. Peterson GM, Stanton LA, Bergin JK, Chapman GA. Improving the prescribing of antibiotics for urinary tract infection. J Clin Pharm Ther. 1997;22:147-153.
13. Avorn J, Soumerai SB, Everitt DE, et al. A randomized trial of a program to reduce the use of psychoactive drugs in nursing homes. N Engl J Med. 1992;327:168-173.
14. Avorn J. Improving the quality and cost-effectiveness of prescribing. Pharmacoeconomics. 1992;1(suppl 1):45-48.
15. Axon RN, Penney FT, Kyle TR, et al. A hospital discharge summary quality improvement program featuring individual and team-based feedback and academic detailing. Am J Med Sci. 2014;347:472-477.
16. Jin M, Gagnon A, Levine M, et al. Patient-specific academic detailing for smoking cessation: feasibility study. Can Fam Physician. 2014;60:e16-e23.
17. Horowitz CR, Goldberg HI, Martin DP, et al. Conducting a randomized controlled trial of CQI and academic detailing to implement clinical guidelines. Jt Comm J Qual Improv. 1996;22:734-750.
18. Fischer MA, Avorn J. Academic detailing can play a key role in assessing and implementing comparative effectiveness research findings. Health Aff (Millwood). 2012;31:2206-2212.
19. Sketris IS, Langille Ingram EM, Lummis HL. Strategic opportunities for effective optimal prescribing and medication management. Can J Clin Pharmacol. 2009;16:e103-e125.
20. NPS MedicineWise. www.nps.org.au/. Accessed April 5, 2015.
21. Kondro W. Academic drug detailing: an evidence-based alternative. CMAJ. 2007;176:429-431.
22. National Resource Center for Academic Detailing. www.narcad.org/. Accessed April 5, 2015.
23. Siegel D, Lopez J, Meier J, et al. Academic detailing to improve antihypertensive prescribing patterns. Am J Hypertens. 2003;16:508-511.
24. Hermes EDA, Sernyak M, Rosenheck R. Impact of a program encouraging the use of generic antipsychotics. Am J Manag Care. 2012;18:e307-e314.
25. Chhina HK, Bhole VM, Goldsmith C, et al. Effectiveness of academic detailing to optimize medication prescribing behaviour of family physicians. J Pharm Pharm Sci. 2013;16:511-529.
26. Van Hoof TJ, Miller NE. Consequences of a lack of standardization of continuing education terminology: the case of practice facilitation and educational outreach. J Contin Educ Health Prof. 2014;34:83-86.
27. Van Hoof TJ, Meehan TP. Using theory and evidence to guide the use of educational outreach to improve patient care. Am J Med Qual. 2012;27:467-471.
28. Van Hoof TJ, Harrison LG, Miller NE, et al. Characteristics of academic detailing: results of a literature review. Am Health Drug Benefits. 2015;8:414-422.
29. Dalkey N, Helmer O. An experimental application of the DELPHI method to the use of experts. Manage Sci. 1963;9:458-467.
30. Ulschak FL. Human Resource Development: The Theory and Practice of Need Assessment. Reston, VA: Reston Publishing Co Inc; 1983.
31. Adler M, Ziglio E, eds. Gazing into the Oracle: The Delphi Method and Its Application to Social Policy and Public Health. London, England: Jessica Kingsley Publishers; 1996.
32. Linstone HA, Turoff M, eds. The Delphi Method: Techniques and Applications. Reading, MA: Addison-Wesley Publishing Co Inc; 1975.
33. Jin M, Naumann T, Regier L, et al. A brief overview of academic detailing in Canada: another role for pharmacists. Can Pharm J (Ott). 2012;145:142-146.e2.
34. Fink A, Kosecoff J, Chassin M, Brook RH. Consensus methods: characteristics and guidelines for use. Am J Public Health. 1984;74:979-983.
35. Mitchell VW. The Delphi technique: an exposition and application. Technol Anal Strateg Manage. 1991;3:333-358.