Immunoglobulins are proteins produced by B-cells that are used by the immune system to identify and act on foreign objects, such as bacteria and viruses. Therapeutic preparations of human immunoglobulin have provided substantive opportunity for their application in a wide variety of clinical contexts. Although the mechanism of action of intravenous immunoglobulin (IVIG) is complex, it is known that in addition to its anti-infective activity, IVIG exhibits immunoregulatory and anti-inflammatory properties.1 IVIG is a fundamental replacement therapy for many debilitating and/or life-threatening immunodeficiency disorders for patients who have decreased or abolished antibody production capabilities2 and is a critical immune-modulating agent in an array of immune dysregulatory disorders.
In the United States, currently 8 uses have been approved by the US Food and Drug Administration (FDA) for IVIG, including primary immunodeficiency disease or primary antibody immunodeficiency, idiopathic thrombocytopenic purpura, Kawasaki disease, B-cell chronic lymphocytic leukemia, bone marrow transplantation, pediatric HIV-1 infection, multifocal motor neuropathy, and chronic inflammatory demyelinating polyneuropathy.3,4
However, in clinical practice, IVIG therapy is frequently used for other conditions, without FDA approval, when clinicians perceive a clinical benefit for its use.5 Common off-label uses include several neurologic conditions, such as relapsing-remitting multiple sclerosis (RRMS), Guillain-Barré syndrome, and Lennox-Gastaut syndrome6,7; infection prophylaxis in patients undergoing transplant, such as cytomegalovirus prevention in lung and bone marrow transplantation8,9; for the treatment of rheumatoid arthritis in certain subgroups10; and for the prevention of miscarriage in individuals with antiphospholipid syndrome, among many others.5,11,12
An exhaustive list of off-label uses is beyond the scope of this overview, but many payers (Medicare or commercial insurers) have approved reimbursement (in some cases provisionally) for various off-label uses of IVIG, by establishing criteria such as a lack of response to conventional therapies or evidence of contraindications. Furthermore, although various levels of clinical evidence are available to support the use of IVIG therapy in these and other conditions, a comprehensive review is not part of this analysis. Rather, the reader may refer to published case reports, review articles, US-specific and international guidelines, and various compendia (eg, DRUGDEX Drug Evaluations) for information about a specific unlabeled use.
For example, the American Academy of Neurology rated evidence for the use of IVIG therapy in adults with Guillain-Barré syndrome as level A, whereas the evidence is insufficient to support or refute its use in children with Guillain-Barré syndrome (Level U)6; however, a previous Cochrane systematic review of immunotherapy for recurrent miscarriage showed that IVIG did not improve the live birth rate in unexplained recurrent miscarriage and, thus, concluded that IVIG should not be offered for this use.11
Although the administration of IVIG therapy can be life-saving, including some of its off-label uses, it can also be associated with many adverse effects, reimbursement challenges, and even limited availability (because its production is dependent on the supply of qualified human plasma).5,13,14 As a result, several national and international treatment guidelines have been developed to direct treatment for conditions requiring IVIG therapy.6,15,16
An ongoing evaluation of IVIG uses is crucial because of the limited donor pool, complex production and distribution process, the high cost of therapy, and the increasing number of clinical conditions supported by at least some experimental evidence. In 2006, expert immunologists from the American Academy of Allergy, Asthma & Immunology (AAAAI) reviewed the use of IVIG therapy from an American perspective and published recommendations based on the degree of available clinical evidence for FDA-approved and unapproved conditions.13 There was limited emphasis placed on differences in opinion during, and especially after, the process of achieving consensus.13 Although these recommendations are now several years old, they provide a reference for the evidence-based medicine (EBM) axis on which other aspects of IVIG therapy utilization priority (ie, the importance of IVIG shortages, disease severity, and resistance or intolerance to alternative therapies to IVIG utilization) can be considered.
More recently, the question arose whether using a single axis in the AAAAI’s algorithm was consistent with the full spirit of EBM for a limited resource such as IVIG, considering the definition of EBM advocating for “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.”17 Furthermore, for patients awaiting treatment with IVIG, decisions made by Pharmacy & Therapeutics (P&T) committees that are typically composed of knowledgeable multispecialty healthcare providers (eg, practicing physicians, pharmacists, nurses, administrators, quality improvement managers) involved in the medication-use process are critical when the availability of IVIG is limited.18,19 However, relatively few (37%) hospitals have a P&T committee that is specifically tasked with determining which patients are to be treated under established priority protocols, and even fewer (27%) have priority protocols specifying which patients are to be treated first.20 This is surprising, given the relatively little guidance provided in the peer-reviewed medical literature.
The current study was undertaken to address these limitations and promote cross-specialty dialogue about EBM for a wide range of FDA-approved and off-label uses of IVIG therapy, based on a previously proposed IVIG therapy prioritization algorithm across approved and off-label conditions by a multispecialty/disciplinary medical panel consisting of experts in IVIG therapy in each respective field.21 The engagement’s primary objective in this study was to use consensus-building methodologies to rate the disease states for IVIG utilization while considering disease severity (Axis 2) and the efficacy of alternative therapeutic options (Axis 3) from the perspective of US multispecialists to strengthen the best practices for the use of IVIG therapy.
Methods
This study evaluated the previously proposed addition of 2 axes—disease severity and the efficacy of therapeutic alternatives—in addition to the AAAAI’s EBM ratings for the consideration of IVIG therapy prioritization for clinical conditions.13,21 To strengthen the application of AAAAI’s ratings, we used a 3-axis IVIG prioritization algorithm for strong evidence-based indications previously considered by a panel of expert immunologists.21 Because the quality of evidence alone does not convey the full range of therapeutic utility of IVIG across all clinical conditions, we considered the hypothesis that the application of the algorithm’s second and third axes can offer enhanced perspective that could prove advantageous in meeting the increased demand for a limited supply of IVIG.
The 3-axis IVIG prioritization algorithm, incorporating 16 ratings and a linear scale, is shown in Figure 1. For example, the highest-priority condition (A1) according to this 3-axis algorithm was classified as an immediately life-threatening disease with no reasonable treatment alternatives.
Initially, a comprehensive list of items was identified, and consensus was built from the feedback provided by 7 experts from the preceding rounds. The modified Delphi method consisted of 2 rounds of e-mail questionnaires and a final face-to-face meeting.
The 7-member panel consisted of leading experts on the use of IVIG therapy within their specific board-certified specialty. In the Delphi methodology, defining consensus is independent of the size of the group. The number of our panel members is consistent with best practices, as evident in the literature.22,23
Because the use of IVIG therapy is ubiquitous, all the experts accounted for experiences on the use of IVIG outside of their specialties. All the experts were physicians representing major academic centers located throughout the United States, including experts from various medical specialties representing the majority of the IVIG applications. This includes hematology, allergy, clinical immunology, dermatology, rheumatology, solid organ transplantation, neurology, and nephrology (Figure 2). All panelists were compensated for the actual time they spent participating in the study, at fair market value rates by ASD Healthcare.
The consensus process encompassed a 3-step modified Delphi method.24,25 This method is advocated for use in healthcare as a dependable tool for establishing consensus for a stated clinical dilemma.26-28 The modified Delphi method uses an iterative systematic progression of repetitive rounds of voting and has been shown to be an effective process for establishing expert group consensus where there is inadequate or absent definitive evidence and where opinion is imperative.27
Specific to the 3 steps of the modified Delphi method, the panel initially convened in a virtual meeting to review the prioritization algorithm and then independently completed an online prioritization survey, with a second virtual consensus meeting to review the results. Subsequently, a follow-up online rating survey was independently completed to achieve consensus on nonconsensus-forming items. The full process is illustrated in Figure 2 and is described in more detail below.
First virtual consensus meeting and initial online survey. The 3-axis algorithm for IVIG therapy prioritization was introduced during the first virtual meeting, which was a web conference with audio and visual capabilities. During this engagement, the 3-axis IVIG prioritization model was introduced. The survey methodology and approach to follow-up meetings was also explained to the participating experts.
Initial online survey. After the first virtual consensus meeting, the experts were asked to assess their level of agreement with the AAAAI’s categorization of evidence rating (Axis 1), the strength of the recommendation, and the interpretation of perceived benefit of IVIG therapy for each condition as published,13 in an online survey that lasted approximately 75 minutes.
These questions were based on a scale from 1 to 7, where 1 meant strongly disagree and 7 meant strongly agree. When no AAAAI rating was available, the experts were asked to provide their own assessment of treatment benefit based only on the existing data (ie, not relative to anything else) using the defined descriptors of perceived benefit (ie, definitely beneficial, probably beneficial, may provide benefit, or unlikely to be beneficial).
Next, the experts were asked to independently rate 50 disease states across 2 domains (axes) on a 4-point Likert scale that included disease severity (Axis 2 [1] immediately life-threatening; [2] life-threatening; [3] life-modifying; [4] other), and perceived efficacy of therapeutic alternatives (Axis 3 [1] none; [2] low; [3] medium; and [4] high). The most frequently used therapeutic alternatives across these 50 disease states are corticosteroids; various immunosuppressive therapies, including cytotoxic agents; and plasma exchange.13 It was up to each panel member to decide what was a viable therapeutic alternative to IVIG in each case, as well as the perception of the treatment efficacy.
Second virtual consensus meeting. The findings from the online survey were presented during the second meeting, and 8 conditions with either low consensus or those representing IVIG utilization in highly variable conditions were identified for further discussion. Consistent with the Delphi method, a group discussion followed, during which panelists with specific expertise presented their views on the selected disease states and IVIG use in the treatment of these conditions.
The discussion allowed the experts to present and rationalize their opinions and facilitated the opportunity to consider the opinions of others, reconsider their own opinions, and assess the relative importance of each opinion presented.
Follow-up online survey. After the group discussion, a second online survey, lasting approximately 30 minutes, was administered. The experts were asked to rerate nonconsensus-forming items on Axis 2 (disease severity) and Axis 3 (efficacy of alternative therapies) as a result of the group discussion. To ensure the soundness of the data, close attention was given to upfront expert selection and motivation, survey construction, and process management. The experts were also instructed to provide any outstanding comments on the IVIG rating for disease states where there was relatively good consensus in the first survey.
Analyses
Interquartile deviation (IQD), a measure of data dispersion, was used to measure the degree of consensus. IQD is based on the lower quartile Q1 (25th percentile) and the upper quartile Q3 (75th percentile) of responses.29 The difference of Q3−Q1 is the interquartile range. The difference of (Q3−Q1) divided by 2 is the semi-interquartile range or the quartile deviation. Raskin identified an IQD of 1.00 or less as an indicator of consensus.30 This study used an IQD of ≤0.5 as an indication of consensus. Disease states reaching consensus across both domains were plotted on a 2 × 4 algorithmic matrix (ie, the consensus rating of Axis 2 on its 4-point Likert scale was plotted against the Axis 3 rating) to establish priority recommendations.
Results
In distinction from the design of many consensus projects, 7 experts, including 2 physicians with 2 or more specialties, were assembled from across 8 clinical disciplines (not mutually exclusive). The first meeting was held as a virtual consensus meeting paired with the initial online survey. No results were presented at this first virtual consensus meeting, because it was a didactic event to provide the experts with the background information necessary regarding the 3-axis IVIG prioritization methodology being used in this study and the completion of the IVIG surveys. The results of the online survey that the experts completed after this meeting showed several points.
Axis 1. Overall, with the exception of 4 conditions (ie, Alzheimer’s disease, fetomaternal alloimmune thrombocytopenia, neonatal sepsis, and pemphigus vulgaris), the results of the online survey demonstrated high agreement among the respondents and the AAAAI’s IVIG ratings. For each disease state, the mean score for Axis 2 was plotted against the mean score for Axis 3 to get an overall rating ranging from A1 to D4 (Figure 1).
Axis 2. Based on an IQD of ≤0.5, the panel reached consensus on the severity (Axis 2) of all 50 disease states; 12 of those showed a tight grouping of scores, with an IQD of 0 (Table 1).
Axis 3. Of the 50 disease states, 39 reached consensus on the efficacy of therapeutic alternatives (Table 1). Four conditions (ie, autoimmune uveitis, posthematopoietic stem-cell transplantation, severe eczema, and specific antibody deficiency) had an IQD of 0 on perceived efficacy of therapeutic alternatives (Axis 3). These 4 disease states, except for specific antibody deficiency, also had an IQD of 0 in the assessment of severity (Axis 2).
The panel was reconvened for a second virtual consensus meeting to discuss the findings from the first survey and to have a more in-depth discussion on selected disease states. Here, the axis-specific concerns were raised, and conditions were selected for consideration.
Axis 2. After evaluating the degree of consensus on the severity rating of all 50 disease states, 4 (ie, myasthenia gravis, toxic epidermal necrolysis, Kawasaki disease, and Guillain-Barré syndrome) with IVIG utilization in highly variable conditions (ie, highly variable in presentation and/or progression) were selected for additional discussion.
Axis 3. The 11 disease states where consensus was not reached on perceived efficacy of therapeutic alternatives (Tables 2 and 3)—none of which were deemed immediately life-threatening on Axis 2—were severe combined immunodeficiency, fetomaternal alloimmune thrombocytopenia, adrenoleukodystrophy, prevention of infection in pediatric AIDS, autoimmune hemolytic anemia, cytomegalovirus pneumonitis in solid organ transplant, autism, idiopathic thrombocytopenic purpura, RRMS, chronic fatigue syndrome, and recurrent spontaneous abortion (to promote successful pregnancy).
Of these, the 4 conditions (ie, severe combined immunodeficiency, autoimmune hemolytic anemia, RRMS, and cytomegalovirus) closest to consensus on the initial survey, and therefore the most likely to reach consensus within the study time frame, were chosen for additional discussion at the second virtual consensus meeting.
Of the 39 conditions reaching consensus, only 2 were rated as immediately life-threatening (ie, prevention of sepsis in neonatal intensive care unit and neonatal sepsis [treatment of]) and were perceived to have medium and high efficacy as therapeutic alternatives, respectively.
After the discussion at the second virtual consensus meeting, the experts completed a follow-up online survey, thus allowing for the evaluation of any alteration in consensus (or lack thereof) for the discussed indications. The follow-up survey demonstrated 2 key points.
Axis 2. Myasthenia gravis, Kawasaki disease, toxic epidermal necrolysis, and Guillain-Barré syndrome were subsequently resurveyed, with only myasthenia gravis showing movement in IQD, moving from an IQD of 0.5 to an IQD of 0 (ie, a higher degree of consensus).
Axis 3. The 11 disease states without consensus in the first survey were resurveyed, with 4 of them (ie, autoimmune hemolytic anemia, severe combined immunodeficiency, RRMS, and cytomegalovirus pneumonitis in solid organ transplantation) subsequently reaching consensus; severe combined immunodeficiency and RRMS also showed consolidations on severity, moving from an IQD of 0.5 to an IQD of 0.
Overall, the discussion among the clinical experts and the resurvey resulted in increasing consensus from 78% to 86% (ie, an IQD ≤0.5 on Axes 2 and 3), with 3 disease states reaching an IQD of 0 on Axes 2 and 3, and a change in the overall IVIG rating of 3 conditions.
Discussion
The number of disease states for which some level of clinical evidence regarding the use of IVIG therapy exists is increasing. However, because IVIG therapy is a finite resource with high costs, the rating of its use is critical. As previously proposed by Orange and colleagues,21 our study investigated the use of a more refined algorithm for IVIG therapy prioritization, incorporating 3 axes to account for strength of clinical evidence, disease severity, and efficacy of therapeutic alternatives. Through a multistep process, a 7-member multispecialty expert panel established consensus in 86% of the disease states evaluated for which IVIG therapy could be considered. The findings were consistent with the previously published guideline recommendations13; however, because of the opinion-based nature of this pilot study, a definitive stratification of IVIG off-label uses was not feasible.
Specific to the degree of consensus, a total of 15 disease states reached an IQD of 0, indicating a high degree of consensus on disease severity, and 4 disease states reached this level of consensus on the efficacy of therapeutic alternatives. Three disease states (ie, autoimmune uveitis, posthematopoietic stem-cell transplantation, and severe eczema) reached an IQD of 0 on Axes 2 and 3.
There was also a change after step 3 (ie, the follow-up online survey) in the overall IVIG therapy rating of 3 conditions, with changes on resurvey in the mean disease severity rating of 2 conditions and the mean efficacy of therapeutic alternatives rating of 2 conditions. Severe combined immunodeficiency increased in severity rating from life-threatening to immediately life-threatening, and autoimmune hemolytic anemia decreased in severity rating from life-threatening to life-modifying. The mean score for the efficacy of therapeutic alternatives changed from medium to high efficacy for autoimmune hemolytic anemia and for myasthenia gravis. The follow-up discussion and the resurvey emphasized the need for case-by-case discussion of the more variable and complex autoimmune diseases to determine IVIG therapy priority.
Although transcripts of the discussions were not collected, an important topic for future research would encompass the consensus-building dialogues and the key subjective drivers of outcomes toward consensus in specific applications. This could potentially identify key decision points to arm and inform local groups, such as multidisciplinary institutional P&T committees, in making decisions regarding IVIG-related formularies.
Overall, our evaluation of consensus might have been potentially further improved by supplementing it with additional axes, having a higher number of meetings, subdividing the disease categories in the AAAAI list, and hosting 1 or 2 virtual consensus meetings as live events. As noted, the analysis of transcripts and subjective research techniques may also be of value in informing future directions.
In this study, conditions with less variability in disease severity and course demonstrated higher consensus and, as a result, may be more amenable to standard policy regarding the priority of the need for IVIG therapy. For highly variable conditions, discussion by a multispecialist panel improved the consensus on the IVIG therapy rating. The experts were likely to reach agreement for the use of IVIG therapy based on dialogue with peers and clinical experts. The follow-up discussion and survey emphasized the need for a case-by-case discussion of the more variable or complex autoimmune diseases to determine the priority of IVIG therapy. Furthermore, the panel concluded that evidentiary evaluation should start with Axis 1 (ie, evidence-supported conditions), and once this threshold is met, then Axis 2 (disease severity) and Axis 3 (efficacy of therapeutic alternatives) should be used for rating.
The complicated nature of many rare diseases, and their high variability, underscores the need for institutions to have a P&T committee or similar decision-making body that includes expert clinicians and decision makers from multiple disciplines for discussions of IVIG therapy application and rating utilizing expert perspective and data from the peer-reviewed medical literature.
Conclusions
An IVIG therapy prioritization algorithm, augmented with disease severity and the efficacy of therapeutic alternatives, is a valuable clinical decision-making tool. A multidisciplinary panel improved consensus building among the 7 panel members on the effective use of IVIG therapy in several clinical conditions. Applying this 3-axis consensus-building approach, an expert multidisciplinary panel reached consensus on almost 90% of disease states considered in this study in which IVIG therapy may be used.
Although there was generally high agreement with the AAAAI evidence ratings for the use of IVIG therapy for the surveyed conditions, dialogue among multispecialty panel members addressed the need for the case-by-case discussion of patients with the most variable and complex diseases and improved consensus for the use of IVIG therapy in these conditions. The use of expert multispecialty dialogue is valuable and something that may be considered on a larger scale for coverage decisions by formulary decision makers, potentially even within the FDA or among specialty medical societies. Nevertheless, it is unlikely that there will be a replacement for considering rare cases on an individual basis regarding their suitability for IVIG treatment.
Source of Funding
This study was funded by ASD Healthcare.
Author Disclosure Statement
Dr Orange is a consultant to ASD Healthcare, Baxalta, CSL Behring, Grifols, and Walgreens and is on the scientific advisory board for ADMA Biologics. Mr Johnson, Ms Lennert, Dr Shields, and Dr Eaddy are employees of Xcenda, which was paid by ASD Healthcare to complete this study.
Dr Orange is Chief of Section of Immunology, Allergy and Rheumatology, Texas Children’s Hospital, Houston, TX; Mr Johnson is Chief Operating Officer, ASD Healthcare, AmerisourceBergen, Frisco, TX; Ms Lennert is Vice President, Operational Excellence, Xcenda, AmerisourceBergen, Palm Harbor, FL; Dr Shields is Director of Oncology Medical Communications, Xcenda, AmerisourceBergen; Dr Eaddy is Vice President, Real-World Evidence, Xcenda, AmerisourceBergen.
References
1. Ballow M. Mechanisms of immune regulation by IVIG. Curr Opin Allergy Clin Immunol. 2014;14:509-515.
2. Sriaroon P, Ballow M. Immunoglobulin replacement therapy for primary immunodeficiency. Immunol Allergy Clin North Am. 2015;35:713-730.
3. US Food and Drug Administration. Immune globulin intravenous (IGIV) indications. www.fda.gov/BiologicsBloodVaccines/BloodBloodProducts/ApprovedProducts/LicensedProductsBLAs/FractionatedPlasmaProducts/ucm133691.htm. Accessed April 25, 2016.
4. Sriaroon P, Ballow M. Immunoglobulin replacement therapy for primary immunodeficiency. Immunol Allergy Clin North Am. 2015;35:713-730.
5. Leong H, Stachnik J, Bonk ME, Matuszewski KA. Unlabeled uses of intravenous immune globulin. Am J Health Syst Pharm. 2008;65:1815-1824.
6. Patwa HS, Chaudhry V, Katzberg H, et al. Evidence-based guideline: intravenous immunoglobulin in the treatment of neuromuscular disorders: report of the Therapeutics and Technology Assessment Subcommittee of the American Academy of Neurology. Neurology. 2012;78:1009-1015.
7. Gray O, McDonnell GV, Forbes RB. Intravenous immunoglobulins for multiple sclerosis. Cochrane Database Syst Rev. 2003:CD002936.
8. Centers for Disease Control and Prevention; Infectious Diseases Society of America; American Society for Blood and Marrow Transplantation. Guidelines for preventing opportunistic infections among hematopoietic stem cell transplant recipients: recommendations of CDC, the Infectious Diseases Society of America, and the American Society for Blood and Marrow Transplantation. MMWR Recomm Rep. 2000;49:1-125, CE1-CE7. Erratum in: MMWR Recomm Rep. 2004;53:396.
9. Schulz U, Solidoro P, Müller V, et al. CMV immunoglobulins for the treatment of CMV infections in thoracic transplant recipients. Transplantation. 2016;100(suppl 3):S5-S10.
10. Katz-Agranov N, Khattri S, Zandman-Goddard G. The role of intravenous immunoglobulins in the treatment of rheumatoid arthritis. Autoimmun Rev. 2015;14:651-658.
11. Wong LF, Porter TF, Scott JR. Immunotherapy for recurrent miscarriage. Cochrane Database Syst Rev. 2014:CD000112.
12. Royal College of Obstetricians and Gynaecologists. The Investigation and Treatment of Couples with Recurrent First-Trimester and Second-Trimester Miscarriage. Green-top Guideline No 17. London, England: Royal College of Obstetricians and Gynaecologists; 2011. www.rcog.org.uk/globalassets/documents/guidelines/gtg_17.pdf. Accessed April 19, 2016.
13. Orange JS, Hossny EM, Weiler CR, et al; for the Primary Immunodeficiency Committee of the American Academy of Allergy, Asthma and Immunology. Use of intravenous immunoglobulin in human disease: a review of evidence by members of the Primary Immunodeficiency Committee of the American Academy of Allergy, Asthma and Immunology. J Allergy Clin Immunol. 2006;117(4 suppl):S525-S553. Erratum in: J Allergy Clin Immunol. 2006;117:1483.
14. Späth PJ, Granata G, La Marra F, et al. On the dark side of therapies with immunoglobulin concentrates: the adverse events. Front Immunol. 2015;6:11.
15. Anderson D, Ali K, Blanchette V, et al. Guidelines on the use of intravenous immune globulin for hematologic conditions. Transfus Med Rev. 2007;21(2 suppl 1):S9-S56.
16. Shehata N, Palda VA, Meyer RM, et al. The use of immunoglobulin therapy for patients undergoing solid organ transplantation: an evidence-based practice guideline. Transfus Med Rev. 2010;24(suppl 1):S7-S27.
17. Centre for Evidence Based Medicine. Prof. Carl Heneghan evidence-based medicine. www.cebm.net/prof-carl-heneghan-evidence-based-medicine/. Accessed September 12, 2014.
18. Council of Medical Specialty Societies. Principles for the development of specialty society clinical guidelines. September 2012. http://cmss.org/wp-content/uploads/2016/02/CMSS-Principles-for-the-Development-of-Specialty-Society-Guidelines-September-20122.pdf. Accessed April 9, 2016.
19. Tyler LS, Cole SW, May JR, et al; for the ASHP Expert Panel on Formulary Management. ASHP guidelines on the pharmacy and therapeutics committee and the formulary system. Am J Health Syst Pharm. 2008;65:1272-1283.
20. Immune Deficiency Foundation. Hospital usage of intravenous immunoglobulin: a 2006 survey of pharmacy directors. http://primaryimmune.org/idf-survey-research-center/idf-surveys/ivig-access-surveys/?aid=1369&sa=1. Accessed July 15, 2014.
21. Orange JS, Ochs HD, Cunningham-Rundles C. Prioritization of evidence-based indications for intravenous immunoglobulin. J Clin Immunol. 2013;33:1033-1036.
22. Clayton MJ. Delphi: a technique to harness expert opinion for critical decision-making tasks in education. Educ Psychol. 1997;17:373-386.
23. Mitchell VW. The Delphi technique: an exposition and application. Technol Anal Strat Manage. 1991;3:333-358.
24. Dalkey N, Helmer O. An experimental application of the Delphi method to the use of experts. Manag Sci. 1963;9:458-467.
25. Dalkey NC. The Delphi method: an experimental study of group opinion. Rand Corp Public RM-58888-PR. Santa Monica: Rand Corporation; June 1969. www.rand.org/content/dam/rand/pubs/research_memoranda/2005/RM5888.pdf. Accessed April 5, 2017.
26. Meshkat B, Cowman S, Gethin G, et al. Using an e-Delphi technique in achieving consensus across disciplines for developing best practice in day surgery in Ireland. J Hosp Adm. 2014;3:1-8.
27. Bennett C, Vakil N, Bergman J, et al. Consensus statements for management of Barrett’s dysplasia and early-stage esophageal adenocarcinoma, based on a Delphi process. Gastroenterology. 2012;143:336-346.
28. Murphy MK, Black NA, Lamping DL, et al. Consensus development methods, and their use in clinical guideline development. Health Technol Assess. 1998;2:i-iv, 1-88.
29. Rayens MK, Hahn EJ. Building consensus using the policy Delphi method. Policy Polit Nurs Pract. 2000;1:308-315.
30. Raskin MS. The Delphi study in field instructions revisited: expert consensus on issues and research priorities. J Social Work Education. 1994;30:75-89.