Nowhere is this caveat from David Blumenthal, MD, MPP, the former National Coordinator for Health Information Technology, more applicable than in the emergency department setting. Although originally designed as the section of a hospital where only the most acutely ill persons should seek care for their maladies, the emergency department has become much more than that. It now serves as the primary care provider for many who have no such physician outside the emergency department.2,3
In addition, the emergency department provides a triage function for nonemergent cases that have no reason to be seen in the emergency department yet continue to increase in number.4,5 Finally, the emergency department coordinates care for individuals who have chronic medical conditions.6-8
Such emergency department care results in increased quality of care.9 Many of the increased expenditures may be directly traced to redundant diagnostic testing.10 Moreover, it is costly for health plans: emergency department care makes up 7% of a health plan’s budget.11 Given that individuals with health insurance drive the increasing use of the emergency department, this issue will continue to be problematic for health plans.12
For these reasons, many tout health information exchanges (HIEs)—where clinical data are exchanged between hospitals, providers, public health administrators, and, potentially, payers—as a method of addressing emergency department overutilization.13 Payer participation in HIEs can promote care coordination and cost control for the end user—the plan’s members—as well as create value for the plan’s customers—the employers. Moreover, because HIEs tend to view payers as receiving the greatest benefit from HIE, many believe that payer support of the exchanges provides a path toward sustainability.14
For payers to invest in HIE voluntarily, they must see the business case for doing so.15 Yet, up to now, only scant evaluations of measured HIE benefits can be found.16 Ultimately, assessing the effectiveness of HIE between multiple facilities in a community can show payers the rationale for having such an exchange from the individual health, population health, and financial perspective. A positive evaluation helps to promote the business case for continued support of these exchanges.17 This was the purpose of the present study.
Beginning in December 2008, Humana in southeast Wisconsin became the first local health plan—and one of the first in the nation—to provide a financial incentive to the local HIE for promoting the querying of a clinical database by emergency department clinicians (as a part of their workflow) for our fully insured members who present to the emergency department for care.18
The Wisconsin Health Information Exchange (WHIE; www.whie.org) serves as the vehicle for linking disparate emergency departments across 5 competitive health systems in Milwaukee County.19 The WHIE Emergency Department Linking program provides clinicians access to comprehensive encounter history data including: patient demographics, encounter location, date and time, chief complaint, allergy and reaction, primary care provider, and diagnosis. (In addition, these Medicaid patient data include details on procedures performed and prescription fills.) These encounter data guide providers toward the specific information they need to manage patients. Providers may also post messages related to patient-specific care coordination and encounters. These messages become part of the patient’s history and are available to other providers in the course of patient care.
Evaluating the effectiveness of using the WHIE (or any HIE, for that matter) is challenging, because “the economic value is diffuse, accrues over time, and is difficult to measure.”20 Yet, to determine if Humana receives “value” for its investment, and to promote the business case for a continued investment in WHIE sustainability, Humana analyzed WHIE’s effectiveness in controlling costs for our members who sought emergency care, as described in this article. If the Humana–WHIE relationship demonstrates mutual benefit, then this study could serve to encourage other plans toward an additional investment in HIE, which will advance HIE sustainability, and to embrace HIE standards and services.21
Study Design: Developing the Sample Population for Evaluation
There is no way for a health plan to know which of its members will seek emergency department care and where. Planning this evaluation to address such concerns served as a focal point of discussion as this proposal advanced to approval through the Humana version of an institutional review board—the clinical “stage gate process.” The planned evaluation presumed an observational and retrospective analysis, with the appropriate statistical techniques, such as propensity scoring methodology, for addressing such a quasi-experimental design.
In developing the member pool from which to draw the evaluation population, Humana and the WHIE had agreed in advance that the plan would provide to the WHIE a financial incentive to cover their costs for promoting emergency department clinicians’ querying of the WHIE database for eligible Humana members who presented to the emergency department for care.22WHIE encouraged emergency department clinicians to make querying the WHIE database a standard part of the emergency department workflow for all patients seeking emergency department care, but, for the purposes of this pilot study, Humana provided the incentive only in cases where a query occurred for an eligible Humana member.
Eligible members were commercial, fully insured members. Self-funded group members were specifically excluded, because we could not ensure savings before the analysis and did not want these groups to assume more financial risk than necessary. Therefore, any non–fully insured members or members otherwise covered by public programs (eg, Medicare or Medicaid) would be ineligible for this study.
WHIE provided Humana, on a quarterly basis, specified data about each health plan member who was fully insured by Humana and who sought emergency department care, as well as when the emergency department clinician accessed the WHIE database for that patient and at which emergency department/facility. The data included the last 4 digits of the Social Security number if known, date/time of emergency department registration, group number, group name, policy number if known, and facility (eg, hospital).
In contrast to the clinical data provided to the emergency department clinicians, WHIE provided no clinical data to Humana for each member, except for such data as would appear on a claim related to the clinical encounter. All communications were compliant with the Health Insurance Portability and Accountability Act and used encrypted files.
The information provided allowed Humana to match emergency department claims data received from providers with the emergency department encounter by date of service and facility. Once we matched member claims for emergency department services with the WHIE file, we then looked at specific procedures and associated costs for an individual member’s given encounter.
In working with claims data, we needed to stipulate explicit parameters for member inclusion in the evaluation sample. Inclusion criteria were:
- All members included in the evaluation must have had at least 12 months of continuous coverage with our health plan
- Members would be excluded from the evaluation if they had either less than 6 months of coverage before the start of the program or less than 3 months of coverage after the start of the program
- Because plan cost would be the key parameter of evaluation, we excluded potential outliers from the analysis.
An “outlier” was defined as someone who had exceeded $10,000 in claims during a single emergency department visit. This exclusion prevented potential skewing of the data by a member who might have been held as a “24-hour observation” in the emergency department rather than admitted to the hospital (where the emergency department costs roll into the total admission cost and would not be included in our analysis). If the member were not admitted and not held as a “24-hour observation,” our data showed that such an individual did not exceed $10,000 in claims for a single emergency department encounter.
Study Design: Criteria for the Test Group and the Control Group
We identified members who were seen in the emergency department when the WHIE database was queried as eligible for the test group; members who were seen in the emergency department where the WHIE database was not queried (because the facility had not yet provided WHIE access at that time) were identified as eligible for the control group.
For the test group, there were 428 plan members presenting for emergency department care with a WHIE database query in both a first emergency department visit and any subsequent emergency department visit. Alternatively, in the control group, there were 1054 plan members who presented for emergency department care without a WHIE database query in either a first emergency department visit or any subsequent emergency department visit.
In addition, for this observational and retrospective study, propensity scoring afforded the best way to match members and minimize bias. Propensity scoring provides “the conditional probability of receiving the treatment given the observed covariates.”23 For our purposes, a member for whom WHIE database querying occurred at the point of care would have a dependent variable in the logistic regression of “y = 1,” whereas a member whose care did not include WHIE database querying would have a dependent variable of “y = 0.”
In their seminal article, Rosenbaum and Rubin showed that the “adjustment for the scalar propensity score is sufficient to remove bias due to all observed covariates.”24 Furthermore, propensity scoring has been found to yield estimates that are not substantially different from typical multivariable methods.25,26
For the logistic regression analysis, we used all the following combinations of cost-related and demographic variables to match the 2 groups—age, gender, medical net costs paid per participant per month (PPPM), net prescription costs paid PPPM, medical plus net prescription costs paid PPPM, medical inpatient net costs paid PPPM, medical outpatient net costs paid PPPM, emergency department net costs paid PPPM, and medical physician net costs paid PPPM. With the exception of age and gender, all these variables represent dollar values, because not only were those the outcomes of interest, but they were also unrelated to the specific dependent variable (WHIE database querying). Simulation studies have shown that one should always include “variables that are unrelated to the exposure but related to the outcome” in a propensity scoring model.27
The econometrician among us, coauthor Victor Lawnicki, PhD, developed the propensity scores with which we matched the participants using the nearest neighbor algorithm. Matching allows for “sampling from a large reservoir of potential controls to produce a control group of modest size in which the distribution of covariates is similar to the distribution in the treated group.”24 MATLAB version 126.96.36.199 was used for member matching.28
Once we completed matching 325 pairs of individuals for the test and control groups, we analyzed the 2 groups for differences in the metrics of interest. SAS Enterprise Guide version 4.2 was used for descriptive, matched-pair t-tests and other statistics.29 We compared claims for the 2 groups for a time period beginning 1 year before an individual’s first emergency department visit to an end date of 1 year after that first emergency department visit: therefore, each individual’s length of time in the pilot was 1 full year. The pilot ran from December 2008 through March 2010.
To assess total emergency department costs for an individual emergency department visit, we used allowed claims dollars to evaluate if the test group achieved savings for second and subsequent emergency department visits compared with the control group.
We also assessed procedure utilization through billed Current Procedural Terminology codes, including imaging studies, laboratory studies, and therapies, to see how changes in utilization potentially affected emergency department costs per visit. In these analyses we compared the differences in emergency department claims dollars or number of emergency department services rendered between a first and subsequent emergency department visit for the 2 populations of interest after adjustment for the trend between the 2 time periods (which would be a factor if fee schedules at a given facility changed between the 2 emergency department visits).
Descriptive results before matching for all eligible control population and test population members are shown in Table 1. These preliminary, unadjusted results show that in the aggregate, the total emergency department cost differences for all members in the test group candidate pool (where querying of the WHIE database occurred for the initial and subsequent emergency department visits) decreased by $186 per emergency department visit compared with the cost differences noted for all members representing the control group candidate pool.
After propensity score matching, Table 2 shows that after a first emergency department visit, the test population has higher costs in nearly all subcategories contributing to the net dollars paid PPPM. This could imply that the test population requires higher intensity care on a total claims dollar basis; it is possible to consider them a “sicker” population based on claims. Second, none of the differences in dollars spent for the test group are significant (a = .05), except for “medical physician” and “emergency department.”
These differences imply that utilizing HIE in the emergency department affects the costs for some care taking place in the emergency department, including physician costs and emergency department facility costs. However, HIE use in the emergency department may not impact medical costs outside the emergency department.
The emergency department subgroup comparison looks at emergency department costs for participants for all services provided during a single emergency depart - ment encounter (Table 3). Those results show that Humana achieved an average savings of $29 for each emergency department visit where the WHIE was queried.
Given that the test group experienced higher overall claim costs than the control group, any dollar impact reducing costs becomes important. Potential drivers of the $29-per-emergency-department visit savings when the WHIE was queried are shown in Table 4. For the top 5 emergency department– based procedures, which are shown in Table 4, we found definitive decreases in the test group for 4 of them. These decreases in test redundancy help to mitigate waste and control costs.
Such findings bear out that substantial change in outcomes that matter is “clinically important,” regardless of statistical significance.30
The overarching goal of this study was to evaluate the WHIE’s effectiveness as a means of cost control for Humana members seeking emergency care. A secondary, albeit important, goal involved the assessment of overall value to Humana: if the business case for a health plan’s investment in HIE can be supported, then this study could validate the literature in proposing that health plans promote HIE sustainability.
Review of these results reveals several interesting points. Consistent with our hypothesis, we found that when the WHIE was queried, our health plan achieved an average savings of $29 per emergency department visit. We believe that the savings are driven by 2 factors—( 1) the availability of medical history at the point of care, and (2) a decrease in redundant diagnostic testing resulting from the availability of that medical history. For physicians, the ability to care for a patient in the emergency department has long been limited by the availability of information. As Cory Wilson, MD, Chair Milwaukee, WI, states in a video about the WHIE, “it [the WHIE] gives us immediate information that we can use at the bedside, and information, for an emergency physician, is gold.”31
Preliminary survey results of emergency department physicians utilizing the WHIE at the point of care for patient management demonstrated that workup or treatment of the patient was altered 42% of the time.32 Furthermore, the time spent gathering data decreased by 42% and the time to the disposition decision decreased approximately 50% of the time.32
An analysis performed by the Agency for Healthcare Research and Quality highlighted the impact that the availability of electronic information can have on patient care; it can improve provider performance when the data are readily accessible.33 From the health plan business case perspective, improving provider performance has a cost-savings implication as well.34
The cost-savings health plans accrue are important for several reasons. First, if health plans save money when their members seek emergency department services, the plan can, among many options, share those savings with providers (both physicians and HIEs) to support HIE utilization and sustainability. In fact, early HIE adopters, such as the California Regional Health Information Organization, created a “shared savings” model whereby providers and payers would share the savings attained through quality improvements and cost reductions.35
Within the initial context of HIE, early national costsavings projections were estimated at $78 billion annually once implemented.36 Estimates of local HIE cost-savings may serve as more appropriate comparators to what our results showed. Overhage and colleagues published data in 2002 from the Indiana Health Information Exchange, which estimated that clinical information shared between facilities saved $26 per emergency department visit by eliminating duplicate tests and other unnecessary activities.37,38
More recently, Daniel and colleagues concluded that the utilization of a payer-based electronic health record in an emergency department resulted in a mean cost-sav-ings of $1560 for each emergency department encounter that led to an admission compared with admissions from the emergency department with no electronic health record access.39
Even cost reductions of as little as $10 per emergency department visit could yield substantial savings for health plans.40 Based on these examples, there is definite potential for health plans to save money by promoting HIE’s sustainability and its use in the emergency department. Such a finding may also help to explain why, after Humana’s promotion of this effort, other national payers have started to support local HIE efforts.41
Second, we theorize that the cost-savings attained relate to decreased ordering of specific tests performed in the emergency department. Overhage and colleagues, along with Frisse and Holmes, noted that operational metrics that eliminate redundancies and reduce costs, such as duplicate imaging and lab tests, specifically decreased when emergency department physicians used HIEs.37,40 Anecdotal evidence before our study had even shown that querying the WHIE avoided the ordering of unnecessary ultrasounds, computed tomography scans, and, in 1 case, an angiogram.42
Moreover, for the top 5 emergency department–based procedures identified in our study, the test group experienced decreases in 4 of them; for the fifth, we hypothesize that because many individuals arrive in the emergency department via ambulance, or because of their chief complaint, they require intravenous (IV) access for potential medication, the fact that IV therapy was seen more frequently in the test group during an emergency department visit may not be a complete surprise for a higher-cost population.
We need to account for several potential limitations to this study. First, with respect to “internal validity,” although the use of propensity scoring methods to create test and control groups should minimize bias, any time data manipulation occurs, new risks from potential bias must be acknowledged. Although it would have seemed ideal to use emergency department visits from the same individual on a “pretest/posttest” basis, this too is fraught with problems. There is no guarantee that the same member may be seen for the same condition in the emergency department each time; “internal validity” may improve but “external validity” is sacrificed.43
In addition to potential internal validity issues, there are potential external validity issues. In previous studies of community health information networks, Wisconsin had experienced cost-savings, whereas other community health information networks had not.44,45 It is possible that outcomes found in Wisconsin may not transfer elsewhere. Furthermore, although Daniel and colleagues had found cost-savings for patients admitted from the emergency department, they did not find any cost-savings for patients discharged from the emergency department.39 This study, however, looked at HIE using clinical data rather than Daniel and colleagues’ payer-based data.
Second, we used a commercial, fully insured population in Milwaukee, WI. It is possible that similar costsavings may not be achieved for a self-funded population, a Medicaid population, or a Medicare population. Contractual reimbursement rates for these differing products may play a role. Nonetheless, in comparing “like” groups through propensity scoring, reimbursement amounts for similar products should, in theory, cancel out. Moreover, because we know that the Indiana HIE experienced cost-savings similar to ours, one could view any savings as functionally related to the use of HIE rather than to extraneous factors.
Finally, when it comes to addressing the financial arrangements between payers and HIEs, the details of such arrangements must be established on a case-by-case basis. In our case, the cost-savings Humana realized in the pilot study exceeded the incentive Humana paid to the WHIE. We have chosen not to state the exact incentive amount, because doing so may compromise current or future stakeholders’ ability to adjust their administrative costs for future obligations.
Still, although we are pleased to report that our return on investment was better than 2:1, the acknowledged point is that the basic “value” realized is greater than the financial commitment: not only do providers and patients benefit, but payers do realize a true return on their investment. Pilot programs from health plans in collaboration with HIEs in other markets may serve to substantiate this point further.
This study demonstrates that using a community based HIE in the emergency department can yield cost savings for health plans; the premise of HIE as a cost minimizer and care enhancer is reinforced, whether savings ensue from decreased service utilization or through the provision of historical medical information. Health plan savings allow this model to develop into a potential revenue stream for HIEs, which helps to ensure HIE sustainability in the absence of public funding. From Humana’s perspective, these positive results encourage us to play a leadership role in other community HIEs as they arise in areas where we have a business presence.
Ultimately, such assessments of community-based HIEs in emergency department care have significant benefits. They provide evidence of cost-savings, service important, these partnerships help to improve the health of the whole community as they address the continuing proliferation of emergency department use by the insured—a predicament that is not going away anytime soon.12
When it comes to addressing the balance between service and sustainability for HIEs, one may ask what factor makes community-based HIEs a necessity. The answer is simple—“a compelling mission whose clinical and economic value is widely acknowledged and measurably demonstrated.”46
We plan to continue the collaboration between Humana and the WHIE; our study demonstrated that we gained an economic value. We found that as payers invest in HIEs, they receive a positive financial return on their investment. Although these cost-savings realized by the health plan may be used in a number of ways, one option allows health plans to invest in HIE funding, thereby improving HIE sustainability.
Such decisions can benefit the community at large in addition to the plan itself. Morrissey perhaps expressed it best, “To put it short and sweet, health information exchange makes business sense now.”47
The authors wish to acknowledge and thank the following individuals for their advocacy and support during the development, implementation, and evaluation of this project—Marcia James, Karen McClay, Andrew Osterman, and Dr Michael Sherman from Humana’s Clinical Leadership & Policy Development organization; Chunmei Wang from Humana’s Business Intelligence and Informatics Competency Center; and members of the Milwaukee Health Care Partnership, the Wisconsin Department of Health Services, and the Wisconsin Hospital Association, as well as other members of the Board of Advisors from the WHIE. Without the support of these individuals, their employers, and the groups they represent, this project could have never come to fruition. The authors also wish to thank Humana, Louisville, KY, for its financial support for this project.
Author Disclosure Statement
Dr Tzeel is employed by and owns stock in Humana and is a nonpaid member of the WHIE Board of Advisors, as well as a volunteer member of the WHIE Sustainability Committee; Dr Lawnicki is an employee and shareholder at Humana; Mr Pemble is employed by the National Institute for Medical Informatics/WHIE
- Blumenthal D. The HITECH Foundation for Information Exchange, a message from Dr. David Blumenthal, National Coordinator for Health Information Technology. Update #4. US Department of Health & Human Services, Office of the National Coordinator for Health Information Technology. November 12, 2009. http://healthit.hhs.gov/portal/server.pt?open=512&objID=1406&parentname=Com munityPage&parentid=6&mode=2&in_hi_userid=10741&cached=true. Accessed April 15, 2010.
- Glick DF, Thompson KM. Analysis of emergency room use for primary care needs. Nurs Econ. 1997;15:42-49.
- Grumbach K, Keane D, Bindman A. Primary care and public emergency department overcrowding. Am J Public Health. 1993;83:372-378.
- Young GP, Wagner MB, Kellerman AL, et al. Ambulatory visits to hospital emergency departments. Patterns and reasons for use. 24 hours in the ED Study Group. JAMA. 1996;276:460-465.
- Tufts Managed Care Institute. Emergency department utilization: trends and management. Nov-Dec 2001. http://www.thci.org/downloads/topic1112_01.pdf. Accessed July 7, 2010.
- Mollica RL, Gillespie J. Care coordination for people with chronic conditions. Partnership for Solutions. January 2003. http://www.partnershipforsolutions.org/DMS/files/ Care_coordination.pdf. Accessed April 14, 2010.
- Kulkarni R, Pell F, Agocs-Holler EJ, D’Onofrio G. 227: Care coordination in the emergency department: avoiding inappropriate hospital admissions. Ann Emerg Med. 2007;50(suppl):S71-S72.
- Brigham and Women’s Hospital. Care coordination. http://www.brighamandwomens.org/Patients_Visitors/pcs/carecoordination/default.aspx. Accessed May 1, 2010.
- Laborers Health and Safety Fund of North America. Emergency room care: slow, questionable and costly. Lifelines Online. Vol 3, No 2. July 2006. http://www.lhsfna.org/index.cfm?objectID=120F0D19-D56F-E6FA-9384F31044F34810. Accessed April 14, 2010.
- van Walraven C, Raymond M. Population-based study of repeat laboratory testing. Clin Chem. 2003;49:1997-2005.
- Page L. Patients bypassing primary doctors for emergency care. American Medical News. February 12, 2001. http://www.ama-assn.org/amednews/2001/02/12/bil10212.htm. Accessed April 14, 2010.
- DeLia D, Cantor J. Research Synthesis Report No. 17: emergency department utilization and capacity. Robert Wood Johnson Foundation. July 2009. http://www.rwjf.org/ files/research/072109policysynthesis17.emergencyutilization.pdf. Accessed November 10, 2010.
- Community Health Integrated Partnership. Emergency Department Over- Utilization: A New Paradigm? May 27, 2009. http://www.dhmh.state.md.us/mma/pdf/2009/ jun09/Emergency_Department_Usage_052209.pdf. Accessed July 14, 2011.
- Goedert J. The payer role in HIEs. Health Data Management Magazine. June 1, 2009. http://www.healthdatamanagement.com/issues/2009_67/-28270-1.html?pg=1. Accessed April 14, 2010.
- Vest JR, Gamm LD. Health information exchange: persistent challenges and new strategies. J Am Med Inform Assoc. 2010;17:288-294.
- Hripcsak G, Kaushal R, Johnson KB, et al. The United Hospital Fund meeting on evaluating health information exchange. J Biomed Inform. 2007;40(suppl 6):S3-S10.
- Veryard R. Making the business case. SCIPIO. February 10, 1999. http://www.users.globalnet.co.uk/~rxv/scipio/swpbc.pdf. Accessed April 14, 2010.
- Neupert P. Re-charting healthcare: innovations to drive a new delivery model for tomorrow’s health system. In: Merritt D, ed. Paper Kills 2.0: How Health IT Can Help Save Your Life and Your Money. Washington, DC: Center for Health Transfor mation Press;2010:15.
- Wisconsin Health Information Exchange. Information exchange service links emergency departments and community health centers. 2009. http://www.whie.org/ edlink.php. Accessed April 14, 2010.
- Ozanich GW, Chrisman K, Jones Dolen R, et al. State health information exchange: factors shaping sustainability and value. J Healthc Inf Manag. 2011;25:48-55.
- Dixon BE, Zafar A, Overhage JM. A framework for evaluating the costs, effort, and value of nationwide health information exchange. J Am Med Inform Assoc. 2010;17:295-301.
- Humana to partner with WHIE on emergency department data exchange. Wisconsin Technology Network News. January 29, 2009. http://wistechnology.com/ articles/5432/. Accessed April 14, 2010.
- Rosenbaum PR. Observational Studies. 2nd edition. New York, NY: Springer- Verlag; 2002:296.
- Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70:41-55.
- Shah BR, Laupacis A, Hux JE, Austin PC. Propensity score methods gave similar results to traditional regression modeling in observational studies: a systematic review. J Clin Epidemiol. 2005;58:550-559.
- Stürmer T, Joshi M, Glynn RJ, et al. A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. J Clin Epidemiol. 2006;59:437-447.
- Brookhart MA, Schneeweis S, Rothman KJ, et al. Variable selection for propensity score models. Am J Epidemiol. 2006;163:1149-1156.
- MathWorks. MATLAB – The language of technical computing. http://www.mathworks.com/products/matlab/. Accessed June 18, 2010.
- SAS Institute. SAS enterprise guide, a graphical user interface for deploying the power of SAS analytics. http://www.sas.com/technologies/bi/query_reporting/guide/. Accessed June 18, 2010.
- Primer on statistical significance and P values. Eff Clin Pract. 2001;4:183-184.
- Microsoft Showcase. Wisconsin Health Information Exchange [video]. 2008. http://www.microsoft.com/showcase/en/US/details/ab8cd114-583c-472f-87ac791 ebf531938. Accessed April 14, 2010.
- Kolbasuk McGee M. Health information exchange enhances decision making. June 16, 2010. InformationWeek. http://www.informationweek.com/story/showArticle. jhtml?articleID=225700387. Accessed June 15, 2011.
- Shekelle PG, Morton SC, Keeler EB. Costs and Benefits of Health Information Technology. Evidence Report/Technology Assessment No. 132. Agency for Healthcare Research and Quality Publication No. 06-E006. Rockville, MD: Agency for Healthcare Research and Quality. April 2006.
- American Medical Association. Terminology used in physician profiling. 2009. http://www.ama-assn.org/ama1/pub/upload/mm/368/profiling-terminology.pdf. Accessed April 14, 2010.
- CalRHIO disbands, in lieu of new ARRA-funded statewide HIE. January 13, 2010. http://www.healthimaging.com/index.php?option=com_articles&view=article& id=20182:calrhio-disbands-in-lieu-of-new-arra-funded-statewide-hie. Accessed April 14, 2010.
- Walker J, Pan E, Johnston D, et al. The value of health care information exchange and interoperability. Health Aff (Millwood). 2005;Suppl web exclusives: W5-10-W5-18.
- Overhage JM, Dexter PR, Perkins SM, et al. A randomized, controlled trial of clinical information shared from another institution. Ann Emerg Med. 2002;39:14-23.
- Patton S. Sharing data, saving lives. CIO. 2005;18:64-68.
- Daniel GW, Ewen E, Willey VJ, et al. Efficiency and economic benefits of a payer-based electronic health record in an emergency department. Acad Emerg Med. 2010;17:824-833.
- Frisse ME, Holmes RL. Estimated financial savings associated with health information exchange and ambulatory care referral. J Biomed Inform. 2007;40(6 suppl): S27-S32.
- UnitedHealthcare. UnitedHealthcare partners with CalRHIO to expand health information statewide. May 20, 2009. http://www.uhc.com/news_room/2009_news_ release_archive/unitedhealthcare_partners_with_calrhio.htm. Accessed April 14, 2010.
- Langreth R. Wiring medicine. Forbes. 2009;183:40-42.
- Shuttleworth M. Pretest-posttest designs. 2009. http://www.experiment-resources.com/ pretest-posttest-designs.html. Accessed May 23, 2010.
- Lassila KS, Pemble KR, DuPont LA, Cheng RH. Assessing the impact of community health information networks: a multisite field study of the Wisconsin Health Information Network. Top Health Inf Manage. 1997;18:64-76.
- Soper P. Realizing the Potential of Community Health Informaton Networks for Improved Quality and Efficiency Through the Continuum of Care: A Case Study of the HRSA Community Access Program and the Nebraska Panhandle Partnership for Health and Human Services. Scientific Technologies Corporation. December 2001. http://www.stchome.com/media/white_papers/WHP023A.pdf. Accessed April 14, 2010.
- Krohn R. The socially responsible HIE: finding the optimal mix of service and sustainability. J Healthc Inf Manag. 2011;25:12-13. 47. Morrissey J. HIE health information exchange. Hospitals Health Networks. February 2011. http://www.hhnmag.com/hhnmag_app/jsp/articledisplay.jsp?dcrpath= HHNMAG/Article/data/02FEB2011/0211HHN_Coverstory&domain=HHNMAG. Accessed April 14, 2011.