Applying Evidence for Medical Technologies: Closing the Gap between R&D and Decision Maker Needs

Robert Henry: Our goal is to draw on your current position at the Center for Medical Technology Policy (CMTP), as well as your previous experience at the Centers for Medicare & Medicaid Services (CMS), to discuss private payer initiatives for medical technologies research and development (R&D) and the role of evidence in developing new technologies.

Sean R. Tunis, MD, MSc: Let me begin by saying that CMTP is a nonprofit organization that began in 2006 with grants from the California Healthcare Foundation and Blue Shield California Foundation, as well as some funding from the Commonwealth Fund. The goal was to create a platform that allows different stakeholders to address the lack of good qualitative evidence for many medical technologies, including existing and emerging technologies, and its impact on policy. By medical technologies I mean drugs, medical devices, procedures, and interventions—especially interventions in a chronic care setting.

The broad question for CMTP was how to bring people together to understand why there are such systematic gaps in the evidence for emerging medical technologies. This was based on my experience as the person in charge of clinical policy for Medicare at CMS, having spent 5 years trying to make policy decisions more explicitly evidence-based. At CMS, no matter what technology we looked at, we always concluded that the evidence was not as good as we would wish for, often leading policymakers to conclude that the evidence is inadequate and request that more trials be done. CMTP was set up to explore why policymakers never have the evidence that they want.

Henry: Do you connect this effort to benefit design?

Tunis: In terms of benefit design, my interest is in designs that encourage the development of evidence concerning the effectiveness of clinical services. The work we did at CMS, and our focus on coverage with evidence development (CED), was an effort to use benefit design as a tool for improving the quality of evidence around specific clinical services and technologies.

CED means that the drug or device will only be reimbursed under the condition that patients participate in a properly designed study. There has been a lot of effort to articulate the legal foundation for CED and to be more specific about the types of studies that would be eligible for reimbursement under CED. Medicare has done much work trying to integrate the clinical trials policy with CED to have a coherent, broad approach to subsidizing clinical research. CMS had meetings with the Medicare Payment Advisory Commission to discuss the research priorities that might inform when Medicare would apply CED.

There have been new coverage decisions based on Medicare’s CED proposals. The most recent decision was about continuous positive airway pressure devices for treating obstructive sleep apnea, where for certain uses, it would be paid for only in the context of clinical trials, and the coverage decision articulates a fairly detailed list of criteria or what sort of studies would be eligible. Medicare is also extending coverage under the CED paradigm for the assist device for artificial hearts. So, CMS is using CED with some degree of frequency, and there is much evidence that Medicare (and the Department of Health and Human Services more broadly) is committed to continuing the efforts to figure out how best to use CED as a tool to generate better evidence.

CMTP is aiming to replicate these efforts in the private sector; it is collaborating with several large payers and purchasers on a specific project to develop potential CED contract language for private payers and purchasers. The contract language would allow payers to apply it across the board of therapeutic areas, but CMTP is also developing specifications for what type of clinical services and what kind of studies would be eligible.

The point of the project is to first think about what kind of contract language would be necessary to allow health plans to do CED when they want to, but also to develop the specifications for under what conditions, in what type of studies, and for what kind of services with the user. For now, the project is not intended to be specific about which clinical services or technologies it would be applied to.

This latest initiative grew out of the regulatory framework that Medicare had developed for CED, based on their reasonable and necessary language. But it became clear that it is not sufficient to have only Medicare doing CED. Many CED questions address populations beyond the Medicare population. At CMTP, we felt it would be worthwhile to explore how to create a legal construct in the contract language that would allow private payers to do CED under the right circumstances. At the payer level, those involved in CED would include the medical directors, the pharmacy directors, and Pharmacy and Therapeutics Committee members—the people who currently do coverage policy, pharmacy, and therapeutics.

Henry: How long has this initiative been in process?

Tunis: It started about one and a half years ago. Last year we had a meeting in Washington, DC, that included participants from Aetna, UnitedHealthcare, Blue Shield of California, National Business Group on Health, and General Electric Healthcare. We also had some patient representatives, including the Service Employees International Union. There was a lot of enthusiasm for the notion of developing a contractual framework for reimbursements for services that are limited to participation in clinical studies as a way of developing evidence for promising, but unproved, technologies and services.

Henry: What are the most promising areas that have an insufficient body of evidence?

Tunis: One area is proton beam therapy for prostate cancer. Others are new imaging technologies, such as positron-emitting tomography scanners, new applications for high-resolution computed tomography (CT) scanning, molecular diagnostic tests, gene expression tests for guided cancer therapy, minimally invasive surgeries for, say, weight reduction, or any minimally invasive variation of an existing surgical procedure. Another area is the next generation of pharmaceuticals, for example, biologic versions for existing drugs for clinical categories where alternative pharmacy interventions are available (eg, osteoporosis, rheumatoid arthritis, or oncology). These next-generation technologies may have benefits over existing therapies, but there often is limited evidence on the true comparative risks and benefits involved.

Henry: Is what you are doing at CMTP going to have an impact on coverage for off-label use, say, in cancer therapy?

Tunis: There may be important clinical benefits to an off-label use of an approved product, but it is unlikely that new indications would receive US Food and Drug Administration (FDA) approval in the near-term. Some of those off-label uses could actually be paid for in the context of clinical trials under a CED initiative.

Henry: Would that generate sufficient evidence to perhaps reconcile the gap between approved indications and those listed in clinical practice guidelines?

Tunis: Yes. Our initiative at CMTP is still in the stage of developing an agenda and priorities. I would agree that this involves the questions that are important particularly to payers, patients, and clinicians—developing the information that they need to provide better care. We are trying to create a forum within which that research agenda becomes a high priority, and push a clinical and health services research agenda that is more reflective of the information needs of decision makers.

Henry: Are providers going to hear about what you are doing, or is this primarily directed at payers and purchasers?

Tunis: CMTP is equally designed to serve payers, providers, and patients—these are the subgroups of decision makers for whom we wish to generate reliable evidence. As such, we have to ascertain their most important questions, areas in which they need evidence that they don’t currently have. The point about the impact of compliance with guidelines on patient outcomes is a good example of a category of questions that are insufficiently addressed in the current research paradigm.

Henry: How did the concept for this project evolve?

Tunis: We had a small work group of about 20 people who were talking about creating “model benefit” parameters for CED. We were not focused on any specific question but were trying to identify what the important unanswered questions were within that particular project. I was focused on the broader notion of creating an analogy to the Medicare CED framework, and how would we set that up for private payers and purchasers.

Henry: Could you explore the issues of CED a bit more?

Tunis: A good example of a promising but unproved medical technology is coronary CT angiography, which is an initial diagnostic test for patients with chest pain. There is a debate about whether the evidence is adequate for that to be paid for, and whether enough is known about that particular technology. Last year Medicare issued a decision saying that the evidence was not adequate, but all of its contractors had been paying for that test already. This is one example of where CMS believed the evidence was not adequate, but it could not do anything, because its contractors were already paying.

Most private health plans that have to make a decision about coronary CT angiography have got to decide whether it is a medically necessary technology or it is an investigational technology—those are their 2 choices. We want to create a third choice, which is the category of “promising but unproved” technologies. This would allow us to determine how the test influences patient outcomes; the plans would pay for it, but only for patients who are enrolled in a large clinical study that will provide more evidence to that end.

Henry: What you are doing is helping to provide a repository for these data?

Tunis: There is no policy framework currently that allows health plans to collect evidence; benefit language says that a new service or a product is either investigational or it is medically necessary. There is no policy language in any benefit contract that says that the plan will pay for things that are promising, under the condition that data collection will be accompanying the provision of the service.

Henry: So, you are trying to construct a benefit design that allows reimbursement for promising technologies in the context of a study that will determine the degree to which they are clinically effective. How are you eliciting the input of the patients?

Tunis: For every one of our work groups and every project that CMTP does, we have at least 2 and sometimes 3 patient/consumer citizen representatives. They are on every work group; no matter how technical the issue is, we make sure we have them there, and as we develop the governance structure for CMTP, we are also in the process of developing our board of directors and our advisory structure to include patient and consumer representatives.

First, the majority of the board of directors of CMTP is going to be patients and providers. We are going to make sure that the agenda and the structure and all our activities are built around the needs of clinicians, providers, and patients, and that will be built into our governance structure. Second, we are going to have an advisory group, or a steering group, that is separate from the board of directors and is filled exclusively with patient and consumer representatives.

The job of that advisory group will be to continually monitor all our projects and activities, as well as our funding mechanisms, to make sure that ultimately our organization is serving the information needs of patients and providers.

Henry: Are these divided across specific disease states?

Tunis: No, we are being ad hoc about it. Obviously, there tend to be the major diseases, such as cancer or heart disease, but we probably have to be more thoughtful about making sure that we also represent less common illnesses.

Henry: Is your focus primarily on chronic diseases or is it split between chronic and acute conditions?

Tunis: We focus on the major causes of morbidity and mortality—cancer, heart disease, joint disease. As the organization matures, presumably our patient and consumer advisory committee will provide us feedback on how to most effectively focus on various disease areas, chronic disease versus acute illness, and so on.

Henry: And how will the research results get transmitted back to decision makers (payers and purchasers) in terms of benefit design and device/drug utilization?

Tunis: There are a number of answers. We have all the major potential users of the research involved up front in helping us to design studies, select topics, and so on. We are engaging the decision makers—the payers, the providers, and patients—in figuring out not only which questions to ask, but also how to design the studies. And by virtue that the studies are designed from the beginning to answer the questions that are important to decision makers, there is presumably a greater chance that the results will later influence their decision-making.

Part of our work on a standard for medical technology policy is built around the assumption that there is a great need for dialogue across the different stakeholders regarding what are the evidence needs versus what type of evidence is conventionally being produced, and that better dialogue would change the nature of postmarket research.

Listening to the arguments on comparative effectiveness research, I wonder what it is that people think a National Center for Comparative Effectiveness would actually do. That is, what kind of questions would it answer, with what kind of methods? And defining those questions seems to be the hard part, yet nobody is giving much thought to that. As such, we need to be facilitating a dialogue with various stakeholders, not just payers but also product developers, clinicians, and patients.

Many of the reasons why clinical research, health services research, and other research done in healthcare does not help decision makers are because they were never involved in the beginning. Decision makers do not regularly weigh in on determining whether a study was designed to answer an important question. So a big part of making sure that there is an impact in short-term practice is to ensure that the consumers of the information are actually involved in designing the research in the first place.

Henry: Are there examples that perhaps would serve as an illustration of the way this ought to be done?

Tunis: The prototype for CED done by Medicare was the National Emphysema Treatment Trial (January 1998 through July 2002) that investigated the efficacy of lung volume reduction surgery for patients with severe emphysema. It was a collaboration between Medicare and the National Heart, Lung, and Blood Institute. It had been determined that this surgical procedure was potentially very important for emphysema —a common cause of morbidity and mortality. The surgical procedures were done in the context of the trial; the results showed that there was little or no benefit with this surgery for most patients, so this surgery is no longer performed today. It was a good example of a procedure that may well have been abandoned eventually, over a long period of experience, but that would probably have also been a long period during which many patients were undergoing a surgery that could have killed many of them.

Another example is the bone marrow transplant for patients with breast cancer. Again, thanks to a collaboration between the National Institutes of Health and the Blue Cross Blue Shield Association of Insurers, a large study was done for that procedure, which was becoming widely popular, because it was very profitable for cancer centers to be doing bone marrow transplants for breast cancer patients. And that too was killing patients, which was only discovered after dozens of states put in legislative mandates that it must be covered by all payers.

If we could potentially create a policy framework that allows for CED as in the case of bone marrow transplant for breast cancer patients, and it becomes a well-defined policy pathway, we could potentially get the legislators out of the business of mandating benefits that have not yet been adequately studied, because payers would have a policy tool that was appropriate for that circumstance. But nobody has tried to create a generalized policy framework for private payers to do this in an organized way.

Henry: Could you briefly discuss how this relates to evidence-based medicine, including randomized versus observational trials and meta-analyses?

Tunis: We are interested in empirical data that provide information about the risks, benefits, and costs of technologies and services. Our scope of interest is mostly in prospective studies, particularly because we are interested in new and emerging clinical services and technologies for which there would not be a lot of data in existing databases that could be analyzed retrospectively. So our focus tends to be on prospective observational or experimental studies, generally comparing an emerging technology or service to an existing alternative.

Henry: Could this be done from administrative databases, such as insurance claims databases orMedicare data?

Tunis: Yes, the data could be collected from electronic medical records or it could be proactively collected from clinical trials, registries, and other types of prospective study mechanisms, depending on what are the appropriate data source and methodology to answer a particular question. It is our view that we need to have the conversations about “what is the appropriate method” before the study is done, because once the study is done, it is too late to go back and do it differently if the decision makers decide they need different information.

For a particular surgical procedure or a disease management program, we want to have the conversations up front about what kind of study will be sufficiently robust for us to be able to make decisions about the procedure or program. If it is a randomized controlled trial (RCT), we have to stop wasting our time collecting observational data, and do an RCT. There are some things for which an RCT is the only sufficiently robust methodology, but that is not true for all questions.

But what happens too often (and I have participated in this many times at Medicare) is that the study has been done, it has been published, and then the argument ensues between the decision makers and the providers of the service about whether that evidence is sufficient. For the most part, our view is that we need a forum within which we can move that discussion up front, before we complete the protocol design.

Henry: Are you achieving success in getting that kind of consensus?

Tunis: We are achieving success in bringing people together to have that discussion. I would not say that we have been particularly successful in reaching consensus, but it is a step forward to recognize how difficult it will be.

A great example, again, is coronary CT angiography. Within our multistakeholder work group, we had all the vendors (GE, Philips, Siemens, Toshiba); the American College of Cardiology and the American College of Radiology; and we had multiple payers. We also had clinical researchers, patients, and a technology for which there are important unanswered questions about how this affects patient health.We asked the following question: What kind of a study do we need to do? And there was a huge range of opinion—some thought we needed to do a large multirandomized trial to measure the impact of this diagnostic on death and acute myocardial infarction; at the other end of the spectrum, some said we already know enough about this technology that we could say that it was the most important clinical advancement in the past decade and it should just be reimbursed. That we did not have any common ground on that question has important implications for policy and for clinical care. But it is important to have a forum within which that discussion can continue, because these are the sorts of issues that will have to be addressed for the next technology.Without this type of dialogue, how would anybody know what kind of studies they need to do?

Henry: What’s the impact of the FDA guidance documents?

Tunis: The FDA guidance documents articulate the FDA’s perspective on what kind of scientific evidence it is going to need to approve products in particular categories—asthma drugs, cardiac defibrillators, and others—and there are hundreds of these documents. They are useful for product developers who can look at an FDA guidance document and have a how-to manual about designing their clinical studies in ways that are most likely to obtain regulatory approval. They know what the end points should be, what the patient inclusion criteria should be, and how long to follow patients.

And those guidance documents are developed through the iterative notice-and-comment process, so they reflect not only the FDA’s perspective but also the perspectives of all the necessary experts and stakeholders. CMTP has started to develop what we are calling “effectiveness guidance documents,” but CMTP’s point is to articulate the evidential requirements from the perspective of decision makers such as payers, providers, and patients, so that product developers and technology developers will be able to know what kind of evidence is going to be required for this to be accepted in the reimbursed marketplace and by patients.

We are going to replicate the FDA notice-and-comment process, so we will develop draft guidance documents and post them on our website (www.cmtpnet.org) for anybody to comment. For example, of the first 2 we are starting with, one is on wound healing, and there are zillions of companies that are developing wound-care products, devices, drugs, and others. You cannot go anywhere to find out what is it that the payers consider to be sufficient evidence of effectiveness for a wound-care product. What is the appropriate primary outcome for a study? How long do the patients need to be followed? What sorts of patients should be enrolled in the study? Which patients is it okay to exclude? All these are answerable questions, but we do not want to hear solely from payers, because they do not know enough about wound care to develop a guidance document independently. So we bring them together with clinicians, research methodologists, patients, and industry representatives to design an appropriate framework for a good research study to answer these questions.

Henry: Who should be included in the creation of these documents?

Tunis: We need to include the product developers, the clinical researchers, and the patients, so that we have a coherent guidance on what is sufficient evidence, not for purposes of regulatory approval (which is a different policy decision), but for purposes of clinical use and reimbursement.

Our second example is gene-expression profiling (GEP) for breast cancer. This type of genetic testing can determine who does and does not need chemotherapy for breast cancer. That guidance document will have broad applications to many types of gene-expression tests, but we are writing it to be specific to breast cancer. We have contracted with researchers at Johns Hopkins, who recently did an evidence report on GEP for breast cancer. They reviewed the current literature on GEP for breast cancer, so they know what has been done well or poorly in clinical studies. We have asked them to complete a draft as if we were going to be providing guidance to a product developer who wants to develop a test in this area, recommending what kind of studies it should plan on doing that would be persuasive to payers, providers, and patients.

Henry: Are there any tools that could help those making the benefit design decisions?

Tunis: Everybody wants to move toward evidencebased decision-making, but our decisions are only as good as our evidence. And so our mantra, if you will, is that the decision makers have to be willing to get involved in the “creation” part of the evidence, not just the “use” part. Or as I like to say, in the world of evidence-based decision-making, whoever controls the evidence controls the decisions. CMTP is trying to provide a forum for decision makers to be actively engaged in that process.

We need a more clinically and scientifically sophisticated approach to respond to the fact that technology and new clinical services are driving increased spending. The kinds of mechanisms and approaches we are proposing are effective at both promoting the rapid adoption and use of high-value technologies and services and restricting the dissemination of new technologies that do not have much value.We are trying to sort out where there is value and make sure that the payers, the providers, and the patients have the information they need to rapidly adopt the technologies that really contribute to better health outcomes.

Disclosure Statement
CMTP receives funding from Aetna, Amgen, Blue Shield of California Foundation, California Healthcare Foundation, The Commonwealth Fund, Johnson & Johnson, Kaiser, National Pharmaceutical Council, Pfizer, and United Healthcare Foundation.

Dr Tunis is Director of the Center for Medical Technology Policy, and former Director of the Office of Clinical Standards and Quality and Chief Medical Officer, Centers for Medicare & Medicaid Services. He can be reached at This email address is being protected from spambots. You need JavaScript enabled to view it..