JNCI Monographs

JNCI Monographs 1998 1998(24):1-28;
© 1998 by Oxford University Press

This Article Extract FREE Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to citation manager Request Permissions Google Scholar Search for Related Content PubMed PubMed Citation Social Bookmarking          What's this?

Journal of the National Cancer Institute Monographs, No. 24, 1-28, 1998
© 1998 Oxford University Press

Integrating Economic Analysis Into Cancer Clinical Trials: the National Cancer Institute-American Society of Clinical Oncology Economics Workbook

	Introduction

Top Introduction Part I: Economic Analysis... Part II: Planning an... Part III: Implementing an... Part IV: Designing an... Part V: Statistical Issues Part VI: Economic Evaluation... Conclusion References

 
Clinical economics is a new and evolving discipline that addresses the economic implications of changes in medical practice. As applied to cancer care, clinical economics assesses the costs and effectiveness of new cancer interventions and can be a valuable endpoint in selected clinical trials. Through the integration of economics into clinical evaluations, information can be developed that contributes to the decisions of patients, clinicians, health care managers, and policymakers as to the most effective allocation of cancer care resources.

To begin a formal effort to promote the development of economic analyses in National Cancer Institute (NCI) clinical trials, NCI sponsored a conference in 1994 with cancer center and cooperative group representatives to initiate discussions on the importance, appropriateness, and complexity of such evaluations. In 1995, the American Society of Clinical Oncology (ASCO) established its Health Economics Working Group with a charge to develop specific guidelines for implementing economic evaluations in cancer clinical trials. As a follow-up to these initiatives, in 1996, NCI and ASCO convened a workshop to consider the practical implementation of economic evaluations in cancer clinical trials. The participants in this small workshop included experts from cooperative groups, NCI staff, and other experts who are actively involved in health economics.

This workbook is the product of the meeting and subsequent discussions by the participants. It is meant to identify and elucidate the important characteristics of economic studies in the context of clinical trials, to indicate the considerations that investigators should address in their planning and implementation of such studies, and to suggest possible approaches. The workbook is neither a definitive text defining how all aspects of such studies should be handled nor an official NCI document prescribing how studies must be done. Rather, it is a developing guide to be used as a practical reference that will be revised as the state of the art progresses. The writing committee hopes that the workbook will serve as a useful tool for NCI cooperative groups as they incorporate economics as a research endpoint into the evaluation of new cancer treatment, prevention, and diagnosis strategies.

	Part I: Economic Analysis and Cancer Clinical Trials

Top Introduction Part I: Economic Analysis... Part II: Planning an... Part III: Implementing an... Part IV: Designing an... Part V: Statistical Issues Part VI: Economic Evaluation... Conclusion References

 
This section briefly presents the framework of economic theory underlying cost-effectiveness analysis and its application to the field of oncology. It is not, however, an in-depth review of the theory of economic evaluation, because many comprehensive sources currently exist (1,2).

Why Economic Analysis?

The Rising Costs of Cancer Care The percentage of total deaths in the United States attributable to cancer has risen from 16.3% in 1965 to 23.3% in 1997 (3). From 1990 through 1996, the estimated costs of cancer treatment increased from $35 billion (4) to $50 billion due to higher inflation, increasing numbers of procedures and cases, and the aging of the population (5). Even conservative estimates, measuring only the direct costs of treatment, show cost increases from $18.1 billion in 1985 to $27.5 billion in 1990 to $41.4 billion in 1994 (6). Cancer will become the foremost cause of death in the United States by the end of the decade.

Statistics and predictions such as these underscore the likelihood that cancer will continue to absorb more of the increasingly limited resources of the U.S. health care system. Some researchers have suggested that increasing costs and demands from a sophisticated patient audience will require both implicit and explicit rationing (7,8). In any event, economic forces will contribute to a growing need to better evaluate treatment practices by all clinicians, thus insuring that we utilize the relatively scarce resources of the health care system in an appropriate manner.

Cancer therapies are increasingly resource intensive, as evidenced by stem cell transplantation for hematologic disorders and solid tumors, paclitaxel for palliative chemotherapy of breast and ovarian cancers as well as non-small-cell lung cancer (9), serotonin-antagonist antiemetics, and growth factors for supportive care during treatment. In an era of capitated physician and hospital payments, the resources available for cancer treatment will be increasingly constrained, and payers and purchasers will want to understand the value of cancer treatments, especially for resource-intensive therapies, before allowing widespread access to them. Furthermore, when health outcomes are identical for alternative therapies, the costs of these treatments may be the most important factor in determining whether to recommend or reimburse one of the treatments.

Encouraging examples of less intensive strategies have the potential to improve efficiency of cancer treatment, such as evidence-based, minimalist follow-up care for breast cancer patients (10), decreased use of tumor markers in breast and colorectal cancers (11), and conservative use of hematopoietic growth factors (12), as well as a shift to less expensive outpatient treatments for stem cell transplantation (13). However, these strategies remain a minority of cases of new treatments in oncology.

The Need to Improve Decision-Making Because few medical interventions that provide health benefits also result in reduced health care expenditures (Table 1), reliable economic analyses must be developed to improve our decision-making about the allocation of resources for cancer treatments (9,14,15). The rapid advancement of science and biotechnology, coupled with increasing fiscal pressures, may severely limit access to cancer treatments for which economic data are not available. It is worth remembering that medical ethicists have argued that the least ethical way to allocate resources is to continue to spend money on unexamined treatments (16,17).

View this table: [in this window] [in a new window]   Table 1. Efficacy and cost-effectiveness of selected cancer and other treatments*

 
As shown in Table 1, most treatments provide additional clinical benefits at an additional cost. Thus, economic efficiency does not imply a reduction in cost of care but provides a measure of the value provided for the cost of the treatment (18).

Economic information can strengthen arguments for adoption of new technologies that are economically attractive, provide additional input for decision-making in indifferent or gray areas, and provide clear-cut answers when costs are a major consideration (19). Most existing cancer treatments have not been evaluated with the stringency applied to newer treatments. For example, we have no "proof" that intensive treatment of metastatic breast cancer, compared with best supportive care, improves quality or quantity of life or saves money, because no randomized clinical trial has been performed in the past 20 years. This is not to say that intensive treatments are ineffective or exceed reasonable cost-effectiveness limits—only that they have not been studied with the same scrutiny as high-dose chemotherapy with stem cell transplantation (20).

Measurement of incremental benefit (additional lives saved by a strategy) and incremental cost (additional costs of a strategy) forms the backbone of comparative economic analysis. The four possible approaches of such analysis are listed in Table 2 and represent the combinations of costs and outcomes measurement. Estimates of absolute incremental differences in cost between trial arms are needed, just as they are for analyses of treatment benefits. For example, the primary outcome measure of a cost-effectiveness analysis is the additional cost incurred per year of life gained or quality-adjusted years of life gained. Such measurement requires explicit delineation of clinical benefit in terms of years of life saved or quality-adjusted years of life saved and of dollars spent on treatment, recurrence, survival, and death. The statistical implications of absolute and relative differences are discussed in Part V.

View this table: [in this window] [in a new window]   Table 2. Potential combinations of costs and outcomes

 
Perspectives on the Use of Resources Cost data are meaningful only when considered from a specific perspective, usually that of patient, caregiver, provider, payer, or society (i.e., the choice of perspective determines what events or expenditures are regarded as costs, as well as how those costs should be calculated). The societal perspective is often the most appropriate one for resource-allocation decisions (1), because its broad approach can minimize bias in comparisons; however, other perspectives may be important in attempts to understand the economic effects of a treatment.

As in clinical medicine, investigators must not make assumptions about how patients value treatment outcomes. For example, a recent clinical trial demonstrated substantial intellectual deficits in glioblastoma patients undergoing radiotherapy; but, because the treatment offered a few weeks of improved survival, most patients reported that they would choose the radiotherapy (21,22).

Multiple perspectives may be adopted within a single analysis to illustrate the views of patients, payers, policymakers, and health care providers. For example, the cost of providing a hospital service is usually different than the charge for the service and the amount paid by the insurance company. An episode of care may cost a hospital $7000, but the hospital charges $10 000 for the service. From the hospital's perspective, the charge is an overstatement of the resources consumed. If the patient must pay the full charge, however, that charge accurately represents cost from the patient's perspective. Alternatively, while a hospital can sometimes decrease its costs by discharging patients early, patients' costs may increase due to increased outpatient expenses that are not covered by health insurance.

When Are Economic Evaluations Justified?

Inclusion of economic evaluations in clinical trials can be justified in a variety of circumstances: when significant resources are at stake; when resource considerations are prominent; and when resource allocation decisions are imminent (23,24). Examples of each circumstance are provided below.

Significant Resources at Stake Antiemetics. Although individual doses of 5HT3-agonists, such as odansetron and granisetron, are not expensive, they carry a high potential volume of use and can account for 5% of a hospital's oncology pharmacy budget (data on file, Medical College of Virginia Hospitals).

Colony-stimulating factors (CSFs). The initial and additional costs of CSFs (granulocyte, granulocyte-macrophage, and erythropoietin) can consume up to 5% of a hospital's oncology pharmacy budget (data on file, Medical College of Virginia Hospitals).

Transplant procedures. High-dose chemotherapy with stem cell transplantation and similar experimental therapies carry a high dollar cost, regardless of volume. The cost of treatment and follow-up for leukemia and lymphoma patients receiving autologous bone marrow transplantations and granulocyte-macrophage CSF reached $79 892 (25).

Genetic predictors. BRCA1 and BRCA2 tests, genetic predictors for breast and ovarian cancer, can cost anywhere from $150 to $1500 per mutation, depending on whether a family's specific mutation has been identified. Lerman et al. (26) note that the high cost of such testing may deter some individuals from undergoing the test, and individuals may hesitate to request coverage from insurers for fear of future discrimination.

Resource Considerations Are Prominent Many therapies carry a high cost, regardless of their clinical outcome, such as high-dose chemotherapy with stem-cell transplantation for metastatic breast cancer and allogeneic bone marrow or stem cell transplantation from an unrelated matched donor (estimates allow $100 000-$250 000 for each uncomplicated case).

Other therapies vary widely in costs but produce similar clinical outcomes. For example, high-dose chemotherapy is commonly used as treatment for first-remission lymphoma, although less expensive standard-dose chemotherapy is known to produce similar or identical outcomes (27).

Resource Allocation Decisions Are Imminent Megakaryocyte growth factors have been proposed to eliminate the need for platelet transfusions and to allow for earlier hospital discharge after high-dose chemotherapy (25). Also, results of a trial of high-dose chemotherapy as adjuvant therapy for high-risk breast cancer (>10 positive lymph nodes) quickly changed the standard therapeutic approach to this disease (28).

When Not to Use Economic Analysis

Economic analysis is unnecessary when a primary therapy works well in a small number of patients—for example, testicular cancer treated with platinum (600 cases/year) (29). Furthermore, some therapies vary little in costs and produce similar clinical outcomes in very common diseases. For example, a number of chemotherapeutic combinations are used as adjuvant therapy for node-positive breast cancer, but they cost about the same and produce similar effects (30).

Also, economic analysis is unnecessary when its results are unlikely to alter clinical practice. For example, an economic analysis of breast self-examination might show that costs are higher in the screened population and fail to demonstrate clinical benefit. But the use of breast self-examination is so widespread that an economic analysis would not change the practice.

Generalizability of Observed Costs of Cancer Care

Cost reflects the resources needed to provide a service and it has two elements: "resource utilization" in some natural units (e.g., one hospital day or one dose of ceftazadime) and the "unit cost" or price (e.g., 1 hospital day in a semiprivate room for $460). Units and prices are multiplied to determine treatment costs. One might calculate physician costs by multiplying physician time by hourly physician earnings. The same principle applies to productivity costs: the cost of patient time can be calculated by multiplying time by a patient's daily wage rate.

Some administrative datasets provide reliable information only on resource utilization, whereas others provide reliable information only on total cost, and still others provide information on both. It is important to recognize that the resource utilization and price components of cost have different determinants. For example, resource utilization may be affected by treatment setting, organizational and technical innovation, and efficient use of fixed resources. Price may be affected by the degree of competition in health care markets, including geographic location, regulatory and legal restraints, and the bargaining power of health care payer organizations. The "learning curve" of better efficiency and lower cost that develops as procedures are performed more often may decrease some costs; alternatively, unforeseen consequences may increase expected costs. It is important to keep these issues in mind, because economic data obtained from one clinical trial may not always be generalizable to another time, setting, or pricing scheme.

Costs related to a disease entity have traditionally been classified into a number of categories, depending on whether the costs are directly related to the expenditure of medical or nonmedical resources related to treatment and care; whether the costs are a reflection of lost economicproductivity due to disease-related disability or premature mortality; or whether the costs are related to the pain and suffering caused by the disease (6). The four categories of costs are listed below:

Direct Medical Costs

• Resource use related to services and procedures delivered by the health care system in the course of treatment and care (hospital and physician costs); patient payment for medical supplies.
  

Direct Nonmedical Costs

• Resources expended by the patient and her family related to treatment and care (travel to treatment facility, parking, etc.).
  

Productivity Costs

• Not a financial transaction, but the monetary value of lost economic production due to morbidity or mortality, often utilized in cost-effectiveness or cost-utility analyses; may be important from a societal perspective or from the perspective of an employer—for example, in the analysis of treatments for substance abuse or treatments that effect the ability of cancer patients to return to work.
  

Intangible Costs

• Also not a financial transaction, but the monetary expression of pain and suffering associated with disease or treatment; various methods for measuring intangible costs, such as "willingness to pay," indicate that such "costs" can be quantified as a monetary value (i.e., the amount a patient would be willing to pay to avoid pain or suffering); for purposes of cost-benefit analysis, intangible costs are typically expressed in monetary terms, because the outcome of cost-benefit analysis is expressed in purely monetary terms; for cost-effectiveness analysis, intangible costs often are ignored, and for purposes of cost-utility analysis, intangible costs are expressed as part of the denominator in units of quality-adjusted life-years (QALYs)
  

While all of the categories of costs are included in cost-of-illness studies and cost-benefit analyses, only direct costs are usually included in cost-effectiveness (or cost-utility) analyses, which have become the most common approach in economic analysis of health care interventions (Table 3).

View this table: [in this window] [in a new window]   Table 3. Outcome and cost studies for presentation to decision makers

 
The Need to Make Decisions

The need for either implicit or explicit decision-making about resource allocation dictates that decision makers should know the total budget of a program, the efficiency or cost-effectiveness of the given modality, and the overall effect of each treatment. It is essential, therefore, that data be presented in such a way that decision makers can assess both the costs and potential clinical consequences of new cancer treatments (Table 3). Although the mechanics of such decision-making vary greatly from centralized planning to individual choice, only limited economic information is currently available to make these important treatment decisions.

Criteria for Evaluating the Soundness of an Economic Analysis

Economic analyses must be evaluated using defined criteria with clear goals in mind, just as clinical trials themselves are evaluated against a variety of accepted criteria. A full economic analysis of any cancer intervention should go beyond a simple identification of costs (Table 4). As the competition for scarce health care dollars intensifies, more therapies will be marketed as "cost-effective." Failure to explain these criteria should make the investigator wary about the quality of the economic analysis (31).

View this table: [in this window] [in a new window]   Table 4. Criteria for identifying a sound economic analysis*

 
Conclusion

In this section, we review the rationale for economic evaluation of new cancer treatments, including a discussion of why economic analysis is increasingly being implemented in cancer clinical trials, a discussion of when economic evaluations are justified, and an overview of the economic framework for evaluation of cancer clinical therapies.

Economic evaluation is an increasingly important component of clinical decision-making in the context of scarce or limited resources. Economic evaluation itself can contribute to decision-making in specific clinical settings as outlined in this section. Furthermore, economic evaluation goes beyond assessing the monetary cost of medical care to include such issues as productivity costs (loss to the economy from morbidity and mortality) and intangible costs (costs of pain and suffering related to illness or injury).

	Part II: Planning an Economic Analysis

Top Introduction Part I: Economic Analysis... Part II: Planning an... Part III: Implementing an... Part IV: Designing an... Part V: Statistical Issues Part VI: Economic Evaluation... Conclusion References

 
The previous section introduced some of the concepts of economic evaluation of new cancer therapies by reviewing the conceptual framework underlying economic evaluation of cancer clinical trials. Parts III through V will review these concepts in more detail and lay out a framework for the collection of economic data in clinical trials. The purpose of this section is to illuminate issues in the planning of economic analyses that may be important to clinical investigators designing protocols with an economic component.

Selection of Appropriate Trials

Economic evaluation can be an important secondary endpoint in clinical trials of new drugs and technologies, especially new cancer therapies. Implementing an economic evaluation, however, requires additional time and effort, including additional data collection, management, and analysis. In deciding whether to make this investment, investigators must consider whether economic analysis is likely to aid in decision-making regarding reimbursement for new therapies or technologies.

In general, investigators have suggested that economic evaluation is most appropriate in pivotal trials of new technologies or strategies when the investigational therapy is resource intensive or is likely to be utilized by a significant number of patients. Economic evaluation is less important in trials of "me-too" therapies, which may be similar in efficacy and cost to other clinical therapies. Here, however, the cost of therapy may be the important distinction between two treatment arms. Clinical investigators should develop economic hypotheses before approaching economic investigators to discuss the inclusion of economic analysis as an endpoint in the clinical trial.

Economic evaluations usually are proposed for inclusion in phase III clinical trials but can be included in phase II trials to allow for collection of pilot data that will be used in plans of pivotal phase III assessments. Phase IV studies are also appropriate for consideration for economic evaluation.

Some estimate of the potential difference in cost between treatment arms, and the resulting cost-effectiveness ratio if the targeted difference in survival is achieved, should be included in the concept sheet and protocol for any study with an economic component. Ideally, this estimate will be based on data collected in a pilot study, but a "back-of-the-envelope" calculation should be performed when pilot data are not available. Such estimates play a critical role in justifying the investment in an economic component and are needed for the planning of an efficient data collection strategy.

Timing of Data Collection

Data collection for economic studies can be based on prospective data that have been collected concurrently with the clinical trial or on retrospective data that have been collected after the trial's completion. The advantages and disadvantages of each approach are discussed in more detail in Part V.

Prospective economic evaluation assures that economic data will be available at the time of a trial's completion. It allows for collection of the four types of economic data (direct medical, direct nonmedical, productivity, and intangible), as well as quality-of-life data from study patients. But prospective collection of economic data requires substantial planning and is, in this sense, no different than prospective collection of clinical data. Because economic evaluation is usually a secondary endpoint of clinical trials, and because there is a risk that the null hypothesis will be adopted at the end of the trial, investigators should carefully consider the potential use of resources for an economic evaluation in a trial of an unknown or untested therapy.

Retrospective economic evaluation offers a more limited approach to assessment of resource use within clinical protocols, typically focusing on direct medical costs, because other types of data often are not available in the medical record. In addition, retrospective approaches can not assess patients' preferences or quality of life. However, in this approach, the expense of collecting economic data will only be required for therapies with proven efficacy.

The choice between prospective and retrospective data collection usually rests on the potential clinical effect of the new therapy and on the amount of resources available to investigators. If the trial is indeed an appropriate candidate for economic evaluation, and if resources are available for proper planning and design of an economic study, a prospective approach should be taken, because it will allow for a more thorough evaluation of the costs of a new therapy or technology.

Choice of Perspective

Economic evaluation of medical care is unique in that it assesses the costs and benefits of therapies from a variety of perspectives, including those of patients, caregivers, employers, payers, and society. However, clinical and economic investigators must determine which perspectives are most relevant to the clinical trial under consideration. For example, economic evaluation from a hospital's perspective requires consideration of inpatient costs but not outpatient or out-of-pocket costs. Data collection from a patient's perspective is more complex and may require primary data collection to yield more useful information on out-of-pocket costs and lost work days. Economic evaluation from a societal perspective is the most comprehensive of all and includes consideration of both patient and provider perspectives.

Unless a therapy is particularly burdensome to patients, the most common perspective for clinical trials is a "modified" societal perspective, which considers only direct medical costs. In this type of analysis, direct medical costs are quantified as social costs, but direct nonmedical and productivity costs are not routinely collected.

Time Horizon

Time horizon is defined as the time from randomization to follow-up for patients in clinical trials. Time horizon can affect the economic evaluation of new therapies by limiting the amount of information available to the assessment of the long-term effectiveness of treatment. Economic evaluation frequently requires a different time horizon than that proposed by clinical investigators.

Ideally, the time horizon of an economic evaluation should be such that all costs related to the therapies being compared are captured for analysis. This concept differs from the idea of time horizon in clinical trials, which can be based on time until a measurable clinical endpoint is achieved or until the end of some predetermined time period (e.g., 5 years). Under the ideal economic scenario, subjects would be randomized to a trial and followed until death or cure (32). Unfortunately, most clinical trials are conducted for a limited length of time.

In general, the time horizon for an economic evaluation of a clinical trial begins at the date of trial randomization. Alternative start dates for the time horizon include time of diagnosis or time of the onset of symptoms. Economic evaluations that begin before the time of randomization usually are attempting to assess important costs incurred during the prerandomization time period (e.g., a trial of bone marrow transplantation for patients with breast cancer may need to provide induction chemotherapy to 10 patients to find one responder for the transplant portion of the study; an investigator may be interested in the costs of screening as well as the costs of the prerandomization hospitalization) (33).

In determining the appropriate length of the time horizon for economic evaluation, it is helpful to distinguish between two types of clinical trials—trials of therapies for metastatic cancer (mortality studies) and trials of adjuvant therapies. Clinical trials in these two settings pose different issues with respect to clinical and economic endpoints. Knowledge of the survival curves for the type of cancer being studied, along with projections of the expected benefit of the therapy, can be used to determine how long patients should be followed for an economic study. In mortality studies, the follow-up time is likely to be relatively short, so resource utilization can be assessed for all patients until time of death for a high proportion of study patients. In clinical trials of adjuvant therapies, some proportion of patients may be cured and thus live a normal life expectancy. Generally, patients can be followed for recurrence. In adjuvant therapy trials, one can consider three potential time frames—short-term (e.g., 6 months, 1 year), intermediate, and lifetime.

Outside the Time Horizon When the time horizon of a clinical study differs from that of the economic study, it may be necessary to track all resource use in both arms of the trial over a period that is longer than the relevant clinical endpoint, because important resource use may occur in the period beyond the time horizon of the clinical trial.

Sequelae Substantial long-term sequelae that may occur in the relatively distant future, whether related to the cancer itself or to treatment of the cancer, should be included in the analysis.

Relating Costs and Benefits The time horizon should be the same for the numerator and denominator of a cost-effectiveness ratio. For example, if an analysis includes 2 years of clinical follow-up data, it should also include 2 years of cost data. In a trial of adjuvant therapy, simply including the cost of treatment in the numerator and a longer follow-up in the denominator does not accurately reflect the relationship between the costs and benefits of the therapy being assessed.

Endpoints

Economic evaluation usually is based on a consideration of final outcomes, such as survival rather than response, disease-free survival, and time to progression. Thus, economic evaluation requires either consideration of a final outcome measure as an endpoint of the clinical trial or an understanding of how clinical trial endpoints will translate into final endpoints (for example, epidemiologic data may be available for estimations of cure rates for patients who demonstrate a complete response to treatment). Specification of outcomes measures must be determined prospectively by both clinical and economic investigators before the trial proceeds.

Baseline Characteristics

Clinical studies, except for some "large and simple" trials and a small number of special cases, are generally underpowered for evaluating secondary cost or quality-of-life measurements. Analytic techniques that assess nontreatment-related variance can be used to address this issue. Thus, the ability to detect differences in costs due to treatment assignment will increase if information on baseline characteristics of study patients is included as a set of covariates in analyses of secondary study endpoints. Baseline characteristics include resource use in the period prior to the trial, clinical status, diagnostic profile, disease stage and severity, level of social support, and health status. Variation in covariates among patients stem from the type of patient rather than from treatment assignment—for example, whether a patient is an inpatient or outpatient at the time of randomization can predict resource utilization after randomization.

Baseline covariates allow investigators to "disentangle" patient characteristics prospectively. Covariates may be used as predictors in regression analysis at the end of a study, or they can be used to stratify the randomization of any parameter that may be a factor in predicting costs to a high degree. Many of these data elements can be collected as part of eligibility determination and tagged as covariates in the economic evaluation. It is also important to consider any and all clinical information obtained prior to randomization as potential covariates. Ideally, covariates will be continuous variables.

External Validity

Where possible, the clinical protocol should be modified to reduce protocol-mandated tests or procedures to reduce protocol-induced charges in medical care (protocol-induced costs or benefits) and to ensure that the protocol mirrors "usual care" as much as possible. Economic investigators should also review the clinical protocol to ensure that there are no economic biases in the structure of the trial (e.g., there are no fixed discharge criteria included in the study and/or there are no differences in prescribed treatment across study areas). In addition, whether or not the clinical protocol procedures are modified to reduce mandated tests, efforts should be made to identify medical procedures that are mandated only for the research portion of the study. If these research resources are not related to treatment, their costs can be excluded from the analysis.

Economic evaluation raises a unique set of issues about the generalizability of clinical trials themselves. These studies are designed to influence clinicians, patients, and decision makers in their perceptions of the importance of a new therapy. Thus, economic evaluation aims to ensure that patients being studied in the clinical trial are as similar as possible to relevant patient populations in the "real world." For appropriate economic evaluation, therefore, clinical trials should be free of economic bias, should include patient entry criteria broad enough to provide support for the external validity of the analysis, and should, to the greatest extent possible, follow strict and objective standards in patient selection.

Trial investigators can reduce economic bias in the design of clinical studies by avoiding prespecified indications for discharge, supportive care interventions, or second-line therapies. Investigators can further reduce bias by identifying and minimizing protocol-induced costs or benefits, thus rendering the phase III clinical trial as realistic as possible in terms of actual practice in the community. Protocol-induced costs can include repeated diagnostic tests or other evaluations that do not directly contribute to clinical care or are not routine elements of clinical practice. Because these tests and evaluations can lead to diagnoses that otherwise might be missed in the usual care of patients, they can also yield clinical outcomes that would not be replicated in usual practice.

Finally, economic analysis is concerned with the generalizability of data developed from clinical trials. The external validity of studies is thus an important consideration in recruitment of subjects to the study. One such example of data that suggest clinical trial results can be replicated in the community are presented in a study of breast cancer mortality in British Columbia by Olivotto et al., showing that after adoption of routine adjuvant chemotherapy, mortality declined with an effect size identical to that of clinical trials (34).

An alternative view is that patient selection will probably continue to be a strong factor in most trials. The task for researchers is to determine how well bias to external validity can be modeled by obtaining good data on the relevant characteristics of patients enrolled in trials compared to patients who are likely to receive the treatment shown to be efficacious in the trial.

Salvage Protocols

In oncology, the potential for salvage or late use of interventions is important to consider in advance for economic evaluation. As mentioned above, the economic period of a treatment may be in discordance with clinical trial measures. Late use of interventions may provide crucial information for economic evaluation of a study, because treatment for disease progression can be very costly.

Economic evaluation can be implemented in this setting by including patients in the clinical trial for a fixed time period, allowing patients to remain in the economic protocol throughout this period even if they reach an endpoint of the clinical trial and withdraw from active treatment. Salvage protocols with economic arms also provide an opportunity for further economic data collection from patients who have failed treatment.

Finally, several unique statistical issues play a role in economic evaluation. Patients may incur fairly low costs until their disease has progressed, after which time they usually are withdrawn from the clinical trial. Yet, it is often at this point that patients become expensive and, thus, important to the economic evaluation. Economic studies often are designed to allow patients the option of continuing with the economic protocol, even after withdrawing from active treatment on the clinical protocol. In this limited way, many economic evaluations have been able to successfully follow large portions of a trial population beyond the time horizon of the clinical trial (25).

Importance of Early Collaboration Among Investigators

To be successful, economic investigators must understand the clinical aspects of the trial being considered as well as the potential clinical implications of the therapy or technology being evaluated. Likewise, clinical investigators must be able to develop economic hypotheses and to understand all elements of the economic study to ensure appropriate implementation of the economic study's design. Thus, clinical-economic evaluation is a truly collaborative undertaking between clinical and economic investigators.

Economic investigators should become familiar with the economic profile of the intervention under study: What resources are required to implement the therapy? For how long will these resources be required? What is the potential benefit of therapy? What are the potential resource offsets or savings from the new therapy compared with existing therapies? In addition, economic investigators should ask these questions in the context of the current standard of care. They should consider the potential risks of the new therapy as well as how the clinical trial's results might change the standard of care.

Clinical investigators should understand that there is no "standard" economic analysis to be simply appended to a clinical trial. Economic analysis is an important secondary endpoint of a study and, to be performed well, must be integrated throughout the clinical trial mechanism. Seamless integration of the economic component of the study with the quality-of-life data collection strategy is particularly important in any study that employs patient diaries as instruments of economic data collection. Most important, clinical investigators must "buy in" to the economic evaluation so that they have a stake in seeing that economic data are treated with as much care as clinical data. Recipes for failure of an economic evaluation include lack of commitment by clinical investigators to the collection of economic data and a lack of understanding by clinical investigators of the purpose of the economic evaluation.

Pilot Testing

Economic evaluation can benefit from pilot testing in much the same way as clinical trials. Pilot testing offers investigators an opportunity to assess the use of case report forms (CRFs) and patient diaries in clinical trials so that they might become more familiar with the process of implementing a protocol and more knowledgeable about the economic profile of the disease under study. In addition, pilot tests can pinpoint a trial's "cost drivers" and help economic investigators characterize the distributions of cost data and predict the differences in cost between treatment arms or the total cost of care for study patients. Identifying cost drivers in a pilot study can reduce the data collection burden in the final study. Furthermore, pilot data can be helpful in identifying the toxicity profiles of a therapy and, in particular, the nature and frequency of high-cost events, such as hospitalizations for febrile neutropenia or cardiac toxicity. Similarly, pilot testing can provide information for estimations of sample size and expected cost differences between study arms (35).

Conclusion

This section reviews issues important for clinical investigators to consider in contemplating whether to implement an economic evaluation within a specific clinical trial. Issues for clinical investigators to consider include the selection of appropriate clinical trials for economic evaluation, timing of data collection, choice of perspective, time horizon, study endpoints, and external validity or generalizability of the study. Investigators must also consider that to design robust economic comparisons of clinical trials, economic evaluation requires input from economic investigators as early collaborators within the clinical team. To implement a successful protocol, clinical investigators must assist in the design of the economic evaluation and in the implementation of the economic component of the protocol.

	Part III: Implementing an Economic Analysis

Top Introduction Part I: Economic Analysis... Part II: Planning an... Part III: Implementing an... Part IV: Designing an... Part V: Statistical Issues Part VI: Economic Evaluation... Conclusion References

 
In the previous section, we reviewed some of the important concepts for a clinical investigator to consider when deciding whether to incorporate an economic evaluation into a clinical trial. In this chapter, we review in more detail many of the same issues from the perspective of someone who is designing an economic evaluation of a clinical trial. In the next chapter, we focus on the collection of data items themselves.

Study Outline

Economic evaluation measures the use of resources and often takes note of the same events of interest to clinical investigators, such as whether a patient receives a magnetic resonance imaging scan (MRI). Yet, while clinical investigators are most interested in the results of the MRI, economic investigators take greater interest in the number of MRIs consumed during the study period. In addition, economic investigators attempt to interpret changes in resource quantities. For example, a specific therapy may reduce length of stay for patients receiving autologous bone marrow transplant (ABMT) yet increase the need for outpatient follow-up visits. Costs are applied to these resource-utilization measures to demonstrate overall economic benefit or harm. For example, if inpatient days cost $700 each and physician visits cost $100 each, the ABMT scenario described above will save money if it reduces length of stay by 1 day and increases outpatient resource use by no more than the equivalent of six physician visits.

Thus, economic evaluation requires both resource utilization and unit cost information to provide an overall estimate of the resources consumed in the course of caring for an individual patient. The following sections focus specifically on the development of resource-use data and unit costs.

Resource-Use Categories

Oncology patients consume a variety of resources in the course of medical treatment, including hospital services, physician and nursing services, diagnostic tests, pharmaceutical products, radiation therapy, and ancillary services. Given the tremendous complexity of oncology care, economic investigators must make explicit decisions about the types and amounts of information they wish to incorporate into their protocols. In general, these data elements should include resources that carry significant costs (e.g., inpatient stays) and services that are likely to differ across treatment groups. The most important of these—known as "cost drivers"—represent resources that are both costly and differ across treatment groups (36) and they should be validated through pilot data collection efforts. Cost drivers are not simply drivers of a study's total cost but of the increments in costs between the study's treatment arms. Investigators should, therefore, assess the impact of a particular resource on the mean costs, the variance in costs, and the difference in costs between treatment arms before eliminating it from the list of data elements to be collected.

Computerized cost-accounting systems, which are increasingly being adopted by health care provider organizations, make it possible to retrieve information on resource use and costs for a variety of resource-use categories. For example, the Decision Support System used by Group Health Cooperative of Puget Sound contains information on resource use and costs for such categories as inpatient days, surgeries, outpatient/short-stay hospitalizations, primary care visits, specialty care visits, mental health visits, emergency room visits, community health services, diagnostic radiology, laboratory services, occupational/physical therapy, respiratory therapy, and pharmacy (37).

Direct Medical Costs: Inpatient and Outpatient Resource Utilization

Hospitalizations Hospitalizations typically represent the most intense period of resource utilization by clinical trial patients. Resources received during a hospitalization include hospital "hotel services" themselves, physician services, nursing services, laboratory and pathology services, blood work, operating rooms, and pharmaceutical products.

Hospital services are provided based on the type of unit to which the patient is assigned, e.g., intensive care, high care, telemetry, general, and rehabilitation. Thus, descriptions of resource utilization for hospital "hotel services" could be a description of length of stay or a description of length of stay by unit type (e.g., a 10-day admission of which 4 days were spent in the oncology unit and 6 were spent on the general medical floor). One distinguishing feature of the different unit types within a hospital is the level of nursing support services available to patients. Thus, location within the hospital is often also used as a description of the intensity of nursing care.

Physician Services Physician services in the hospital setting include those provided by the attending physician (either medical or surgical) and by consulting physicians. In the outpatient setting, they can include oncology services as well as services provided by primary care and consulting physicians. In an economic investigation, measures of physician effort can be characterized in terms of time (a 30-minute consultation) or workload units (resource-based relative value units [RVUs]) (38,39). These data can be collected for all physicians treating patients in the protocol or by the type of physician providing the service.

Surgical Procedures Surgical procedures are an important component of physician services. Performed for diagnostic and therapeutic purposes, surgical procedures can occur in and out of the hospital setting. The most global measure of surgical services is a description of the procedure itself (e.g., laparoscopy, colectomy). This description can be text based, or it can be coded using a standardized system such as the International Classification of Diseases (ICD) or the Common Procedural Terminology (CPT). Surgical procedures can be further described using a second level of detail—for example, the time in the operating room or the use of specific supplies, including blood products or isotopes. A third measure, which is especially important for outpatient surgical services, is a description of the location where the service is performed (e.g., hospital, outpatient surgical center, free-standing surgical center, physician's office). These levels of detail can be used to describe resource utilization by different treatment modalities.

Diagnostic Tests Some of the most important services provided to patients both in and out of the hospital are diagnostic tests. As mentioned above, economic investigators are interested in recording the number and types of services provided to patients. These resources can be captured in a variety of ways, including listing the number of radiologic or laboratory tests as either an aggregate measure or an itemized listing of the occurrence of specific types of tests and procedures.

Pharmaceutical Services Pharmaceutical utilization during hospitalization periods may already be captured within clinical CRFs' concomitant medication sections. If not, investigators may wish to detail the amount of medication received by patients during a hospital stay. Medication use can be characterized by medication type and dose (total daily dose) or by total dose received during a specific time period (i.e., daily dose, total dose during a course of chemotherapy, total dose during a hospitalization). In addition, economic investigators should record the route of administration, because intravenous administration may be more costly than oral administration. Data collection can include all concomitant medications or a subset of specific medications of interest to the investigators.

Radiation Oncology Physician services for radiation oncology treatment can be captured as described in the Physician Services section above. Use of radiation treatment facilities can be tracked by recording the types of services received by patients, the personnel effort required to provide such services, the duration of treatment, the dose of isotope delivered, and the setting where the treatment occurred. In radiation oncology protocols, more detailed treatment or service data may be collected.

Ancillary Services Many other services are available to patients in and out of the hospital. These services can be especially important to the care of oncology patients, although they are not delivered by physicians. Social work, speech therapy, pain management, occupational therapy, physical therapy, and clinical pharmacology services, to name a few, are usually measured by tracking the occurrence of specific consultations or by measuring the amount of time the provider spent in consultation with study patients.

Subacute-Care Facilities Cancer patients may utilize a variety of subacute-care facilities, including nursing homes, rehospitalization centers, and inpatient hospices. These resources may be quantified by the type of institution and the length of stay in the institution. For more specific studies, data can be collected on specific services received during the stay.

Direct Nonmedical Costs

Direct nonmedical costs include the costs of transportation to and from physician visits or hospitalizations and other costs related to receiving medical care. For extended stays at a treatment center, these costs can include the cost of hotel services or of moving families to a treatment center. Investigators should determine the specific resource quantities of interest to be captured as a direct nonmedical cost. These can include the number of hours of parking at specific treatment centers or the number of days spent at a hotel adjacent to a treatment center.

Measuring Resource Utilization

Data on resource measures can be collected from medical records and medical bills and through patient self-report. In this section we review the data sources available to economic investigators.

Medical Records Medical records include source documents, such as medical charts and abstracted flow sheets. These documents may contain data on all elements of resource utilization during a specific time period, such as the detailed information found on a hospital chart, which can be abstracted by data collectors to provide information for each of the resource categories reported in the previous section. Interpretation may be required for some data elements, such as unit type by day of hospitalization. Medical records are limited in that they often are not linked between inpatient and outpatient treatment facilities. Patients typically have multiple providers and, thus, multiple records, making it difficult to follow patients from one site to another.

Administrative Data Sources Many types of administrative data are available to economic investigators, including hospital and physician billing records and datasets from large managed care organizations (MCOs) or the Medicare program. Administrative datasets include information on resource quantities and can be used as a direct measure of resource utilization or abstracted to track utilization of specific services. As with medical records, certain administrative datasets may be available only for individual episodes of care (hospital datasets) and may not be available as linked records of inpatient and outpatient services. Also, Medicare and managed care administrative data may be available only for defined populations of patients and usually do not include information on noncovered services and specific service elements (for example, they may include little information on resources utilized during a hospitalization).

Patient Self-Report Patient self-report enables data to be collected for all medical care received by patients, whether or not it is provided at the study site. Patient self-report data can be used in several ways in an economic study. First, it may be the only means of collecting certain data items (direct nonmedical costs, productivity costs, intangible costs). Second, patients are often the only people with complete knowledge of all of the services they have received. Thus, patient self-report can be used to assess direct medical costs received by patients. However, patient self-report data may not be reliable for several types of information, including data on resource utilization during acute-care hospitalizations or for detailed information on treatments received. In these cases, patient self-report information should not be used as the sole data collection strategy for economic studies. Often, patient self-report is combined with source-document validation. In this strategy, patients report the occurrence of a specific service at a facility, which then leads to follow-up data collection with that facility.

In addition to direct medical resource-use data, patient self-report may include assessment of the amount of caregiver time required for care at home and the number of days lost from work or other activities. Alternatively, work loss could also be abstracted directly from patients' work records, though it is unlikely that this would be feasible in a typical clinical trial unless the trial were restricted to patients with a single employer (for example, in a workplace cancer screening trial).

Several methods exist for collecting patient self-report data for resource use items. Patients may be interviewed, either during a protocol visit or over the telephone. Alternatively, specially designed questionnaires may be mailed to patients at regular intervals. Problems occur with patient self-report data for a number of reasons. Patient recall of events becomes problematic when the recall period is extended, if the patient is a heavy user of medical care services, and/or illness interferes with the patient's mental status. Thus, scheduled visits or telephone or mail contact must be sufficiently frequent to avoid recall problems. An alternative way to avoid recall problems is to ask the patient to complete a diary at home as care is received. The diary should then be brought to the study site at each visit, used by the patient as a reference during telephone follow-up, or used as a reference when completing a mail questionnaire. Diary problems can be minimized through reminder telephone calls to the patient at regular intervals between visits and by means of a letter sent before each study visit reminding the patient to complete the diary. For patients who are illiterate or mentally incompetent, resource data can be obtained from a proxy, such as a family member or close friend.

For an example of the use of combined patient interviews and medical billing records for obtaining direct nonmedical costs, see Bennett et al. (40). Probably the most detailed and comprehensive survey instrumentation developed for the purpose of obtaining information on medical resource by way of patient interview is that developed for the National Medical Expenditure Survey (41).

Assigning Unit Costs: Direct Medical Costs

Data on the costs of resources used by patients may be available from standard costing data, from sources of care for individual patients, or from administrative data sets. Most U.S. hospitals maintain billing systems that can be used to record the costs of resources consumed by patients during a hospitalization. However, hospital billing information will only include a record of the specific services supplied by the hospital and may not include the cost of physician services during the hospital period. Except in specific settings, hospital billing information does not include information on the care received by patients on an outpatient basis. Thus, if a protocol follows patients over an extended period of time, data collection mechanisms should be available to collect both inpatient and outpatient resource use as well as the cost of these resources.

Administrative databases, such as the claims payment records of an MCO or large health plan, are becoming increasingly important sources of resource-use data in clinical protocols. However, given the fragmented nature of the U.S. health insurance system, the use of administrative datasets is not a feasible means of tracking resource use unless the study is designed around specific patient populations, such as patients more than 65 years of age (Medicare) or patients enrolled in a specific health plan.

Hospital Costs The most basic description of costs for hospital care is the cost of an entire hospital period. For example, a study in which the measure of resource utilization is hospital admission will assign an overall cost to each admission. The most common means of developing an estimate of aggregate cost is by first classifying the admission by diagnosis-related group (DRG), which combines the primary diagnosis and procedure with information about comorbidities. Medicare or other insurance payment information can then be used to assign a proxy for cost by the specific DRG. Another simple costing method tracks length of stay and assigns a cost per day, or a "per diem."

Generally, more specific information is collected on hospital resource utilization. Hospital financial information is available in the form of hospital charges. Because hospital charges are thought to overstate the actual costs of service, separate analysis of these data must be undertaken (42).

Hospitals in the United States must report their overall costs and charges to the Health Care Financing Administration (HCFA) on an annual basis. This Medicare "cost report" has been used by investigators to develop a "cost-to-charge ratio" that is used to convert hospital charges to hospital costs for the purpose of economic analysis. These cost-to-charge ratios can be developed either at an aggregate level for the institution or on a more specific level based on the hospital departments providing the relevant services. The relationship between costs and charges at hospitals actually varies considerably between hospital departments. When available, departmental cost-to-charge ratios may be a better proxy for costs of specific services than total cost-to-charge ratios. One severe limitation to this more detailed approach to assessment of hospital costs is that each hospital may have its own system of assigning costs to uniform billing (UB-92) categories. Thus, fresh frozen plasma may be included in one institution's blood bank cost center report but in the operating cost center in a different institution. These differences make aggregation of departmental cost-to-charge ratios a labor-intensive task. A study developed by the Cancer and Leukemia Group B (CALGB) is currently attempting to evaluate the capabilities of administrative datasets within CALGB member institutions. The group is expected to develop recommendations for strategies of requesting cost information from CALGB member institutions for economic evaluations within group studies.

Given the increasing financial pressures resulting from changes in the health care system, many hospitals have developed their own cost-accounting systems to provide more detailed cost information for management decisions. However, not every institution has such an accounting system in place. Where these datasets exist, they can be used to assess the costs of services and may offer a more accurate reflection of the true costs of services than the cost-to-charge ratio (43).

It is difficult to develop costs for specific services from detailed hospital bills. Hospitals often keep track of every service received by an individual patient on a disaggregated basis. Thus, for a specific procedure or treatment—for example, an intensive care unit day or an hour of operating room time—the hospital may bill for each of the hundreds of components of that service separately. In fact, it may be impossible to develop an overall cost for specific services for an institution. One approach to resolving this issue is to develop a regression-based model that will use the resource counts in the economic protocol as predictor variables in decomposing the overall hospital bill. This technique could result in the development of bundled costs for the specific clinical services collected within a CRF.

More recently, hospitals have revised their billing procedures due to changes in market forces. For example, hospitals may agree to a contract price for procedures that are part of the research protocol, resulting in a gap between the actual bill and the contract price. Furthermore, some hospital accounting systems simply record the contract price. In these institutions, the process of tracking costs is predominantly being used for internal cost accounting but not for generating bills to external organizations. Some hospitals continue to generate bills but apply a contractual allowance to represent the difference between the charge and the contract price. Other hospitals have a separate accounting system for procedures priced through research contracts. If the data collection strategy for a proposed protocol is based solely on the use of administrative datasets, investigators must ensure that facilities are continuing to track resource utilization by specific patients in either the billing or cost-accounting systems at each center.

In a multi-institutional study, the costing strategy may be complicated by the inability to collect cost data at some study sites (most often because of lack of sufficient investigator resources to collect these data). In these cases, cost data may be obtained from a subset of study institutions. Statistical techniques to reflect this data collection strategy are discussed in Part V.

Physician Costs Physicians assign CPT codes when billing for their services. In a manner analogous to hospitals, charges for specific physician services may not be directly related to the costs of providing those services. However, there is no physician cost-to-charge ratio. In 1992, HCFA developed the Resource-Based Relative Value Scale (RBRVS) as a measure of the resource intensity of the specific physician services for each CPT code. The Medicare system currently implements this resource intensity measure in physician payment. RVUs may be used to calculate standard costs for physician services (44). It should be noted that there are facility charges associated with many outpatient medical procedures that are not included in the RBRVS fee schedule. These facility costs may need to be developed from other data sources. This is especially true for outpatient surgical services.

Nursing Services Costs for nursing services are usually developed on an hourly basis using an average wage rate for the type of nurse (registered nurse, nurse practitioner, etc.).

Diagnostic Tests Costs for diagnostic tests and procedures can include the costs of physician services as well as the costs of conducting the tests. The costs of physician services for diagnostic tests are reported as described above. The costs of conducting laboratory tests are available using a set of standard laboratory workload units (Lab RVUs). The costs of diagnostic tests include the facility fees for the service. The facility costs for radiology services may need to be developed from other data sources.

Pharmaceutical Services Pharmaceutical charges also vary depending on the pharmacy. One way of standardizing pharmaceutical costs is to use the average wholesale price available for specific pharmaceutical products (45), with the addition of a standard dispensing fee.

Subacute-Care Facilities Costs for subacute-care facilities may be developed based on published price lists using per diem rates. For studies that focus on the use of subacute-care facilities, a more detailed costing approach may need to be developed for the study. This would be the case in a study of alternative hospice treatment programs.

Capital Costs Investigators should also consider that some treatments require the purchase of capital equipment; for example, cancer centers may already own much of the equipment required for new treatments, but community hospitals may have to purchase equipment to implement certain services. Capital equipment purchases become fixed costs that are independent of the standard process of costing for an individual patient. In other words, there may be costs associated with a program of treatment that are shared by many patients, and this cost would not appear on patient billing data (this may especially be the case in phase III trials of experimental, unreimbursed technologies for which there is no market price for the service). Derivation of these costs requires specific costing efforts to be conducted in conjunction with the clinical trial.

Protocol Costs The costs of data collection in a research study should not be included in the economic assessment of the therapy. In addition, the costs of research-related archived materials not used for clinical care should be excluded from the assessment.

Assigning Unit Costs: Direct Nonmedical Costs

Often, these costs are measured from the patient's perspective and include the costs of direct nonmedical services purchased by patients. While these data may be captured as both resources and costs, direct nonmedical costs are more often captured solely as costs of resources. This is usually assessed by asking patients to record their out-of-pocket expenses for these services.

Aggregating Price and Quantity

There are four general approaches to data analysis that investigators may consider in developing the economic component of clinical trials. These include resource use collected for all trial patients and costs collected for all trial patients; resource use collected for all trial patients and costs collected in a subset of study patients; resource use collected for all trial patients and costs developed from administrative data sets; and prospective collection of cost data only (resource utilization would then be derived from the cost data). In economic evaluation of cancer clinical trials, the second and third strategies are the most common.

Some economic evaluation strategies actually omit measurement of resource consumption and concentrate instead on economic measures of resource use, such as hospital bills. These studies are undertaken when collecting resource consumption measures may not be feasible or may be prohibitively expensive, forcing the investigator to choose an alternative "billing" approach.

Generic Economic Evaluation Strategies Resource categories in clinical protocols must be specifically enumerated for both prospective and retrospective studies. The level of precision in the costing exercise is dependent upon the type of study being implemented.

Example. In a trial of low-molecular-weight heparin (outpatient) versus intravenous heparin (inpatient), no difference is expected in treatment outcome; however, a large difference is expected in the utilization of inpatient hospital resources. A list of resource categories to be collected in the clinical protocol might include hospital days by unit type, as well as physician visits.

Example. In a comparison of hospital settings in the administration of a 5HT3 odansetron bolus push versus intravenous administration, it is necessary to microcost the administration of the drugs. Such an approach represents a higher level of precision in the data than is required for the previous example. This study requires a microcosting exercise to track nursing time (and intensity) in providing antiemetic services to patients, with an effort to develop specific prices for these services.

Example. In a trial of adjuvant therapy for breast cancer assessing four versus eight cycles of chemotherapy, there is similar outpatient use of drug administration, pharmacy charges, diagnostic testing, and professional fees. The difference may lie in the hospitalization utilization rate for febrile neutropenia and in outpatient care for nausea, mucositis, dehydration, etc. Thus, investigators must evaluate outpatient resource utilization and occurrence of hospitalizations.

Inpatient Studies Inpatient studies are clinical trials that either occur within the hospital setting or enroll a high proportion of patients that are expected to receive hospital treatment during the course of the clinical trial. Examples include trials of new cytokine therapies as supportive care for bone-marrow transplantation, and studies of new surgical techniques for solid organ tumors, studies of intensive chemotherapy regimens administered on an inpatient basis but potentially having different effects on bone-marrow suppression. Data collection would focus on the intensity of treatment during the hospitalization, including tracking resource utilization by unit type within the hospital, physician visits, surgical procedures, diagnostic tests and procedures, and high-cost medications.

Outpatient Studies Outpatient studies are clinical trials that either occur outside the hospital setting or enroll a high proportion of patients who are not expected to receive hospital treatment during the course of the clinical trial. Data collection would focus on the utilization of services on an outpatient basis, including physician services, nursing services, surgical procedures, diagnostic tests and procedures, and use of high-cost medications (including chemotherapy). Information about acute-care hospitalization would be recorded. However, unless there were an explicit hypothesis about the intensity of the hospital services by treatment arm, data collection on use of resources within the hospital setting could be minimized. For example, data could be collected on length of stay by unit type but not on diagnostic tests or procedures or medication use.

Subacute-Care Studies Studies of patients within subacute-care facilities (i.e., hospice care, nursing home care, or rehabilitation care) require information about the use of the subacute-care facility but often do not require information about other outpatient treatments. Information about acute-care hospitalization would be recorded. However, unless there were an explicit hypothesis about the intensity of the hospital services with treatment, data on use of resources within the hospital setting could be minimized.

Conclusion

In summary, this section addresses some of the major issues in designing and implementing an economic evaluation within a clinical trial. Issues highlighted in this section include the potential direct medical and direct nonmedical resources that can be captured in a clinical trial, the assessment of productivity and intangible costs, the sources of data for capturing resource utilization information, the methodology for assigning unit costs to the resources collected in the protocol, and methods of refining data collection instruments for different types of clinical trials. There is no "boiler-plate" economic evaluation. Clinical and economic investigators must review the requirements for the different data elements within their protocol. They must also assess alternative strategies for collecting cost information for the resources collected within the economic protocol.

	Part IV: Designing an Effective Data Collection Strategy

Top Introduction Part I: Economic Analysis... Part II: Planning an... Part III: Implementing an... Part IV: Designing an... Part V: Statistical Issues Part VI: Economic Evaluation... Conclusion References

 
Designing Data Collection Forms

When integrating economics into an existing clinical trial design, investigators should attempt to collect only those items that are necessary for the economic analysis. Existing forms, such as clinical trial report forms and flow sheets, often contain much of the required information (see Appendix I). It may be necessary to add only a few variables to collect sufficient information to conduct the study. Variables should include any subset of those listed below, depending on the type of trial and the study analysis plan.

View larger version (67K): [in this window] [in a new window]   Appendix I. Excerpt from Clinical Trial Case Report Form

 
When collecting data centered around actual resource use, the CRFs used to collect the data must be designed and pilot tested prior to data collection. They should allow the investigators to record core areas of resource use required for the study. Specific forms are developed to track major resource use by study patients. These forms often are divided into hospital resource use, outpatient resource use, other institutional resource use, caregiver support, and covariates.

Hospital Resource Data The following are important data elements to consider for collection when patients are hospitalized during the clinical trial:

• facility name and location;
  
• admission and discharge dates;
  
• reason for admission (e.g., protocol treatment, elective admission, and urgent admission);
  
• hospital room type (e.g., ward, step-down, intensive care unit);
  
• surgical procedure (type as well as duration of procedure);
  
• transfusions of blood or blood products (date, type, number);
  
• major laboratory tests (date, type, number);
  
• major radiologic tests (date, type, number);
  
• medications (date, type, route, dosage);
  
• chemotherapy (dates, types, routes, dosage);
  
• radiotherapy (dates, fractionation, site, dosage); and
  
• physician visits (date, type, duration in minutes).
  

Outpatient Resource Data The following are important data elements to consider for collection when patients are treated in an outpatient setting:

• date, type, and duration of visit;
  
• type of facility;
  
• reason for visit (routine, elective follow-up, emergent follow-up);
  
• surgical procedure (type and duration of procedure);
  
• major laboratory tests (date, type, number);
  
• major radiologic tests (date, type, number);
  
• medications (date, type, route, dosage);
  
• chemotherapy (date, type, route, dosage);
  
• radiotherapy (date, fractionation, site, dosage);
  
• physician visits (date, type, duration in minutes); and
  
• home care (dates/frequency of visits, skill requirements).
  

Subacute-Care Institutional Resource Use The following are important data elements to consider for collection when patients are treated in subacute-care institutions:

• nursing home (date of admission, date of discharge, level of care); and
  
• hospice care (date of admission, date of discharge, level of care).
  

Caregiver Support The following are important data elements to consider for collection when assessing caregiver burden:

• caregiver burden (number of hours of paid and unpaid caregiver support);
  
• days of usual activity missed (work, school, home activities);
  
• out-of-pocket costs;
  
• transportation costs; and
  
• caregiver quality of life.
  

Covariates The following are important covariates to be collected within the economic trial:

• comorbidities;
  
• patient's demographic characteristics;
  
• geographic proximity of patient to site of care;
  
• insurance coverage status;
  
• caregiver status;
  
• prior hospitalizations (within the past year);
  
• employment status;
  
• disease severity;
  
• prior physician visits (within the past year); and
  
• cancer history (prior chemotherapy, prior radiation therapy, prior surgery, date of diagnosis).
  

Collecting Data

Beginning Data Collection There are essentially three approaches to collecting trial data on a prospective basis—1) collection at a fixed time period; 2) collection at the beginning of a designated medical event; and 3) collection after a treatment cycle or hospitalization. The strategy for data collection should be planned well in advance and will depend on the needs of the specific trial (for example, whether outpatient treatment is being compared with inpatient treatment).

Collection at fixed time periods. Most economic studies query patients about resource utilization at fixed time periods—for example, on a monthly or quarterly basis. This method assures that economic data are not omitted in the study protocol and that follow-up is comparable across the study areas. This data collection strategy is also the best method to assess quality-of-life data for study patients.

Starting from an event. Clinical trials that measure resource utilization from the time of a predesignated event often use toxicity as a marker. For example, a data collection form might instruct, "In the case of toxicity, please indicate (on another form) what resources were used to treat the toxicity." The event-triggered method is limited in that a significant amount of resource utilization may not be linked to toxicity.

Starting after a treatment cycle or hospitalization. Data can also be collected at the end of a treatment cycle, such as a cycle of chemotherapy, or after a computed tomography (CT) scan or other procedure or test. This method provides reasonable certainty—if interviews are conducted at least quarterly—that all instances of resource utilization and the reasons for each will be captured. More frequent interviews may be required for sicker patients, though the data collection strategy should be standardized to keep costs low.

Why Collect Nonstudy-Site Data? Although nonstudy-site data are difficult to collect, the data do reflect resource utilization at sites outside the treatment center. The costs of these resources can be substantial when significant portions of care are provided in these settings. For example, bone-marrow transplant patients may receive much of their follow-up care at nonstudy sites.

Obtaining Informed Consent

Primary economic data collection that requires access to medical records or other confidential sources must have the clearance of the study subject. Trials that follow patients to nonstudy centers must develop more elaborate consent procedures to ensure that all patient records are included in the consent. In general, a consent form should be designed to encompass as many potential sources of information as possible, particularly if the investigators anticipate having to collect information from outside the study center. The consent form should allow the study investigators access to the patients' financial records at whatever sites the patients receive care.

Medical resource-use data collection may involve abstraction by trial personnel or data from the patient's medical records or medical bills. To obtain a patient's medical bill in the United States, the principal investigator must include a release-of-information statement in the patient consent form. This can be a requirement for enrollment in the study or optional depending on the design of the study. Patient trial numbers should be substituted for patient names to maintain anonymity for all confidential data sources for analysis, especially on financial data. Strict confidentiality procedures should be applied to all economic data.

Maintaining Communication Between Research Teams

Most large, multisite clinical trials are monitored at the sites by a team of clinical research associates. These individuals play an important role in maintaining high data quality for both clinical and economic data. Often, they are more experienced at reviewing clinical data than economic data; therefore, it is usually important to conduct training sessions at the beginning of the trial in both the general requirements of the economic study and the specific review that should be conducted. Those items for which source-document review can and should be done must be specified in advance; often the clinical research associate will be a good judge of what is feasible. This training must be followed by regular communication between the economic investigators and the clinical research associate during the data collection period to correct problems as they occur.

Conclusion

In this section, we review in more detail the specific data elements that often are considered for inclusion in an economic evaluation. We further discuss procedural issues, such as the need for informed consent and for maintaining effective communication within the research team. Economic evaluation requires a comprehensive approach to the development of data collection instruments and a data collection strategy. This section presents a general framework for this assessment.

	Part V: Statistical Issues

Top Introduction Part I: Economic Analysis... Part II: Planning an... Part III: Implementing an... Part IV: Designing an... Part V: Statistical Issues Part VI: Economic Evaluation... Conclusion References

 
In this section, we aim to describe the unique characteristics of economic data as they are encountered in controlled clinical trials; we intend, furthermore, to identify a number of the statistical complications associated with such data. We suggest a variety of statistical methods that might be used to address these complications. This chapter is not meant to be taken as a cookbook that offers a standard menu of tools that can be uniformly applied to any economic data collected in the course of controlled clinical trials. Rather, it is meant to alert trial planners and analysts to the statistical concerns that may need to be addressed in the design, implementation, and analysis of trials oriented toward economic issues and to provide some guidance to the rich and growing statistical literature that has been engendered by the increasing practice of collecting economic data alongside clinical trials.

The Importance of a Statistical Analysis Plan

The same care and rigor must be applied to the statistical analysis of economic data as is routinely applied to clinical endpoints in clinical trials. Detailed analysis plans are as important for economic evaluations as they are for clinical evaluations. The primary economic outcome should also be specified as closely as possible, as should covariates that are intended to be included in analysis for descriptive or predictive purposes. Included in the study protocol should be a specification of the statistical strategies that will be used. In deciding on appropriate statistical methods to use, such features of the data, such as its distributional characteristics, the degree and type of censoring, and other types of missing data, should be considered. A properly conducted sample-size analysis should be conducted for economic data. Also, the analyst should consider whether economic data from a clinical trial alone are sufficient for addressing the economic question of interest or whether such data must be supplemented with data derived from other sources or from modeling or extrapolation techniques.

Many of the statistical issues that apply to clinical endpoints are also applicable to economic data. However, economic data are in some ways different and more complicated than clinical data, and additional issues may arise in regard to the analysis of economic data. These characteristics of economic data should be considered before deciding on a strategy for statistical analysis.

Important Properties of Economic Data

Anticipated Uses of Economic Data Clinical endpoints often are reported as ratio terms; for example, treatment A resulted in a 30% mortality reduction compared with treatment B. It is also possible to report costs such that average medical costs for treatment A were 20% greater than for treatment B. This approach may be appropriate when the only aim of the study is to test the null hypothesis of equal costs between two treatment arms.

It is often anticipated, however, that the economic results of clinical studies will be used in economic analyses in which the absolute difference of cost is of interest. So, in a cost-effectiveness or cost-utility analysis, average medical costs for treatment A might be $10 000 greater than cost for treatment B. The same point applies to the clinical outcome in cost-effectiveness and cost-utility analyses. That is, an estimate of absolute, rather than relative, benefit is required for the denominator of the cost-effectiveness or cost-utility ratio. For example, patients receiving treatment A survived 2 years longer than patients receiving treatment B. The anticipated use, in economic analysis, of a ratio estimate, such as the cost-effectiveness ratio—consisting of the ratio of the absolute cost difference divided by the absolute treatment effect difference—may have implications for the statistical design of a trial in which economic considerations are prominent.

Economic analysis may be focused on estimating the economic impact of a treatment for a patient group or treatment setting that is not typical of the trial setting. In such cases, covariate analysis becomes essential. Because different covariates may be more important for economic effects than for clinical effects, it may not be possible to control for these covariates through such strategies as block randomization. Covariates that are of interest in economic analysis and may or may not be controlled for in clinical trials include those reviewed earlier, including sociodemographic variables (sex, age, race, immigration status, income, education, occupation, employment status, census tract of residence, and marital status), comorbidity status, functional status, treatment setting prior to randomization, treatment setting before initial trial care, cost prior to randomization, availability of caretaker at home, availability of social support networks, type of health insurance, and structure of health care provider organization.

The Complexity of Economic Data While a variety of clinical measures may be recorded in trials and used in complex ways in covariate analysis, primary outcome measures in clinical trials often are measurable by a single index, such as death from or recurrence of cancer, etc. By comparison, economic data—not unlike quality-of-life data—tend to be complex.

First, economic data usually are not measurable as a single event but as an accumulation of resource use over time. Second, economic data may consist of a variety of components that may, in turn, serve as substitutes or complements to each other (i.e., positively or negatively correlated). For example, costs may consist of the components of hospital-based costs, outpatient costs, and family costs. Costs may be equivalent for patients A and B, but patient B's outpatient or family costs may have been substituted for hospital costs. Third, the longitudinal nature of costs present specific analytic issues. For example, if certain components of cost have a systematic tendency to be measured with less complete data in treatment arm B, and this is not recognized in the analysis, an incorrect inference could be drawn that costs are lower for B.

Within these broad components of cost, cost is measured as a collection of finer components. For example, hospital costs may consist of the summation of many procedure or service components, such as operating room use, intensive care days, normal hospital days. The case is similar for outpatient care. Because cost components may be substitutes or complements, it is important that collection of all relevant cost data be as comprehensive as possible. In addition, it follows from this property of costs that tests of statistical significance should not be performed on individual components of cost, nor should individual cost components be excluded from the summation of costs because of statistical insignificance (46).

The second way in which economic data are complex is that cost is the sum of products of resource use (quantity) and unit cost (price). Cost may not always be separated into its component quantity and price vectors, but often it will be, because there is great uncertainty about the validity of the price vectors that are embedded in the usual accounting systems available from hospitals and physician office records. When the quantity and price vectors that compose cost are obtained separately, the question of how to treat the separate and joint variability of these quantity and price vectors arises.

Because economic data tend to be complex, the statistical analyst is more likely to have to deal with the problem of missing data. This is due both to the variety and complexity of data elements necessary to measure economic variables and to the fact that economic data elements may be drawn from administrative data systems that are not designed primarily for research purposes. Also, to the extent that some data elements are derived from questionnaires administered to patients or caregivers, missing data will be a problem.

Distributional Characteristics of Economic Data Cost data may have a relatively skewed distribution. An idea of what the distributional characteristics of cost data from cancer clinical trials might look like can be inferred from observational cost data from the SEER¹-Medicare database (47). This database links records of incident cancer cases from NCI's SEER system to claims data from the Medicare system maintained by HCFA.

Tables 5 and 6 show the distributional characteristics of actual payments made by Medicare for treatment of colorectal cancer by stage of diagnosis, covering all costs from the date of diagnosis to 6 months following those dates (only patients surviving at least 6 months from the date of diagnosis are included in the analysis). Table 5 shows data using charges submitted to Medicare as a measure of cost, and Table 6 shows data using actual reimbursements paid by Medicare as a measure of cost. In all cases, these data do not fit the assumptions of normality, and for all cases except one—reimbursements for distant-stage disease—the data tend to be more highly skewed and dispersed than data from an exponential distribution.

View this table: [in this window] [in a new window]   Table 5. Distributional characteristics of SEER/Medicare Data: submitted charges for colorectal cancer—cumulative charges to 6 months from diagnosis*

 

View this table: [in this window] [in a new window]   Table 6. Distributional characteristics of SEER/Medicare Data: Medicare reimbursements for colorectal cancer—Cumulative reimbursements to 6 months from diagnosis*

 
Actual economic data encountered in clinical trials may be less skewed than the data displayed in Tables 5 and 6 because the population and treatments studied in the context of clinical trials may be more homogeneous. On the other hand, economic data encountered in clinical trials may be more skewed because, unlike the SEER-Medicare data, it may be derived from a wide variety of accounting systems. Also, the inclusion of additional components of cost, such as patient out-of-pocket or time costs, may introduce additional components of variance and skewedness. Dudley et al. (48) present another example of highly skewed cost data from a clinical trial of coronary artery bypass graft surgery.

Statistical Methods for Analyzing Economic Data

Descriptive Methods Mean costs are perhaps the most important univariable summaries, and in the absence of censoring problems, they can be simple arithmetic means. Means are important because their multiplication by population size is an estimate of total system costs, and the comparison of total system costs conditional on comparison treatments can have economic meaning for policy evaluation. However, when cost data are heavily skewed, mean costs may not be descriptive of typical patients, so median costs and other quantiles should always supplement the presentation.

The standard deviation of costs may not be very informative, as its interpretation depends on normality of costs (one frequently sees illogical published figures such as "the mean cost was $10 000 plus or minus $12 000"). An alternative set of summary statistics would be the mean, median, and quantiles such as 0.05, 0.1, 0.25, 0.75, 0.9, and 0.95. Graphical presentation of the empirical cumulative distribution function of costs is even more informative.

Univariable Methods Analysis of economic data without informative censoring. In some trials, the economic data of interest derives from a single event or a series of events unrelated to the subsequent flow of costs or survival. Also, cumulative costs over a fixed duration from the date of randomization are of interest, and observations on costs up to this date are complete. In cases such as these, there will be no informative censoring. A variety of statistical methods applicable to cost data are available in this case.

Testing the entire distribution. As stated above, the difference in mean costs will be of most interest. However, it is useful to conceptualize cost comparisons by comparing the entire cost distribution between treatment groups. The most general method for comparing distributions is the Kolmogorov-Smirnov test, which is based on the maximum absolute difference between two cumulative distribution function estimates. If the sample is large enough to make relative efficiency a nonissue, the robust Kolmogorov-Smirnov test would not have many competitors. However, for moderate to small studies, this general nonparametric test does not have as much power as tests directed at detecting differences in specific parameters of the distribution, such as the mean or median.

Tests for difference on means, medians, and other parameters. For directed tests, one can use either parametric or nonparametric tests. The latter may be preferred because 1) if normality holds, nonparametric tests such as the Wilcoxon test still have 0.96 efficiency with respect to the optimal test (Student't test ); and 2) if normality does not hold (the likely condition for cost data), nonparametric tests can be much more efficient than the Student't test. For symmetric distributions, the Wilcoxon two-sample test addresses whether the mean or median cost differs between groups, because under symmetry the mean equals the median. For the general case in which the distribution may be asymmetric, the Wilcoxon test is still a valid test for differences in central tendency. As such, the Wilcoxon test is most sensitive in test of the pseudomedian, defined as the median of midpoints of all possible pairs of costs. More important, however, the Wilcoxon two-sample test is a valid test of the null hypothesis that the probability that costs in group A exceed costs in group B, for randomly chosen pairs of patients, regardless of the distribution.

The Wilcoxon statistic is a simple translation of an estimate of the probability that a random A patient has incurred costs exceeding those of a random B patient. Thus, the Wilcoxon test has many advantages for univariable resource-use comparisons. Other nonparametric tests, such as the log-rank test, can also be considered. The log-rank test has the advantage of being a special case of the Cox semiparametric regression model, which may come into play when covariable adjustment is needed.

When the log of costs is thought to be normally distributed, log-transformed comparisons testing for differences in the median cost (equivalent to testing for differences in mean costs if the group variances are equal) have often been used. Zhou et al. (49,50) have developed a method, the Z-score test, to test for differences in means when group variances are not equal by recognizing that the log of the mean of the untransformed data is equal to the mean of the log-transformed data plus one half the variance of the transformed data. The Z-score test gives type I errors closest to nominal levels when variances differ between the two distributions being tested.

The validity of parametric tests, such as the Z-score, depend on the correctness of the assumption about the distributional characteristics of the data, which may be difficult to establish with relatively small samples. One way to obviate this difficulty is to use tests whose validity does not depend on the distribution of the data, such as permutation t tests (51) or bootstrap techniques (52). These approaches are preferred over the Z score when it is uncertain that the distribution of log-transformed cost data approximates a normal distribution. The disadvantage of permutation tests and the bootstrap technique is that these methods generally are not available in the form of "canned" software packages and, therefore, require a moderate amount of custom programming by the analyst. The bootstrap technique is further described later in this section.

Covariable Adjustment and Multivariable Modeling Methods In general, two reasons for using covariable adjustment in the context of a randomized clinical trial are to gain power to shorten the confidence interval (CI) for the treatment effect and to estimate costs and health effects for important subgroups when trial participants are heterogeneous. Some authors have found that for binary endpoints or censored failure time outcomes, covariable adjustment may not result in any appreciable gain in precision or power (53,54). However, in heterogeneous patient populations, estimating crude effect ratios (unadjusted odds or hazard ratios) can result in different estimates than adjusted estimates, and the crude estimates may not apply to any patient. Also, unadjusted models may lack fit. For example, the treatment may operate in a nonproportional hazards fashion without adjustment but satisfy the Cox model's proportional hazards assumption with adjustment (55,56). When patients have a moderately wide prognostic spectrum, unadjusted tests (e.g., unstratified log-rank test) may have poor power in comparison with adjusted tests (e.g., from the Cox model) (57).

In economic analysis, the outcomes of interest are the absolute cost difference and the absolute effects of treatments. Even when power and precision are not issues, modeling multiple patient descriptors or prognostic factors is necessary when one wishes to translate relative treatment benefit into absolute benefit. It is easy to demonstrate that, even when relative treatment effects are constant across all types of patients, absolute effects vary when the variation of prognostic factors is moderate across enrolled patients. The absolute survival advantage of a treatment is strongly dependent on initial survival in the absence of treatment as well as the relative survival advantage associated with the treatment.² Thus, covariable modeling is necessary in estimations of absolute effects of treatment on event probabilities or survival time that comprise the denominators of cost-effectiveness ratios, and covariables may also be needed to increase power.

Similarly, for cost endpoints, one expects that covariable adjustment will almost always increase power, and it will also be useful for estimating cost distributions (or point estimates and confidence limits) for different types of patients.

Regression models. Ordinary multiple regression currently is the most commonly employed regression model for analysis of costs. These models can be used to derive unbiased estimates, but when the distribution of cost data is highly skewed, the estimates may have high variance, and confidence limits for mean costs may not have the desired coverage probabilities. As discussed briefly above, one of the traditional methods for regression analysis of costs is to model the log of costs. For example, Rutten-van Moken et al. (58) use a log transformation of costs in their analysis of economic data after first applying a linear transformation to deal with the problem of zero costs. They perform regression analysis on the log-transformed costs and then use the Duan "smearing" estimator (59) to retransform the regression-adjusted cost estimates back to the linear domain. This method may not be entirely satisfactory because the residuals may still not be normally distributed, and the assumed multiplicative relationship between patient variables and cost (on the original scale) may not be sensible. For example, analyses of data from a clinical trial on the cost of coronary artery bypass graft surgery found that one would have to take sixfold logarithms of charges to achieve normality (48).

An alternative to using log transformation models is to use robust regression techniques. The Cox semiparametric regression model (60,61) has demonstrated several advantages in analyses of resource-utilization data (48,62,63) characterized by a skewed distribution. Regression coefficient estimates for the Cox model are identical whether one analyzes costs on the original scale or takes any monotonic transformation of costs (such as the logarithm). Antilogs of regression coefficients are hazard ratios, but these can be converted to cost ratios (64). Once regression coefficients are estimated, the entire cost distribution, or the mean, median, or any other quantile of cost, can easily be estimated for individual types of patients. By assigning treatment as one of the predictors, one can readily estimate differences in mean or median costs using the fitted model. As the Cox model is distribution-free for any single type of patient, there is no a priori assumption about how the mean and median are related mathematically. Model fit can be checked using standard methods, i.e., smooth-scaled Schoenfeld residual plots for checking the proportional hazards assumption, and smoothed martingale residual plots or regression spline fits for determining linearity of predictors (64). Note that the Cox model in no way assumes that any censoring is present, so it can be used to analyze continuous response variables of many types.³

It should be recognized that while the standard Cox model is often used to analyze clinical outcome data in the presence of censoring, this approach to censoring may result in bias in the cost realm (although the bias resulting from ignoring censoring may be even greater). This issue is discussed in more detail below.

Analysis in the Presence of Informative Censoring

Cumulative costs, which are the integral of the flow of costs over time, often are the economic data of primary interest in clinical trials. This flow may not be constant over time, and in many cases, the flow will increase beginning sometime prior to death. Observations of this cumulative flow from a trial may include patients with various times accrued since the time of randomization and may include some patients who have died, others who are close to death, some who will not die for many years, and some who are lost to economic follow-up at various times from randomization.

Survival Techniques

Standard techniques of survival analysis have been recommended for analyzing cost data in clinical trials in the presence of censoring. For example, Fenn et al. (62) have recently advocated the use of the Kaplan-Meier estimator for comparing costs in clinical trials. This approach has obvious appeal because it is parallel to the usual method of analyzing clinical survival in cancer trials.

However, the scale of cumulative cost is not strictly equivalent to the scale for the clinical endpoint of time. Time has a constant rate of flow while cost does not. In addition, changes in the flow of costs are not random; rather, increases in the flow of costs tend to be indicative of death. Thus, when cost data are censored prior to death, this censoring is informative with regard to costs and survival. Even when no deaths are present in the data, the different scales for cost and censoring can cause informative censoring.

A number of variations on the Kaplan-Meier estimator have been proposed to take this complication into account. Etzioni et al. (65) have proposed the Kaplan-Meier sample average (KMSA) estimator. For each interval, the KMSA estimates the sample average cost for a given time interval as the average cost of all individuals who were alive and observed at the beginning of that period. Total cumulative cost is then given as the weighted sum of all interval costs where the weights are the Kaplan-Meier estimates for the probability of survival up to that interval (since the true probabilities are not known because of censoring). This estimator works well when 1) cost data are available at frequent intervals; and 2) the censoring mechanism is independent of survival and cost in the sense that individuals are not censored because they appear to be at especially high or low risk of death. In addition, individuals are not censored because of accumulation of especially high or low costs relative to uncensored cases.

A variation on the KMSA estimator is the Pathwise estimator (66), which requires only data on cumulative costs for individuals who have died (or reached some other "terminal" follow-up point, such that the flow of costs at that point are not correlated with the probability of death). The Pathwise estimator of average cumulative cost is the weighted sum of the average cumulative costs (of individuals who died) in each interval, where the weights are the Kaplan-Meier estimates for the probability of survival up to that interval. The Pathwise estimator does not depend on the availability of cost data for short intervals (data based on annual intervals can be used), and it yields an estimate of the distribution of cumulative costs as well as the mean. The efficiency of the Pathwise estimator is limited by the proportion of cases in which death is observed. A variant of the KMSA/Pathwise approach has been used to estimate cumulative cancer treatment costs using Medicare data linked to the SEER tumor registry system by treatment phase (67). Efforts are currently underway to extend the KMSA/Pathwise approach to Cox regression (66). Etzioni et al. (68) use SEER-Medicare data combined with simulated censoring to demonstrate the bias in estimating costs when using standard Kaplan-Meier and Cox regression techniques and the improvement achieved by using the KMSA and pathwise methods. Etzioni et al. (65) has discussed the bias that may be introduced in cost extrapolations using the KMSA approach when costs are discounted.

Parametric survival models have also been advocated as a method for extrapolating costs beyond the endpoint of trial observation (69,70). Fenn et al. (62) describes this approach using standard survival techniques. The results of parametric extrapolation may be very sensitive to the specification of the survival function and, within trial data, may be inadequate to allow strong discrimination between specifications. In this case, the variance introduced into cost predictions by model specification uncertainty should be recognized (71).

Missing Data

Several types of missing-data problems are likely to occur in relation to economic data collected in clinical trials. The types of missing data discussed in this section are different from the types of missing data that can be dealt with using the survival methods reviewed above. This section discusses data that are missing due to nonresponse or a failure to collect or record specific data items due to technical or logistic problems—not as a result of censoring inherent to the study's time frame.

Item Nonresponse Item nonresponse occurs when one or several of the elements of an outcome variable at a specific point in time are missing but when there is still a great deal of data available on the patient. For example, data on the number of physician visits for a patient might be missing for a specific week but be available for all other weeks; or unit cost data for a specific procedure may be missing from the accounting system of one study hospital but available at others. Item nonresponse can occur at intermediate points in time, in which observations exist before and after the missing item, as well as at the point of a final outcome. Because item nonresponse implies that there is sufficient information on the patient already available, it is reasonable to include the patient in the analysis despite the missing elements. It is acceptable to impute missing data when the analyst believes that the study will not be biased by imputing the missing values.

Unit Nonresponse Unit nonresponse occurs when all or most of the information is missing on the unit of observation, such as all hospital days for a particular patient or all unit cost data for a particular study hospital. One type of unit nonresponse, in contrast to censoring caused by the time frame of the study, is the type of missing data not handled well by the survival methods discussed above (i.e., when the cost histories of patients who "drop out" are correlated to the reason for dropping out and are very different from patients who do not drop out).

Imputation Methods for Missing Data Methods of imputation for missing unit data are reviewed by Little and Rubin (72) and Rubin (73), and specific application to health care data is described by Rubin and Shenker (74). Imputation strategies range from using the last observed value of the missing item or unit to using mean values of the missing item or unit to modeling the missing values using regression or simulation models that link the missing values to observed data. The appropriate imputation strategy depends on the analyst's judgment regarding the nature of the missing data. If the missing data are believed to be missing completely at random (MCAR) (i.e., missing data are independent of observed data or unobserved data if the latter were known), rather simple methods of imputation are appropriate. If data is not MCAR, but the probability of missing is related only to observed data, then various structural imputation models may be appropriate. Lavori et al. (75) apply Rubin's method of multiple imputation to unit data missing from a clinical trial due to patient nonadherence. They then compare the results to those obtained using the simpler last value approach and using only complete data records. Multiple imputation gives values of clinical outcomes that are between the extreme values of the other two methods and yields higher (more realistic) variances. Bootstrapping can also be used to compute variance estimates and CIs that take into account the increased variance resulting from having imputed missing data (76). McGuigan et al. (77) compared the results of the two-stage Heckman sample selection model and the results of inverse sample weights based on logistic regression to results actually obtained when additional follow-up data were obtained for a cohort study on a school-based drug prevention program. The weighting strategy performed better than the sample selection model and better than analysis of data without the additional follow-up. Performance of the sample selection model was very sensitive to the particular specification of the model used.

When billing or cost information are not available for all patients, cost data can be collected on a subset of patients with resource utilization data being collected for the entire sample. Regression techniques can then be used to impute charges and predict bills. It should be remembered that the selected institutions from which bills are obtained are not necessarily representative of other institutions. For example, institutions with the best billing systems may also have more efficient delivery care systems. Or, the billing systems upon which unit costs are imputed may have higher unit costs, because they are less efficient and have less advanced billing systems. To prevent bias, resource utilization must be compared across institutions.

Although much time and energy are involved in collecting bills at every institution, it is recommended that an effort be made to obtain complete billing data for major resource utilization events when possible. The likelihood that the distribution of expenses for an institution with missing billing data are different than other institutions should be addressed. If a difference in resource use is found, costs from the hospitals that seem to be driving those differences in resource utilization should be obtained.

How should the collection of data for both resource consumption and pricing studies be handled for hospitals that may be in pricing consortia and will not disclose the prices of some resources or certain pharmaceuticals? A price may have to be assigned. For example, a military institution, which is a large accruer to a number of trials, will have a different price on almost every resource than a nonmilitary site. Access to technology and length of stay may radically alter pricing information among institutions. Therefore, one may only be able to assign a reasonable across-the-board price in the estimate. This must be explicitly stated in the analysis. Speaking directly to Internal Review Boards (IRBs) concerning the financial use of the data, especially in terms of confidentiality, may assist in disclosure of needed information.

Variance of Price and Quantity Vectors

In some studies, direct measures of monetary cost will be available. In others, costs will be constructed by multiplication of price (unit cost) vector and a quantity (resource utilization) vector. In the latter situation, data on resource utilization often will be obtained from observations on study subjects, while data on the price vector will be obtained from data sources not directly related to these observations. In this case, it will be difficult or impossible to determine the variance of the price vector and the covariance between the price and quantity vectors. In this case, it is permissible to treat the price vector as if it were a parameter of fixed values. But, it should be recognized that this will result in an optimistic estimate of the variance of cost. One way to account for this is to perform a sensitivity analysis in which the values of the price vector are varied over a "plausibility interval." Upper and lower bounds for the price values might be determined from observational data, such as billing, reimbursement, and cost-accounting records of various health care providers or third-party payers.

Statistical Issues Related to Cost-Effectiveness Ratios

What Should Be in the Denominator? The typical quantity to use in the denominator of a cost-effectiveness ratio is the increase in expected (mean) life length, considering death due to all causes. There is nothing inherently wrong with using the increase in median life length, and this quantity requires less extrapolation and in some cases does not require parametric survival modeling. Another choice is the mean restricted life length, the area under the estimated quality-adjusted survival curve from zero until the end of the follow-up period of interest (78,79). This measure also eliminates the need for extensive extrapolation. Another choice for the denominator is the difference in t-year survival probabilities. Mark et al. (80) compared many of these measures in one setting. Similarly, the denominator for a cost-utility ratio is the life length adjusted for quality.

Variance of the Cost-Effectiveness Ratio When economic and clinical outcome data from clinical trials are used to construct a cost-effectiveness (or cost-utility) ratio, the statistical properties of this kind of ratio estimator should be appreciated. The full variance/covariance structure of costs and clinical outcomes will usually not be known. A variety of techniques have been developed to provide estimates of the variance of the cost-effectiveness ratio in this situation. These are reviewed by Manning et al. (81) and Polsky et al. (82,83). These techniques include univariate sensitivity analysis, multivariate sensitivity analysis, computation of confidence regions, and overall CIs using the delta method or simulation techniques. Naïve use of univariate analysis will most often result in estimates of CIs that are unrealistically narrow, and multivariate sensitivity analysis may result in CIs that are too wide if only extreme bounds are examined. Manning et al. (81) compare the results of univariate and multivariate sensitivity analysis, the delta method and Monte Carlo simulation using an empiric example of the cost-effectiveness of breast cancer screening, where cost-effectiveness is a function of nine epidemiological, technical, and economic parameters.

Application of the bootstrap method to obtain joint variance plots of the numerator and denominator of the cost-effectiveness ratio and to perform threshold analysis is described by Hlatky et al. (84). The bootstrap can also be used to simultaneously take into account other sources of variation such as missing data and model selection (85).

Interpretation of the confidence interval from a bootstrap assessment is based on the proportion of times a result falls into each of the four possible quadrants of the cost-effectiveness plane, as well as an assessment of how often the cost-effectiveness ratio is below a prespecified threshold (e.g., $50 000 per QALY gained).

Fig. 1 provides an example of the results of a bootstrap procedure that was used to estimate the confidence intervals for the comparison of costs and effects of a new therapy and usual practice. The x axis represents the difference in mortality between the two therapies. Points to the right of the line representing zero (i.e., no difference) indicate that the new therapy increased the probability of survival, while points to the left of the line indicate that the new therapy decreased the probability of survival. The y axis represents the difference in costs between the therapies. Points above the line representing zero indicate that the new therapy increased costs, while those to the right indicate that it decreased costs. The diagonal line represents the combinations of changes in costs and effects that bear a ratio of $50 000 per death averted, one example of a prespecified level of cost-effectiveness that might be proposed to justify new investments.

View larger version (36K): [in this window] [in a new window]   Fig. 1. Imputation of missing data.

 
In quadrant I of the Fig. 1, the 104 points below and to the right of the line represent cases in which the new therapy has a ratio below $50 000 (i.e., satisfy the acceptability criterion), while the 12 points above and to the left of the line represent cases in which the new therapy has ratios that fail to satisfy the criterion.

In quadrant III, the new therapy is less costly and less effective. The 120 points below and to the right of the $50 000 line indicate instances in which the use of the more expensive, more effective usual practice has ratios that exceed the $50 000 ratio (i.e., the new therapy would save enough money to potentially justify the trade-off in health), while the 8 points above the $50 000 line indicate instances in which the adoption of more expensive, more effective usual care would have ratios that satisfy the criterion.

In all, in 974 of the 1000 replications, the new therapy had a change in costs and effects that satisfied the $50 000 criterion (i.e., given a two-tailed test, we can conclude that there was just over a 5% chance that the new therapy satisfied the acceptability criterion). While there is evidence that the therapy satisfied the $50 000 acceptability criterion, the statistical tests for the individual comparisons of cost and mortality differences between the two therapies indicate that there were no differences between the therapies at the 95% CI (in 122 of 1000 cases, the costs were increased by the new therapy, while in 878 they were decreased; in 865 of 1000 cases, mortality was reduced by the new therapy, while in 135 it was increased).

Stinnett notes that the bootstrap method can also be used to correct for bias in the cost-effectiveness ratio that may be present when the ratio is based on the mean cost and effect estimates derived from relatively small samples (86).

Sample-Size Analysis Since the outcome of interest for economic analysis is the absolute magnitude of cost difference, it is appropriate to conduct sample size analysis from the perspective of gauging the likely precision of a CI (87,88). The results of economic studies from clinical trials may also be used to compute cost-effectiveness or cost-utility ratios. It is important, then, that clinical trials with important economic components be designed so that the clinical benefit, cost or resource utilization, and cost-effectiveness ratios can be estimated reliably. This can be accomplished by determining the sample size that will yield acceptably small confidence limits for the three quantities of interest (89). The limiting factor in most cases will be the precision of the cost-effectiveness ratio, as this ratio inherits noise from both numerator and denominator, and imprecision in the denominator will cause great fluctuations in the ratio (81). Other factors to consider in sample size analysis are the magnitude of the cost difference that is considered economically meaningful, the distributional characteristics of cost data, and the efficiency of estimation methods used to analyze cost data. A naïve evaluation of cost data like those in Tables 5 and 6 might indicate that a few hundred subjects per arm are adequate to establish cost differences of a few thousand dollars with 95% confidence. But lower precision would be expected for a cost-effectiveness ratio, and precision would be expected to deteriorate because of nonnormality of the data and missing and censored data. Application of Fieller's Theorem may be useful in estimating sample sizes based on confidence intervals for cost-effectiveness ratios (90).

Because trials tend to be underpowered for some or all economic outcomes, an alternative approach to demanding strict adherence to conventional hypothesis testing is to establish a sample size that is sufficient for establishing that the specified economic outcome is true with some probability. For example, one could establish the probability that the cost-effectiveness ratio of a treatment is less than $20 000 per life-year saved. Definitive probabilities of effects such as this can only be achieved when the Bayesian posterior distribution is sufficiently sharp or traditional confidence limits are sufficiently narrow (91).

Example. Distributions of charges and reimbursements for cancer treatment can exhibit strong positive skew and, thus, cannot be assumed to be normally distributed. A similarly shaped distribution with positive skew is the log-normal. While the log-normal may only provide a first-order approximation to the true underlying distribution of charges, reimbursements, or costs, it can in some limited examples provide a useful method to estimate sample sizes needed to test for differences in cancer treatment costs between two samples.

To calculate sample size for tests of differences in costs, it is possible to convert everything so that calculations can be performed with normal distributions.⁴ One usually generates a log-normal distribution by first generating the Zs (for some µ and ) and then exponentiating to get X =exp(Z). This is the reverse problem, in that the moments of the log-normally distributed Ys are given and it is desired to return to the Zs to conduct the power calculations.

The following simplifying assumptions are made: the costs have a two-parameter log-normal distribution, and CV_X = 1 (where CV_X is the coefficient of variation or the standard deviation of X divided by the mean of X).

We then solve following equality:

so that ß can then be calculated as ß = E(X)/exp(²/2), with the result that µ = ln ß.

In Fig. 2, we fix Type I error level, , as 0.05 (one-sided) and power, or 1-ß as, 0.80. The expected difference between groups, , is specified as a percentage of the mean. Note that these calculations do not depend on a particular value for the mean but rather depend on the ratio of the standard deviation relative to the mean and as a function of the mean.

View larger version (18K): [in this window] [in a new window]   Fig. 2. A case study of sample size requirements. Sample size required for = 0.05 (one-sided), 1 - ß = 0 for comparing costs between two samples. N refers to the combined sample sizes of the two equally sized groups. CV refers to the coefficient of variation. Lines indicate sample size need for % differences over ranges of CV. (Unpublished figure and data provided by Kimberly McGuigan and R. D. Fricker at the Rand Corporation.)

 
Price Adjustments

Unit prices change over time and vary by geographic location. When trial costs occur over a significant period of time and/or multiple locations, price adjusters should be used to allow costs to be expressed in a common "base." Even when trial costs are incurred over a short period of time and/or at a single location, adjustments should be made if costs are to be related to a different time or place (e.g., update of 1990 trial costs to 1997 costs or extrapolation of trial costs conducted in Olmstead County, Minnesota, to the United States overall).

Methods of adjusting prices are readily available (92) and include the medical component of the Consumer Price Index (CPI), which tends to overestimate overall inflation of health care costs, and the Fixed Weight Index (FWI) from HCFA, which results in lower estimates of inflation than the CPI. The FWI relies more on changes in the input prices of goods and services that go into the production of health care services. Two components of the FWI that may be of particular interest are the Prospective Payment System (PPS) input price index for hospital costs and the Medical Economic Index (MEI), which measures changes in the cost of physicians' time and operating expenses. Price adjustment has also been accomplished using the Geographic Adjustments Factors (GAF) for hospital wage costs and physician practice costs, including costs at the county level.⁵

Discounting the Costs and Benefits of Cancer Care

Economic theory indicates that assessments that look at costs and benefits of medical therapies over periods of time longer than a year should incorporate discounting. This concept embodies the fact that the costs and outcomes may occur at different points in time and that their value changes with time.

Costs incurred and effects realized in the present are generally greater than costs and effects of similar nominal value incurred later—if not used in the present, the resources could otherwise be invested in the interim. Hence, it is necessary to express all costs and benefits at one point in time, usually the present. This is done by "bringing back" all future costs and effects of a therapy and expressing them in terms of the present value.

Conclusion

Resource utilization and cost data collected within cancer clinical trials pose a series of important analytic challenges for economic investigators. These challenges stem from the need to analyze the data collected on a longitudinal basis within clinical trials as compared with dichotomous outcomes such as survival. Given the nature of data collected within economic studies, the potential for missing data is greatly increased over that usually seen in clinical trials without economic components. This section presents a state-of-the-art review of methodologic issues in economic evaluation of cancer clinical trials and highlights the analytic issues for investigators pursuing analysis of economic data in clinical trials. The section also services as an important reference to help all members of the economic evaluation team better understand the complexity of economic study design.

	Part VI: Economic Evaluation in the Cooperative Group Setting

Top Introduction Part I: Economic Analysis... Part II: Planning an... Part III: Implementing an... Part IV: Designing an... Part V: Statistical Issues Part VI: Economic Evaluation... Conclusion References

 
This section addresses practical issues that affect the ability of cooperative groups to collect data in multisite clinical trials. Because data will be gathered at different sites, clear and specific arrangements must be made from the beginning of the trial regarding data collection processes. It is recommended that discussion about an economic component—including how appropriate, complicated, and extensive it is—should be part of the concept review of a trial. The following issues are relevant to the design of an economic study in a clinical trial.

Are the Data Required?

If resource utilization or cost data are included in a trial, they should have the status of required clinical data and be part of any monitoring and reporting system. But should the commitment to collecting resource utilization or cost data be required for patient registrations? The best way to address this problem is to link therapeutic trial registration to completion of ancillary outcomes such as the economic study (i.e., submission of resource data is designated as a formal eligibility requirement in the protocol). Most resource-utilization data required for economic analysis are routinely collected for clinical outcomes. A resource-utilization strategy should not, therefore, be burdensome to clinical research associates (CRAs) or investigators and should not result in substantial biases in recruitment to the economic study. A separate consent for release of billing data may be required. If resource-utilization data are being collection, the patient can be informed in the main consent form that the medical charts also will be reviewed for cost outcomes.

However, even in a single cooperative group trial, some centers do not enter patients into every part of a trial. Some groups, for example, do not require that patients participate in ancillary studies (e.g., provide quality-of-life outcome data); a patient can still be eligible for the treatment component of the trial as long as the ancillary study is offered to the patient. This approach has the potential to bias the ancillary study outcomes, as physicians might only offer the quality-of-life study to a less ill subset of patients. Economic investigators are also concerned about this potential for bias. If it is not possible to include all patients, one compromise is to select a priori subsamples of patients; for example, include the first 500 patients randomized to the trial in the cost study (given that they meet the eligibility requirements). Data obtained with this approach can also be biased depending on historical cohort factors, such as the introduction of a new supportive care treatment midway through the trial. In this case, the cost-saving or cost-inducing effects will be missed.

Who Will Collect the Data at Cooperative Group Institutions?

CRAs collect and submit clinical data in most cooperative group settings. Will CRAs also collect the cost data, or should these data come directly from patient accounts offices at the institutions? It is also possible to collect cost data directly from patients. If collection of data from patients is necessary, this should be done by the local site and not the central data center (unless outside support is obtained for this effort, and then only with a communication linkage with the clinical investigator). Collection of data directly from patients by central office staff may also involve additional informed consent issues requiring local IRB approval. A combination of these data sources can be considered for cooperative group research.

How Will Data be Collected, Monitored, and Submitted?

The mechanism for data collection should be specified in the protocol. Part IV of this workbook presents a number of data collection sources for this effort. Forms represent the most common method used in cooperative group trials, and they can be mailed, faxed, or electronically submitted to the coordinating center. In addition, data can be submitted to the central office on diskette or tape. A number of procedural issues must be specified in the design of the economic study:

Highlight up front the importance of monitoring and data quality.

Set standards for data collection.

Include all resource-utilization and cost data in the group's system for monitoring data submission (e.g., "expectation systems"). This may require early input from programming staff.

To ensure that the necessary data are collected, be certain that protocols are written clearly and that terms and procedures are defined precisely.

Develop an analysis plan for the economic study.

Example. Detailed data collection procedures should be developed for each component of the protocol. An example of the issues to be addressed in developing these procedures follows for the collection of economic information from patients by telephone.

Who will be the contact person?

Is the contact person from the local hospital or a centralized location?

Are sufficient staff available?

Is there an informed consent issue? Does the clinical trials consent form require special permission for this data collection approach? Does it require the consent of the physician contact?

How will confidentiality be monitored? Are there additional IRB issues to consider in economic studies? Does clinical trial medical chart access obtained in consent forms for clinical data needs cover the level of information required for economic studies? At the very least, amend consent forms to include the cost objectives of the trial. If patient bills are evaluated, additional IRB approval may be required, depending on the local IRB. Speaking directly to IRBs concerning the financial use of the data especially in terms of confidentiality may assist in disclosure of needed information.

Do existing quality control procedures implemented in cooperative group trials address data quality issues for economic studies, particularly for billing or resource-utilization data collection from the patient? Economics investigators, coordinating center programmers, and statisticians should communicate early on about any particular features of economic data that may need to be incorporated into data entry verification systems and tracking systems. Procedures for standardizing the collection of all economic data are necessary but this is particularly important for data obtained for care delivered "off-site" (e.g., at other hospitals, outpatient clinics, nursing homes, in-home). Data that require additional documentation at the institution level (e.g., data obtained through a telephone interview) and allowable source documentation must be clearly specified.

Given these arrangements and approvals, the monitoring plan can incorporate early and ongoing review of the correctness of economic data as it is received at the coordinating center. This would allow changes to be made in the data collection process if appropriate data are not being collected. It is important to specify how long economic data will be collected and monitored.

Training: An Important Quality-Control Procedure

Training at cooperative group meetings is one way to achieve standardization in procedures for economic studies. Explanation of economic study objectives and definition of terms are important when new research outcomes are introduced in cooperative group trials. Roundtable sessions on various research topics are held for CRAs at cooperative group meetings, and these meetings could easily incorporate sessions on economic evaluation. These training sessions can also provide a vehicle for obtaining feedback on data collection forms for economic studies. Investigator meetings, group meetings, or nurse/CRA training sessions can also provide an opportunity for reviewing data collection forms. All of these meetings provide the opportunity to ask volunteer institutions to conduct pilot tests for the economic study forms and procedures. If possible, it is ideal to have separate training sessions for economic studies of cancer treatment protocols. However, time is very limited at these meetings, and sessions separate from the main training sessions may not be well attended. It is advisable to include at least limited training on economic studies during the regular training sessions at cooperative group meetings.

Consider terminology in training efforts. Confusion can arise around semantics in collecting quantities of resource use. It is important for the staff to recognize that they will be collecting familiar information; for example, numbers of procedures and surgeries. Spreadsheets, made up before the economic study is implemented, should list all major categories of data collection for economic studies. These can be used to explain the economic study to the CRAs. Probably 90% of the data for the economic study are already being collected for the clinical study—but in a different format. Few additional data items really need to be checked. Unit price information, beyond hospital bills, usually is not obtained at the institutional level. If patient bills are required for the study, they could be sent from the hospital finance center directly to the data center, given that appropriate arrangements have been agreed upon (e.g., confidentiality safeguards).

If resources allow, in-depth, on-site training can occur for CRAs at selected institutions (e.g., institutions where more detailed cost data might be collected). Cooperative group newsletters can provide another vehicle for training. It may be useful to add economic outcomes to the training provided for new study coordinators for cooperative group trials. A video tape is a useful approach for communicating economic study procedures and, if distributed to all participating institutions, can address staff turnover occurring between group meetings.

Small-Scale Pilot Studies

Small-scale pilot studies for data collection forms can be conducted to develop standard data collection forms for economic evaluations and to identify data collection problems. These small-scale pilot studies would address feasibility issues at the institutional level, as well as statistical center issues (e.g., integration into the current database system, extra programming requirements). Logs of time and effort required to use the forms and collect the data should be completed at institutions and statistical centers. Quantification of the extra effort involved, in terms of time and money (e.g., amount of time required to review charts, the source of the data required, and the cost in terms of staff time) can identify ways to reduce the data collection burden and to prepare budgets for future funding requests.

Several questions should be addressed in a pilot study. An assessment of the acceptability of the data collection forms can be done to ensure that the forms will be completed. The wording and structure of the questions in the forms must be assessed to ensure that the desired information is obtained.

Who Will Analyze and Share the Data?

Sharing of Data For most cooperative groups, all data flow through a central statistical office. There are many safeguards to prevent the establishment of "shadow" databases. The clinical trials group may insist that all data remain at the statistical center, with output required by the economics group generated by the central office. However, this may cause economic investigators some concern. With the economics expertise in a separate environment, biostatisticians at the statistical center probably do not have an analysis plan appropriate for economic outcomes or have access to the full extent of available related data. Even in this context, however, most statistical centers will prefer maintaining the database for the trial. One solution is to allow for substantial input from the economists regarding data analysis. In these collaborative relationships, therefore, communication between economic investigators and group biostatisticians should occur early in the design of a trial to address appropriate statistical methods for the cost data and the use of clinical data for the cost analyses.

Statistical center personnel often collaborate with outside economists, and some arrangements for data sharing may already have been discussed. Also, there may be a person who has the appropriate expertise for these analyses at the coordinating center, and there will be no need to split up the information. It may well be the case that the economic investigators will need all of the data to integrate resource consumption for patients with specific clinical practice outcomes. If these arrangements can be agreed upon, it is important to establish some mechanism to make sure that both groups are given continuous and consistent data. Before the clinical study begins, a decision should be made concerning the amount and type of data that will be given to the economic investigators and the publications rules for the economic study. Decisions need to be specified clearly and within the constraints established for that cooperative group.

If a decision is made to have economic data analyzed elsewhere, most cooperative groups will prefer to have all data sent to the statistical or coordinating center for forwarding to the economic investigators. If data are to be submitted directly to an economics center, the procedures for this separate submission should be clearly stated in the protocol.

Data Analysis Issues Economic analysts should remain blinded until the last step in the analysis. This will avoid implicit conflict-of-interest issues and maintain the integrity of the overall trial. If data analysis involves the clinical data, the analysis ideally should not be done until the data are mature. Any use of major clinical endpoints will require waiting until data are mature. These decisions will be made by the data monitoring committee established for the particular trial or by arrangements established during the design phase of the trial.

Intergroup Collaboration Issues Specific to Cost Studies

It is generally unworkable to have one cooperative group execute and coordinate an ancillary study that is being coordinated by a different cooperative group because of the complications associated with transfer of large amounts of data. Therefore, it is reasonable to expect that cost studies will usually need to be executed and coordinated by the group that is coordinating the therapeutic trial. There are, of course, potential difficulties associated with this policy:

• negotiations of "ownership of economic data";
  
• no salary support for economists;
  
• noncoordinating group often does not receive appropriate credit (i.e., publications as primary author for economic study results); and
  
• awkward when the participating group may have more expertise in the area (quality of life, economics) than the coordinating group.
  

There are ways to encourage collaboration:

• the use of outside or separate funding (grants) to help support various areas of the trial;
  
• subcontracting of specific, external (to the group) staff to obtain required expertise;
  
• development of a liaison committee of economic experts among cooperative groups;
  
• attendance at each other's group meetings;
  
• individuals that focus on specific sites (i.e., breast or lung) can attend those committees at other cooperative group meetings to monitor upcoming trials that might be candidates for an economic study;
  
• the standardization of data collection instruments for economic protocols across groups, whenever possible; and
  
• it is recommended that discussion about an economic component including how appropriate, complicated, and extensive it is should be part of the concept review of an intergroup trial.
  

Quality Control Quality control issues become more challenging in the intergroup trial context because of the variation in procedures employed by the different groups. Quality-control procedures suggested for single group trials are equally relevant to the intergroup context. In particular, training at the meetings held by each group would be very important to standardize data collection procedures. Regular communication between intergroup staff associated with the cost study would help identify problems with forms and procedures and allow for improvements in procedures.

Intergroup Pilot Test Clinical and economic investigators across cooperative groups interested in a specific protocol could identify clinical research areas with important economic issues and identify trials planned in those areas; the cooperation of the Cooperative Group Chairs Committee would assist this process. Pilot work is particularly important for intergroup trials, given existing variations in procedures. Each group could evaluate the acceptability of the data collection forms, particularly with respect to access to required information and the time required to collect the information. Quantification of the extra effort involved, in terms of time and money (e.g., amount of time required to review charts, the source of the data required, and the cost in terms of staff time) could document whether or not intergroup economic studies were feasible. The intergroup pilot study could identify ways to reduce the data collection burden and to prepare budgets for future funding requests.

Data Issues

Information-Sharing The same questions addressed for single-group economic studies involving outside economic expertise apply to intergroup studies. Who will have access to cost (and clinical) data? What is the process for sharing data, and how will the information be copied and distributed to others? Which data are relevant to the analysis of economic and clinical parameters?

How Will Economic Studies Be Funded in Cooperative Group Trials?

The conduct of economic studies requires expenditure of considerable staff resources, both at the coordinating center and at individual institutions. Because these studies are new, case report forms, data collection procedures, data base preparation, and data analysis all require considerable start-up effort. Economic outcome data must be accorded the same importance as clinical data, meaning that, if collected, they are subject to the same strict quality control procedures that drive collection of clinical endpoint data. Funding is required to support this effort. Currently, we recognize the importance of including economic outcomes in selected clinical trials. Introduction of quality-of-life outcomes to cooperative groups is a case in point, where substantial central staff time for quality control and monitoring has been necessary (93). However, identifying funding sources that fit the specific timing and procedural constraints of cooperative group research is a challenge.

To date, several approaches to funding economic studies have been tried in cooperative group trials. Pharmaceutical industry funding has supported several economic studies in the cooperative groups: CALGB 9411 (an acute lymphocyte leukemia trial) and SWOG 9509 (an advanced-stage non-small-cell lung cancer trial) are examples.

Third-party payer examples are less common, but one currently in place supports an intergroup economics investigation in a metastatic breast cancer trial by U.S. Healthcare. The extent to which other large managed care organizations can be convinced to support these studies is yet to be determined.

The NCI has two potential mechanisms for supporting economic studies: direct grants, and credits based on accrual to either treatment or cancer control trials. Through a request for applications, NCI is supporting trials of minimally invasive surgery that incorporate an economic component. An example is an intergroup trial (INT 0146) of open colectomy versus laparoscopic-assisted colectomy for colon cancer. Pursuing the RO1 mechanism involves delays between the time applications are submitted and funds are awarded. The cooperative group will not be able to delay activation of a trial while funding for the economic component is being sought.

Cancer control and treatment credits represent another potential avenue for funding. Treatment credits translate into data management support based on accrual effort during the previous grant cycle. Cancer control credits are assigned to cancer prevention and control clinical trials following review and approval by the Division of Cancer Prevention (DCP) Cancer Control Protocol Review Committee (CCPRC). The process is analogous to the review of treatment protocols in the Cancer Treatment Evaluation Program.

Credit is a measure of the data management burden needed to implement an intervention in a clinical trial. Most treatment clinical trials receive one credit/accrual. Credits for cancer prevention and control clinical trials vary depending on the nature of the intervention, the amount of data collection, and the length of follow-up. Community Clinical Oncology Programs (CCOPs) receive budgets based in part on their projected accrual as measured in treatment and cancer control credits. Cooperative groups serving as CCOP research bases receive "pass-through" finding for members and affiliates that accrue to cancer control clinical trials.

To date, economic evaluation in cooperative group treatment studies have not received cancer control credits. Quality-of-life studies have received cancer control credit when they were stand-alone protocols or when the outcomes have been incorporated into treatment trials. Economic outcomes could follow the same model, but this will require a reassessment of DCPC policy. Another possibility is award of additional treatment credits for registrations to companion economic trials or treatment trials with an economic outcome.

As can be seen by this brief summary, routine funding mechanisms are not available to support economic studies in cooperative group trials. Access to stable funding for this research is a first step in our efforts to encourage cooperative groups to incorporate economic outcomes data in cancer clinical trials.

This section addresses specific issues related to the implementation of economic evaluationswithin the NCI cooperative group setting. The NCI cooperative groups are unique and specialized research institutions with well-established protocols for collection of clinical data. However, economic evaluation is a new endpoint within cancer clinical trials for which investigators may have to modify their study procedures. Given the need for change, training and communication become key factors for success in the implementation of economic evaluation in the cooperative group setting. Finally, given the current structure of funding for NCI clinical trials, economic evaluation often will require external funding arrangements, including peer-reviewed grants or industry support.

	Conclusion

Top Introduction Part I: Economic Analysis... Part II: Planning an... Part III: Implementing an... Part IV: Designing an... Part V: Statistical Issues Part VI: Economic Evaluation... Conclusion References

 
Economic evaluation of cancer clinical trials is a new and evolving discipline. As outlined in this workbook, implementation of economic evaluation requires a multidisclipinary research team and extensive planning and forethought analogous to the process of designing a study to assess the clinical endpoints of a clinical trial. Economic evaluation requires training of cooperative group personnel as well as training of site personnel including physicians and the clinical research study team. Through this collaboration, economic data can be successfully collected within cancer clinical trials.

	Acknowledgments

 
We thank the many people who participated in this project, including the writing committee, the workshop participants, and outside reviewers at the NCI and other institutions across the country. We also thank Joy Lopez for her assistance in coordinating this project and the Amgen Corporation for its generous support of this activity.

	Notes

 
¹Editor's note: SEER is a set of geographically defined, population-based central tumor registries in the United States, operated by local nonprofit organizations under contract to the National Cancer Institute (NCI). Each registry annually submits its cases to the NCI on a computer tape. These computer tapes are then edited by the NCI and made available for analysis.

²These indices are available from Carolyn S. Donham, Office of the Actuary, Health Care Financing Administration, telephone 410-786-7947.

³For example, the absolute increase in 5-year survival probability, when a proportional hazards model holds, is S^r - S. Here, S is the 5-year survival for control subjects, and r is the experimental control hazard ratio. This absolute survival difference is strongly influenced by S. If the survival distribution for control patients is exponential with a constant hazard rate h and proportional hazards model holds, the difference in mean life length due to treatment is µ(1-r)/r, where µ is the control group's mean life length (1/h). When one wishes to estimate absolute treatment effects for patients different from those enrolled, these effects will depend on prognostic factors.

⁴The more general case is a three-parameter log-normal distribution, Y ~ LN(, ß, ), with location parameter , scale parameter ß, and shape parameter . The expectation is E(Y) = + ß exp(²/2), and the variance, var(Y) = ß² exp(²) [exp(²) - 1]. For the two-parameter log-normal distribution, the location parameter is assumed to be 0.

⁵In limited experience thus far, the proportional hazards assumption has been remarkably well satisfied. A case study of the Cox model in estimating mean and median costs along with methods for checking model assumptions may be obtained from the World Wide Web: http://www.med.virginia.edu/medicine/clinical/hes/heshp4.htm

	References

Top Introduction Part I: Economic Analysis... Part II: Planning an... Part III: Implementing an... Part IV: Designing an... Part V: Statistical Issues Part VI: Economic Evaluation... Conclusion References

 

1 Gold MR, Siegel JE, Russell LB, Weinstein MC, editors. Cost-effectiveness in health and medicine. New York: Oxford University Press, 1996.

2 Drummond MF, O'Brien B, Stoddart GL, Torrance DW. Methods for the economic evaluation of health care programmes: second edition. New York: Oxford University Press.

3 Parker SL, Tong T, Bolden S, Wingo PA. Cancer statistics, 1997.Cancer 1997;47:5-27.

4 Brown ML. The national economic burden of cancer. J Natl Cancer Inst 1990;82:1811-4.[Free Full Text]

5 Rundle RL. Salick pioneers selling cancer care to HMOs. The Wall Street Journal 1996;August 12, 1996, B1-B2.

6 Brown ML, Hodgson TA, Rice DP. Economic impact of cancer in the U.S. In: Schottenfeld D, Fraumeni J, editors. Cancer, Epidemiology, and Prevention. New York: Oxford University Press, 1996.

7 Callahan D. Controlling the costs of health care for the elderly: fair means and foul. N Engl J Med 1996;335:744-6.

8 Levinsky NG. The purpose of advance medical planning: autonomy for patients or limitation of care? N Engl J Med 1996;335:741-3.[Free Full Text]

9 Hillner BE. Potential evaluation of the incremental cost-effectiveness of paclitaxel in advanced non-small-cell lung cancer (Eastern Cooperative Oncology Group 5592). Monogr Natl Cancer Inst 1995;19:65-7.

10 American Society of Clinical Oncology. Recommended breast cancer surveillance guidelines. ASCO Breast Cancer Surveillance Expert Panel. J Clin Oncol 1997;15:2149-56.[Abstract/Free Full Text]

11 American Society of Clinical Oncology. Clinical practice guidelines for the use of tumor markers in breast and colorectal cancer. ASCO Tumor Marker Expert Panel. J Clin Oncol 1996;14:2843-77.[Abstract/Free Full Text]

12 American Society of Clinical Oncology. Recommendations for the use of hematopoietic CSFs: evidence-based clinical practice guidelines. ASCO Ad Hoc Colony-Stimulating Factor Guidelines Expert Panel. J Clin Oncol 1994;12:2471-508.[Abstract/Free Full Text]

13 Peters WP, Ross M, Vredenburgh JJ, Hussein A, Rubin P, Dukelow K, et al. The use of intensive clinic support to permit outpatient autologous bone marrow transplantation for breast cancer. Semin Oncol 1994;21(4 suppl 7):25-31.

14 Bennett CL, Westerman IL. Economic analysis during phase III clinical trials: who, what, when, where, and why? Oncology 1995;9(suppl):1-7.

15 Hillner BE, Kirkwood JM, Atkins MB, Atkins ER, Johnson ER, Smith TJ. Economic analysis of adjuvant interferon alfa-2b in high-risk melanoma based on projections from Eastern Cooperative Oncology Group 1684. J Clin Oncol 1995;15:2351-8.[Abstract/Free Full Text]

16 Bochner F, Martin ED, Burgess ND, Somogyi AA, Misan GM. How can hospitals ration drugs? Drug rationing in a teaching hospital: a method to assign priorities. Br Med J 1994;308:901-5.[Free Full Text]

17 Smith TJ, Bodurtha JN. Ethics in oncology: balancing interests of the patient, oncologist, and society. J Clin Oncol 1995;13:2464-70.[Abstract/Free Full Text]

18 Eisenberg JM. Clinical economics: a guide to the economic analysis of clinical practices. JAMA 1989;262:2879-86.[Abstract/Free Full Text]

19 Bennett CL, Matchar D, McCrory D, McLeod DG, Crawford ED, Hillner BE. Cost-effective models for flutamide for prostate carcinoma patients: are they helpful to policy makers? Cancer 1996;77:1854-61.[CrossRef][Web of Science][Medline]

20 Hillner BE, Smith TJ, Desch CE. Efficacy and cost-effectiveness of autologous bone marrow transplantation in metastatic breast cancer: estimates using decision analysis while awaiting clinical trial results. JAMA 1992;267:2055-62.[Abstract/Free Full Text]

21 Davies E, Clarke C, Hopkins A. Malignant cerebral glioma—I. Survival, disability, and morbidity after radiotherapy. Br Med J 1996;313:1507-12.[Abstract/Free Full Text]

22 Davies E, Clarke C, Hopkins A. Malignant cerebral glioma—II. Perspectives of patients and relatives on the value of radiotherapy. Br Med J 1996;313:1512-6.[Abstract/Free Full Text]

23 Drummond MF. Survey of cost-effectiveness and cost-benefit analyses in industrialized countries. World Health Statistics Quarterly 1985;38:383-401.

24 Drummond MF, O'Brien B. Economic analysis alongside clinical trials: practical considerations. The Economics Workgroup. J Rheumatol 1995;22:1418-9.[Web of Science][Medline]

25 Schulman KA, Dorsainvil D, Yabroff KR, DiCesare J, Feltzer J, Glick HA, et al. Prospective economic evaluation of rhGM-CSF/rhIL-3 bone marrow transplant for leukemia and lymphoma (manuscript in preparation).

26 Lerman C, Narod S, Schulman KA, Hughes C, Gomez-Caminero A, Bonney G, et al. BRCA1 testing in families with hereditary breast-ovarian cancer: a prospective study of patient decision-making and outcomes. JAMA 1996;275:1885-92.[Abstract/Free Full Text]

27 Haioun C, Lepage E, Gisselbrecht C, Coiffier B, Bosly A, Tilly H, et al. Comparison of autologous bone marrow transplantation with sequential chemotherapy for intermediate-grade and high-grade non-Hodgkin's lymphoma in first complete remission: a study of 464 patients. J Clin Oncol 1994;12:2543-51.[Abstract/Free Full Text]

28 Peters WP, Ross M, Vredenburgh JJ, Meisenberg B, Marks LB, Winer E, et al. High dose chemotherapy and autologous bone marrow support as consolidation after standard-dose adjuvant therapy for high-risk primary breast cancer. J Clin Oncol 1993;11:1132-43.[Abstract/Free Full Text]

29 Feuer EJ, Frey CM, Brawley OW, Nayfield SG, Cunningham JB, Geller NL, et al. After a treatment breakthrough: a comparison of trial and population-based data for advanced testicular cancer. J Clin Oncol 1994;12:368-77.[Abstract]

30 Goldhirsch A, Wood WC, Senn HJ, Glick JH, Gelber RD. Meeting highlights: international consensus panel on the treatment of primary breast cancer. J Natl Cancer Inst 1995;87:1441-5.[Free Full Text]

31 Drummond MF, Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. Br Med J 1996;313:275-83.[Free Full Text]

32 Luce BR, Manning WG, Siegel JE, Lipscomb J. Estimating costs in cost-effectiveness analysis. In: Gold MR, Siegel JE, Russell LB, Weinstein MC, editors. Cost-effectiveness in Health and Medicine. New York: Oxford University Press, 1996.

33 Schulman KA, Glick HA. Economic evaluation of high dose chemotherapy and BMT for patients with stage 2 and stage 3 breast cancer with greater than ten positive nodes at the time of surgical excision. J Natl Cancer Inst 1995;19:45-50.

34 Olivotto IA, Bajdik CD, Plenderleith IH, Coppin CM, Gelmon KA, Jackson SM, et al. Adjuvant systemic therapy and survival after breast cancer. N Engl J Med 1994;330:805-10[Abstract/Free Full Text]

35 Mauskopf JK, Schulman K, Bell L, Glick H. A strategy for collecting pharmacoeconomic data during phase II/III clinical trials. PharmacoEconomics 1996;9:264-77.[Web of Science][Medline]

36 Kukull WA, Koepsell TB, Conrad DA, Immanuel V, Prodzinski J, Franz C. Rapid estimation of hospitalization charges from a brief medical record review: evaluation of a multivariate prediction model. Med Care 1986;24:961-6.[CrossRef][Web of Science][Medline]

37 Nefcy P. Challenge for health care: information for managing costs, utilization and quality. In: QQPHS 1992 Conference Proceedings. Atlanta: Institute of Industrial Engineers, 1992.

38 Hsiao WC, Braun P, Dunn D, Becker ER. Resource-based relative values: an overview. JAMA 1988;260:2347-53[Abstract/Free Full Text]

39 Federal Register. FR 58. Washington (DC): National Archives and Records Administration, 1993.

40 Bennett CL, Cvitanic M, Pascal A. The costs of AIDS in Los Angeles. J Acq Immune Defic Syndr 1991;4:197-203.

41 Edwards WS, Berlin M. National Medical Expenditure Survey: Questionnaires and Data Collection Methods for the Household Survey and the Survey of American Indians and Alaska Natives, Methods 2. DHHS Publ No. (PHS) 89-3450. Rockville (MD): DHHS, 1989.

42 Finkler S. The distinction between cost and charges. Ann Intern Med 1982;96:102-9.

43 Finkler SA. Cost Accounting for Health Care Organizations: Concepts and Applications. Gaithersburg (MD): Aspen Publishers, 1994.

44 UB-92 Editor: A Guide to Medicare Billing. Falconer C, editor. Reston (VA): St. Anthony, 1994.

45 Red Book 1995. Drug topics. Montvale (NJ): Medical Economics Data, 1995.

46 Potosky AL, Riley GF, Lubitz JD, Mentnech RM, Kessler LG. Potential for cancer related health services research using a linked Medicare-tumor registry database. Med Care 1993;31:732-48.[Web of Science][Medline]

47 Coyle D. Statistical analysis in pharmacoeconomic studies: a review of current issues and standards. PharmacoEconomics 1996;9:506-16.[Web of Science][Medline]

48 Dudley RA, Harrell FE Jr, Smith LR, Mark DB, Califf RM, Pryor DB, et al. Comparison of analytic models for estimating the effect of clinical factors on the cost of coronary artery bypass graft surgery. J Clin Epidemiol 1993;46:261-71.[CrossRef][Web of Science][Medline]

49 Zhou XH, Gao S. Confidence intervals for log-normal mean. Stat Med 1997;16:783-90.[CrossRef][Web of Science][Medline]

50 Zhou XH, Gao S, Hui SL. Methods for comparing the means of two independent log-normal samples. Biometrics. In press.

51 Edgington ES. Randomization Tests. New York: Marcel Dekker, 1986.

52 Efron B, Tibshirani RJ. An Introduction to Bootstrap. New York: Chapman and Hall, 1993.

53 Robinson LD, Jewell NP. Some surprising results about covariate adjustment in logistic regression models. Inter Stat Rev 1993;59:227-40.

54 Schoenfeld DA. Sample size formulae for the proportional hazards regression model. Biometrics 1983;39:499-503.[CrossRef][Web of Science][Medline]

55 Anderson GL, Fleming TR. Model misspecification in proportional hazards regression. Biometrika 1995;82:527-41.[Abstract/Free Full Text]

56 Ford IJ, Norrie J, Ahmadi S. Model inconsistency, illustrated by the Cox proportional hazards model. Stat Med 1995;14:735-46.[Web of Science][Medline]

57 Akazawa KT, Nakamura T, Palesch Y. Power of log-rank test and Cox regression model in clinical trials with heterogeneous samples. Stat Med 1997;16:583-97.[CrossRef][Web of Science][Medline]

58 Rutten-van Moken MP, van Doorslaer EK, van Vliet RC. Statistical analysis of cost outcomes in a randomized controlled clinical trial. Health Economics 1994;3:333-45.[Web of Science][Medline]

59 Duan N. Smearing estimate: a nonparametric retransformation method. J Am Stat Assoc 1983;78:605-10.[CrossRef][Web of Science]

60 Cox DR, Oakes D. Analysis of survival data. London: Chapman and Hall, 1984.

61 Lin DY, Wei LJ. The robust inference for the Cox proportional hazards model. J Am Stat Assoc 1989;84:1074-8.[CrossRef][Web of Science]

62 Fenn P, McGuire A, Backhouse M, Jones D. Modeling programme costs in economic evaluation. J Health Economics 1996;15:115-25.[CrossRef][Web of Science][Medline]

63 Smith TJ, Desch CE, Hillner BE. Ways to reduce the cost of oncology care without compromising the quality. Cancer Invest 1994;12:257-64.[Web of Science][Medline]

64 Harrell FE Jr, Lee KL, DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996;15:361-87.[CrossRef][Web of Science][Medline]

65 Etzioni RD, Urban N, Baker M. Estimating the costs attributable to a disease with application to ovarian cancer. J Clin Epidemiol 1996;49:95-103.[CrossRef][Web of Science][Medline]

66 Lin DY, Feuer EJ, Etzioni R, Wax Y. Estimating medical costs from incomplete follow-up data. Biometrics. In press.

67 Riley GF, Potosky AL, Lubitz JD, Kessler LG. Medicare payments from diagnosis to death for elderly cancer patients by stage at diagnosis. Med Care 1995;33:828-41.[CrossRef][Web of Science][Medline]

68 Etzioni RD, Feuer RJ, Sullivan SD, Lin D, Hu C, Ramsey S. On the use of survival analysis techniques to estimate medical care costs. Working Paper, Fred Hutchison Cancer Research Center, Seattle, 1997.

69 Cole BF, Gelber RD, Anderson KM. Parametric approaches to quality-adjusted survival analysis. Biometrics 1994;50:621-31.[CrossRef][Web of Science][Medline]

70 Cole BF, Gelber RD, Kirkwood JM, Goldhirsch A, Barylak E, Borden E. Quality-of-life-adjusted survival analysis of interferon alfa-2b adjuvant treatment of high-risk resected cutaneous melanoma: an Eastern Cooperative Oncology Group study. J Clin Oncol 1996;14:2666-73.[Abstract/Free Full Text]

71 Draper D. Assessment and propagation of model uncertainty. J R Stat Soc 1995;57:45-97.

72 Little T, Rubin D. Statistical analysis with missing data. New York: John Wiley & Sons, 1987.

73 Rubin DB. Multiple imputations for nonresponse in surveys. New Work: John Wiley & Sons, 1987.

74 Rubin D, Shenker N. Multiple imputation in health-care data bases: an overview and some applications. Stat Med 1991;10:585-98.[Web of Science][Medline]

75 Lavori P, Dawson R, Shera D. A multiple imputation strategy for clinical trials with truncation of patient data. Stat Med 1995;14:1913-25.[Web of Science][Medline]

76 Efron B. Missing data, imputation, and the bootstrap. J Am Stat Assoc 1994;89:463-79.[CrossRef][Web of Science]

77 McGuigan KA, Ellickson PE, Hays RD, Bell RM. Adjusting for attrition in school-based samples: bias, precision, and cost trade-offs of three methods. Eval Rev 1997;21:554-67.[Abstract/Free Full Text]

78 Karrison TG. Restricted mean life with adjustment for covariates. J Am Stat Assoc 1987;82:1169-76.[CrossRef]

79 Karrison TG. Use of Irwin's restricted mean as an index for comparing survival in different treatment groups—Interpretation and power considerations. Controlled Clinical Trials 1997;18:151-67.[CrossRef][Web of Science][Medline]

80 Mark DB, Hlatky MA, Califf RM, Naylor CD, Lee KL, Armstrong PW, et al. Cost effectiveness of thrombolytic therapy with tissue plasminogen activator as compared with streptokinase for acute myocardial infarction. N Engl J Med 1995;332:1418-24.[Abstract/Free Full Text]

81 Manning WG, Fryback DG, Weinstein MC. Reflecting uncertainty in cost-effectiveness analysis. In: Gold MR, Russell LB, Siegel JA, Weinstein MC, editors. Cost-effectiveness in health and medicine. New York: Oxford University Press, 1996.

82 Polsky D, Glick HA, Willke R, Schulman K. Confidence intervals for cost-effectiveness ratios: a comparison of four methods. Health Economics 1997;6:243-52.[CrossRef][Web of Science][Medline]

83 Willan AR, O'Brien BJ. Confidence intervals for cost-effectiveness ratios: an application of Fieller's theorem. Health Economics 1996;5:297-305.[CrossRef][Web of Science][Medline]

84 Hlatky MA, Boothroyd DB, Johnstone IM, Marcus FI, Hahn EH, Hartz V, et al. Long-term cost-effectiveness of alternative management strategies for patients with life-threatening ventricular arrhythmias. J Clin Epidemiol 1997;50:185-93.[CrossRef][Web of Science][Medline]

85 Chaudhary X, Mohammad A, Stearns SC. Estimating confidence intervals for cost-effectiveness ratios: an example from a randomized trial. Stat Med 1996;15:1447-58.[CrossRef][Web of Science][Medline]

86 Stinnett AA. Adjusting for bias in C/E ratio estimates. Health Economics 1996;5:470-2.[CrossRef][Web of Science][Medline]

87 Hornberger JC, Brown BW, Halpern J Jr. Designing a cost-effective clinical trial. Stat Med 1995;14:2249-59[Web of Science][Medline]

88 Borenstein, M. The case for confidence intervals in controlled trials. Controlled Clin Trials 1994;15:411-28.[CrossRef][Web of Science][Medline]

89 Borenstein M. Planning for precision in survival studies. J Clin Epidemiol 1994;47:1277-85.[CrossRef][Web of Science][Medline]

90 Willan AR, O'Brien BJ. Confidence intervals for cost-effectiveness ratios: an application of Fieller's Theorem. Health Economics 1996;5:297-305.

91 Spiegelhalter DJ, Freedman LS, Parmar MK. Bayesian approaches to randomized trials. J R Stat Soc 1994;157:357-416.[CrossRef]

92 Sensenig AL, Heffler SK, Donham CD. Health care indicators: hospital, employment, and price indicators for the health care industry—third quarter 1995. Health Care Financing Review 1996;17:269-79.[Web of Science][Medline]

93 Moinpour CM. Costs of quality-of-life research in Southwest Oncology Group trials. J Natl Cancer Inst 1996;20:11-6.

94 Smith TJ, Desch CE, Hillner BE. Efficacy and cost-effectiveness in oncology: rational allocation of cancer care resources. J Natl Cancer Inst 1996;85:1460-71.[Free Full Text]

95 Smith TJ, Hillner BE, Schmitz N, Linch DC, Dreger P, Goldstone AH, et al. Economic analysis of a randomized clinical trial to compare filgrastim-mobilized peripheral blood progenitor cell transplantation and autologous bone marrow transplantation in patients with Hodgkin's and non-Hodgkin's lymphoma. J Clin Oncol 1997;15:5-10.[Abstract/Free Full Text]

96 Elit LM, Gafni A, Levine MN. Economic and policy implications of adopting paclitaxel as first-line therapy for advanced ovarian cancer: an Ontario perspective. J Clin Oncol 1997;15:632-9.[Abstract/Free Full Text]

CiteULike    Connotea    Del.icio.us    What's this?

This article has been cited by other articles:

				R. Wells, L. Stensland, and J. Vieira The Human Cytomegalovirus UL112-113 Locus Can Activate the Full Kaposi's Sarcoma-Associated Herpesvirus Lytic Replication Cycle J. Virol., May 1, 2009; 83(9): 4695 - 4699. [Abstract] [Full Text] [PDF]

				H. Feng, J. L. Taylor, P. V. Benos, R. Newton, K. Waddell, S. B. Lucas, Y. Chang, and P. S. Moore Human Transcriptome Subtraction by Using Short Sequence Tags To Search for Tumor Viruses in Conjunctival Carcinoma J. Virol., October 15, 2007; 81(20): 11332 - 11340. [Abstract] [Full Text] [PDF]

				J. Douglas, B. Dutia, S. Rhind, J. P. Stewart, and S. J. Talbot Expression in a Recombinant Murid Herpesvirus 4 Reveals the In Vivo Transforming Potential of the K1 Open Reading Frame of Kaposi's Sarcoma-Associated Herpesvirus J. Virol., August 15, 2004; 78(16): 8878 - 8884. [Abstract] [Full Text] [PDF]

This Article Extract FREE Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to citation manager Request Permissions Google Scholar Search for Related Content PubMed PubMed Citation Social Bookmarking          What's this?

Online ISSN 1745-6614 - Print ISSN 1052-6773

Copyright © 2008 Oxford University Press

Oxford Journals Oxford University Press

• Site Map
• Privacy Policy
• Frequently Asked Questions

Other Oxford University Press sites: Oxford University Press Oxford Journals China Oxford Journals Japan American National Biography Booksellers' Information Service Children's Fiction and Poetry Children's Reference Corporate & Special Sales Dictionaries Dictionary of National Biography Digital Reference English Language Teaching Higher Education Textbooks Humanities International Education Unit Law Medicine Music Online Products Oxford English Dictionary Reference Rights and Permissions Science School Books Social Sciences Very Short Introductions World's Classics