© 2004 by Oxford University Press
2004 © Oxford University Press
Article |
Cancer Outcomes Research and the Arenas of Application
J. Lipscomb (Rollins School of Public Health, Emory University, Atlanta, GA, and formerly National Cancer Institute, Bethesda, MD); M. S. Donaldson (National Cancer Institute); R. A. Hiatt (Division of Prevention and Surveillance, UCSF Comprehensive Cancer Center, University of California at San Francisco, San Francisco, CA)
Correspondence to: Molla S. Donaldson, DrPH, Senior Scientist for Quality of Care Research and Policy, Outcomes Research Branch, National Cancer Institute, National Institutes of Health, 6130 Executive Blvd., Rm. 4028, Bethesda, MD 20892 (e-mail: molla.donaldson{at}nih.gov).
 |
INTRODUCTION |
|---|
 TopNotes Introduction Defining Cancer Outcomes...The Arenas of ApplicationUnderstanding and Informing...References |
|---|
In 2004, an estimated 1.5 million persons in the United States will be diagnosed with cancer; about 560 000 will die from cancer; and more than 9.5 million will be undergoing curative treatment, coping with progressive disease, or living free of cancer after successful therapy (1). But many of these survivors still feel the aftershocks and downstream side effects arising from diagnosis and treatment, and many are fearful of recurrence. Substantial progress in reducing the suffering and death caused by cancer is being pursued by the National Cancer Institute (NCI) and cancer agencies and organizations worldwide through a variety of initiatives, programs, and projects. At the NCI, these efforts emphasize the joint importance of basic and applied scientific discovery, the development and testing of promising interventions, and the delivery of quality care to prevent, detect, and treat cancer and to improve the length and quality of life of cancer survivors (2).
It is therefore vital for decision makers, at all levels, to have a firm understanding of just how effectively the fruits of discovery and development are being applied to enhance cancer care delivery in ways that reduce suffering and death. The scientific pursuit of such decision-relevant information is the central business of cancer outcomes research.
The overall aim of this Monograph is to provide an empirically grounded review and evaluation of the peer-reviewed literature in cancer outcomes research. The intent is to identify both important recent contributions and the challenges that remain in bringing scientifically sound information to bear in cancer care decision making.
|
DEFINING CANCER OUTCOMES RESEARCH |
|---|
|
TopNotesIntroductionDefining Cancer Outcomes...The Arenas of ApplicationUnderstanding and Informing...References |
|---|
There is, as yet, no consensus definition of outcomes research, much less cancer outcomes research. But recent statements by the U.S. Agency for Healthcare Research and Quality (AHRQ) and the NCI convey a common sense of purpose and scope of application. According to AHRQ, "... outcomes research seeks to understand the end results of particular health care practices and interventions. End results include effects that people experience and care about, such as change in the ability to function. In particular, for individuals with chronic conditions—where cure is not always possible—end results include quality of life as well as mortality" (3,4).
At NCI, outcomes research "describes, interprets, and predicts the impact of various influences, especially (but not exclusively) interventions, on `final' endpoints that matter to decision makers" (5). These decision makers may include patients, families, individuals at risk for cancer, providers, private and public payers and purchasers of cancer care, regulatory agencies, health care accrediting organizations, and society at large. In cancer, final endpoints (outcomes) include such traditional and important biomedical outcomes as survival and disease-free survival, but also health-related quality of life (HRQOL), perceptions of and satisfaction with health care, and economic burden. Final outcomes are distinguished from "intermediate" outcomes (e.g., whether the individual was screened for cancer, or quit smoking, or received appropriate adjuvant therapy following cancer surgery) and "clinical" outcomes (e.g., whether the patient's tumor shrank or disappeared, or whether the tumor recurred). Although such measures are frequently pivotal in assessing the proximate success of particular interventions, the ultimate test is, or should be, whether improvement in the clinical or intermediate outcome predicts success in improving final outcomes.1
Prominent biomedical outcomes like survival and disease-free survival have long been employed both in clinical investigations and in widely circulated reports on progress against the cancer burden at the population level. Although pockets of controversy remain about matters of definition and measurement (for example, the ongoing discussion about cause-specific versus relative survival measures), most researchers and many in the lay public would likely believe they have a basic understanding of what these particular final outcomes mean, if not also how to construct them.2 On the other hand, there remains much debate about, and wide variations in the use of patient-reported outcomes like HRQOL and measures of patient satisfaction and economic burden. Consequently, this Monograph focuses largely on the actual, and potential, contributions of such patient-reported outcomes on decision making along the cancer continuum, which includes prevention, early detection, diagnosis, treatment, life after a cancer diagnosis (survivorship), and end-of-life care. As will be seen, the bulk of the existing outcomes research literature is devoted to screening, diagnosis, and treatment applications.
For outcomes research to achieve its potential to enhance care delivery, there are at least three prerequisites: 1) technically sound and decision relevant outcome measures; 2) persuasive evidence about the impact of interventions on those outcomes; 3) the capacity, determination, and ingenuity to translate findings into useful information for decision making (5,6).
There are clear interconnections between outcomes research and health services research, though there is no consensus yet about the precise relationship (owing in part to varying definitions of both enterprises). The National Library of Medicine MESH heading "health services research" connotes "the integration of epidemiologic, sociological, economic, and other analytic sciences in the study of health services..." (7). To date, there is no MESH heading for outcomes research; the closest option appears to be "outcome assessment (health care)," although this appears to encompass not only what we term final outcomes but also, "... abnormal states (such as elevated blood pressure)" (7). In their detailed review of the history and role of outcomes research in oncology, Lee et al. (7) appear to settle upon a definition that does not differ significantly from that of health services research, as spelled out by the Association for Health Services Research (AHSR) in the late 1990s.3 In the introduction to a volume of papers discussing future directions in cancer outcomes research, Ramsey (10) concluded that it may be premature to impose a consensus definition "until we know what matters to whom"4.
To pursue its goal of improving the quality of cancer care by strengthening the scientific basis for public and private decision making, NCI has created a research and applications agenda that is supported by a range of outcomes research and health services research projects, data resources, and collaborations that extend well beyond the NCI itself (15). These various efforts are discussed in the Monograph's final article. In designing its agenda, NCI has been guided by the Institute of Medicine's well-known definition, "Quality of care is the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge" (16). In this definition, desired means desired by individuals receiving interventions—underscoring the idea of patient-centered outcomes. NCI's comparatively broad conceptualization of outcomes research, and its recognition of the interplay between outcomes research and HSR, underscores the idea that increasing the likelihood of desired health outcomes requires sustained and well-coordinated progress in several areas. Specifically, we should measure outcomes that matter; investigate the impact of interventions on these outcomes; use the findings to improve quality of cancer care in the community; and monitor progress over time to identify successes, shortcomings, knowledge gaps, and new opportunities for research and application.
As Donabedian (17) emphasized years ago, quality of care may be indexed, alternatively, by structural variables (the quantity and quality of inputs), process variables (what is done, or not done, to the patient), or outcomes (by which he meant final outcomes, as defined here). But in the end, the validity of structural or process variables as quality measures hinges on the strength of the evidence that they are predictive of outcomes that matter. As AHRQ has noted, "By linking the care people get to the outcomes they experience, outcomes research has become the key to developing better ways to monitor and improve the quality of care" (3).
With these functional definitions and policy aims in mind, we turn now to the specific framework employed in this Monograph for categorizing and characterizing the applications of cancer outcome measures, as seen in the peer-review literature over the decade of the 1990s. This framework provides a functionally useful backdrop for our efforts to present a systematic review and evaluation of a major segment of that literature and inferences about future directions for cancer outcomes research.
|
THE ARENAS OF APPLICATION |
|---|
|
TopNotesIntroductionDefining Cancer Outcomes...The Arenas of ApplicationUnderstanding and Informing...References |
|---|
The framework adopted here recognizes three broad categories, or arenas, for the application of cancer outcome measures5 (Table 1).
|
Macro. Population surveillance of trends in cancer-related outcomes and progress against the cancer burden—including survival but ideally also capturing selected patient-reported outcomes.
Meso. Descriptive and analytical studies to understand the impact of cancer, patterns of service use, and effects of intervention on cancer-related outcomes. Included are a diverse range of analyses:
- Randomized clinical trials examining intervention efficacy.
- Observational investigations of the effectiveness of interventions in
real-world, community practice and of the burden of cancer on patients,
survivors, and families.
- Patterns-of-care studies that not only examine variations in service
utilization, but also relate these variations to differences in outcomes
across population groups; and studies to monitor the quality of cancer care by
tracking adherence of individuals or populations to consensus recommendations
about appropriate care.
- Clinical modeling, evaluation, and priority-setting analyses, which
typically integrate and synthesize information on outcomes (and related
explanatory variables) from a variety of sources to identify the best
course(s) of action for the cancer-related decision at issue. Included here
are clinical decision modeling analyses to select an optimal intervention for
the population (or subgroup) of interest; cost-effectiveness and cost-benefit
analyses (which may, or may not, employ a decision-model); and studies to
evaluate existing cancer intervention programs or to guide the establishment
of new ones.
Micro. Patient-clinician decision making enhanced by the use of patient-reported outcome measures, risk and outcome prediction models, or other tools to improve the quantity and quality of information available at the bedside, in the clinic and office, and in electronic (telephone or computer-assisted) communications.
This Monograph presents 10 invited papers,6 that are distributed across the arenas of application as follows: one macro, seven meso, one micro, and one paper that suggests how to connect the schema in Table 1 to a more general "health outcomes framework" that envisions a creative interplay among applications at all levels. Five of the seven meso papers examine outcomes research applications, in turn, for the four most prevalent adult cancer disease sites (breast, colorectal, lung, and prostate) and one major childhood cancer (acute lymphoblastic leukemia). Two additional meso papers provide critical assessments of the application of HRQOL measures and economic cost measures, respectively, within and across cancer disease sites. The Monograph concludes with a paper by NCI staff scientists reflecting on possible future directions for cancer outcomes research.
What follows is a snapshot of these invited papers within the arenas-of-application framework.
Macro-level. Macro-level studies seek to document variations in important outcomes to assist decision makers in establishing and evaluating policies aimed at benefiting entire populations (20). In particular, Clauser (20) discusses how these studies typically monitor trends over time, trends within or across geopolitical units (e.g., within the United States, across states, across countries), and/or trends among population subgroups (e.g., racial-ethnic differences) in outcomes describing the burden of cancer (e.g., survival rates, functional status measures), or the use of services thought to influence that burden (e.g., cancer screening rates). Such surveillance studies are not sufficient alone for tightly drawn causal inferences about the determinants of variations in population outcomes. But frequently they suggest important hypotheses (e.g., insurance coverage and racial-ethnic factors play distinct but interactive roles in explaining population disparities in patterns of cancer care) that can be examined more rigorously through meso-level investigations. Clauser (20) concludes there is a need to supplement traditional biomedical endpoints, like survival, with patient-reported outcomes, like health-related quality of life, to improve our ability to monitor progress against the suffering, as well as death, caused by cancer.
There is no intent here to imply that cancer surveillance, as it is traditionally and broadly construed, is synonymous with cancer outcomes research. Rather, a range of outcome measures important to decision makers can and should be part of what is being tracked and analyzed by the surveillance enterprise.
Meso-level. Meso-level studies, although diverse in form and specific purpose, have in common the broad aim of understanding (and sometime also predicting) the impact of factors, especially interventions, on outcomes that matter. Typically, these analyses have as the unit of observation the individual cancer patient, survivor, or person at risk. Meso-level studies may serve to test hypotheses suggested by macro-level findings. They may illuminate process-outcome linkages that support micro-level applications (see below). Or they may provide important findings that feed directly into decision making about cancer care insurance coverage and reimbursement; regulation, including drug registration and marketing; and a range of quality improvement activities, including standards setting, clinical guidelines use, and performance measurement.
The five disease-specific papers—Mandelblatt et al. (21), Provenzale and Gray (22), Earle (23), Collins et al. (24), and Pickard et al. (25)—began with a common charge, literature search strategy, and general format for reporting findings. As the work proceeded, however, it became clear there were notable differences across breast, colorectal, lung, prostate, and childhood cancers in the nature of the available relevant literature, how best to search for it, and how to report findings most effectively. Consequently, the papers appearing here have been tailored to accommodate these differences, while maintaining certain important common features.
Each paper builds on and adapts a Medline search strategy we developed in several iterations with the NIH reference library staff. After applying inclusion/exclusion criteria, each paper analyzes the literature covering (at least) the 11-year period from 1990 through 2000. And importantly, each paper focuses largely on applications of patient-centered outcome measures—such as health-related quality of life, patient satisfaction, and economic burden—and the conduct of decision-analytic modeling and economic analyses. Thus, articles that examine only survival-related and other biomedical endpoints were not included.
Although the set of five papers differ in their particular findings, conclusions, and recommendations, certain common themes emerge, including the following:
- Patient-centered outcome measures can provide valuable information for
interpreting the impact of interventions, e.g., indicating the net effect on
patient functioning of treatments that reduce cancer-induced pain while
generating toxic side effects.
- There is little consensus about how to measure health-related quality of
life for any of the five cancer types examined here. Because multiple
competing instruments are used across studies (and sometimes even within a
given study), our capability to carry out meta-analyses and other cross-study
syntheses of findings is compromised. Underlying this absence of consensus
about the most appropriate HRQOL measure is a lack of agreement about the
criteria for selecting a "best" measure. Similarly, there is wide
variability in how the economic costs of cancer and cancer interventions are
defined and computed.
- Compared with HRQOL assessment, many fewer studies measure patient
perceptions of the quality of cancer care or their satisfaction with aspects
of the care process.
- Greater attention should be paid to measuring patient preferences for
outcomes, particularly for those treatment or screening options where HRQOL
effects are significant, complex, and important to decision making.
- There is substantial variability in the scientific quality of studies
examining patient-centered outcomes, particularly in randomized clinical
trials where a biomedical outcome (e.g., short-term cancer-free survival) is
the primary endpoint and thus the focus of data collection and analysis. For
published findings about HRQOL effects to be taken seriously by decision
makers, study sponsors and investigators must measure and analyze these
outcomes with the same scientific rigor as more traditional, biomedical
endpoints.
In a wide-ranging analysis of the use of HRQOL measures across cancer disease sites, Gotay (26) urges greater attention to understanding the performance of selected, high-quality instruments within specific types of applications. For example, better estimates of the covariation between a generic measure like the SF-36 and cancer-specific measures, like the EORTC QLQ-C30 and FACT G, in cancer treatment trials would enhance understanding of the comparative contributions of each measure. Similarly, the ability to determine whether a given HRQOL change score is "clinically meaningful" would be strengthened by the availability of normative data on the instrument's performance in diverse populations.
In contrast to the field of HRQOL assessment, defining and measuring the resource burden of cancer and cancer care poses little conceptual problem (because the idea of economic opportunity cost has been long understood and broadly accepted). Yet, again in contrast to HRQOL, Fryback and Craig (27) report that there are virtually no widely used, standardized instruments vying to become the accepted standard for cost measurement. This has resulted in significant variation in how cancer costs are defined and measured, which greatly reduces the comparability of findings across studies. These authors recommend developing and testing standardized questionnaires and data collection forms for costs.
Micro-level. Micro-level studies use outcome measurement to understand, evaluate, and improve patient-provider decision making. Donaldson (28) examines the emerging literature on the use, acceptability, and effectiveness of patient-centered measures like HRQOL in enhancing communication, providing feedback on the progression of disease and the impact of therapy, and improving the quality of cancer care. Although there have been methodological advances and reports in the literature of successful applications, HRQOL (and patient-centered outcomes generally) are not yet routinely used in oncology practice in the United States. The way forward, Donaldson argues, is not exhortation or top-down requirements that clinicians do HRQOL assessment. Rather, the focus should be on facilitating adoption by improving information technology in the office setting; efficiently embedding HRQOL assessment into the processes of care delivery; and redesigning aspects of cancer care to reduce barriers and improve incentives to assess and use HRQOL data.
The Monograph's invited papers conclude with Erickson's (29) proposal for a "health outcomes framework" to link macro, meso, and micro applications by embracing a core set of HRQOL questionnaires for use at all levels, possibly supplemented at each level by additional survey items. The aim is to improve the interpretability and usefulness of HRQOL within each arena of application, whether the focus is a given disease (cancer) or a set of diseases. The framework could apply, as well, to any cancer outcome measure of interest. The key point is that because the core measures are used in all arenas, the interpretability within any one arena benefits from that broader perspective. In turn, the broad use of such core measures would foster a much better understanding of health-related quality of life across multiple populations and diseases.
|
UNDERSTANDING AND INFORMING CANCER CARE DECISION MAKING: ASCENDING THE OUTCOMES RESEARCH PYRAMID |
|---|
|
TopNotesIntroductionDefining Cancer Outcomes...The Arenas of ApplicationUnderstanding and Informing...References |
|---|
In The Outcomes of Outcomes Research at AHCPR—a compelling report commissioned by what is now the U.S. Agency for Healthcare Research and Quality—a useful framework was introduced for evaluating progress and identifying future research directions (30). The framework takes the form of a four-level pyramid, with the possible impact of research on final outcomes moving progressively closer as one ascends. Specifically, one moves from research that enhances the knowledge base, and the quality of future research (level 1), but does not directly influence medical policies, practices, or final outcomes; to research that does influence practice policies (level 2); to research that influences care delivery (level 3); to research whose influence on patient outcomes in the community is clearly demonstrable (level 4); see Fig. 1.
|
From an analysis of survey responses from AHCPR-supported investigators over the 1989-1997 period, the report concluded that outcomes research studies had been most successful in describing health care delivery processes, developing methodological tools for measuring patient-centered outcomes and economic costs, and identifying future research topics—all level-1 analyses. There were impressive advances in such areas as meta-analysis, clinical decision modeling, and cost-effectiveness analysis. But, the report noted, there were "few examples" demonstrating that research findings had influenced medical policies, practices, or final outcomes. [One prominent level-4 success story was the use of data feedback and continuous quality improvement techniques to achieve a substantial reduction in coronary artery bypass surgery mortality rates (30).]
Such conclusions, as well as the myriad findings reported in this Monograph's papers, underline the importance of developing and supporting a cancer outcomes research agenda that enables us to scale the pyramid. Although Fig. 1 might suggest that such upward advances are linear, they are more likely recursive, interactive, and dynamic. That is, successful efforts at each level will not only influence activity at the next level, but also inform work at that level and at lower level(s). It is frequently observed that improving the quality of care requires attention not only to the underuse of appropriate interventions but also to the misuse and overuse of services (31). Similarly, advancing up the pyramid means promoting evidence-based decision making that is attentive to overuse and underuse, as well as failure to complete an appropriate plan as intended. In the end, the aim is to produce and provide the right information, at the right time and place, to the right decision maker(s) across the arenas of application. In the Monograph's final paper, NCI staff scientists draw upon their experiences and perspectives (as further informed by the 10 papers that follow herein) to provide observations about the current state of the science and possible future directions in cancer outcomes research (32).
Our ascent up the pyramid has clearly begun—but is far from complete. As the Monograph's papers indicate, there has been much progress in recent years toward understanding the impact of interventions on final outcomes, including the patient-centered outcomes emphasized here. However, the literature is thin, and our knowledge more anecdotal than systematized, about exactly how such level-1 information is used (or misused or ignored) in decisions about practice policies (level 2) or the delivery of cancer care (level 3). The challenges in acquiring a deeper understanding of the ways in which information at each level comes to influence decision making, and ultimately the outcomes that matter to decision makers, point to a new "frontier" area for outcomes research. We return to this theme in the Monograph's final paper (32), illustrating with a cancer example how the pyramid has been scaled—and also the challenges in establishing the chain of evidence that links research to improvement in outcomes.
|
NOTES |
|---|
The authors benefited significantly from comments by Rachel Ballard-Barbash, MD, MPH, and Martin L. Brown, PhD.
1 A potentially useful way to conceptualize the idea of final outcomes is to
invoke the economist's traditional model of rational choice: the decision
maker acts so as to maximize "utility" subject to whatever
constraints on behavior are relevant and binding. In symbols, the decision
maker maximizes a function U(H, x) subject to specified
constraints, where H is a vector of health-related outcomes and
x is a vector of all other goods that affect utility (happiness).
Within this stylized framework, H comprises what we have termed final
outcomes, and, likewise, all relevant final outcomes are in H.
Intermediate and clinical outcomes can then be modeled as influencing the
generation of H via the production function H =
f(I, C, Y), where I and C are relevant
intermediate and clinical outcomes, respectively, and Y is a vector
of all other factors influencing final outcomes. Further, the potential
interplay between I and C could be modeled, recognizing that
achieving particular intermediate outcomes influences clinical outcomes. All
this underscores the distinction between final outcomes, which are the
"ends" we seek, and intermediate outcomes, which are means to
those ends. Note also that what we have termed intermediate outcomes are
closely related to what quality-of-care researchers call "process
measures" of quality. For the three intermediate measure examples in the
text, we can construct the corresponding QOC process measures: percent of
eligible population screened for (a particular) cancer, percent of smokers who
quit, and percent of patients eligible for adjuvant therapy following surgery
who elect to receive it and do. 
2 To be sure, there is much ongoing discussion and different viewpoints about
the appropriate role of such prominent surrogate endpoints as rate of tumor
progression in cancer treatment trials. But these are what we would term
clinical, not final, biomedical outcomes.
3 Academy Health, the successor organization to AHSR, now defines health
services research similarly to NLM, and it defines outcomes research in the
same spirit as AHRQ and NCI
(9).
4 Yet another perspective on these matters of definition and function comes
from the NCI itself, where its Outcomes Research Branch (ORB)
(11)
and Health Services and Economics Branch (HSEB)
(12)
operate in parallel and complementary ways within the Applied Research Program
(ARP)
(13)
of the Division of Cancer Control and Population Sciences
(14).
In terms of the three prerequisites for effective outcomes research cited
earlier, the ORB has focused substantially on assessing and improving outcome
measures. HSEB (and ARP more broadly) has initiated a number of projects to
strengthen the evidence base and analyze the impact of interventions on
outcomes. ORB, with support from HSEB and many units and organizations within
and beyond the NCI, has devoted substantial resources to synthesizing and
translating evidence for cancer care decision making.
5 The macro-meso-micro rubric has been previously used by both Sutherland and
Till
(18)
and Osoba
(19),
but in each of these papers the terms are defined somewhat differently than in
Table 1 here.
6 Each of these invited papers was supported through contracts from the
National Cancer Institute and was subsequently subjected to the peer review
process that is standard for all Journal of the National Cancer Institute
Monographs. Two of this paper's authors, Lipscomb and Donaldson, were
designated by the Journal of the National Cancer Institute as the
scientific editors for the Monograph.
|
REFERENCES |
|---|
|
TopNotesIntroductionDefining Cancer Outcomes...The Arenas of ApplicationUnderstanding and Informing...References |
|---|
1 American Cancer Society. Cancer facts & figures, 2004. Atlanta (GA): American Cancer Society; 2004 .
2 National Cancer Institute. Discovery, development, and delivery as a seemless process. In The Nation's investment in cancer research: a plan and budget proposal for fiscal year 2005. NIH Publication No. 03-5446, October 2003, Bethesda, MD, p. 1-2. Available at: http://plan.cancer.gov. [Last accessed April 25, 2004].
3 Agency for Healthcare Research and Quality. Outcomes research fact sheet: what is outcomes research? Available at: http://www.ahrq..gov/clinic/outfact.htm. [Last accessed April 25, 2004].
4 Clancy CM, Eisenberg JM. Outcomes research: measure the end results
of health care. Science 1998; 282: 245
-6.
5 Lipscomb J, Snyder CF. The outcomes of cancer outcomes research. Med Care 2002;40 [supplement]: III-3-III-10.[Medline]
6 Lipscomb J, Donaldson MS. Outcomes research at the National Cancer Institute: measuring, understanding, and improving the outcomes of cancer care. Clin Ther 2003;25: 699 -712.[CrossRef][ISI][Medline]
7 National Library of Medicine. Scope Note (on Health Services Research). Available at: http://www.nlm.nih.gov/cgi/mesh/2004/MB_cgi. [Last accessed April 22, 2004].
8 Lee SJ, Earle CC, Weeks JC. Outcomes research in oncology: history,
conceptual framework, and trends in the literature. J Natl Cancer
Inst 2000;92: 195
-204.
9 Academy Health. Glossary of terms commonly used in health care. Available at: http://www.academyhealth.org/publications/glossary.pdf. [Last accessed April 21, 2004].
10 Ramsey S. What is cancer outcomes research? A diversity of perspectives for an emerging field. Med Care 2002; 40[suppl]: III-1 -III-2.[Medline]
11 National Cancer Institute. Outcomes Research Branch. Available at: http://outcomes.cancer.gov. [Last accessed April 24, 2004].
12 National Cancer Institute. Health Services and Economics Branch. Available at: http://healthservices.cancer.gov. [Last accessed April 24, 2004].
13 National Cancer Institute. Applied Research Program. Available at: http://appliedresearch.cancer.gov. [Last accessed April 24, 2004].
14 National Cancer Institute. Division of Cancer Control and Population Sciences. Available at: http://cancercontrol.cancer.gov. [Last accessed April 24, 2004].
15 National Cancer Institute. Improving the quality of cancer care. In The nation's investment in cancer research: a plan and budget proposal for fiscal year 2005. NIH Publication No. 03-5446, October 2003, Bethesda (MD); 2003, p. 51-7. Available at: http://plan.cancer.gov. [Last accessed April 25, 2004].
16 Institute of Medicine. Medicare: a strategy for quality assurance. K. N. Lohr, editor. Washington (DC): National Academy Press; 1990.
17 Donabedian A. The definition of quality and approaches to its assessment. Ann Arbor (MI): Health Administration Press; 1980 .
18 Sutherland HJ, Till JE. Quality of life assessments and levels of decision making: differentiating the objectives. Qual Life Res 1993; 2: 297 -303.[CrossRef][ISI][Medline]
19 Osoba D. A taxonomy of the uses of health-related quality of life instruments in cancer care and the clinical meaningfulness of the results. Med Care 2002;40: III31 -III38.[Medline]
20 Clauser SB. Use of cancer performance measures in population
health: a macro-level perspective. J Natl Cancer Inst Monogr 2004; 33: 142
-54.
21 Mandelblatt J, Armetta C, Yabroff KR, Liang W, Lawrence W. A
descriptive review of the literature on breast cancer outcomes: 1990-2002. J Natl Cancer Inst Monogr
2004; 33: 8
-44.
22 Provenzale D, Gray RN. Colorectal cancer screening and treatment: a
survey of outcomes research. J Natl Cancer Inst Monogr 2004; 33: 45
-55.
23 Earle CC. Outcomes research in lung cancer. J Natl Cancer
Inst Monogr 2004;33: 56
-77.
24 McNaughton-Collins MM, Walker-Corkery EW, Barry MJ.
Health—related quality of life, satisfaction, and economic outcome
measures in studies of prostate cancer screening and treatment, 1990-2000. J Natl Cancer Inst Monogr
2004; 33: 78
-101.
25 Pickard AS, Topfer LA, Feeny DH. A structured review of studies on
health-related quality of life and economic evaluation in pediatric acute
lymphoblastic leukemia. J Natl Cancer Inst Monogr 2004; 33: 102
-25.
26 Gotay CC. Assessing quality of life in cancer across a spectrum of
applications. J Natl Cancer Inst Monogr 2004; 33: 126
-33.
27 Fryback DG, Craig BM. Measuring economic outcomes in cancer. J Natl Cancer Inst Monogr
2004; 33: 134
-41.
28 Donaldson MS. Taking Stock of health-related quality-of-life
measurement in oncology practice in the United States. J Natl Cancer
Inst Monogr 2004;33: 155
-67.
29 Erickson P. A health outcomes framework for assessing health status
and quality of life: enhanced data for decision making. J Natl Cancer
Inst Monogr 2004;33: 168
-77.
30 Tunis S, Stryer D. The outcomes of outcomes research at AHCPR. Publication No. AHCPR 99-R044, 1999. Available at: http://www.ahrq.gov/clinic/out2res/. [Last accessed on March 29, 2004].
31 Chassin MR, Galvin RW, and the National Roundtable on Health Care
Quality. The urgent need to improve health care quality. Institute of Medicine
National Roundtable on Health Care Quality. JAMA 1998; 280: 1000
-5.
32 Lipscomb J, Donaldson MS, Arora NK, Brown ML, Clauser SB, Potosky
AL, et al. Cancer outcomes research. J Natl Cancer Inst Monogr 2004; 33: 178
-97.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  J. Lipscomb Estimating the Cost of Cancer Care in the United States: A Work Very Much in Progress J Natl Cancer Inst, May 7, 2008; 100(9): 607 - 610. [Full Text] [PDF]
|
|
|
|
 |
|
 J. Lipscomb, C. C. Gotay, and C. F. Snyder Patient-reported Outcomes in Cancer: A Review of Recent Research and Policy Initiatives CA Cancer J Clin, September 1, 2007; 57(5): 278 - 300. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 E. Grunfeld Looking Beyond Survival: How Are We Looking at Survivorship? J. Clin. Oncol., November 10, 2006; 24(32): 5166 - 5169. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. B. Clauser Use of Cancer Performance Measures in Population Health: A Macro-level Perspective J Natl Cancer Inst Monographs, October 1, 2004; 2004(33): 142 - 154. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||