© The Author 2005. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oxfordjournals.org.
Building a Research Consortium of Large Health Systems: The Cancer Research Network
Affiliations of authors: Group Health Cooperative Center for Health Studies, Seattle, WA (EHW, SMG, GH); Meyers Primary Care Institute, University of Massachusetts, Worcester, MA (TSF); Department of Ambulatory Care and Prevention, Harvard Pilgrim Health Care, Boston, MA (SF); Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA (AMG); Division of Research, Kaiser Permanente Northern California, Oakland, CA (LJH); Henry Ford Health System, Detroit, MI (CCJ); Kaiser Permanente Colorado, Denver, CO (JM); HealthPartners Research Foundation, Minneapolis, MN (SJR); Center for Health Research, Kaiser Permanente Northwest, Portland, OR (MCH, VJS); National Cancer Institute, Bethesda, MD (SHT); Kaiser Permanente Georgia (DT); Center for Health Research Hawaii, Kaiser Permanente Hawaii (TMV)
Correspondence to: Edward Wagner, MD, MPH, Group Health Cooperative Center for Health Studies, 1730 Minor Ave., Ste. 1600, Seattle, WA 98101 (e-mail: wagner.e{at}ghc.org).
 |
ABSTRACT |
|---|
 TopNotes Abstract IntroductionCRN STRUCTURE AND PROCESSESPARTICIPATING HEALTH SYSTEMSCRN STRUCTURECRN PROGRESSCHALLENGESLESSONS LEARNEDCONCLUSIONSReferences |
|---|
Critical questions about cancer prevention, care, and outcomes increasingly require research involving large patient populations and their care delivery organizations. The Cancer Research Network (CRN) includes 11 integrated health systems funded by the National Cancer Institute (NCI) to conduct collaborative cancer research. This article describes the challenges of constructing a productive consortium of large health systems, and explores the CRN's responses. The CRN was initially funded through an NCI cooperative agreement in 1999 and has since received a second 4-year grant. Leadership and policy development are provided through a steering committee, subcommittees, and an external advisory committee. The CRN includes integral and affiliated research projects supported by a Scientific and Data Resources Core. Three characteristics of the CRN intensified the general challenges of consortium research: 1) its members are large health systems with legitimate concerns about confidentiality of data about enrollees, providers, and the organization; 2) CRN research projects often generate highly sensitive data about quality of care; and therefore 3) each participating organization wants a strong voice in CRN direction. CRN experience to date confirms that a consortium of health systems with internal research capacity can address a range of important cancer research questions that would be difficult to study in other venues. The advantages and challenges of consortium research are explored, with suggestions for the development, execution, and management of multisystem population laboratories.
|
INTRODUCTION |
|---|
|
TopNotesAbstractIntroductionCRN STRUCTURE AND PROCESSESPARTICIPATING HEALTH SYSTEMSCRN STRUCTURECRN PROGRESSCHALLENGESLESSONS LEARNEDCONCLUSIONSReferences |
|---|
Consortium or cooperative group research has been the foundation of American clinical cancer research. Through their large investigative teams and access to large, diverse patient populations, research networks like Cooperative Groups and the related Community Clinical Oncology Programs (CCOPs) are critical resources for evaluating therapeutic and preventive interventions. Large networks figure prominently in the new NIH Roadmap effort to reengineer the clinical research enterprise (1). More recently the cohorts and databases assembled from large clinical trials and other databases have been used to study critically important clinical and policy questions included under the general rubric of outcomes research (2). These include studies of a broader array of outcomes such as quality of life, cost-effectiveness, prognostic factors, and the impacts of therapy on important subgroups such as racial minorities or older adults.
Although this is an important evolutionary step in clinical cancer research, the highly selective nature of clinical trial patient populations and databases limits their ability to answer many critical questions about care, long-term outcomes, costs, and other important issues. Such questions require more representative populations, longer follow-up periods, and a greater range of patient data. Many questions need data on the preventive and medical care experience as well as cancer characteristics of patients to describe and assess current cancer care, and to evaluate its impacts on patient outcomes and health care costs. Such data now exist through the SEER–Medicare data link for non-HMO data cancer patients who are 65 years and older (3). But population data of this sort are generally unavailable for HMO enrollees, those younger than 65 years, or those at risk of cancer.
To this end, the National Cancer Institute (NCI) issued a Request for Applications (RFA) in 1997 entitled "Cancer Research Network Across Health Care Systems." NCI's goal was to enhance research on cancer etiology, prevention, detection, and management through collaboration among large health care delivery organizations that emphasized ambulatory care; served large, stable populations of patients; and maintained databases with a rich array of patient information. Established large integrated health systems, especially those with internal research capabilities, seemed well positioned to respond to this RFA. The Cancer Research Network (CRN) was funded in 1999 (CRN 1), responded to a second competitive RFA in 2002, and has been funded for another 4 years (CRN 2).
There are compelling reasons for building a consortium of research-oriented integrated delivery systems to study cancer questions. These organizations provide comprehensive services, from prevention to palliation, to a defined population over many years. Characteristics and care of all enrollees are well documented in medical charts and automated data systems. Further, a consortium of geographically dispersed health systems increases the size and diversity of potential study populations, as well as variation in medical care practices for study. Thus, these systems can address key research questions that cannot be addressed in many other settings. Finally, the major participation of large, integrated delivery systems brings a substantial and innovative component of U.S. health care delivery to the cancer research endeavor.
The CRN has wrestled with the problems of any research consortium—communication, equitable sharing of opportunities for new projects and publications, and variation in resources, experience, and data. Also, research in health care delivery organizations, even those with internal research centers, includes other challenges related to concerns about privacy, litigation, and market competition.
In this paper, we describe the CRN's origin and progress, elucidate challenges to building a productive consortium of large delivery organizations, and discuss steps taken to address these challenges.
|
CRN STRUCTURE AND PROCESSES |
|---|
|
TopNotesAbstractIntroductionCRN STRUCTURE AND PROCESSESPARTICIPATING HEALTH SYSTEMSCRN STRUCTURECRN PROGRESSCHALLENGESLESSONS LEARNEDCONCLUSIONSReferences |
|---|
Our initial response to the RFA was a consortium of research programs based in 10 large health systems: Group Health Cooperative (GHC); Harvard Pilgrim Health Care (HPHC); Henry Ford Health System/Health Alliance Plan (HFHS/HAP); HealthPartners Research Foundation (HPRF); Meyers Primary Care Institute of Fallon/University of Massachusetts (Meyers); and Kaiser Permanente divisions in Hawaii (KPH), Oregon and Washington (KPNW), Northern California (KPNC), Colorado (KPCO), and Southern California (KPSC). In response to the 2002 RFA, we added an 11th site—Kaiser Permanente Georgia (KPGA)—to further augment geographic representation and population diversity. The CRN emanated from an established organization of health system–based research organizations—the HMO Research Network (4).
The CRN's initial goals were to 1) create a productive multipotential cancer research consortium that would become a national resource for research on cancer prevention, care and outcomes; 2) develop a research theme and new projects consistent with the theme; 3) develop and use standardized approaches to data collection, management, and analysis; 4) increase enrollment of health systems' patients in NCI-supported clinical trials; 5) continuously improve the CRN's performance through ongoing evaluation.
|
PARTICIPATING HEALTH SYSTEMS |
|---|
|
TopNotesAbstractIntroductionCRN STRUCTURE AND PROCESSESPARTICIPATING HEALTH SYSTEMSCRN STRUCTURECRN PROGRESSCHALLENGESLESSONS LEARNEDCONCLUSIONSReferences |
|---|
Patient Population and Clinical Care
The CRN's 11 participating members have enrolled populations ranging from 207 000 to 3 129 000, for an aggregate population of more than 10 million enrollees that represents 3.5% of the U.S. population. As shown in Table 1, the populations vary in the prevalence of ethnic and racial minorities, but in aggregate include many Asian Americans, African Americans, and Latinos. For example, there are nearly 1 million African American enrollees across the CRN. Most systems have substantial Medicare enrollments, ensuring access to the full age spectrum.
|
These are relatively stable health systems. All have been in operation for at least 25 years. Table 1 shows the 1-, 3-, and 5-year retention rates across the 11 sites for all age groups. Disenrollment rates ranged between 10% and 17% for 1 year, 16% and 33% over 3 years, and 24% and 46% over 5 years. Evidence from CRN studies to date suggests that disenrollment rates diminish with increasing age and are still lower for enrollees diagnosed with cancer. In a recent CRN study, the likelihood of disenrollment after a cancer diagnosis was less than 1% per year.
Each of these health systems has a group/staff model component, and several have network model components that cover various proportions of enrollees (Table 1). All provide comprehensive cancer services from primary prevention to end-of-life care such as hospice. All have medical and surgical oncologists and radiation therapists on staff or in their affiliated medical groups.
HMO Research Programs
All participating health systems have well-established public domain research programs. The year of research program initiation, research budget, and number of doctoral-level investigators are shown in Table 2. Some research programs are decades old, have budgets exceeding $10 million, and have 20 or more doctoral-level investigators, whereas a few are small and of recent origin. Nine programs are exclusively HMO sponsored, whereas two (Harvard Pilgrim and Meyers) represent joint programs with local academic institutions (Harvard University and the University of Massachusetts Medical Schools, respectively). A third, Henry Ford Health System, owns the HMO (Health Alliance Plan), but the researchers are not HMO employees; they are housed in the Medical Group. Across the centers, research tends to fall into four related areas: epidemiologic, clinical, health services, and behavioral research. Laboratory-based research is unusual among these institutions, except at Henry Ford and HealthPartners, although collaboration with academically based bench scientists is increasing.
|
Table 3 summarizes the automated data availability and longevity. Essentially all HMOs have computerized data on enrollment, usage, health care costs, inpatient and outpatient diagnoses, laboratory tests, and pharmacy (5). For most CRN sites, these data systems have been in place for many years, facilitating studies with lengthy periods of follow-up. Seven systems contribute to local SEER registries, and their research units have established linkages that give them access to SEER data on their enrollees. Two other organizations use local registries, and the remaining two use computerized diagnoses and related data to identify cancer patients. A few sites have established rapid ascertainment systems to facilitate study of incident cases.
|
The CRN's Virtual Data Warehouse provides a quantitative snapshot of cancer cases available for study. Based on automated data from 2003, the number of incident cancer cases at eight CRN member organizations was as follows: 3505 colorectal, 3868 lung, 6520 breast, and 5926 prostate cancers. The three CRN sites not included have a combined enrollment of approximately 1.3 million. The CRN also provides rich opportunities to study less common cancers. For example, in 2003 the eight organizations recorded 308 esophageal, 589 stomach, 667 pancreatic, and 575 ovarian cancers. Thus, for studies of most tumors, the CRN has access to sizable numbers of patients.
|
CRN STRUCTURE |
|---|
|
TopNotesAbstractIntroductionCRN STRUCTURE AND PROCESSESPARTICIPATING HEALTH SYSTEMSCRN STRUCTURECRN PROGRESSCHALLENGESLESSONS LEARNEDCONCLUSIONSReferences |
|---|
The CRN is an "Alliance Structure," more specifically a lateral alliance, as described in the book Managing a Health Care Alliance, Improving Community Cancer Care by Kaluzny and colleagues (6). They define "lateral" alliances as "those in which similar types of organizations pool information and resources to pursue mutually beneficial goals and interests." Evaluation of these alliances, especially CCOPs, has provided important insights into elements of a successful research alliance (7): interdependence for meeting key research needs, effective communication, mutual adaptation and collective problem solving. To foster interdependence and collective problem solving, the structure and policies of the CRN encourage the participation of all members of the consortium in decision making and all aspects of the research process. Successful alliances pay close attention to the processes of interaction and communication among alliance partners, and they opt for increasing the number of opportunities for interaction. We formed a communications committee to assess routinely whether CRN members feel that their concerns and ideas are being heard, are informed of decisions and developments, and to make recommendations to enhance communication.
CRN Leadership Structure
From its inception, the CRN has been governed by a steering committee that includes the CRN Principal Investigator (PI), a "site" PI at each health system, the PIs of the core research projects (if not a site PI), and the NCI project officer. Despite its large size, this group operates by consensus and makes all major decisions that way. The steering committee meets monthly by conference call and in person twice a year.
The Steering Committee receives advice from an advisory group of distinguished cancer researchers, the Academic Liaison Committee (ALC), and three committees (Fig. 1). In addition to providing general scientific guidance, ALC members review new proposals and help identify research opportunities. The committees play central roles in the conduct of CRN business. The Publications Committee formulates policy concerning publications and conducts prepublication review of all manuscripts. The New Proposals Committee reviews brief descriptions of proposed new projects for scientific merit, duplication or interference with other CRN activities, and feasibility. The Communications Committee assesses the quality of communication among CRN participants and makes recommendations to the Steering Committee for improvements such as the development of a newsletter, "CRN Connection," and revisions to the CRN's Web site.
|
Scientific Core Resources
The original CRN structure included a Data Resources Coordinating Center (DRCC), which bore the primary responsibility for meeting the data standardization goal, and seven Expert Teams. The evolving role of the DRCC has been a primary topic of discussion throughout the CRN's lifecycle. Its functions and progress toward data standardization are discussed more fully below.
Seven Expert Teams were established in CRN1. By design, the teams functioned more as interest groups than as typical cores of program project grants. Therefore, they received very limited funding and were expected to determine their own course. Teams included biostatistics, health economics, survey measurement, clinical trials, survivorship, pharmacoepidemiology, and genetics. Several served as incubators for new research ideas and proposals. For example, the Pharmacoepidemiology Expert Team has been funded by the Agency for Healthcare Research and Quality as a Center for Education and Research in Therapeutics (8). Interest groups now play a crucial role in the scientific life of the CRN, generating new collaborative relationships and proposals. The topics have expanded to include obesity, end-of-life care, quality of cancer care, the development of biorepositories, and racial disparities. Interest groups meet regularly by conference call or in-person during CRN annual meetings.
The CRN1 Survey Measurement and Economics Expert Teams provided advice and products useful to several projects—for example, a literature review on the assessment of race and ethnicity (9) and a review of practices around the use of incentives for surveying providers (10). In CRN2, these teams were combined with the DRCC in a new core resource, the Scientific and Data Resources Core (SDRC), whose more recent activities are described below.
|
CRN PROGRESS |
|---|
|
TopNotesAbstractIntroductionCRN STRUCTURE AND PROCESSESPARTICIPATING HEALTH SYSTEMSCRN STRUCTURECRN PROGRESSCHALLENGESLESSONS LEARNEDCONCLUSIONSReferences |
|---|
Create a Productive Multi-institutional Cancer Research Consortium
The CRN's initial focus was to build confidence and trust in the consortium. The involved research institutions had legitimate concerns about the new entity. Delivery system–based research programs have expended considerable time and resources to build population laboratories, and they feared loss of control over access to and use of their laboratories. The Steering Committee concluded that they as a group would make all major policy decisions and that they would operate by consensus. It was also decided that involvement in any of the scientific activities of the Network is voluntary. As trust built over time, attention shifted to the development of structures and processes that could efficiently conduct collaborative research, generate new proposals, and attract new investigators.
The initial RFA made explicit NCI's intent that this network or consortium ultimately serve as a national resource for researchers at NCI and other cancer research organizations, not just researchers based in the member institutions. CRN members view this expectation with cautious optimism, as CRN research benefits from collaboration with biomedical and clinical cancer researchers who are in short supply in most health system–based research centers. But valid analyses of organizational, administrative, and clinical data require local expertise and experience, and injudicious use could expose the contributing organizations to possible litigation or competitive disadvantage if measures of care quality or other sensitive indicators could be linked to particular sites. Thus, the Steering Committee felt the need to have investigators who are familiar with the data play a major role in analysis and interpretation, and decided to provide access to CRN populations and data to outside researchers only when there are CRN collaborators on the project. The CRN has also taken steps to increase opportunities for collaboration. First, the Academic Liaison Committee, in addition to its advisory function, provides linkages to researchers and resources outside the CRN. Second, the CRN has begun to establish formal relationships with comprehensive cancer centers and other research organizations with relevant scientific interests and expertise. Third, the CRN has established a public Web site (http://crn.cancer.gov) to augment its visibility to cancer researchers.
Develop a Research Theme and New Projects Consistent With the Theme
The Institute of Medicine's report, Ensuring Quality Cancer Care, highlighted deficiencies in current services for cancer patients and the need for cancer health services research "to assess patterns of cancer care and factors associated with the receipt of good care" (11). The President's Cancer Panel described managed care as "both a problem and an opportunity for cancer care" (12). The problem is the still unproven fear that cost-related restrictions on care will limit access to optimal care for managed-care patients. One opportunity is that large, integrated delivery and financing systems can develop policies and programs that promote high-quality cancer care. A second opportunity is that managed-care organizations can become population research laboratories for examining the impact of policies, new technologies and interventions, provider practices and perspectives, and features of the delivery of cancer services on patient outcomes. Thus, our research theme—"to conduct research on the effectiveness of cancer prevention and treatment modalities and programs to identify those patient, treatment, and delivery system factors associated with differences in outcomes"—was selected because of its consistency with the goals of the RFA, the NIH Roadmap, and the interests and expertise of CRN investigators.
The original and competitive renewal RFAs required proposals for specific research projects consistent with the research theme. Our initial integral projects focused on: tobacco control (13–15), the effectiveness of breast and cervical cancer screening (16–19), and efficacy of preventing breast cancer mortality in high-risk women through either early screening or prophylactic mastectomy (20–24). All projects employed rigorous, observational research designs and capitalized on the CRN's rich resources by involving multiple sites, their populations, and data resources. All studies were population based, i.e., able to identify either the entire population of interest (e.g., all women with invasive cervical cancer; all women dying of breast cancer), or a random sample of the larger population (e.g., current cigarette smokers). The major findings from the two breast cancer projects (18,23) have important implications for care. Even in the presence of organized, active programs to increase participation in screening, the major predictor of invasive disease is the failure to be screened (18). And contralateral prophylactic mastectomy reduces mortality significantly among women with unilateral breast cancer (23). These findings have already influenced screening and treatment policies among CRN HMOs.
The Steering Committee elected to have a rigorous open competition to select the required research projects for the renewal proposal. We sought proposals consistent with the objectives of the NCI RFA, which included a much broader array of research areas than our initial research theme. We intensively reviewed 22 concept proposals ranging in scope from etiology to end-of-life care and selected three projects. One project explores the possible prognostic significance of various biologic characteristics among women with ductal carcinoma in situ of the breast. This study includes approximately 3700 women, one of the largest cohorts with this important condition yet assembled. It is the first CRN study to combine a population-based epidemiological cohort study with sophisticated analyses of pathological material for biological prognostic factors. A second was a direct outgrowth of the tobacco project in CRN1—a randomized trial evaluating the effects of electronic medical record system enhancements on adherence to tobacco control guidelines and cessation rates. The third study is a randomized trial examining the feasibility and efficacy of providing tailored Web-based nutritional information to improve the dietary behaviors of HMO enrollees. The last two studies provide CRN investigators with opportunities to explore the research and intervention potential of two important technological advances in medical practice—electronic medical records and interactive patient-oriented Web sites.
The generation of fundable new projects has been a primary aim of the CRN from the beginning. To date, 21 projects have been funded in addition to the six core projects, and several others are under review. Table 4 lists all funded projects. They collectively study several different cancer sites and cover the natural history of cancer from prevention to end of life care. Of the 21 noncore projects, six are investigator-initiated grants from NCI; six are NCI administrative or minority supplements; two are contracts from the Centers for Disease Control and Prevention; one is an NCI contract; and six were supported through pilot funds awarded to the CRN in its renewal. Most projects involve multiple sites, and several involve collaborators from outside research institutions.
|
Develop and Use Standardized Approaches to Data Collection, Management and Analysis
From the outset, the sensitivity of health system data, especially data related to quality or delivery of care, has been a major concern. Therefore, the Steering Committee rejected the notion of the DRCC serving as a repository of generic data on the enrollees of each HMO for use in current and future studies. HMO leaders were reluctant to send generic data outside their organizations because of concerns regarding patient and institutional confidentiality. The challenge then was to develop a strategy for efficiently assembling analytic datasets using standardized data without a centralized database.
Next, the Steering Committee considered whether the DRCC should function as a typical statistical coordinating center: collecting project-specific data from each site, creating analytic files, and conducting the analyses. This notion was also rejected because the participating research centers sought to enhance their data and biostatistical resources and competency through involvement in the data management and analysis functions of CRN projects and activities.
Lack of consensus and the rapid startup of the core research projects slowed progress toward data standardization in CRN1. But as new proposals developed and were implemented, demands for more efficient aggregation and use of automated data across HMOs became more urgent. The Steering Committee combined the functions of the DRCC, Survey Measurement, and Economic Expert Teams in a single Scientific and Data Resources Core (SDRC). The overall goal of the SDRC is to increase the use of common approaches to measurement of key constructs, as well as the collection, transfer, and management of data. The SDRC developed the strategy of producing a Virtual Data Warehouse (VDW)—a uniform data dictionary and related site-specific code that guides the assemblage of standardized site-specific databases in each organization. The administrative and clinical data systems across CRN sites vary widely, making it a considerable task to generate comparable data across sites. Prior to the development of the VDW, each project had to resolve these issues for itself. If a given multisite project wanted to identify enrollees with certain demographic and disease characteristics prior to the VDW, project investigators would have to agree on variable definitions and programmers at each site would have to create code to run against their organization's administrative and clinical data systems. The VDW now contains the relevant conceptual and operational definition(s) of variables and the computer code used at the 11 sites to collect those variables. A projectwide programmer at any one site can now write a single program for the other collaborating sites to use that identifies the relevant subjects in each participating site from the VDW database maintained at each site.
To increase the visibility of this core and enhance communication with projects, we invite all project PIs to join the SDRC and the Steering Committee in a bimonthly conference call, the Project Leaders Forum. Agendas for these calls are scientific, including discussion of project-specific research issues, or broader conversations about expansion of the VDW or other methods issues.
Increase Enrollment of HMO Patients in NCI-Supported Clinical Trials
Enrollment of adults in cancer clinical trials continues to be a substantial barrier to progress (25) in cancer control. Two CRN projects have addressed NCI's priorities to increase the enrollment of adult cancer patients, especially racial and ethnic minorities, in clinical trials. One project evaluated barriers to clinical trial enrollment from the perspective of an organization's leaders and oncologists (26). The other is assessing strategies for increasing enrollment of Vietnamese and Chinese women in breast cancer prevention and treatment trials. These two projects explore barriers and facilitators to clinical trial recruitment and enrollment that can inform the design and testing of interventions to increase trial participation. Also, one CRN site is testing an automated system for proactively and routinely notifying clinicians of cancer trials.
Continuously Improve the CRN's Performance Through Ongoing Evaluation
In the RFA, NCI stipulated that the awardee periodically evaluate its performance. We chose to make this evaluation a central element of CRN management. The evaluation regularly assesses the following aspects of CRN structure and function: performance of research projects, effectiveness of core resources, effectiveness of infrastructure (PI's office, committees, web site), extent of collaboration and quality of communication, impact on CRN staff and organizations, scientific productivity.
This formative evaluation is undertaken annually, with broad discussion of results and development of specific plans for improvement.
|
CHALLENGES |
|---|
|
TopNotesAbstractIntroductionCRN STRUCTURE AND PROCESSESPARTICIPATING HEALTH SYSTEMSCRN STRUCTURECRN PROGRESSCHALLENGESLESSONS LEARNEDCONCLUSIONSReferences |
|---|
Several important challenges faced by the CRN have relevance for other multisite research structures.
Representativeness of the CRN Systems and Populations
CRN proposals and manuscripts have been criticized by reviewers who feel that our health systems' populations, practices, and data are unrepresentative. We feel that these concerns are often unjustified. The enrollees in our systems cover the demographic spectrum of the American population, as several organizations serve Medicaid and Medicare recipients in addition to commercial clients, and most have minority representation similar to or even larger than that of their surrounding communities. The growing diversification of these health systems means that the professional caregivers and practice systems are more typical than in the traditional HMO setting. That these systems are innovative may render them less representative, but, in our view, more interesting to study.
Privacy and Confidentiality
New privacy regulations have added to already serious concerns about patient and institutional confidentiality for the CRN. CRN organizations have their own institutional review boards (IRBs), each of which has developed local interpretations of new federal privacy regulations. These interpretations are not uniform, and individual projects have had to vary consent and survey procedures across sites. For each project, participating health systems must obtain IRB approval for all study components, materials and modifications—an iterative process that has occasionally compromised efficiency (27).
Protecting the confidentiality of quality data on individual patients has proven to be especially challenging, and a significant threat to projects (28). For example, the project exploring the effectiveness of breast and cervical cancer screening programs and practices specifically assessed the prevalence of false-negative mammograms and Pap smears as well as screen-positive patients who did not receive follow-up. Several participating organizations and their clinical leaders were understandably concerned that research rereadings of specimens, especially from patients presenting with advanced cancer, could be subpoenaed for malpractice litigation. Because of this, some sites would not provide access to specimens until we could assure the sites of protection of the research data from legal discovery. To this end, we requested a certificate of confidentiality, an assurance document from the Department of Health and Human Services, which prevents individually identifiable research data from being subject to legal discovery. Upon receipt of the certificate, all participating sites agreed to submit their patient material for research.
Concurrently with the reissuance of the CRN RFA, NCI and the Agency for Healthcare Research and Quality (AHRQ) had been discussing the need for closer collaboration in the area of quality-of-cancer-care research. As a result of these discussions, it was determined that AHRQ cosponsorship of NCI research networks that have a substantial health services research element would be appropriate. As a result, AHRQ is a cosponsor of CRN2. One benefit of cosponsorship is that AHRQ-sponsored projects receive broad protection from legal discovery beyond that conferred by the certificate of confidentiality.
Financial Barriers to Collaboration
The CRN has encountered financial disincentives inherent in multisite research. In addition to the more evident costs of collaboration associated with contracts, administration, and communication across sites, there are less obvious costs that exert an impact on project budgets. A major issue in the CRN has been differences in IRB-related consent requirements. These multiple recruitment and consent protocols add substantial costs to data collection. For example, a telephone survey center may have to program multiple consent approaches into their telephone interviewing system and train interviewers to use different procedures. Differences in data collection logistics may also necessitate development of multiple versions of protocols to address the unique features of some sites. For example, essentially all CRN sites have multiple clinical locations each with their own medical record facility. In some organizations, researchers can access medical records in a central location, reducing travel requirements. In other HMOs, research staff must travel long distances to review records. Grant reviewers often do not understand the nuances in conducting research in different settings and insist on the application of standardized data collection costs, often pegged at the costs of lower cost sites. As a result, data collection may be substantially underfunded.
Project Leadership
CRN projects involve multiple research teams and their associated delivery systems. Many CRN investigators had limited experience with multisite research, and the associated administrative and scientific complexities of participating in, much less managing, multisite projects. This produced opportunities for misunderstandings, miscommunications, failures to appreciate important differences among sites, and failures to acknowledge and respond to the research aspirations of participating investigators. While not unique to health system research centers, the risk of these problems may be higher in the CRN because leadership of the individual projects is spread across the 11 sites by design. This structure places major responsibility for project performance and morale on the individual project PIs.
Increasing Complexity of Integrated Health Systems
Most CRN member organizations began as prepaid group practices—the original HMOs. But market pressures have induced many of these organizations to diversify, most often by developing networks of physicians paid by fee-for-service reimbursement, or by increasing the fee-for-service activities of the group practice. Prepaid group practice facilitated population-based research because of its defined population of enrollees, committed clinicians, and relatively complete data on, e.g., usage, drugs, and laboratory results. Although we can still define network enrollees, their providers have less organizational commitment to the health system and are harder to engage in data collection or intervention studies. Also, network data are far less complete, as they rely on claims. Further, patients outside of the prepaid group practice are less likely to use laboratories or pharmacies that make relevant data available for research. The growing diversity of financing and delivery arrangements within health systems increases not only the representativeness but also the complexity of research in these settings. This trend underscores the importance of locally knowledgeable researchers able to navigate the political and informational complexities of each organization.
|
LESSONS LEARNED |
|---|
|
TopNotesAbstractIntroductionCRN STRUCTURE AND PROCESSESPARTICIPATING HEALTH SYSTEMSCRN STRUCTURECRN PROGRESSCHALLENGESLESSONS LEARNEDCONCLUSIONSReferences |
|---|
The experience of the CRN confirms the rich research opportunities provided by a coordinated research program involving large health systems. The CRN has already generated many new projects reflecting questions posed by investigators from the 11 sites, affiliated academic institutions, and NCI. The rich array of automated data available in the CRN has been, as expected, integral to most projects and proposals. However, finding acceptable and efficient ways to assemble and analyze these data in this era of heightened privacy concerns and regulations remains a constant challenge.
The Participating Institutions
The CRN's progress confirms the potential of large health plans with internal research programs to address critical cancer research questions. Population size and diversity across CRN sites facilitate examination of questions related to important disease or population subgroups—e.g., women with DCIS, Vietnamese Americans, or recent smoking quitters. The presence of local researchers familiar with the system and the data has proven essential. These health systems are innovative, especially in the use of information technology. Nine of the 11 health systems will have sophisticated electronic medical record systems in full operation shortly, and all but one will use the same software. Most have or are developing interactive patient Web sites. These system changes create extraordinary opportunities to collect clinical and patient data efficiently, as well as intervene with clinicians and patients. CRN investigators will soon be able to conduct detailed studies of clinical cancer care relying exclusively on existing electronic data.
Protecting Privacy and Confidentiality
To retain anonymity of sites in presenting potentially sensitive organizational data, we have developed methods for protecting site identification. For example, since organization size and ethnic and racial composition vary widely and could point to the organization, we have agreed to present only site-specific data when it is crucial to the research question being addressed. If site-specific data are presented, it is done in a manner that does not reveal sample size or demographic characteristics that would identify the health system.
Leadership Development
The evaluation helped us identify problems in project management and opportunities to resolve them. Ongoing discussion of the project leader role and the establishment of clear expectations for project communications has improved morale and efficiency. There is a steep learning curve for project and site principal investigators. The Project Leaders Forum, an important communication vehicle in CRN 2, provides support and mentoring for less experienced project PIs.
|
CONCLUSIONS |
|---|
|
TopNotesAbstractIntroductionCRN STRUCTURE AND PROCESSESPARTICIPATING HEALTH SYSTEMSCRN STRUCTURECRN PROGRESSCHALLENGESLESSONS LEARNEDCONCLUSIONSReferences |
|---|
Consortia of large health systems providing comprehensive care to many patients at risk of or suffering from cancer can serve as rich laboratories for studying a variety of important questions about prevention and management of cancer or other health conditions. Such laboratories should play a central role in accelerating the translation of research findings into practice (1). The CRN represents one strategy for building such a laboratory by constructing a consortium of integrated care systems with established research programs. Its development was facilitated by the prior collaborative experience of CRN sites in the HMO Research Network. The CRN has made substantial progress toward its goal of producing high-quality collaborative research projects and proposals. Its democratic approach to decision making, emphasis on communications, supportive infrastructure, focused core resources, and ongoing evaluation have created a culture and processes supportive of collaboration. However, in our years of operation, we have become much more aware of the added costs, financial and interpersonal, of multisite collaboration and the pressures on developers of new projects to identify the optimum number of sites needed to achieve aims of a given project. We hope that the experience of the CRN will contribute to building a body of evidence that will facilitate generation of more and better large health care delivery laboratories and provide a model for the emerging NIH Roadmap.
|
NOTES |
|---|
The CRN has benefited enormously from the leadership and support of its NCI project officer, Dr. Martin Brown.
Funded by the National Cancer Institute (U19 CA 79689, Increasing Effectiveness of Cancer Control Interventions, Edward H. Wagner, MD, MPH, Principal Investigator).
|
REFERENCES |
|---|
|
TopNotesAbstractIntroductionCRN STRUCTURE AND PROCESSESPARTICIPATING HEALTH SYSTEMSCRN STRUCTURECRN PROGRESSCHALLENGESLESSONS LEARNEDCONCLUSIONSReferences |
|---|
(1) Zerhouni E. Medicine. The NIH Roadmap. Science 2003;302:63–72.
(2) Lipscomb J, Donaldson MS, Arora NK, Brown ML, Clauser SB, et al. Cancer outcomes research. J Natl Cancer Inst Monogr 2004;33:178–97.
(3) Warren JL, Klabunde CN, Schrag D, Bach PB, Riley GF. Overview of the SEER-Medicare data: content, research applications, and generalizability to the United States elderly population. Med Care 2002;40(8 Suppl):IV-3–18.
(4) Vogt TM, Lafata JE, Tolsma D, Greene SM. The role of research in integrated health systems: the HMO Research Network. Am J Manag Care 2004;10:643–8.[ISI][Medline]
(5) Selby JV. Linking automated databases for research in managed care settings. Ann Intern Med 1997;127(8 Pt 2):719–24.
(6) Kaluzny AD, Warnecke RB and Associates, editors. Managing a health care alliance: improving community cancer care. San Francisco (CA): Jossey-Bass; 1996.
(7) Klabunde CN, Kaluzny AD. Accrual to the breast cancer prevention trial by participating Community Clinical Oncology Programs: a panel data analysis. Breast Cancer Res Treat 1995;35:43–50.[CrossRef][ISI][Medline]
(8) Platt R, Davis R, Finkelstein J, Go AS, Gurwitz JH, Roblin D, et al. Multicenter epidemiologic and health services research on therapeutics in the HMO Research Network Center for Education and Research on Therapeutics. Pharmacoepidemiol Drug Saf 2001;10:373–7.[CrossRef][ISI][Medline]
(9) Ford ME, Hill DD, Nerenz D, Hornbrook M, Zapka JG, Meenan R, et al. Categorizing race and ethnicity in the HMO Cancer Research Network. Ethn Dis 2002;12:135–40.[Medline]
(10) Field TS, Cadoret CA, Brown ML, Ford ME, Greene SM, Hill D, et al. Surveying physicians: do components of the "Total Design Approach" to optimizing survey response rates apply to physicians? Med Care 2002;40:596–606.[CrossRef][ISI][Medline]
(11) Hewitt M and Simone JV, editors. National Cancer Policy Board, Institute of Medicine and National Research Council. Ensuring Quality Cancer Care. Washington (DC): National Academy Press; 1999. p. 223.
(12) President's Cancer Panel. Voices of a Broken System: Real People, Real Problems. Annual Report, 2000–2001. Available at: http://deainfo.nci.nih.gov/advisory/pcp.
(13) Rigotti NA, Quinn VP, Stevens VJ, Solberg LI, Hollis JF, Rosenthal AC, et al. Tobacco-control policies in 11 leading managed care organizations: progress and challenges. Eff Clin Pract 2002;5:130–6.[Medline]
(14) Solberg LI, Hollis JA, Stevens VJ, Rigotti NA, Quinn VP, Aickin M. Does methodology affect the ability to monitor tobacco control activities? Implications for HEDIS and other performance measures. Prev Med 2003;37:33–40.[CrossRef][ISI][Medline]
(15) Solberg LI, Quinn VP, Stevens VJ, Vogt TM, Rigotti NA, Zapka JG, et al. Tobacco control efforts in managed care: what do the doctors think? Am J Manag Care 2004;10:193–8.[ISI][Medline]
(16) Puleo E, Zapka JG, White MJ, Mouchawar J, Somkin C, Taplin SH. Caffeine, cajoling and other strategies to maximize clinician survey response rates. Eval Health Prof 2002;25:169–84.
(17) Zapka JG, Taplin SH, Solberg LI, Manos MM. A framework for improving the quality of cancer care: the case of breast and cervical cancer screening. Cancer Epidemiol Biomarkers Prev 2003;12:4–13.
(18) Taplin SH, Ichikawa L, Yood MU, Manos MM, Geiger AM, Weinmann S, et al. Reason for late-stage breast cancer: absence of screening or detection, or breakdown in follow-up? J Natl Cancer Inst 2004;96:1518–27.
(19) Zapka JG, Puleo E, Taplin SH, Goins KV, Yood MU, Mouchawar J, et al. Processes of care in cervical and breast cancer screening and follow-up-the importance of communication. Prev Med 2004;39:81–90.[ISI][Medline]
(20) Reisch LM, Fosse JS, Beverly K, Yu O, Barlow WE, Harris EL, et al. Training, quality assurance, and assessment of medical record abstraction in a multisite study. Am J Epidemiol 2003;157:546–51.
(21) Geiger AM, Greene SM, Pardee R, Hart G, Herrinton LJ, Macedo AM, et al. A computerized system to facilitate medical record abstraction in cancer research. Cancer Causes Control 2003;14:469–76.[CrossRef][ISI][Medline]
(22) Herrinton LJ, Barlow WE, Yu O, Geiger AM, Elmore JG, et al. Efficacy of prophylactic mastectomy in women with unilateral breast cancer. J Clin Oncol. Epub 2005 Mar 28.
(23) Geiger AM, Yu O, Herrinton LJ, Barlow WE, Harris EL, et al. A Population-Based Study of Bilateral Prophylactic Mastectomy Efficacy in Women at Elevated Risk for Breast Cancer in Community Practices. Arch Int Med 2005;23:4275–86.
(24) Elmore JG, Reisch LM, Barton MB, Barlow WE, Rolnick S, et al. Efficacy of breast cancer screening in the community according to risk level. J Natl Cancer Inst 2005;97:1035–43.
(25) Sateren WB, Trimble EL, Abrams J, Brawley O, Breen N, Ford L, et al. How sociodemographics, presence of oncology specialists, and hospital cancer programs affect accrual to cancer treatment trials. J Clin Oncol 2002;20:2109–17.
(26) Somkin CP, Altschuler A, Ackerson A, Geiger AM, Greene SM, et al. Organizational barriers to physician participation in cancer clinical trials. Am J Manag Care 2005;11:413–21.[ISI][Medline]
(27) Greene SM, Geiger AM, Harris El, Altschuler A, Nekhlyudov L, et al. Impact of IRB Requirements on a Multicenter Survey of Prophylactic Mastectomy Outcomes. Ann Epidemiol 2005; Jul 6 [Epub].
(28) Carney PA, Geller BM, Moffett H, Ganger M, Sewell M, Barlow WE, et al. Current medicolegal and confidentiality issues in large, multicenter research programs. Am J Epidemiol 2000;152:371–8.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  D. J. del Junco, S. W. Vernon, S. P. Coan, J. A. Tiro, L. A. Bastian, L. S. Savas, C. A. Perz, D. R. Lairson, W. Chan, C. Warrick, et al. Promoting Regular Mammography Screening I. A Systematic Assessment of Validity in a Randomized Trial J Natl Cancer Inst, March 5, 2008; 100(5): 333 - 346. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. S.M. Buist, L. Ichikawa, M. N. Prout, M. U. Yood, T. S. Field, C. Owusu, A. M. Geiger, V. P. Quinn, F. Wei, and R. A. Silliman Receipt of Appropriate Primary Breast Cancer Therapy and Adjuvant Therapy Are Not Associated With Obesity in Older Women With Access to Health Care J. Clin. Oncol., August 10, 2007; 25(23): 3428 - 3436. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. G. Glass, J. V. Lacey Jr, J. D. Carreon, and R. N. Hoover Breast Cancer Incidence, 1980 2006: Combined Roles of Menopausal Hormone Therapy, Screening Mammography, and Estrogen Receptor Status J Natl Cancer Inst, August 1, 2007; 99(15): 1152 - 1161. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. Mandelblatt To Screen or Not to Screen Older Women for Breast Cancer: A New Twist on an Old Question or Will We Ever Invest in Getting the Answers? J. Clin. Oncol., July 20, 2007; 25(21): 2991 - 2992. [Full Text] [PDF]
|
|
|
|
|
|
J. Z. Ayanian and P. B. Jacobsen Enhancing Research on Cancer Survivors J. Clin. Oncol., November 10, 2006; 24(32): 5149 - 5153. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||