Maternal & Child Health Bureau - Training Report - Close Window
Click here to download the PDF Version
Introduction & Table of Contents: The MCH Training Program: An Evaluation
Chapter 1: Introduction & Methodology
Chapter 2: A Statistical Snapshot of the MCH Training Program
Chapter 3: Training for Leadership
Chapter 4: Supporting Faculty in Leadership Roles
Chapter 5: Contributing to Advances in the Field
Chapter 6: Promoting Collaboration
Chapter 7: The Economics of MCH Training
Chapter 8: Recommendations
Notes
Appendix
Chapter 1: Introduction and Methodology Illustration of people in a group talking.
image of male and female in a classroom facing instructor

The Maternal and Child Health Bureau (MCHB) has initiated a series of measures designed to increase accountability and improve decision-making. One of these efforts is an assessment of the impact of discretionary grants in the category of Special Projects of Regional and National Significance (SPRANS), which comprise an array of demonstration, research, and training grants.The evaluation of the Maternal and Child Health (MCH) Training Program described in this document is part of this effort.

The National Center for Education in Maternal and Child Health (NCEMCH) at Georgetown University was awarded a grant that included as one of its objectives the development of a model for evaluating SPRANS projects. The first program to be evaluated was the Healthy Tomorrows Partnership for Children Program.The MCH Training Program is the second SPRANS program to be evaluated. The evaluation focuses on the 13 long-term training priorities supported by MCHB; the continuing education grants, which are quite different from the long-term priorities, are not included in the study.

The training evaluation consisted of two phases. Because little had been written describing the Training Program, the purpose of phase I was to chronicle the program's history and development and to identify themes common to the 13 priorities. The product of phase I, Building the Future: The Maternal and Child Health Training Programwas based on a review of Training Program documents, interviews with current and former federal staff associated with the program, and information obtained in focus groups from grantees in seven of the training priorities. An important outcome was greater clarity about the overall goals of the program. Phase II, the results of which are presented here, is designed to broadly assess the program's accomplishments, identify problems, and provide recommendations focused on program operations and management.

BACKGROUND

MCHB and its predecessor agencies have funded long-term training in maternal and child health since the 1940s.The Children's Bureau took a holistic approach to the care of children and families, viewing health, social, and emotional needs as inseparable and equally important. This perspective has permeated the MCH Training Program throughout its history. Another hallmark of the program is its long-standing focus on vulnerable populations, including children with special health care needs and underserved women, children, and adolescents.

Much more than just a mechanism to support the education of individuals, the Training Program was designed to be a vehicle for national MCH infrastructure building. Training Program grantees were to be key partners with the federal government and the states in improving the health of women and children through their work with professional associations, public agencies, and voluntary organizations, and to constitute a ready and willing cadre of individuals with expertise, dedication, and commitment to children.

THE TRAINING PROGRAM AND THE MCH MISSION

The Training Program is a key resource for MCHB as it strives to address the following goals articulated in its strategic plan:

  • Goal 1: Eliminate Barriers and Health Disparities: The Training Program promotes this goal through an educational focus on health disparities, development of outreach services for children and families who have poor access to health services,and policy work,such as service on advisory committees or task forces.

  • Goal 2: Ensure Quality of Care: A major aspect of the Training Program is quality improvement in the provision of health services. Grantees develop practice guidelines, assist states and communities with evaluation, disseminate research findings to various communities, and provide quality training for a new generation of MCH leaders and practitioners.

  • Goal 3: Improve the Health Infrastructure and System: Trainees are taught the value of comprehensive systems of care, cultural competence, and family-centered care. Many grantees function as local, state, and national advocates to improve the health care system.

Interrelationship of the Training and Block Grant Programs

The MCH Block Grant program and the Training Program represent complementary approaches to addressing the health of women and children. In addition, the Training Program both directly and indirectly supports the Block Grant program. Examples of direct support include technical assistance and continuing education provided by Training Program faculty for state Title V programs, and examples of indirect support include activities such as standards development, policy work, information dissemination, applied research, and the education of a new generation of practitioners. The Training Program thus enhances MCHB's ability to serve as a catalyst for change and strengthens the context for the delivery of MCH services.

The MCH pyramid (Figure 1) is a graphical representation of the activities supported by MCHB. It identifies the levels of services provided through Title V. The Training Program is located in the base of the pyramid. Without this foundation, the other MCH functions would be severely compromised. The base consists of infrastructure-building services, including assessment and assurance functions and training. These infrastructure-building services were noted as critical areas of emphasis for public health programs in the landmark study conducted by the Institute of Medicine, The Future of Public Health.

Interrelationship of Special MCH Initiatives and the Training Program

MCHB supports a number of special initiatives, and the alliances MCHB has established with universities through the Training Program are critical to the success of these initiatives. A few illustrative examples are provided below:

• Children with Special Health Care Needs: MCHB, in particular its Division of Services for Children with Special Health Care Needs, works to improve services for children with a variety of disabilities. States receive MCH Block Grant funds to ensure that services are adequate and of high quality. Several training grant priorities focus on children with special health care needs. The interdisciplinary approach of priorities such as LEND ensures that children with complex health and social needs receive coordinated care from a variety of disciplines. Most LEND projects work collaboratively with their state offices; in two states, the LEND program actually administers the state program for children with special health care needs. LEND grantees provide many of the experts (LEND program faculty) who are equipped to treat and diagnose children with neurodevelopmental disabilities, and LEND grantees deliver an array of clinical services within most states, serving as a referral source for the state programs. LEND grantees also provide community training and advocacy for special needs children, supporting the work of MCHB and state offices.Other Training Program priorities, such as pediatric pulmonary centers, behavioral pediatrics, pediatric dentistry, communication disorders, and the occupational and physical therapy projects, also focus on children with special health care needs.

Figure 1: an image of the MCH Pyramid

  • Office of Adolescent Health: Established in statute, MCHB's Office of Adolescent Health strives to improve the health of the nation's adolescents through special discretionary grants, policy work, support of Title V programs in improving adolescent health, and interagency collaborationThe LEAH grants are integral partners in this work. They provide technical assistance to the state adolescent health coordinators, conduct research that furthers the goals of the Office of Adolescent Health, and generate policy documents that foster awareness of adolescent health issues. LEAH grantees form the core of the Society for Adolescent Medicine, the key professional association focused on adolescent health, and through this association they advocate for new policies and treatment approaches to benefit adolescentsThe Office of Adolescent Health's two policy center grants are housed in the same university departments as two of the LEAH grants; both the LEAH projects and the policy centers are strengthened through the complementary activities of the two categories of grants.

  • Crosscutting Initiatives: From time to time, MCHB supports initiatives that cut across all its offices and divisions. The Training Program enables MCHB to more effectively accomplish such initiatives, as shown by the example of Bright Futures. Designed to improve quality of care for children and their families, Bright Futures is a set of expert guidelines and a practical developmental approach to providing health supervision for children from birth through adolescence, and consists of a variety of tools for health professionals, families, and communities. The Training Program has been pivotal to the success of Bright Futures. For example, many of the experts on the panels assembled to develop the Bright Futures guidelines were either graduates of the Training Program or current faculty.In addition, several training projects visited as a part of this evaluation have fostered curriculum changes, both within their own universities and nationwide, that build on the Bright Futures guidelines. The Training Program has played a central role in Bright Futures, albeit one that has gone largely unrecognized. A great strength of the MCH Training Program over time has been its implicit recognition of the way in which social change occurs through the synergy created by service providers, policymakers, academics, and family members working in concert. The Training Program is integral to this process.

GOALS OF THE MCH TRAINING PROGRAM

The MCH Training Program addresses a diverse set of needs. And yet it has developed a cohesive set of goals that distinguishes it from other federal health training programs. The five goals of the MCH Training Program are to:

  • train leaders;

  • address the special health and social needs of women, infants, children, and adolescents;

  • foster interdisciplinary care;

  • change attitudes and practice (e.g., toward family-centered and culturally competent care); and

  • emphasize the public health approach.

Prior to the publication of Building the Future: The Maternal and Child Health Training Program,These goals had not been clearly articulated in written documents.

Training Leaders

The MCH Training Program aims to train a new generation of leaders who can advocate for children and their families, provide quality clinical services, teach, and conduct research. Leadership training is a strategy chosen by MCHB to maximize the impact of a program with limited resources relative to need. Although it could be argued that the MCH Training Program has always trained leaders, this aspect of the program has recently become explicit and more central to the MCH Training Program mission.

Addressing the Special Needs of Women, Children, and Adolescents

A key characteristic of the MCH Training Program is its focus on women, children, and adolescents. Historically, health professionals in a number of fields have not received adequate training in serving the special health and social needs of these populations, a situation that continues to the present day. The MCH Training Program is intended to address this gap.

Fostering Interdisciplinary Care

As children began to survive previously untreatable complications of birth, and as once-fatal illnesses became treatable, some health care providers turned their attention to the complex health and social needs of children with chronic health problems. Single disciplines cannot address the needs of many of the children who have special needs, and so, in the 1960s, an interdisciplinary model of care emerged from the experience of the University Affiliated Facilities (later renamed as University Affiliated Programs).This model fosters collaboration among faculty and trainees from various disciplines as they work together to address the multifaceted issues of children with special health care needs. The MCH Training Program is currently one of the only sources of support for this type of training.

Changing Attitudes and Practice

Quality health care services are community-based, family-centered, and culturally competent.In addition, health care should be coordinated, and health services should be integrated with other systems that serve women, children, and families (including education, justice, and social services). Noted in the Title V legislation that defined the Children with Special Health Care Needs (CSHCN) program, these aspects of service delivery have come to constitute core MCH values, and the Training Program attempts to ensure that they are integrated in each training project and that graduates of the Training Program reflect these values in their practices.

Emphasizing the Public Health Approach

The MCH Training Program has attempted to broaden the perspective of clinicians to an understanding of public health, of preventing problems from occurring among population groups. The public health approach recognizes that many health problems are rooted in the behavior of individuals and in their social context and that the environment plays a major role in health.In contrast with the clinical medical approach, which explores the history and health conditions that may have led to health problems in a single individual, the public health approach focuses on identifying patterns among groups. It has four basic steps: (1) clearly define the problem; (2) identify risk and protective factors; (3) develop and test interventions; and (4) implement interventions.The public health approac h is a rational and organized way to marshal prevention efforts and ensure that they are effective.

Our faculty have learned about the public health perspective, advocacy, cultural competency, and family-centered care. Faculty who come from a clinical background were not trained with such a model. This is at the forefront of exemplary practice.

Project director, Occupational Therapy

NEEDS ADDRESSED BY THE MCH TRAINING PROGRAM

Although the grantees of all 13 MCH Training Program priorities incorporate the general goals of the Program in their projects, the specific needs that individual priorities address vary considerably. Although MCHB recently funded a graduate training and continuing education needs assessment, it has not done so in the past rather, the priorities have arisen over time in an ad hoc way. This evaluation found that training needs, as reflected in the 13 priorities, are qualitatively different and may be conceptualized in several ways, as follows:

• The scope and/or trajectory of a particular health problem: A health problem may affect many people, be quite severe, and/or be dramatically increasing in scope, and the resources to address it may be inadequate. Asthma is an example: it is sometimes fatal, affects hundreds of thousands of children, and is growing in prevalence.However, the resources to address asthma are not commensurate with its scope. The PPC training projects are working to understand and control asthma, along with other significant pulmonary diseases. Dental disease in children, and adolescent suicide, are other problems of great scope, ones that are preventable; these are being addressed by the pediatric dentistry and adolescent health priorities, respectively.

• Lack of a doctoral-level professoriate: In some health fields, the master's degree is the terminal degree, and persons capable of effectively teaching trainees (i.e., persons with doctorates) are few. An example is the field of communication disorders. The master's degree is the certifying degree for practicing audiologists and speech/language pathologists. In a robust economy, there is little or no economic incentive for practitioners to pursue a doctoral degree.Alternatively, a field may experience a decrease in the number of doctoral-trained individuals, with universities then having difficulty recruiting qualified faculty for available positions. MCH programs in schools of public health, for example, report difficulties in finding and attracting appropriately trained faculty. Pediatric dentistry is another field with difficulty recruiting academics. Of those trained in pediatric dentistry, most tend to pursue private practice.

image of man and woman looking at document

  • Complexity of clinical problems: Some children are particularly difficult to treat, especially those with multiple disabilities and/or illnesses. An example is a child who is both autistic and blind. Such children typically require the services of a variety of health care professionals who have had special training, but these professionals may be in short supply. A recent study found that pediatricians lack training in providing medical care to children with special health care needs.Children with special health care needs face not only complex clinical issues, but often have social and educational needs that must be met as well. Individuals trained in an interdisciplinary model that focuses on addressing such complex needs in collaboration with other professionals (both health- and non health-related) are well-suited to provide this type of care, but may be even more difficult to find. The LEND priority addresses the need for training specialists to work with children with neurodevelopmental and related disorders.

  • Special needs of subpopulation groups: Some population groups may be quite large and have special needs that have gone unmet. This is the case with adolescents. Adolescents have high rates of certain risk behaviors, such as use of cigarettes, alcohol, and other drugs, and also high rates of obesity and sexually transmitted dis-eases.And yet, adolescent medicine is a relatively new subspecialty with few trained practi-tioners.The LEAH projects train professionals in several disciplines to serve adolescents and promote improvements in adolescent health.

  • Perceived urgency of a problem: A health care problem may be viewed as urgent, perhaps because new research has documented its prevalence or because practitioners in the field find that they confront it daily and lack the resources or knowledge to address it. Behavioral problems of children is an example: primary care practitioners are encountering increasing numbers of children with mental health and/or behavioral problems, such as attention-deficit hyperactivity disorder (ADHD) or depression, but most have neither the knowledge nor the requisite skills to treat children with these problems.The behavioral pediatrics training projects aim to address this deficiency.
  • Inadequacy of MCH content in basic training programs: Some professional training programs are designed to educate generalists who can serve the needs of a variety of patients or clients. However, these programs may lack appropriate MCH content. Examples include the fields of social work, occupational therapy, physical therapy, respiratory therapy, nursing, and nutrition. The MCH Training Program priorities in these disciplines address gaps in basic professional education.

  • Lack of racial and ethnic diversity: In the absence of racial and ethnic diversity in a field, important issues may be overlooked in the provision of services and quality of care may be compromised. To ensure access and enhance quality, individuals from diverse backgrounds may need to be encouraged to receive training and then supported financially, academically, and emotionally. The Historically Black Colleges and Universities (HBCU) priority is intended to increase the number of professionals from diverse backgrounds providing primary care in community-based settings, with an emphasis on the special needs of families of African-American and Hispanic descent. Additionally, efforts must be made to increase cultural competency among nonminority MCH professionals.

The Training Program has evolved over time to address needs as they have emerged. As social and medical issues change, new needs may be identified.

FOCUSING THE EVALUATION

The MCH Training Program is both large and complex. To focus the evaluation and ensure that the most important questions were asked, an MCH Training Program logic model (Figure 2) was developed in collaboration with the project's advisory board. A logic model helps to clarify the theory of any program and elucidates presumed relationships among different levels of action. The Training Program logic model shows that the outputs of the Program include technical assistance, consultation, and continuing education; research; clinical services innovations; faculty development; curricular changes; and increased numbers of students receiving training in MCH. These outputs lead to a set of intermediate outcomes that include dissemination of knowledge to the field; increased knowledge of how to serve the health and social needs of the MCH populations; improved delivery of clinical care; and the training of leaders, all of which generate better-quality care, more-integrated systems, and more-informed policy decisions, with the ultimate outcome of improved health for families.

Using the logic model as a guide, a set of evaluation questions was developed again in collaboration with the advisory board and a methodology appropriate to each question was identified.

Issues selected for analysis included the ways in which resources are utilized by training projects; the types of activities supported by the Training Program; the experiences of beneficiaries of the Training Program, including trainees and recipients of continuing education and technical assistance; the perceived impact of training projects on trainees; the ways in which projects are integrated into trainees universities; and policy and administrative issues of potential interest to MCHB.

STUDY METHODOLOGIES

The methodologies selected for the evaluation included a review of the FY 1999 continuation applications for all 101 projects (record review); site visits to 31 training projects with interviews of multiple individuals at each site; focus groups with state Title V program staff and federal regional MCH consultants; and telephone interviews with 110 trainees who graduated from the training projects in either 1990 or 1995. Each of these methods is briefly described below. Technical documents, including questionnaires and other data collection instruments, will be posted on the NCEMCH Web site (http://www.ncemch.org/spr/default.html# mchbtraining).

Record Review

A review of the FY 1999 continuation applications for all 101 long-term training grants was undertaken first. This provided evaluation staff with an in-depth understanding of the program and was used to collect data that could be aggregated across projects. In addition, the record review allowed the evaluation staff the opportunity to review information from all projects, not just those that were site-visited. Data collected from the record review guided the development of protocols for the site visits; an effort was made to solicit only that information not available from materials that grantees had already provided to MCHB. General topics on which information was collected included the following:

Figure 2: Diagram of MCH Training Program Logic Model

  • Budget

  • Administrative and organizational structure

  • The educational program

  • Demographic and other information on both current and former trainees

  • Continuing education activities

  • Technical assistance services, including policy work

  • Research and publications of faculty

A form was developed to record quantitative data abstracted from continuation applications so that the data could be aggregated across the projects. However, a number of problems were apparent in this aspect of the study: (1) The variability in the types of training provided in different projects means that the validity of cross-category aggregations are suspect at best; and (2) there is no consistency in definitions among projects, even within the same priority, and thus aggregating data, particularly on such variables as the number of individuals receiving technical assistance or continuing education, is problematic. Nevertheless, this analysis represented the first time that all projects had been systematically reviewed for this type of information, and it provided a snapshot of aspects of the entire program at one point in time.

All data collected were stored in a database in FileMaker Pro for Mac OS Version 4.1 (Claris Corporation, Santa Clara, CA). Quantitative data were analyzed using SPSS for Windows, Release

10.1 (SPSS Incorporated, Chicago, IL), whereas narrative responses were summarized and examined for patterns in FileMaker Pro.

Site Visits

Site visits to training projects were undertaken in order to collect information on the major themes that emerged in phase I of the evaluation, to probe for additional information, and to provide an opportunity to interview beneficiaries of the projects. In order to ensure that the full scope of the Training Program was adequately reflected in the site visits, a set of criteria was developed to guide the selection of projects. The criteria included:

  • geographic diversity;

  • projects at publicly as well as privately funded universities;

  • projects located in universities with multiple MCHB training grants, as well as those in universities with only a single grant; and

  • projects that have been funded for a long period of time, as well as those that were more recently funded. In addition, projects representing each of the 13 priorities were included, and priorities with the greatest dollar investment by MCHB were over-sampled.

The site visits provided rich and in-depth information about the projects. A potential weakness of the site visits was the necessity of relying on the project directors to identify interviewees. Thus, there may be an inherent bias towards a favorable view of the projects. Nevertheless, the fact that many individuals at each site were interviewed enhanced the validity of the findings. Site visits are one of the best methods for developing a clear picture of a project.

Thirty-one training project sites were visited over the course of 8 months. (See Appendix B for a list of site-visited projects and project directors.) During the visits, interviews were conducted with the project director, dean and/or department chair, faculty, current trainees and recent graduates, and recent recipients of continuing education and/or technical assistance. Interview protocols were developed for each category of interviewee.

Data gathered during the site visits were stored in a FileMaker Pro database, and narrative site visit reports for each site were prepared describing the team's findings.

Title V Focus Groups

Because Title V agencies should be key partners of training projects, the experiences of Title V directors in working with faculty and trainees of training grants were explored through a series of focus groups. An in-person focus group was conducted with five state Title V directors at the 2000 meeting of the Association of Maternal and Child Health Programs (AMCHP). Because the project budget did not permit additional in-person focus groups with other state Title V program staff, telephone focus groups were substituted. Title V directors (both state MCH and CSHCN directors) representing 6 of the 10 Health Resources and Services Administration (HRSA) regional offices (regions I, IV, V, VII, VIII, and IX) participated in these calls, along with federal regional office staff. Because most focus group participants in this study knew each other and were accustomed to meeting via monthly telephone conference calls, this approach may have been as fruitful as an in-person focus group.

image of 3 smiling and happy children

Focus groups are an effective way to obtain the opinions of several individuals on a broad array of topics. The ability of participants to build on each other's ideas stimulates thinking and tends to result in comprehensive information. Thus, focus groups have become an important qualitative method for obtaining opinion-based information.

Narrative data from the focus groups were summarized and analyzed for patterns.

Interviews of Former Trainees

A significant outcome of the training projects is the trainees who complete the programs. Consequently, the evaluation included an appraisal of former trainees perceptions of the impact of the Training Program on their professional development. In particular, the study attempted to determine whether trainees who completed the program either 5 or 10 years ago believe that they have become leaders and whether they attribute their success as a leader to the training they received.

A sample of 423 former trainees across 12 training priority areas was generated. Former trainees from the HBCU priority were excluded from the sample, as this category of grants does not financially support long-term trainees. Details of the sample selection process are included in Appendix C. The former trainees were contacted to either participate in a brief telephone interview or to provide written responses to the interview questions, which were mailed to them. Nonrespondents were followed up on with a postcard and multiple telephone calls. A total of 110 interviews were completed, yielding a 26 percent overall response rate, and a 35 percent response rate among trainees for whom addresses and/or telephone numbers were presumed valid.

Both quantitative and qualitative data were obtained from the interviews and stored in a FileMaker Pro database. Quantitative data were analyzed in Stata statistical software release 7.0 (Stata Corporation, College Station, Texas), whereas the narrative data were summarized and examined for trends in the FileMaker Pro database.

The interviews provided the perceptions of the respondents regarding the extent to which they currently exercise leadership and their assessment of the impact of the Training Program on their careers. Budget constraints precluded the use of additional methodologies to further verify the former trainees beliefs about these issues. However, the perceptions of the individuals most directly affected by the Training Program provide strong evidence of its impact.

SUMMARY

This report presents the findings of an evaluation of a large, complex, and multifaceted program. Several qualitative methods were used to describe and analyze the program. Study methods selected were those most appropriate to the particular questions being addressed. The use of multiple methodologies helped validate the findings from each individual method. Findings are presented in the following chapters of this report, along with a set of recommendations designed to improve the MCH Training Program and help it accomplish its mission.

Back to the top of the page.