Recommendation #2: Request legislation for an MCH Training Program advisory committee. In the meantime, organize and convene an expert panel on an annual basis.

Historically, decisions about the Training Program have been made by MCHB staff, following consultation with grantees and others. Such consultation has been extremely useful but limited, and a consistent, field-based source of information through an advisory committee could enhance decision-making and program stewardship. Many individuals have a strong interest in the Training Program and need to be represented in the deci-sion-making process.

The National Institutes of Health have addressed the problem of securing advice on a regular basis through the establishment of policy advisory committees that include professionals in the field as well as consumers. Although the formation of a similar standing advisory committee requires legislation, its benefits to MCHB would be considerable. An advisory committee would provide the opportunity for MCHB to obtain information and suggestions from a diverse group of highly respected individuals who, although unrelated to the program, would become very familiar with it over time.

In the absence of legislation permitting an advisory committee, the MCH Training Program could periodically call together an expert panel, as it has done in the past. However, an ad hoc committee is less efficient than a standing one because ad hoc groups must devote considerable time to learning about a program's goals and activities, leaving less opportunity for generating useful suggestions and advice. The inability of such a group to develop an in-depth understanding of the program may render its recommendations less valuable, unless it could be structured to span a considerable length of time. Nevertheless, a panel would be preferable to no panel at all.

The responsibilities of an advisory committee could include provision of advice on various policy issues, feedback on program management, identification of emerging issues that the Training Program might address, and identification of strategies to communicate program successes.

Recommendation #3: Organize a comprehensive training priority review every 5 years.

A strong and compelling case can be made for MCHB to continue to support each of the existing priorities; all priorities address important MCHB goals and promote the MCH vision. Other funding sources do not exist for the activities currently funded through the MCH training grants. Moreover, it takes several years for a university to “ramp up” a training program, so it is not cost-effective for MCHB to frequently change the priorities.

On the other hand, many training needs are not currently represented in the existing priorities, and there is no process for regularly and systematically reviewing priorities to determine if changes need to be made. As discussed in the introduction, the MCH Training Program addresses qualitatively different types of needs, and decisions as to which needs will be emphasized in the program are ultimately based on values and judgment as well as data.

Several of the existing priorities have been funded for decades, for good reason, and deserve continued support for the foreseeable future. Other priorities continue to document impressive accomplishments, leverage a significant amount of funds, and/or address major problems that promise to be with us for many years to come. All the existing priorities address gaps in training that would go unmet without MCH Training Program support. However, given the changing needs in training, MCHB may need to enhance and provide additional funding to some priorities, whereas others may need to be phased out or refocused over time. A process to regularly review training priorities could be established that would explore the relationship of the needs addressed by the Training Program to evolving MCHB and HRSA goals. Such a process could also assess new training needs based on emerging issues.

The review process might consist of the formation of an ad hoc advisory panel, a literature review, and an opportunity for constituent groups to address the panel. Decisions about altering the existing priorities or adding new ones should be based on answers to the following questions:

• Does the priority promote the MCH vision and address current MCHB goals?

• Does the priority have a significant regional or national impact? (For a new priority, does it have the potential for such impact?)

• Does the priority address a significant problem, one that is not otherwise addressed?

• Is there a compelling case for MCHB support?

• Are there any ways to generate savings in the priority?

Recommendation #4: Utilize and support studies focused on workforce needs and research on emerging issues to help inform decisions related to the funding of different priorities.

Although workforce studies are often fraught with problems and must be based on many (often shaky) assumptions, they are nevertheless important in identifying future training needs and can help inform policy decisions related to the funding of different priorities. MCHB can review workforce studies that have been commissioned by others, collaborate with other organizations that sponsor workforce studies such as those recently supported by the Bureau of Primary Health Professions and others so as to incorporate questions of special interest to MCHB, and occasionally support its own special studies.

Research on emerging issues and reviews of pop-ulation-based data are other important sources of information for the Training Program. Such research can identify new directions for the Training Program and enable it to address new problems quickly. Examples include health problems that appear to be increasing (such as autism), the effects of new technologies (such as gene-based treatments), and service delivery problems (such as lack of access to health care for certain populations).

*Recommendation #5: Develop a comprehensive and multidimensional evaluation plan that includes project-level studies, analyses across program priorities, and evaluations of the entire MCH Training Program portfolio.

Evaluation is a necessary aspect of program management, partly to ensure that a program stays faithful to its mission, and also to identify ways to improve it. A large, complex program such as the MCH Training Program requires several approaches to evaluation. In particular, MCHB might consider the following:

• Require projects to review and critically evaluate themselves, using annual continuation applications and self-assessment tools such as those used for accreditation. One such self-assessment tool is currently under development by the LEND program network.

• Provide funding for projects to conduct their own evaluations. One respondent commented that funders often provide only enough evaluation money to fail:
  “If you put in an expectation for evaluation but do not provide for technical expertise, it's a huge problem.” (Faculty member, School of Health-Related Professions)

Recommendation #6: Support cost-benefit analyses, cost-effectiveness evaluations, and other studies to assess the value of funding different lengths of training.

This study identified some areas requiring further research and analysis. For example, most directors of training projects with short- and intermediate-term trainees see considerable merit in providing training to these individuals and emphasize the effects exposure to MCH issues will have on the trainees’ future careers. However, others believe that the training funds would be more effective if they were focused solely on long-term trainees who, as leaders, presumably will have a larger impact on the field. That is, they question whether it is more fruitful to devote scarce resources to providing a large number of people with a small amount of training as opposed to providing a smaller number of people with extensive training. This evaluation was not designed to address that question, but it is one that deserves attention.

A related issue is the relative value of certain types of continuing education. Many individuals interviewed believe that continuing education is a central component of the mission of an integrated training program, whereas others believe that the money currently devoted to continuing education might provide better value if used for another purpose.

PORTFOLIO POLICIES

Recommendation #7: Include geographic and population-based distribution as explicit funding criteria and develop a technical assistance capacity to assist potential applicants from states that are underrepresented in the MCH Training Program.

With the exception of grants in California and Washington, few MCH training grants are located in the western half of the United States, whereas a disproportionate number are found in the northeast. Because grantees are better able to work collaboratively with the states that are proximate to them, some observers consider such a distribution to be counterproductive to the national objectives of the Training Program. On the other hand, the Training Program aims to support the highest quality projects as determined by review committees, and universities in some states have not submitted applications judged to be competitive, whereas others do not have the necessary university infrastructure to apply for MCH training grants.

Grants in certain priorities need equitable geographic distribution more than do others. If the focus of a grant is primarily national rather than local or within a state, it may make little difference where it is located. For example, if there are only two or three projects in the nation in a given priority, and those projects are small and focused on training a professoriate, geographic distribution may be relatively unimportant. However, the MCH Training Program should strive to balance quality in an application with geographic distribution, especially for priorities with relatively large numbers of grantees, such as LEND and the Schools of Public Health.

Recommendation #8: Implement incentives designed to foster a stronger commitment to cultural competence in curricula and racial and ethnic diversity among trainees and faculty.

Many projects see the value of addressing cultural competence within their curriculum and have taken strides to do so. However, many trainees felt that this was an area in which the projects could improve, especially in light of the relatively few racial and ethnic minorities among faculty and trainees and the desire to provide culturally competent services to the populations they serve.

This evaluation suggests that a truly serious commitment to racial and ethnic diversity pays off. Although attracting minority trainees and faculty members may be challenging in some disciplines, a variety of strategies do appear to work. Projects that show progress in achieving cultural diversity devote resources to it and make minority recruitment and retention a high priority, and they develop creative approaches to ensuring diversity. Projects appear to be most successful when the university as a whole is committed to racial and ethnic diversity.

Several ideas were suggested as methods of improving racial and ethnic diversity:

• Add an evaluation criterion for progress reports and competitive renewals that addresses the ability of projects to attract and retain diverse faculty and trainees.

• Provide increased funding to projects that are able to document diversity.

• Earmark funding for projects to use in special outreach efforts designed to increase diversity.

• Encourage grantees to develop partnerships with Historically Black Colleges/Universities (HBCUs), Hispanic-serving institutions, and tribal colleges and universities.

• Provide consultation to training projects that are having difficulty achieving diversity.

Some of the problems related to minority recruitment and retention must be addressed by the nation as a whole—they are too intractable for MCHB and its grantees to solve on their own. But MCHB can seek opportunities to collaborate with other federal agencies in addressing diversity and it should continuously review the approaches that other agencies have taken for models that might be applicable to the Training Program.

Recommendation #9: Support a series of forums to obtain guidance on modifying existing requirements for the number of disciplines in interdisciplinary projects and appropriate ways of instituting centers of excellence within specific priorities.

Considerable variability exists among grantees with respect to how they interpret and operationalize MCHB’s Training Program requirements. Nevertheless, many grantees believe that the requirements are too prescriptive and that better results could be obtained with greater latitude. These grantees suggest that MCHB should more clearly identify the outcomes to be achieved, but allow grantees the flexibility to identify the methods for achieving these outcomes. Some grantees also suggest that MCHB scale back some of its requirements, especially the requirements for very small grants. Specific suggestions for ways to capitalize on the strengths of different grantees include the following:

• Reduce the number of disciplines required to be on staff, while maintaining the substantive focus. Or, instead of reducing the total number of disciplines, allow projects more flexibility in determining which disciplines to include in the training project; for example, projects could select from several disciplines the ones in which they are strongest.

• Within a priority area, encourage (or require) centers of excellence on specific topics. For example, a school of public health could develop special expertise on cultural competency training, translating research into practice, or other topics that might be identified.

Recommendation #10: Revise the Training Program grant guidances to require evidence of policy and public health foci at both the national and regional levels and to encourage research as one component of a comprehensive program.

MCHB can support only a very small fraction of the training that is needed to ensure that the health needs of women and children are addressed. For that reason, the program emphasizes leadership training as opposed to training for the provision of local clinical services. It is clear from this evaluation that projects that are the most involved in regional and national policy have the greatest overall effect. Thus, MCHB should consider an even stronger emphasis in the MCH Training Program on training for pop-ulation-based policy work.

MCHB should also more strongly encourage grantees to undertake research that will help develop the knowledge base needed to grow the field. A few project directors interviewed in the site visits mistakenly believed that MCHB does not allow attention to research in its programs, rather than understanding that the Training Program does not directly fund research. The MCH Research Program may be a natural source of funding that training grantees can tap into.

BUDGET POLICIES AND GUIDELINES

Recommendation #11: Strive to support at least six projects in every priority, unless there are clearly articulated policy reasons to fund fewer.

In 1999, the amount of funding among the 13 priorities ranged from $357,813 to $18.2 million and the number of projects per priority ranged from 3 to 35. It was not clear from this evaluation that the current distribution is inappropriate, even with the existing disparities. However, the evaluation team did observe that, unsurprisingly, a larger total amount of support, and especially a larger number of projects in a given priority, has a greater national impact. Just as it is important to have a critical mass of faculty focused on MCH to effect curriculum changes within a university, so it is important to have a critical mass of projects to have a national impact, as reflected in significant policy and service change. Although there was insufficient information available on which to base a firm conclusion, the perception of the evaluation team is that the minimum number of projects needed for an observable national impact is between six and nine. Where there are fewer than six funded projects in a priority, grantees may still accomplish their project goals and make important contributions to MCH, but they are hampered in their ability to make a truly national impact.

Recommendation #12: Consider increasing the maximum allowable amount of student stipends.

Trainee stipends in some projects are quite low. One faculty member commented that the stipends for trainees “are so low that they only pay for parking.” Other faculty commented that larger stipends would help increase diversity, especially because the financial needs of racial and ethnic minority students are frequently great and many of these students cannot attend school without adequate financial assistance.

Recommendation #13: Review the different priorities with regard to sustainability expectations to determine if annual increases in grantee budgets should be allowed.

In some projects, flat budgets have led to a situation in which faculty who are supported by the MCH training grant cannot receive raises or cost-of-living increases, creating morale problems. In other projects, faculty must see more patients to receive salary increases, which takes time away from project-based activities. On the other hand, an argument can be made that grantees have the ability to leverage their MCH funds and that they must take responsibility for doing so; grantees who do will flourish, and those who do not must abide with the consequences. However, because the opportu-nities for securing additional funding vary among the priorities, each priority must be reviewed independently on this issue.

*Recommendation #14: Employ a variety of strategies to increase the total amount of money available for MCH training.

Project administrators are concerned about the declining funding available for MCH training grants; some projects have experienced cuts, whereas many other projects have seen their purchasing power dwindle due to budgets that have been flat for many years. While the funds have shrunk, the needs have increased. Several suggestions were made by grantees to address this problem, including requiring a match from universities that receive MCH Training Program funding, developing joint training initiatives with other federal agencies, such as the Bureau of Health Professions, the National Institutes of Health and the Bureau of Primary Health Care, and developing funding partnerships with private groups, such as foundations.

PROGRAM STEWARDSHIP

*Recommendation #15: Develop and implement a communications plan for the Training Program designed to enhance its integration with state Title V agencies and the larger MCH community.

The accomplishments and successes of the MCH Training Program have not been effectively communicated; consequently, the Training Program is not widely understood. The Training Program’s goals are complex, and its activities are numerous. In addition, the needs it addresses are unclear to many people. One result is that the Training Program is not always well integrated with the rest of the MCH community.

A communications strategy could help address these problems. Such a strategy should include development and dissemination of materials that clearly describe the MCH Training Program in lay language and document its achievements, that explain the need for the program, and that show why MCH goals will not be met in the absence of funding for training. The communications strategy should also clearly differentiate this leadership training program from a manpower training program.

As part of the communications strategy, MCHB should collapse the existing 13 priorities into three or four larger groupings. Thirteen is simply too large a number to grasp, and listing all the priorities separately obfuscates rather than clarifies the mission of the Training Program.

One goal of the communications strategy should be to facilitate collaboration among training projects and Title V offices. MCHB should market the Training Program to Title V programs by providing more information about the program to Title V agencies and other groups that are natural partners. For example, MCHB might encourage Title V agencies to solicit MCH training grant recipients as MCH Block Grant reviewers. One project director commented,“The Title V programs need to lean on us more, to know what we can offer.”

Recommendation #16: Implement a variety of activities designed to increase opportunities and incentives for collaboration among grantees, including support of grantee meetings and revision of grant evaluation criteria.

This evaluation found benefits when collaboration occurred among grantees, but it also found that there is considerable room for improvement in terms of collaboration. There are a number of actions that MCHB can take to encourage further collaboration among grantees:

• Support annual meetings of all priorities. Although MCHB currently supports meetings of some grantees, not all priorities meet regularly. Collaboration within a priority area is valuable for fostering joint efforts on distance learning, continuing education, and work within professional associations.

• Support occasional conferences of all grantees. A conference that encompassed all priority areas could facilitate the adoption of new teaching models (such as “mini-fellowships” and certificate programs) and curricula (such as those on cultural competence). Meetings of grantees across priorities could enhance the ability of grantees to learn from each other and to disseminate more quickly and surely new ideas and approaches to leadership training.

• Add an evaluation criterion for progress reports and competitive renewals requiring documentation of collaboration among grantees in the same priority area. There is considerable diversity in the extent to which grantees within a given priority work together, but those who do stress the benefits they derive from collaboration. Collaboration among grantees within a priority can also help foster national policy work.

• Add an evaluation criterion for progress reports and competitive renewals requiring documentation of collaboration among MCH grantees located in the same university. Some grantees are effective at collaborating across departmental boundaries and others are not. Collaboration among MCH grantees within a university can generate positive results and reduce the financial

  burden on individual grantees through the pooling of resources.

Recommendation #18: Review each existing priority in terms of its special issues and modify the guidances as needed in order to improve the ability of grantees to meet MCHB goals.

Some of the Training Program priorities have special issues that deserve attention. Many of these issues are complicated, and no consensus exists on the best way to deal with them. However, MCHB needs to examine each of these issues and propose solutions to the problems they present. Priority-specific issues identified in this study are as follows:

• Behavioral pediatrics: The effect of subspecialty status in behavioral/developmental pediatrics on the behavioral pediatrics priority needs to be examined. How should subspecialty status affect training in both behavioral pediatrics and LEND? MCHB also needs to increase the funding levels of these grants, as they are quite low.

• The presentation of behavioral pediatrics fellows’ research projects at annual meetings is exemplar y and is a model that should be employed by other priorities.
• HBCUs: Some HBCUs have limited (or nonexistent) endowments and are unable to provide such basic infrastructure as computers to faculty. MCHB needs to explore how best to support projects in these universities, which are at a competitive disadvantage and yet are so important in training diverse professionals. In addition, MCHB needs to consider expanding this priority to include institutions that serve a predominantly Hispanic or Native American student body.

The high school component of these projects is innovative; however, it has not been evaluated. MCHB should consider providing funds for evaluation.

• LEAH: LEAH projects are highly focused on policy and advocacy. This is to be commended and should be further encouraged. LEAH fellows should be encouraged to continue to present their research at an annual meeting of colleagues.

• LEND: An important issue in LEND, one that project directors and faculty acknowledge but have not been fully able to address, is the scalability of the interdisciplinary model. Although the trainees receive exposure to an excellent model, and children with neurodevelopmental problems seen through the projects receive the best possible services, a continuing concern is the impossibility of replicating this model outside of the Training Program, given its high cost and the unwillingness of insurance providers to pay. However, there is clearly value in interdisciplinary training, regardless of whether the model can be widely replicated. MCHB may need to reassess its goals with respect to LEND and consider a revised approach to both training and services, or to encourage research that documents costs relative to the quality of services the model provides. LEND projects would also benefit from developing a strong advocacy component that would help promote and sustain comprehensive services outside the setting of the Training Program.

MCHB should explore expanding the focus of LEND to all children with special needs, rather than restricting services to children with neurodevelopmental problems. Such an expansion might help the program to be more fully integrated with state Title V and CSHCN programs. All LEND grantees need to develop partnerships with state CSHCN directors.

The advent of subspecialty status for behavioral/developmental pediatrics and for neurodevelopmental disabilities needs to be reviewed. How should subspecialty status affect training both in LEND and behavioral pediatrics?

The LEND category is unique in terms of the ver y wide differences in the amount of support provided for each grant. Yet, there is no apparent rationale to support these differences. MCHB should review the distribution of the money within the LEND priority to determine if it accords with the overall mission of MCHB.

LEND grantees should provide opportunities for fellows to present their research at annual meetings of LEND directors.

• Nursing: There is a particularly large spread in the amount of support MCHB provides the grantees in this priority. MCHB should consider increasing the amount of funds for the smallest grants. If the spread remains great, MCHB needs to provide a rationale. MCHB should support and foster a policy debate on the appropriate way for the nursing priority to create national MCH leaders and foster changes in the nursing field given the crisis in human resources in this field.

• Nutrition: As in the nursing priority, there are great differences among the nutrition projects in terms of grant size. The smallest grants need to be increased and a rationale provided for any remaining significant differences. Considering the national epidemic of childhood obesity, there is a great need for the development of leadership in this field to support state Title V programs and policies to address this issue. Additionally, the nutrition grants need to reconsider a focus on maternal nutrition, as that appears to have been largely lost.

• Pediatric dentistry: The field of pediatric dentistry faces two major challenges: (1) the need for academicians to train the next generation of pediatric dentists; and (2) tremendous disparities in access to dental care, with a shortage of dentists who will accept Medicaid patients. It is difficult to see how a total of two grants can address both of these needs. MCHB should consider increasing the number of grants in this priority and should clarify the desired outcomes from these grants: Is it to produce academicians, clinicians who will treat low-income children, or both?

• Pediatric pulmonary centers. The PPCs collaborate among themselves in an exemplary manner. However, PPCs also need to develop strategic partnerships with other key MCH partners, including state directors of children with special

health care needs programs and programs for emergency medical services for children.

As with the other interdisciplinary programs, PPC projects need to emphasize policy and advocacy as opposed to clinical services—sever-al grantees already do so.

PPCs should be encouraged to have fellows present their research at annual meetings of colleagues.

• Physical therapy, occupational therapy, and communication disorders: These three categories of grants are small in terms of the dollar amount of the grants, and they are few in number. Several grantees are closely affiliated with LEND programs. Of the projects site visited, one has a strong research component, which is helping to build the field, and another focuses on training doctoral-level individuals who can serve as a new generation of leaders. However, both the size and number of these grants seem to preclude much national impact.

•Schools of public health: The student base of schools of public health has changed dramatically in recent years. Many MPH students now do not have a clinical degree—the norm in years past when the priority was first established—but rather enter the program directly from undergraduate school. How should the new educational background of MCH students affect MCH training in public health? This question needs to be fully considered, and the goals of the MCH Training Program in public health reassessed.

As a group, the school of public health projects tend to have strong, positive relationships with Title V programs. Although the evaluation team saw little evidence of current collaboration among projects, in the past schools of public health have worked together to produce the Association of Teachers of Maternal and Child Health (ATMCH) competencies, a notable achievement. Projects in this priority should be more strongly encouraged to develop collaborative relationships with each other.

This priority lends itself well to the centers of excellence concept, in which different grantees would develop a special focus on a particular topic and provide national technical assistance on that topic.

• Social work: As with some other priorities, there are too few grants in this category to have a strong national impact. However, the focus that these grantees have taken—namely, to strive to influence the profession—is appropriate. Methods include development and dissemination of national curricula, and support of doc-toral-level trainees.

CONCLUSION

The recommendations presented here are designed to improve a strong program with a long history of impressive accomplishments. A few of the recommendations will be relatively easy to implement, whereas others will require a considerable investment of time, energy, and money. However, given its size and scope, the MCH Training Program deserves this attention.

The faculty and graduates of the MCH Training Program account for many of the important accomplishments in MCH over the last half-centu-ry despite the fact that this is a modestly funded program. Some of these accomplishments, such as curriculum development, technical assistance, and the policy work of a committed faculty, are directly and immediately attributable to the Training Program. Others derive from the achievements of the Training Program’s graduates over a period of many years, or they represent an effect of leveraging. Overall, the cumulative impact of this program is impressive.

Traditionally, MCHB funds projects in the SPRANS category that have great potential for true excellence and for building capacity where none previously existed. Such projects fill a unique niche. Although sometimes risky, this strategy has often paid off, as documented in this report. In addition, more funding for evaluation and dissemination of information about the accomplishments and contributions of the MCH Training Program will build further support for funding training programs for MCHB in the 21st century.