Return to the Office of Program Development & Evaluation
Site Map | Search | Privacy Notice | Feedback | TFHRC Home | FHWA | U.S. DOT

Prepared for

National Cooperative Highway Research Program Project No. 20-38A

July 1998

DOCUMENTING PEER EXCHANGE ADMINISTRATIVE EXPERIENCES

Prepared by

Barbara T. Harder B. T. Harder, Inc.

DOCUMENTING PEER EXCHANGE ADMINISTRATIVE EXPERIENCES

NCHRP Project No. 20-38A

DOCUMENTING PEER EXCHANGE ADMINISTRATIVE EXPERIENCES

NCHRP Project No. 20-38A

TABLE OF CONTENTS

Back to the Top

DOCUMENTING PEER EXCHANGE ADMINISTRATIVE EXPERIENCES

INTRODUCTION

Peer Exchanges for state departments of transportation research, development, and technology (RD&T) activities originated with the Intermodal Surface Transportation Efficiency Act of 1991. Through the process of implementing the regulation, state research managers, with guidance from the Federal Highway Administration (FHWA), developed a practical and effective tool to foster excellence in RD&T program management. In particular, this process encourages the states to apply successful program management strategies among the participating research manager peers.

This report documents the administrative experiences of the host states and the peer exchange teams for those exchanges having been conducted through December, 1997. The information in the report was received through interviewing the host state research directors and the peer exchange team leaders. The report is organized in a chronological manner following the general timeline of a peer exchange process. The report begins discussing the host state planning and preparation activities, continues with team leader and team member selection and continues through various aspects of an exchange and concludes with discussion of the administrative benefits and winner strategies as well as an overall assessment of the process.

This report is presented as an exchange tool -- furthering the peer exchange process. The purpose of the report is, foremost, to share administrative experiences from the states having conducted an exchange with those states planning exchanges -- to convey what was successful, what was not productive, the expected time and costs involved, and other informative aspects of the administrative activities involved in conducting an exchange. Secondly, this document is a record of the conduct of the exchanges demonstrating a very-well accepted process of continuously improving the transportation RD&T programs throughout the nation. It is hoped that this material will also present an opportunity for states having conducted an exchange to think through how the next exchange will continue to be a value-added activity for RD&T efforts.

Back to the Table of Contents

Documenting Peer Exchange Administrative Experiences

SCOPE AND BACKGROUND OF THE STUDY

Scope of Study

This study examines the administrative experiences resulting from conducting a peer exchange. In general, the report focuses on the procedures and activities required by the host state organizations as they prepare to conduct their own exchanges -- whether it be the first exchange conducted or the second round. The study did not attempt to identify or collect information on research program management principles learned from the exchanges. The management principles are extremely diverse, however, administrative issues are common to all states. Having documentation on what administrative items were successful, and the process of how exchanges were conducted, was seen as being the initial area of investigation for the overall peer exchange experience.

This study is based on three information sources: 1) responses from a telephone interview of the host state research managers and 2) the peer exchange team leaders as well as 3) examination of the reports produced by the exchange teams. The data collected represents experiences from all of the earliest exchanges conducted up until the end of 1997. All research managers, but one were interviewed (12 of 13) using the Host State Telephone Interview Script found in the Appendix. In addition, all team leaders but one were interviewed (six of seven) using the Team Leader Telephone Interview Script also included in Appendix A. (The two gaps in interviewing were from two different exchanges.) In all cases the combination of data from two and/or three of the above described sources provided more than sufficient material for consideration for this report. Exchanges included in the study are as follows (listed in chronological order):

• South Dakota
• New Mexico
• Nebraska
• Kansas
• Pennsylvania
• North Carolina
• Connecticut
• Montana
• Virginia
• Minnesota
• Texas
• Wyoming
• Maine

The host state interviews covered many aspects of the peer exchange process from the planning and preparation made by the host state and including the selection of the team leader and team members, team building activities, location, length, cost and other items such as, what was notably successful, or what would not be repeated during the next peer exchange conducted by that state. Interviews ranged from 60 to 80 minutes. The team leader interview was approximately one third the length of the host state interview. Questions for the leaders focused

Documenting Peer Exchange Administrative Experiences

on team-building efforts and other items such as report preparation and successful leadership strategies used during the exchange.

Guidelines and Regulations

The regulation instituting peer reviews (now peer exchanges) became effective on August 22, 1994. The authorizing language for these reviews can be found in the Title 23 Code of Federal Regulations Part 420, Subpart B -- Research, Development and Technology Transfer Program Management (Section 420.207 - Conditions for grant approval). The regulations state that a condition for grant approval is:

(b) Each State shall conduct peer reviews of its RD&T program and should participate in the review of other States' programs on a periodic basis. To assist peer reviewers in completing a quality and performance effectiveness review, the State shall disclose to them information and documentation required to be collected and maintained under this subpart.... At least two members of the peer review team shall be selected from the FHWA list of qualified peer reviewers. The peer review team shall provide a written report of its findings to the State. The State shall forward a copy of the report to the FHWA Division Administrator with a written response to the peer review findings.

In November 1994, the FHWA issued program administrative guidelines for the changes in the SP&R Program resulting from the new regulations. In these guidelines the conduct of peer exchanges (then called peer reviews) was described. In that document the reviews were to be conducted once every three years, have two members from a list of approved peer reviewers compiled by the FHWA, may have travel and associated costs be included as a SP&R program Part II line item, and were to produce a report of the activities of the reviewing team Further information was distributed to the states in April, 1996 containing guidelines for the conduct of peer exchanges prepared by David Huft, South Dakota Department of Transportation and Ken Eschmeyer, FHWA South Dakota Division Office. Many aspects of the general administrative profile of exchanges comes from the original thinking of these two individuals as they conducted South Dakota's peer exchange in 1995. (See Appendix B.)

The date on which peer exchanges were to begin was June 30, 1995, yet the FHWA provided a six month period for states to prepare to meet this new regulation. Therefore, the first triennium for the exchanges began on January 1, 1996 and will end on December 31, 1998.

Back to the Table of Contents

Documenting Peer Exchange Administrative Experiences

HOST STATE PLANNING AND PREPARATION

Length of Time Needed to Plan

Over half of the respondent state research managers considered that starting three months ahead of the date for the exchange was adequate time for the active preparatory planning. Several states' managers began their "in earnest planning" at the four month prior mark. Almost all states' managers, however, had been thinking through the issues and had had preliminary discussions with staff and others in their respective departments well before three or four months prior to the exchange. These preliminary activities began as early as a year to eight months prior to the exchange date. In several cases research managers did the major preparation in as little as one and one half months. This timeframe was considered too short. One research manager said:

"I started getting approvals four to six months ahead... at three to four months I set the date, and did actual planning one and one half months ahead -- too short a time for this part, people are busy and you must get on their schedules. As soon as you think of a date, start calling for your team leader."

Participation in Another Peer Exchange Prior to Conducting One's Own

The majority of research managers had not attended another exchange prior to conducting their own. The first exchanges however, had no or few predecessor exchanges, so they could not have attended another. Only two research managers clearly said they would not recommend attending another state's exchange. These people didn't think such attendance was necessary as long as the team members were well chosen and good communicators, or if the manager had first attended the one-day peer exchange training course given by the FHWA Office of Research and Development. Others not having attended an exchange prior to conducting their own, indicated that doing so could be quite helpful. "I might have learned some items to take back." and "It would be beneficial for administrative set ups." and a third said, "I strongly recommend it -- you learn from the experience." Several research managers indicated that having attended another exchange might bias one's opinions especially if that earlier exchange was not particularly successful. This has shown not to be an issue of great concern, considering the high approval rate given by all host state research managers. (See discussion in the last section of the report, Overall Assessment).

Three research managers had attended another peer exchange prior to conducting their own. All three definitely recommended that doing so was very helpful. One of these managers said:

"It really helped me out, especially in organization for my exchange; it helped me know what to expect."

Documenting Peer Exchange Administrative Experiences

Each of these three were very positive about their experience and very clearly expressed that being on another team or being an observer at an exchange was worthwhile.

It is interesting to note that those having attended another exchange prior to conducting their own were very positive supporters of this practice. Those not having attended another exchange were much more likely to say it wasn't necessary. Are the comments only justifying the actions taken? The comments don't seen to indicate this; those who hadn't attended another exchange were open to the idea and thought it might be beneficial.

Several team leaders strongly endorsed attending or observing an exchange prior to leading an exchange team. Several said they thought it was not necessary, but it would do no harm. One team leader who had a great deal of confidence in the peer exchange process said, "If the people really want to get something out of their exchange, they will, regardless if the team leader has attended another state's exchange." The predominant opinion, however, was that this may be a good preparatory practice and may expand the pool of people available for team leadership.

Peer Exchange Training

As the peer exchange training became available, host states' research managers attended the one-day course. The first two exchanges were conducted prior to the training becoming available, although one of these two managers was instrumental in shaping the training course. The peer exchange course was a series of one-day workshops sponsored by the FHWA Office of Research and Development, held in various locations throughout the country during the later half of 1996. Overall there are mixed opinions of the value of the training course. One manager said the training wasn't valuable for things to do, but did show what not to do. For the most part research managers said the training was helpful in some manner. A significant number of the research managers reported that the session on communications and personality traits proved not to be useful A manager suggested that this part of the course should be condensed to a much shorter module. Overall, the comments were supportive of the training and complementary on the procedural guidance given for the conduct of the exchange. One manager said, "The introduction, purpose, and close-out questions were good, as well as were the risks and uncertainties questions, they helped establish the bounds for the exchange." Others said, "At the training I saw how much planning was necessary." and "The training helped the staff to understand why they were doing the exchange."

Other Preparation Items

The host state research managers and the peer exchange team leaders were asked, "What would have helped you be more prepared (if possible) to conduct/lead the peer exchange."

Documenting Peer Exchange Administrative Experiences

• The majority of respondents considered themselves to have been well prepared.
• One of the team leaders credited the host state research manager for having the leader be completely prepared. The research manager had extensive knowledge of the team members and spent considerable time with the leader discussing objectives, issues for consideration, and other aspects of the exchange.
• Several reiterated the message that attending another exchange was or would have been helpful.
• A suggestion was made for the FHWA to make staff available for or to facilitate discussions and brainstorming sessions to determine what is the best way to conduct a state's exchange. There is tremendous value to tailoring the process for the specific state and not addressing the exercise with a cookie-cutter approach.
• One manager said, "I'd like to have had more preliminary study time, more contact with the team members -- see how they do their work."

Back to the Table of Contents

TEAM LEADERS -- SELECTION AND CHARACTERISTICS

Team Leader Selection

Without exception all host state research managers said finding a team leader was not difficult. Also in nearly every case the host state research manager was flexible in scheduling the exchange so that the selected team leader would be available. Many of the leaders were chosen because of an existing personal relationship with the host state research manager. Each of these leaders were peer research managers and included one FHWA representative co-leader. Because of the many qualified research manager peers, the existing relationships provided the host state manager with a pool of people having known leadership and facilitation skills. For research managers not having been in their positions for a sufficient amount of time to develop such relationships, referrals to people of team leader capabilities were readily available from officers of the American Association of State Highway and Transportation Officials (AASHTO) Research Advisory Committee (RAC).

Of the first 13 exchanges conducted, seven people served as peer exchange team leaders. Each person served one time and one person, Dave Huft, served as leader of six exchanges. Considering Dave Huft was instrumental in preparing guidelines for the exchanges, many host state research managers requested his knowledge and skill for their exchanges.

Documenting Peer Exchange Administrative Experiences

Team Leader Characteristics

The host state research managers provided some insight to what they thought were the most important strengths of a team leader. The characteristics most frequently mentioned as highly important were:

! be a good communicator, including being a good listener, able to handle sensitive areas with tact and consideration, and able to draw out those being interviewed by asking non-judgmental (neutrally-based) questions,

! able to synthesize and summarize material well, including being able to see the "big picture" and

! able to effectively facilitate a group, including maintaining control over agenda and time.

Other desired characteristics that were mentioned were:

• experienced at having managed a program like the host state's program -- this was considered as only moderately to less important. What was seen as important, however, was that the team leader understood the nature of transportation research and the general workings of transportation agencies. It was also considered necessary that as appropriate the leader should have an understanding of research partner relationships such as those found with academia or the private sector.
• possess good organizational skills,
• able to make others feel comfortable in a new situation,
• be open to a variety of ways to accomplish goals, including not using own experience as the best, only, and right way to do a task, and
• be at ease with a broad range of people including top administrators and field personnel.

From the perspective of the team leaders, the most important characteristics they identified were similar to those mentioned by the host state managers. With the leaders concurring, several of the characteristics stand out: being a facilitator, including getting people to participate and contribute as well as at the same time keeping to a schedule and maintaining focus of the discussion; being a synthesizer; and being able to handle well, many different kinds of people.

Back to the Table of Contents

Documenting Peer Exchange Administrative Experiences

TEAMS -- MEMBER SELECTION AND COMPOSITION

Team Member Selection

There were several very important criteria given by the host state research manager for selecting team members. Most often mentioned was the people selected were known to be knowledgeable professionals in their respective areas of expertise. Secondly and closely linked with expertise was understanding that the person would be forthright in his or her discussion. Host state research directors looked for individuals with whom they had an already established rapport, thus starting the team building process at the point of team member selection.

The host state managers said that considering the activity was an exchange, knowledge of research processes and state agency research programs was very important. The concept of sharing experiences was a true goal of the exchanges. The requirement of being able to contribute successful ideas, methods, and procedures was a very real requirement placed on the team selection process. Not as important as research program knowledge was the existence of a related technical expertise, however respondents (research managers and team leaders) determined that technical expertise was necessary.

Team member communication skills were also considered as being a very important characteristic. Team members were chosen not only because of expertise, but because they could convey meaning well. The teams were required to discuss a wide variety of issues with a broad group of individuals from various levels within the host agency. Well honed communication skills helped the team in every aspect of the exchange.

As with team leaders, it was not considered critical that a team member have been involved in another peer exchange as a prerequisite for selection. A number of the research managers and leaders indicated peer exchange experience is always helpful, but not necessary. It was noted that persons having been part of another exchange tended to be more "up-to-speed" on the general process of an exchange and may have better understood the need for quickly building a cohesive team.

Other characteristics that were emphasized present wise counsel. One research manager who also served as a team leader said team members must also be learners. Team members were expected to discuss, share, synthesize, and most importantly learn from the experience. Several managers said they was looking for people who could "see the big picture" as well as be able to deal with details of a program. One other person indicated that people who could think beyond the obvious would be an asset to any team.

Documenting Peer Exchange Administrative Experiences

Once the team was selected and members accepted the invitations, there were remarkably few cancellations from team members. Only in two exchanges did an individual have to cancel. In those exchanges, the host state research manager had sufficient talent on the team and did not make any effort to replace the person who had cancelled. One host state manager changed the date for the exchange to avoid a schedule conflict for the team leader it wanted, but this was done early in the exchange planning activities.

Team Composition

On the average, peer exchange teams consisted of eight members including those that were observers. Of these eight, two or three people were from the FHWA. In every exchange the FHWA division offices were represented and in almost every exchange the regional technology engineer was present for all or a considerable part of the exchange. FHWA representatives from the Turner-Fairbank Highway Research Center also were invited as team members for a number of the exchanges. Observers invited to exchanges tended to be FHWA representatives. Although often some participants were originally considered observers, the prevailing attitude at the majority of exchanges was that anyone present was expected to participate. Observers were given the same status as any of the official team members. When asked directly, most host state research managers said they didn't think there should be differences in those invited to participate -- no observers, all invited should be an equal part of the team.

In general the typical team profile was:

• up to four state departments of transportation research managers,
• one or two representatives from academia -- an institution performing research for the host agency or a well respected transportation researcher -- or one private sector representative; and
• two or three representatives from the FHWA.

In the first 13 exchanges conducted, those examined in this study, over 70 individuals participated as team members (including host research managers and team leaders as well as observers) with 20 of the total being from the FHWA.

As the list of qualified peer exchange participants compiled by the FHWA became available, host state research managers met requirements of having two members from this list on their teams. Host state research managers indicated this requirement had no or very little effect on team member selection. In general, the managers' choices for team members were people that were also on the FHWA list. This requirement became a "non-issue" with the research managers interviewed. Several research managers wondered why having two from the list was a requirement. There was feeling that the list was one more item of bureaucracy that was

Documenting Peer Exchange Administrative Experiences

unnecessary. For host state managers who are not familiar with their peers, it was noted that such a list could speed up the process of team selection and provide some consistency to the level of expertise of the team.

Knowing that familiarity with research and research program management was considered very important, research managers were asked about the type of program experience that was a desirable for having on the team. Host agency managers were asked how important it was to have team members from states having similar programs in size (dollars expended), geographic location, or type of operation, or in other respects similar. The overwhelming response was that host research managers were looking for a diversity of input and purposefully organized the team to have a mix of perspectives. The host research manager wanted balance on the team more than they wanted programs similar in size, although team members were selected knowing that they could relate to the host agency program -- not being so much larger that there was no common ground or so much smaller that all seemed to be beyond comprehension to the team member. The exception to this model is with the very small state programs. They tended to invite research managers from programs of a relatively small size.

For team composition, the second most important team criteria related to research or research program management was experience with a program that has a similar type of operation, such as in-house research, contract research, or university research The states that perform considerable in-house or contract research were sure to have several team members familiar with these processes. Similarly, if much research was done with an academic institution, then several team members were selected from states having a similar approach to their research efforts. Although these areas of expertise were included on teams, the diversity of membership was still a primary objective. Most host states did not consider it important to have team members from states within their geographic region of the country. Some thought was given to extreme differences in location or climate which could have introduced disconnects in technical areas. Several states' research managers indicated that teams should include research partner representatives particularly from the private sector and university areas. (See Appendix C for Team Member invitation letters and other correspondence to the Team Member from the Host State.)

Back to the Table of Contents

TEAM BUILDING

For a majority of the peer exchanges, a team telephone conference call was the first team building activity. This telephone call, for the most part, involved 1) logistics associated with arrival and attending the team meetings and 2) strategy for the conduct of the exchange. Often the focus areas for the exchange were discussed and team members were solicited for input to them at that early date, usually two weeks or more prior to the exchange. By the time the conference call occurred the team members had had an opportunity to review material sent to

Documenting Peer Exchange Administrative Experiences

them by the host agency. The effect of the conference call was that it helped to place the team members all "on the same page" when they arrived for the exchange.

Each peer exchange had some activities that were designed to draw the team closer together. Over half the exchanges examined in this study incorporated a team dinner on the evening prior to the start of the exchange. The dinners were informal, social affairs and were designed to let the team members get to know one another. Other exchanges were preceded by informal meetings in which the team had opportunity to establish an identity. Some of these meetings were held the night before the official start of the exchange and several as the first activity of the exchange meeting. Most all exchanges had some sort of organizational time as the initial activity. One team leader said, "We worked hard the first morning. It was very important to get all the members on board and to work out any differences. We got expectations in line so we were all together -- the pre-meeting was very important." A number of exchanges were held in part in a resort setting. The nature of getting away to a relatively isolated spot assisted in developing a team esprit de corps.

All host state research managers said building a team and providing opportunities for promoting its cohesiveness were very important. The dinners prior to the exchanges proved to be very successful events; enjoyable for the team members as well as accomplishing the task of developing the team.

Back to the Table of Contents

INFORMATION RESOURCES FOR TEAM AND AGENCY PARTICIPANTS

Information for Team Leaders and Team Members

For all exchanges the team members including the leader received a package of materials from host agency. These materials consisted of all or some subset of:

• invitation letter
• purpose and objectives of the exchange and/or focus items for discussion,
• research program manual and the most recent SP&R Part II Work Program,
• organization charts for the agency and the RD&T group,
• department strategic plan,
• summaries of questionnaire responses from RD&T customers,
• preliminary agenda,
• sample research reports and products of research,
• program status documentation, and
• personal service agreement and expense vouchers.

Documenting Peer Exchange Administrative Experiences

Information was conveyed to the team by letter, fax, email, and telephone. There was considerable contact with the team through all of these mechanisms prior to the conduct of the exchange. During the team conference telephone calls, team members were encouraged to request any material they thought was appropriate for enabling them to better understand the workings of the RD&T function in the host organization.

One agency research manager succeeded in having his chief administrative officer send a letter of invitation to the chief administrative officer of the department of transportation employing the selected team member. This process gave credibility to the peer exchange activity for both the host agency as well as the agency from which team members were drawn. The host research manager as well as a team member (who subsequently hosted an exchange) reported this technique was very helpful for their experiences.

In general for all exchanges the material sent to team leaders and other team members was the same. For one exchange, the observers were sent a less comprehensive package of materials. In another, they were only sent an invitation letter -- in this case the observers were from the FHWA and already had the general material that had been sent to the team.. (See Appendix C for correspondence to Team Members.)

Information to Host Agency Participants

Information to agency personnel focused on the purpose of the peer exchange and the importance of the agency personnel's participation. Agendas, itineraries, and in some cases information about the RD&T function was sent with the invitation to participate. In one state, the agency personnel received the same notebook as did team participants. However that exchange focused solely on the research function and did not perform an agency wide RD&T customer interview exercise.

The host state research manager used a multitude of communication methods to enlist the participation of members of his or her agency. Personal visits to agency managers, invitation letters, fact sheets, and all sorts of confirmation acknowledgements were used. One manager prepared an advisory to his agency managers that contained the purpose of the exchange, the itinerary, the tentative interview schedule, and ten example interview questions. In every case the research managers were diligent about reminder memos and last minute logistics. All research managers indicated they had some formal schedule for distribution of materials to the agency staff. The agency staff tended to be divided into two major groups. One group being those scheduled to be interviewed and the other being senior managers invited to the close-out session. Materials and the approach to each of these two groups was highly customized for their respective needs.

(See Appendix C for invitation letters to agency staff.)

Back to the Table of Contents

Documenting Peer Exchange Administrative Experiences

LOCATION OF PEER EXCHANGE

The peer exchanges were held in a variety of DOT offices, with two primary sites being the RD&T offices and the central headquarters offices. All exchanges used a facility at the DOT for some part of their meetings. Most exchanges conducted their close-out meeting at the DOT in a location readily accessible by senior management and research oversight committees. Nearly every exchange had more than one location for its full series of meetings. For some portion of the exchange, about half of the exchanges used a meeting room in the hotel in which team members stayed while at the exchange. Many of the pre-exchange informal strategizing meetings were held at the hotel the night prior to the beginning of the exchange. Only two exchanges used facilities in a resort setting. These host research managers appreciated the more remote locations which helped focus the team. Yet both said the travel to that secondary location needed to be planned carefully considering the travel time could be used as an additional opportunity for the team to interact. Of the varying locations used by the host states, several conducted parts of the exchanges at a university or at the offices of the Local Technical Assistance Program, Technology Transfer Center.

Most meal times were not officially parts of the exchanges, but restaurants rapidly became important locations for the teams to discuss issues prompted by events during the formal meetings. Some host research managers incorporated times at meals for team building and for continued discussion of exchange topics. One exchange used meal times as a more formal part of the exchange agenda, using the time as opportunities for presentations by invited speakers.

In nearly every exchange, host state managers considered the informal time critical to the success of the exchange. Finding a location that provided an atmosphere for open discussion in a more relaxed setting was an important part of each exchange.

Some of the comments regarding the advantages or disadvantages of having the exchange in the selected location(s) follow:

Advantages:

"We had lunch together, not an organized thing, just did it -- worked out very well, it was time for informal but important discussion."

"Having our exchange at the main office was a plus for us. We got a very nice meeting room, which allowed flexibility for interviewees' schedules if changes had to be made. The team report material generated on laptops was just taken to the department word processing resource center. We had our staff available and

Documenting Peer Exchange Administrative Experiences

conveniently located. The senior staff was only one floor away -- this was a positive. If they had to travel to get to the close-out meeting, they may not have come."

"The major advantage of being at the DOT offices was it optimized time for those being interviewed."

"The DOT location provided easy access to all research personnel and gave an opportunity to tour the facility which improved the team's understanding of the state's research program."

"At a center away from the office, we could develop relationships better. In a more relaxed atmosphere, people are more free to say what is on their minds. If one can mix the human side with the business side, results are always better. I had a second motive, that being getting my own staff team away for some time together."

"The university is a partner, the atmosphere was better, logistics easier, and it was more convenient."

There were remarkably few disadvantages articulated about the locations chosen for the exchanges. Several comments were:

"We found no disadvantages in our selection of location."

"The change of venue from one location to another allowed the exchange team to decompress from stressful interviewing, yet the disadvantage was the time spent in moving our location -- this wasn't a big issue."

Back to the Table of Contents

LENGTH OF PEER EXCHANGE

The average length of a peer exchange meeting was approximately three days of elapsed time, which included meal time and other informal sessions. The full series of exchange meetings ranged from under two days to four days. Most every exchange had one late night session -- the night prior to the last day of the exchange -- which was dedicated to preparing the close-out meeting report.

Seventy five percent of the host state research managers considered the length of time they committed for the conduct of the exchange "about right." The remaining 25 percent considered

Documenting Peer Exchange Administrative Experiences

the time too short, and each said they could have used another half day. One manager would have used the time to synthesize the material learned. This manager felt rushed to get the comments and discussion into a report format. Another, indicating more time would have been useful, said that more time was needed between interviews for exchange team assessment.

Back to the Table of Contents

CONDUCT OF PEER EXCHANGE

Scope of the Exchanges

Most of the peer exchanges were broad-based exercises which concentrated on the interaction of the RD&T office with its internal department customers and/or external (outside-the-department) partners. Two exchanges focused exclusively on the research group, and one other focused on the research group, but expanded its attention to other areas in the department that were involved in research performance.

Interviews

The broad-based exchanges required interviewing a wide variety of department personnel and outside-the-department research program partners. The host research managers mentioned the following types of individuals as those being part of the interview process:

• CAO (Chief Administrative Officer -- Secretary/Commissioner/Director)
• Deputy(ies) to CAO, Chief Engineer
• Staff to the CAO
• Bureau or Division Managers
• Project Managers
• Technical Professionals -- Engineers, Planners, Administrative Staff
• Research Managers
• Researchers
• Others outside of agency: Academia, Private Sector

For all broad-based exchanges nearly every type of person listed above was interviewed. There were a few exceptions. Only in several exchanges were CAOs interviewed directly and representatives of the private sector partners, e.g., consultants and representatives from research laboratories were generally not included. One research manager described the interviewing process as:

"We did our interviewing in five groups, administrators (director, deputy director, and my boss); Research Advisory Committee (division heads); technical contacts

Documenting Peer Exchange Administrative Experiences

in department divisions (bridge, roadway customers); infrequent customers (where we had a chance to do some marketing); and researchers from the university."

Three exchanges particularly included people from their non- headquarters district offices. The exchange presented an opportunity to further integrate the research program into the operational part of the department business.

On the average interviews were scheduled for approximately one hour with a short period of time between each interview session. Many of the host state research managers indicated that they needed more time between the interview sessions to assess what had been discussed.

With the exception of one exchange, all interviewing was done in groups of approximately two to five people. The one exception was an exchange that focused only on the research group and interviews were conducted individually with each of a series of senior researchers.

The average number of people interviewed in the broad-based exchanges was 30 people. All host state research managers determined this number was "about right." One manager did note that some of the presentations/interviews got a bit long.

Getting Agency Personnel to Participate

Two messages that nearly every host state research manager conveyed in his or her internal correspondence to agency staff were 1) the peer exchange is an opportunity to improve the research processes, and 2) it is a confidential session with the peer exchange team. Two research managers gave opposing recommendations on how to handle the regulatory nature of the exchange. One research manager said the fact that this was a requirement helped get the attention and participation of his department's more senior managers. One other host manager purposefully did not mention that the exchange was a requirement because that took away, in his opinion, the emphasis he wanted placed on the process improvement focus. (See Appendix C for example correspondence.)

All host managers responded that there was some special effort made to get the participation from agency personnel. The correspondence was tailored to the various people; face-to-face invitations were given, and a host of other methods were used to assure strong participation in the interview and close-out sessions.

Documenting Peer Exchange Administrative Experiences

The following are some quotes taken from host state research managers' comments on agency staff participation:

"Get them to come!!! Do anything you can to lock in a commitment from the top people."

"The invitation letters had the 'flavor' of -- this is an opportunity for you to tell us what we could do for you; the interviewed people were made to feel very positive about coming and talking."

"We presented this as an opportunity to improve research. Their presentations would be an asset to the process. We didn't give the impression that this was something we had to do because of the regulation."

"We spoke of the regulatory nature of the exchange, that helped. People could not casually dismiss the invitation -- says, 'this is serious'."

"We not only included the reason for conducting the exchange but what they would get out of the time spent -- what would be learned by all."

Research managers and team leaders responding to the questions for this study considered the participation of the CAO in the close-out session as reasonable or desirable. In several cases, the CAO was not the type of manager that dealt with details of the operation and was not expected to attend the close-out meeting. However, the close-out meeting was seen as an opportunity to present the research program to senior managers. The general consensus was to do all one can to see that these senior managers attend the close-out session. Streamlining logistics, scheduling appropriately, and adequate up-front information about the purpose of the exchange were all seen as necessary to facilitate the senior managers' attendances.

Atmosphere of Trust or Threat

In three exchanges there was some initial misunderstanding about the nature of the exchange process. One of the earliest exchanges required considerable discussion with the senior managers to change perceptions -- that the peer group was not coming in to tell the department how to run its business. The initial discussions focused on the exchange nature of the experience and dispelled the impression that the exercise would be a compliance review. "There was a happy ending, by the conclusion of the exchange this manager was very pleased and wrote a personal letter of thanks to each team member." One of the other host state research managers said the research office staff had misunderstandings about the purpose of the exchange. Once these people attended the peer exchange training course, "they felt good about what they were

Documenting Peer Exchange Administrative Experiences

going to do." One other research manager stressed the importance of the team leader taking the time to discuss the true exchange nature of the meeting to everyone interviewed. Some of the agency personnel expected a review regardless of the explanations that were given in the pre-meeting correspondences.

All other exchanges were conducted with no appearance of threat to those being interviewed or to the agency. In fact, once the initial hurdles discussed above were crossed, the exchanges took on the atmosphere of high trust, promoting open and frank discussions by all. Many of the team leaders and host research managers remarked on the productivity of the experiences because of the openness with which all approached the exchange.

Objectives of Exchange Articulated

In all exchanges, the objectives of the exchange were clearly articulated. The team leader and members were given ample opportunity to review these objectives prior to their participation. The objectives were articulated to department senior managers as well as those being interviewed. Several research managers stressed the importance of developing concisely worded objectives. That helped others in the department have a positive impression of the purpose for the exchange. For all exchanges, the objectives are presented in the report produced by the exchange team as well as were emphasized during the close-out meeting.

A number of host managers also detailed focus areas to help steer the direction of the exchange meetings. These focus areas provided a structure for the research manager to organize his or her personal implementation agenda resulting from the items learned during the exchange.

Computer Resources

Although exchange host states were geared up for giving computer assistance for note taking and report production, in many cases the exchange team members' laptops were put into service. One research manager said, "We used the laptops brought by team members, we didn't use as much of the in-house resources as I thought we'd need." Notes were taken during the exchange on laptops and in general the host state provided systems resources to produce the final report.

Report Generation

Producing the report was a rushed activity for all exchange teams. For the most part the late afternoon and evening prior to the last day of the exchange was committed to producing the report. A frequent comment from host managers was to keep daily notes such that they could be easily incorporated into the final report. All host state research managers and team leaders agree that producing the final report prior to the end of the exchange is necessary. The report is then

Documenting Peer Exchange Administrative Experiences

available for presentation to the department senior managers and FHWA as a product of the exchange. There is no after-the-fact guessing at the intent of comments, and all team members can begin implementation of their respective agenda items.

In all but one exchange daily notes were taken. In about one third of the exchanges there were host state staff or others specifically appointed to take notes. In the other exchanges various members of the team took turns taking notes or shared notes they took with the rest of the team when the time came to write the report. For report generation, many of the host states had what is being called the "Huft model format." This is a diskette which includes a final report from an earlier exchange showing a basic model format for a final report. It is concise and presents material in a brief bullet point format. Having a framework for the report greatly assisted in getting material written. Several host state managers said it was helpful to see what a final report looked like prior to being responsible to produce their own. There was a wholehearted acceptance of producing something in a concise bullet point format.

The host state research managers and team leaders were asked if the peer exchange team could have been any more productive in generating the report. Over half of the respondents said that there wasn't more that could be done. One manager's response was, "The key is to compile notes as you go along, at the end, condense these during a 2 to 3 hour session to draw the report together -- it's an incremental element of work that just must be done." Others said, "There is a certain amount of work to be done, spend one evening producing the report and get it done before the close-out meeting." and "Keep it concise, the process was hurried, the report could have been fine-tuned, but we were satisfied with the product." (See Appendix C for Example Report Format)

Every host state research manager agreed that the level of detail of the exchange and the report met their needs as well as those in their agency. All were complementary of the concise key-thought type of reporting.. For most exchanges, the report was distributed to every participant in the exchange. Managers said the report contained enough detail to go forward and that extravagance is not necessary.

One of the requirements of the peer exchange process is to transmit a copy of the peer exchange report to the FHWA division office. In all cases either the FHWA was part of the peer exchange team or attended the close-out meeting -- thus receiving a report. For the most part no formal letters of transmittal were written; the reports were transmitted personally. One state prepared a short summary of the report and exchange and submitted that document.

Documenting Peer Exchange Administrative Experiences

Back to the Table of Contents

FOLLOW-UP ACTIVITIES

Nearly all peer exchange reports contain implementation items for each of the team members. Often some peer contact will help team members accomplish their stated action items. Seventy-five percent of the host state respondents said they have or were planning to conduct some activity to follow the exchange for the purpose of team member encouragement and accountability. The primary means to accomplish this were a telephone conference call or letter. Host state research managers thought an appropriate time for this follow-up to be done was between six months and one year after the exchange took place. One manager said, "A follow-up is good for accountability. I didn't do a conference call with the team. I made one-to-one contacts which was more hit-or-miss, I should have done follow-up formally."

Back to the Table of Contents

OTHER ADMINISTRATIVE ISSUES

Time, Cost, and Reimbursements

The amount of time committed to planning and preparing as well as for post-exchange activities varied considerably among the exchanges examined for this study. Most research managers did not track the time it took them and their staffs to host the exchange. The best estimates given by a majority of the research managers averaged a little over three weeks of effort including both the manager and the staff. This amount does not include the actual time spend during the exchange sessions. It is useful to note that the range of times reported was 5 days to over 50 days. The distribution was relatively even among the values: several in the low end of the range, a number in the middle of the range, and several on the high end of the range. For most exchanges the work was divided between the research manager and the staff, with no research manager reporting spending more than 15 days on the effort -- exclusive of the time spent during the exchange.

For the host states where costs were tracked or an estimate could be made, the average direct (or out-of-pocket) cost of the exchange was $4,000. The two extremes were $500 for one exchange and $9,000 for another. Even with these two extremes excluded from the average, the remaining exchanges averaged $3,875 each.

Five host states specifically mentioned that they included a line item in their SP&R budget to reserve funds to pay for the exchange. This was not a direct question in the host state research manager telephone interview, so other states may have used this convention. Eight of the thirteen states indicated there were no administrative problems associated with paying expenses for the peer exchange team. Several state research managers reported there was some work in setting up the financial processes. One noted that each state process is different. "You must start

Documenting Peer Exchange Administrative Experiences

early to be assured you meet the time frame required by the various agencies." One manager also reported the payment process was difficult due to his own state's regulations of having to treat everyone as a service contractor.

Administrative Lessons Learned

The following are quotes from the host state research managers:

"We spent an evening just talking, blunt and honest, constructive. It was one of most useful times. It was not for outside audiences, and nothing was written down. There has to be some off-the-record time with the team, to purposefully plan this would be good -- perhaps after the 8 to 5 discussions at night. Everyone learns."

"Start planning early, and give enough lead time"

" For some of the people being interviewed that had not used research, they felt a bit uncomfortable, perhaps calling them before the exchange or having a pre-meeting with them would have helped."

"Get to the people that do the paperwork (not just the chiefs and bureau directors) include fiscal and contracts divisions if you are interested in process improvement."

"Do real-time recording of notes."

"Develop a contracting mechanism so team members get a fair and timely expense reimbursement. It is absolutely critical for people's out-of- pocket expenses to be paid in a timely fashion and fairly." and "Direct bill team expenses as much as possible."

"During the exchange, go through the organization chart several times, it benefits the panel."

"Leave some time for reflection during the exchange."

"One thing was driven home, you can say all states are different, but you don't know this until you are around a program -- this experience was very informative. It was truly an exchange -- there is still some tension about this in several states. They are looking at this process as an audit -- it isn't, it is an exchange!

Documenting Peer Exchange Administrative Experiences

Frequency of Future Exchanges and Participation

Host state research managers and team leaders on the average said they would consider participating in a peer exchange about two times per year. Many research managers and experienced team leaders said they would consider being a team leader. One person who was a host state manager as well as a team leader said he would participate as many times as asked. He thought the opportunity to learn from one's peers would provide a significant benefit for his program. Another host research manager and team leader stated, "My participation would depend on the goals of the exchange. If the state really wants an honest assessment of the program and how it an be improved, I'm interested. If it is just to check off a box to fulfill a regulation, then it's not for me."

The vast majority of host state research managers and team leaders said they thought conducting the exchange every three years was "about right." Several said every three or four years was a good cycle, and two said 5 to 7 years. The host state providing the farthest horizon was the state having spent the most amount of effort in conducting its exchange. The primary concern of all the research managers was being able to address the full set of action items identified at the exchange in the three year period between exchanges.

Back to the Table of ContentsBENEFITS

The host state research managers and team leaders were asked what benefited them the most from the peer exchange experience. The following is a list of their responses:

• the exchange focused attention on research in the department, this was 3 hours of senior management thinking about research -- a very good opportunity;
• because customers were interviewed, found out what they thought -- important
• the team helped my thinking outside the box -- talking with the team, listening to them, gained a great deal;
• the positive feedback was very important, everything was presented in a positive way, confirmed what we are doing, showed us areas for improvement;
• got excellent input from other research directors and people interviewed; the fiscal people made things simpler for us, improved communications;
• executed a major cooperative agreement about 4 months after the exchange; it was based on what was learned from two other states' research managers participating in the exchange -- the new agreement was a hybrid of these two states' agreements --- we took a 180 degree turn based on what we learned from the exchange;
• gained a greater understanding and sense of awareness, adopted new means for disseminating project results, learned how important it is for reporting in a timely way;

Documenting Peer Exchange Administrative Experiences

• the number of people the exchange drew in to participate helped strengthen the program, people were pleased to be invited.
• spending three days in very focused wide ranging discussion on how to lead a research program from people who have done it for many years... being with people can learn from and with whom can exchange experiences and information is most valuable;
• learning from others who have been doing this longer than I have, finding out I'm not alone in this, I have partners in the process; and
• found out I had a resource I haven't tapped into before.

Back to the Table of Contents

WINNER STRATEGIES

The following list identifies items that research managers and team leaders would do again in the next peer exchange they conduct or lead:

• keep format as a free flowing type discussion;
• definitely have report to top managers as a face-to-face experience, one-on-one was very good, and conduct the exchange at the main office, offered lots of advantages;
• a conference call prior to meeting is pretty critical -- important to have materials sent to the team so can talk through goals and other preliminary items during the conference call;
• for interviews of one hour, take 15 minutes in between for synthesis and assessment;
• have real-time note taking;
• incorporate field views of RD&T efforts; helped team understand program;
• the exchange was a winner, we will continue to host an exchange even if it is not in the next legislation;
• include staff interviews -- as many as possible, as well as contracted principal investigators;
• involve all transportation sectors -- highway and non-highway;
• recruit excellent team members, especially the team leader;
• include "evening-before" activities and team dinner evening before presentation (with working session to hammer out report);
• take time in the beginning of the exchange to say what is expected for the next several days; identify top issues of concern to senior management and the issues detailed by the RD&T group;
• limit interviews to be a reasonable amount of time for customer interaction;
• sent an invitation letter to the CAO of the states of selected team members, this went a long way in gaining credibility inside the organization as well as with the other states;
• have private sector, university involvement, and FHWA as full team member, diversity in the team was an important thing;

Documenting Peer Exchange Administrative Experiences

• liked having the FHWA region representative there, it was a chance to get to know him better; ! have more senior manager involvement, some folks would liked to have had opportunity to participate;
• I'd take a refresher on the training if I did this again.
• include academia and private sector researchers as part of the team; and ! interview people based on exchange objectives.

Back to the Table of Contents

ITEMS FOR CHANGE IN ROUND TWO

The following list identifies items that research managers and team leaders would consider changing for the next peer exchange they conduct or lead:

• I wish had involved the research staff more during the conduct of the peer exchange --had two experienced research managers on the team, it would have been good for my staff to learn from them.
• I would have liked to revisit some issues and didn't have time -- I think this has been a common failure -- I'll make each area a shorter time rather than longer...
• I'll try to interview more users of the program next time, we looked at the first level of users this time and would try to go down one more level to designers and similar technical employees.
• We didn't interview the CAO, I'd like to do this next time -- would be productive.
• We had no private sector representation on the team, I'd get some next time.
• I'd narrow the focus and go into one area in more detail -- because it will be second time doing an exchange.
• We didn't have enough time for informal interaction. I'll try to have an evening activity in the middle of the meeting for some informal interaction -- it is a trade-off for time;
• Senior managers weren't sufficiently involved, I'd invite chief engineer or deputy secretary for highways to cover entire exchange.
• There wasn't enough time to have team interact with the department staff -- I'd like to invite policy director as well as chief engineer and bureau directors to meals, so can interact in less formal situation.
• We were too heavy in highway mode, add other modes next time.
• 2 or 3 per interview group was optimal, 4 too many, will change next time.
• We were too rushed, I'll do larger interview groups and go to one and one half hour sessions.
• We didn't allow enough time for evaluating results and effectiveness analysis -- we will add this time to our next exchange.

Documenting Peer Exchange Administrative Experiences

• We dumped a lot of data on panel in a hurry, too much dumping and not enough interacting.
• Next time we may be looking for review rather than an exchange.
• I will include an executive staff member as observer or team member. This might also be especially good for states that need executive level buy-in.
• We didn't have any team wrap-up, include this next time -- finish well.

Back to the Table of Contents

ADVICE TO FHWA

The following are thoughts presented by both host state research managers and team leaders:

• FHWA should do a peer exchange as well.
• FHWA should definitely be involved and be there to help and support the process, they should be invited to be on the team, if they had not been on the team the exchange would not have been as productive.
• Continue to emphasize this is an exchange, encourage flexibility.
• Do not develop a cookie cutter approach to the process -- each state has its own profile; make sure the peer exchange process is responding to the states' as well as national priorities.
•  Offer training periodically from now on.
• FHWA has done a good job of keeping us informed, they've given information at RAC meetings, presentations, and training, there has been good advance material.
• This is such good experience for the states, don't let people (states) off the hook, I think there are states waiting for this to go away...
• Continue in present format, don't change it yet, it's just starting to walk, help a bit more in follow-up.
• Because the process was so successful, the FHWA should encourage other technical areas to do the same.
• Let division offices in partnership have a dialogue with state... collaboratively figure what is best approach for the next exchange; perhaps through the division partnering they could custom tailor a team and the objectives.
• I am is pleased that the FHWA has let this evolve, FHWA has opted to give freedom to the peer exchange process rather than tightening up to be more of a review. This has given the states the opportunity to improve themselves without any heavy handed direction from FHWA.

Back to the Table of Contents

OVERALL ASSESSMENT

Without exception all host state research managers and team leaders were very positive about the peer exchange process. A representative sample of comments given are:

•  Excellent, nothing negative at all, it's a good thing for everyone involved.
• If you are doing the exchange just to fulfill the federal regulation, don't bother, that will be a waste of time, however if you really want material to help you improve your program, then by all means do it.
• Positive impression, this is a much more effective system to help states make sure their research programs are relevant and focused than the classic government review that is a "check the box" type review; additionally I feel more comfortable with being a team member at this time.
•  Very helpful, very positive and constructive.
• It was a lot of work, but it was worth it; we learned a lot of good items. It was a good experience for the staff -- good news, the exchange is a good experience.
• It was a great experience, we really benefited from the exchange. It clarified where we were weak in our program, and it was an extremely helpful way of finding out items to work on. We implemented all five major recommendations of the team.
•  Very enlightening, definitely recommend to others.

There were no downsides mentioned to the exchanges. All expressed the positive nature of the learning experience. Additionally, many of the host state research managers expressed their intent to conduct another peer exchange in three or four years whether the exchange was a continuing requirement or not.

Back to the Table of Contents

APPENDIX A

Peer Exchange Data Collection -- Research Manager

Host State Research Manager Telephone Interview Script

Peer Exchange Data Collection -- Research Manager

Peer Exchange Data Collection

HOST STATE TELEPHONE INTERVIEW SCRIPT

Directed to the Research Manager of the host state:

1. State:

2. Dates of Peer Exchange
- get start time including pre-meetings and ending time

3. Research Manager:
- name
- address
- telephone
- fax
- email

4. Team Leader:
- name
- address
- telephone
- fax
- email

5. Team Members and (non-FHWA) observers:
- name and affiliation
-
-
-
-
-

6. Did having observers have an impact on the peer exchange (positive or negative)?

Peer Exchange Data Collection -- Research Manager

7. FHWA Representation:
- name, office, what level of participation (team member, observer)
-
-
-

8. What are your thoughts regarding having to use two individuals from the FHWA list of qualified people?

9. Location of Peer Exchange:
- DOT main offices - other DOT facility - hotel/conference center - university
- if more than one location, percentage of time or days spent at each location (if schedule available would be good to have)

10. Did you have informal gatherings (hospitality hour or receptions and dinner(s) before or during the peer exchange? How were they beneficial to the peer exchange process?

11. What were the advantages and/or disadvantages of holding the peer exchange in the location(s)?

12. Was the time allocated for the peer exchange about right, too long, or too short? If too long or too short, why?

Peer Exchange Data Collection -- Research Manager

14. How far ahead of the date of the exchange did you begin planning for the exchange? If you did another peer exchange would you change the planning time? If yes to what?

15. How much staff time did the exchange require (in hours or 8 hour person-days)? (Please approximate if you haven't calculated this number.) - your personal time for - planning and preparation - post peer exchange administrative wrap-up - staff time for - planning and preparation - during the peer exchange - post peer exchange administrative wrap-up

16. What were the total direct expenses for the peer exchange (this would not include agency employee labor)? (Please approximate if you haven't calculated this number -- to nearest $1,000)

17. Were there any administrative problems associated with paying expenses of the peer exchange team? If yes, what were they?

18. How did you determine who was to be on your Peer Exchange Team? What are the most useful method(s)? (check all that apply) - your own personal contacts - FHWA list of approved peer exchange team members - recommendations by others - based on the team member's participation in other exchanges - other ---

19. Was selecting/finding a team leader in any way difficult or not as easy as anticipated?
______
yes
_______
no (How difficult -- on a scale of 5 very difficult to 1 not difficult). If yes, why, and what could be done to ease this task for others?

20. What are the most important strengths of a team leader? (This is a general question, not an evaluation of your team leader.) (How important -- on a scale of 5 very important to 1 not important).

Peer Exchange Data Collection -- Research Manager

- communicator - synthesizer

- manages a program like/unlike yours - other ---

21. What were important strengths of those selected for the peer exchange team? (How important --on a scale of 5 very important to 1 not important).

- research management expertise - a relevant technical expertise - prior experience in peer exchanges - communications ability - other ---

22. Was it important (and how important) to have people on the peer exchange team that manage or are involved with programs similar/different to your program? (How important -- on a scale of 5 very important to 1 not important).

- in size ($$$)

- in geographic location

- in type of operation (in house research focus, contract research focus....) - in other respects (from academia, from private sector...) - other ---

23. Once the team was selected and accepted the invitation to participate, were there any team member cancellations? Did you substitute another individual in place of this cancellation? (This is asked to determine whether we should advise a backup team member for others who will be doing exchanges in the future.)

24. How did you prepare those who were to participate? - team leader - sent written materials (mail, fax, or email) - telephone calls - personal meeting - other - what material was conveyed - team members - sent written materials (mail, fax, or email) - telephone calls - personal meeting - othe

Peer Exchange Data Collection -- Research Manager

- what material was conveyed - observers

- sent written materials (mail, fax, or email) - telephone calls - personal meeting - other - what material was conveyed

- agency personnel and other partners who were interviewed - sent written materials (mail, fax, or email) - telephone calls - personal meeting - other - what material was conveyed

- agency senior managers (those to whom close-out presentation was given) - sent written materials (mail, fax, or email) - telephone calls - personal meeting - other - what material was conveyed

25. Who was interviewed by the peer exchange team?
- CAO (Secretary/Commissioner/Director)
- Deputy(ies) to CAO
- Staff to the CAO
- Bureau or Division Managers
- Project Managers
- Technical Professionals -- Engineers, Planners
- Research Managers
- Researchers
- Laboratory Technicians
- Others in agency
- Others outside of agency: FHWA, Academia, Private Sector

26. Were interviews done in a group setting or individually?

27. What was the total number of people interviewed? _______ Was this too many, too few, or about right?
____________

28. What advice would you give to others regarding getting participation from people in your agency for the interviewing process?

Peer Exchange Data Collection -- Research Manager

29. Did you articulate detailed objectives to accomplish for the peer exchange team?

Are these in your report? If not, I'd like a copy of them. Did the activities of the peer exchange and the report fulfill these objectives? If not, why not?

30. What computer/word processing resources (computer systems and people) were available to the team during the peer exchange?

     - no resources during, just at end for report preparation -- computer systems, people - minutes taken during the whole exchange - summary notes prepared at specified points during exchange - used team's laptops and in-house systems - other Was the level of systems support optimal? If not what would have been more useful?

31. How could the peer exchange team be more productive in the report generation?

32. Has your state formally endorsed the report prepared by the peer exchange team?

If yes, was an endorsement letter and a copy of the report sent to the FHWA Division Office? If no, what were the principal reasons the state did not endorse the report?

33. As a general comment regarding R&D management issues, did the peer exchange process (including the team report) address the correct level of detail for your state? For the senior managers? For you as research manager? (too little, too much, about right?)

34. Did you participate in another state's peer exchange as a team member/leader (or observer) prior to conducting your own state's peer exchange? What function?

(How strongly recommend -- on a scale of 5 very strongly recommend to 1 mild recommendation.)

Peer Exchange Data Collection -- Research Manager

Would you recommend others to be part of another state's peer exchange prior to conducting their own effort? As a team member
_____
; as an observer
______
.

35. Did you or are you planning to contact the peer exchange team to determine if changes in R&D programs/management have been made as a result of the time spent together at your state's peer exchange? How was (is this to be) done and in what timeframe?

36. Although this effort was an exchange, did you think any part of it was threatening to you or the employees of the agency? If yes, can you give any advice on how others may avoid this situation?

37. Did the peer exchange training program assist you in conducting your peer exchange? Was there anything that should have been emphasized more during the training?

38. What would have helped you be more prepared (if possible) to conduct the peer exchange?

39. What three items would you definitely do again in the next peer exchange you conduct?

1. 2. 3.

40. What will you change next time you conduct a peer exchange? (If there are lots of things you would change, give the top three items.)

1.

2.

3.

41. Have you submitted any follow-up comments to FHWA? If yes, may I have a copy?

Peer Exchange Data Collection -- Research Manager

42. What advice do you have for FHWA regarding the peer exchange process?

43. What administrative lessons did you learn in conducting your peer exchange that you would like to pass on to others who have not yet conducted their peer exchanges?

44. What benefited you the most from this experience?

45. What is your overall assessment of the peer exchange process?

46. How many times each year would you be willing to participate as a team leader ______ as a member of a team
_____
?

47. How often should a peer exchange be conducted in your state? Every ____ years (for example).

THANK YOU for sharing your experience. Your information combined with others will be very helpful for those who have not yet conducted their peer exchanges.

Back to the Table of Contents

Peer Exchange Data Collection -- Team Leader

Team Leader Telephone Interview Script

Peer Exchange Data Collection -- Team Leader

Peer Exchange Data Collection

TEAM LEADER TELEPHONE INTERVIEW SCRIPT

Directed to the Team Leader of a Peer Exchange

1. For what state(s) were you team leader?

2. What characteristics are the most beneficial for a team leader?

(How beneficial -- on a scale of 5 very beneficial to 1 not beneficial).

- communicator - synthesizer

- experience like/unlike the host agency research program - other ---

3. What are important strengths for members of a the peer exchange team? (This is a general question, not an evaluation of the team you led.) (How important -- on a scale of 5 very important to 1 not important).

- research management expertise - a relevant technical expertise - prior experience in peer exchanges - communications ability - other ---

4. Was it important (and how important) to have people on the peer exchange team that manage or are involved with programs similar/different to the host state's program? (How important --on a scale of 5 very important to 1 not important).

- in size ($$$)

- in geographic location

- in type of operation (in house research focus, contract research focus....) - in other respects (from academia, from private sector...) - other ---

5. What administrative items helped you do your job more effectively? for example:

Peer Exchange Data Collection -- Team Leader

- word processing support available to write report - good up-front communication prior to the exchange - well articulated goals to accomplish - other ---

6. What could the host state agency have done to make the exchange more effective?

7. What improvements can be made in the process or format of the report production?

8. Did you participate in another state's peer exchange as a team member or observer prior to leading a state's peer exchange? What function?

(How strongly recommend -- on a scale of 5 very strongly recommend to 1 mild recommendation).

Would you recommend others to be part of another state's peer exchange prior to leading a team? As a team member
______
; as an observer
_______
.

9. What would have helped you be more prepared (if possible) to lead the peer exchange?

10 . What three items would you definitely do again in the next peer exchange you lead?

1.

2.

3.

11. What three items would you change if you lead another peer exchange?

1.

2.

3.

Peer Exchange Data Collection -- Team Leader

12. Have you submitted any follow-up comments to FHWA? If yes, may I have a copy?

13. What advice do you have for FHWA regarding the peer exchange process?

14. What administrative lessons did you learn in leading the peer exchange that you would like to pass on to others who have not yet conducted their peer exchanges?

15. What benefited you the most from this experience?

16. What is your overall assessment of the peer exchange process?

17. How many times each year would you be willing to participate as a team leader ______ as a member of a team
_____
?

THANK YOU for sharing your experience. Your information combined with others will be very helpful for those who have not yet conducted their peer exchanges.

Back to the Table of Contents

APPENDIX B

FHWA Memo SP&R Program Administration Guidelines Peer Reviews (Exchanges)

FHWA Memo -- SP&R Program Administration Guidelines Peer Reviews (Exchanges)

     The new regulations for conducting the State Planning and Research (SP&R) program became effective on August 22. Our goal is to have all States operating under the new procedures by June 30, 1995.

     To assist the Regions, Divisions, and States in implementing Subpart B. "Research. Development, and Technology Transfer Program (RD&T) Management," we have developed the attached guidelines. The guidelines expand on the basic requirements in the regulations and are provided as "guidelines."

     A State's RD&T management process must include as a minimum the items listed in Section 420.207 of Subpart B. Beyond that, we encourage the Regions and Divisions to work with the States to develop the best program possible. Please remember the guidelines are "guidelines," and each State's management process will be different.

     Attached are sufficient copies of the guidelines to provide one copy to each region and Division Office and State highway agency.

     Please contact Mr. Charles W. Niessner at 703/285-2100 with any questions concerning the new SP&R regulations or guidelines

signed

John A. Clements, P.E.

3 Attachments

cc: HOA-1, HOA-2, HOA-3

FHWA Memo -- SP&R Program Administration Guidelines Peer Reviews (Exchanges)

STATE PLANNING AND RESEARCH PROGRAM ADMINISTRATION

FEDERAL HIGHWAY ADMINISTRATION GUIDELINES

For 23 Code of Federal Regulations Part 420, Subpart B, RD&T Program Management

INTRODUCTION

The Intermodal Surface Transportation and Efficiency Act (ISTEA) of 1991 instituted a number of substantive changes pertinent to the planning and research program. In addition to retitling it from Highway Planning and Research to State Planning and Research (SP&R) it: (1) increased the set aside of funds apportioned to the States for SP&R activities from 1.5 percent to 2.0 percent; (2) included planning, research, and technology transfer as eligible activities under the National Highway System and Surface Transportation Programs; (3) permitted the use of 23 U.S.C. funds for other modes of transportation planning, research, and technology transfer; and (4) required the expenditure of 25 percent of the State's annual SP&R funds for research, development, and technology transfer (RD&T) activities, unless the State certifies that it will use more than 75 percent for planning. The ISTEA sets forth the minimum Federal requirements for carrying out RD&T activities using planning and research funds and establishes the foundation under which the Federal Highway Administration (FHWA) may allow States increased flexibility for directing and controlling their program initiatives, subject to certain program standards and conditions. Title 23, Code of Federal Regulations, Part 420, Subpart B provides for new procedures pertaining to the management of RD&T activities.

These guidelines have been prepared to assist the FHWA staff and the State highway agencies in implementing the regulations contained in Part 420, Subpart B. Guidance is also provided for processing RD&T activities during the transition period from the current process to the new management procedures.

CONDITIONS FOR GRANT APPROVAL

There are four conditions that each State must meet to be eligible for FHWA planning and research funds for RD&T activities. These are:

1.Implement a program of RD&T activities for planning, design, construction, and maintenance of highways and public and intermodal transportation systems.

2. Develop, establish, and implement a management process that identifies and implements RD&T activities expected to address high priority transportation issues.

3. Agree to peer reviews of its RD&T program and be willing to participate in the review of other States' programs.

FHWA Memo -- SP&R Program Administration Guidelines Peer Reviews (Exchanges)

4. Maintain documentation of its management process.

Each State is permitted to tailor its process to meet State and local needs. However, the process must comply with the above minimum conditions. The following items should be considered in determining a State's compliance with the above conditions.

1. Implement a Program of RD&T Activities

In developing its program, a State may include RD&T activities necessary in connection with planning, design, construction, and maintenance of highway and public and intermodal transportation systems. It is not mandatory that a State include all of these areas in its RD&T program. However, when identifying and prioritizing its RD&T activities (see item 2a below), the State should consider all of these areas. Each State should develop a program that addresses its highest priority transportation RD&T needs. The priorities will vary from State to State depending on the size of the State, population, size and number of urban areas, etc.

Not less than 25 percent of the State's apportioned SP&R funds shall be spent on these activities. The FHWA Division Office should review the State's work program to ensure that the appropriate percentage is being spent on RD&T activities.

A waiver of the 25 percent requirement may be requested if a State believes its total expenditures during the fiscal year for transportation planning will exceed 75 percent of the amount apportioned for the fiscal year. The approval authority for a waiver of the 25 percent requirement has been delegated to the FHWA Associate Administrator for Research and Development.

Prior to submitting a request for a waiver, a State shall ensure that:

1. The additional planning activities are essential and that there are no other reasonableoptions available for funding these activities.
2. The planning activities have a higher priority than the RD&T activities in the overall needs of the State for a given year.
3. The total level by the State in RD&T is adequate.

Requests for a waiver along with supporting justification shall be forwarded through the FHWA Divisions and Regions to the Associate Administrator for Research and Development. In reviewing the State's request, the Associate Administrator's decision will be based on the following considerations:

FHWA Memo -- SP&R Program Administration Guidelines Peer Reviews (Exchanges)

If the State's request for a waiver is approved, it is valid only for the fiscal year in which the waiver is approved. A new request must be submitted in subsequent fiscal years.

2. Develop, Establish, and Implement a Management Process

As a minimum, a State's management process should include procedures for:

1. Identifying and prioritizing RD&T activities for inclusion in a work program.
2. Tracking program activities, schedules, accomplishments, and fiscal commitments.
3. Using the TRIS database for program development, reporting of current RD&T activities, and input of the final report information.
4. Determining the effectiveness of the State's management process in implementing the RD&T program and determining the utilization of the State's RD&T outputs.

FHWA Memo -- SP&R Program Administration Guidelines Peer Reviews (Exchanges)

e. Documenting RD&T activities through the preparation of final reports.

The specific details concerning these procedures need to be developed on a State-by-State basis. However, the following guidance is provided to assist in developing the management process.

a. A procedure for identifying and prioritizing RD&T activities for inclusion in a work program.

The procedure for identifying RD&T activities should involve the other offices (highway, transit, intermodal) within the transportation agency, the FHWA Division Office, and the universities within the State. Within the State Highway agency, problems should be solicited from the various offices (design, construction, maintenance, etc.). It would be desirable to use a standard form to solicit problem statements (Attachment No. 1). Criteria for prioritizing the problem statements should be developed. A single committee or several committees (i.e. pavements, structures, traffic, safety, etc.) for States with large SP&R programs can be used to prioritize and rank the problem statements. FHWA Division Office participation in this process and on this committee(s) is strongly encouraged.

The problem statements selected for funding using the above process could then be expanded to add the research approach, time required, and cost estimate (Attachment No. 2). These expanded problem statements plus the financial summaries, could be combined to constitute the State's work program.

A State work program may be either annual or biennial. The work program, as a minimum, shall include:

The work program would be submitted to the FHWA Division Office for approval and authorization. Copies of the approved work program should be forwarded to the Regional Office (number of copies to be determined by each region) and two copies should be forwarded to the Associate Administration for Research and Development (HRD-10).

FHWA Memo -- SP&R Program Administration Guidelines
Peer Reviews (Exchanges)

Work Program Changes

Administrative requirements for grants and cooperative agreements to State and local governments are
covered under 49 CFR part 18. Under 49 CFR 18.30, a State may make budget transfers among
individual RD&T activities without FHWA prior approval unless the total of such transfers over the
period of the work program will, or is expected to, exceed the larger of $100,000 or 10 percent of the
total approved work program budget. For example, if an RD&T work program totals $2,000,000, a
State may transfer $200,000 among RD&T activities included in the work program without prior
FHWA approval. At the discretion of the FHWA Division Administrator, this prior approval
requirement may be waived.

A budget change that involves an increase in the total funds authorized for the work program still
requires prior FHWA approval and authorization. Similarly, programmatic changes (adding a line
item, contracting out, etc.) specified in 49 CFR 18.30 (d) require prior FHWA approval.

After approval and authorization of the work program, it is not necessary for a State to submit to the
FHWA individual, detailed work statements or proposals for review and approval. A State, at its
discretion, may wish to submit specialized or highly technical proposals or work statements to the
FHWA for comments or technical assistance. These requests may be forwarded through the Division
and Regional Offices to the Associate Administrator for Research and Development (HRD-10).
HRD-10 will coordinate the review with the appropriate R&D Division.

b. Procedures for tracking program activities, schedules, accomplishments, and fiscal commitments.

A quarterly or semiannual progress reporting system, using a standard format for the reports, should
be developed. Progress reports should indicate work completed, plans for the next period (quarter or
semiannual), results (if any), problems, and fiscal expenditures. Copies should be provided to the
FHWA Division Office.

c. Procedures for using the TRIS database for program development, reporting of current RD&T
activities, and input of the final report information.

The State should develop procedures for using the TRIS database when developing their work
program to determine if similar work is under way or has been completed by someone else. A
process for the timely entering of new studies, updating ongoing studies, and reporting completed
studies also needs to be developed.

Under a cooperative agreement between the Transportation Research Board (TRB) and the FHWA, a
standardized system will be created for managing summaries of past, current, and future State and

FHWA Memo -- SP&R Program Administration Guidelines
Peer Reviews (Exchanges)

Federal RD&T activities. The system will integrate document management with the TRIS system by
providing each State with a compatible data entry and text retrieval system that permits uploading of
research summaries. The FHWA Region and Division Offices will be able to read and download
summaries, but will not be able to input data. Each State will be provided with a dedicated computer
and software to access TRIS.

d. Procedures to determine the effectiveness of the State's management process in implementing the
RD&T program and to determine the utilization of the State's RD&T outputs.

The effectiveness of an RD&T management process and program is d