TFHRC Home > Human Centered Systems Research > Human Centered Systems Research Publications > Human Factors Literature Reviews on Intersections, Speed Management, Pedestrians and Bicyclists, and Visibility > Appendix B

APPENDIX B. GUIDE FOR DOCUMENT REVIEWERS

The guide for document reviewers provided in this appendix was developed for use by the three individuals who were responsible for producing reviews of the documents/reports presented in section 3.0 of this report. The guide’s purpose was to provide a structure and framework for the reviews that: (1) would inform and help the reviewers, (2) was consistent with the project’s scope and objectives, (3) would provide accurate and technically defensible reviews, and (4) would provide some measure of consistency across the reviews.

PAGE 1 FIELDS

General:

• Try to quote directly from the report whenever possible, especially for the objective and conclusions.
• One guide for determining what information to include or to which degree elements should be elaborated or explained, is to write the review so that the researchers conducting the summary phase will be able to comprehend the key information from the study without having to re-read (or refer back to) the original report.

Title

Definition: The title of the report.

Usage: Include the report title, followed by the report number in parentheses, if it is a technical report. American Psychological Association (APA) format should be used (e.g., capital letters only for the first word of each sentence).

Authors

Definition: The primary authors of the report.

Usage: APA format should be used.

Report Date

Definition: The report publication date.

Usage: Include publication month and year, if available.

Number of Pages

Definition: The number of pages comprising the entire report.

Usage: Appendixes, front matter, etc., should be included in the page count.

Funding Agency and Contact Info

Definition: The contact address of the primary funding agency.

Usage: If available, the COTR should be identified under the address.

Document Web Site

Definition: The website where the document can be found.

Source Type

Definition: Preset identifiers describing the main approaches used for collecting/synthesizing data or information.

Usage: Categorical field that consists of one or more of the following terms:

• Crash/Demographic Statistical Analysis
• Literature Review
• Workplan
• Workshop
• Technical Analysis
• Survey
• Focus Group
• Laboratory Study
• Driving Simulator Study
• Closed-Track Study
• On-Road Study
• Field Study
• Integrative Research Review
• System Documentation
• Guidelines and Recommendations

Driving Conditions

Definition: Describes the roadway/environmental conditions associated with the study/project being reviewed.

Usage: Use the terms defined below (e.g., Degraded, Imminent Crash, Collision Warning System (CWS)). Indicate which of the following conditions apply (include all that apply):

• Normal: Applies if a test condition or analytical situation in the report involves driving conditions that are not degraded (e.g., dry weather/roadway, daytime, etc.). Driver distraction research should be categorized as occurring under Normal driving conditions.
• Degraded: Applies if a test condition or analytical situation in the report involves reduced visibility, inclement weather, driver fatigue, and other degraded driving conditions that make crashes more likely to occur by impairing a driver’s perception of the driving environment or of his/her own physical condition. Note that driver distraction is not applicable in this category (it is classified as a Normal driving condition).
• Imminent Crash: Applies if a technology or research paradigm addresses a specific type of collision. This is most likely to be relevant for CWS, driver warning systems, and vehicle control devices. Indicate in parentheses, which of the following apply (e.g., Imminent Crash (RCWS, RDCAS) ):
  • Rear-End Collision Warning Systems (RCWS)
  • Road Departure Collision Avoidance Systems (RDCAS)
  • Lane-Change Collision Avoidance Systems (LCAS)
  • Intersection Collision Avoidance (ICA)
  • Vehicle Stability (VS)
  • Not Specified (NS)
• All: Applies if all above driving conditions apply to a research report.
• Not Specified: Applies if it is not possible to determine driving conditions.

Vehicle Platform

Definition: Describes the class of vehicle(s) studied in the report.

Usage: Use the terms defined below (e.g., Commercial Vehicles, Transit Vehicles). Indicate which of the following vehicle platforms apply (include all that apply):

• Light Vehicles: Passenger vehicles, light trucks, vans, and sport utility vehicles.
• Commercial Vehicles: Heavy trucks and interstate buses.
• Transit Vehicles: Nonrail vehicles operated by transit agencies.
• Specialty Vehicles: Emergency response (e.g., snowplows), enforcement, and highway maintenance vehicles.
• All: Applies if report is relevant to all vehicles listed above.
• Not Specified: Applies if vehicle platform is not specified.

Objective

Definition: A list of research questions that the authors are attempting to answer in the study.

Usage: This field should consist of a single broad objective (perhaps obtained in the abstract) describing the overall purpose of the study and, if applicable, bulleted subobjectives describing additional research goals that support or are related to the main objective (perhaps obtained in the Introduction or Background sections). If possible, avoid combining multiple subobjectives into a single bullet, since keeping them separate provides a clearer description of the different tasks.

The objective should be stated in the authors’ words, if possible.

General Approach

Definition: Briefly describes how the researchers performed their research. Core methodological details (e.g., number of participants, roadway type) should be included in this section.

Usage:

• One sentence describing the test conditions, such as the apparatus and/or location of the study.
• One sentence describing the general procedure, while not providing excessive detail about the methods.

A common format should be used for describing elements that occur repeatedly (e.g., 40 participants drove an instrumented vehicle on a 0.5-km closed-loop test-track...).

Methods

Definition: This section provides additional details about the methods used. For empirical studies, this section primarily covers the main Independent and Dependent measures used in the study.

Usage (Nonempirical Study): Describes specific details about the methods that were not reported in the General Approach section. This might include:

• Type(s) of analysis performed.
• Methods for review articles.
• Specific activities for workshops.
• Sampling procedures for surveys.

Usage (Empirical Studies): Attempts should be made to list all of the variables examined in the report. If there are too many variables to list, priority should be allocated in the following manner:

1. Variables that form the basis for subsequent Key Results and/or Conclusions. It is important to include all variables covered in these sections.
2. Variables that are not covered by the previous criterion, but are relevant to the objective.
3. Variables that are not covered by the previous criteria, but are the stated priorities of the authors.

Independent Variables: These should be consistent with the following format:

• Factor (Levels): Within/between subjects.
• For example, Age (16-35, 36-64, 65+), Between subjects.

Note that if the experiment design is not obvious from the variable list or if there are other aspects not captured by the list (e.g., pre/post test), it may be necessary to provide additional information about the experiment design. However, this should be avoided if possible.

Dependent Variables: These should be consistent with the following format:

• Variable (Microvariables, ...).
• For example, Speed Control (average speed, speed drift, # speed fluctuations).

If there are too many microvariables to list, restrict the listings to global variables.

• For example, Driving Performance Measures (lane-keeping, speed control, headway, etc.).
• The Key Findings could then state that “some elements of lane-keeping were significantly impaired/improved, etc.”

If the variable name is not sufficient to provide a clear description of the nature of that variable, further detail can be included in a footnote.

Include the abbreviations used in the report if possible.

For surveys and focus groups, the Independent Variables are typically the demographic or market segment variables—whatever key factors are used to group populations (e.g., age, sex, etc.). If the authors do not segment participants into groups, then there are no Independent Variables (use N/A).

Key Terms

Definition: One- or two-word terms that describe important elements of the report.

Usage: The first letter in each word should be capitalized and a comma should separate each term. Use either:

• Key words provided in the reports, or
• Reviewer-determined key words based on summary text (e.g., abstract). Note that it is important that reviewer-generated key words use terminology that is consistent with the report summary.

PAGE 2 FIELDS

Key Results

Definition: A detailed list of the main empirical or analytical results of the study. If there are too many individual results to list, priority should be allocated in the following manner:

1. Significant results that form the basis of subsequent conclusions.
2. Results that are not covered by the previous criterion, but are relevant to the objective.
3. Results that are not covered by the previous criteria, but are the stated priorities of the authors.
4. Results for the variables listed in the Methods section that did not achieve significance.

Usage:

• Results should be described in as quantitative a manner as possible and they should refer to the relevant Independent Variables (e.g., braking times were 100 milliseconds (ms) slower in condition X).
• It is not necessary to provide quantitative values for abstract measures (e.g., subjective scales) or for measures that are not clearly understandable (e.g., RMSE).
• It is not necessary to provide measures of significance (e.g., p-values).
• If there are too many results to treat each key finding separately, group all of the related findings that follow the same trend (e.g., increasing or decreasing effects) into a single sentence that does not provide specific quantitative information (e.g., braking times, steering variability, and the number of lane excursions increased in condition X).
• The most important results should be presented as a graphic (two figures) or table (one table). This same criterion for determining which results should be included in this section can be used to select which graphics to present. Statistical tables (e.g., ANOVA tables) should be avoided.
• Use previously defined abbreviations from the report to save space.
• It is important that the stated objectives be addressed by some of the Key Results.

Conclusions, Recommendations, Best Practices, Design Implications, or Design Guidelines

Definition: Major conclusions, etc., that the authors indicate are important.

Usage: Stay true to the authors’ wording and meaning, and avoid making judgments on the validity or value of the conclusions. Inclusion of conclusions, etc., should be based on:

1. Consistency with the stated objectives.
2. Support from the data.
3. Connection with the methodology.

For guideline documents that contain many specific guidelines, list the major headings (e.g., Visual Display, Controls, etc.) and indicate the number of separate guidelines pertaining to each major heading in parentheses (e.g., Visual Controls (8)). It is important that the conclusions explicitly address the stated objectives.

It is important that the stated objectives be addressed by some of the conclusions.

General Comments

This section contains relevant information not covered in other sections. It can include:

• Surprising or unexpected results.
• Reviewer comments:
  For example, “This review covers experiment 1 of a three-experiment report.”
• Methodological lessons learned.

Definition:

• A list of issues/problems that may be encountered when using a particular methodology, or
• Caveats or cautions that researchers should be mindful of when designing a study, or
• Recommendations for improving flaws or fixing problems inherent in a methodological approach.

Usage: The authors’ recommendations about how to improve the study in the future should be included; however, truisms (e.g., “This study should be replicated with noncollege students.”) should be avoided.

Previous | Table of Contents | Next

TFHRC Home | FHWA Home | Feedback United States Department of Transportation - Federal Highway Administration