Virginia Gains Public Trust, November/December 2003 Public Roads

The Virginia Department of Transportation took measures to address project data and information problems and opened up its agency's operations to the public.

Says author William Pollard, “Information is a source of learning. But unless it is organized, processed, and available to the right people in a format for decisionmaking, it is a burden, not a benefit.”

The Virginia Department of Transportation (VDOT) found itself in such a situation; it had a big problem with data and information management, which led to other more serious problems. On the one hand, VDOT took pride in being known as an innovative engineering and research leader to peers in the transportation industry. However, VDOT also discovered that Virginia's elected officials, reporters, and citizens didn't necessarily retain the same view and positive impression, especially when it came to construction projects.

Within the agency, there appeared to be a disconnect between data collections and the information that was needed, desired, and communicated internally and externally, contributing significantly to budgetary and project management issues, and a negative outward image. Part of the difficulty was where information was stored, access to that information, and what information was needed to enable managers to make proactive decisions. Volumes of data, past and present, were stored in a variety of methods, locations, and databases, making it onerous to gather and analyze information about one project, much less all projects. Problems with cost estimating also could be linked to inaccurate, incomplete, and “guesstimated” information.

“Project management and cost estimating were our two biggest trouble spots,” says VDOT Commissioner Philip A. Shucet. “I asked about several major projects' costs and progress. Each person I asked gave me different answers because each answer included different data ... presented in such a way that it was hard to get a good handle on what was really going on with the projects.”

He adds, “You can't do good business that way, so we developed our project management system, Dashboard, and our Cost Estimation System within a year, and we're already reaping the benefits ... huge leaps from where we were.”

Through these two new Webbased systems developed in-house with off-the-shelf software, VDOT is gaining public trust and a reputation for accountability and openness.

The Mixing Bowl Catalyst The VDOT problems came to light with an improvement project near Washington, DC, for the I-95 Springfield Interchange, known locally as the "Mixing Bowl." According to an audit report about the project from the U.S. Department of Transportation's (USDOT) Office of the Inspector General (OIG), "The Springfield project, when completed, will improve traffic flow at the junction of Interstates (I)-95, 395, and 495 in Fairfax County, VA, which is one of the busiest and most congested interchanges in the country." The massive interchange feeds I-95 traffic into the District of Columbia, northern Virginia, and Maryland through I-395 and I-495. Like sometimes happens with other complex urban projects, this huge, seven-phase effort with its frequently changing costs and scope was starting to pick up a negative perception.

The project's estimated cost in June 1994 was $241 million, however, this increased by 180 percent to at least $676.5 million by June 2002, according to the OIG's November 2002 Audit of the Springfield Interchange Project: Federal Highway Administration Report Number (IN-2003-003). The audit also maintained that "this 180 percent cost increase occurred due to (1) the addition of new features after the initial estimates were prepared, (2) consistent exclusion of certain reasonably anticipated and known costs from earlier estimates, and (3) unanticipated cost increases." The audit found that not only had the Springfield project cost estimates increased, but that "construction problems have increased costs and could delay project completion, and funding to pay for cost increases has come at the expense of other State highway projects. We [Office of the Inspection General] made six recommendations to improve planning, cost-estimating, project management, and oversight. Responding to a draft of this report, FHWA [Federal Highway Administration] agreed to implement all six recommendations. VDOT also agreed with all of the recommendations and committed to work with FHWA to implement them."

Because of its problems, the project served as a catalyst for VDOT to make major changes in how the agency manages highway and bridge construction projects. It set the stage for real change in the agency that would be viewed as permanent instead of political. VDOT considers the changes to be groundbreaking, because they demonstrate how a State agency can do business openly under the unblinking public eye.

"Also significant is the fact that Virginia is now requiring finance plans for all [its] large projects, not just the federally mandated plans for megaprojects. The Office of Program Administration views Virginia's 'voluntary' approach to finance planning as a best practice," says Stewardship and Oversight Leader Tom Sorel from FHWA's Office of Program Administration. "Sound financial planning is almost always associated with good monetary stewardship and is a key to maintaining public trust and confidence with all large projects."

Today, the interchange improvement is under control with a single project manager, a realistic budget of $676.3 million, and a stable completion date in 2007.

Upon arriving at VDOT in April 2002, Shucet, a former official in the West Virginia and Arizona transportation departments and a Michael Baker Corporation executive vice president, asked his new staff members if they had any way of telling how VDOT was doing with its projects and program deliveries. Their responses indicated that this was an area for agency improvement.

Dr. Gary Allen, VDOT's chief of Technology, Research, and Innovation, wondered if a simple tracking tool with colored lights, or something similar, to show the status of every project in VDOT's Six-Year

Improvement Program might do the trick. After talking through the idea and working with a handful of people, Allen quickly-and literally-sketched out the concept. The group also came up with an initial set of rules to determine project statuses. Within less than 2 weeks, the group presented a template for what became VDOT's project management tool, Dashboard.

The agency's Dashboard is an online, one-stop information source that highlights the status of Virginia projects ready to go to construction and those under construction. Dashboard shows the latest on a project's progress, costs, and work orders (contract approvals to move forward with the work). The statuses are pictured using the red, yellow, and green lights on a traffic signal icon:

"Our goal is to maintain the green, manage the yellow, and deep-six the red," says John DePasquale, P.E., northern Virginia construction engineer. To do that requires spending resources, with "yellow" generating the biggest cost-to-benefit ratio in moving a project to "green," he explains. The tool earned its name because, just as an automobile's dashboard keeps the driver informed of the car's performance, Dashboard keeps VDOT project managers (and the public) tuned in to construction performance. It alerts them to the projects that are on track and those that are at risk. With this information at their fingertips, the project managers' jobs are to get the work back on track when possible.

"It's a great tool," says Tom Hawthorne, P.E., VDOT's Richmond District administrator. "Dashboard lets us focus on simple metrics to judge our progress, good or bad, against our goal of managing our transportation program more effectively."

Dashboard graphically displays information collected from several mainframe systems through a data warehouse. Those systems include VDOT's Program Project Management System, showing the status of project schedules from planning to advertisement, and Trns·port, a construction contract monitoring system developed by the American Association of State Highway and Transportation Officials (AASHTO). Because the systems are updated and queried daily, the Dashboard view is always less than a day old.

"It tells me in a moment what would have required going through reams of individual project reports to pull together before," says Connie Sorrell, VDOT's Director of Policy and Organizational Development who was tasked with coordinating the Dashboard effort. "Before Dashboard, we were data rich, but information poor."

VDOT's Chatham Resident Engineer Randy Hamilton adds, "Dashboard helps us easily track a project from advertisement through construction and keep it moving forward. Before, we had to rely on others to go through various information systems to pull the information together. Now we can get it ourselves quickly and we can answer our customers' questions on details." No special software was developed or purchased. VDOT's information technology staff relied on Microsoft® software products already used by the agency.

Dashboard has three views: a summary level, a second level that lists the projects, and a third with project details. The levels are on VDOT's external Web site, but within the third is a section reserved for internal viewing that includes some project history. To view the Dashboard system, go to http://dashboard.virginiadot.org on the Web. Projects can be sorted into statewide, regional, local, or highway system listings. The system is transparent: anyone can see the information without a password

One of the major challenges in creating Dashboard was establishing criteria that would determine whether a project was "red," "yellow," or "green" in each of the four categories. A project is considered "green," for example, if it is on schedule according to its contract; "yellow" if it is between 1 and 9 percent behind schedule; and "red" if the contract's completion date has passed, or if the project is 10 percent or more behind schedule, or if 90 percent of the time for the project has lapsed but less than 90 percent of the work is complete, or if critical data about the schedule are missing.

Missing or out-of-date information was the other big challenge. Dashboard tracks about 24 of the most significant project activities in VDOT's Program Project Management System. When Dashboard went live internally, many projects that showed up "red" really were simply missing some data. VDOT staff spent several weeks on updating data, and additional time testing Dashboard.

Less than a year later, on March 12, 2003, and with more than a little admitted internal squeamishness about opening itself up for public scrutiny, VDOT launched Dashboard on its external Web site, www.virginiadot.org.

"It just made sense that if we were spending $3 billion of taxpayers' money a year, they had a right to see how it was being used and to ask questions of the person responsible for the project," says Shucet. "Yes, it made some of our folks nervous, but we have to be accountable for what we do. No excuses. No hiding." Others agreed. Instead of receiving verbal derision for its "red light" projects, VDOT heard praise for sharing the good, bad, and even ugly details with the public.

"I think it's the most advanced instance of a really transparent project management system," says Jonathan Gifford, director of Master's in Transportation Policy, Operations, and Logistics at George Mason University in northern Virginia.

"In the public sector, it is difficult to know how a project is developing," says Jim Comstock, of Comstock Communications Concepts, a project management expert who has conducted training in several State DOTs, including VDOT. "VDOT's Dashboard is on the cutting edge in providing the needed information to all of the project's stakeholders."

Dwight L. Farmer, deputy executive director for transportation for the Hampton Roads (VA) Planning District Commission, adds, "The public's as well as the (local) governments' opinion of VDOT was that progress and budgets were a great mystery, at best. Dashboard has added enormously to the openness and ease of availability of budget and progress information."

"Governor Warner is a firm believer in 'that which gets measured gets done,' says Virginia Secretary of Technology George C. Newstrom. "VDOT's Dashboard is an excellent example of how State agencies can allow our customers, the citizens of the Commonwealth, to gauge our performance in serving them."

Comments from news reporters and editors include: "Lots of great information, very transparent-you don't get this a lot in government."-Lane Ranger column, Potomac News, March 17, 2003. "Transportation Commissioner Philip Shucet says he expects a lot of feedback because of the site. Here's some: Good job."-Richmond Times-Dispatch editorial, March 20, 2003.

"The ability to immediately review the physical and fiscal condition of a particular project aids in communicating with my constituents regarding VDOT projects," says Virginia State Senator Marty Williams, who, as chair of the Virginia Senate Transportation Committee, receives many questions about the transportation agency's work. "VDOT employees are aware of their actions, and the quality of their projects is only a few clicks away from their potential critics." The system makes employees aware of accountability for their actions.

Sharing Dashboard on the Web means that users can obtain the details at their convenience. And anyone can ask questions of the project managers through e-mail links contained in each project's description.

"The way Dashboard is presented, anyone can pick up information without having to get hold of someone between 9 and 5," says Young Ho Chang, P.E., director of transportation for Fairfax County, a densely populated area just outside Washington, DC.

VDOT to improve its relationships with local government and citizens," Chang observes. "I respect VDOT for saying, 'Here's what we're doing and how we're doing it . . . You're going to see us every step of the way.' The true benefit will come with what VDOT does with the information. The key is to improve by using this tool."

Shucet agrees and emphasizes that seeing red lights on Dashboard can only help VDOT project teams. "There's nothing wrong with a light being red," he wrote to VDOT employees.

"If a light is red, and if all of the information turning the light red is accurate, then we can learn from the experience to make the next job better. And the one after that even better again ... That's using knowledge to improve our business," he emphasized.

"Dashboard has been an organizational catalyst for us," adds DePasquale. "It's making everyone focus on the same goals, and it's changing our project management behaviors and practices."

For example, Dashboard and other reports clearly showed that VDOT was doing a better job of keeping its projects within budget, with some notable exceptions, than completing them on time. Three months after Dashboard's public launch, 72 percent of the projects shown were within budget while only 35 percent were within their contract deadlines.

Many of the projects have "calendar day" contracts, meaning that they must be completed within a certain number of days, instead of "fixed date" contracts, which must be completed by a certain date. Now VDOT is transitioning to all fixed-date contracts unless there is a compelling reason not to.

"This change will get the projects completed quicker," says Sorrell, "help us give citizens more specific information on when the projects will be completed, and emphasize better planning on the contractors' part."

The Springfield Interchange work was the most recognized of the projects that were underestimated in terms of cost. VDOT engineers now agree that, over the years, projects were added to Virginia's Six-Year Improvement Program, the agency's blueprint for distributing anticipated funds to transportation projects, with unsound cost estimates developed early in the projects' lives. This underestimating led to public and political criticism because, year after year, costs in the program document would increase, sometimes dramatically.

Bill Cannell, formerly a VDOT public affairs manager for Virginia's huge Hampton Roads urban area recalls, "It was very difficult to explain to a savvy reporter who had covered VDOT for years why a project's cost estimate increased 50 percent from 1 year to the next."

A month after starting Dashboard, Shucet tackled another major problem. VDOT did not have enough money for the construction projects in its Six-Year Improvement Program pipeline. In a sample of projects statewide, the difference between early project estimates and the final costs was more than 200 percent.

"In the newspapers, it seemed that you didn't see the phrase 'VDOT project' without the term 'over budget' lurking nearby," says Stephen Haynes, a VDOT engineer. The problem was not unique to VDOT. An AASHTO Task Force on Project Oversight found that many State DOTs across the country have struggled with keeping their statewide transportation improvement programs in line with available funding. Some, such as Utah, dramatically changed their programming to make it more realistic, says Tom Stephens, P.E., now-retired director of the Nevada DOT and chair of the task force. Accurate, consistent, welltimed cost estimating has become such a challenge for many States that, in addition to the Project Oversight Task Force, AASHTO's Standing Committee on Highways Subcommittee on Design has formed a Cost Estimating Task Force to come up with solutions.

In VDOT's case, Virginia reduced the allocations in its program in 2002 by nearly a third from the previous year. Many early project cost estimates included too much guesswork and were far off the mark, usually on the low side. Many estimates were based on employees' judgment, expertise, and experience rather than a sound, consistent, methodological approach. The estimates often were created at differing points in a project cycle or in differing ways. Inflation was rarely factored in. Certain environmental treatments and unusual construction cost items were not always considered early in the projects' development, which resulted in costly work orders when those items came up after construction began.

VDOT needed a method for accurately estimating project costs early on that considered the many different factors that can change costs. The method had to be something that VDOT's nine districts would use uniformly in a State where the topography ranges from beaches and swamps to mountains and valleys.

And it had to be user-friendly. Commissioner Shucet tasked Allen to lead an effort to develop a definitive, consistent, well-documented approach for estimating construction projects. A task force that included VDOT engineers and researchers, two Commonwealth Transportation Board members, and a metropolitan planning organization member either had to locate the best available early project estimating system or develop one.

Through research, the task force found that many State DOTs were attempting to address the problem, but none had a "silver bullet." Washington State DOT was using a range of estimates for risk factors. California's transportation agency, Caltrans, was using three estimates: optimistic, pessimistic, and realistic. Although these approaches had merit, the task group did not think that they would help VDOT.

The method with the most potential revealed itself in VDOT's own backyard. Haynes, a member of the task force, had developed a project scoping estimation method 3 years earlier. The method, a feature-based system using Microsoft® Excel software, was estimating the district's road costs accurately using historical data. Estimation errors were lower in his district than the statewide average. The Haynes method included an annually compounded inflation factor. It was applicable for primary, secondary, and urban highways. Although the method did not take into account all project attributes, it did include a comprehensive list of the higher cost items and showed more promise than anything else the task force had seen. VDOT found the beginning of a solution from within the organization.

Using the Haynes method as the foundation, the task force collected extensive project data and evaluations from VDOT project management staff to build a project information center that includes:

VDOT had no consistent, uniform method to define a project's need, or how it could best be built-and by whom. This led to serious scope creep during the life of a project, so estimates did not hold up over time. The scope and the cost crept, yet often the original estimate was never adjusted to allow for those changes.

For uniformity and accuracy in the scoping process, part of the solution was to migrate all projects to a Web-based Intranet site that includes the project description, purpose, need, project narrative, team structure, schedule, basic project information, cost estimates, project tracking, documentation, approvals, and other details about the project. Each element is essential to the success of the estimating system. Each project has a site that serves as a repository for all information about that project-and only that project, Lee says.

"We call it the Cost Estimation System, but it is 10 times more than that," says Mike Branscome, assistant location and design engineer in VDOT's Staunton District. "It's a project information clearinghouse. Our contract administrators love it because they can go to one place, and within 10 clicks or 60 seconds, they can get all the details they need without having to run reports." The district's preliminary engineering staff uses the system for project information to help prepare for scoping meetings, he notes.

The estimating part of the system includes spreadsheets that prompt users to enter details that can affect project costs, helping ensure that nothing is overlooked. While considering the usual features-road length, lanes, curbs, gutters, and the like-the possibility of landscaping, lighting, retaining walls, hazmat removals, and wetland or cultural mitigation also is considered.

Other worksheets require details for estimating bridge costs, right-of way transactions, and utility relocations. Costs associated with inflation, construction engineering and inspection, and preliminary engineering are computed automatically and added to the estimate for a total project cost.

Every spreadsheet component must be completed, which requires collaboration among staff members representing all aspects of a project, from the local office to VDOT's central office. Although Virginia's diverse geography can create variations in project costs across the Commonwealth, the system accounts for the variations by using district-specific adjusters. A 3 percent interest rate rate is factored into the estimates.

"Communications is the key here," says Lee. "And everyone has to do their homework on the project before that communication can happen. Much more work will be done prior to the scoping meeting than ever before, which should save significant time and money later in a project's development."

The goal is for estimates to be as close as possible to the final project cost. Applied to projects within VDOT's current Six-Year Improvement Program, the system is helping create a more realistic, credible, achievable program than in the past. And how is it working so far? Early analysis of the system's performance indicates that cost estimates tend to be much closer to final project costs than ever before. "The foundation of our business is getting the estimate right," says John H. Neal, Jr., VDOT's Hampton Roads District construction engineer. "Now we have one official source to go to for cost information. It's pretty explicit about what went into the estimate. It's producing realistic estimates."

Richard Caywood, P.E., VDOT's Petersburg resident engineer, adds, "It's given us a repeatable process framework that will increase our estimating precision every time we use it. It's taken something that was an art and made it into a science."

Dashboard's technical team plans to add several improvements before the system's first anniversary. Users will have a better picture of the projects-literally. A "Map It" feature will provide a geographic information system (GIS) map of the project area, "Picture It" will supply aerial photos of the project site, and "Drive It" will create a virtual drive of the project.

The Cost Estimation System will include major revisions to the right-of-way and utilities worksheets, details to address additional lanes and varying terrains, plus statewide training on using the system. A preliminary engineering tracking system module lets users tailor inquiries about schedules and budgets spanning the life of the projects.

A new plan sheet module to enable users to view and print design plan sheets-a time-saving feature- soon will be added. The agency also has plans to integrate the project documentation feature (for videos, photographs, e-mails, memos, and the like) with VDOT's Falcon, a repository for road designs and other project documentation.

"Your technology linking independent databases and allowing public access and oversight through that technology is the best financial watchdog ever created," wrote Virginia's Deputy Secretary of Transportation Pierce Homer, in a memo to employees involved with Dashboard and the new Six-Year Improvement Program. "It simply will not be possible to create a $10 billion plan with only $7 billion of revenues. While I admire the technical elegance of these efforts, I admire even more their impact on responsible, democratic governance."

Shucet adds, "Dashboard and the Cost Estimation System changed our problems into opportunities to build on. With these tools, we will never go back to the way we did business before them. We owe it to ourselves and to our customers to only go forward-and to do it openly."

Finally, in their efforts to be good stewards of taxpayer investment in transportation, VDOT would like to offer other State and local transportation agencies the opportunity to use and adapt their project management and cost-estimating systems to their own agencies.

Donna Purcell Mayes is an assistant public affairs director for VDOT, a position she has held since 1990. She joined the agency as an editorial assistant in 1977. She is a member of the Public Relations Society of America and the Virginia Government Communicators.

For more information on Dashboard, contact Connie Sorrell, chief of Policy and Organizational Development, VDOT, connie.sorrell @virginiadot.org, 804-786-1476. For the Cost Estimation System, contact Gary R. Allen, Ph.D., chief of Technology, Research, and Innovation, VDOT, gary.allen@virginiadot.org, 434-293-1938.

November/December 2003

Virginia Gains Public Trust by Donna Purcell Mayes

Virginia Gains Public Trust
by Donna Purcell Mayes