Summary
The US government spends hundreds of billions of dollars on infrastructure each year, about 40% of which is spent on building, repairing, and maintaining roads and bridges. A new paper studies the auctions that are used to determine which contractor should carry out a particular project. In particular, “scaling auctions” are used by many state Departments of Transportation, partly to share risks between the government and contractors. The researchers set out and estimate an economic model of how risk-averse bidders will bid in such auctions, and then study whether alternative forms of auction would lower the costs to the taxpayer. The researchers find that the scaling auctions currently used have much to recommend them.
One challenge in spending on infrastructure is that projects often see unexpected changes. Sometimes projects require more work and materials than anticipated; other times less. Bidders submit a unit price bid for each item on a full list of the tasks and materials needed for a project. The winning bidder is determined by the lowest sum of unit bids multiplied by item quantity estimates produced by Department of Transportation (DOT) project designers. The winner is then paid based on its unit bids and the quantities ultimately used to complete the project. This approach helps to share some of the project risks between the DOT and the chosen contractor, and this can help to lower taxpayer expenditures.
The authors of the new study show that contractors skew their bids: Contractors place high unit bids on items that tend to over-run the DOT quantity estimates and low unit bids on items that tend to under-run. Further analysis shows that they optimize not only with respect
to expected profits, but also with respect to the risk that any given expectation will turn out to be wrong. Drawing on these observations, the authors set out and estimate an economic model of how risk-averse bidders will bid in scaling auctions. The model can then be adapted to study what would happen under alternative auction designs.
The most common alternative for infrastructure procurement is a simpler “lump sum” auction, in which bidders commit to a total payment at the time of the auction, and are liable for all implementation costs afterward. Contractors then bear the risk, but because of this, they are likely to submit higher bids, raising the cost to taxpayers. This can also change the mix of bidders, but in simulations by the authors, moving from the scaling format to a lump sum format would substantially increase DOT spending for the median auction. This is the case even if “moral hazard” plays a role in scaling auctions: Over-runs may be caused by contractors ultimately choosing to use more of the items on which they placed high bids. A lump sum auction avoids this risk, but in the analysis of the authors, it remains more expensive for the DOT.
The authors then ask whether scaling auctions could be improved upon by a hybrid format: Bidders would commit to a fixed payment at the time of bidding, but then be able to renegotiate for a higher payment ex-post. The authors find that even this format is unlikely to improve upon the status quo, and this is also true of other policies to lower uncertainty, the benefits of which may not hold up in practice. Overall, the authors conclude that scaling auctions are less costly for taxpayers than the various alternative approaches they consider.
Main article
Hundreds of billions of dollars of public spending each year goes toward funding infrastructure in the US, about 40% of which goes towards building, repairing, and maintaining roads and bridges. Given the level of spending and the economic and social importance of the investment, local transportation authorities have sought to apply efficient mechanisms for procuring construction work. A key challenge for the procurement process is that infrastructure projects often see unexpected changes. For instance, a repair job on a supporting structure of a bridge may uncover, mid-project, that additional components had deteriorated and need to be repaired. These unexpected changes are often small, but sometimes quite large. Perhaps because designers try to account for the possibility of extensions, project changes go in both directions: sometimes they require more work and materials than anticipated; other times they require less.
Project uncertainty can be costly to the contractors that do the work – a lot of their business is often centered on public works – and to the government. The extent to which contractors are exposed to risk depends not only on the project design, but also on how contracts are allocated. Contracts with lower risk exposure may be more lucrative and thus invite lower, more competitive bids. As such, if contractors and the government can share the risk between them, this can help to lower taxpayer expenditures.
This study considers the mechanism by which contracts for construction work are allocated by the Highway and Bridge Division of the Massachusetts Department of Transportation (MassDOT or “the DOT”). Along with about 40 other state DOTs, MassDOT uses a “scaling auction”. Bidders submit a unit price bid for each item on a comprehensive list of tasks and materials required to complete a project. The winning bidder is determined by the lowest sum of unit bids multiplied by item quantity estimates produced by MassDOT project designers. This winner is then paid based on its unit bids and the quantities ultimately used to complete the project.
Scaling auctions thus have several key features. First, they are widespread and common in public infrastructure procurement. Second, they collect bids over units (that is, tasks and materials) that are standardized and comparable across auctions. Third, they allow the government and private contractors to share some of the risks arising from project uncertainty.
“if contractors and the government can share the risk between them, this can help to lower
taxpayer expenditures”
We use a detailed dataset, obtained through a partnership with MassDOT, to study what the partial risk-sharing of scaling auctions implies for bidding behavior and MassDOT procurement spending. For each auction in our study, we observe the full set of items involved in construction, along with the ex-ante DOT estimate of the quantity of each item, and the ex-post realization of the quantity once the project has been completed. We also observe a DOT estimate of the market unit rate for the item, and the unit price bid submitted by each bidder who took part in the auction.
We first show that – as in other settings, such as Athey and Levin (2001) and Bajari, Houghton and Tedalis (2014) – contractors skew their bids. They place high unit bids on items that tend to over-run the DOT quantity estimates and low unit bids on items that tend to under-run. Figure 1 shows this for the top two bidders in each auction. The figure plots a “residualized bin-scatter” of item-level over-bids – the percent difference between the unit price that the bidder requested for each item, and the unit cost that the DOT estimated for that item – against over-runs, namely the percent difference between the quantity of each item ultimately used and the quantity that the DOT had estimated in specifying the project.
As the figure shows, high over-bids and high over-runs tend to go together: the relation between over-bids and over-runs is positive. This suggests that contractors are generally able to predict the direction of ex-post changes to project specifications, and bid so as to increase their ex-post earnings. Furthermore, while the second place bidders (and in fact, also the third, fourth and fifth place bidders) all show a similar positive relation between over-bids and over-runs, the winning bidder exhibits the strongest relation. This suggests that bid skewing across items plays a role in how bidders construct competitive bids.
Figure 1
Next, we show that contractors submit bids that are consistent with rational sets of choices when facing risks (and hence represent a form of “portfolio optimization” for risk-averse contractors). As noted in Athey and Levin (2001), risk-neutral bidders would be predicted to submit “penny” bids – unit bids of essentially zero – on all but the items that are predicted to over-run by the largest amount. By contrast, the vast majority of unit bids observed in our data are not that extreme, even though no significant penalty for penny bidding has ever been exercised in our setting. Furthermore, we show that while contractors bid higher on items predicted to over-run, holding all else fixed, they also bid lower on items that are more uncertain.
To show this, Figure 2 plots the percent difference in the revenue share of each item in a winning bidder’s bid and the comparable share based on the DOT’s cost estimate, against the standard deviation (the dispersion) of the predicted quantity of that item that would be needed, given the project specification and a number of project and item characteristics. The negative relation between these two quantities shows that, holding all else fixed, bidders allocate smaller amounts of their revenue portfolios to project components that are hard to predict accurately. This suggests that contractors optimize not only with respect to expected profits, but also with respect to the risk that any given expectation will turn out to be wrong.
Figure 2
Building on these observations, we specify and estimate an economic model of how risk- averse bidders will bid in scaling auctions. In our model, risk-averse bidders dislike uncertainty and hence place higher overall bids in order to insure themselves sufficiently to take part in the auction. Consequently, auction rules – such as the risk-sharing mechanism in scaling auctions – that decrease bidders’ exposure to significant losses can effectively lower overall bids and reduce total DOT payments to the winning bidder.
However, the extent to which risk sharing is beneficial depends on how competing bidders differ from each other. If bidders who can build more cheaply are also larger and less risk averse, then it is possible for risk sharing to have a converse effect, encouraging less efficient bidders to take part in the auctions. Fitting our model to observed bidding behavior, we estimate the levels of risk, efficiency, and risk aversion among bidders in our data. We find that, while the nature of contractor competition varies widely across auctions, there is a strong positive relation between efficiency and risk aversion.
In order to gauge what these estimates mean for policy design, we conduct a series of “counterfactual simulations” to investigate approaches that would differ from the status quo. We use these to evaluate the cost of uncertainty to the DOT, as well as the performance of scaling auctions relative to alternatives used in other procurement settings. The starting point is an “independent private values” or IPV framework, and a model in which potential bidders decide whether to take part in auctions, in the spirit of Samuelson (1985). We first simulate what happens under a counterfactual setting in which uncertainty about item quantities is reduced to zero. When bidder predictions are held fixed – the only change is that uncertainty about these predictions is eliminated – we find that DOT spending decreases by 14.5% for the median auction. This suggests that project uncertainty contributes to a substantial risk premium.
“auction rules – such as the risk-sharing mechanism in scaling auctions – that decrease bidders’ exposure to significant losses can effectively lower overall bids and reduce total DOT payments to the winning bidder”
However, scaling auctions perform quite well on the whole, given the level of uncertainty in these projects. The most common alternative mechanism for procurement is a “lump sum” auction, in which bidders commit to a total payment at the time of the auction and are liable for all implementation costs afterward. Lump sum auctions require less planning by the DOT and they give bidders incentives to be economical when they can be. But for projects that are highly standardized and monitored, such as the bridge projects in our data, lump sum auctions primarily shift risk from the DOT to the risk-averse bidders. Seen in this light, scaling auctions provide a powerful lever for the DOT to lower its costs: not only do scaling auctions provide insurance by reimbursing bidders for every item ultimately used, they also allow bidders to hedge their risks by making rational portfolio choices.
In our simulations, moving from the scaling format to a lump sum format would increase DOT spending by 42% for the median auction. However, this result compounds two opposing effects. Bidders in a lump sum auction need to bid higher overall in order to compensate for their increased liability. These higher bids lead to higher costs for the DOT. On the other hand, higher liability may also cause the least competitive bidders to be priced out, and to choose not to take part in the auction at all. This reduces the number of participating bidders on average, but increases the competitiveness of the bidders who do take part. In our sample, the marginal bidder willing to participate under the lump sum format was 20% more cost-efficient and 28% less risk averse than the marginal bidder under the scaling format. This positive selection effect cuts the overall cost of moving to a lump sum format by more than half: were participation held fixed, lump sum auctions would be 96% more costly to the DOT than scaling auctions.
Importantly, there may be other benefits to using a lump sum format that are not accounted for in our model. Key among these is the possibility of moral hazard: some of the advantageous over-runs in our data may be caused by contractors choosing to use more of the items on which they placed high bids.
“even under the maximal amount of moral hazard consistent with our data, lump sum auctions are still substantially more costly than scaling auctions”
Our model assumes that the item quantities ultimately deemed necessary are determined “exogenously” (for example, by engineering considerations) and cannot be influenced by the contractor implementing the project. This assumption is supported by our discussions with DOT officials and by the formal procurement rules, which require that a DOT manager supervise construction onsite and approve any changes to project specifications. However, it is possible that some amount of moral hazard, at least on certain items, still arises. For instance, the outlying dots at the bottom right of Figure 1 correspond to several items that often have substitutes included in the project specification, such as signage for diverting traffic. The figure shows that, for these items, the auction winner bid much lower on items that wound up not being used at all than the second place bidder. This suggests that the winning bidder chose not to use these items, whereas the second place bidder might have intended to use another alternative.
While it is difficult to measure the extent of moral hazard that arises in our setting, we conduct a robustness exercise, to bound the maximal extent to which our results would change if moral hazard drove our observations. To do this, we suppose that all advantageous over-runs are driven entirely by moral hazard. We then re-run our simulations, allowing bidders to choose quantity over-runs up to the observed levels at the same time as choosing their bids. The possibility of moral hazard makes lump sum auctions more attractive for the DOT: the lump sum format makes contractors liable for over-runs, since their compensation does not increase with higher usage. This gives contractors incentives to incur fewer over- runs and potentially reduces the final bill for the DOT. However, our simulations show that even in this maximal case, moral hazard reduces the costliness of the lump sum format by only about 30%. That is, even under the maximal amount of moral hazard consistent with our data, lump sum auctions are still substantially more costly than scaling auctions.
Given these results, we ask whether scaling auctions can be improved through a policy that might reasonably be considered by the DOT. We consider a hybrid format in which bidders commit to a fixed payment at the time of bidding, but are able to renegotiate for a higher payment ex-post. This eliminates most of the added DOT costs from lump sum liability. In our sample, renegotiation with 2:1 bargaining power after a lump sum auction reduces added DOT costs to 14%, while renegotiation with equal bargaining power reduces added costs to only 8.5%. Still, both renegotiation options would increase costs relative to the baseline scaling auction, and neither greatly affects the mix of participants. As we do not find evidence of sufficient moral hazard to overturn these results in our setting, we conclude that switching to any type of lump sum format is unlikely to improve upon the status quo.
Earlier in our analysis, we found evidence of a substantial risk premium, by eliminating uncertainty and holding everything else fixed. But a policy to reduce uncertainty – through better monitoring of infrastructure conditions, planning or training, for instance – may not be effective at reducing costs in practice.
Uncertainty in our data is fairly symmetric: under-runs and over-runs both occur frequently, both in quantities and in DOT spending. As such, when we compare the no-uncertainty counterfactual against the status quo, DOT costs actually increase by nearly 2% for the median auction, after accounting for changes in who bids. Eliminating uncertainty gives bidders access to the exact quantities that will ultimately be used, allowing them to avoid making “mistakes” (from an ex-post perspective) that had benefited the DOT under uncertainty. In many cases, costs to the DOT from this difference in predictions counteract the savings from eliminating the risk premium. As a result, we conclude that the benefit from policies to lower uncertainty may not hold up in practice. The scaling auctions currently used have much to recommend them.
This article summarizes ‘Scaling auctions as insurance: A case study in infrastructure procurement’ by Valentin Bolotnyy and Shoshana Vasserman, published in Econometrica in July 2023.
Valentin Bolotnyy is a Kleinheinz Fellow at the Hoover Institution, Stanford University. Shoshana Vasserman is an Assistant Professor of Economics at the Stanford Graduate School of Business and a Faculty Research Fellow at the National Bureau of Economic Research.
