The race between automation and new work

Summary

Technological change transforms economies and labor markets, reshaping the types of jobs that are available, the wages they pay, and the skills they require. Economists have long studied how some new technologies displace workers from existing job tasks, but much less is known about how different types of innovation generate new tasks and new forms of work. The research summarized here uses detailed data and recently developed tools to investigate these questions for the U.S. over 1940-2018. Among other findings, the research casts new light on the race between automation and the emergence of new work, and raises urgent questions about the future evolution of labor markets.

Many current forms of work emerged at some point after 1940. We document this by first constructing a database that spans eight decades of new job titles. We find that new job specialties introduced since 1940 accounted for around three-fifths of employment in 2018. Much of this new work requires specialist expertise, in tasks that are distinct from pre-existing jobs.

Despite its importance, this process has been hard to study, since there has been little consistent measurement of the emergence of new work tasks. In turn, this has limited understanding of the technological and economic forces that change the nature of work. Our study makes progress by using detailed, newly constructed data, partly based on modern methods for text analysis that can be used to interpret patent documents and occupational descriptions.

We show that new forms of work coincided with growing occupational employment over two long time intervals, 1940-1980 and 1980-2018. These shifts have created many new opportunities for college-educated workers. At the same time, they have eroded the middle-skill production and clerical jobs that offered relatively high-paid careers to non-college workers in earlier decades.

Next, we study where new work comes from. We distinguish between two forms of new technology: “augmentation” innovations and “automation” innovations. By combining data from patents and occupational descriptions, we show that augmentation and automation innovations have distinct, asymmetric relationships with the creation of new work. Augmentation innovations boost occupational labor demand while automation innovations erode it.

The findings point to an implicit “race” between task displacement and new task creation in displacing and reinstating labor demand, and suggest that automation may now be pulling ahead in this race. The findings raise urgent new questions about how labor demand changes as new work emerges, and how AI technologies will reshape tasks and occupations in the decades ahead.

Main article

Technological change transforms economies and labor markets, reshaping the types of jobs that are available, the wages they pay, and the skills they require. Much recent empirical work by economists has measured the displacement of labor from existing job tasks by automation technologies, including robots and software. But this literature has often treated the set of human job tasks as finite and static, implying that as automation proceeds, labor is confined to an ever-narrowing scope of activities.

Casual observation and historical evidence suggest this picture is incomplete: as employment in previously labor-intensive sectors such as agriculture, textiles, mining, and manufacturing has eroded, new varieties of labor-demanding activities have emerged, including in medicine, software, electronics, healthcare, finance, entertainment, recreation, and personal care.

These new forms of work require specialized human expertise, making them meaningfully distinct from pre-existing jobs. While this is now recognized in recent models used by economists (e.g., Acemoglu and Restrepo, 2018), a major challenge for research remains: there is almost no direct, consistent measurement of the emergence of new work tasks in our economies, or the technological and economic forces thought to give rise to them. 

We investigate these questions using a newly-constructed database, spanning eight decades of new job titles.

In the research summarized here, we address three central questions in this area. These questions cover the role of newly-emerging job categories (‘new work’) in counterbalancing the effect of task-displacing automation on the demand for workers. First, how much new work is there and what forms does it take, across the economy and over time between 1940 and 2018? Second, what explains when and where new work emerges? And third, what effect does the emergence of new work have on occupational labor demand?

Learning from new data

We investigate these questions using a newly-constructed database, spanning eight decades of new job titles. We link these job titles to detailed US Census “microdata” and measures of the exposure of particular occupations to innovations. These measures of exposure are based on information from patents. Our analysis relies on purpose-built data, allowing us to explore these questions more deeply than previous work. In the first step, we define ‘new work’ itself, as the introduction of new job tasks or job categories requiring specialized human expertise. Following path-breaking work by Lin (2011), we construct a database of new job tasks introduced over eight decades. These are sourced from nearly a century of internal reference volumes used by U.S. Census Bureau employees to classify the free-text job descriptions of census respondents into occupation and industry categories in each decade. 

This figure reports the distribution of employment in 1940 and 2018 across broad occupational categories, ordered from lowest to highest paying. The first set of bars shows 1940 employment. The second set of bars shows 2018 employment, divided into estimated 2018 employment in occupational titles that existed in 1940 (bottom), and estimated 2018 employment in occupation titles that were added since 1940 (top). Employment in 2018 is estimated by constructing a cumulative new-title share in each broad occupation – summing the number of new titles added over 1940-2018, and dividing this by the total number of titles in the 2018 index, adjusted for (the small number of) titles that were removed.

Using this new work measure, we find that the majority of employment in 2018 – slightly above 60% – is found in new job specialties introduced since 1940. This finding, documented in Figure 1, implies that new work emergence matters for understanding how employment evolves. As shown in Figure 2, new work emerges at the same time as growing occupational employment over two long time intervals, 1940-1980 and 1980-2018.

This figure illustrates the relationship between log new-title counts and occupational employment growth (converted into a rate per decade), controlling for initial-year log title counts, and using initial-year employment shares as weights. Plotted lines correspond to weighted partial fitted values. The weighted partial correlation between log employment growth and the log of new-title flows is 0.333 for 1940-1980 and 0.482 for 1980-2018. Axis labels are exponentiated.

Augmentation innovations are technologies that increase the capabilities, quality, variety, or other benefits generated by individual occupations.

But this persistent correlation does not reflect consistent growth in the same occupations. Some of the fastest growing occupations – also those adding the most new titles – between 1940 and 1980 are largely blue collar and clerical jobs, including Bookkeepers, Foremen, and Mechanics and repairmen. In the subsequent four decades, occupational employment growth and new title creation shifts towards information technology occupations, health services, and personal services, such as Computer software developers, Registered nurses, and Hairdressers and cosmetologists.

This figure reports the difference in the share of employment in new work versus the share of employment in existing work separately by education group and period. Each panel reports this difference across 12 broad occupational categories ordered from lowest to highest paying. The first set of bars represent the average difference in employment shares over 1940-1980, while the second set of bars represent the difference over 1980-2018.

This pattern is unmistakable in Figure 3. The figure shows that the locus of new work creation has shifted from middle-paid production and clerical occupations over 1940-1980 to high-paid professional and, secondarily, low-paid services since 1980. New work has thus been integral to seismic shifts in occupational structure. These shifts have created many new opportunities for college-educated workers. At the same time, they have eroded the middle-skill production and clerical jobs that offered relatively high-paid careers to non-college workers in earlier decades.

The sources of new work

We next study where new work comes from. To make progress, we distinguish between two forms of new technology: “augmentation” innovations and “automation” innovations. Armed with this distinction, we measure their rates of arrival over eight decades. Augmentation innovations are technologies that increase the capabilities, quality, variety, or other benefits generated by individual occupations. These can lead to new demands for specialized worker knowledge and expertise. Conversely, automation innovations are technologies that can take the place of the labor inputs of individual occupations. These innovations may replace workers performing these tasks.

We show that demand shifts that raise occupational demand lead new work to emerge more quickly, while adverse shifts in occupational demand slow it down.

We construct augmentation and automation measures using tools for natural language processing. These allow us to map between the full text of all U.S. utility patents issued between 1920 and 2018 and the domain of individual occupations. Using methods introduced by Kogan et al. (2021), we represent both patent documents and occupational descriptions as weighted averages of “word embeddings”, or geometric representations of what words mean. We then gauge the semantic closeness between patent texts and occupational descriptions by measuring how close they are in “embedding space”. For any given occupation, each patent may be classified as an automation innovation, an augmentation innovation, both, or neither.

Using these measures, we show that, while new work emerges in response to technological innovation, not all technology generates new work. Augmentation and automation innovations have distinct, asymmetric relationships with the creation of new work. Augmentation innovations strongly predict where new tasks mostly emerge, as measured by the appearance of new job titles across occupations and over time. By contrast, automation innovations do not predict where new work emerges. As we document in our research, since augmentation and automation innovations often arrive together for particular occupations, our ability to separate their effects is a stringent test of our approach.

Innovation is not the only driver of new work emergence: changing demands for occupational outputs also play a role. To investigate this channel, we make use of demographic changes that can sometimes raise the demand for labor, and changes in import competition with China that can sometimes reduce the demand for labor. We show that demand shifts that raise occupational demand lead new work to emerge more quickly, while adverse shifts in occupational demand slow it down. This matters for understanding the diverse nature of new work, since new work engendered by demand shifts may not have a high technological component. It also indicates that demand shifts may affect not only the sheer amount of work, but also the demand for specialized human expertise.

Our approach allows us to document the sources of new work emergence. Finally, we consider how new task emergence, alongside task displacement, affects employment and earnings. Our main finding is that augmentation and automation innovations have countervailing effects on labor demand. Augmentation innovations boost occupational labor demand while automation innovations erode it. As shown in Figure 4, these effects are present in both halves of our time period. They are also evident in both manufacturing and outside manufacturing. These findings point to an implicit “race” between task displacement and new task creation in displacing and reinstating labor demand, as envisaged in some of the models used by economists, most notably Acemoglu and Restrepo (2018).

The figure reports bin scatters of the employment-weighted conditional correlation between percent employment growth (converted into a rate per decade) and exposure to augmentation innovations (left side) and automation innovations (right side). Panel A uses consistently defined census industry × occupation cells over 1940-1980 (N = 6,520). Panel B uses consistently defined census industry × occupation cells over 1980-2018 (N = 27,380).

In the upper and lower panels of Figure 4, we compare the relationship between automation and augmentation innovations and employment growth, for the first and second halves of our sample respectively. The demand-eroding effects of automation innovations have intensified in the last four decades, while the demand-increasing effects of augmentation innovations have not. This tentatively suggests that automation may be pulling ahead in this race. 

The demand-eroding effects of automation innovations have intensified in the last four decades, while the demand-increasing effects of augmentation innovations have not.

New questions

These findings lead to urgent new questions about how labor demand changes as new work emerges. First, is ‘new work’ more labor-augmenting than simply ‘more work’? The finding that augmentation innovations increase occupational wage-bills by boosting both employment and wages suggests that the answer may be yes. This is plausibly because ‘new work’ demands novel expertise and specialization that, at least initially, commands a scarcity premium. But identifying and quantifying these premia would require us to observe directly the earnings associated with job tasks and the earnings of individual workers engaged in new work. This is largely infeasible in the Census public use microdata. If, as we suspect, new work brings additional opportunities to increase skills and raise earnings growth, beyond ‘more work’, then policies that foster new work creation may be of particular interest.

A second question concerns whether the locus and pace of augmentation and automation respond strongly to incentives. If so, which incentives matter most for steering the direction of innovation? While our research establishes that augmentation and automation move the demand for labor in countervailing directions, we have taken the shifting locus of innovation over time as given, without studying its sources.

Lastly, will ongoing advances in Artificial Intelligence shift the balance of innovation towards more rapid automation, across an expanding set of occupational domains? And to the degree that AI is not exclusively automating, what novel job specialties will it catalyze, and what skill sets will it complement?

++

This article summarizes “New Frontiers: The Origins and Content of New Work, 1940-2018” by David Autor, Caroline Chin, Anna Salomons, and Bryan Seegmiller, published in the Quarterly Journal of Economics in August 2024.

David Autor is the Daniel (1972) and Gail Rubinfeld Professor in Economics at MIT. Caroline Chin is a thesis writer at the Department of Economics at MIT. Anna Salomons is an Instituut Gak endowed Professor at Utrecht University and a Professor at Tilburg University. Bryan Seegmiller is an Assistant Professor at the Kellogg School of Management, Northwestern University.

References and further reading 

Acemoglu, D., & Restrepo, P. (2018). “The Race between Man and Machine: Implications of Technology for Growth, Factor shares, and Employment. American Economic Review, 108(6), 1488-1542.

Autor, D., Chin, C., Salomons, A. and Seegmiller, B. (2024). “New Frontiers: The Origins and Content of New Work, 1940-2018”, The Quarterly Journal of Economics, 139(3), 1399-1465.

Kogan, L., Papanikolaou, D., Schmidt, L. D. W. and Seegmiller, B. (2021). “Technology-Skill Complementarity and Labor Displacement: Evidence from Linking Two Centuries of Patents with Occupations”, National Bureau of Economic Research working paper no. 29552.

Lin, J. (2011). “Technological Adaptation, Cities, and New Work”, Review of Economics and Statistics, 93(2), 554-574.