Parse and visualize methods from NBER Abstracts

There is a long tradition of economists being seriously concerned about the methods used to answer their questions, sometimes with even more concern that the answers to their questions. It has been documented since Richard S. Thorn's The Methodologies of Economists: A Survey.

There is a figure produced by The Economist about the latest fad in the discipline being machine learning.

The figure is constructed in 2016. I want to update it.

I want to collect the NBER abstracts over time, and then parse them for different methods, and maybe have a look at the JEL codes over time.

Ben Davies from Stanford has created a nice R package that has details about all of the NBER papers. You can access it here

I have scraped the abstract content from the NBER website for nearly 30,000 papers. I've made a Shiny App that displays the frequency of terms you input over time. You can see what it looks like below and use it here

I may in time improve upon it, perhaps by doing some CI scraping to keep it up to date, but for now I think this is fine.

People have even used it! Sevrin Waights shared a plot on twitter looking at the decline of instrumental variables mentions over time.