Who does it best? Comparing universities by number of papers, inventions and industry research funding

A university’s ability to create and share innovative technology and know-how should be evaluated in a holistic way that includes both academic and commercial activities. In this article I compare the innovation transfer activities of U.S. research universities in a new, multi-faceted way: by counting and mapping universities according to their ability to

1. publish papers
2. to generate new inventions
3. and to attract industry research funding.

Why these three axes? A university’s scholarly publishing equals its ability to share knowledge via traditional channels; its invention activity reflects faculty interest in, and whether commercialization activity is valued on campus; industry funding equals the value of informal interactions between university and industry scientists (I’ll explain this one later). Combined, these arenas provide a holistic picture of a university’s activity in generating and sharing new technologies and scientific know-how.

To visually depict these comparisons, I made four bubble charts. The first bubble chart maps the usual suspects — the top 22 best-funded large U.S. research institutions. The remaining charts look at a new playing field, one where universities are compared according to their performance per million dollars of federal research funding, a view that triggers the emergence of a refreshing new set of highly performing universities.

This analysis represents university activity for the year 2010. The data on publications comes from the ISI Web of Science database. The data on disclosures and industry funding come from annual metrics collected by the Association of University Research Managers (AUTM, 2010). (If you spot data oddities or omissions for your university, let me know.)

1. Comparing the top 22 research universities

This bubble chart compares the biggest U.S. research universities. Here’s how to make sense of this chart (it’s best to view this on a large screen):

• the vertical axis represents total number of publications for the year 2010
  
• the horizontal axis represents how many inventions university researchers disclosed that year
• the size of the bubble represents how much industry funding the university got that year

So, if a university bubble is high up on the chart, that university produces a lot of papers. If a university sits out to the far right, it creates a lot of new inventions. The bigger the bubble representing a particular university, the more industry funding that university received in 2010.

Not surprisingly, Harvard researchers publish a significantly larger total number of papers than those at other universities. Duke, University of Colorado and Washington University of St. Louis have high levels of research funding from industry sources. CalTech researchers are strong in both paper publishing and creating inventions: on average, for each invention reported by a CalTech researcher, six scholarly papers were published.

The University of Texas and University of California systems aren’t depicted here for the simple reason that their numbers are so large they compress the rest of the university bubbles into a messy blob. If you’d like to imagine these two gargantuan university systems in this chart, visualize two bubbles roughly one-third larger than the big red bubble that depicts the University of Colorado floating in the upper right hand corner. In other words, when it comes to the absolute number (not corrected according to federal funding) of papers published, new inventions and industry research funding, Texas and California perform very well.)

2. Universities that publish the most papers per federal dollar

Scholarly publications, or what some people call “open science” remains the largest, most critical source of university research to industry product development efforts. For this chart, I set up a level playing field. To figure out a university’s publication activity independent of the size of its federal research budget, I calculated how many publications each university churned out per million dollars of federal funding. This way, a new group of universities emerge as top performers.

On this chart, the venerable Harvard shrinks in comparison to the Universities of Arkansas and Alabama. In fact, the number of publications from the University of Arkanses was so large — 165 papers per million dollars of federal research, almost three times more than the next-up university — that I checked and double-checked the count in ISI’s web of Science. (If this number is incorrect or deserves further explanation, please let me know or comment below. I can re-make the chart if needed.) The University of Akron appears to be a well-rounded university as it ranks in the top twenty according to publishing per federal dollar, and also dominates its peers according to new inventions and industry funding.

3. Universities that attract the most industry research funding per federal dollar

To represent the informal interactions between university and industry scientists, I chose to map how much industry funding a university receives. First, I’ll explain what went into this chart, then I’ll explain why industry research funding represents the vitality and quality of the informal relationships between university and industry scientists.

To make the chart, first, I pulled out the 20 universities that attract the most industry funding per million dollars of federal funding. To do this, I divided their total 2010 industry funding by their 2010 federal funding. In other words, the universities depicted on this chart are high performers, attracting large amounts of industry funding for their size and amount of resources.

Next, I charted the top 20 universities in this group according to their publications (vertical axis) and invention disclosures (horizontal axis). In this chart, the size of the bubble represents each university’s industry funding divided (or normalized by) how much federal funding it received in the same time period. The numbers of publications and new inventions on the axes in this chart, however, are the actual number generated by each university. The size of the bubbles represents how much industry funding that university received, again, a larger bubble representing a greater amount of money.

Out of this group, Duke receives the most industry research funding per million federal dollars. Next are the West Virginia University and University of Alabama. Out of this group of 20 well-funded universities, the Columbia and the University of Maryland generate high numbers of both scholarly publications and new inventions.

Quantifying informal knowledge exchange between university and industry scientists is notoriously difficult (for example, how would one measure conversations, consulting engagements, informal collaborations?). Yet, industry funding is a strong proxy that indicates the value and intensity of the informal interactions between a university’s and industry scientists. Turns out that the amount of funding a university researcher receives from a company is likely to be the downstream result of having strong connections in the chimerical, yet widely acknowledged informal channel of university knowledge transfer. Why?

According to research described in a February, 2007 article by Branco Ponomariov and P. Craig Boardman, “We find that involvement in informal interaction is associated with higher probability of undertaking collaborative research with industry as well as with a higher allocation of research time to collaborative research with industry.” In other words, if individual faculty members are intensely and productively involved with their industry-based colleagues, they are more likely to eventually attract an industry sponsor for their on-campus research.

Interestingly, this research offers another compelling reason that university patents are not good measures of innovation. Ponomariov and Boardman discovered that although active faculty partnered with industry scientists to bring a commercial product to market, these joint research projects did not involve university-owned patents. Nor did owning or working in a private company (e.g. a startup) increase the odds that a university researcher would have strong connections to industry. Instead, researchers state that

“Curiously, having worked with industry personnel in work that has resulted in a patent does not have a statistically significant effect on the dependent variable. Having worked at a private company as an owner, partner or employee is also not statistically significant.”

4. Universities that invent the most new inventions per federal dollar

Finally, new university inventions. I selected this group of 20 universities by dividing their total number of formally disclosed new inventions by how much federal funding they received. Similar to the chart above, the vertical axis = number of publications per million dollars of federal funding; the horizontal axis = number of inventions per million dollars of federal funding; the size of the bubble = industry research contracts.

First, a disclaimer on counting new inventions as a measure of university innovation transfer: reporting a high number of invention disclosures is a laudable achievement. It’s a good sign that university researchers trust and value their university’s formal technology transfer process. However, keep in mind that the majority of university researchers (even those considered high-performing) disclose few or no inventions to their university’s technology transfer office. Yet, university faculty and graduate students are keenly aware of how their research applies to real-world challenges, and they continue to perform cutting-edge research that makes tremendous contributions to industry innovation.

The reason I chose the number of new inventions a university generates each year as a meaningful measure of a university’s innovation transfer ability is that new inventions serve as a significant indicator of a university’s commitment to, and skill in commercializing research. For this reason I like new inventions more than patents as a measure of a university’s innovation climate; patents tend to reflect local policy and the size of a university’s patent budget.

Here’s what I like about correcting for differences in federal funding: some of these universities actually disclosed a relatively small absolute number of total inventions. Yet, if you count invention bang for the buck, these small schools are actually turning their research into reported inventions at a brisk rate, e.g. University of Akron, Michigan Tech and South Dakota State.

Overall, Brigham Young University turns its federal funding into the largest number of new inventions, disclosing on average, five inventions per million dollars. Louisiana Tech, Auburn and the New Jersey Institute of Technology do well here too, although the relatively small size of their bubbles indicates that they earn somewhat less in industry funding.

What’s next?

Policy makers, university administrators and others spend a lot of time and effort trying to figure out how to track and chart a university’s skill in generating and sharing new knowledge. Where they fall short is that most metrics today count contractual units of knowledge, for example patents issued, new startups and license revenue earned by university-owned patents. Instead, we need to expand how we measure how effectively universities translate federal research funding into new knowledge and new technologies by honoring channels that do not involve intellectual property. Of course the formal university technology transfer process is a valuable channel that should be valued and maintained. However, the fact remains that most industry scientists keep tabs on innovative university research by reading scholarly articles, informally interacting with their university-based colleagues and if that goes well, by eventually funding on-campus research.