Across industries, we’re getting better at picking metrics

Everywhere you look, people are optimizing bad metrics. Sometimes people optimize metrics that aren’t in their self interest, like when startups focus entirely on signup counts while forgetting about retention rates. In other cases, people optimize metrics that serve their immediate short term interest but which are bad for social welfare, like when California corrections officers lobby for longer prison sentences.

The good news is that as we become a more data-driven society, there seems to be a broad trend — albeit a very slow one — towards better metrics. Take the media economy, for example. A few years ago, media companies optimized for clicks, and companies like Upworthy thrived by producing low quality content with clickbaity headlines. But now, thanks to a more sustainable business model, companies like Buzzfeed are optimizing for shares rather than clicks. It’s not perfect, but overall it’s better for consumers.

In science, researchers used to optimize for publication counts and citation counts, which biased them towards publishing surprising and interesting results that were unlikely to be true. These metrics still loom large, but increasingly scientists are beginning to optimize for other metrics like open data badges and reproducibility, although we still have a long way to go before quality metrics are effectively measured and incentivized.

In health care, hospitals used to profit by maximizing the quantity of care. Perversely, hospitals benefited whenever patients were readmitted due to infections acquired in the hospital or due to lack of adequate follow-up plan. Now, with ACA policies that penalize hospitals for avoidable readmissions, hospitals are taking real steps to improve follow-up care and to reduce hospital-acquired infections. While the metrics should be adjusted so that they don’t unfairly penalize low income hospitals, the overall emphasis on quality rather than quantity is moving things in the right direction.

We still are light years from where we need to be, and bad incentives continue to plague everything from government to finance to education. But slowly, as we get better at measuring and storing data, I think we are getting at picking the right metrics.