When the optimization mindset fails us

Technologists are always on the lookout for quantifiable metrics. 

But simple metrics can take us further away from the important goals we really care about, which may require multiple or more complicated metrics or, more fundamentally, may not lend themselves to straightforward quantification. This results in technologists frequently substituting what is measurable for what is meaningful. 

The realm of worthy ends is vast, and when it comes to world-changing technologies that have implications for fairness, privacy, national security, justice, human autonomy, freedom of expression, and democracy, it’s fair to assume that values conflict in many circumstances. Solutions aren’t so clear cut and inevitably involve trade-offs among competing values. This is where the optimization mindset can fail us.

Rob Reich, Mehran Sahami and Jeremy M. Weinstein, System Error

Not all encouragement is the same

Praising or criticizing outcomes tends to lead to a fixed mindset. Tell me I'm good at science and I'll start to think my skills are innate; tell me I'm terrible at math and I'll begin to believe there's no hope for me. 

Praising effort and application tends to lead to a growth mindset. Praise me for working hard on a project and I'll begin to believe that effort makes anything possible. Praise me for hanging in there even though I initially failed, and I'll begin to believe that perseverance makes eventual achievement possible. Praise me for taking a risk, and I'll begin to believe that trying new things--especially things I'm not good at--is a natural step on the road to achievement.

Jeff Haden writing in Inc.

When Optimization Rules

A focus on optimization can lead technologists to believe that increasing efficiency is inherently a good thing. There’s something tempting about this view. Given a choice between doing something efficiently or inefficiently, who would choose the slower, more wasteful, more energy-intensive path?

The problem here is that goals such as connecting people, increasing human flourishing, or promoting freedom, equality, and democracy are not goals that are computationally tractable. 

Rob Reich, Mehran Sahami and Jeremy M. Weinstein, System Error