You don’t need to work on hard problems

College, 2012—Internship recruiting season. “What are you looking for in your internship?” the recruiter asks. “I’d like to solve hard technical problems,” I reply. I end up at Jane Street writing software to calculate numeric integrals of a function that is costly to evaluate. (They don’t tell me what the function is—it’s too secret for interns.)

After two weeks, I come up with some of my own tweaks that make the algorithm work a bit better. I happily add “built a state-of-the-art library for numerical integration, with novel improvements on the best techniques in the academic literature” to my resume.

Back to campus—time to declare a major. I’m falling asleep in all my lectures, except the math ones, which use enough jargon that I am confused instead of bored. Guess it’s math then. After declaring, I finally get assigned an adviser who doesn’t tell me to take easier courses.

Real world, 2014—Math classes haven’t saved me from getting bored of college. I decide to take senior year off to work for a sexy machine learning startup. I spend months reading papers and optimizing our models. Our company doesn’t grow because we can’t close sales. (We have three engineers and no sales person.)

Real world, 2015—My classmates have graduated and gone to math grad school or Google or Facebook. I’ve switched from earning to give to directly trying to improve the world, at Wave.

Wave is saving immigrants a lot of money. Unfortunately, we’re also saving a lot of money for fraudsters, who are using us to charge other people’s debit cards. I spend three hours building a trigram logistic regression that differentiates Kenyan names (real users) from non-Kenyan names (likely fraudsters), which puts a lid on our fraud for the next year.

Real world, 2016—Wave’s new second-biggest problem is that we have outgrown Quickbooks. Our biggest problem is that the accountant-cum-engineer we hired to fix this is not good at either.

I’m left holding the bag, closing our 2015 books with a god-awful combination of raw Postgres queries and janky half-baked Twitter Bootstrap UIs. I spend 20 hours a week on the phone with our auditors, explaining our terrible decisions. Somehow, improbably, I am not bored.


For some reason, a lot of smart college students end up with the idea that “solving hard technical problems” is the best thing they can do with their life. It’s a common refrain in Hacker News comments, job ads and interview questions.

Why does this happen? Probably because that’s the only thing they’ve been rewarded for over the past 15 years.

School is a closed-world domain—you are solving crisply-defined puzzles (multiply these two numbers, implement this algorithm, write a book report by this rubric), your solution is evaluated on one dimension (letter grade), and the performance ceiling (an A+) is low. The only form of progression is to take harder courses. If you try to maximize your rewards under this reward function, you’ll end up looking for trickier and trickier puzzles that you can get an A+ on.

The real world is the polar opposite. You’ll have some ultra-vague end goal, like “help people in sub-Saharan Africa solve their money problems,” based on which you’ll need to prioritize many different sub-problems. A solution’s performance has many different dimensions (speed, reliability, usability, repeatability, cost, …)—you probably don’t even know what all the dimensions are, let alone which are the most important. The range of plausible outcomes covers orders of magnitude and the ceiling is saving billions of lives. The habits you learn by working on problem sets won’t help you here.


Because of these differences, most graduates of elite schools—including me—start out being completely unable to identify which work is actually important. (And if some important work does happen to hit us over the head, it won’t come in the form of a puzzle with a grading rubric, so we won’t know how to execute it well.) Instead, we’ll keep trying to run our college playbook, and look for hard problems.

Frequently, we’ll find them by making easy problems hard, with hilarious/depressing results. The upper ranks of Big Tech are filled with people who made their careers writing bizarre custom databases, or building Big Data infrastructure that could be replaced with a laptop.

When I first graduated, I was afraid that if I worked with boring technology, I’d get bored. Instead, I learned it was possible—and fun—to optimize on other dimensions, or play a different game. Rather than competing for an A+ on a hard problem, I could try to solve an easy problem as quickly as possible (like Wave’s accounting), or find the easiest problem whose solution would be useful (like identifying Kenyan names), or hire a team to solve easy problems faster than I ever could myself.

In fact, these turned out to be even more interesting! Why? Because “hard technical problems” wasn’t my root goal—my root goal was to use my skills to get the most possible leverage on improving the world.

In school, if you pick an easy problem instead of a hard one, you lose leverage because your extra problem-solving ability goes to waste. But in real life, you can redirect it to prioritizing which problems to solve, or working more quickly, or building a machine that solves the problems for you.

Because of that, I’m confident that my ability to solve “hard problems” has translated into having way more impact on the lives of people in sub-Saharan Africa, even though there’s a sense in which “all I’m doing” is helping build a CRUD app.

I won’t claim that the returns to intelligence are literally equal everywhere. But given a hairy enough goal, they’re a lot more evenly distributed than you think. So don’t look for hard problems—important ones are ultimately more fun!

Thanks to Eve Bigaj, Alexey Guzey, Jeff Kaufman, Dan Luu, Lincoln Quirk, and Yuri Vishnevsky for reading a draft of this post. If you’re looking for easy but important problems to solve, get in touch :)

Comments

email me replies

format comments in markdown.

Your comment has been submitted! It should appear here within 30 minutes.
me

Seriously, the “I found a hard problem I could solve the hard way” is an endemic dysfunction in my everyday life of software engineering managers at BigCo.

Identifying the problems that matter most and finding imperfect ways to improve quickly upon them is the core of contributing to the businesses actual success, but every stage of value generation is tough due to low signal to noise ratios.

email me replies

format comments in markdown.

Your comment has been submitted! It should appear here within 30 minutes.

Miikka

Jacob Kaplan-Moss recently wrote about The Innovation/Execution Spectrum. His line of thinking is similar to yours when he points out that a lot of worthwhile work is really in the execution end of spectrum and not in the innovation end.

Ben

Thanks for the link, that’s a great post! Just subscribed to Jacob’s blog :)

email me replies

format comments in markdown.

Your comment has been submitted! It should appear here within 30 minutes.