Stop Rejecting Your Best Engineers.

We’ve fallen for a collective delusion: that “puzzle-solving” equals “engineering capability.”

During my time at MSFT and Arm, we had long suspected that “clever puzzles” didn’t filter for good engineers. Google’s later internal data proved our suspicions correct: former SVP of People Ops, Laszlo Bock, revealed that after analyzing tens of thousands of interviews, the correlation between these brain teasers and actual job performance was “zero”.

Fig. 2: They serve primarily to make the interviewer feel smart

I still remember interviewing a brilliant, highly senior engineer. He was so nervous at the whiteboard he was visibly shaking, fumbling a simple for-loop. I pulled him into a conference room with just a pen and paper. Thirty minutes later, he had produced one of the most elegant system designs I’d ever seen.

What does the real business battlefield require? Handling ambiguity.

A diagnosis from Microsoft Researchidentifies this as a combination of two key traits: being “Self-reliant” (solving complex, novel problems with limited guidance) and the ability to “Execute” (knowing when to stop analyzing and start building, avoiding “analysis paralysis”).

LeetCode rewards the perfect solution before analysis paralysis. The real world rewards execution despite ambiguity. This confirms your mechanism isn’t filtering for “practitioners”; it’s rewarding “puzzle performers.”

My advice is blunt: stop the “algorithm theater.” As companies, we don’t need “actors”; we need “craftsmen.”

The solution is to shift to “practical competence assessment” that simulates the real work. The data contrast is staggering. A traditional “unstructured interview” has a pathetically low predictive validity, with a correlation coefficient of just r = 0.20-0.38.

This isn’t surprising to me in the slightest. The latter (Take-Home) measures “Craft,” while the former (LeetCode) merely measures “Memory.”

Fig 3: LeetCode vs. Take-Home

The same study found the correlation with “maintainable production code” was even higher, at r = 0.67.

This confirms what I’ve seen work: the closer the test is to the actual job, the stronger its predictive power. The core of the assessment model I built is this: “asynchronous projects,” “debugging challenges,” and “trade-off conversations.”

These mechanisms work because they deliberately simulate the “uncertainty,” “ambiguous requirements,” and “communication costs” of a real job. They force a candidate out of the whiteboard’s vacuum to truly demonstrate their “trade-off decisions” and “engineering mindset”—the true markers of Ownership.

When you stop using the wrong ruler (LeetCode) to measure the wrong thing (puzzle-solving), you stop rejecting A-Players like Max Howell.

For you, the Engineering VP, this means you can finally identify talent that demonstrates true “Craftsmanship”.

This model filters for people who… know the “difference between ‘getting it done’ and ‘getting it right.'” They “don’t take shortcuts.” Critically, they “don’t stop caring” after the code is checked in; their Ownership extends to deployment and maintenance.

This is my definition of an A-Player.

Fig. 4: New definition of an A-Player

This is the real ROI companies need: a team of problem-solvers who can deliver maintainable code in the face of ambiguity, not a team of algorithm test-takers.

You’re reclaiming the A-Players your old mechanism was systematically filtering out.

My core belief is this: The greatest cost of hiring isn’t the “False Positive” (a bad hire). It’s the “False Negative” (a great hire you missed).

As an Engineering VP or Hiring Manager, your challenge isn’t to design harder puzzles. It’s to answer this: is your current assessment mechanism filtering for talent, or is it systematically rejecting it?

I urge you to re-examine your entire interview process. Is it measuring “puzzle-solving,” or is it assessing “practical engineering competence”?

Have you had a “near-miss” experience where you almost rejected an A-Player? Or do you remember a candidate who failed the whiteboard but thrived elsewhere?

As a “practitioner”: Have you ever been the “Max Howell” in the room? Did you lose a massive opportunity because of one broken, irrelevant question?

Share your diagnosis in the comments—how many A-Players has your old mechanism cost you?