Our hiring process is what we can get away with

Here’s why Google hires the way it does but you shouldn’t.

Every month or so on HackerNews, there’s a thread about how interviewing is broken which usually devolves into “Google (and the rest of the FAANGs) suck at interviewing”.

Last year, I spent a few months working on The Holloway Guide to Technical Hiring and Recruiting, over the course of which I got to spend a lot of time talking to people who have been really successful at designing hiring processes, conducting interviews and being interviewed. I learned far more about interviewing than I could have imagined, and it was a great chance to reflect on a lot of these interviewing debates.

One quote from my discussions with hiring managers stood out, and it was from a frustrated high-ranking VP at one of the FAANGs: “Our hiring process is what we can get away with”. He was making the point that many companies blindly copy their hiring process in general and their interviewing process specifically, assuming it’s the reason behind their success, but in reality, their hiring process isn’t actually that good and they’ve just been able to get away with it because they have a strong enough brand and can pay well enough (to quote him, “we still think we’re the only game in town”).

I think that’s partly true, and definitely would fit into the “big companies suck at interviewing narrative”, but I also think that if you just take it at face value, it’s a little simplistic. These big tech company’s have spent years looking at their hiring data and feeding that back into their hiring process (coining the term “people analytics” along the way). Yes, they could all probably be a little more successful if they just dialed down the arrogant “you’d-be-blessed-to-work-here” attitude that’s ingrained in their hiring processes, but in reality, their interviewing processes work quite well for them.

Why? Well if you ask someone why big tech co’s do the types of algo/coding (aka “leetcode”) interviews they do, one answer you might get is that you need to have solid algorithm skills to succeed there. In fact, Dan Luu cites that as “the most common answer” for why algorithmic interviews are necessary. I actually don’t think that’s the most common answer (at least, not from the people I asked, who were hiring managers and recruiters). The most common answer is actually: we want to hire smart problem-solvers with strong analytical skills, and since we stopped asking brain-teasers because they’re irrelevant, algorithmic questions are the next best thing. In other words, algorithmic questions are just a better way of assessing analytical skills.

That’s bullshit. Anyone who has been on either side of an algo interview knows that you can totally prepare for them. Dozens of best-selling books, venture-backed startups, and cottage industry coaching practices have made money helping people improve their performance on these interviews. So it doesn’t actually assess your raw analytical skills. If you look at criticisms of algo interviews, what you’ll hear is that they actually assess:

  • willingness to spend time preparing for these types of interviews.
  • pattern-matching skills (map a problem to something you’ve seen before).

But this still works for big tech co’s.

If you’re a large tech co with a big brand and a salary scale that ranks at the top of Levels.fyi, a good interview weeds out people who wouldn’t do well at your company. To do well at a large tech company, you need to (and I’m painting with a really broad brush, but this is true for 90% of roles at these companies):

  1. Some sort of problem-solving skill that’s a mix of raw intelligence and/or ability to solve problems by pattern-matching to things you’ve seen before.
  2. Ability/commitment to work on something that may not always be that intrinsically motivating, in the context of getting/maintaining a well-paying job at a large, known company.

Hopefully you can see where I’m going with this. Basically, the very criticisms thrown at these types of interviews are the reason they work well for these companies. They’re a good proxy for the work you’d be doing there and how willing you are to do it.

Not that there’s anything wrong with that type of work. I spent several years at big tech co’s, and the work was intellectually stimulating most of the time. But a lot of times it wasn’t. It was a lot of pattern-matching. Looking at how someone else had solved a problem in a different part of the code-base, and adapting that to my use-case.

You really only need one Dan Luu per like 10 or 100 engineers at a FAANG. Most people aren’t going to be optimizing at the level he is, they’re going to be doing work that’s mostly a mix of problem-solving by pattern matching, and ideally, they’re motivated enough to have that job for as long as possible.

Now, unless you are one of those large companies, the type of people you want to hire will be a little different. You might need people who are passionate about a particular domain, or are really strong creative problem-solvers—and sometimes the very things that make someone a strong creative problem-solver can make them a weak pattern-matcher. Entrepreneurs tend to be creative problem-solvers; VCs tend to be strong pattern-matchers. With few exceptions, strong entrepreneurs are shit VCs and vice versa.

To tie this back to the original quote, you also probably don’t have the brand or money that big tech co’s do. So you might incorporate algorithms into your interview process, but you might also consider hands-on in-person interviews or take-home interviews (though those also have pro’s and con’s). The point is, don’t dismiss what big tech co’s do, but don’t blindly copy them either.

The Software Over-specification Death Spiral

I see a common pattern with startups and teams I’ve advised or been a part of. I call it the Software Over-specification Death Spiral, or SODS for short.

It looks like this:

  1. Product Manager (or CEO, Engineering Manager, etc) drafts up some sort of specifications or requirements for a new feature or product.
  2. Product Manager (or CEO, Engineering Manager, etc) drafts up some sort of specifications or requirements for a new feature or product.
  3. The requirements are handed over to the engineering team to implement.
  4. The engineering team “implements” it, but gets some things wrong. One bucket of things that are prone to being wrong could be things that were so obvious that the PM thought they weren’t worth making explicit. Another bucket are corner or edge-cases that the PM didn’t think of.
  5. The PM is surprised at this, and work has to be redone.
  6. In an effort to prevent this in the future, both the PM and the engineering team agree that requirements need to be more detailed.
  7. Surprise: despite the increased details in requirements, the engineering team still gets it wrong.

In each iteration of this loop, everyone agrees the requirements just need more details, but every time that happens, things are still wrong. What’s going on?

Breeding Code Monkeys

You’re breeding code monkeys, is what’s happening. Software is complex and malleable, and no set of specifications or requirements will ever be complete. There will always be behavior that requires some “filling in the blanks” at implementation time. The person doing the implementation needs to be able to either:

  • Recognize when they should ask for clarification, or, preferably,
  • Be able to fill-in-the-blanks correctly.

Counterintuitively, once you have this problem, the more you try to weed out ambiguity in requirements, the more likely you are cause the opposite effect of what you’ve want. Engineers turn off the part of their brain that they would use to think through product decisions, and become, essentially, “code monkeys” that just do what they’re told.

A Better Way

Does that mean you shouldn’t write specifications or requirements, or you should right less? Well, let’s not throw the baby out with the bathwater just yet. Specifications are important, but if you’re missing key pieces, they can make your problem worse.

At a high-level, when you’re building software, there are three questions to answer:

  1. What the software does—requirements and behavior
  2. Why it does what it does—actual problem it solves for its users
  3. How it does what it does—the actual implementation

These three are all related. SODS occurs when engineers try to do #1 and #3 without understanding #2. As Simon Sinek would say, “you must start with why”.

It’s on both the PM and the engineering team to understand “the why”. A few suggestions for PMs (and engineers):

  • Hire or worth with engineers who are inclined to understand the product, because they could be users themselves or because they have some other interest in the product space (or, as an engineer, try to help build products that you would use or that you find interesting).
  • Make sure you explain why something is worth building (or, as an engineer, make sure you understand that, and if you don’t, ask).
  • Push engineers to be involved in product decisions (or, as an engineer, try to be involved in various pre-implementation parts of the product lifecycle).
  • Undertake other activities that help engineers build product intuition. Engineers can spend time with users, spend time understanding your products analytics, etc.
  • If you really want to test whether an engineer is thinking about “the why”, have an engineer write the specifications for a feature of reasonable size, and then have the PM review them. If they can’t even attempt that, that’s usually a bad sign (though the worst sign of all is if they absolutely don’t want to even try).

Don’t fall victim to SODS.

Software as a Liability

On many teams I’ve advised or been a part of, code is generally viewed as an asset. Only some code, the “bad code”, is considered technical debt. The highest-performing teams, however, viewed things differently. For them, all code is technical debt on some level.

Programming vs. Software Engineering

Software requires two broad classes of effort. There is the immediate effort to write the software, and then future effort to maintain it. (Titus Winters of Google would call the former simply “programming”, and the sum total of both as “software engineering”).

Software engineering is programming integrated over time. Engineering is what happens when things need to live for longer and the influence of time starts creeping in.

— Titus Winters

It turns out that both the initial effort (programming) and the eventual effort (software engineering) are hard to estimate. If you’ve been in the software world long enough, the notion of a project’s initial implementation being delivered on time is so rare, it’s almost a joke. And of course, as you get to the “future effort” part, things become even harder to estimate and predict.

A Taxonomy of Technical Debt

Once you’ve built something, you (or someone else) become probably responsible for maintaining it. This cost is usually referred to as “technical debt”. We can break the cost of this future work into two broad classes:

  • Interrupts: Interrupts are when existing systems are taxing your time through reactive work like fixing bugs, fire-fighting, etc. Writing code now that creates interrupts in the future means you (or someone else) will be able to spend less time on making progress on other work later. Both the quantity and severity of interrupts matter. Interrupts are particularly hazardous to engineering teams because they are hard to plan for and usually result in forced context switching.
  • Inertia: Inertia means that a system is hard to make new changes to (because it is hard to understand or reason about, because it’s brittle, not modular and hence hard to change, and so on). This makes forward work difficult for you (ie even when you can spend time doing forward work, it’s really slow) or for others (e.g. because the system is hard to understand, it is a tax on the time of people who need to learn more about it, as well as on people who need to explain to others how it works).

It’s worth noting that if you had two systems that were identical in quality, you’d find that the costs of interrupts increased with system usage—how many people use your product, how often they use it, and how diverse their usage patterns are. In fact, Hyrum’s law tells us that the more people use your product, the more diverse their usages will be. But with the pressure of increased and different usage, your system finds new and different ways to fail, and the cost of failure (since your system is highly-used and depended upon) increases, too.

On the other hand, the cost of inertia increases with the quantity/scope of future changes you need to make to your product. And, of course, for poorly-designed systems, inertia and interrupts create vicious feedback loops. High inertia means you create bugs as you change your code, resulting in interrupts. And when interrupts happen and you need to fix them, it will be really costly because your system has inertia.

The point here is that all software is costly. Poorly written software is obviously more costly, but all software requires effort both now and in the future. Hence, it all creates technical debt—you will be paying interest on it, and occasionally you (or someone else) may need to pay down the principal with some refactoring or re-architecture. In other words, you are creating a liability.

A lot of people think of liability from a financial (debt) or legal perspective, but literally, a liability is simply “the state of being responsible for something”. And when you write software, you or someone else will be responsible for it.

Why Write Code?

But if all code is technical debt, why write any software at all? Well, the functionality that software enables is an asset. At some level, any valuable piece of software is solving some problem. And that’s the important distinction here. Software is the means, not the end.

Another way to frame this comes from “Uncle Bob” Martin (author of the seminal book Clean Code—though this framing is covered in another of his books, Clean Architecture). He views software as having two dimensions: behavior and structure.

Every software system provides two different values to the stakeholders: behavior and structure. Software developers are responsible for ensuring that both those values remain high.

— Bob Martin

Uncle Bob goes on to argue that structure (the ability to modify a piece of software) is more important than behavior (its current functionality). His argument is compelling: a perfectly functional but inflexible system will be impossible to change when change is required, but a poorly functioning system that is extremely flexible can easily be fixed.

I think that statement is mostly true. I use the word “mostly” because you could argue that there are some systems where existing functionality matters more than future flexibility. There are some critical contexts in which being absolutely certain that software is operating correctly outweighs any increased costs in flexibility (e.g. a car, an airplane, medical equipment). And there are some contexts in which not having some functionality really quickly will mean that it doesn’t matter how flexible that software is in the future, because it will have no future (e.g. an early-stage startup). But I don’t want to go too deep on this topic, since it’s tangential to the point I’d like to get to (and we’re getting there, I promise!).

Future Scope and Likelihood

So, putting this all together, software makes sense to build if and only if the value it creates now (through its functionality), and the value it enables in the future (through its ability to change its functionality), outweigh the costs it takes to build it now and maintain it in the future. But that “future” part is hard to predict.

I’ve always been amazed at how financial analysts can put together a spreadsheet to value an asset or investment. They’ll confidently forecast out a series of cash-flows, often in perpetuity. When they can’t forecast perpetuity with a straight face, they’ll slap a terminal value on it instead.

In software, it’s not that easy to predict (or pretend to predict) the future. So it’s easy to just, like Bob Martin, say that flexible software is better than inflexible software. But that’s a truism and it doesn’t really solve the problem of how, exactly, to think about building your software.

Software purists will make the case that “good software is good software”. Practice good design patterns, use the SOLID principles, remove (or encapsulate) complexity, etc. And you should. There are things that are almost universally good or bad architectural decisions in software, and we’ve got some great literature to help guide us.

But remember, writing good software is the means, not the end. Your goal is to build software in a way where current and future functionality outweigh current and future costs. And to get that right, we have to understand the scope and likelihood of future changes. Without that, we’re flying blind.

Domains, Users, and Problems

I’m a big fan of Domain-Driven Design because it shifts focus off of the code into the domain you’re trying to model.

The most significant complexity of many applications is not technical. It is in the domain itself, the activity or business of the user. When this domain complexity is not dealt with in the design, it won’t matter that the infrastructural technology is well-conceived.

— Domain Driven Design

The promise is simple: model the underlying domain correctly, and not only should the code and architecture fall into place now—they should be able to adapt and evolve as your requirements change.

Deeply understanding the domain you’re working with is a great start, but focusing too much on domain modeling can be misguided. Your users don’t care about how the domain is modeled—they care about whether your software solves their problems. So you actually need to understand three things:

  1. The domain.
  2. Your users.
  3. Your users’ problems.

How to actually do that probably requires a separate article(s), but it basically comes down to spending time thinking about, discussing, and analyzing those three aspects.

Don’t Gatekeep

As a final thought, there’s a risk of taking “software as a liability” to an extreme.

You’ve probably worked with one of these developers before: the type that gatekeeps software. Asked to implement something by a Product Manager or a colleague, their go-to response is “no, that’s too complicated”. Then they walk off feeling good that they have just prevented adding a bunch of complexity into the code base, and the future stream of liabilities that would create.

Any principle can be abused with the wrong attitude. So yes, all software is a liability, and it all has costs, but by truly understanding the domain you’re building in, the users you’re building for, and the problems you’re solving, you can help manage the trade-off between the cost of software and the benefits it provides.