Diversity | Programming | Software Engineering | Open source Why Coding Interviews are the Worst August 5, 2022
Neil Chaudhuri (He/Him)

Neil Chaudhuri (He/Him)

LinkedIn Neil Chaudhuri (He/Him)

Tech interviews are broken. Much like with vegan cheese, we all agree there is a problem, but we all differ on how to fix it. How did we get here?

Over the last several decades as tech became a dominant industry, the new personalities that became very rich very fast also became very Gavin Belson from Silicon Valley—supremely convinced of their own outside-the-box cleverness not only at producing revolutionary technology but also at revolutionizing the way it’s built. The media decided this was the beginning of a new era where the novel approaches tech companies took to everything should be a model for everyone else. Remember when open floor plans with foosball tables and Keurigs were the key to unleashing workplace productivity restrained too long by dehumanizing cubicles? After enough writeups in even mainstream outlets like CBS News, everyone was doing it.

But we ended up just using headphones to fashion our own mental cubicles so we could work in some kind of awkward peace—particularly awkward for women, people of color, introverts, and many others.

A similar thing happened with tech interviews. After all, boring accounting firms ask about your resume, but not us! The media were awestruck at profound interview questions like ”Why are manhole covers round?” or ”How would you calculate the number of cars passing through a busy bridge?” or the famous Elon Musk riddle ”You’re standing on the surface of the Earth. You walk one mile south, one mile west, and one mile north. You end up exactly where you started. Where are you?”

This is all about getting deeper than any traditional interview could, you see, to reveal how candidates think!

Yeah, OK.

Coding interviews are the worst. How do we know?

While the more absurd questions have lost some luster, one relic of our unconventional interview style that endures is the coding interview. You know it. A panel of dudes all vaguely reminiscent of Howard from The Big Bang Theory asks candidates to go to the whiteboard or a laptop broadcasting to a screen and write code for something they’ve probably never seen (by design). It could be implementing a doubly-linked list or merge sort or a function that determines if a string is a palindrome. It could be performing a breadth-first search or converting a loop into a recursive function. And on and on.

This is a problem—not only for job candidates but also for the companies considering them. It’s an especially acute problem now with the shortage of software engineers and the popularity of remote work particularly in the aftermath of a global pandemic. Then there is all the uncertainty. Uncertainty tech companies feel in an era of inflation and evolving economic fundamentals and uncertainty candidates feel in an era of layoffs and evolving personal responsibilities and professional goals.

Coding interviews solve for the Computer Science Department valedictorian or the most short term knowledge. Not the best long term culture fit

The biggest technical problem with coding interviews is the disconnect between what they measure and what actually adds value to the business. For 99% of you, hiring someone who understands Kruskal’s algorithm for finding a minimum spanning tree isn’t relevant to making your UI accessible, scaling in the cloud, improving your engineering practices, or driving revenue.

Even if you do better than silly college computer science questions and focus on the tech you use today, you focus on the wrong thing. It isn’t what candidates know; it’s how much they can learn. If you’re a React shop, it’s impressive if a candidate understands how useEffect is about syncing UI with state rather than a new spin on componentDidMount, but what if you decide to move to Solid? Or what if an important new initiative demands a different set of skills entirely and you don’t have the resources to hire a subject-matter expert right away?

Instead, you want to hire engineers who fit with your culture and make their teams better with hard and soft skills. They do this in many ways. They learn quickly because they think in abstractions so they can translate prior knowledge into new skills that serve their professional growth and your bottom line. They aren’t afraid to experiment and fail in order to learn new things. They listen. They are eager to help. They are patient and kind. They manage their time well. They can communicate with customers. They care about good code and write good documentation. Their teammates trust them to support them and to assume leadership when the moment calls for it.

Your business needs Vision. Coding interviews at best get you Ultron.

Coding interviews further exclude systematically excluded communities

I am a passionate advocate for diversity in tech. Prejudice against anyone on the basis of race, gender, ethnicity, nationality, religion, orientation, ability, or even academic background not only limits our individual growth but also limits the creative energy necessary to develop the best software. Sadly, there are countless institutional barriers to diversity in tech, and coding interviews are one of them.

Don’t believe me? Check out this study by North Carolina State University and Microsoft:

But the format may also serve as a barrier to entire classes of candidates. For example, in our study, all of the women who took the public interview failed, while all of the women who took the private interview passed. Our study was limited, and a larger sample size would be needed to draw firm conclusions, but the idea that the very design of the interview process may effectively exclude an entire class of job candidates is troubling.

Much as society writ large is convinced DNA evidence and AI are “objective,” tech is at best convinced coding interviews are also free from bias or at worst happy to capitalize on their bias. Bias in coding interviews could be favoritism, intentional or otherwise, reflected in choice of coding problem or leniency in evaluation. They are also biased against realistic forms of working. In real life, we don’t have people staring at us to scrutinize our work and mannerisms as we type. We have access to Google, Stack Overflow, GitHub Copilot, and other resources. The discomfort produced by contrived awkwardness in coding interviews hurts otherwise talented candidates—particularly those in communities already systematically excluded in tech like the women in the study.

But that’s one study you say. OK, but consider anecdotal evidence from elite engineers. Jenn Creighton, senior engineer at Netflix on the NodeJS team and host of the single-threaded podcast exposed the problem on her premier episode with her guest Erin Fox. Senior engineer, Apple whistleblower, and activist Cher Scarlett described her own experience a few years ago:

Of course if things can get so bad for cis white women in tech that coding interviews become a vanity exercise at their expense, just imagine how much worse it must be for women of color, trans people, anuerotypical people, even people who learned tech at boot camps instead of MIT or have degrees in psychology. And so many others.

Let’s use the interview process to welcome as many people as possible into tech because the challenges that lie ahead are too great to leave anyone behind.

Coding interviews are the basis for an entire cottage industry dedicated to help candidates “ace” them

Precisely because coding interviews are so divorced from the reality of our lives as software engineers, job candidates need time to prepare for the experience. It takes time to build composure as people stare at you at the front of the room or judge every keystroke. It takes time to remember, or learn for the first time, fundamental concepts of computer science or low-level implementation details typically abstracted from you by frameworks and open-source libraries. It takes time to memorize it all because you may not be allowed access to the resources you normally have at your disposal.

Everyone knows it, so a cottage industry has emerged to get candidates through it. There are literally hundreds of books and online courses promising to help them “ace” the tech interview and get that prestigious, high-paying job so you can get that Cybertruck because you find its ugliness endearing.

This is costly. At best, it takes time away from personal life. At worst, it costs serious dollars. Either way, this is another way coding interviews serve as a gatekeeper excluding communities from tech—software engineers who can’t afford the costs of interview prep.

What’s the alternative?

Coding interviews are bad for the business and bad for the candidate. Instead, you should conduct interviews that are inclusive, find the best long term fit for your business, and convey the joy of solving problems there.

Make the interview a conversation. Share your business goals and details about your mission, the role you’re hiring for, your expectations technically and culturally, and your values as an organization. Then consider questions like these.

“What are some examples of problems you solved at work? Take me through how you solved them.”

This question helps you understand the experiences candidates have had on the job. It helps you evaluate how these experiences apply to your business and, more importantly, how your candidates approach challenges. Feel free to explore the topic in depth and follow threads of conversation as candidates raise points you find particularly interesting. You don’t have to be Oprah, but there is an art to navigating a conversation to elicit more information. You will get a good sense of candidates’ technical knowledge, their creativity, amd their ability to gain new knowledge.

After all, game recognizes game.

“Can you describe a situation at work where you were particularly proud of what you accomplished?”

This question helps lighten the mood and bring positivity into an otherwise tense process. It gives candidates a chance to brag a little but in a way that lends value to you. You can glean if your own business offers the kinds of opportunities for joy the candidates want to experience, and you can again get some detail into their experience, expertise, and potential for growth.

“It’s common to have differences of opinion on a team on how to solve problems. How have you worked through those?”

This question addresses the reality of working on a team and the likelihood of a culture fit. Anyone who spends about three minutes on Tech Twitter knows we like to argue—often about the most trivial, self-serving things. Candidates have certainly had their share of disagreements at work, and it’s important to understand how constructively they handle them and if their solutions fit with your company culture. You will also get a sense of what candidates think is important—tech or otherwise. The question is if your value system is compatible with theirs.

“If you had full control of the architecture and engineering of your current code base, what would you change about the code or the process?”

This question again helps you learn about a candidate’s expertise and room for growth but from a different angle. We all have regrets about technical debt, yearnings for different approaches, and other aspects of the product and process that stick in our craw. Give candidates a chance to vent. In the process, you will learn about what candidates value, their knowledge of the engineering and product delivery landscapes, the direction they want to go, and their tact with constructive criticism. Does their vision comport with yours for your business?

“What kinds of support do you need to do your best work?”

This question addresses your responsibility to your people to achieve the goals of the business and the support candidates can expect if they join. You might have an issue with this question because it implicitly undermines conventional wisdom that the best engineers do their best work of their own accord if they have “passion” for tech implied by a robust GitHub presence, lots of side projects, eagerness to work weekends, etc.

Nope.

For reasons why the responsibility is yours to provide “flow state” rather than candidates’ responsibility to have passion, read this excellent post by Sarah Drasner, Director of Engineering, Core Developer Web at Google.

The best engineering managers understand how to create flow state. Even if candidates are unfamiliar with management theory, this question informs them that you have their best interests at heart and recognize your own responsibility in making them successful. You can also evaluate whether their answers are aligned with your business values.

“Do you have any questions for us?”

Hopefully you are asking this question anyway regardless of whether coding interviews are part of your hiring process. Give candidates a chance to ask you about the things that matter to them as they consider making the considerable, potentially life-changing commitment to your business. They should direct the conversation here. Expect questions about titles, commitment to and evidence of diversity, opportunities for advancement, remote work and tooling, and other aspects of business process and company culture.

Be careful though not to interpret the absence of questions as apathy or really anything negative. This is their time, and if they don’t need it, you shouldn’t let that influence your decision one way or another.

As an aside, prefer diverse panels of interviewers. Also pay attention to the demeanor of candidates. Any toxic behavior like arrogance, abruptness, or something particularly awful like refusing to acknowledge female interviewers should make your decision easy.

But OK, if you must do coding interviews…

Let’s face it. Many of you will never accept that coding interviews are the worst. The idea is simply too entrenched in tech. You’re convinced instant recall of computer science fundamentals is critical to bring value to your business. Besides, tech is full of imposters! You must expose them with cleverly chosen coding exercises they have never seen lest they compromise your business with their duplicity.

There is no evidence any of that is true, but it sure feeeeeeels true! The funny thing is there are actually a lot of job candidates who also think it’s true. They see coding interviews as a perfectly reasonable expectation and happily, if sometimes nervously, prepare accordingly.

Fine. If you must, at least adapt your coding interviews like this:

  • Offer candidates the option either to live code or to do a short, paid(!) project to do at home.
  • Have them solve a real but non-critical and non-sensitive problem you have at work.
  • If it’s a live coding exercise, collaborate with candidates rather than sit back and watch, and give them access to resources.

We have a lot of important work ahead of us as an industry. I hope you consider the lessons I’ve learned to help your business grow to meet the challenges ahead.