“Talent is the limiting resource in everything we do right now.” If that observation rings true, you’re not alone. Despite every effort, most startups struggle to hire the best people.
My partners and I believe developing a competency in hiring is one of the most important skills a founder can develop. And, while most founders we know are hungry for data — and create the best processes they can to collect and apply data to all parts of their business — when it comes to growing a team, it’s often all too easy to treat hiring and interviewing as art, not science.
All start-ups make hires that don’t work out. That’s just the way it is. But, we believe a strong interviewing process that focuses on relevant data collection can increase your success rate, and save you tons of lost time and emotional energy.
To sharpen our portfolio’s hiring practices, we’ve been conducting workshops for our founders with Burton Advisors and several of our general partners that focus on a data-centric approach to hiring and interviewing. Jordan Burton’s breadth and depth of experience with talent assessment, coaching, and development, particularly with startups, made him a natural choice. Our portfolio companies gave us such great feedback about these workshops that we decided to share the advice with our wider community of startups.
Matrix General Partner David Skok has written about the importance of hiring an internal recruiter early in your company’s life and developing recruiting capabilities in-house. In this piece, we focus on interviewing.
Table of Contents:
- What are you really looking for?
- Great Data Flows From A Structured Process
- Tracking Your Data
- Common Interviewing Traps
When it comes to effective, data-driven interviewing, it all starts with knowing who you are and what you want — and then taking a structured approach to find candidates that meet your criteria.
What are you really looking for?
I. Know What You Want For This Role
Before you start interviewing, define the role you are hiring for as best you can, listing the specific expectations and goals you have for your new hire. Burton teaches founders to use this list to develop a one-page “target document” for the role. This document is then used to ensure you probe in the right areas during an interview to get the data you need and identify whether a candidate will be a great fit for the given role, and for your company’s culture.
This type of list will help you stay focused when you are tempted by a smart, accomplished person who fundamentally isn’t right for the role in question. Hiring this type of person may work out in some rare cases. But, Burton says, “turnover rates tend to be higher when companies change the role to fit a candidate they like, or (worse) hire a star without any specific role in mind at all. There can be considerable distraction and confusion when someone joins your company without a clear mandate, which is costly for a fast-growing startup.”
Here is an example of a target document we created to hire a Matrix Partners Associate.
II. Define Your Culture: Hold Up A Mirror
What defines your company? The explosive growth inherent to a startup’s early years, and the need to expand your team quickly, make it critical to articulate your answer to this question as early as you can. What is the culture you want to build? What trade-offs will you make? Do you want a team that’s leader-driven or consensus-oriented? Structured and disciplined or loose and fluid? Are you looking for people with like or different styles than you?
It’s your company and the initial hiring choices you make will greatly affect your company’s culture–and ultimately–its destiny. When you begin the interview process with a well-calibrated sense of who you are, you’ll be better equipped to determine which candidates will be the strongest additions to your team.
Burton also recommends:
-
- List the aspects of your culture that truly stand out that would not necessarily apply to your competitors. Go beyond generic, undifferentiated competencies that describe what every startup wants: team players who have high integrity, are smart, and think outside the box.
- Keep your list to between four and seven characteristics. Long lists are difficult to recall when you are in the interview itself.
- Turn the list into short descriptions of who you are that everyone can internalize.
To the extent you’ve started to build out your management team, involve them or even the entire team if your company is small enough, in the discussion. Being an inclusive leader in these foundational culture-building discussions will help you bring the team along for the journey and equip them to support the vision.
Examples of good cultural descriptors:
-
“Quietly confident.”
-
“Rewarding dissent.”
-
“Qualitative and non-linear.”
-
“Metrics-obsessed.”
-
“Selfless servants.”
-
“Celebrating productive failure.”
-
“Street fighters.”
Great Data Flows From A Structured Process
We believe a data-centric approach to interviewing will improve the quality of your hires and reduce your risk of making expensive mistakes.
Commit to developing a structured interviewing process in your organization earlier than you might expect—as early as right after your seed round—so you don’t find yourself winging it later. Having a structured interview process, where every candidate goes through a similar set of steps, will give you the data you need about all candidates. You won’t forget important questions or (perhaps subconsciously) find yourself grading candidates on a subjective curve.
In our workshops, we outline the most important elements of a high-quality interview process:
I. Double Down on Rapport
From your first interaction to communicating the offer letter, everyone who makes contact with your candidates should focus first and foremost on building a friendly, positive interpersonal connection with them.
Even if you’re asking all the right questions, if you don’t have good rapport with the candidate, you won’t build trust—and that limits your ability to collect a high quality and quantity of data.
Burton offers three pieces of advice for creating great rapport:
- In the first two to three minutes of an interview, pay attention to the candidate, how they carry themselves, and what their general energy level is. “I advise being slightly above their energy level—but not way above it. If you bowl them over with energy, it can freak them out. If you are even slightly below their energy level, it can bring them down, and next thing you know, you’re both fighting to stay awake.”
- Be the person that you would actually be on the job. Burton advises, “You’re probably a fun, interesting, engaging person to work with. So be fun, interesting and engaging in the interview! A lot of very Type-A, driven people can unknowingly intimidate the candidate by bringing an interrogation-type energy into the room. They’re in the candidate’s face, frowning or scowling trying to judge the candidate in the interview. Focus on gathering information now, and save judgment for later. Keep in mind, the candidate only knows you based upon this experience. You don’t want anyone to think, ‘If I end up in this job, I’m going to have to deal with this every day?’”
- Show that you are genuinely interested in the candidate. People love to open up with people who show a real interest in them—it is hard-wired into our psychology as social animals. We share openly when it is rewarded with smiles and nods, and we close up when others judge us or appear disinterested. Burton suggests getting into the right mindset before an interview. “Take a look at the candidate’s resume, and look down at their outside interests. Ask yourself how they might have learned those skills or gotten involved in those activities. You may not ask them about their outside interests, but this will help get you in a place where you are fundamentally curious about them. The candidate will pick up on that, and will be more likely to open themselves up.”
II. Screen Thoroughly
A great screen is convenient for you and the candidate and should never be handled as the first in a series of onsite interviews—it should be an independent step, with a subsequent decision point on whether to bring a candidate onsite. This saves both you and the candidate valuable time if there is clearly not a fit. Have a 30-45 minute conversation over Skype or phone, and position the screen as an opportunity for both parties to get to know each other more deeply. After some brief small talk, take charge and let them know your proposed agenda for the call. Remember that you’re gauging the candidate’s skills and fit as much as they are learning about your company, so be sure to leave time at the end for them to ask questions.
Screen everyone, without exception. And, track your ratio of candidates receiving screens to those coming onsite—some companies have a ratio of 5:1 or even higher. This minimizes time wasted in extensive onsite interviews with unqualified candidates, freeing up your team to dive deeper with the best candidates.
Don’t assume when hiring for senior positions that you should skip the screening process. Senior candidates are busy, and they’ll appreciate not making an in-person visit before you both know there is a strong potential fit.
III. Assign Roles to Your Onsite Team
One of the most common mistakes startups make with onsite interviews is failing to clarify who on the hiring team is responsible for what aspects of the interview process. You have a target document that spells out the specific “facets” of the job (goals/objectives, personal qualities)—use it to identify which facets naturally cluster together, and give each cluster to one of your onsite interviewers. Ask them to focus their interviews on their assigned facets to minimize redundant questions and ensure the full hiring team has covered the essential characteristics comprehensively.
“If you are buying a house,” Burton says, “would you prefer to have five inspectors each do the same run-through and fight over their superficial evaluations, or would you rather have one go deep on the foundation, another on the plumbing, and so on — and create one comprehensive report as a team? When you ‘divide and conquer,’ you greatly increase the quality and quantity of data you can gather, you come across as a well-coordinated team, and you improve the candidate experience.”
These “facet” interviews typically follow the “deep dive” interview discussed next.
IV. Prioritize Demonstrated Performance
In terms of the content of interviews, put a heavy emphasis on learning about each candidate’s actual experiences and accomplishments. Keep your questions as open-ended and “unaided” as possible, and ask the candidate for specific examples if they speak in generalities. Burton’s own research shows simpler, open-ended questions tend to elicit responses that are (1) more honest, (2) more representative of the candidate’s true nature and (3) more dramatic in scope or magnitude. Plus, these anecdotes tend to be easier for candidates to recall quickly and in detail, which helps avoid awkwardness in the room..
Avoid overly narrow or leading questions (“Tell me about a time when you had to do X and Y under constraints Z”). This may force your candidate to come up with second-tier, less noteworthy anecdotes from their career histories to satisfy your criteria. Also, avoid hypothetical questions about the future (“How would you handle a situation where…”) This gives your candidate license to paint a picture of the future which may have no basis in what he or she actually has the ability to deliver.
Burton recommends gathering past performance information comprehensively in a single “deep dive” interview which may last two hours or more, depending on the seniority of the candidate. This is typically conducted by a single person, the hiring manager. “Walk through their academic and professional career chronologically. Focus on gathering multiple accomplishments and mistakes in each job or academic program along the way. Get the specifics on what results they personally delivered (or failed to deliver), and how this compared to what was expected of them. Don’t be fobbed off with answers about how well the company did, as that may have had nothing to do with what they personally contributed. Look for evidence of fast career progression and promotions.
And be sure to talk through the transition points between roles—what was behind each departure and what motivated them to choose the new opportunity. Look for evidence of a steep upwards trajectory throughout these career moves. Ask yourself if they were able to explain a thoughtful approach to how they change companies, and some effort applied to evaluate whether the company they are joining was going to be successful. It’s OK for a candidate to make one bad career move, and leave soon after they have seen the issue. But it’s a significant red flag to see them repeat the error when it should have been possible with the right work, to determine up front that this was not a going to be a good career move.
Digging into this question will also let you know what really drives them to take on a new role. This “implicit” information can be a powerful asset to help you sell the best candidates.”
The information you gather in this deep dive interview can be used to assign specific target areas for team members to probe in subsequent interviews.
David Skok, one of my partners, advises you to put yourself in the candidate’s shoes as they describe their accomplishments and failures in each of their roles. Think about what you would have done in their shoes, and whether what they did achieved real results. Would you have expected better results and different outcomes?
*Introvert or extrovert? Know that extroverts tend to deliver better answers to problem solving interviews or questions about the future. Introverts can find it more difficult to generate this kind of content “on the fly” in an interview situation. The best leveler? Past-performance-based questions that ask about actual career history.
V. Ask for the Bad Stuff
If you’ve built great rapport, it’s much easier to ask the candidate for negative information.
Interviewers tend to not ask about a candidate’s shortcomings because they’re afraid to make candidates uncomfortable. But great candidates love difficult questions — they want to know you have a rigorous process and are keeping a high bar on all hires.
A candidate’s negative attributes fall into two broad categories: mistakes (unfavorable actions) and weaknesses (unfavorable qualities). Having a large repertoire of ways to probe each of these areas lets you flex in the moment. So, rather than asking a candidate to describe a fourth mistake — which might come across as badgering — ask for his biggest development area from his last review.
When the candidate responds to tough questions, don’t react negatively. Ask what happened but avoid judgment. Don’t be apologetic. Approach these questions the way you approach every other aspect of the interview — in a spirit of total curiosity and interest in the person.
What does this kind of conversation look like? Burton role-plays an interview that uncovers a real problem — the candidate was slow to address a poor hiring decision — in a way that’s not judgmental and, critically, doesn’t break rapport.
INTERVIEWER: What’s another thing that didn’t go as well in that job?
CANDIDATE: Well, I had one product that went out with a pretty major defect on it.
INTERVIEWER: Oh, what happened? [Bright voice, curious eyes]
CANDIDATE: You know, we had a QC process and we did a bit of a shortcut, because we got in a rush and I frankly hired a QC engineer that was not up to the task, and I knew it.
INTERVIEWER: Okay, what happened next?
CANDIDATE: Well, um, I was under a lot of pressure to remove him, that QC engineer, so I did.
INTERVIEWER: When did that happen?
CANDIDATE: Oh, that was about a year later.
Tracking Your Data
To make this a fully data-driven process, there are two types of data you need to be tracking:
- Candidate scores for each item on your target document
- Funnel metrics
For the first set of data, you will want to assess how the candidate scored against your “table stakes” and “nice to have” attributes – the basis of your “target document”. It’s ideal if your ATS (Applicant Tracking System) provides you the tools to do this. For example, here’s a screen shot from Lever, an ATS with this tracking functionality built in:
*Full disclosure: Matrix Partners is an early investor in Lever, and I sit on their board. We use Lever as an example to show the type of tracking you will want in whatever ATS tool you use. Though after much due diligence, we do think Lever is one of the best tools available to enable this data-driven process.
For the second type of data, you’ll need to track how your interview process is working and where improvements are needed. To do this, track your “funnel metrics” and identify the number of people going through each stage and what percent are successfully transitioning to the next stage. This data is very helpful to identify where you might have problems. For example, if you have lots of offers extended, but few accepted you will want to dig deeper into this last step. Or, if you have a lot of onsite interviews but few move on to the next step, you’ll want to review your screening process to understand where it’s breaking down. In each case, you’ll want to review your funnel, make the necessary changes, and track whether your conversion rates through the funnel improve. Again, an example from Lever to show the type of tracking tool you’ll want to be using:
Common Interviewing Traps
I. Use Live Problem-Solving Questions — Carefully
For intellectually or analytically demanding roles, problem solving or “business case” interviews can be a powerful tool in your arsenal. For engineering hires, technical problem solving questions are invaluable, and may comprise a significant percentage of your time with the candidate (especially for more junior-level hires who have less career history to discuss).
Yet real-time problem solving questions have drawbacks. They are best used when you can afford a few “false negatives”—sometimes an otherwise capable candidate may go off-track or get flustered, leading you to reject a potential strong performer. To ensure your problem-solving interviews are as predictive as possible, Burton advises ensuring the content is highly relevant to the job you are actually expecting them to do. “Don’t ask someone to solve a random brain teaser, unless you’re hiring a VP of Solving Brain Teasers. Focus on problems that look like what they will face on the job.”
Burton shared some additional best practices for problem-solving interviews, both technical and non-technical:
- Choose a problem that offers multiple paths to the solution. Your candidate will be more likely to find a solution, even if it’s not the best one, and you’ll learn how they evaluate tradeoffs between potential solutions.
- Keep the discussion interactive, and encourage the candidate to talk through their logic. Avoid “black box” questions that don’t allow you to see the candidate’s process.
- Offer “safety nets” or “nudges” that allow the candidate to get back on track if they stumble. This improves the candidate experience, which is important even for those candidates you may not choose to hire.
- Set up the discussion in phases, allowing the candidate to showcase different aspects of their thinking. A common break out is (1) Diagnosis, (2) Solution Design, (3) Solution Implementation.
- Use a checklist or other measurement system for your problem-solving question to help you gauge candidate performance objectively.
II. Staying in Sell Mode
When you’re a small startup with limited resources and a low profile, you might find it tempting to jump into sell mode during your interview. But telling a candidate all about your company won’t help you determine whether he or she is right for the job.
Unfortunately, you can find yourself in sell mode before you even realize it’s happening, Burton explains: “You and the candidate walk from reception to your interview room. There’s 30 seconds of chit-chat, and the candidate asks about the product you just launched. You tell them more about it, and suddenly you find yourself in pitch mode, talking about how amazing everything you’re doing is. But what you’re actually doing is saying, ‘You are clearly awesome, and I’m here to impress you.’ You’ve put them on a pedestal.”
The power dynamics of human interaction can make transitioning back to the interview somewhat tricky.
“It’s hard to say, ‘You know all this amazing stuff that I told you about our company? You may not have access to that. So let me elevate my pedestal and now interview you.’ It’s an uncomfortable, rapport-breaking moment.”
Instead, preempt any shift into a sell conversation. Tell the candidate you’ll create a separate space to go through all of their questions at the end. Be friendly — say that you’re excited to tell them all about the company — but that you’re going to start off just by getting to know them.
III. Hiring for An Ambiguous Future
Startups frequently face the challenge of hiring for today’s needs vs. hiring for the future, and end up bringing on someone for a role that really doesn’t exist yet. For example, you may know you need a strong individual sales person to generate your first non-founder-led sales, but you also hope this person can transition into a sales leader as the team scales. You hire for tomorrow’s need, and end up with a frustrated new team member who was ready to build a team. How do you accommodate this when it comes time to evaluate candidates? Burton recommends splitting the facets on your target document into “Table Stakes” and “Nice To Haves.”
Table Stakes are those facets of the role that are absolutely essential for your near-term needs. Nobody gets through your process if they do not rate well against those facets. Nice To Haves are a “bonus”—it’s great if the candidate rates well against these longer-term facets, but it’s not required. If you have two candidates, one who meets the Table Stakes, and one who is great on the Nice To Haves but marginal against the Table Stakes, hire the first one. And if both candidates are marginal on the Table Stakes? Continue your search!
All too often startups see hiring as a relative game—in other words, they are looking for the best candidate out of the five they’ll be seeing in a given week. This is a dangerous approach—if none of your candidates rates well against the results and competencies on your Table Stakes list, go find another five candidates, even if some of those you have right now look solid on the Nice To Haves.
IV. Avoid At All Costs
While obvious, these are worth calling out. Be careful in the frantic pace of interviewing your entire team does not make these mistakes, which can otherwise kill a great interviewing process:
- Discussing a candidate within earshot
- Letting your schedule overrun forces a candidate to wait
- Picking up your phone — don’t even bring it into the room
- Frowns, scowls, glares, and other ambiguous or negative body language — consider having your colleagues “shadow” you in interviews, and ask them about the impression your body language creates. Is your “curious” face in fact a scowl?
Make Data Your Foundation
My partners and I have seen firsthand that the best companies make hiring a priority. And, more than that, we’ve observed that the most successful founders are those who are prepared to create and implement a high-quality, data-driven interview process. The results can make all the difference – as there is no single factor more critical to a startup’s success than getting the very best people on your team.
Comment below or share this post:
“Talent is the limiting resource in everything we do right now.” A data-centric approach to interviewing. @dcstalder https://t.co/qu0x0bgaQy
— Matrix Partners (@MatrixPartners) December 3, 2015
