Interviewing
March 4, 2007 | By Joel Dehlin | 13 Comments
Whatever you think about the book Good to Great it’s hard to argue one of its premises–that great companies don’t exist without great people. I’m a believer.
In my experience a great engineer can be equal to two, three or even more average engineers. They have good attitudes. They’re productive. They do things right and minimize re-work. They’re not defensive. They communicate with others effectively. They look for things to do when they’ve got spare capacity. They’re easy to talk with. And they inspire others. I just love them. People like this are easily worth what their skills and experience demand in the market.
So how do you find them?
That’s the key, huh? With a crack (or even average) HR department, finding people to bring in for interviews isn’t terribly difficult. It’s a little harder at the Church as the intersection of individuals who have temple recommends, want to live in Utah, have the skills & aptitudes we’re looking for and are willing to work for less than market wages is small. Still we’re able to get people in the door. A good HR department will not only bring many people in the door, but a healthy percentage of them will turn out to be the folks we want to hire.
Once folks are in the door for interviews, however, the hard work begins for the rest of the team. People do not inherently know how to interview. Discovering “great people” in an interview is not intuitive. It takes training, preparation and a good measure of thoughtfulness.
Here are some suggestions that can improve your interviewing techniques.
- Interviewing for experience is one of the biggest mistakes people make. You can read someone’s experience on a resume. Prepare prior to the interview by reviewing the resume carefully. Don’t waste your time asking questions you could have found out with a little preparation.
- Figure out what you’re going to ask ahead of time. Write down the questions you’ll ask and think about what you’re trying to discover with each question.
- Save time to write down your feedback after the interview. It helps you process the information you’ve gathered and will help down the line when you’re looking back.
- Smart candidates ask lots of questions and keep the interviewer talking. People love to talk about themselves. I’ve had many, many managers tell me they loved such-and-such candidate, where afterward I’ve asked detailed questions and found the person was duped into telling the candidate about his job, the organization, his family, the weather, what’s it like to live in Utah, etc. The “sell” you think you’re accomplishing with an interview like that just isn’t that important.
- I look for three things when interviewing a candidate: intelligence, passion for technology, attitude. Finding questions that test the second two is easy. The first is more difficult. I like asking problem-solving questions. In my opinion, questions that require an “a-ha” moment, some flash of non-intuitive or non-deducible inspiration, aren’t that useful. I prefer questions where you can watch them think. Encourage them to think out loud and/or to use the board. And I’m perfectly willing to let the candidate stew while they think through the hard questions. You need people who can think.
- Leave your technology biases at home. So they love Linux and you don’t–so what? So they use a different editor than you do, or they put their curly braces in the wrong place–so what? I don’t care if a candidate speaks COBOL, Java, .net, Ruby, Fortran, Forth, LISP, or any other language, for that matter. A candidate’s opinion on which operating system is most secure is just not super relevant. If she can solve problems then her technical biases don’t matter.
- Ask real questions–even simple ones. It helps you see them think in context. If you’re hiring a developer, ask them coding questions. If you’re hiring an architect, have them create architectures. If you’re hiring a business manager, have them write a business plan with you.
- Try to use the same questions for a set of applicants coming through for the same job. It lets you get a relative view.
These are just a few ideas. The one point I’d get across is this: take recruiting and interviewing seriously. Getting great people is the most important thing you can do as a manager or as an organization.
What additional tips do you have?
Eric Poulin said...
I have to agree, the “think on the spot” questions you pose in interviews are great to identify great thinkers and those not in that category. Too many times I’ve had people in interviews ask very specific code-level questions based on their experience (which is often not shared by me), and when I get stuck on those its disappointing. I agree that the resume should clear up experience.
Its important to see how the candidate can solve a problem with code, if thats what they are being hired for, but the problem should be a logic problem that doesn’t require using data structures or API they may not be used to.
[JPD: Couldn’t agree more.]
March 5, 2007 5:51 am #
Michael Schaffner said...
Joe,
Great tips.
To me the ability to communicate is very important. One of the things I like to do is have them tell me “story”. I want them to describe a problem situation and how they resolved it. This does tell me something about there problem solving skills but more importantly it demonstrates their communication skills. Can they formulate a concept, describe it well and in a way that I can understand? Does it hold my interest? Do they develop a flow and logic in their communication that naturally draws to the conclusion.
I’d rather have a good communicator that need technical development than vice versa.
Mike
[JPD: Good suggestion, Mike. Sometimes I’ll ask people to think of the technology in which they’re deepest. Then I’ll ask them to explain the technology to me as if I were their grandfather. You’d be surprised how many can’t come close to doing this acceptably.]
March 5, 2007 6:43 am #
Barry Sandall said...
One thing I’ve put in place, which has worked well for us, are interview teams. These are individuals who are particualry adept at interviewing. We set up the teams based on technical focus areas. For example, we have a .Net interviewing team of which I am also a member. Anytime we receive a promising candidate with .Net, regardless of the manager he or she will be reporting to, they have to go before the interview team. Over countless interviews, we have found a standard set of questions which work exceptionally well. Every candidate is asked the same basic questions. From there we may may delve deeper into specific areas, but atleast the basics are covered. It’s very similar to the temple interview process. Each candidate is given a score on a scale from 1 to 10 by each member of the interview team, and the scores are averaged. This provides us with a comapny wide standard we can use to measure .Net candidates. The hiring manager then has the information he or she needs about a candidates technical abilities to make a hiring decision. This has worked very well, and we have some of the best and the brightest employees because of it.
March 5, 2007 8:03 am #
Matthew said...
You have mentioned Ruby in your blog entry. Ruby is developed by the member of the Church, Yukihiro Matsumoto from Japan.
March 5, 2007 8:15 am #
Tom Welch said...
One ounce of “great attitude” is worth a pound of “great intelligence”. I’ve found that someone that is brilliant but has a lousy attitude is more of a hindrance to the team or an organization than a guy that has a great attitude but is not quite as smart.
Tom
March 5, 2007 8:52 am #
Larry Beck said...
I used to interview people for help desk positions. I got “dumped” into doing these interviews because I had Macintosh skills and no one else did, so when my company needed to hire a lot of Mac techs they needed someone who could talk the talk. That was me. I hadn’t done any interviewing prior to that time, so I checked out a couple of books at the library and talked with a HR person who attended the same ward as I did. I thought long and hard on what qualities I perceived were essential to answering tech questions over the phone, and came up with a short list of questions that would determine if they had the “right stuff”. It takes a certain personality type to be successful at phone support, and since I was also training those I hired, I wanted to find people I know will succeed.It turned out that the most important question I asked was: “given what you know about this position, what three personality qualities do you think are essential to be successful?” I of course came up with my own three, which are: persistence, enthusiasm, and curiosity. I hired some who were very smart, but lazy, or didn’t follow through,, or just flat out didn’t care what answers they gave. But if I hired someone that came close to my three qualities, they almost always were successful.
And now, long after the fact, I realize that those three qualities will take you far no matter what the job or task.
Say what you will, but the spirit of revalation played a big part in my hiring decisions. There were some I just knew would either make it or not, but not based on anything they said. That happened more than once and it was a great comfort to me, knowing that I have “the ultimate source code” on my side.
March 5, 2007 1:37 pm #
Benjamin Hofmann said...
This might not be exactly on topic but the phrase “work for less than market wages” seems inconsistent with the Church’s current technology strategy. The Church seems to spend money on expensive technologies, like Vignette, and you would think they would spend the money on talented people to support it. I am pleased that the Church is frugal with donated funds but I would imagine it would be easier to find good people if you at least paid industry average for the geographic location. Why doesn’t the Church pay at least industry average for its technology staff?
March 5, 2007 3:48 pm #
John Lockwood said...
Mr. Sandall mentions “interviewing teams” as a potentially good process for us to seriously consider, yet a particular candidate may be the brilliant introvert who does a Rubik Cube in record time, writes in Binary, but is totally intimidated in groups, especially those he is unfamiliar with. I do like the “standard set of questions” principle.
March 5, 2007 10:28 pm #
Doug said...
“…afterward I’ve asked detailed questions and found the person was duped into telling the candidate about his job, the organization, …what’s it like to live in Utah, etc”
I agree, you probably don’t want to spend the whole interview answering questions, because you need to find out about the candidate. On the other hand, if you aren’t willing to spend at least some of the time answering questions that are important to the person being interviewed, you aren’t going to get the good candidates. Especially when the market is good (like right now), the good people usually have other options, too. If you can’t give them good, honest, satisfactory responses to the questions that will influence their decision of where to work, they will likely go somewhere that they are more comfortable that they know what they are getting into.
You probably won’t sign someone on if they don’t (or can’t) answer your questions, so you probably shouldn’t expect them to sign on if you won’t spend at least part of the time answering theirs. Just my two bits as one who has sat on the other side of the interview desk (not with the Church, though) in the not-so-distant past.
[JPD: Agree 100%, Doug. We should always spend time answering questions, but just not cut ourselves short by spending too much time. I usually try to leave 5-10 minutes for questions in an hour long interview.]
March 6, 2007 2:28 pm #
Catch the Best » Blog Archive » Interviewing Tips said...
[…] Dehlin, CIO for the Church of Jesus Christ of Latter-day Saints, has some good pointers on interviewing. Having gone through a number of rounds of interviewing and hiring, I can agree with what […]
March 7, 2007 12:59 pm #
Doug Von Feldt said...
As a follow up to the “Interviewing Teams” comment and “Intelligence” comment, I had an interview at a company several years ago that was very interesting. It started out as a “Team Interview” with about 12 people who made of the senior management of the company. We talked for about 1/2 hour as a group initially, which was rather intimidating and interesting.
After that, I met individually with four of the department heads from various areas. One was the head of Engineering, one was the head of Finance, one was the COO, and last was the head of the HR department. During the interviews they asked a lot of questions, none of which were technical (this was for the CIO role). They asked many situational questions or “how would you do this” or “how would you respond to this”. I answered them and thought that I gave good answers.
After the individual interviews were completed, we met as a group again. The CEO then asked each of the department heads that I interviewed with how I did in the interview with them. They were honest and candid about what they liked and didn’t like about me. That was interesting being in an interview and being critiqued and given feedback on the interview in a large group setting. Then came the intelligence test. The department heads discussed with the group what questions they asked me and then they asked me to summarize my response to each of the questions. This may not seem a big deal, but when you just spent several hours answering questions (sometimes the same questions) and then trying to remember exactly what you said, it is not that easy. What I realized after a few minutes is that the department manages wrote down a summary of my answers and if I strayed from what I was repeating back they would let me know. So I ended up having to summarize hours of answers in front of 12 people. I suppose I did ok because they offered me the job, but I decided it was not the right job for me for some other reasons.
This was a very interesting interview technique and I have never forgotten it. I use this in my interviews I do now and find that it does help me know if people can remember well and if they are paying attention.
March 7, 2007 3:23 pm #
LDS CTO Guy said...
It’s always fun to probe into the candidate to see how dogmatic they are… I mean, a lot of time people hear things from influential people and without much research or thought of their own, regurgitate the same as if on auto pilot. I call these people Affiliates. I seek to hire Leaders, not Affiliates. So, for example, when I discover a candidate to be a staunch .NET dev (or Java or PHP for that matter), I ask them questions around why one is “better than the other” or “preferable, and in what context”… Depending on the answer, I ask them to defend the design/architecture choices of the particular technology they are quick to criticize as if they were the architect responsible for it… A simple game of paradigm shifting and it yields an amazingly thorough and better yet (sometimes) introspective interview. I have been interviewing for years and it never ceases to amaze me how little people actually think for themselves, either due to lack of intelligence, confidence, or experience (or too much experience for that matter).
March 13, 2007 8:16 pm #
Justin Bradley said...
It has been interesting to read these comments. Many of the commenters here are interviewers. But few seem to be interviewees on a regular basis. I am currently graduating with an MS in Electrical Engineering so I have been interviewing for jobs like crazy. Let me offer a few suggestions from the opposite side of the table.
Some of the most obnoxious interviews I’ve been in are when I am asked to describe when I might use the C++ keyword “volatile” and what it means. Honestly, does this say anything about my competence as an engineer? Isn’t that why I spent $50.00 on the C++ reference book sitting on my book shelf (by the way, I did answer the question correctly)?
I also don’t agree with this interviewing technique of making me “think on the spot.” This is not real engineering. No one is asked to design a controller for a memory device in 10 minutes.
Basically here, I am disagreeing with Joel. I think the latter two “passion for technology,” and “attitude” are the important things to interview for. I think he was right when he said not to interview for experience. I would add to that, not interviewing for intelligence. This can be gleaned from a resume or transcripts. Anyone graduated with an MS or higher with a reasonable GPA can be considered intelligent. In fact, I have felt nearly insulted at times when interviewing and the interviewer asks me some basic programming question. In my mind I’m thinking “didn’t I learn that in CS 101?” Basically, if I am called onsite to interview then I assume you recognize there is some level of intelligence and it is adequate for your needs. The real question is whether or not I can solve problems.
So now that I’ve said what I don’t think is important, let me offer some suggestions for what IS important in my mind. I appreciate when interviewers ask detailed questions about my past projects and or research. This shows they care about my ability to solve problems. That’s what engineering is all about after all. It is not about my ability to recall all the C++ keywords on the spot. I think we can all agree that that does not make a good programmer.
After initial screening, during the interview process, the goal should be to find out how good an engineer I am. How well can I solve problems? The kind of problem doesn’t matter. A good engineer is characterized by how well he can take what he knows and combine that with what he doesn’t know, but will find out, to solve a problem. This information can be discovered by discussing in detail my previous work. When I say “discussing in detail” I mean very literally have the applicant draw block diagrams on the board. Ask them to write down UML diagrams and schematics. Ask them to describe the thought process and the approach to solving the problem. And for software engineering please ask about their coding style, and debugging techniques they have used. These may be just as important as how well they solve problems.
Lastly, there are things I want to say about these greuling interviews. As I have interviewed often the process goes like this. Fly 1000 miles somewhere (often at a very inconvenient time since I had other things going on). Pick up a rental car and try to navigate a foriegn city. Get to my hotel late at night and try to get some sleep. Wake up at 5:00 a.m. so I can get ready and have enough time to navigate the foreign city again. Now I’m at the site and begin interviewing. Somehow I’m supposed to answer technical questions even though I’m dog tired, and I need to have the presence of mind to ask good questions which I will probably forget anyway since I’m interviewing with 8 people in one day. Aside from me being interviewed, I am also interviewing them as a potential employer. That means that in spite of all that’s going on, I need to make some sort of judgments about this place and the projects and people here. After interviews I have to navigate the strange city again back to my hotel, or maybe straight to the airport for another 1000 mile trip home most likely arriving at a very awkward hour.
My point is that, being an interviewee is very taxing. Keep that in mind. Also keep in mind in interviewing that the employer is also being judged. I remember one place I interviewed with in which one gentlemen bad-mouthed the other places I had interviewed with. I immediately formed a negative opinion and it was easy to decline their offer.
I hope some of my rambling has been useful for seeing things from the other side of the interviewing table.
[JPD: Thanks for the comments, Justin. You would be shocked at the number of people who come into interviews with what seem like ideal pedigrees and yet perform miserably in very basic questions: reverse a string in the language of your choice, help me understand the architecture of a major coding project you worked on, etc. Divining which candidates will work out for your company is such an important thing, and is hardly a science. I hope you do well in your interviews!]
March 17, 2007 12:30 pm #