Fascinating use of "Big Data" to cut through the bullshit. Wonder if it will change anything. I suspect the "tough" interview plays well into a company's PR.
from: "The trick Max Levchin used to hire the best engineers at PayPal"
Levchin realized the best engineers wanted to be challenged both in their jobs and in the interview process. “We cultivated a very public culture of being incredibly hard to get in. Even though it was actually very hard to get good people to even interview, we made a point of broadcasting that it's incredibly hard to even so much as get into the door at PayPal. You have to be IQ of 190 to begin with, and then you have to be an amazing coder, and then five other requirements. The really, really smart people looked at it and said, "That's a challenge. I'm going to go interview there just to prove to these suckers that I'm better." Of course, by end of the conversation, I'm like, "Maybe you want to come get a job here because you're pretty amazing.”
http://firstround.com/article/the-trick-max-levchin-used-to-...
The simple truth is large companies end up with a few geniuses randomly but there rare enough to not be worth optimizing for. What companies really want are people willing to work ridiculously hard for little reason and that's what 'hard' interviews are optimized for. O your willing to put up with hours of BS on the off chance we will higher you, great let's just see how you like 60h+ weeks.
Well that, or: http://en.wikipedia.org/wiki/Dunning–Kruger_effect
(which might not be a problem)
Seems like it would fit into a normal database. Or maybe even an unwieldy Excel spreadsheet.
"sometimes I think Big Data is just Excel on 128GB of RAM"
We get it, big data it's not really "big" unless you talk giga(tera?)bytes.
Do not take upon yourself to educate any single person that misuses the term. It's not worth it. :)
Its "not" big data in the sense that it needs a cluster to process but it is a pretty large sample set of the current population of engineers who might want to work there.
I realize that reporters like to throw buzzwords into anything to cater to the "simpler" readers. But come one, this is outright silly.
But, more recently, in conversations with non-programmers, I see that 'big data' to them, means 'broad data' - it means trying to track everything possible and make sense of it. The average business user is really excited to be able to cross-relate disparate types of data - in an effort to make things better. 'Big data' enables the breaking down silos and enabling of cross references. It's about making empirical decisions based on data rather than opinion or intuition. That's really good, in my opinion.
So, 'big data' in that way is more amorphous than just the size of the data. With services and networks, the question becomes where does the data begin and end? Big data is potentially everything.
eBay has a 9 petabyte DW that cuts across all of the types of data on their whole site: listings, bids, feedback, categories, clicks, etc.
Sometimes big data is actually big data, both in terms of raw size as well as complexity.
http://gigaom.com/2013/03/27/why-apple-ebay-and-walmart-have...
"Monopoly" is a perfect example of a brain teaser and anyone using that would be treated pretty harshly by the committee that reviews interview feedback. Estimate questions are not: there's no expectation of a "right answer" and the important fact is the working.
Some programming questions border on brain teasers to non programmers but that doesn't matter because you are asking programmers and again, it's the working that matters.
Finally different roles get different types of interviews. I worked in PM and there were analytical (these questions), product (design a better x) and technical (basic engineering interviews). The behavioral type was not one I encountered in PM or eng but maybe used elsewhere.
This is almost exactly the same effect as the fact that SATs do not predict college GPA. SAT scores are used as one of the major factors for admission to colleges; having separated the students into cohorts based (partially) on SAT, it is completely expected that SAT scores have minimal correlation with grades assigned within each cohort.
http://www.nytimes.com/2007/01/03/technology/03google.html?e...
Which there have already been a few HN discussions about:
https://www.google.com/search?client=safari&hl=en&ei=ZfzCUYa...
It shows several stories at once, not just the google brainteasers stories.
2. In general, people would rather read the real article instead of a summary of the article. When someone submits a summary of an article to a site like HN or reddit, it is usually flagged as blog-spam because we'd rather read/support the original content than a summary with questionable value.
3. For long form articles, nothing beats reading the print-preview page to get rid of all the sidebars, comments, ads. Look at the print preview page: it is not possible to get less distraction free than that. Any other format has more distractions.
Even aside from that, the New York Times has some of the best information architecture in the business. These are the guys who did NYTProf. Their web team is awesome.
http://www.nytimes.com/2013/06/20/business/in-head-hunting-b...
4. Some visual issues I had with quartz:
4.1: No left/right whitespace around images.
4.2: I see a vertical scroll bar in the middle of my screen on Firefox.
4.3 The black header bar which is fixed and stays on the screen all the time even though it conveys no useful information to me.
4.4: A bunch of text blurbs on the left side of the screen that convey no useful information to me.
You say you're trying to be as distraction free as possible, but that's not actually true because it isn't possible to have your business model and be as distraction free as possible. The print preview page is as distraction free as possible.
1 - Viewing it in Android Chrome just doesn't work. See this comment and its responses: https://news.ycombinator.com/item?id=5907648
2 - On some pages (with full-screen window), some or all of the thumb of the scroll bar is hidden on my browser under the black bar.
3 - As I drag the scrollbar down the size of the page jumps so my mouse pointer is no longer on the thumb.
4 - Sometimes I use the space bar to scroll because my hands are on the keyboard, not on the mouse. It doesn't work unless I click first.
5 - I often tile windows on my machine. This window is exactly width of the left-hand half of my screen (2011 MBP). Because of the responsive design I get a ToC when I visit the home page, which is annoying. It feels like the developers said "all users will have maximised windows".
Other frustrations I've had in the past but can't remember now, perhaps it was changed since last time I tried to use it on the desktop. I just get the feeling that (and Quartz isn't alone in this) the developers tried to re-implement functionality and didn't do it well enough to be worth it.
Screenshot: http://imgur.com/fFqt2ud
Maybe I'm a bit irrational about this, I know scrolling isn't hard, but I came to read your text, dammit.
Edit:
Also, seriously... HUGE thanks for asking!
Edit 2:
Scrolling down I'm noticing that your image height is actually bigger than the height of the viewport (minus the height of your top bar). Even if you disagree with the idea that the image shouldn't push the text below the fold, I hope you agree that the image should at least fit my viewport.
- Scrolling down into the next article and having the article list as a sidebar is a neat way to encourage exploration. It makes the site a bit more "sticky".
- I like that ads are unobtrusive and placed at the end of the articles. And unlike nytimes.com, there's no paywall.
- I find the site's overall design crisp and relatively uncluttered compared to most news sites.
I say cruft because I'm reading on mobile. I've got a Galaxy Note 2, and I'm using Chrome. It's hardly a slouch of a machine.
First, after clicking the link, I have to watch your spinner for a ~5 seconds. I'm on 65Mbps ADSL - just how much content are you sending me?
Second, the sheer weight of the JavaScript slows everything down. My browser is perfectly capable of scrolling through a page - yet because you've overloaded that, the scrolling is slow, jumpy, and the text renders poorly.
Thirdly, the Note has a huge screen, so I don't mind your static header. If I had a small screen it would piss me off.
Finally, when I switch from Portrait to Landscape, your page jumps all over the place.
Now, compare that to the Mobile NYT page. Yours looks more beautiful, but the NYT is quicker, easier to read, and doesn't get in my way.
You have great content - and an interesting product - but it needs to go on a diet and be user tested on a wider range of devices. I dread to think how it performs on low end phones.
T
On that article, I was completely unable to bring the location/page-controls back into view -- I was stuck on that page, trying to figure out how to escape -- until I clicked a link - then the controls showed up while it was loading the new content.
huge fixed banner at the top of the page, and a fixed sidebar.
We get feedback specifically calling this out as good and relatively distraction-free (hence your comment struck me as odd)
Also - the width of the page dedicated to text/images is the same as the NYT mobile site (600px aprox)
It's a waste of space and it makes it much much harder for me to read on my laptop. Vertical space is way too valuable to throw away like that.
After all, they're just changing the mechanism, not lowering the standard.
Then again, they also have some of the best Engineers. But I don't think that's a testament to a great hiring process, as that could be the result of great marketing. The "we allow you the freedom to actually do stuff and to work with the best" type marketing. From what I read/saw, the great Engineers didn't seem that happy, so maybe those marketing claims aren't really true, but perhaps things have changed since then.
And if you're eliminating the former kind of hard without increasing the latter, then your interviews should get easier. Heck, for people in the latter group it will get easier even if you decrease the former and increase the latter, to a point.
Just because you're not lowering standards doesn't mean your interviews aren't getting easier.
Which is also a thing with exams now I think about it - just because your exam's hard doesn't mean it's worthwhile.
For instance - "how many hard drives does Gmail need?" requires a rough guess of how many users Gmail has (if you're interviewing at Google, you should know it's 1e8-1e9). How much space each one takes (probably nowhere near a gigabyte on average - let's say 1e8 bytes). And that the current capacity of hard drives is (1e12 bytes).
Then you can say that they probably need 1e5 hard drives, link it to redundancy, availability, deduplication, backups etc. You can comment that it's feasible to build a datacenter with that many hard drives.
No one cares that the actual number is 12,722 - but you've demonstrated a broad set of knowledge about the current state of technology. Saying "dunno - a billion?" is not going to get you anywhere, and with good reason.
The Monopoly question is crap, though.
I'd like to know how useful http://google-tale.blogspot.com/2008/07/google-billboard-puz... was.
However, I used to ask one estimation question (How many hours have you spent coding over the course of your life) on all my interviews. Over time, I lost interest in it because almost everyone got it "right" (took an acceptable route and arrived at a reasonable estimate). The only people that got it wrong were ones I decided to reject for other reasons (this being a semi-technical but mostly ask-about-experience phone interview).
So, although I agree it's a useful skill, I don't think it's worth askin estimation questions from my personal experience.
Why? Is the interview a trivia exam like in college?
Not sure if you were referencing this bit from comedian Pat Brice, but here's a relevant video: http://www.youtube.com/watch?v=J2nrKdECSGU
These estimates are completely useless in real life, because in real life nobody guesses how many drives you need for GMail, or how many gas stations there are in LA.
Oh yes they are, for getting a grasp on what real life entails and what's possible.
With all the NSA scandal/hysteria going around, lots of people are approaching the issue with the presumption "gee, they can't possibly record everyone's phone calls". With a quick estimate I figured recording everyone, all the time, in CD quality, would take just 5% of the federal budget - making it doable instead of improbable or impossible, and making subsets of the scenario (i.e.: just recording phone calls) likely. For those of us who remember 10MB hard drives and 5.25" floppies, such a data scale is staggering - but it's a current reality, and a little estimating provides a reality check.
Likewise grasping the concept of, or even implementing, high-res "eye in the sky" drones. Gigapixel cameras seem like a novel futuristic impractical concept ... but then a little estimating involving HD-quality cell phone cameras, you can realize that a 24/7 flying 30fps gigapixel camera drone is in fact quite possible for a relatively modest sum (speaking in jurisdictional law enforcement budget terms). (Takes less than 200 cell phone cameras and a suitable multiplexer & high-bandwidth downlink BTW.)
I had an epiphany about accounting (!) when touring a billion-dollar timeshare (hotel/condo) project. Wanna make a billion dollars? Pick some large expensive project, then estimate your way down to a plan for pulling a few dollars out of a LOT of wallets by dividing, dividing, dividing away into manageable chunks people are willing to shell out a few bucks for.
Such estimates are exercises in how to mentally manage very large scale money, personnel, opportunities, and processes. Wanna make a billion dollars? Charge a buck profit per window to wash a thousand windows for each of a thousand businesses every week for 20 years. Don't laugh, there's some really rich people who made a lot of money charging a buck at a time - because they estimated their way into a profitable vision.
Your point that interviewers read the wrong signals from candidates' responses is a very good one, but it's not specific to estimation questions. It applies as well to straight-up programming questions and probably a lot more.
It's also good for sanity testing, it's a useful skill to be able to spot that something is out by an order of magnitude as it can allow you to catch problems early on.
In any case it's one of those things where you tend to do well if you've prepared for it and do pretty poorly if you face it for the first time. Probably a better fit for interviewing management consultants than programmers for sure though
Do you never do sanity checks? If you want to compute the number of drives you need for GMail, you estimate, do the computation, make sure they agree, and then have someone check your work.
You now need to make decisions about UI (map or list? what's a good default map zoom level?) and infrastructure (how much gas station data will a single user need in a single request? how does that impact my storage?) and a whole lot of other places.
You need a nice representative metro area that first a realistic worst-case scenarion, say LA.
How is any of this unrealistic?
> How much should you charge to wash all the windows in Seattle?
> Design an evacuation plan for San Francisco
> How many times a day does a clock’s hands overlap?
> A man pushed his car to a hotel and lost his fortune. What happened?
> You are shrunk to the height of a nickel and your mass is proportionally reduced so
> as to maintain your original density. You are then thrown into an empty glass blender.
> The blades will< start moving in 60 seconds. What do you do?
The problem with trick questions and standardized answers is that the nature of the question makes the candidate uneasy and even if they eventually figure it out, nobody will have learned anything during the process. It's more like a hazing, not a hiring interview.
horrible question, it took me a few seconds to figure out they are talking about the game Monopoly.
On that last point, I recall interviewing at Microsoft: Asked questions about automatic control of venetian blinds, for one question I knew I was missing some obvious simple checkbox-type answer. I told her "I know I'm missing the obvious here, so I'm just gonna pick some alternative solution and talk thru it so you can see how I think" and proceeded to elaborate on a complex yet viable & marketing-impressive implementation. Wasn't gonna let some "correct" answer stand in my way...
The point of the question is to establish how well you gather additional information when the initial description is unsatisfactory. Anyone who's been an engineer knows you have to do this every single day.
(Not that it's a good interview question.)
It seems like a hazing but perhaps it's a bias on the part of the person interviewing. Meaning, the interviewer wants to work with people very much like themselves, instead of hiring the best candidate.
I think one of the best interview strategies was a recent posting: https://news.ycombinator.com/item?id=5779406
...or anything similar to the FizzBuzz problem.
Long before this was reported in the New York Times, this was the finding of research in industrial and organizational psychology. A valid hiring procedure is a procedure that actually finds better workers than some different procedure, not a hiring procedure that some interviewer can make up a rationale for because it seems logical to the interviewer. We have been discussing home-brew trick interview questions here on Hacker News for more than a year now.
https://news.ycombinator.com/item?id=4879803
Brain-teaser or life-of-the-mind interview questions do nothing but stroke the ego of the interviewer, without doing anything to identify job applicants who will do a good job. The FAQ on company hiring procedures at the Hacker News discussion linked here provides many more details about this.
http://ycombinator.com/newsguidelines.html
(I mention this because many comments in this thread will be very confusing to newcomers if it is not made clear that the thread used to point to Quartz but now points to the New York Times.)
I still maintain that the piece was original. It was based on (and expanded on) one of the eight points made within an NYT article.
To me "In Head-Hunting, Big Data May Not Be Such a Big Deal" could not describe the same article as "Google admits those infamous brainteasers were completely useless for hiring".
Even the headlines indicate two separate directions.
Please read again and compare this (relevant passage on NYTimes will be Highlighted):
http://www.nytimes.com/2013/06/20/business/in-head-hunting-b...
to this:
http://qz.com/96206/google-admits-those-infamous-brainteaser...
Serious question - am I completely off the mark?
From the original New York Times article that Quartz has linkspammed
Link spam is defined as links between pages that are present for
reasons other than merit.[9] Link spam takes advantage of
link-based ranking algorithms, which gives websites higher
rankings the more other highly ranked websites link to it.
These techniques also aim at influencing other link-based
ranking techniques such as the HITS algorithm.
Lets be clear, there is link-spam and then there is writing an original piece based on information from elsewhere.
The NYT article is about 8 questions and answers from a HR person at Google.
The "puzzle" aspect is 1 of those 8 questions.
From that Quartz references that, links directly to the piece and then expands upon it and links out to other relevant and related information.
A similar equivalent in coding interviews would be "what does obscure function/feature do?"
There must be thousands of people on HN who interviewed at Google over the years. Did anyone ever get a question like this?
Sometimes the questions feel a little bit gimmicky in that they aren't really representative of what software developers spend most of their time doing. I understand that one guy at Google had to come up with a clever algorithm to figure out a snippet to use in the search results, but that can't be what the thousands of developers at Google are doing all day.
> "On the hiring side, we found that brainteasers are a complete waste of time"
That implies Google has some data to back it up, whether they themselves previously asked those types of questions, or they derived it from some other means. Either way, they aren't doing themselves any favors to dispel that urban legend. Most reading that will just assume they used to ask brainteasers, but no longer.
There are many posts online about the actual, CS-y questions that you can expect in a Google interview, I had just assumed that the mentions of brainteasers were merely urban legend.
Ever since, whenever I've interviewed someone, I ask them to demonstrate their strengths to me first.
Interestingly, I believe Google are slowly moving over all their coding interviews from whiteboards to Chromebooks - this is what I was told by my Google recruiter when I last interviewed with them, anyway.
The whiteboard can be a bit polarising...I love whiteboarding code, but I suspect many people detest it with a passion (I used to teach CS, so it's something I picked up on the job). I do think it is rather unfair to have candidates whiteboard and demand syntactically correct code, especially when under pressure. There's room for flexibility.
Nice. That agrees with some of the other comments here. For example, about asking about past work or projects that they are proud of or that demonstrate their skills.
Another hard issue is what to do, as an interviewer, if things start to go downhill to the point where the candidate becomes flustered and you can tell they're not at their best.
He asked how many plumbers worked in the city, to which I replied you could check the industry registry for qualified plumbers, you can probably filter them out by city. There was silence then I had the question clarified to how many 'plumbing businesses' where there not individual plumbers.
To which I replied you could get the company registrar office but it was impossible to calculate as so many plumbers work full time while also holding businesses of there own as free agents. A very unimpressed look came across the guys face and I was told there is a very simple way to find out and asked to try again.
I sat in silence for a 30 seconds or so trying to think of something that would be more thorough than the registry offices, I think offered a few alternatives like tax department records, government statistics office. All things I could think of that would keep fine grained data. But I could see the guy growing impatient with me so I stared at him and asked him what a better metric was than what I had offered.
After a few moments I was told the correct answer was to check the phone book, any practicing plumber business would be listed.
Startled but what seemed like a completely faulty answer I pointed out what seemed obvious to me... not every business needs to have a public listing... some deal directly as sub contractors ... some could be umbrella companies for subbies ... again some are free agents... some might use unlisted cellphones... not everyone is a legal company, not all plumbers where qualified. It was a terrible way to get a dataset you could rely on.
Angry swept across the guys face and I was told sternly I was wrong the data was perfectly suitable, onto the next question... which was all down hill from there as he didn't want to hear my answers, didn't challenge me back, just rip through the rest.
To this day I laugh when ever I think back to that interview. It was probably the most uncomfortable interview I've ever been in.
Because fixing interviews is harder than just working out hwat questions to ask.
I had fun and wouldn't mind it again, it didn't feel like a bunch of stupid random brain teasers like I've experienced before (how many t-shirts would it take to make sea worthy sail? why are manholes round?) etc.
Interesting; when I interviewed for a management position (test manager) it was nothing but coding questions, including the infamous "reverse a string" question. ("Would like that optimized for space or speed? In-place, or do I get a buffer? Can you tell I've heard this a zillion times before?") I can understand wanting a test manager to be more than an empty suit, but yoiks.
1. Invent a bunch of silly riddles that a non-technical reader might accept as tech interview questions.
2. Pull a major tech company out of a hat (today it's Google), and claim with no evidence that their interviews are based around silly riddles. The article will be cited for years as proof that people working at $COMPANY are weird and obtuse.
3. Wait a couple years. Ignore all evidence that $COMPANY does not use silly riddles in interviews.
4. Once traffic on the original article dies down, write another article claiming $COMPANY has "admitted" silly riddles aren't useful for interviews.
http://www.nytimes.com/2013/06/20/business/in-head-hunting-b...
That said, the riddles listed are of course a bit clickbaity but they did not conjure the story out of thin air.
For example, write a program to find every possible word in a given Boggle board is a puzzle, but one you're going to solve by coding...rather than "how many piano tuners are there in New York", which is a rather different matter. I've interviewed on-site with Google several times, and always found the CS puzzles to be challenging but fair.
Isn't the only difference between "brainteasers", "puzzles", and real engineering challenges, just the usefulness of the result?
I get what you are saying though. Asking someone challenges rooted in technology seems so much more useful and natural than something involving ping pong balls and Lake Michigan.
We used it to build our hiring process for http://www.thinkful.com/ and it consistently proves valuable.
We also use it to help our students prepare for job interviews.
Have you gathered much data on the success of this book's approach in your firm? I'd love to hear a positive take on it.
I had a couple questions like this at a couple of interviews more than a few years back now. In both cases, I sat for a minute, and asked a few questions back, like "do you mean the city limits of Raleigh, or the metro area?", "how do you define gas station - do we include public-only, or private fueling places?", etc. Part of this was buying some time, because the question caught me off guard, but I think my questions back caught him off guard a bit too.
That interviewer told me I was the only person who asked clarifying questions before blurting out an answer or walk through. Another one was "take this marker and design a house on the whiteboard for me". So I took the marker and asked questions like "how many people will live here, do you want one or two story, do you need a garage/shed/basement, etc?" And again, was told I was the only person who'd asked questions before starting to draw.
I don't think the intention behind those brain teasers was necessarily to determine how you react to those sorts of problems, but it may have been a useful determining factor for some interviewers nonetheless.
I once got negative feedback from an interviewer: I'd asked too many questions about the questions.
Good logical thinking shown here could indicate an ability to rapidly prototype systems without getting hung up on too fine detail.
Well, for one, the second isn't syntactically correct.
I can't decide what's worse, brainteasers or brainless trivia questions.
kill [-s sigspec | -n signum | -sigspec] [pid | jobspec] ...
Even so, with /bin/kill -sigspec is still valid and common usage, even if it is not documented in the manpage.
Anyways, there was one part in the movie where they start talking about auditions. All four or five of the actors they were interviewing for the movie unanimously spoke badly about the typical audition process. Some quotes taken from memory:
"I love acting, but I hate auditioning"
"You've seen my demo reel, you've seen me when I was on Star Trek, you know I can act, then why not just give me the part? Why make me go through this tedious audition process"
"90% of acting is reacting. You can't fully demonstrate your full acting abilities when you're standing in front of a panel of producers 'acting' out a scene that consists of 5 lines of dialog"
What the actors were saying about how they hate the audition process reminded me a lot of my frustrations surrounding hiring during tech interviews. Making an engineer do puzzles like FizzBuzz is a lot like making an actor act out a 20 second scene without any time to prepare or a proper "scene partner" to act alongside of.
I wish I could like to a youtube of the movie, but I can't find one. Its on netflix though.
FizzBuzz is self-contained tho, so maybe a better comparison would be to asking for a dramatic poetry reading?
It's hard, when using problems that are common, to really understand how the candidates gets to the answer. Often, he's building on pre solved sub problems he encountered on his professional life, so the resolution process didn't even occur at the interview.
I personally don't use brain teasers, because they stress out valid candidates who do not work well under pressure. However, I think teasers, when properly used, are valid tools in an interviewers toolbox.
This has the potential to reveal a certain high level problem solving ability which the lack thereof will not necessarily be revealed by more concrete "write pseudocode for X" type of interview questions. What I mean by that is that there is a continuum of skills ranging from rote copying of solutions all the way through synthesizing solutions to business problems and designing architectures to fulfill a malleable list of requirements. A mediocre engineer can inch their way up the continuum through raw pattern matching ability (which humans excel at) without ever attaining mastery of the high level abstraction that are driving the implementation detail. Such engineers can appear tremendously productive at the ground level, but they are dangerous for an technical organization to have many of them because they tend not to see where technical debt is piling up and can often paint themselves into corners because they're not considering the bigger picture. Knowing someone has strong reasoning skills from very high level human tasks down is a good hedge against this.
Can we see the study?
Also note that performance on the job is a noisy measurement, because people who get to work on impactful projects (through luck or people skills) get rated higher than others. I wouldn't be surprised if interview scores were a better measurement of "true" skills.
Possibly, but in a sense "true" skills don't really matter. What matters to Google, ultimately, is Google's opinion of the worker. It's almost certainly skewed / flawed / distorted in some way from the individual's true skills, and that's unfortunate but mostly a fact of life.
Anyone can help maximing it out. I know for myself that having 'clue' reduces self-esteem. Which can be balanced by having the right co-workers. End result: Maxing out the potential.
A(Technical intelligence) + B(Social Intelligence) = (Innovative potential)
And these things are important, because job candidates are not people, they are OEM replacement parts being order from Pep Boys. Call up the recruiter and requisition a J6-252: Programmer, seasoned 5 years, with degree from MIT. Oh, those ones are too expensive. Guess I'll take the knock-off version, but I refuse to pay full price!
Hopefully, because it's Google saying it, everyone will cargo-cult on this bandwagon too.
The comment about brainteasers vs structured rubrics is sort of surprising to me, given Google's reputation for quantitative data. Speaking from a very high level, structure was really what was emphasized for interviews. It's interesting how culture can get in the way of proven 'fact,' and I love that Google is using their own (much larger data sets) to make these improvements and in/validate other research
I like estimation questions in general for many of the reasons other commenters have cited. However, I wish those using them would consider the knowledge implicitly required of a candidate.
http://en.wikipedia.org/wiki/Fermi_problem
Knowing how to quickly estimate something is useful.
I imagine that Larry Page does a few quick estimates every day. How many Loon balloons would it take to bring Internet to 90% of Africa?
But not everybody at Google has a job like Larry Page. It's gotten to be a big company full of accountants, HR people, and other jobs that don't require much thinking in unfamiliar territory.
In other words, guesstimation is a useful skill, but not for every Google employee, so it's not going to show up as useful on average.
When trying to work out what best predicts job performance, the quality of your data is by far the most important thing to focus on. I would very much like to know more about the details of their internal studies. There are a lot of difficult problems in trying to use statistics to improve interview processes. One of the big problems is that you will always have a truncated sample of only those people who were selected: you would then expect the importance of certain variables, such as GPA or test scores, to be lowered because those who scored lower on such metrics will have had compensating characteristics...
"It’s also giving much less weight to college grade point averages and SAT scores"
In 2004 I interviewed for a Creative Maximizer position. I received a glowing review from my brother who was a Googler. I studied all the ins-and-outs of adwords back then and the British interviewer confirmed: "You did very good on the assessment" (which was working through real ads that needs to be maximized). My opinionated experience has been that in these kinds of situations, Brits embellish less than Americans.
However, she told me that my college GPA was "a major question mark" because it was 2.99 and Google only hires people with 3.0 and above (I didn't know what I wanted to do in college). Looking back I'm glad I was never hired, but that burned me bad for a while.
Contract to hire is one way, another is what they have done previously as a good predictor. It is a risk for sure but that really is the only true way in the end.
Plenty can be gained from just letting the interviewee talk and maybe looking at some of their code they have done previously while they talk about it. Whiteboard coding should not apply as it is completely out of element for many coders.
The type of person they are can't really be detected correctly until they are in the team and delivering because everyone is selling themselves on an interview.
http://www.ted.com/talks/mihaly_csikszentmihalyi_on_flow.htm...
While the analysis is correcting some beliefs about interviewing techniques, do I sense them draw a conclusion again not supported by data? How did they conclude the lack of correlation is "because" the skills required are different and people think differently a few years out from college.
I STILL go to interviews where I am ONLY asked these kinds of questions...It's embarrassing. If you ask me these questions for a 2 hour long interview then I'm not going to work for you...it's that simple