tomx

Given billions of rows of data across a few tables, how do you best make sense of it?

My current method involves thinking up interesting questions and writing database queries. I then plug the resulting data into gnuplot to examine it.

Is there generally a better way? I am kinda hoping a Mathematica/Matlab type shell or similar for databases or other data sources exists. Just type a query and view a graph. Even better, type queries, output graphs into a web page.

Or is the method to hire a data scientist to build specialised reports?

The data format is agnostic, interested in how this works across all ecosystems.

I find it's easy to forget many developers are effectively completely invisible online. I guess people just don't feel the need to broadcast information to everyone. So, out of interest:

What percentage of your coworkers - maintain a blog? - post code online? is it a big company or a startup? location? (i.e. the valley or elsewhere?)

I am interested whether we find the 1% rule ( http://en.wikipedia.org/wiki/1%25_rule_(Internet_culture) ) applies to developers&blogging, developers&posting code, or not. In my experience at least (small UK company), it's true for blogging, but not for posting code. About 1/4 the people I work with have a github account.

In this post http://news.ycombinator.com/item?id=3670014 , the commenter described Google as having a 'certain monoculture'.

Do developers within big companies (or I guess within units of big companies) have such a 'monoculture', i.e. people are pretty similar in some respect?

As someone who works at a small company, I'd be interested in what types of cultures exist.

Any ideas how cultures become defined? (I would guess at hiring similar people, working on similar technologies, office environment...)

The UK government are introducing programming lessons into British schools: http://www.bbc.co.uk/news/education-16509298

Does anyone have any opinions on how this may affect the programming landscape of the UK in the future (say, 5-10 years from now)?

I thought it might be interesting to consider how such a scheme may change code quality, employment prospects, programming language prevalence, the number of programmers and so on.

I'm sent 100 resumes a month, then interview about ~10 people. Of those, about 50% can do FizzBuzz style questions. On paper, all of these ~10 people would be qualified to do the work, however (I think) some have become rusty, or focused more on management or other areas.

What percentage of your job interview candidates pass FizzBuzz style questions? Any tips to improve the signal to noise ratio?