Ask HN: Do you think intuition is as valuable as rational thinking?

I've found that there is a trend in software development to throw out both reason and intuition, and rely upon a form of retarded "test driven" empiricism. You can't possibly be smart or experienced enough to KNOW how your for loop is going to work, it has to be thoroughly tested. The same goes for product development. We can't possibly come up with an idea that people might want, without first taking a few polls where we ask them what they want...

"The intuitive mind is a sacred gift and the rational mind is a faithful servant.

We have created a society that honors the servant and has forgotten the gift."

Albert Einstein

Please don't apologize for trying things. :)

To answer your question. I have personally been more right than wrong when i have followed a gut feeling. I also act when i go with it more often than when i dont. When i am 'rational' i often DO NOT act because rationality tends to lead to pessimism. Atleast with me.

The value of the two are intricately linked but ultimately not comparable.

Is money as valuable as a marketable skill? Is getting somewhere quickly as valuable as a vehicle?

The intuition of a person who doesn't value rational thinking is going to be worthless in an empirical field. The gut feeling of someone who can't show they are right is not likely to be correct.

Intuition is just the fast-read cache of pre-computed notions. If a problem looks similar but not exactly like a dozen or so others that you have solved in the past, you take a sort of snapshot of all that past work and distill it into an intuitive grasp of the problem.

If you see a problem unlike any you have ever encountered, you are unlikely to have an intuitive idea of the solution. Sometimes people will pull past-solved problems out of their fast-read cache and apply them to unrelated problems in an irrational way; sometimes people will feel their intuition guiding them in a certain way and rationally realize they are projecting subconsciously an unrelated situation onto the problem.

So intuition is only as valuable as its rational basis. Intuition is much more efficient, much like using a rainbow table to crack a password; the table is only as effective as its precomputed hashes; it will never crack a password that hasn't been precomputed. However, compared to a traditional brute-force approach, it's orders of magnitudes more efficient. A brute-force approach however, with enough time will crack any password.

They are two different tools for different scenarios. Intuition is only valuable when you are experienced in something. Rational thinking, however, can be applied to areas where you have no or little experience.

Say, if a person doesn't know programming, his 'intuition' in programming would just be wild guesses and unlikely to be useful. However, he can still gather facts on programming and think rationally on it, no matter how slow and painful.

So in your original context of your post, I would suggest that we should only use intuition when we have extensive experience in that market or working with the target users. Otherwise, stay close with rationality.

People who believe that they make all their decisions rationally are delusional.

I say this with confidence because there is a large amount of research showing that people make decisions with a good deal of input- if not entirely- from the non-conscious parts of their brain. In fact, there are some people that argue that our conscious brains don't make decisions at all, they merely rationalize it post facto. (This goes too far, for me)

The classic example of this are the split brain studies, where they would show one half of a person's brain a sign that said "stand up." The patient would stand up. Then they would ask the other half of the brain (the one that was capable of speaking; in most humans, only one half of the brain can) why they stood up, and they would be completely convinced it was because they wanted to get a drink or water, or go to the bathroom.

That said, I'm pretty much the most gung-ho person on reason I know. The reason for this is that, like someone mentioned on a blog post on here, intuition is not transferable. If someone wants to convince me of anything at all, they're going to have to make reasoned arguments, not emotional ones.

I think intuition is the only way to proceed, actually.

-- But let me clarify! I want to directly attack those who you refer to thinking that "if you can't explain or prove something rationally then it's not of any value". I don't think anyone is a constant calculator like that. It's important to learn because we're building computers that operate that way, but it doesn't appear that strict rationality (whatever that is) is our true operating mode.

So if you want to get the gains of rationality, the best you can do is use rational structure to train and shape your intuition. If you're skilled at the math required then you can use it to empower your intuition -- but never replace it.

I say this with strong fear of thought paralysis. Those who spend too long rationalizing every thing seem likely to trap themselves in local minima, to argue endlessly over two similar choices while missing out benefit of either.

I like to think about MCMC algorithms, actually. They're guaranteed to converge to the most liable posterior beliefs, but do so by jumping randomly. Each step forward is technically blind and hopeful, but by keeping a goal in mind and learning from every jump you improve, even considering the immense ignorance that Markov methods maintain.

Intuition can do better.

Logic is great for answering life's questions; intuition is great for knowing which questions to ask.

Intuition represents all the reasoning that your brain does but which you aren't able to put into words. In many cases, that includes important information and heuristics which you would be ill advised to ignore. However, that also includes information and heuristics that may be incorrect; so if your intuition disagrees with your explicit reasoning, you should debug both to figure out which one's wrong and how.

Intuition and rational thinking are most powerful when you get them to agree. If I reason my way to a conclusion that doesn't feel right, that's a good indication that I missed something or discounted some aspect that I shouldn't have. And if I have a gut feeling, I won't act on it until I can rationalize it.*

* Of course, rationalizing a gut feeling is far from objective. But at least it makes you think things through.

Emotions seem to be essential to decision-making. People with brain injuries preventing them from having emotions find many things impossible to decide. [1]

But however much we rely on emotions and intuition in our own thinking, they're not easy to communicate, especially to skeptics. Those of us with a scientific and skeptical mindset try not to accept irrational arguments from other people. [2]

So, in some sense, an idea without a rational justification isn't worth very much (yet), because you can't convince most people that it's correct. But it still might have a lot of potential! And if we're talking about hacking then there are other forms of persuasion (code and demos).

[1] http://changingminds.org/explanations/emotions/emotion_decis...

[2] This is especially true of mathematics where rational arguments (that is, proofs) are valued far above any other kind of argument.

Is intuition as valuable as rational thinking?

At first glance, yes.

You've re-discovered the fundamental dilemma of historical Chinese civilization. "In Taoism, intuition was the guide to wisdom. Reason and reading the classics were the basis of Confucianism." (The alphabet versus the goddess, Leonard Shlain. p. 187) The question you've posed has a very, very ancient pedigree - and, of course, no set answer.

They're both important; beware of trusting the decision making of anyone who suggests otherwise. Rational analyses are almost always by necessity incomplete. There will always be assumptions we think are true that turn out to be false, facts we're unaware of, options we don't consider, or externalities we're oblivious to. Intuition often picks up on those things that we can't obviously articulate, or recognizes patterns that aren't otherwise apparent. On the other hand, people often use "intuition" as an excuse for shooting from the hip, justifying their predispositions, ignoring dissent, or simply being too lazy to do a rigorous analysis.

There's no substitute for doing your best to rationally analyze something, gather data, analyze arguments, etc., and not doing so in a particular situation is simply sloppy decision making. But that's merely one input into a decision making process that can also include things that fall under the category of intuition but which can't easily be articulated: ideas that give you a bad feeling or make you nervous, gut feelings about things, consideration of taste and aesthetics.

In the end there's no substitute for having good judgment, and good judgment and wisdom come from being able to use both rational arguments/data and intuition/feelings to arrive at the best choice.

I've definitely found intuition to be valuable, in the sense that my own intuition has been correct more often than otherwise. But intuition is basically the result of my unconscious reasoning based on lots of past experience, so it absolutely has to be trained--it's worthless without that background.

However, if I've made an important choice intuitively and it turns out right, I absolutely dig back into it and try to discover why it was the right choice in a rational fashion. Understanding that choice rationally lets me learn how to apply what I learned later, and sometimes helps me see second- and third-order effects which I can't see intuitively.

Basically, your brain has two ways of operating. You have your slow, deliberate "rational" side, and a fast heuristic side. There are well documented[1] biases that influence our answers.

After you've practised and gotten good at something, you've got your brain to start using the fast side to handle things. For instance, bike riding. At first, you're consciously thinking and trying to control everything. After a while, you get heuristics that take over and it becomes natural.

This happens in a lot of subjects, as far as I can tell. So, you shouldn't trust your intuition blindly. But understand the biases you're likely to have and how your brain is immediately popping up these ideas can be useful. Perhaps intuition can be more powerful because it's our brain using some fast ways of coming to a conclusion?

http://www.amazon.com/Heuristics-Biases-Psychology-Intuitive...

I think it's necessary to stick with your intuitive thought to a degree... at least until rationality takes over. Keep in mind that anything, taken to the extreme, can become unhealthy. I think you've hit the nail on the head in saying that "decisions and actions based on intuition usually end up having a rational explanation" and while I've found this to be true more often than not I feel the need sometimes to act purely in rational thought. It's something that, for me, needs to be decided on the fly. I can't plan when I'll act rationally, nor when I go with my gut. It just happens.

I don't think it's in our nature as human beings to act purely in a rational realm, but some degree of it has to exist or we'd never get anywhere. When I focus on matters of personal finance and productivity I need to think and act rationally or I lose objectivity (and ultimately much more). However in my work and much of the rest of my life, intuition goes hand in hand with experience and I find myself relying on what I "feel" to be the best course of action given whatever situation.

Again, this intuitive process that makes use of my experience (and that of those I've entrusted my thoughts to) is something I've given myself over to on a rational basis. If my experience-based intuition has served me well in the past, isn't using that to my advantage the rational thing to do?

It is by logic we prove, it is by intuition that we invent. - Henri Poincaré

I think it depends on the kind of person you are. Use what you have.

I'm the type that will routinely head off to a restaurant with only vague information about where it is or what its name is.

"It's probably down here, this seems like a street that would have mexican restaurants on it..."

"I've been there once before, and I remember it was on a block that was on the edge of a hill that sloped off toward the morning sun..."

It seems crazy, but the reason I end up doing this is because it works a surprising amount of the time. I end up getting there, somehow.

With work, I find that I end up relying on intuition the same way. The downside is that until I'm familiar with the problem space, I tend to be more unsure of myself than my rational peers. The upside is that once I learn the space, I can move more quickly with less information than they do.

Edit: Like others said below, the other downside is that this can sometimes lead to surprisingly bad results, especially in new and unfamiliar areas. I've been trying to exercise my logic muscles more recently to help with this.

Intuition is very often a direct result of experience--- I still remember my math professors all chanting various mantras about training my 'mathematical intuition' and over the years, I'd have to agree with them. I don't think in terms on one versus the other, if you are not using both, something is wrong. Easy to understand if you don't have any intuition, but if you do and you ignore it for the sake of 'rationality', I'm not sure that would in fact be rational! We get our ideas from a lot of different places, some more off the wall than others--- once we have them we can examine them and proceed or discard based on some presumably rational evaluation. Is intuition as valuable as rational thinking? You betcha!

Pragmatic Thinking and Learning[1] is a great read for anybody interested in understanding how the features of our mind affect our work as programmers.

The leading thought of the book is that we should put more attention to the R-mode of thinking, traditionally associated with the right brain, which is responsible for creativity, pattern matching and intuition.

Moreover there is a fragment which states that depending on intuition works only when you reach a certain level of competence and gather enough experience to "feed" the unconsciousness processing.

http://pragprog.com/titles/ahptl/pragmatic-thinking-and-lear...

This book is worth a read on the topic: http://www.amazon.com/Sources-Power-People-Make-Decisions/dp...

It's been a while since I read it, but the long and short of it is that intuition is the only game in town, especially when making decisions under pressure, and the people with good intuition are the one's with experience.

Each approach has liabilities that should be recognized. Intuitive decisions might "feel right" but there are so many cognitive biases that can make something intuitively appealing but surprisingly wrong. Analytical, rational approaches are likewise poor at developing creative solutions but is good at removing bias from reasoning by clearly stating every step.

I would argue that there isn't so much of a different between intuition and rational thinking as one might think.

Rational thinking is mostly understood to be a deductive form of "thinking" that a electrical/mechanical device could simulate with the technology we have no. (Evidenced by, say, Mathematica.)

Intuition is a more organic form of rational though; generally unchecked for errors (before your rational thought process gets ahold of it and examines it thoroughly), but exploring possibilities for solutions that may not immediately follow from the data you have in front of you. It skips the intermediate steps and delivers an abstract pathway to your solution, which you follow, and prove to work, with your rational thought.

Intuition is more like +N un-processed rational steps at once, with a higher probability for error. As long as we assume there is some difference, they're are co-dependent and just as valuable as the other; you need equal "amounts" of both to solve the difficult problems.

As you go further into the path of becoming a master in some subject of your choice, knowledge becomes part of you, and is therefore more intuitive than rational. You don't rationalize how to walk. You just do it. Now, if we're talking of a beginner in something, rationalization is a must. He can't relay on intuition, because there's none, or perhaps some but innaplicable to the subject, because it comes from previous unrelated experiences. The walking seems to be a bad example here, but if you think about it, a baby learns also by trial and error. He can't truly rationalize, but he wants it so bad (and is also very much encouraged to it) that he learns and grows intuition without thinking about it. Decisions are made based on whether you can decide using your guts or your brain. And that depends on your previous experience with similar decisions, either learnt from others or on your own.

I'm not sure I see the distinction. If following your intuition leads to better results than a more explicitly analytical method, then obviously the rational thing to do is to place more weight on your intuition. If "rationality" systematically underperforms an "irrational" approach, then your definitions are exactly backwards.

Jonah Lehrer dives deep into this his book, How We Decide. In an interview on the question, he answered -

"For the first time in human history, we can look inside our brain and see how we think. It turns out that we weren't engineered to be rational or logical or even particularly deliberate. Instead, our mind holds a messy network of different areas, many of which are involved with the production of emotion. Whenever we make a decision, the brain is awash in feeling, driven by its inexplicable passions. Even when we try to be reasonable and restrained, these emotional impulses secretly influence our judgment."

Put another way, there is no real bright-line divide between rational, intuitive and emotional. Rather they form a decision-making fabric and even when we believe we are deciding in an entirely rational manner, we're not.

Intuition is good at finding what to do and what approach to take. This is finding the new dot in the unknown.

Rational thinking is good at explaining why does it work, then. This is connecting the new dot back to what you already know.

Further, my observations suggest that those two are not mutually exclusive. Intuition and feelings can indeed be based on what you already rationally know. They just operate on the unknown area that your consciousness can't reach yet. Intuition isn't too helpful unless you know something about the problem domain. Similarly, rational thinking can't really warp to anything truly new without intuition: the rational mind per se can only take what is already known and expand it a bit. To do otherwise would be, to the rational mind, well, irrational.

The book "Hare Brain, Tortoise Mind: How Intelligence Increases When You Think Less" http://www.amazon.com/Hare-Brain-Tortoise-Mind-Intelligence/... goes into detail about how intuition works, where it comes from, what it's good at and what it's not good at. IMO, Gladwell's "Blink" was heavily based on "Hare Brain".

tldr: Intuition has a larger short-term memory space. It can process more complex systems than verbalized thinking. However, because it is not verbalized, it is difficult to componentize, communicate or reapply. It also does not work well on-demand or under pressure.

Intuition and rationality are reflexive. At all times one informs the other. Some people prefer to lead or focus on rationality, where their rational mind trains their intuition, and some lead or focus on their intuition, where their intuition is checked by their rational mind.

I don't believe the two approaches can be meaningfully separated, but some people try to do so. Which can only cause problems as each belongs to a different type of problem. One doesn't rationalized where a lobbed ball will land when going to catch it, but a non-expert doesn't intuit where a mortar will land upon firing it.

An intuitive decision making skill worth having is always based in reason.

We might not base those intuitive decisions on clear cut reasons (that guarantee results, 100%), but we base them on past experiences and observed phenomena (that increase the likelihood of the result, sometime even to 100%.)

If you are asking about an intuitive decision that is not based on any of the above, then your question is really, "Should I make random decisions?" Sure, just because something has a minute chance doesn't mean it won't happen. It might just work.

IMHO, depending on the situation, it might be even more valuable than rational thinking. Especially on situations where rational thinking fails or where you have to make instant decisions.

I suggest you read 'Blink' by Malcolm Gladwell. The author gives many examples of situations where spontaneous decisions yield better results compared to rational thinking methods.

You make the implicit statement that intuition isn't rational, which is interesting. I find my intuition is often most rational.

Ask Doug Bowman: http://stopdesign.com/archive/2009/03/20/goodbye-google.html

    And [that] data eventually becomes a crutch for every
    decision, paralyzing the company and preventing it from
    making any daring design decisions.

People tend to make decisions based on emotions and rationalize it afterwards. Intuition seems to be a refreshing outbreak from the above: it seems to be an early information you consciously have and before any emotional attachment - liking or disliking - is tied to it.

So yes, let's use it, with care, and be honest to yourself.

Intuition is like your nose leading you to yummy food. But you need to evaluate (chew and taste) that food before you are certain that you want to swallow it. It's an indicator that you are on the right or wrong track, and it can point to where you want to go.

But it is quite fallible in a way that "pure" rational thought is not.

There are three tools for discovering truth and making decisions: reason, intuition, and revelation.

Revelation means ask an expert, read the documentation. It is most appropriate when you don't know what you're doing at all -- when you have no sound first principles to feed into the engine of reason, and no experience on which to build intuition. Revelation is fast but limited; you instantly gain a conclusion as sound as your expert, but you cannot improve upon or critique it.

Reason is most appropriate when you have moderate experience in a field. Through revelation and limited experience, you have developed some sound, inviolable principles, and can reason your ways to new ones. You know what must go here because you know what must go there, and you can figure out how this works because you know how that works. Reason converges to truth slowly but inexorably. It will eventually get there. But it takes a long time to process a lot of input or to navigate a complex landscape.

Intuition is an appropriate tool when you have high experience in a field. Even presented with a complex problem, you know what to do. You just know that OO is the wrong paradigm for this, and that code must go on the server, and this bug is almost certainly caused by a mismatched type somewhere in the parser. Given time, you could probably justify it. Doing so would be equivalent work to writing a paper -- there's just a lot of stuff to consider and weigh. But the power of intuition is that you can decide nearly instantly, and often decide right. Intuition as a tool doesn't converge to truth; it quickly leaps, and then it either gets there or it doesn't. And even when does get there, you don't know immediately whether it actually worked or not. It always uncertain, being only as good as your necessarily incomplete mental models. But there are times when a fast guess is way better than a slow conclusion. And there are times certainty isn't possible anyway. One problem with intuition is that you can't improve its results for a particular problem. If it's wrong, it's just plain wrong. You can't doggedly grind on like you could with reason, or ask another source like you can with revelation.

The highest level of competence comes from using all three tools in concert. With a lot of experience, revelation is sharp: you know who the experts are and what they are likely to know. Reason is sharp: you know a lot of useful rules, and the fastest ways to check truth for certainty. Intuition is sharp: you have a feel for the rhythm of the hills and valleys in the problem landscape. You can guess the existence of a distant mountain the hill-climbing rationalist would take forever to find, quickly and soundly check the local landscape for its unexpected little pits and spikes, and borrow a good map to cross a desert quickly.

This engine of canny guesses and rapid checks and good borrowed maps is precisely what makes communities of hackers work so impossibly, frighteningly fast.

I like to think that intuition is the statistical part of the brain, and it can learn larger sample sets than you can handle rationally. As so, you should trust intuition when you know you have enough experience on the subject.

When I started to think about this question I just ended up getting sucked into a debate with myself about the definitions of rationality and intuition.

Could the two concepts merely be component parts of a more complex feedback loop?

I think the hard part about working off of intuition is communicating it to others. I often find myself working backwards to justify out my intuition, and I think it comes across as me just making stuff up.

I rely on both, plus a generous helping of muscle memory, sense memory and random chance - I sometimes roll a die and go by the result.

Intuition, for me, is just natural rationality that has been 'learned'.

i found out that most times my intuition is right, even when rationally thinking about something would come up with a different decision.

but i guess this strongly depends on your intuition :)

I think so. Logic is only as good as its premises.

Intuition + rational thinking = valuable/powerful

Trust your gut until it lets you down.

no

The most important thing is to apply rational thinking to irrational thinking, what I mean by that is things like A/B testing where you discover the irrationality of others and rationally exploit it.

Intuitive thinking is highly valuable when there is no time to collect the necessary data for rational thinking, which is most of the time. When data is available there is little point to intuitive thinking.

They are both very valuable tools, but like any tool it's value is realized by applying it to the right task.

Rational thinking can sometimes be like a wall...

Intuition will sometimes lead to ludicrous solutions...

Guess you have to use rationality to filter out the ridiculous ideas you can come up with intuitively...