I have never regretted it and only feel relief whenever I think about it. I walked straight into another job.
If I were to stop freelancing and get a job again, I would just leave it off my CV.
I've done something similar twice, except there were no interesting technical challenges. Both were government jobs where I was lured in to do all this cool work and then get told to sit there and do nothing. The pay and hours were awesome, but for my 20 something self it was a terrible fit.
What I'm actually doing could be considered a cross between 'IT guy' and 'generic office staff' (I go to tons of meetings, write lots of system-engineering-esque documents no one will read, and because I am 'a computer guy', I'm bombarded with annoying IT technician-level questions all the time). I'm extremely overpaid for what I'm doing (I'm bored out of my skull most of the time), but the local job market is such that I'd probably spend several months unemployed if I were to just up and quit.
Just keep looking for something else. It's easiest to find a job when you already have one. That's because there is no financial pressure to take whatever comes along. You can take a little time to find a good fit. Don't feel that this reflects in any way on your attitude toward your current job. Do your best at that, but when you go home spend at least a few hours a week looking for something else. If you stay in your present job for too long (years) it will stunt your professional (and possibly personal) growth.
A few years ago, the single most powerful thing I did was knock out all our non-mortgage debt. It's freeing to know that if something happened, I could take a pay hit of $Xk/month and still have the same lifestyle.
Being unemployed for several months is way less scary if your monthly expenses are lower and you have cash in the bank.
They pay you alot of money to be there to do stuff they don't understand, but outside of that, they don't really want you interfering with their day to day life with all that "tech stuff".
Definitely find a better job, and don't underestimate the value increase in you quality of life by being around better people who challenge you. Nothing is worth more than that. So outside of minimum financial obligations, make plans to move elsewhere even if it will take a while. If you are the breadwinner in your situation for yourself and or other people, they will have to deal with it.
It may take even 3-4 years to move if you have long term commitments and get ready to go and do it without being risky financially, but honestly its worth it. If you are the one providing for people and you are the one donating half of your waking life and you are the one smart enough to be dissatsified with the lack of mental stimulation then you deserve to have a job you enjoy.
Being around people at work who challenge you and being on a good team will without a doubt increase your personal joy in life as well. Being emotionally isolated in a company who designates you as the computer guy and wants to drain your brain more by asking to create documentation to make the integration of technology in the company appear more valid than it really is, or save them if you don't show up work one day, is an abuse to your mental situation as much as you allow it to happen.
If you are already that smart, rising to the challenges you wish you could will only result in more income later, so don't let the current financial benefits/perceived location limitations stop you from pursuing a more rewarding career.
A couple years may seem like a long time but...what are you going to do, rot in a n IT office for the next 40 + years of your life? Make the difficult decisions/sacrifices now and even if you are in your 30s or even more, still youll get decades of happiness and challenges you won't if you throw your hands up in the air now.
The rest of the time at work, I spend inventing projects to helped me learn some interesting technology. My bosses were happy because I stopped bugging them for work to do.
Out of curiosity, in what direction?
(Also in the process-driving-me-round-the-bend camp, but concerned that a lot of alternatives might be a case of "out if the frying pan, into the scrum"...)
The agile bits were standups before we'd even got anything to do, card votes on how long features would take and I was saying months and everyone saying days, but I was the only one with any significant experience. Lo and behold 4 weeks later when I left it still wasn't done. There was some nonsense around the kanban board when they'd move things when nothing happened, tasks were really poorly split up so some bits would just get stuck, etc.