There were definitely times when I was extremely frustrated and wanted to give up. There were a couple times when I had 5+ hour sessions and got nowhere, then I'd come back the next day to realize it was a stupid mistake.

One thing I can't stress enough (and James has been saying it too), if you don't have a final site, product, or app in mind you're probably going to get bored and walk away for good. I tried to learn C++ a while back and gave up because I didn't have anything in mind and the boring repetitive exercises lead to nothing in the end.