Some countries were in the same position as USA of assets seized by the regime, but they accepted symbolic paybacks. USA didn't, probably because they didn't like Cuba destabilizing the Caribean region in the Cold War. At the time, they probably believed that the embargo will damage Cuba regime and cause a quick coup, nobody thought that the regime will survive more than a decade.
Through time this has become a political matter (softening the embargo loses votes in USA, specially in Florida; paying back would expose the regime failure in Cuba when the boogeyman is gone), so both sides at political level are very interested in mantaining the embargo. I tend to see politics as a game of staying in power as long as possible, and here we have a Nash equilibrium where trying to end the embargo goes against both parts. Some people believes (I do) that the embargo helps the regime.
There's no "anti-Cuba" lobby: there's a "anti-Castro lobby". Sadly, this lobby believes in the embargo, and sees its lifting as a weak position against Castro regime.