The whole point of got is that it is distributed, so just clone it on your local machine surely?
Alternatively you could use rsync or lsyncd which are much faster and better than SSHFS.
Anyway I don't think this was created for use over SSH. Most people that use it are going to be working locally, in which case a proper GUI is much better.
I don't rsync to store /etc out of dozens of servers just to occasionally see some diffs.
CUI apps are really meant over SSH imo. Like you say, local use has wider alternatives including GUI in which case I prefer SourceTree over any CUI tools.
