What made you accept the offer on such short notice? I get that startups move faster than other companies, but if someone wanted to hire me right now or not at all, I'd take it as a serious warning sign. Most likely it means they're hard-ass negotiators who are manipulating you or that they're genuine but too focused on the short term--either way, a red flag.
I got a tentative offer for my current job just as I was starting a one-year sabbatical backpacking trip through South-East Asia. They had something time sensitive they needed me for as soon as possible. Despite that, they were totally understanding, reassuring me that they still wanted to work with me whenever I was ready.