Consider that surviving severe overpopulation is trauma. Survivors are physically and/or psychologically sick. They're not good breeders and at this point.
Happens frequently in aquariums. Guppies breed like guppies. Overpopulate, then die back. The proximate causes can be varied. Maybe there's an ammonia spike, oxygen deficiency or other consequence of overabundance.
When the "crash" happens, survivors tend to be a precocious bunch. They're past breeding prime, and probably injured by the events of the crash and its precedents. Starting a colony with "poor stock" is always more difficult, and that's what you now have.
It doesn't have to be overpopulation that causes a crash. Say you accidentally pollute the water with soapy hands. Some fish will die today. Some tomorrow. Dead fish rotting is also polluting. Fish surviving the ordeal will probably have reduced lifespans and breeding potential.
My unqualified guess is that most extirpation events leave survivors, usually in less than ideal shape. The technical extirpation happens later, as survivors gradually drop off without successful reproduction.
Good question about minimum population size for boom/bust cycles to occur. I suspect this is more about environmental richness than pure numbers. Some predation probably stabilizes things, and some boom/bust might be possible. If you had multiple colonies connected by 500m of tubing, this "distance" between territories might allow for "local" booms and busts to occur.
