> I'm not saying it would happen often. But things do regularly plummet into the sun
The flow of mass through the region near the black hole (and well inside the surface of our sun) is minuscule in comparison to that in binary systems where the BH's companion is a Roche lobe filling star.
The event horizon of the sun would be about 6km in diameter, assuming that we swap the sun for a black hole with little angular momentum. Within the volume 10x that diameter, one might expect violence in objects on intersecting geodesics; with a diameter 50x that of the event horizon, one can start using linearized GR safely; with a diameter 500x that of the event horizon, post-Newtonian corrrections start to become small.
The diameter of the sun is a bit more than 450 times larger than that (or about 232 000 times the diameter of the black hole).
While there are lots of objects that are known to pass within 10 solar radiuses of the sun's centre-of-mass, it's hard to be definitive about why the number falls off with decreasing radius since the sun does a good job of disintegrating comets, and non-comets are hard to observe that close to the sun. Collisions with our sun are rare, and most colliders are unlikely to have been on orbits that would take them into the central regions discussed in the paragraphs above.
Objects that do find themselves only a thousand or two km from the event horizon will certainly take a post-Newtonian long time to emerge from that region, according to a surviving human observer on Earth. I'm not an expert on accretion discs (most of what I know involves the inner edge) nor astrophysical black holes, but absent a binary companion, the disc will be pretty sparse in general, and in particular maybe a factor of ten beyond the ISCO it'd be so sparse that even if it's super-hot, it's not going to do much to an asteroid or comet.
Although it's just a guess (you'd have to solve it numerically, good luck) I think tidal dismantling of a comet or asteroid would not put the debris on to many intersecting geodesics, so you need to have the debris linger around the black hole long enough for a different comet or asteroid on a significantly different orbit to add its debris. The bits of the second would almost certainly collide violently with bits of the first. I think that the timing involved for such small objects is so tight that "it would[n't] happen often" is a bit of an understatement, on human scales. :-)
Moreover, the energy densities (assuming comets, say) are small, so even being optimistic about the amount and location of inverse Compton scattering, the gamma flux at 1 AU isn't especially worrying. You're not going to destroy the Earth. (Consider a direct conversion of a pair of comets each of 6 * 10^7 kg into gammas radiated isotropically in one second; now extend the duration and reduce the amount converted, both by a lot... Then for convenience, the solid angle of the Earth from the the collision point would be about 6*10^-9 sr).
> every few years
I think maybe every few million years is more plausible, frankly.
There'd be more to see with a young star, since those tend to get a lot more things crashing onto them.
> compact black hole
Is there a non-compact black hole? :-) :-) :-)
ETA : extremely luminous events proposed to be tidal disruption events (and found at https://tde.space/ ) are almost wholly from jets from the "pancake flambé" star being disrupted close a supermassive black hole (whose diameter is much much larger than the disrupted star's). The UV emissions are largely absorbed or compton-scattered by the disrupted star's remains on their way out, and so mostly what's seen by us (or anyone outside a few tens of thousands of Schwarzschild radiuses) is in the IR.