Looking through my recent comments, the majority are from people filling in the box on my posts. So, yes, I would lose a lot if I switched them off.
You'll also notice that I use WebMentions to import those 3rd party discussions (or links to them) into my blog.
That way users aren't subject to the advertising on platform A, the poor moderation on platform B, or the difficulty of even finding the comments on platform C.