It actually can be a pretty huge deal. On their own it's not the biggest issue, but they can compound a lot of invasive plants. Most native flora evolved without earthworms. Earthworms are incredibly effective at breaking down matter into easily accessible nutrients for plants. Native plants aren't built to take advantage of this. You know which plants are though? Non-native invasive plants from the rest of the earthworm-having world

Earthworms are generally a very welcome addition to any soil ecology, but they're currently playing an extremely destructive role. A role that probably wouldn't be as problematic if it weren't for all the compounding effects of other invasive species