It's really the alkalinity (e.g. the Mg++ or Ca++), which silicate rocks often have (but technically not limited to silicates).

As an aside, we need to dissolve roughly one large mountain into the mix layer (top ~50m) of the ocean to have it fully take up atmospheric CO2. Without dissolving, the reaction is very slow (co2 in atmosphere => slightly lower pH rain => reaction with mostly passivated rock + erosion).