Pushed secrets in public repos are automatically archived by third parties so removing commits containing them would not be enough to prevent their use, just rotating the secrets is the way to go here.
https://twitter.com/andrzejdyjak/status/1324360905237372929 https://news.ycombinator.com/item?id=25013756