1. The rate of change-per-generation is very much not constant, as described by the Punctuated Equilibria theory. Sudden changes in the environment can cause sudden bursts of evolution. We don't know if there were any (and how many) mass extinctions / sudden change events before the fossil record starts, which is already hundreds of millions of years into the existence of single-celled life existing!

2. The time elapsed per generation has changed over time too, and we have virtually no direct evidence of the actual rate for the earliest epochs of life, before multi-cellular life.

These are particularly bad problems for any theory trying to extrapolate backwards, with compounding issues that can blow out any naive error estimates massively.

For example:

RNA-based vs DNA-based life. We know that DNA is more stable and resistant to mutation than RNA, which was the foundation of the earliest life forms. But we have no idea how that difference specifically affected early life evolutionary rates! We can guess... but only guess. However, almost certainly, early life had a much higher mutation rate per generation than modern life AND a consistently short generation time.