Biodegradable is only one type of degradation. Some of those compounds break down over time, or with exposure to uv or random stuff in the ground, into nasty compounds that you certainly don't want entering the water cycle or food cycle. An additional attribute of biodegradable therefore is: keeps (non CO2 pollution down).

In addition, things that biodegrade don't immediately just turn into CO2. Things like biomass (that is everything alive and dead that isn't decomposed) use a lot of that carbon. A significant fraction of the carbon in rotting stuff doesn't end up in the atmosphere for decades or longer. The carbon cycle isn't just "CO2 becomes plants which become CO2"... there's a lot more steps in between (for example, next time you eat... you are a direct next step!). Some of those steps take a very, very long time.