If you have a true full-time AWD vehicle such as a Subaru or Audi with longitudinal engine, then mismatched tire wear will cause increased wear and tear on the center differential, which is expensive and difficult to get to. This is especially true if the differential is limited-slip.

If you have a torsen or similar FWD car that only engages the rear wheels when slip is detected (most Audis with transverse mounted engines, VWs, etc), then you're fine.

Tire diameter in passenger cars does not change with loading.