The shrinking population thing is basically happening in every first world country, we only notice it in Japan because their immigration policy is essentially "No."
Dramatically over-simplified model: Mormons in the US have 3.4 children compared to the national average of 2.1. They currently number about 6M, or 2% of the US population. If they grow by a factor of 3.4/2.1=1.62 (i.e., 62%) each generation, then it takes 8.1 generations, or a bit over 200 years, for them to become the majority.
- Reality is Broken, Jane McGonigal
Life without challenge is lived poorly and never punctuated by fiero.
Lack of money requires both partners to work in order to sustain the family. Typical Japanese work at least 8 hours, Plus one hour resting in the middle as the law demands it so it took 9 hours of a day.
But we don't have a luxury to live next to the working place. Most Japanese live 1-2 hours away from the working place. Going to work requires 2 to 4 hours.
It took 11 or 13 hours of a day.
Not all jobs are 9-5 jobs. The holidays are also differs.
So, partners work hours and holidays doesn't overlap.
Which makes really narrow margin of time spending together with the partner.
And this is the case of a family without having any kids. No wonder the birth rate is historically low.
Before somebody mentioned it, remote work isn't possible for many type of works. Most jobs still requires physical person's presence.