I don't mean to reduce the work of whichever researchers published in 1983 (are you thinking of Rumelhart et al 1986?), but gradient descent and reverse mode automatic differentiation are way older than 1983. Adaptive filters were being "trained" in the 50s using stochastic gradient descent.

> One year effect? What exactly are they speaking about?

From article:

> What may take a cluster to compute one year takes a consumer machine the next.

So, one year after introducing something you don't need cluster for computing, consumer machine will be sufficient.

From first comment:

> Is that not partly because the hardware is ever improving? [...] Certainly improving the model will help, who hasn't marvelled at how better his code ran when he fixed that On^2 hot spot. But I can't help but think improving hardware plays a role too. > Am I off base here?

Now my comment was about original commenter being off base. Hardware improvement plays a role, but not during one year. 1000x speedup in a year is due to algorithms.