Nice explanation. For future advances, part of it could be solved by photonics, getting rid of most of the interconnects. Massively parallel simple computations could be carried out on a photonic neuromorphic substrate, making NNs use orders of magnitude less power.