By modeling the neural network as a time-discrete nonlinear dynamical system, we interpret the memorization property as a problem of simultaneous or ensemble controllability. This problem is addressed by constructing the network parameters inductively and explicitly, bypassing the need for training or solving any optimization problem.

Additionally, we establish that such a network can achieve universal approximation in Lp(Ω;R+), where Ω is a bounded subset of Rd and p∈[1,∞), using a ReLU deep neural network with a width of d+1. We also provide depth estimates for approximating W1,p functions and width estimates for approximating Lp(Ω;Rm) for m≥1. Our proofs are constructive, offering explicit values for the biases and weights involved.