Neural networks are computational systems inspired by the biological neural networks that constitute our and animal brains. At their core, these networks consist of layers of nodes, or "neurons," each of which applies a simple computation to its inputs. The Universal Approximation Theorem provides the theoretical foundation for these systems, offering assurance that neural networks have the capacity to model a wide variety of functions given sufficient complexity and proper configuration.

At the simplest level, a neural network can be thought of as a function $f: \mathbb{R}^n \rightarrow \mathbb{R}^m$ where is the dimensionality of the input vector and is the dimensionality of the output vector. A basic feed-forward neural network with one hidden layer can be mathematically represented as: