Neurons are terminally differentiated cells that specialize in cell-to-cell signaling through electrical mechanisms. Under resting conditions, neurons maintain a net negative membrane potential through the coordinated activation of multiple types of energy-independent and energy-dependent ion channels. Each of the tens of billions of neurons in the human brain can connect to up to approximately 10,000 neighbors, through close approximation of axons and dendrites (see Fig. 1 ). Neurotransmitters like acetylcholine and gamma-aminobutyric acid are released at axon terminals and stimulate ion transmission through channel receptors embedded in dendrite membranes. Information in the human brain is exchanged through repetitive, stepwise activation of axon–dendrite pairings, also known as synapses. Receptor activation leads to a temporary change in the overall membrane potential. The amplitude and duration of this change depends on the type of receptor activated. If the summed change in membrane potential exceeds a set threshold known as the actional potential, a rapid influx of sodium ions triggers the neuron promotes a net positive membrane potential and the neuron “fires.”

On firing, electrical depolarization propagates along all axons and triggers neurotransmitter release, conveying information to connected neurons. The coordinated firing of individual neurons across complex networks represents the fundamental basis for human cognition. Synapses are constantly, dynamically reorganized through long-term potentiation, which adjusts the strength of connections based on patterns of activity, where neurons that fire simultaneously develop stronger connections. Mathematical modeling of these fundamental units paved the way for simulations of human cognitive processes.

The United States Congress defines AI as “a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations or decisions influencing real or virtual environments.” Machine learning encompasses the subset of AI techniques that focus on modeling patterns within large datasets. Supervised learning supports models that try to match a human-defined output , while unsupervised learning extracts novel patterns from high-dimensional data. The deep learning techniques that have captivated media attention in recent years rely on complex neural networks to model several layers of abstraction from high-dimensional input data. Neural networks are particularly useful for modeling nonlinear relationships. The most current models are the product of iterative improvement on earlier mathematical representations of neurons and their connections. Given strong theoretic links to human cognition, we will focus on these models throughout the rest of the article.

Alan Turing’s 1936 analysis of mathematical theory is often cited as foundational in the field of AI, given that it was the first description of an autonomous computing machine. Seven years later, Warren McCulloch and Walter Pitts proposed a foundational mathematical representation of human cognition. They drew parallels between neuronal and electrical circuits, theorized on the substrates underlying human cognition, and created the first mathematical model of the neural networks that support thought.

Several advances over the following decades, in both the theory of artificial neural networks and the engineering underlying modern computing, supported LeCun and Haffner’s successful implementation in the late 1990s of a neural network capable of recognizing handwritten digits. In brief summary, critical improvements included the description of the “perceptron” for information storage and encoding by Frank Rosenblatt, which introduced the concept of layers within artificial neural networks; a mathematical formulation of Hebbian learning by Bernard Widrow and Marcian Hoff, which guided how networks might begin to capture relationships between input and output data; the application of backpropagation to layered artificial networks by Paul Werbos among others, which solidified how networks might learn meaningful relationships with low error rates; and the conceptualization of recurrent networks with the capacity for parallel processing by J.J. Hopfield among others, which expanded the complexity of input–output relationships that could be captured. Collectively, these advancements formulated the modern artificial neural network, which is multilayered, intricately interconnected, and powered by complex learning rules and activation formulae for individual neurons.

In practice, modern artificial neural networks are implemented as a series of matrix operations on vectors of input data. A review of the mathematical representation of artificial neurons and their layers has been well reviewed. Two of the most popular implementations—Google’s tensorflow and Meta’s pyTorch —support the creation of tensors, or data structures that organize high-dimensional data in a computationally efficient manner. These Python libraries also make use of parallel processing via central, graphical, or tensor processing units to improve compute time, which scales nonlinearly with the size of input data.

Most neuronal network implementations process information serially in a feedforward network , usually composed of artificial neurons organized into 3 types of layers: input, hidden, and output ( Fig. 2 ). Connections between neurons are referred to as the weights or strengths. Input layers represent or are “activated” by a vector of input data and connect to the first hidden layer. Their size is constrained by the size of the input data; that is, an input representing a coordinate in 7 dimensional space would require a 7 neuron input layer. Output layers summarize the information contained in the network’s hidden layers and compute values that are recognizable to the user and comparable to the desired output. For example, networks designed to classify input data into a binary outcome might end in a 1 neuron output layer.

The structure of a feedforward neuronal network. In this very basic neural network, an input layer ( blue ) represents the individual values an input vector and connects to a hidden layer ( green ). Arrows represent the strength or “weight” of connections between input neurons and the hidden layer. The hidden layer then connects to an output layer ( orange ), which summates the output of the network. Bias nodes ( purple ) can be employed to enhance fitting for complex data.

Depending on the size of the input data or the complexity of the relationships or patterns the user attempts to model, artificial neural networks may have any number of hidden layers. Hidden layers may take any size and can be fully interconnected to each other or sparsely connected, that is, individual connections are removed at random. Sparse networks lead to fewer computations and can be more energy and time efficient.

At each calculation step, the value of individual neurons in hidden layers is determined as the sum of the input multiplied by the learned weight of the connection between the neuron and each of its inputs. In many networks, a bias may also be applied, to adjust the value of individual neurons at each stage in calculation and enhance fits for complex data. Just like biological neurons, artificial neurons have action potentials or thresholds above which they activate, and below which their feed-forward value is nullified. Input strengths activate the input layer, which feeds forward to activate the first hidden layer, which in turn activates the following input layer, and so on, until the output layer is activated to deliver a conclusion. There are now several widely used activation functions that differ in terms of their sensitivity in information processing, for example, the stricter rectified linear unit and the more relaxed sigmoid function. Similarly, there are many kinds of optimization formulae, including stochastic gradient descent that updates all weights according to the same learning rate and Adam, which calculates unique learning rates for all connections. These specializations may prevent overfitting to training data, among other benefits.

Finally, the expected and calculated output values can be compared to calculate an error which, together with the model’s desired learning rate and optimization function, are used to adjust connection weights and biases during training as part of supervised learning. This training process allows the model to capture how complex relationships within the input data relate to output values. Neural networks can also be deployed in an unsupervised manner. For example, they have been used to cluster high-dimensional data such as scholarly texts and to restructure high-dimensional data into lower dimensional representations that preserve similarities between varied inputs. These and other applications make use of intricate network structures and training paradigms that expand on the traditional feedforward network to carry out complex target functions.

Self-organizing maps were first introduced in the early 1980s and were quickly realized for their potential to support the unsupervised classification of high-dimensional data. The design of these networks was initially motivated by prior observations of the cortical organization of information collected from the retina, which was in turn motivated by foundational studies of the organization of the visual cortex by David Hubel and Torsten Wiesel. These networks rely on an alternative form of training, in which the association between input and hidden neurons is competitively strengthened throughout the training process. As in long-term potentiation, the strength of connections between input and hidden layer neurons dynamically reorganizes based on mutual activity, leading individual hidden layer neurons to associate selectivity to individual input neurons. These network structures remain useful for transformations of high-dimensional data into lower dimensional representations.

Feed forward networks remain suboptimal for tasks in which input data have spatial dependencies, such as image recognition. Inspired again by the visual cortex, Kunihiko Fukushima developed a theoretic framework through which network substructures might learn or “extract” individual parts of an input. Later, Homma and colleagues refined this approach, allowing networks to filter information. They developed convolutional neural networks (CNNs), which use convolution and pooling layers to filter information. For example, individual layers might recognize the ears, eyes, and nose in a CNN trained for facial recognition. After filtering, CNNs employ a fully connected layer to summarize the learned information and relate it to an output. This network structure represents the current state of the art that supports the models behind many advances in the automated processing of medical images, for example, ResNet. These models are adept at extracting features from input images and using those features to relate the source to a desired output.

Similarly, feedforward networks are not particularly well suited to learning temporal relationships within long sequences of input data. Several researchers have developed specialized network models that exchange information in both directions, relying on backpropagation of information to earlier network layers in a recurrent neural network structure. One of the most widely used structures is known as long-short term memory (LSTM), which employs specialized memory units that selectively retain the influence of errors from prior calculations throughout training. These memory units contain “forget” gates, allowing them to dynamically ignore the influence of select inputs. This unique structure renders LSTMs adept at completing sequences and providing high-quality forecasts.

Large language models (LLMs) are relatively newer than the above-described models and represent a revolutionary achievement in machine learning. Building on advancements in convolutional and recurrent neural networks, a Google-based AI research team developed the transformer architecture to better encode and decode the relationships between individual tokens in input strings of written language. Transformer units rely on a self-attention mechanism to learn how individual parts of any input relate to each other in context. The ability to encode context gave rise to exponential improvements in text processing and language tasks.

Today, frontier models like ChatGPT are pretrained on large amounts of input text and then subject to reinforcement learning from human feedback , whereby model conclusions are fine-tuned by expert reviewers. LLMs powered by the transformer architecture excel at interpreting a text prompt and expanding it by supplying the most likely tokens that match the context inherent in the input. This structure in turn allows LLMs to function as the strongest examples of generative AI, whereby they synthesize novel output such as text and images.