acquisition (in relation to associative learning): the process by which a new association between a stimulus and a response is formed (i.e., “acquired”) based on new experience
associative learning: the ability to associate a stimulus with a reflexive response, such that the next time that stimulus occurs that same reflexive response is more likely to occur
adaptation (in relation to the responses of neurons): the property of neurons whereby they change the relationship between a given stimulus strength and the resulting firing rate; for example, neurons will gradually decrease their firing rate in response to a constant stimulus over time
affect/affective state: a way to categorize the behavioral state of an animal along the dimensions of valence (either positive valence or negative valence) and arousal (either high arousal or low arousal)
agranular prefrontal cortex (aPFC): the region of frontal neocortex that evolved in early mammals. It is called “agranular” because it is a region of neocortex that is missing layer 4 (the layer that contains “granule cells”)
auto-association: a property of certain networks of neurons whereby neurons automatically build associations with themselves, enabling the network to automatically complete patterns when given an incomplete pattern
backpropagation: an algorithm for training artificial neural networks; computes the impact of changing the weight of a given connection on the error (a measure of the difference between the actual output and the desired output) at the end of the network, and nudges each weight accordingly to reduce the error
bilaterian: a group of species with a common ancestor around 600 million years ago, in whom bilaterial symmetry emerged as well as the first brains
bilateral symmetry: animal bodies that contain a single plane of symmetry, which divides the animal into roughly mirror image right and left halves
blocking (in relation to associative learning): one of the solutions to the credit assignment problem that evolved in early bilaterians; once an animal has established an association between a predictive cue and a response, all further cues that overlap with the predictive cue are inhibited (i.e., “blocked”) from making associations with that response
catastrophic forgetting: an outstanding challenge of sequentially training neural networks (as opposed to training them all at once); when you teach a neural network to recognize new patterns, it tends to lose the memory of previously learned old patterns
continual learning: the ability to automatically learn and remember new things as new data is provided
convolutional neural network: a type of neural network designed to recognize objects in images by looking for the same features in different locations
credit assignment problem: when an event or outcome occurs, what cue or action do you give “credit” for being predictive of that event or outcome?
extinction (in relation to associative learning): the process by which previously learned associations are inhibited (i.e., “extinguished”) due to a conditional stimulus no longer occurring alongside a subsequent reflexive response (i.e., a buzzer sounding that used to occur before food, but no longer occurs before food)
firing rate (also spike rate): the number of spikes per second generated by a neuron
generative model: a type of probabilistic model that learns to generate its own data, and recognizes things by comparing generated data with actual data (a process some researchers call “perception by inference”)
granular prefrontal cortex (gPFC): the region of frontal neocortex that evolved in early primates. It is called “granular” because it is a region of prefrontal neocortex that contains a layer 4 (the layer that contains “granule cells”)
Helmholtz machine: an early proof of concept of Helmholtz’s idea of perception by inference
mentalizing: the act of rendering a simulation of one’s own inner simulation (i.e., thinking about your own thinking)
Model-based reinforcement learning: the type of reinforcement learning whereby possible future actions are “played out” (i.e., simulated) ahead of time before selecting an action
model-free reinforcement learning: the type of reinforcement learning whereby possible future actions are not “played out” (i.e., simulated) ahead of time; instead, actions are automatically selected based on the current situation
neuromodulator: a chemical released by some neurons (“neuromodulatory neurons”) that has complex and often long-lasting effects on many downstream neurons. Famous neuromodulators include dopamine, serotonin, and adrenaline
overshadowing (in relation to associative learning): one of the solutions to the credit assignment problem that evolved in early bilaterians; when animals have multiple predictive cues to use, their brains tend to pick the cues that are the strongest (i.e., strong cues overshadow weak cues).
primate sensory cortex (PSC): the new regions of sensory neocortex that evolved in early primates, these include the superior temporal sulcus (STS) and temporoparietal junction (TPJ)
reacquisition (in relation to associative learning): one of the techniques to deal with changing contingencies in the world and enable continual learning in early bilaterians; old-extinguished associations are reacquired faster than entirely new associations
sensory neocortex: the back half of the neocortex, the area in which a simulation of the external world is rendered
spontaneous Recovery (in relation to associative learning): one of the techniques to deal with changing contingencies in the world and enable continual learning in early bilaterians; broken associations are rapidly suppressed but not, in fact, unlearned; given enough time, they reemerge
superior temporal sulcus (STS): a new region of sensory neocortex that evolved in early primates
synapse: the connection between neurons through which chemical signals are passed
temporal credit assignment problem: when an event or outcome occurs, what previous cue or action do you give “credit” for being predictive of that event or outcome? This is a subcase of the credit assignment problem when having to assign credit between things separated in time
temporal difference learning (TD learning): the model-free reinforcement learning process whereby AI systems (or animal brains) reinforce or punish behaviors based on changes (i.e., “temporal differences”) in predicted future rewards (as opposed to actual rewards)
temporal difference signal (TD signal): the change in predicted future reward; this signal is used as the reinforcement/punishment signal in temporal difference learning systems
temporoparietal junction (TPJ): a new region of sensory neocortex that evolved in early primates
theory of mind: the ability to infer another animal’s intent and knowledge
valence: the goodness or badness of a stimulus, behaviorally defined by whether an animal will approach or avoid the stimulus