LearningSignal

Overview

A LearningSignal is a type of ModulatorySignal that is specialized for use with a Learning Mechanism and one or more LearningProjections, to modify the matrix parameter of the MappingProjections to which they project. A LearningSignal receives the value of a learning_signal calculated by the Learning Mechanism to which it belongs, which in general is a matrix of weight changes to be made to the matrix parameter of the MappingProjection(s) being learned. The LearningSignal assigns its learning_signal as the value of its LearningProjection(s), which convey it to the MappingProjections` MATRIX ParameterState(s), which in turn modify the matrix parameter(s) of the MappingProjection(s) being learned.

Creating a LearningSignal

A LearningSignal is created automatically whenever a MappingProjection is specified for learning and the Projection belongs to the same Composition as the Learning Mechanism. LearningSignals can also be specified in the learning_signals argument of the constructor for a Learning Mechanism. Although a LearningSignal can be created directly using its constructor (or any of the other ways for creating an OutputState), this is neither necessary nor advisable, as a LearningSignal has dedicated Components and requirements for configuration that must be met for it to function properly.

Specifying LearningSignals

When a LearningSignal is specified in the learning_signals argument of the constructor for a Learning Mechanism, the ParameterState(s) of the MappingProjection(s) being learning must be specified. This can take any of the following forms:

  • an existing LearningSignal, or a reference to the class (in which case a default LearningSignal is created);
  • a ParameterState, which must be for the matrix parameter of the MappingProjection to be learned;
  • a tuple, with the name of the parameter as its 1st item. and the Projection to which it belongs as the 2nd; note that this is a convenience format, which is simpler to use than a specification dictionary (see below), but precludes specification of any parameters for the LearningSignal.

  • a specification dictionary, that must contain at least the following two entries:

    • NAME:str
      the string must be the name of the MappingProjection to be learned; the LearningSignal is named by appending “_LearningSignal” to the name of the Projection.
    • PROJECTION:MappingProjection
      the MappingProjection must be valid projection specification for the one to be learned.

    The dictionary can also contain entries for any other LearningSignal attributes to be specified (e.g., a LEARNING_RATE entry); see below for a description of LearningSignal attributes.

Structure

A LearningSignal is owned by an Learning Mechanism, and “trains” one or more MappingProjections by modulating the value of their matrix parameters. This is governed by three attributes of the LearningSignal, as described below.

Projections

When a LearningSignal is created, it can be assigned one or more LearningProjections, using either the projections argument of its constructor, or in an entry of a dictionary assigned to the params argument with the key PROJECTIONS. These will be assigned to its efferents attribute. See State Projections for additional details concerning the specification of Projections when creating a State.

Note

Although a LearningSignal can be assigned more than one LearningProjection, all of those Projections will convey the same learning_signal (received from the LearningMechanism to which the LearningSignal belongs). Thus, for them to be meaningful, they should project to MappingProjections that are responsible for identical or systematically-related error signals (e.g., as in convolutional networks.

Modulation

A LearningSignal has a modulation attribute that determines how the LearningSignal’s value (i.e., its learning_signal) is used by the ParameterState(s) to which it projects to modify the matrix parameter(s) of their MappingProjection(s) (see ModulatorySignal Modulation for an explanation of how the modulation attribute is specified and used to modify the value of a parameter). The default value is set to the value of the modulation attribute of the LearningMechanism to which the LearningSignal belongs; this is the same for all of the LearningSignals belonging to that LearningMechanism. The default value of modulation for a LearningMechanism is ADDITIVE, which causes the learning_signal (i.e., its matrix of weight changes) to be added to the matrix parameter of the MappingProjection being learned. The modulation can be individually specified for a LearningSignal using a specification dictionary where the LearningSignal itself is specified, as described above. The modulation value of a LearningSignal is used by all of the LearningProjections that project from that LearningSignal.

Learning Rate and Function

A LearningSignal has a learning_rate attribute that can be used to specify the learning_rate parameter for its LearningProjection(s) (i.e., those listed in its efferents attribute). If specified, it is applied multiplicatively to the LearningProjection`s learning_signal and thus can be used to modulate the learning_rate in addition to (and on top of) one specified for the Learning Mechanism or its function. Specification of the learning_rate for a LearningSignal supersedes the learning_rate for its LearningProjections, as well the learning_rate for any Process(es) and/or the learning_rate for any System(s) to which the LearningSignal’s owner belongs (see learning_rate of LearningMechanism for additional details).

The function of a LearningSignal converts the learning_signal it receives from the LearningMechanism to which it belongs to its value (i.e., the LearningSignal’s learning_signal). By default this is an identity function (Linear with slope=1 and intercept=0), that simply uses the LearningMechanism’s learning_signal as its own. However, the LearningSignal’s function can be assigned another TransferFunction, or any other function that takes a scalar, ndarray or matrix and returns a similar value.

Note

The index and assign attributes of a LearningSignal are automatically assigned and should not be modified.

Execution

A LearningSignal cannot be executed directly. It is executed whenever the Learning Mechanism to which it belongs is executed. When this occurs, the LearningMechanism provides the LearningSignal with a learning_signal, that is used by its function to compute its value (i.e., its own learning_signal for that TRIAL. That value is used by its LearningProjection(s) to modify the matrix parameter of the MappingProjection(s) to which the LearningSignal projects.

Note

The changes in a MappingProjection’s matrix parameter in response to the execution of a LearningSignal are not applied until the MappingProjection is next executed; see Lazy Evaluation for an explanation of “lazy” updating).

Class Reference

class psyneulink.components.states.modulatorysignals.learningsignal.LearningSignal(owner, function=Linear(), modulation=ModulationParam.MULTIPLICATIVE learning_rate=None params=None, projections=None, name=None, prefs=None)

A subclass of ModulatorySignal used by a Learning Mechanism to modify the matrix parameter of one more more MappingProjection(s).

Parameters:
  • owner (LearningMechanism) – specifies the Learning Mechanism to which to assign the LearningSignal.
  • function (Function or method : default Linear) – specifies the function used by the LearningSignal to generate its learning_signal.
  • learning_rate (float or None : default None) – specifies the learning_rate for the LearningSignal’s LearningProjections (see learning_rate for details).
  • modulation (ModulationParam : default ModulationParam.MULTIPLICATIVE) – specifies the way in which the value of the LearningSignal is used to modify the value of the matrix parameter for the MappingProjection(s) to which the LearningSignal’s LearningProjection(s) project.
  • params (Dict[param keyword: param value] : default None) – a parameter specification dictionary that can be used to specify the parameters for the LearningSignal and/or a custom function and its parameters. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
  • projections (list of Projection specifications) – specifies the LearningProjection(s) to be assigned to the LearningSignal, and that will be listed in its efferents attribute (see Projections for additional details).
  • name (str : default see ModulatorySignal name) – specifies the name of the LearningSignal; see LearningSignal name for additional details.
  • prefs (PreferenceSet or specification dict : default State.classPreferences) – specifies the PreferenceSet for the LearningSignal; see prefs for details.
owner

LearningMechanism – the Learning Mechanism to which the LearningSignal belongs.

variable

number, list or np.ndarray – used by function to generate the LearningSignal’s learning_signal.

function

TransferFunction : default Linear(slope=1, intercept=0) – converts variable into the LearningSignal’s learning_signal. The default is the identity function, which assigns the LearningSignal’s variable as its learning_signal.

learning_rate

float : None – determines the learning rate for the LearningSignal. It is used to specify the learning_rate parameter for its LearningProjection(s) (listed in the efferents attribute). See Learning Rate and Function for additional details.

value

number, list or np.ndarray – result of the LearningSignal’s function; same as its learning_signal.

learning_signal

number, ndarray or matrix – result of the LearningSignal’s function; same as its value.

efferents

[List[LearningProjection]] – a list of the LearningProjections assigned to (i.e., that project from) the LearningSignal.

modulation

ModulationParam – determines the way in which the value of the LearningSignal is used to modify the value of the matrix parameter for the MappingProjection to which the LearningSignal’s LearningProjection(s) project.

name

str – name of the LearningSignal; the first LearningSignal is always assigned the name LEARNING_SIGNAL; any additional LearningSignals without the name argument of their constructor specified are assigned a default name (see name).

Note

Unlike other PsyNeuLink components, State names are “scoped” within a Mechanism, meaning that States with the same name are permitted in different Mechanisms. However, they are not permitted in the same Mechanism: States within a Mechanism with the same base name are appended an index in the order of their creation.

prefs

PreferenceSet or specification dict – the PreferenceSet for the LearningSignal; if it is not specified in the prefs argument of the constructor, a default is assigned using classPreferences defined in __init__.py (see PreferenceSet for details).