LearningSignal

Contents:

• Learning_signal_Overview

• Creating a LearningSignal

  • Specifying LearningSignals

• Structure

  • Projections

  • Modulation

  • Learning Rate and Function

• Execution

• Class Reference

Overview

A LearningSignal is a type of ModulatorySignal that is specialized for use with a LearningMechanism and one or more LearningProjections, to modify the matrix parameter of the MappingProjections to which they project. A LearningSignal receives the value of a learning_signal calculated by the LearningMechanism to which it belongs, which in general is a matrix of weight changes to be made to the matrix parameter of the MappingProjection(s) being learned. The LearningSignal assigns its learning_signal as the value of its LearningProjection(s), which convey it to the MappingProjections` MATRIX ParameterPort(s), which in turn modify the matrix parameter(s) of the MappingProjection(s) being learned.

Creating a LearningSignal

A LearningSignal is created automatically whenever a MappingProjection is specified for learning and the Projection belongs to the same Composition as the LearningMechanism. LearningSignals can also be specified in the learning_signals argument of the constructor for a LearningMechanism. Although a LearningSignal can be created directly using its constructor (or any of the other ways for creating an OutputPort), this is neither necessary nor advisable, as a LearningSignal has dedicated Components and requirements for configuration that must be met for it to function properly.

Specifying LearningSignals

When a LearningSignal is specified in the learning_signals argument of the constructor for a LearningMechanism, the ParameterPort(s) of the MappingProjection(s) being learned must be specified. This can take any of the following forms:

• an existing LearningSignal, or a reference to the class (in which case a default LearningSignal is created);

• a ParameterPort, which must be for the matrix parameter of the MappingProjection to be learned;

• a Projection, which must be either a LearningProjection, or a MappingProjection to which the LearningSignal should send a LearningProjection. In both cases, it is assumed that the LearningProjection projects to the MATRIX ParameterPort of a MappingProjection.

• a tuple, with the name of the parameter as its 1st item. and the Projection to which it belongs as the 2nd; note that this is a convenience format, which is simpler to use than a specification dictionary (see below), but precludes specification of any parameters for the LearningSignal.

• a specification dictionary, that must contain at least the following two entries:

  • NAME:str

    the string must be the name of the MappingProjection to be learned; the LearningSignal is named by appending “_LearningSignal” to the name of the Projection.

  • PROJECTION:MappingProjection

    the MappingProjection must be valid projection specification for the one to be learned.

  The dictionary can also contain entries for any other LearningSignal attributes to be specified (e.g., a LEARNING_RATE entry); see below for a description of LearningSignal attributes.

Structure

A LearningSignal is owned by an LearningMechanism, and “trains” one or more MappingProjections by modulating the value of their matrix parameters. This is governed by three attributes of the LearningSignal, as described below.

Projections

When a LearningSignal is created, it can be assigned one or more LearningProjections, using either the projections argument of its constructor, or in an entry of a dictionary assigned to the params argument with the key PROJECTIONS. These will be assigned to its efferents attribute. See Port Projections for additional details concerning the specification of Projections when creating a Port.

Note

Although a LearningSignal can be assigned more than one LearningProjection, all of those Projections will convey the same learning_signal (received from the LearningMechanism to which the LearningSignal belongs). Thus, for them to be meaningful, they should project to MappingProjections that are responsible for identical or systematically-related error signals (e.g., as in convolutional networks.

Modulation

A LearningSignal has a modulation attribute that determines how the LearningSignal’s value (i.e., its learning_signal) is used by the ParameterPort(s) to which it projects to modify the matrix parameter(s) of their MappingProjection(s) (see ModulatorySignal Modulation for an explanation of how the modulation attribute is specified and used to modify the value of a parameter). The default value is set to the value of the modulation attribute of the LearningMechanism to which the LearningSignal belongs; this is the same for all of the LearningSignals belonging to that LearningMechanism. The default value of modulation for a LearningMechanism is ADDITIVE, which causes the learning_signal (i.e., its matrix of weight changes) to be added to the matrix parameter of the MappingProjection being learned. The modulation can be individually specified for a LearningSignal using a specification dictionary where the LearningSignal itself is specified, as described above. The modulation value of a LearningSignal is used by all of the LearningProjections that project from that LearningSignal.

Learning Rate and Function

A LearningSignal has a learning_rate attribute that can be used to specify the learning_rate parameter for its LearningProjection(s) (i.e., those listed in its efferents attribute). If specified, it is applied multiplicatively to the LearningProjection`s learning_signal and thus can be used to modulate the learning_rate in addition to (and on top of) one specified for the LearningMechanism or its function. Specification of the learning_rate for a LearningSignal supersedes the learning_rate for its LearningProjections, as well the learning_rate for any Process(es) and/or the learning_rate for any System(s) to which the LearningSignal’s owner belongs (see learning_rate of LearningMechanism for additional details).

The function of a LearningSignal converts the learning_signal it receives from the LearningMechanism to which it belongs to its value (i.e., the LearningSignal’s learning_signal). By default this is an identity function (Linear with slope=1 and intercept=0), that simply uses the LearningMechanism’s learning_signal as its own. However, the LearningSignal’s function can be assigned another TransferFunction, or any other function that takes a scalar, ndarray or matrix and returns a similar value.

Note

The index and assign attributes of a LearningSignal are automatically assigned and should not be modified.

Execution

A LearningSignal cannot be executed directly. It is executed whenever the LearningMechanism to which it belongs is executed. When this occurs, the LearningMechanism provides the LearningSignal with a learning_signal, that is used by its function to compute its value (i.e., its own learning_signal for that TRIAL. That value is used by its LearningProjection(s) to modify the matrix parameter of the MappingProjection(s) to which the LearningSignal projects.

Note

The changes in a MappingProjection’s matrix parameter in response to the execution of a LearningSignal are not applied until the MappingProjection is next executed; see Lazy Evaluation for an explanation of “lazy” updating).

Class Reference

class psyneulink.core.components.ports.modulatorysignals.learningsignal.LearningSignal(owner=None, reference_value=None, variable=None, size=None, index=0, assign=None, function=None, learning_rate=None, modulation=None, modulates=None, params=None, name=None, prefs=None, **kwargs)

LearningSignal( owner, function=Linear(), modulation=MULTIPLICATIVE learning_rate=None params=None, projections=None, name=None, prefs=None)

A subclass of ModulatorySignal used by a LearningMechanism to modify the matrix parameter of one more more MappingProjection(s). See ModulatorySignal for additional arguments and attributes.

functionFunction or methoddefault Linear

specifies the function used by the LearningSignal to generate its learning_signal.

learning_ratefloat or Nonedefault None

specifies the learning_rate for the LearningSignal’s LearningProjections (see learning_rate for details).

modulationstrdefault ADDITIVE

specifies the way in which the value of the LearningSignal is used to modify the value of the matrix parameter for the MappingProjection(s) to which the LearningSignal’s LearningProjection(s) project.

modulateslist of Projection specifications

specifies the LearningProjection(s) to be assigned to the LearningSignal, and that will be listed in its efferents attribute (see Projections for additional details).

functionTransferFunctiondefault Linear(slope=1, intercept=0)

converts variable into the LearningSignal’s learning_signal. The default is the Identity Function, that assigns the LearningSignal’s variable as its learning_signal.

learning_ratefloatNone

determines the learning rate for the LearningSignal. It is used to specify the learning_rate parameter for its LearningProjection(s) (listed in the efferents attribute). See Learning Rate and Function for additional details.

valuenumber, list or np.ndarray

result of the LearningSignal’s function; same as its learning_signal.

learning_signalnumber, ndarray or matrix

result of the LearningSignal’s function; same as its value.

efferents[List[LearningProjection]]

a list of the LearningProjections assigned to (i.e., that project from) the LearningSignal.

modulationstr

determines the way in which the value of the LearningSignal is used to modify the value of the matrix parameter for the MappingProjection to which the LearningSignal’s LearningProjection(s) project.

projection_type

alias of psyneulink.core.components.projections.modulatory.learningprojection.LearningProjection

exception psyneulink.core.components.ports.modulatorysignals.learningsignal.LearningSignalError(message, component=None)