LearningFunctions¶

BayesGLM
Kohonen
Hebbian
ContrastiveHebbian
Reinforcement
BackPropagation
TDLearning

Functions that parameterize a function.

class psyneulink.core.components.functions.learningfunctions.LearningFunction(default_variable, params, owner=None, name=None, prefs=None, context=None, **kwargs)¶

Abstract class of Function used for learning.

variable¶

most LearningFunctions take a list or 2d array that must contain three items:

the input to the parameter being modified (variable[LEARNING_ACTIVATION_INPUT]);
the output of the parameter being modified (variable[LEARNING_ACTIVATION_OUTPUT]);
the error associated with the output (variable[LEARNING_ERROR_OUTPUT]).

However, the exact specification depends on the funtion’s type.

Type: list or array

default_learning_rate¶

the value used for the function’s learning_rate parameter if none of the following are specified: the learning_rate for the LearningMechanism to which the function has been assigned, the learning_rate for any Process or the learning_rate for any System to which that LearningMechanism belongs. The exact form of the value (i.e., whether it is a scalar or array) depends on the function’s type.

Type: numeric

learning_rate¶

generally used to multiply the result of the function before it is returned; however, both the form of the value (i.e., whether it is a scalar or array) and how it is used depend on the function’s type.

Type: numeric

Returns

The number of items returned and their format depend on the function’s type.
Most return an array (used as the learning_signal by a LearningMechanism), and some also return a similarly formatted array containing either the error received (in the third item of the variable) or a version of it modified by the function.

class psyneulink.core.components.functions.learningfunctions.Kohonen(default_variable=None, learning_rate=None, distance_function=None, params=None, owner=None, prefs=None)¶

Calculates a matrix of weight changes using the Kohenen learning rule.

The Kohonen learning rule is used to implement a Kohonen netowrk, which is an instance of the more general category of self organizing map (SOM), that modifies the weights to each element in the network in proportion to its difference from the current input pattern and the distance of that element from the one with the weights most similar to the current input pattern.

function calculates and returns a matrix of weight changes from an array of activity values (in variable[1]) and a weight matrix that generated them (in variable[2]) using the Kohonen learning rule:

\[learning\_rate * distance_j * variable[0]-w_j\]

where \(distance_j\) is the distance of the jth element of variable[1] from the element with the weights most similar to activity array in variable[1], and \(w_j\) is the column of the matrix in variable[2] that corresponds to the jth element of the activity array in variable[1].

Parameters

variable (List[array(float64), array(float64), 2d array[[float64]]] : default class_defaults.variable) – input pattern, array of activation values, and matrix used to calculate the weights changes.
learning_rate (scalar or list, 1d or 2d array, or np.matrix of numeric values: default default_learning_rate) – specifies the learning rate used by the function; supersedes any specification for the Process and/or System to which the function’s owner belongs (see learning_rate for details).
distance_measure (GAUSSIAN, LINEAR, EXPONENTIAL, SINUSOID or function) – specifies the method used to calculate the distance of each element in variable[2] from the one with the greatest value.
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
owner (Component) – component to which to assign the Function.
name (str : default see name) – specifies the name of the Function.
prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).

variable¶

input pattern, array of activation values, and weight matrix used to generate the weight change matrix returned by function.

Type: List[array(float64), array(float64), 2d array[[float64]]]

learning_rate¶

used by the function to scale the weight change matrix returned by the function. If specified, it supersedes any learning_rate specified for the Process and/or System to which the function’s owner belongs. If it is a scalar, it is multiplied by the weight change matrix; if it is a 1d array, it is multiplied Hadamard (elementwise) by the variable <Kohonen.variable>` before calculating the weight change matrix; if it is a 2d array, it is multiplied Hadamard (elementwise) by the weight change matrix; if it is None, then the learning_rate specified for the Process to which the owner belongs is used; and, if that is None, then the learning_rate for the System to which it belongs is used. If all are None, then the default_learning_rate is used.

Type: float, 1d or 2d array

default_learning_rate¶

the value used for the learning_rate if it is not otherwise specified.

Type: float

function¶

calculates a matrix of weight changes from: i) the difference between an input pattern (variable <Kohonen.variable>`[0]) and the weights in a weigh matrix (variable[2]) to each element of an activity array (variable[1]); and ii) the distance of each element of the activity array (variable <Kohonen.variable>`[1])) from the one with the weights most similar to the input array (variable <Kohonen.variable>`[0])) using distance_measure.

Type: function

owner¶

Mechanism to which the Function belongs.

Type: Component

prefs¶

the PreferenceSet for the Function (see prefs for details).

Type: PreferenceSet or specification dict : default Function.classPreferences

_validate_variable(variable, context=None)¶

Validate variable and return validated variable

Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:

VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]

Perform top-level type validation of variable against the self.class_defaults.variable;: if the type is OK, the value is returned (which should be used by the function)

This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is True (which it is by default)

IMPLEMENTATION NOTES:

future versions should add hierarchical/recursive content (e.g., range) checking
add request/target pattern?? (as per _validate_params) and return validated variable?

Parameters

variable – (anything other than a dictionary) - variable to be validated:
context – (str)

Return variable

validated variable

_validate_params(request_set, target_set=None, context=None)¶: Validate learning_rate

_function(variable=None, context=None, params=None)¶

Parameters

variable (array or List[1d array, 1d array, 2d array] : default class_defaults.variable) – input pattern, array of activation values, and matrix used to calculate the weights changes.
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

Returns

weight change matrix – matrix of weight changes scaled by difference of the current weights from the input pattern in variable[0] and the distance of each element from the one with the weights most similar to that input pattern.

Return type

2d array

class psyneulink.core.components.functions.learningfunctions.Hebbian(default_variable=None, learning_rate=None, params=None, owner=None, prefs=None)¶

Calculate a matrix of weight changes using the Hebbian (correlational) learning rule.

function calculates a matrix of weight changes from a 1d array of activity values in variable using the Hebbian learning rule:

\[\Delta w_{ij} = learning\_rate * a_ia_j\ if\ i \neq j,\ else\ 0\]

where \(a_i\) and \(a_j\) are elements of variable.

Parameters

variable (List[number] or 1d array : default class_defaults.variable) – specifies the activation values, the pair-wise products of which are used to generate the a weight change matrix.
learning_rate (scalar or list, 1d or 2d array, or np.matrix of numeric values: default default_learning_rate) – specifies the learning rate used by the function; supersedes any specification for the Process and/or System to which the function’s owner belongs (see learning_rate for details).
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
owner (Component) – component to which to assign the Function.
name (str : default see name) – specifies the name of the Function.
prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).

variable¶

activation values, the pair-wise products of which are used to generate the weight change matrix returned by the function.

Type: 1d array

learning_rate¶

used by the function to scale the weight change matrix returned by the function. If specified, it supersedes any learning_rate specified for the Process and/or System to which the function’s owner belongs. If it is a scalar, it is multiplied by the weight change matrix; if it is a 1d array, it is multiplied Hadamard (elementwise) by the variable <Hebbian.variable>` before calculating the weight change matrix; if it is a 2d array, it is multiplied Hadamard (elementwise) by the weight change matrix; if it is None, then the learning_rate specified for the Process to which the owner belongs is used; and, if that is None, then the learning_rate for the System to which it belongs is used. If all are None, then the default_learning_rate is used.

Type: float, 1d or 2d array

default_learning_rate¶

the value used for the learning_rate if it is not otherwise specified.

Type: float

function¶

calculates the pairwise product of all elements in the variable, and then scales that by the learning_rate to generate the weight change matrix returned by the function.

Type: function

owner¶

Mechanism to which the Function belongs.

Type: Component

prefs¶

the PreferenceSet for the Function (see prefs for details).

Type: PreferenceSet or specification dict : default Function.classPreferences

_validate_variable(variable, context=None)¶

Validate variable and return validated variable

Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:

VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]

Perform top-level type validation of variable against the self.class_defaults.variable;: if the type is OK, the value is returned (which should be used by the function)

This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is True (which it is by default)

IMPLEMENTATION NOTES:

future versions should add hierarchical/recursive content (e.g., range) checking
add request/target pattern?? (as per _validate_params) and return validated variable?

Parameters

variable – (anything other than a dictionary) - variable to be validated:
context – (str)

Return variable

validated variable

_validate_params(request_set, target_set=None, context=None)¶: Validate learning_rate

_function(variable=None, context=None, params=None)¶

Parameters

variable (List[number] or 1d array : default class_defaults.variable) – array of activity values, the pairwise products of which are used to generate a weight change matrix.
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

Returns

weight change matrix – matrix of weight changes generated by the Hebbian Learning rule, with all diagonal elements = 0 (i.e., hollow matix).

Return type

2d array

class psyneulink.core.components.functions.learningfunctions.ContrastiveHebbian(default_variable=None, learning_rate=None, params=None, owner=None, prefs=None)¶

Calculate a matrix of weight changes using the Contrastive Hebbian learning rule.

function calculates a matrix of weight changes from a 1d array of activity values in variable using the ContrastiveHebbian learning rule:

\[\Delta w_{ij} = learning\_rate * (a_i^+a_j^+ - a_i^-a_j^-) \ if\ i \neq j,\ else\ 0\]

where \(a_i^+\) and \(a_j^+\) are the activites of elements of variable in the plus_phase of execution, and \(a_i^-\) and \(a_j^-\) are the activities of those elements in the minus phase of execution.

Parameters

variable (List[number] or 1d array : default class_defaults.variable) – specifies the activation values, the pair-wise products of which are used to generate the a weight change matrix.
learning_rate (scalar or list, 1d or 2d array, or np.matrix of numeric values: default default_learning_rate) – specifies the learning rate used by the function; supersedes any specification for the Process and/or System to which the function’s owner belongs (see learning_rate for details).
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
owner (Component) – component to which to assign the Function.
name (str : default see name) – specifies the name of the Function.
prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).

variable¶

activation values, the pair-wise products of which are used to generate the weight change matrix returned by the function.

Type: 1d array

learning_rate¶

used by the function to scale the weight change matrix returned by the function. If specified, it supersedes any learning_rate specified for the Process and/or System to which the function’s owner belongs. If it is a scalar, it is multiplied by the weight change matrix; if it is a 1d array, it is multiplied Hadamard (elementwise) by the variable <ContrastiveHebbian.variable>` before calculating the weight change matrix; if it is a 2d array, it is multiplied Hadamard (elementwise) by the weight change matrix; if it is None, then the learning_rate specified for the Process to which the owner belongs is used; and, if that is None, then the learning_rate for the System to which it belongs is used. If all are None, then the default_learning_rate is used.

Type: float, 1d or 2d array

default_learning_rate¶

the value used for the learning_rate if it is not otherwise specified.

Type: float

function¶

calculates the pairwise product of all elements in the variable, and then scales that by the learning_rate to generate the weight change matrix returned by the function.

Type: function

owner¶

Mechanism to which the Function belongs.

Type: Component

prefs¶

the PreferenceSet for the Function (see prefs for details).

Type: PreferenceSet or specification dict : default Function.classPreferences

_validate_variable(variable, context=None)¶

Validate variable and return validated variable

Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:

VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]

Perform top-level type validation of variable against the self.class_defaults.variable;: if the type is OK, the value is returned (which should be used by the function)

This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is True (which it is by default)

IMPLEMENTATION NOTES:

future versions should add hierarchical/recursive content (e.g., range) checking
add request/target pattern?? (as per _validate_params) and return validated variable?

Parameters

variable – (anything other than a dictionary) - variable to be validated:
context – (str)

Return variable

validated variable

_validate_params(request_set, target_set=None, context=None)¶: Validate learning_rate

_function(variable=None, context=None, params=None)¶

Parameters

variable (List[number] or 1d np.array : default class_defaults.variable) – array of activity values, the pairwise products of which are used to generate a weight change matrix.
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

Returns

weight change matrix – matrix of weight changes generated by the ContrastiveHeabbian learning rule, with all diagonal elements = 0 (i.e., hollow matix).

Return type

2d array

class psyneulink.core.components.functions.learningfunctions.Reinforcement(default_variable=None, learning_rate=None, params=None, owner=None, prefs=None)¶

Calculate error term for a single item in an input array, scaled by the learning_rate.

function takes an array (activation_output) with only one non-zero value, and returns an array of the same length with the one non-zero value replaced by \(\Delta w\) – the error_signal scaled by the learning_rate:

\[\Delta w_i = learning\_rate * error\_signal_i\ if\ activation\_output_i \neq 0,\ otherwise\ 0\]

The non-zero item in activation_output can be thought of as the predicted likelihood of a stimulus or value of an action, and the error_signal as the error in the prediction for that value.

To preserve compatibility with other LearningFunctions:

the variable argument of both the constructor and calls to function must have three items, although only the 2nd and 3rd items are used; these are referenced by the activation_output and error_signal attributes, respectively (the first item is used by other LearningFunctions as their activation_input attribute).

function returns two copies of the error array (the first is a “place-marker”, where a matrix of weights changes is often returned).

Parameters

default_variable (List or 2d array : default class_defaults.variable) –
template for the three items provided as the variable in the call to the function (in order):
- activation_input (1d array) (not used);
- activation_output (1d array with only one non-zero value);
- error_signal (1d array with a single scalar element).
learning_rate (float : default default_learning_rate) – supersedes any specification for the Process and/or System to which the function’s owner belongs (see learning_rate for details).
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
owner (Component) – component to which to assign the Function.
name (str : default see name) – specifies the name of the Function.
prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).

variable¶

specifies three values used as input to the function:

activation_input,

activation_output, and

error_signal.

Type: 2d array

activation_input¶

first item of variable; this is not used (it is implemented for compatibility with other LearningFunctions).

Type: 1d array

activation_output¶

second item of variable; contains a single “prediction” or “action” value as one of its elements, the others of which are zero.

Type: 1d array

error_signal¶

third item of variable; contains a single scalar value, specifying the error associated with the non-zero item in activation_output.

Type: 1d array

learning_rate¶

the learning rate used by the function. If specified, it supersedes any learning_rate specified for the Process and/or System to which the function’s owner belongs. If it is None, then the learning_rate specified for the Process to which the owner belongs is used; and, if that is None, then the learning_rate for the System to which it belongs is used. If all are None, then the default_learning_rate is used.

Type: float

default_learning_rate¶

the value used for the learning_rate if it is not otherwise specified.

Type: float

function¶

the function that computes the weight change matrix, and returns that along with the error_signal received.

Type: function

owner¶

Mechanism to which the Function belongs.

Type: Component

prefs¶

the PreferenceSet for the Function (see prefs for details).

Type: PreferenceSet or specification dict : default Function.classPreferences

_validate_variable(variable, context=None)¶

Validate variable and return validated variable

Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:

VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]

Perform top-level type validation of variable against the self.class_defaults.variable;: if the type is OK, the value is returned (which should be used by the function)

This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is True (which it is by default)

IMPLEMENTATION NOTES:

future versions should add hierarchical/recursive content (e.g., range) checking
add request/target pattern?? (as per _validate_params) and return validated variable?

Parameters

variable – (anything other than a dictionary) - variable to be validated:
context – (str)

Return variable

validated variable

_validate_params(request_set, target_set=None, context=None)¶: Validate learning_rate

_function(variable=None, context=None, params=None, **kwargs)¶

Parameters

variable (List or 2d np.array [length 3 in axis 0] : default class_defaults.variable) –
must have three items that are (in order):
- activation_input (not used);
- activation_output (1d array with only one non-zero value);
- error_signal (1d array with a single scalar element);
(see note above).
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

Returns

error array – Both 1d arrays are the same, with a single non-zero error term (see note above).

Return type

List[1d array, 1d array]

class psyneulink.core.components.functions.learningfunctions.BayesGLM(default_variable=None, mu_0=None, sigma_0=None, gamma_shape_0=None, gamma_size_0=None, params=None, owner=None, seed=None, prefs=None)¶

Use Bayesian linear regression to find means and distributions of weights that predict dependent variable(s).

function uses a normal linear model:

\[dependent\ variable(s) = predictor(s)\ \Theta + \epsilon,\]

with predictor(s) in variable[0] and dependent variable(s) in variable[1], and a normal-gamma prior distribution of weights (\(\Theta\)), to update the weight distribution parameters mu_n, Lambda_n, gamma_shape_n, and gamma_size_n, and returns an array of prediction weights sampled from the multivariate normal-gamma distribution [based on Falk Lieder’s BayesianGLM.m, adapted for Python by Yotam Sagiv and for PsyNeuLink by Jon Cohen; useful reference:

Bayesian Inference.]

Hint

The mu_0 or sigma_0 arguments of the consructor can be used in place of default_variable to define the size of the predictors array and, correspondingly, the weights array returned by the function (see Parameters below).

Parameters

default_variable (3d array : default None) – first item of axis 0 must be a 2d array with one or more 1d arrays to use as predictor vectors, one for each sample to be fit; second item must be a 2d array of equal length to the first item, with a 1d array containing a scalar that is the dependent (to-be-predicted) value for the corresponding sample in the first item. If None is specified, but either mu_0 or sigma_0 is specified, then the they are used to determine the shape of `variable <BayesGLM.variable>`. If neither **mu_0 nor sigma_0 are specified, then the shape of variable is determined by the first call to its function, as are mu_prior, sigma_prior, gamma_shape_prior and gamma_size_prior.
mu_0 (int, float or 1d array : default 0) – specifies initial value of mu_prior (the prior for the mean of the distribution for the prediction weights returned by the function). If a scalar is specified, the same value will be used for all elements of mu_prior; if it is an array, it must be the same length as the predictor array(s) in axis 0 of default_variable. If default_variable is not specified, the specification for mu_0 is used to determine the shape of variable and sigma_prior.
sigma_0 (int, float or 1d array : default 0) – specifies initial value of sigma_prior (the prior for the variance of the distribution for the prediction weights returned by the function). If a scalar is specified, the same value will be used for all elements of Lambda_prior; if it is an array, it must be the same length as the predictor array(s) in axis 0 of default_variable. If neither default_variable nor mu_0 is specified, the specification for sigma_0 is used to determine their shapes.
gamma_shape_0 (int or float : default 1) – specifies the shape of the gamma distribution from which samples of the weights are drawn (see documentation for numpy.random.gamma.
gamma_size_0 (int or float : default 1) –
specifies the size of the gamma distribution from which samples of the weights are drawn (see documentation for numpy.random.gamma.
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
owner (Component) – component to which to assign the Function.
name (str : default see name) – specifies the name of the Function.
prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).

variable¶

samples used to update parameters of prediction weight distributions. variable[0] is a 2d array of predictor vectors, all of the same length; variable[1] is a 2d array of scalar dependent variables, one for each predictor vector.

Type: 3d array

mu_0¶

determines the initial prior(s) for the means of the distributions of the prediction weights; if it is a scalar, that value is assigned as the priors for all means.

Type: int, float or 2d array

mu_prior¶

current priors for the means of the distributions of the predictions weights.

Type: 2d array

mu_n¶

current means for the distributions of the prediction weights.

Type: 2d array

sigma_0¶

value used to determine the initial prior(s) for the variances of the distributions of the prediction weights; if it is a scalar, that value is assigned as the priors for all variances.

Type: int, float or 2d array

Lambda_prior¶

current priors for the variances of the distributions of the predictions weights.

Type: 2d array

Lambda_n¶

current variances for the distributions of the prediction weights.

Type: 2d array

gamma_shape_0¶

determines the initial value used for the shape parameter of the gamma distribution used to sample the prediction weights.

Type: int or float

gamma_shape_prior¶

current prior for the shape parameter of the gamma distribution used to sample the prediction weights.

Type: int or float

gamma_shape_n¶

current value of the shape parameter of the gamma distribution used to sample the prediction weights.

Type: int or float

gamma_size_0¶

determines the initial value used for the size parameter of the gamma distribution used to sample the prediction weights.

Type: int or float

gamma_size_prior¶

current prior for the size parameter of the gamma distribution used to sample the prediction weights.

Type: int or float

gamma_size_n¶

current value of the size parameter of the gamma distribution used to sample the prediction weights.

Type: 2d array with single scalar value

random_state¶

private pseudorandom number generator

Type: numpy.RandomState

weights_sample¶

last sample of prediction weights drawn in call to sample_weights and returned by function.

Type: 1d array

owner¶

Mechanism to which the Function belongs.

Type: Component

prefs¶

the PreferenceSet for the Function (see prefs for details).

Type: PreferenceSet or specification dict : default Function.classPreferences

_handle_default_variable(default_variable=None, size=None)¶

Finds whether default_variable can be determined using default_variable and size arguments.

Returns

a default variable if possible
None otherwise

initialize_priors()¶: Set the prior parameters (mu_prior, Lamba_prior, gamma_shape_prior, and gamma_size_prior) to their initial (_0) values, and assign current (_n) values to the priors

reset(default_variable=None, context=None)¶: If the component’s execute method involves execution of an IntegratorFunction Function, this method effectively begins the function’s accumulation over again at the specified value, and may update related values on the component, depending on the component type. Otherwise, it simply reassigns the Component’s value based on its default_variable.

_function(variable=None, context=None, params=None)¶

Parameters

variable (2d or 3d array : default class_defaults.variable) –

If it is a 2d array, the first item must be a 1d array of scalar predictors,
and the second must be a 1d array containing the dependent variable to be predicted by the predictors.

If it is a 3d array, the first item in the outermost dimension must be a 2d array containing one or more
1d arrays of scalar predictors, and the second item must be a 2d array containing 1d arrays each of which contains a scalar dependent variable for the corresponding predictor vector.
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

Returns

sample weights – array of weights drawn from updated weight distributions.

Return type

1d array

sample_weights(gamma_shape_n, gamma_size_n, mu_n, Lambda_n, context)¶: Draw a sample of prediction weights from the distributions parameterized by mu_n, Lambda_n, gamma_shape_n, and gamma_size_n.

class psyneulink.core.components.functions.learningfunctions.BackPropagation(default_variable=None, activation_derivative_fct=Logistic().derivative, learning_rate=None, loss_function=None, params=None, name=None, prefs=None)¶

Calculate and return a matrix of weight changes and weighted error signal from arrays of inputs, outputs and error terms.

This implements the standard form of the backpropagation learning algorithm, using a form of loss determined by the error_signal of the LearningMechanism to which it is assigned.

function calculates a matrix of weight changes using the backpropagation (Generalized Delta Rule) learning algorithm, computed as:

\[weight\_change\_matrix = learning\_rate * activation\_input * \frac{\delta E}{\delta W}\]

where

\[\frac{\delta E}{\delta W} = \frac{\delta E}{\delta A} * \frac{\delta A}{\delta W}\]

is the derivative of the error_signal with respect to the weights, in which:

\(\frac{\delta E}{\delta A} = error\_matrix \bullet error\_signal\)

is the derivative of the error with respect to activation_output (i.e., the weighted contribution to the error_signal of each unit that receives activity from the weight matrix being learned), and

\(\frac{\delta A}{\delta W} = activation\_derivative\_fct(input=activation\_input,output=activation\_output)\)

is the derivative of the activation function responsible for generating activation_output at the point that generates each of its entries.

The values of activation_input, activation_output and error_signal are specified as items of the variable both in the constructor for the BackPropagation Function, and in calls to its function. Although error_matrix is not specified in the constructor, it is required as an argument of the function; it is assumed that it’s value is determined in context at the time of execution (e.g., by a LearningMechanism that uses the BackPropagation LearningFunction).

The BackPropagation function returns the weight_change_matrix as well as the error_signal it receives weighted by the contribution made by each element of activation_output as a function of the error_matrix \(\frac{\delta E}{\delta W}\).

Parameters

variable (List or 2d array [length 3 in axis 0] : default class_defaults.variable) – specifies a template for the three items provided as the variable in the call to the function (in order): activation_input (1d array), activation_output (1d array), error_signal (1d array).
activation_derivative_fct (Function or function) – specifies the derivative for the function of the Mechanism that generates activation_output.
learning_rate (float : default default_learning_rate) – supersedes any specification for the Process and/or System to which the function’s owner belongs (see learning_rate for details).
loss_function (Loss : default None) – specifies the operation to apply to the error signal (i.e., method of calculating the derivative of the errror with respect to activation) before computing weight changes.
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
owner (Component) – component to which to assign the Function.
name (str : default see name) – specifies the name of the Function.
prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).

variable¶

contains the three values used as input to the function:

activation_input, activation_output, and error_signal.

Type: 2d array

activation_input¶

the input to the matrix being modified; same as 1st item of variable.

Type: 1d array

activation_output¶

the output of the function for which the matrix being modified provides the input; same as 2nd item of variable.

Type: 1d array

activation_derivative_fct¶

the derivative for the function of the Mechanism that generates activation_output.

Type: Function or function

error_signal¶

the error signal for the next matrix (layer above) in the learning pathway, or the error computed from the target (training signal) and the output of the last Mechanism in the sequence; same as 3rd item of variable.

Type: 1d array

error_matrix¶

matrix, the input of which is activation_output and the output of which is used to calculate the error_signal; if it is a ParameterPort, it refers to the MATRIX parameterPort of the MappingProjection being learned.

Type: 2d array or ParameterPort

learning_rate¶

the learning rate used by the function. If specified, it supersedes any learning_rate specified for the process and/or system to which the function’s owner belongs. If it is None, then the learning_rate specified for the process to which the owner belongs is used; and, if that is None, then the learning_rate for the system to which it belongs is used. If all are None, then the default_learning_rate is used.

Type: float

default_learning_rate¶

the value used for the learning_rate if it is not otherwise specified.

Type: float

loss_function¶

the operation to apply to the error signal (i.e., method of calculating the derivative of the errror with respect to activation) before computing weight changes.

Type: Loss or None

owner¶

Mechanism to which the Function belongs.

Type: Component

prefs¶

the PreferenceSet for the Function (see prefs for details).

Type: PreferenceSet or specification dict : default Function.classPreferences

_validate_variable(variable, context=None)¶

Validate variable and return validated variable

Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:

VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]

Perform top-level type validation of variable against the self.class_defaults.variable;: if the type is OK, the value is returned (which should be used by the function)

This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is True (which it is by default)

IMPLEMENTATION NOTES:

future versions should add hierarchical/recursive content (e.g., range) checking
add request/target pattern?? (as per _validate_params) and return validated variable?

Parameters

variable – (anything other than a dictionary) - variable to be validated:
context – (str)

Return variable

validated variable

_validate_params(request_set, target_set=None, context=None)¶

Validate learning_rate and error_matrix params

error_matrix argument must be one of the following

2d list, np.ndarray or np.matrix
ParameterPort for one of the above
MappingProjection with a parameterPorts[MATRIX] for one of the above

Parse error_matrix specification and insure it is compatible with error_signal and activation_output

Insure that the length of the error_signal matches the number of cols (receiver elements) of error_matrix: (since it will be dot-producted to generate the weighted error signal)
Insure that length of activation_output matches the number of rows (sender elements) of error_matrix: (since it will be compared against the result of the dot product of the error_matrix and error_signal
Note: error_matrix is left in the form in which it was specified so that, if it is a ParameterPort: or MappingProjection, its current value can be accessed at runtime (i.e., it can be used as a “pointer”)

_function(variable=None, context=None, error_matrix=None, params=None, **kwargs)¶

Note

Both variable and error_matrix must be specified for the function to execute.

Parameters

variable (List or 2d array [length 3 in axis 0]) – must have three items that are the values for (in order): activation_input (1d array), activation_output (1d array), error_signal (1d array).
error_matrix (List, 2d array, np.matrix, ParameterPort, or MappingProjection) – matrix of weights that were used to generate the error_signal (3rd item of variable from activation_output; its dimensions must be the length of activation_output (rows) x length of error_signal (cols).
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

Returns

weight change matrix, weighted error signal – the modifications to make to the matrix, error_signal weighted by the contribution made by each element of activation_output as a function of error_matrix.

Return type

List[2d array, 1d array]

class psyneulink.core.components.functions.learningfunctions.TDLearning(default_variable=None, learning_rate=None, params=None, owner=None, prefs=None)¶

Implement temporal difference learning using the Reinforcement Function (see Reinforcement for class details).

_validate_variable(variable, context=None)¶

Validate variable and return validated variable

Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:

VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]

Perform top-level type validation of variable against the self.class_defaults.variable;: if the type is OK, the value is returned (which should be used by the function)

This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is True (which it is by default)

IMPLEMENTATION NOTES:

future versions should add hierarchical/recursive content (e.g., range) checking
add request/target pattern?? (as per _validate_params) and return validated variable?

Parameters

variable – (anything other than a dictionary) - variable to be validated:
context – (str)

Return variable

validated variable