# LearningFunctions¶

Functions that parameterize a function.

class psyneulink.core.components.functions.learningfunctions.LearningFunction(default_variable, params, owner=None, name=None, prefs=None, context=None, **kwargs)

Abstract class of Function used for learning.

variable

most LearningFunctions take a list or 2d array that must contain three items:

• the input to the parameter being modified (variable[LEARNING_ACTIVATION_INPUT]);

• the output of the parameter being modified (variable[LEARNING_ACTIVATION_OUTPUT]);

• the error associated with the output (variable[LEARNING_ERROR_OUTPUT]).

However, the exact specification depends on the funtion’s type.

Type

list or array

default_learning_rate

the value used for the function’s learning_rate parameter if none of the following are specified: the learning_rate for the LearningMechanism to which the function has been assigned, the learning_rate for any Process or the learning_rate for any System to which that LearningMechanism belongs. The exact form of the value (i.e., whether it is a scalar or array) depends on the function’s type.

Type

numeric

learning_rate

generally used to multiply the result of the function before it is returned; however, both the form of the value (i.e., whether it is a scalar or array) and how it is used depend on the function’s type.

Type

numeric

Returns

• The number of items returned and their format depend on the function’s type.

• Most return an array (used as the learning_signal by a LearningMechanism), and some also return a similarly formatted array containing either the error received (in the third item of the variable) or a version of it modified by the function.

class psyneulink.core.components.functions.learningfunctions.Kohonen(default_variable=None, learning_rate=None, distance_function=None, params=None, owner=None, prefs=None)

Calculates a matrix of weight changes using the Kohenen learning rule.

The Kohonen learning rule is used to implement a Kohonen netowrk, which is an instance of the more general category of self organizing map (SOM), that modifies the weights to each element in the network in proportion to its difference from the current input pattern and the distance of that element from the one with the weights most similar to the current input pattern.

function calculates and returns a matrix of weight changes from an array of activity values (in variable[1]) and a weight matrix that generated them (in variable[2]) using the Kohonen learning rule:

$learning\_rate * distance_j * variable[0]-w_j$

where $$distance_j$$ is the distance of the jth element of variable[1] from the element with the weights most similar to activity array in variable[1], and $$w_j$$ is the column of the matrix in variable[2] that corresponds to the jth element of the activity array in variable[1].

Parameters
• variable (List[array(float64), array(float64), 2d array[[float64]]] : default class_defaults.variable) – input pattern, array of activation values, and matrix used to calculate the weights changes.

• learning_rate (scalar or list, 1d or 2d array, or np.matrix of numeric values: default default_learning_rate) – specifies the learning rate used by the function; supersedes any specification for the Process and/or System to which the function’s owner belongs (see learning_rate for details).

• distance_measure (GAUSSIAN, LINEAR, EXPONENTIAL, SINUSOID or function) – specifies the method used to calculate the distance of each element in variable[2] from the one with the greatest value.

• params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

• owner (Component) – component to which to assign the Function.

• name (str : default see name) – specifies the name of the Function.

• prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).

variable

input pattern, array of activation values, and weight matrix used to generate the weight change matrix returned by function.

Type

List[array(float64), array(float64), 2d array[[float64]]]

learning_rate

used by the function to scale the weight change matrix returned by the function. If specified, it supersedes any learning_rate specified for the Process and/or System to which the function’s owner belongs. If it is a scalar, it is multiplied by the weight change matrix; if it is a 1d array, it is multiplied Hadamard (elementwise) by the variable <Kohonen.variable> before calculating the weight change matrix; if it is a 2d array, it is multiplied Hadamard (elementwise) by the weight change matrix; if it is None, then the learning_rate specified for the Process to which the owner belongs is used; and, if that is None, then the learning_rate for the System to which it belongs is used. If all are None, then the default_learning_rate is used.

Type

float, 1d or 2d array

default_learning_rate

the value used for the learning_rate if it is not otherwise specified.

Type

float

function

calculates a matrix of weight changes from: i) the difference between an input pattern (variable <Kohonen.variable>[0]) and the weights in a weigh matrix (variable[2]) to each element of an activity array (variable[1]); and ii) the distance of each element of the activity array (variable <Kohonen.variable>[1])) from the one with the weights most similar to the input array (variable <Kohonen.variable>[0])) using distance_measure.

Type

function

owner

Mechanism to which the Function belongs.

Type

Component

prefs

the PreferenceSet for the Function (see prefs for details).

Type

PreferenceSet or specification dict : default Function.classPreferences

_validate_variable(variable, context=None)

Validate variable and return validated variable

Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:

VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]

Perform top-level type validation of variable against the self.class_defaults.variable;

if the type is OK, the value is returned (which should be used by the function)

This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is True (which it is by default)

IMPLEMENTATION NOTES:
• future versions should add hierarchical/recursive content (e.g., range) checking

• add request/target pattern?? (as per _validate_params) and return validated variable?

Parameters
• variable – (anything other than a dictionary) - variable to be validated:

• context – (str)

Return variable

validated variable

_validate_params(request_set, target_set=None, context=None)

Validate learning_rate

_function(variable=None, context=None, params=None)
Parameters
• variable (array or List[1d array, 1d array, 2d array] : default class_defaults.variable) – input pattern, array of activation values, and matrix used to calculate the weights changes.

• params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

Returns

weight change matrix – matrix of weight changes scaled by difference of the current weights from the input pattern in variable[0] and the distance of each element from the one with the weights most similar to that input pattern.

Return type

2d array

class psyneulink.core.components.functions.learningfunctions.Hebbian(default_variable=None, learning_rate=None, params=None, owner=None, prefs=None)

Calculate a matrix of weight changes using the Hebbian (correlational) learning rule.

function calculates a matrix of weight changes from a 1d array of activity values in variable using the Hebbian learning rule:

$\Delta w_{ij} = learning\_rate * a_ia_j\ if\ i \neq j,\ else\ 0$

where $$a_i$$ and $$a_j$$ are elements of variable.

Parameters
• variable (List[number] or 1d array : default class_defaults.variable) – specifies the activation values, the pair-wise products of which are used to generate the a weight change matrix.

• learning_rate (scalar or list, 1d or 2d array, or np.matrix of numeric values: default default_learning_rate) – specifies the learning rate used by the function; supersedes any specification for the Process and/or System to which the function’s owner belongs (see learning_rate for details).

• params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

• owner (Component) – component to which to assign the Function.

• name (str : default see name) – specifies the name of the Function.

• prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).

variable

activation values, the pair-wise products of which are used to generate the weight change matrix returned by the function.

Type

1d array

learning_rate

used by the function to scale the weight change matrix returned by the function. If specified, it supersedes any learning_rate specified for the Process and/or System to which the function’s owner belongs. If it is a scalar, it is multiplied by the weight change matrix; if it is a 1d array, it is multiplied Hadamard (elementwise) by the variable <Hebbian.variable> before calculating the weight change matrix; if it is a 2d array, it is multiplied Hadamard (elementwise) by the weight change matrix; if it is None, then the learning_rate specified for the Process to which the owner belongs is used; and, if that is None, then the learning_rate for the System to which it belongs is used. If all are None, then the default_learning_rate is used.

Type

float, 1d or 2d array

default_learning_rate

the value used for the learning_rate if it is not otherwise specified.

Type

float

function

calculates the pairwise product of all elements in the variable, and then scales that by the learning_rate to generate the weight change matrix returned by the function.

Type

function

owner

Mechanism to which the Function belongs.

Type

Component

prefs

the PreferenceSet for the Function (see prefs for details).

Type

PreferenceSet or specification dict : default Function.classPreferences

_validate_variable(variable, context=None)

Validate variable and return validated variable

Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:

VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]

Perform top-level type validation of variable against the self.class_defaults.variable;

if the type is OK, the value is returned (which should be used by the function)

This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is True (which it is by default)

IMPLEMENTATION NOTES:
• future versions should add hierarchical/recursive content (e.g., range) checking

• add request/target pattern?? (as per _validate_params) and return validated variable?

Parameters
• variable – (anything other than a dictionary) - variable to be validated:

• context – (str)

Return variable

validated variable

_validate_params(request_set, target_set=None, context=None)

Validate learning_rate

_function(variable=None, context=None, params=None)
Parameters
• variable (List[number] or 1d array : default class_defaults.variable) – array of activity values, the pairwise products of which are used to generate a weight change matrix.

• params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

Returns

weight change matrix – matrix of weight changes generated by the Hebbian Learning rule, with all diagonal elements = 0 (i.e., hollow matix).

Return type

2d array

class psyneulink.core.components.functions.learningfunctions.ContrastiveHebbian(default_variable=None, learning_rate=None, params=None, owner=None, prefs=None)

Calculate a matrix of weight changes using the Contrastive Hebbian learning rule.

function calculates a matrix of weight changes from a 1d array of activity values in variable using the ContrastiveHebbian learning rule:

$\Delta w_{ij} = learning\_rate * (a_i^+a_j^+ - a_i^-a_j^-) \ if\ i \neq j,\ else\ 0$

where $$a_i^+$$ and $$a_j^+$$ are the activites of elements of variable in the plus_phase of execution, and $$a_i^-$$ and $$a_j^-$$ are the activities of those elements in the minus phase of execution.

Parameters
• variable (List[number] or 1d array : default class_defaults.variable) – specifies the activation values, the pair-wise products of which are used to generate the a weight change matrix.

• learning_rate (scalar or list, 1d or 2d array, or np.matrix of numeric values: default default_learning_rate) – specifies the learning rate used by the function; supersedes any specification for the Process and/or System to which the function’s owner belongs (see learning_rate for details).

• params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

• owner (Component) – component to which to assign the Function.

• name (str : default see name) – specifies the name of the Function.

• prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).

variable

activation values, the pair-wise products of which are used to generate the weight change matrix returned by the function.

Type

1d array

learning_rate

used by the function to scale the weight change matrix returned by the function. If specified, it supersedes any learning_rate specified for the Process and/or System to which the function’s owner belongs. If it is a scalar, it is multiplied by the weight change matrix; if it is a 1d array, it is multiplied Hadamard (elementwise) by the variable <ContrastiveHebbian.variable> before calculating the weight change matrix; if it is a 2d array, it is multiplied Hadamard (elementwise) by the weight change matrix; if it is None, then the learning_rate specified for the Process to which the owner belongs is used; and, if that is None, then the learning_rate for the System to which it belongs is used. If all are None, then the default_learning_rate is used.

Type

float, 1d or 2d array

default_learning_rate

the value used for the learning_rate if it is not otherwise specified.

Type

float

function

calculates the pairwise product of all elements in the variable, and then scales that by the learning_rate to generate the weight change matrix returned by the function.

Type

function

owner

Mechanism to which the Function belongs.

Type

Component

prefs

the PreferenceSet for the Function (see prefs for details).

Type

PreferenceSet or specification dict : default Function.classPreferences

_validate_variable(variable, context=None)

Validate variable and return validated variable

Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:

VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]

Perform top-level type validation of variable against the self.class_defaults.variable;

if the type is OK, the value is returned (which should be used by the function)

This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is True (which it is by default)

IMPLEMENTATION NOTES:
• future versions should add hierarchical/recursive content (e.g., range) checking

• add request/target pattern?? (as per _validate_params) and return validated variable?

Parameters
• variable – (anything other than a dictionary) - variable to be validated:

• context – (str)

Return variable

validated variable

_validate_params(request_set, target_set=None, context=None)

Validate learning_rate

_function(variable=None, context=None, params=None)
Parameters
• variable (List[number] or 1d np.array : default class_defaults.variable) – array of activity values, the pairwise products of which are used to generate a weight change matrix.

• params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

Returns

weight change matrix – matrix of weight changes generated by the ContrastiveHeabbian learning rule, with all diagonal elements = 0 (i.e., hollow matix).

Return type

2d array

class psyneulink.core.components.functions.learningfunctions.Reinforcement(default_variable=None, learning_rate=None, params=None, owner=None, prefs=None)

Calculate error term for a single item in an input array, scaled by the learning_rate.

function takes an array (activation_output) with only one non-zero value, and returns an array of the same length with the one non-zero value replaced by $$\Delta w$$ – the error_signal scaled by the learning_rate:

$\Delta w_i = learning\_rate * error\_signal_i\ if\ activation\_output_i \neq 0,\ otherwise\ 0$

The non-zero item in activation_output can be thought of as the predicted likelihood of a stimulus or value of an action, and the error_signal as the error in the prediction for that value.

To preserve compatibility with other LearningFunctions:

• function returns two copies of the error array (the first is a “place-marker”, where a matrix of weights changes is often returned).

Parameters
• default_variable (List or 2d array : default class_defaults.variable) –

template for the three items provided as the variable in the call to the function (in order):

• learning_rate (float : default default_learning_rate) – supersedes any specification for the Process and/or System to which the function’s owner belongs (see learning_rate for details).

• params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

• owner (Component) – component to which to assign the Function.

• name (str : default see name) – specifies the name of the Function.

• prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).

variable

specifies three values used as input to the function:

Type

2d array

activation_input

first item of variable; this is not used (it is implemented for compatibility with other LearningFunctions).

Type

1d array

activation_output

second item of variable; contains a single “prediction” or “action” value as one of its elements, the others of which are zero.

Type

1d array

error_signal

third item of variable; contains a single scalar value, specifying the error associated with the non-zero item in activation_output.

Type

1d array

learning_rate

the learning rate used by the function. If specified, it supersedes any learning_rate specified for the Process and/or System to which the function’s owner belongs. If it is None, then the learning_rate specified for the Process to which the owner belongs is used; and, if that is None, then the learning_rate for the System to which it belongs is used. If all are None, then the default_learning_rate is used.

Type

float

default_learning_rate

the value used for the learning_rate if it is not otherwise specified.

Type

float

function

the function that computes the weight change matrix, and returns that along with the error_signal received.

Type

function

owner

Mechanism to which the Function belongs.

Type

Component

prefs

the PreferenceSet for the Function (see prefs for details).

Type

PreferenceSet or specification dict : default Function.classPreferences

_validate_variable(variable, context=None)

Validate variable and return validated variable

Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:

VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]

Perform top-level type validation of variable against the self.class_defaults.variable;

if the type is OK, the value is returned (which should be used by the function)

This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is True (which it is by default)

IMPLEMENTATION NOTES:
• future versions should add hierarchical/recursive content (e.g., range) checking

• add request/target pattern?? (as per _validate_params) and return validated variable?

Parameters
• variable – (anything other than a dictionary) - variable to be validated:

• context – (str)

Return variable

validated variable

_validate_params(request_set, target_set=None, context=None)

Validate learning_rate

_function(variable=None, context=None, params=None, **kwargs)
Parameters
• variable (List or 2d np.array [length 3 in axis 0] : default class_defaults.variable) –

must have three items that are (in order):

(see note above).

• params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

Returns

error array – Both 1d arrays are the same, with a single non-zero error term (see note above).

Return type

List[1d array, 1d array]

class psyneulink.core.components.functions.learningfunctions.BayesGLM(default_variable=None, mu_0=None, sigma_0=None, gamma_shape_0=None, gamma_size_0=None, params=None, owner=None, seed=None, prefs=None)

Use Bayesian linear regression to find means and distributions of weights that predict dependent variable(s).

function uses a normal linear model:

$dependent\ variable(s) = predictor(s)\ \Theta + \epsilon,$

with predictor(s) in variable[0] and dependent variable(s) in variable[1], and a normal-gamma prior distribution of weights ($$\Theta$$), to update the weight distribution parameters mu_n, Lambda_n, gamma_shape_n, and gamma_size_n, and returns an array of prediction weights sampled from the multivariate normal-gamma distribution [based on Falk Lieder’s BayesianGLM.m, adapted for Python by Yotam Sagiv and for PsyNeuLink by Jon Cohen; useful reference:

Hint

The mu_0 or sigma_0 arguments of the consructor can be used in place of default_variable to define the size of the predictors array and, correspondingly, the weights array returned by the function (see Parameters below).

Parameters
• default_variable (3d array : default None) – first item of axis 0 must be a 2d array with one or more 1d arrays to use as predictor vectors, one for each sample to be fit; second item must be a 2d array of equal length to the first item, with a 1d array containing a scalar that is the dependent (to-be-predicted) value for the corresponding sample in the first item. If None is specified, but either mu_0 or sigma_0 is specified, then the they are used to determine the shape of variable <BayesGLM.variable>. If neither **mu_0 nor sigma_0 are specified, then the shape of variable is determined by the first call to its function, as are mu_prior, sigma_prior, gamma_shape_prior and gamma_size_prior.

• mu_0 (int, float or 1d array : default 0) – specifies initial value of mu_prior (the prior for the mean of the distribution for the prediction weights returned by the function). If a scalar is specified, the same value will be used for all elements of mu_prior; if it is an array, it must be the same length as the predictor array(s) in axis 0 of default_variable. If default_variable is not specified, the specification for mu_0 is used to determine the shape of variable and sigma_prior.

• sigma_0 (int, float or 1d array : default 0) – specifies initial value of sigma_prior (the prior for the variance of the distribution for the prediction weights returned by the function). If a scalar is specified, the same value will be used for all elements of Lambda_prior; if it is an array, it must be the same length as the predictor array(s) in axis 0 of default_variable. If neither default_variable nor mu_0 is specified, the specification for sigma_0 is used to determine their shapes.

• gamma_shape_0 (int or float : default 1) – specifies the shape of the gamma distribution from which samples of the weights are drawn (see documentation for numpy.random.gamma.

• gamma_size_0 (int or float : default 1) –

specifies the size of the gamma distribution from which samples of the weights are drawn (see documentation for numpy.random.gamma.

• params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

• owner (Component) – component to which to assign the Function.

• name (str : default see name) – specifies the name of the Function.

• prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).

variable

samples used to update parameters of prediction weight distributions. variable[0] is a 2d array of predictor vectors, all of the same length; variable[1] is a 2d array of scalar dependent variables, one for each predictor vector.

Type

3d array

mu_0

determines the initial prior(s) for the means of the distributions of the prediction weights; if it is a scalar, that value is assigned as the priors for all means.

Type

int, float or 2d array

mu_prior

current priors for the means of the distributions of the predictions weights.

Type

2d array

mu_n

current means for the distributions of the prediction weights.

Type

2d array

sigma_0

value used to determine the initial prior(s) for the variances of the distributions of the prediction weights; if it is a scalar, that value is assigned as the priors for all variances.

Type

int, float or 2d array

Lambda_prior

current priors for the variances of the distributions of the predictions weights.

Type

2d array

Lambda_n

current variances for the distributions of the prediction weights.

Type

2d array

gamma_shape_0

determines the initial value used for the shape parameter of the gamma distribution used to sample the prediction weights.

Type

int or float

gamma_shape_prior

current prior for the shape parameter of the gamma distribution used to sample the prediction weights.

Type

int or float

gamma_shape_n

current value of the shape parameter of the gamma distribution used to sample the prediction weights.

Type

int or float

gamma_size_0

determines the initial value used for the size parameter of the gamma distribution used to sample the prediction weights.

Type

int or float

gamma_size_prior

current prior for the size parameter of the gamma distribution used to sample the prediction weights.

Type

int or float

gamma_size_n

current value of the size parameter of the gamma distribution used to sample the prediction weights.

Type

2d array with single scalar value

random_state

private pseudorandom number generator

Type

numpy.RandomState

weights_sample

last sample of prediction weights drawn in call to sample_weights and returned by function.

Type

1d array

owner

Mechanism to which the Function belongs.

Type

Component

prefs

the PreferenceSet for the Function (see prefs for details).

Type

PreferenceSet or specification dict : default Function.classPreferences

_handle_default_variable(default_variable=None, size=None)

Finds whether default_variable can be determined using default_variable and size arguments.

Returns

• a default variable if possible

• None otherwise

initialize_priors()

Set the prior parameters (mu_prior, Lamba_prior, gamma_shape_prior, and gamma_size_prior) to their initial (_0) values, and assign current (_n) values to the priors

reset(default_variable=None, context=None)

If the component’s execute method involves execution of an IntegratorFunction Function, this method effectively begins the function’s accumulation over again at the specified value, and may update related values on the component, depending on the component type. Otherwise, it simply reassigns the Component’s value based on its default_variable.

_function(variable=None, context=None, params=None)
Parameters
• variable (2d or 3d array : default class_defaults.variable) –

If it is a 2d array, the first item must be a 1d array of scalar predictors,

and the second must be a 1d array containing the dependent variable to be predicted by the predictors.

If it is a 3d array, the first item in the outermost dimension must be a 2d array containing one or more

1d arrays of scalar predictors, and the second item must be a 2d array containing 1d arrays each of which contains a scalar dependent variable for the corresponding predictor vector.

• params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

Returns

sample weights – array of weights drawn from updated weight distributions.

Return type

1d array

sample_weights(gamma_shape_n, gamma_size_n, mu_n, Lambda_n, context)

Draw a sample of prediction weights from the distributions parameterized by mu_n, Lambda_n, gamma_shape_n, and gamma_size_n.

class psyneulink.core.components.functions.learningfunctions.BackPropagation(default_variable=None, activation_derivative_fct=Logistic().derivative, learning_rate=None, loss_function=None, params=None, name=None, prefs=None)

Calculate and return a matrix of weight changes and weighted error signal from arrays of inputs, outputs and error terms.

This implements the standard form of the backpropagation learning algorithm, using a form of loss determined by the error_signal of the LearningMechanism to which it is assigned.

function calculates a matrix of weight changes using the backpropagation (Generalized Delta Rule) learning algorithm, computed as:

$weight\_change\_matrix = learning\_rate * activation\_input * \frac{\delta E}{\delta W}$

where

$\frac{\delta E}{\delta W} = \frac{\delta E}{\delta A} * \frac{\delta A}{\delta W}$

is the derivative of the error_signal with respect to the weights, in which:

$$\frac{\delta E}{\delta A} = error\_matrix \bullet error\_signal$$

is the derivative of the error with respect to activation_output (i.e., the weighted contribution to the error_signal of each unit that receives activity from the weight matrix being learned), and

$$\frac{\delta A}{\delta W} = activation\_derivative\_fct(input=activation\_input,output=activation\_output)$$

is the derivative of the activation function responsible for generating activation_output at the point that generates each of its entries.

The values of activation_input, activation_output and error_signal are specified as items of the variable both in the constructor for the BackPropagation Function, and in calls to its function. Although error_matrix is not specified in the constructor, it is required as an argument of the function; it is assumed that it’s value is determined in context at the time of execution (e.g., by a LearningMechanism that uses the BackPropagation LearningFunction).

The BackPropagation function returns the weight_change_matrix as well as the error_signal it receives weighted by the contribution made by each element of activation_output as a function of the error_matrix $$\frac{\delta E}{\delta W}$$.

Parameters
• variable (List or 2d array [length 3 in axis 0] : default class_defaults.variable) – specifies a template for the three items provided as the variable in the call to the function (in order): activation_input (1d array), activation_output (1d array), error_signal (1d array).

• activation_derivative_fct (Function or function) – specifies the derivative for the function of the Mechanism that generates activation_output.

• learning_rate (float : default default_learning_rate) – supersedes any specification for the Process and/or System to which the function’s owner belongs (see learning_rate for details).

• loss_function (Loss : default None) – specifies the operation to apply to the error signal (i.e., method of calculating the derivative of the errror with respect to activation) before computing weight changes.

• params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.

• owner (Component) – component to which to assign the Function.

• name (str : default see name) – specifies the name of the Function.

• prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).

variable

contains the three values used as input to the function:

Type

2d array

activation_input

the input to the matrix being modified; same as 1st item of variable.

Type

1d array

activation_output

the output of the function for which the matrix being modified provides the input; same as 2nd item of variable.

Type

1d array

activation_derivative_fct

the derivative for the function of the Mechanism that generates activation_output.

Type

Function or function

error_signal

the error signal for the next matrix (layer above) in the learning pathway, or the error computed from the target (training signal) and the output of the last Mechanism in the sequence; same as 3rd item of variable.

Type

1d array

error_matrix

matrix, the input of which is activation_output and the output of which is used to calculate the error_signal; if it is a ParameterPort, it refers to the MATRIX parameterPort of the MappingProjection being learned.

Type

2d array or ParameterPort

learning_rate

the learning rate used by the function. If specified, it supersedes any learning_rate specified for the process and/or system to which the function’s owner belongs. If it is None, then the learning_rate specified for the process to which the owner belongs is used; and, if that is None, then the learning_rate for the system to which it belongs is used. If all are None, then the default_learning_rate is used.

Type

float

default_learning_rate

the value used for the learning_rate if it is not otherwise specified.

Type

float

loss_function

the operation to apply to the error signal (i.e., method of calculating the derivative of the errror with respect to activation) before computing weight changes.

Type

Loss or None

owner

Mechanism to which the Function belongs.

Type

Component

prefs

the PreferenceSet for the Function (see prefs for details).

Type

PreferenceSet or specification dict : default Function.classPreferences

_validate_variable(variable, context=None)

Validate variable and return validated variable

Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:

VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]

Perform top-level type validation of variable against the self.class_defaults.variable;

if the type is OK, the value is returned (which should be used by the function)

This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is True (which it is by default)

IMPLEMENTATION NOTES:
• future versions should add hierarchical/recursive content (e.g., range) checking

• add request/target pattern?? (as per _validate_params) and return validated variable?

Parameters
• variable – (anything other than a dictionary) - variable to be validated:

• context – (str)

Return variable

validated variable

_validate_params(request_set, target_set=None, context=None)

Validate learning_rate and error_matrix params

error_matrix argument must be one of the following
• 2d list, np.ndarray or np.matrix

• ParameterPort for one of the above

• MappingProjection with a parameterPorts[MATRIX] for one of the above

Parse error_matrix specification and insure it is compatible with error_signal and activation_output

Insure that the length of the error_signal matches the number of cols (receiver elements) of error_matrix

(since it will be dot-producted to generate the weighted error signal)

Insure that length of activation_output matches the number of rows (sender elements) of error_matrix

(since it will be compared against the result of the dot product of the error_matrix and error_signal

Note: error_matrix is left in the form in which it was specified so that, if it is a ParameterPort

or MappingProjection, its current value can be accessed at runtime (i.e., it can be used as a “pointer”)

_function(variable=None, context=None, error_matrix=None, params=None, **kwargs)

Note

Both variable and error_matrix must be specified for the function to execute.

Parameters
Returns

weight change matrix, weighted error signal – the modifications to make to the matrix, error_signal weighted by the contribution made by each element of activation_output as a function of error_matrix.

Return type

List[2d array, 1d array]

class psyneulink.core.components.functions.learningfunctions.TDLearning(default_variable=None, learning_rate=None, params=None, owner=None, prefs=None)

Implement temporal difference learning using the Reinforcement Function (see Reinforcement for class details).

_validate_variable(variable, context=None)

Validate variable and return validated variable

Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:

VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]

Perform top-level type validation of variable against the self.class_defaults.variable;

if the type is OK, the value is returned (which should be used by the function)

This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is True (which it is by default)

IMPLEMENTATION NOTES:
• future versions should add hierarchical/recursive content (e.g., range) checking

• add request/target pattern?? (as per _validate_params) and return validated variable?

Parameters
• variable – (anything other than a dictionary) - variable to be validated:

• context – (str)

Return variable

validated variable