LearningFunctions

Functions that parameterize a function.

class psyneulink.core.components.functions.learningfunctions.BayesGLM(default_variable=None, mu_0=0, sigma_0=1, gamma_shape_0=1, gamma_size_0=1, params=None, prefs=None)

Use Bayesian linear regression to find means and distributions of weights that predict dependent variable(s).

function uses a normal linear model:

\[dependent\ variable(s) = predictor(s)\ \Theta + \epsilon,\]

with predictor(s) in variable[0] and dependent variable(s) in variable[1], and a normal-gamma prior distribution of weights (\(\Theta\)), to update the weight distribution parameters mu_n, Lambda_n, gamma_shape_n, and gamma_size_n, and returns an array of prediction weights sampled from the multivariate normal-gamma distribution [based on Falk Lieder’s BayesianGLM.m, adapted for Python by Yotam Sagiv and for PsyNeuLink by Jon Cohen; useful reference:

Bayesian Inference.]

Hint

The mu_0 or sigma_0 arguments of the consructor can be used in place of default_variable to define the size of the predictors array and, correspondingly, the weights array returned by the function (see Parameters below).

Parameters:
  • default_variable (3d array : default None) – first item of axis 0 must be a 2d array with one or more 1d arrays to use as predictor vectors, one for each sample to be fit; second item must be a 2d array of equal length to the first item, with a 1d array containing a scalar that is the dependent (to-be-predicted) value for the corresponding sample in the first item. If None is specified, but either mu_0 or sigma_0 is specified, then the they are used to determine the shape of `variable <BayesGLM.variable>`. If neither **mu_0 nor sigma_0 are specified, then the shape of variable is determined by the first call to its function, as are mu_prior, sigma_prior, gamma_shape_prior and gamma_size_prior.
  • mu_0 (int, float or 1d array : default 0) – specifies initial value of mu_prior (the prior for the mean of the distribution for the prediction weights returned by the function). If a scalar is specified, the same value will be used for all elements of mu_prior; if it is an array, it must be the same length as the predictor array(s) in axis 0 of default_variable. If default_variable is not specified, the specification for mu_0 is used to determine the shape of variable and sigma_prior.
  • sigma_0 (int, float or 1d array : default 0) – specifies initial value of sigma_prior (the prior for the variance of the distribution for the prediction weights returned by the function). If a scalar is specified, the same value will be used for all elements of Lambda_prior; if it is an array, it must be the same length as the predictor array(s) in axis 0 of default_variable. If neither default_variable nor mu_0 is specified, the specification for sigma_0 is used to determine their shapes.
  • gamma_shape_0 (int or float : default 1) – specifies the shape of the gamma distribution from which samples of the weights are drawn (see documentation for numpy.random.gamma.
  • gamma_size_0 (int or float : default 1) –

    specifies the size of the gamma distribution from which samples of the weights are drawn (see documentation for numpy.random.gamma.

  • params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
  • owner (Component) – component to which to assign the Function.
  • name (str : default see name) – specifies the name of the Function.
  • prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).
variable

3d array – samples used to update parameters of prediction weight distributions. variable[0] is a 2d array of predictor vectors, all of the same length; variable[1] is a 2d array of scalar dependent variables, one for each predictor vector.

mu_0

int, float or 2d array – determines the initial prior(s) for the means of the distributions of the prediction weights; if it is a scalar, that value is assigned as the priors for all means.

mu_prior

2d array – current priors for the means of the distributions of the predictions weights.

mu_n

2d array – current means for the distributions of the prediction weights.

sigma_0

int, float or 2d array – value used to determine the initial prior(s) for the variances of the distributions of the prediction weights; if it is a scalar, that value is assigned as the priors for all variances.

Lambda_prior

2d array – current priors for the variances of the distributions of the predictions weights.

Lambda_n

2d array – current variances for the distributions of the prediction weights.

gamma_shape_0

int or float – determines the initial value used for the shape parameter of the gamma distribution used to sample the prediction weights.

gamma_shape_prior

int or float – current prior for the shape parameter of the gamma distribution used to sample the prediction weights.

gamma_shape_n

int or float – current value of the shape parameter of the gamma distribution used to sample the prediction weights.

gamma_size_0

int or float – determines the initial value used for the size parameter of the gamma distribution used to sample the prediction weights.

gamma_size_prior

int or float – current prior for the size parameter of the gamma distribution used to sample the prediction weights.

gamma_size_n

2d array with single scalar value – current value of the size parameter of the gamma distribution used to sample the prediction weights.

weights_sample

1d array – last sample of prediction weights drawn in call to sample_weights and returned by function.

owner

ComponentMechanism to which the Function belongs.

prefs

PreferenceSet or specification dict : default Function.classPreferences – the PreferenceSet for the Function (see prefs for details).

initialize_priors()

Set the prior parameters (mu_prior, Lamba_prior, gamma_shape_prior, and gamma_size_prior) to their initial (_0) values, and assign current (_n) values to the priors

function(variable=None, execution_id=None, params=None, context=None)
Parameters:
  • variable (2d or 3d array : default class_defaults.variable) – if it is a 2d array, the first item must be a 1d array of scalar predictors, and the second must be a 1d array containing the dependent variable to be predicted by the predictors; if it is a 3d array, the first item in the outermost dimension must be 2d array containing one or more 1d arrays of scalar predictors, and the second item be a 2d array containing 1d arrays each of which contains a scalar dependent variable for the corresponding predictor vector.
  • params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
Returns:

sample weights – array of weights drawn from updated weight distributions.

Return type:

1d array

sample_weights(gamma_shape_n, gamma_size_n, mu_n, Lambda_n)

Draw a sample of prediction weights from the distributions parameterized by mu_n, Lambda_n, gamma_shape_n, and gamma_size_n.

class psyneulink.core.components.functions.learningfunctions.Kohonen(default_variable=None, learning_rate=None, distance_measure=GAUSSIAN, params=None, name=None, prefs=None)

Calculates a matrix of weight changes using the Kohenen learning rule.

The Kohonen learning rule is used to implement a Kohonen netowrk, which is an instance of the more general category of self organizing map (SOM), that modifies the weights to each element in the network in proportion to its difference from the current input pattern and the distance of that element from the one with the weights most similar to the current input pattern.

function calculates and returns a matrix of weight changes from an array of activity values (in variable[1]) and a weight matrix that generated them (in variable[2]) using the Kohonen learning rule:

\[learning\_rate * distance_j * variable[0]-w_j\]

where \(distance_j\) is the distance of the jth element of variable[1] from the element with the weights most similar to activity array in variable[1], and \(w_j\) is the column of the matrix in variable[2] that corresponds to the jth element of the activity array in variable[1].

Parameters:
  • variable (List[array(float64), array(float64), 2d array[[float64]]] : default class_defaults.variable) – input pattern, array of activation values, and matrix used to calculate the weights changes.
  • learning_rate (scalar or list, 1d or 2d array, or np.matrix of numeric values: default default_learning_rate) – specifies the learning rate used by the function; supersedes any specification for the Process and/or System to which the function’s owner belongs (see learning_rate for details).
  • distance_measure (GAUSSIAN, LINEAR, EXPONENTIAL, SINUSOID or function) – specifies the method used to calculate the distance of each element in variable[2] from the one with the greatest value.
  • params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
  • owner (Component) – component to which to assign the Function.
  • name (str : default see name) – specifies the name of the Function.
  • prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).
variable

List[array(float64), array(float64), 2d array[[float64]]] – input pattern, array of activation values, and weight matrix used to generate the weight change matrix returned by function.

learning_rate

float, 1d or 2d array – used by the function to scale the weight change matrix returned by the function. If specified, it supersedes any learning_rate specified for the Process and/or System to which the function’s owner belongs. If it is a scalar, it is multiplied by the weight change matrix; if it is a 1d array, it is multiplied Hadamard (elementwise) by the variable <Kohonen.variable>` before calculating the weight change matrix; if it is a 2d array, it is multiplied Hadamard (elementwise) by the weight change matrix; if it is None, then the learning_rate specified for the Process to which the owner belongs is used; and, if that is None, then the learning_rate for the System to which it belongs is used. If all are None, then the default_learning_rate is used.

default_learning_rate

float – the value used for the learning_rate if it is not otherwise specified.

function

function – calculates a matrix of weight changes from: i) the difference between an input pattern (variable <Kohonen.variable>`[0]) and the weights in a weigh matrix (variable[2]) to each element of an activity array (variable[1]); and ii) the distance of each element of the activity array (variable <Kohonen.variable>`[1])) from the one with the weights most similar to the input array (variable <Kohonen.variable>`[0])) using distance_measure.

owner

ComponentMechanism to which the Function belongs.

prefs

PreferenceSet or specification dict : default Function.classPreferences – the PreferenceSet for the Function (see prefs for details).

function(variable=None, execution_id=None, params=None, context=None)
Parameters:
  • variable (array or List[1d array, 1d array, 2d array] : default class_defaults.variable) – input pattern, array of activation values, and matrix used to calculate the weights changes.
  • params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
Returns:

weight change matrix – matrix of weight changes scaled by difference of the current weights from the input pattern in variable[0] and the distance of each element from the one with the weights most similar to that input pattern.

Return type:

2d array

class psyneulink.core.components.functions.learningfunctions.Hebbian(default_variable=None, learning_rate=None, params=None, name=None, prefs=None)

Calculate a matrix of weight changes using the Hebbian (correlational) learning rule.

function calculates a matrix of weight changes from a 1d array of activity values in variable using the Hebbian learning rule:

\[\Delta w_{ij} = learning\_rate * a_ia_j\ if\ i \neq j,\ else\ 0\]

where \(a_i\) and \(a_j\) are elements of variable.

Parameters:
  • variable (List[number] or 1d array : default class_defaults.variable) – specifies the activation values, the pair-wise products of which are used to generate the a weight change matrix.
  • learning_rate (scalar or list, 1d or 2d array, or np.matrix of numeric values: default default_learning_rate) – specifies the learning rate used by the function; supersedes any specification for the Process and/or System to which the function’s owner belongs (see learning_rate for details).
  • params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
  • owner (Component) – component to which to assign the Function.
  • name (str : default see name) – specifies the name of the Function.
  • prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).
variable

1d array – activation values, the pair-wise products of which are used to generate the weight change matrix returned by the function.

learning_rate

float, 1d or 2d array – used by the function to scale the weight change matrix returned by the function. If specified, it supersedes any learning_rate specified for the Process and/or System to which the function’s owner belongs. If it is a scalar, it is multiplied by the weight change matrix; if it is a 1d array, it is multiplied Hadamard (elementwise) by the variable <Hebbian.variable>` before calculating the weight change matrix; if it is a 2d array, it is multiplied Hadamard (elementwise) by the weight change matrix; if it is None, then the learning_rate specified for the Process to which the owner belongs is used; and, if that is None, then the learning_rate for the System to which it belongs is used. If all are None, then the default_learning_rate is used.

default_learning_rate

float – the value used for the learning_rate if it is not otherwise specified.

function

function – calculates the pairwise product of all elements in the variable, and then scales that by the learning_rate to generate the weight change matrix returned by the function.

owner

ComponentMechanism to which the Function belongs.

prefs

PreferenceSet or specification dict : default Function.classPreferences – the PreferenceSet for the Function (see prefs for details).

function(variable=None, execution_id=None, params=None, context=None)
Parameters:
  • variable (List[number] or 1d array : default class_defaults.variable) – array of activity values, the pairwise products of which are used to generate a weight change matrix.
  • params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
Returns:

weight change matrix – matrix of weight changes generated by the Hebbian Learning rule, with all diagonal elements = 0 (i.e., hollow matix).

Return type:

2d array

class psyneulink.core.components.functions.learningfunctions.ContrastiveHebbian(default_variable=None, learning_rate=None, params=None, name=None, prefs=None)

Calculate a matrix of weight changes using the Contrastive Hebbian learning rule.

function calculates a matrix of weight changes from a 1d array of activity values in variable using the ContrastiveHebbian learning rule:

\[\Delta w_{ij} = learning\_rate * (a_i^+a_j^+ - a_i^-a_j^-) \ if\ i \neq j,\ else\ 0\]

where \(a_i^+\) and \(a_j^+\) are the activites of elements of variable in the plus_phase of execution, and \(a_i^-\) and \(a_j^-\) are the activities of those elements in the minus phase of execution.

Parameters:
  • variable (List[number] or 1d array : default class_defaults.variable) – specifies the activation values, the pair-wise products of which are used to generate the a weight change matrix.
  • learning_rate (scalar or list, 1d or 2d array, or np.matrix of numeric values: default default_learning_rate) – specifies the learning rate used by the function; supersedes any specification for the Process and/or System to which the function’s owner belongs (see learning_rate for details).
  • params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
  • owner (Component) – component to which to assign the Function.
  • name (str : default see name) – specifies the name of the Function.
  • prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).
variable

1d array – activation values, the pair-wise products of which are used to generate the weight change matrix returned by the function.

learning_rate

float, 1d or 2d array – used by the function to scale the weight change matrix returned by the function. If specified, it supersedes any learning_rate specified for the Process and/or System to which the function’s owner belongs. If it is a scalar, it is multiplied by the weight change matrix; if it is a 1d array, it is multiplied Hadamard (elementwise) by the variable <ContrastiveHebbian.variable>` before calculating the weight change matrix; if it is a 2d array, it is multiplied Hadamard (elementwise) by the weight change matrix; if it is None, then the learning_rate specified for the Process to which the owner belongs is used; and, if that is None, then the learning_rate for the System to which it belongs is used. If all are None, then the default_learning_rate is used.

default_learning_rate

float – the value used for the learning_rate if it is not otherwise specified.

function

function – calculates the pairwise product of all elements in the variable, and then scales that by the learning_rate to generate the weight change matrix returned by the function.

owner

ComponentMechanism to which the Function belongs.

prefs

PreferenceSet or specification dict : default Function.classPreferences – the PreferenceSet for the Function (see prefs for details).

function(variable=None, execution_id=None, params=None, context=None)
Parameters:
  • variable (List[number] or 1d np.array : default class_defaults.variable) – array of activity values, the pairwise products of which are used to generate a weight change matrix.
  • params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
Returns:

weight change matrix – matrix of weight changes generated by the ContrastiveHeabbian learning rule, with all diagonal elements = 0 (i.e., hollow matix).

Return type:

2d array

class psyneulink.core.components.functions.learningfunctions.Reinforcement(default_variable=None, learning_rate=None, params=None, name=None, prefs=None)

Calculate error term for a single item in an input array, scaled by the learning_rate.

function takes an array (activation_output) with only one non-zero value, and returns an array of the same length with the one non-zero value replaced by \(\Delta w\) – the error_signal scaled by the learning_rate:

\[\Delta w_i = learning\_rate * error\_signal_i\ if\ activation\_output_i \neq 0,\ otherwise\ 0\]

The non-zero item in activation_output can be thought of as the predicted likelihood of a stimulus or value of an action, and the error_signal as the error in the prediction for that value.

Note

To preserve compatibility with other LearningFunctions:

  • the variable argument of both the constructor and calls to function must have three items, although only the 2nd and 3rd items are used; these are referenced by the activation_output and error_signal attributes, respectively (the first item is used by other LearningFunctions as their activation_input attribute).
  • function returns two copies of the error array (the first is a “place-marker”, where a matrix of weights changes is often returned).
Parameters:
  • default_variable (List or 2d array : default class_defaults.variable) –

    template for the three items provided as the variable in the call to the function (in order):

  • learning_rate (float : default default_learning_rate) – supersedes any specification for the Process and/or System to which the function’s owner belongs (see learning_rate for details).
  • params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
  • owner (Component) – component to which to assign the Function.
  • name (str : default see name) – specifies the name of the Function.
  • prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).
variable

2d array

specifies three values used as input to the function:

activation_input

1d array – first item of variable; this is not used (it is implemented for compatibility with other LearningFunctions).

activation_output

1d array – second item of variable; contains a single “prediction” or “action” value as one of its elements, the others of which are zero.

error_signal

1d array – third item of variable; contains a single scalar value, specifying the error associated with the non-zero item in activation_output.

learning_rate

float – the learning rate used by the function. If specified, it supersedes any learning_rate specified for the Process and/or System to which the function’s owner belongs. If it is None, then the learning_rate specified for the Process to which the owner belongs is used; and, if that is None, then the learning_rate for the System to which it belongs is used. If all are None, then the default_learning_rate is used.

default_learning_rate

float – the value used for the learning_rate if it is not otherwise specified.

function

function – the function that computes the weight change matrix, and returns that along with the error_signal received.

owner

ComponentMechanism to which the Function belongs.

prefs

PreferenceSet or specification dict : default Function.classPreferences – the PreferenceSet for the Function (see prefs for details).

function(variable=None, execution_id=None, params=None, context=None, **kwargs)
Parameters:
  • variable (List or 2d np.array [length 3 in axis 0] : default class_defaults.variable) –

    must have three items that are (in order):

    (see note above).

  • params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
Returns:

error array – Both 1d arrays are the same, with a single non-zero error term (see note above).

Return type:

List[1d array, 1d array]

class psyneulink.core.components.functions.learningfunctions.BackPropagation(default_variable=None, activation_derivative_fct=Logistic().derivative, learning_rate=None, params=None, name=None, prefs=None)

Calculate and return a matrix of weight changes and weighted error signal from arrays of inputs, outputs and error terms.

function calculates a matrix of weight changes using the backpropagation (Generalized Delta Rule) learning algorithm, computed as:

\[weight\_change\_matrix = learning\_rate * activation\_input * \frac{\delta E}{\delta W}\]

where

\[\frac{\delta E}{\delta W} = \frac{\delta E}{\delta A} * \frac{\delta A}{\delta W}\]

is the derivative of the error_signal with respect to the weights, in which:

\(\frac{\delta E}{\delta A} = error\_matrix \bullet error\_signal\)

is the derivative of the error with respect to activation_output (i.e., the weighted contribution to the error_signal of each unit that receives activity from the weight matrix being learned), and

\(\frac{\delta A}{\delta W} = activation\_derivative\_fct(input=activation\_input,output=activation\_output)\)

is the derivative of the activation function responsible for generating activation_output at the point that generates each of its entries.

The values of activation_input, activation_output and error_signal are specified as items of the variable both in the constructor for the BackPropagation Function, and in calls to its function. Although error_matrix is not specified in the constructor, it is required as an argument of the function; it is assumed that it’s value is determined in context at the time of execution (e.g., by a Learning Mechanism that uses the BackPropagation LearningFunction).

The BackPropagation function returns the weight_change_matrix as well as the error_signal it receives weighted by the contribution made by each element of activation_output as a function of the error_matrix \(\frac{\delta E}{\delta W}\).

Parameters:
  • variable (List or 2d array [length 3 in axis 0] : default class_defaults.variable) – specifies a template for the three items provided as the variable in the call to the function (in order): activation_input (1d array), activation_output (1d array), error_signal (1d array).
  • activation_derivative_fct (Function or function) – specifies the derivative for the function of the Mechanism that generates activation_output.
  • learning_rate (float : default default_learning_rate) – supersedes any specification for the Process and/or System to which the function’s owner belongs (see learning_rate for details).
  • params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
  • owner (Component) – component to which to assign the Function.
  • name (str : default see name) – specifies the name of the Function.
  • prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).
variable

2d array – contains the three values used as input to the function: activation_input, activation_output, and error_signal.

activation_input

1d array – the input to the matrix being modified; same as 1st item of variable.

activation_output

1d array – the output of the function for which the matrix being modified provides the input; same as 2nd item of variable.

activation_derivative_fct

Function or function – the derivative for the function of the Mechanism that generates activation_output.

error_signal

1d array – the error signal for the next matrix (layer above) in the learning sequence, or the error computed from the target (training signal) and the output of the last Mechanism in the sequence; same as 3rd item of variable.

error_matrix

2d array or ParameterState – matrix, the input of which is activation_output and the output of which is used to calculate the error_signal; if it is a ParameterState, it refers to the MATRIX parameterState of the MappingProjection being learned.

learning_rate

float – the learning rate used by the function. If specified, it supersedes any learning_rate specified for the process and/or system to which the function’s owner belongs. If it is None, then the learning_rate specified for the process to which the owner belongs is used; and, if that is None, then the learning_rate for the system to which it belongs is used. If all are None, then the default_learning_rate is used.

default_learning_rate

float – the value used for the learning_rate if it is not otherwise specified.

owner

ComponentMechanism to which the Function belongs.

prefs

PreferenceSet or specification dict : default Function.classPreferences – the PreferenceSet for the Function (see prefs for details).

function(variable=None, execution_id=None, error_matrix=None, params=None, context=None, **kwargs)

Note

Both variable and error_matrix must be specified for the function to execute.

Parameters:
Returns:

weight change matrix, weighted error signal – the modifications to make to the matrix, error_signal weighted by the contribution made by each element of activation_output as a function of error_matrix.

Return type:

List[2d array, 1d array]

class psyneulink.core.components.functions.learningfunctions.TDLearning(default_variable=None, learning_rate=None, params=None, owner=None, prefs=None)

Implement temporal difference learning using the Reinforcement Function (see Reinforcement for class details).