LearningFunctions¶
Functions that parameterize a function.
- class psyneulink.core.components.functions.learningfunctions.LearningFunction(default_variable, params, owner=None, name=None, prefs=None, context=None, **kwargs)¶
Abstract class of Function used for learning.
- variable¶
most LearningFunctions take a list or 2d array that must contain three items:
the input to the parameter being modified (variable[LEARNING_ACTIVATION_INPUT]);
the output of the parameter being modified (variable[LEARNING_ACTIVATION_OUTPUT]);
the error associated with the output (variable[LEARNING_ERROR_OUTPUT]).
However, the exact specification depends on the funtion’s type.
- Type
list or array
- default_learning_rate¶
the value used for the function’s
learning_rate
parameter if none of the following are specified: thelearning_rate
for the Learning Mechanism to which the function has been assigned, thelearning_rate
for anyProcess
or thelearning_rate
for anySystem
to which that LearningMechanism belongs. The exact form of the value (i.e., whether it is a scalar or array) depends on the function’s type.- Type
numeric
- learning_rate¶
generally used to multiply the result of the function before it is returned; however, both the form of the value (i.e., whether it is a scalar or array) and how it is used depend on the function’s type.
- Type
numeric
- Returns
The number of items returned and their format depend on the function’s type.
Most return an array (used as the
learning_signal
by a Learning Mechanism), and some also return a similarly formatted array containing either the error received (in the third item of thevariable
) or a version of it modified by the function.
- class psyneulink.core.components.functions.learningfunctions.Kohonen(default_variable=None, learning_rate=None, distance_function=None, params=None, owner=None, prefs=None)¶
Calculates a matrix of weight changes using the Kohenen learning rule.
The Kohonen learning rule is used to implement a Kohonen netowrk, which is an instance of the more general category of self organizing map (SOM), that modifies the weights to each element in the network in proportion to its difference from the current input pattern and the distance of that element from the one with the weights most similar to the current input pattern.
function
calculates and returns a matrix of weight changes from an array of activity values (invariable
[1]) and a weight matrix that generated them (invariable
[2]) using the Kohonen learning rule:\[learning\_rate * distance_j * variable[0]-w_j\]where \(distance_j\) is the distance of the jth element of
variable
[1] from the element with the weights most similar to activity array invariable
[1], and \(w_j\) is the column of the matrix invariable
[2] that corresponds to the jth element of the activity array invariable
[1].- Parameters
variable (List[array(float64), array(float64), 2d array[[float64]]] : default class_defaults.variable) – input pattern, array of activation values, and matrix used to calculate the weights changes.
learning_rate (scalar or list, 1d or 2d array, or np.matrix of numeric values: default default_learning_rate) – specifies the learning rate used by the
function
; supersedes any specification for theProcess
and/orSystem
to which the function’sowner
belongs (seelearning_rate
for details).distance_measure (GAUSSIAN, LINEAR, EXPONENTIAL, SINUSOID or function) – specifies the method used to calculate the distance of each element in
variable
[2] from the one with the greatest value.params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
owner (Component) – component to which to assign the Function.
name (str : default see
name
) – specifies the name of the Function.prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the
PreferenceSet
for the Function (seeprefs
for details).
- variable¶
input pattern, array of activation values, and weight matrix used to generate the weight change matrix returned by
function
.- Type
List[array(float64), array(float64), 2d array[[float64]]]
- learning_rate¶
used by the
function
to scale the weight change matrix returned by thefunction
. If specified, it supersedes any learning_rate specified for theProcess
and/orSystem
to which the function’sowner
belongs. If it is a scalar, it is multiplied by the weight change matrix; if it is a 1d array, it is multiplied Hadamard (elementwise) by thevariable
<Kohonen.variable>` before calculating the weight change matrix; if it is a 2d array, it is multiplied Hadamard (elementwise) by the weight change matrix; if it isNone
, then thelearning_rate
specified for the Process to which theowner
belongs is used; and, if that isNone
, then thelearning_rate
for the System to which it belongs is used. If all areNone
, then thedefault_learning_rate
is used.- Type
float, 1d or 2d array
- default_learning_rate¶
the value used for the
learning_rate
if it is not otherwise specified.- Type
float
- function¶
calculates a matrix of weight changes from: i) the difference between an input pattern (variable <Kohonen.variable>`[0]) and the weights in a weigh matrix (
variable
[2]) to each element of an activity array (variable
[1]); and ii) the distance of each element of the activity array (variable <Kohonen.variable>`[1])) from the one with the weights most similar to the input array (variable <Kohonen.variable>`[0])) usingdistance_measure
.- Type
function
- prefs¶
the
PreferenceSet
for the Function (seeprefs
for details).- Type
PreferenceSet or specification dict : default Function.classPreferences
- _validate_variable(variable, context=None)¶
Validate variable and return validated variable
Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:
VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]
- Perform top-level type validation of variable against the self.class_defaults.variable;
if the type is OK, the value is returned (which should be used by the function)
This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is
True
(which it is by default)- IMPLEMENTATION NOTES:
future versions should add hierarchical/recursive content (e.g., range) checking
add request/target pattern?? (as per _validate_params) and return validated variable?
- Parameters
variable – (anything other than a dictionary) - variable to be validated:
context – (str)
- Return variable
validated variable
- _validate_params(request_set, target_set=None, context=None)¶
Validate learning_rate
- _function(variable=None, context=None, params=None)¶
- Parameters
variable (array or List[1d array, 1d array, 2d array] : default class_defaults.variable) – input pattern, array of activation values, and matrix used to calculate the weights changes.
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
- Returns
weight change matrix – matrix of weight changes scaled by difference of the current weights from the input pattern in
variable
[0] and the distance of each element from the one with the weights most similar to that input pattern.- Return type
2d array
- class psyneulink.core.components.functions.learningfunctions.Hebbian(default_variable=None, learning_rate=None, params=None, owner=None, prefs=None)¶
Calculate a matrix of weight changes using the Hebbian (correlational) learning rule.
function
calculates a matrix of weight changes from a 1d array of activity values invariable
using the Hebbian learning rule:\[\Delta w_{ij} = learning\_rate * a_ia_j\ if\ i \neq j,\ else\ 0\]where \(a_i\) and \(a_j\) are elements of
variable
.- Parameters
variable (List[number] or 1d array : default class_defaults.variable) – specifies the activation values, the pair-wise products of which are used to generate the a weight change matrix.
learning_rate (scalar or list, 1d or 2d array, or np.matrix of numeric values: default default_learning_rate) – specifies the learning rate used by the
function
; supersedes any specification for theProcess
and/orSystem
to which the function’sowner
belongs (seelearning_rate
for details).params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
owner (Component) – component to which to assign the Function.
name (str : default see
name
) – specifies the name of the Function.prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the
PreferenceSet
for the Function (seeprefs
for details).
- variable¶
activation values, the pair-wise products of which are used to generate the weight change matrix returned by the
function
.- Type
1d array
- learning_rate¶
used by the
function
to scale the weight change matrix returned by thefunction
. If specified, it supersedes any learning_rate specified for theProcess
and/orSystem
to which the function’sowner
belongs. If it is a scalar, it is multiplied by the weight change matrix; if it is a 1d array, it is multiplied Hadamard (elementwise) by thevariable
<Hebbian.variable>` before calculating the weight change matrix; if it is a 2d array, it is multiplied Hadamard (elementwise) by the weight change matrix; if it isNone
, then thelearning_rate
specified for the Process to which theowner
belongs is used; and, if that isNone
, then thelearning_rate
for the System to which it belongs is used. If all areNone
, then thedefault_learning_rate
is used.- Type
float, 1d or 2d array
- default_learning_rate¶
the value used for the
learning_rate
if it is not otherwise specified.- Type
float
- function¶
calculates the pairwise product of all elements in the
variable
, and then scales that by thelearning_rate
to generate the weight change matrix returned by the function.- Type
function
- prefs¶
the
PreferenceSet
for the Function (seeprefs
for details).- Type
PreferenceSet or specification dict : default Function.classPreferences
- _validate_variable(variable, context=None)¶
Validate variable and return validated variable
Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:
VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]
- Perform top-level type validation of variable against the self.class_defaults.variable;
if the type is OK, the value is returned (which should be used by the function)
This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is
True
(which it is by default)- IMPLEMENTATION NOTES:
future versions should add hierarchical/recursive content (e.g., range) checking
add request/target pattern?? (as per _validate_params) and return validated variable?
- Parameters
variable – (anything other than a dictionary) - variable to be validated:
context – (str)
- Return variable
validated variable
- _validate_params(request_set, target_set=None, context=None)¶
Validate learning_rate
- _function(variable=None, context=None, params=None)¶
- Parameters
variable (List[number] or 1d array : default class_defaults.variable) – array of activity values, the pairwise products of which are used to generate a weight change matrix.
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
- Returns
weight change matrix – matrix of weight changes generated by the Hebbian Learning rule, with all diagonal elements = 0 (i.e., hollow matix).
- Return type
2d array
- class psyneulink.core.components.functions.learningfunctions.ContrastiveHebbian(default_variable=None, learning_rate=None, params=None, owner=None, prefs=None)¶
Calculate a matrix of weight changes using the Contrastive Hebbian learning rule.
function
calculates a matrix of weight changes from a 1d array of activity values invariable
using the ContrastiveHebbian learning rule:\[\Delta w_{ij} = learning\_rate * (a_i^+a_j^+ - a_i^-a_j^-) \ if\ i \neq j,\ else\ 0\]where \(a_i^+\) and \(a_j^+\) are the activites of elements of
variable
in the plus_phase of execution, and \(a_i^-\) and \(a_j^-\) are the activities of those elements in the minus phase of execution.- Parameters
variable (List[number] or 1d array : default class_defaults.variable) – specifies the activation values, the pair-wise products of which are used to generate the a weight change matrix.
learning_rate (scalar or list, 1d or 2d array, or np.matrix of numeric values: default default_learning_rate) – specifies the learning rate used by the
function
; supersedes any specification for theProcess
and/orSystem
to which the function’sowner
belongs (seelearning_rate
for details).params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
owner (Component) – component to which to assign the Function.
name (str : default see
name
) – specifies the name of the Function.prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the
PreferenceSet
for the Function (seeprefs
for details).
- variable¶
activation values, the pair-wise products of which are used to generate the weight change matrix returned by the
function
.- Type
1d array
- learning_rate¶
used by the
function
to scale the weight change matrix returned by thefunction
. If specified, it supersedes any learning_rate specified for theProcess
and/orSystem
to which the function’sowner
belongs. If it is a scalar, it is multiplied by the weight change matrix; if it is a 1d array, it is multiplied Hadamard (elementwise) by thevariable
<ContrastiveHebbian.variable>` before calculating the weight change matrix; if it is a 2d array, it is multiplied Hadamard (elementwise) by the weight change matrix; if it isNone
, then thelearning_rate
specified for the Process to which theowner
belongs is used; and, if that isNone
, then thelearning_rate
for the System to which it belongs is used. If all areNone
, then thedefault_learning_rate
is used.- Type
float, 1d or 2d array
- default_learning_rate¶
the value used for the
learning_rate
if it is not otherwise specified.- Type
float
- function¶
calculates the pairwise product of all elements in the
variable
, and then scales that by thelearning_rate
to generate the weight change matrix returned by the function.- Type
function
- prefs¶
the
PreferenceSet
for the Function (seeprefs
for details).- Type
PreferenceSet or specification dict : default Function.classPreferences
- _validate_variable(variable, context=None)¶
Validate variable and return validated variable
Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:
VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]
- Perform top-level type validation of variable against the self.class_defaults.variable;
if the type is OK, the value is returned (which should be used by the function)
This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is
True
(which it is by default)- IMPLEMENTATION NOTES:
future versions should add hierarchical/recursive content (e.g., range) checking
add request/target pattern?? (as per _validate_params) and return validated variable?
- Parameters
variable – (anything other than a dictionary) - variable to be validated:
context – (str)
- Return variable
validated variable
- _validate_params(request_set, target_set=None, context=None)¶
Validate learning_rate
- _function(variable=None, context=None, params=None)¶
- Parameters
variable (List[number] or 1d np.array : default class_defaults.variable) – array of activity values, the pairwise products of which are used to generate a weight change matrix.
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
- Returns
weight change matrix – matrix of weight changes generated by the ContrastiveHeabbian learning rule, with all diagonal elements = 0 (i.e., hollow matix).
- Return type
2d array
- class psyneulink.core.components.functions.learningfunctions.Reinforcement(default_variable=None, learning_rate=None, params=None, owner=None, prefs=None)¶
Calculate error term for a single item in an input array, scaled by the learning_rate.
function
takes an array (activation_output
) with only one non-zero value, and returns an array of the same length with the one non-zero value replaced by \(\Delta w\) – theerror_signal
scaled by thelearning_rate
:\[\Delta w_i = learning\_rate * error\_signal_i\ if\ activation\_output_i \neq 0,\ otherwise\ 0\]The non-zero item in
activation_output
can be thought of as the predicted likelihood of a stimulus or value of an action, and theerror_signal
as the error in the prediction for that value.To preserve compatibility with other LearningFunctions:
the variable argument of both the constructor and calls to
function
must have three items, although only the 2nd and 3rd items are used; these are referenced by theactivation_output
anderror_signal
attributes, respectively (the first item is used by other LearningFunctions as theiractivation_input
attribute).
function
returns two copies of the error array (the first is a “place-marker”, where a matrix of weights changes is often returned).
- Parameters
default_variable (List or 2d array : default class_defaults.variable) –
template for the three items provided as the variable in the call to the
function
(in order):activation_input
(1d array) (not used);activation_output
(1d array with only one non-zero value);error_signal
(1d array with a single scalar element).
learning_rate (float : default default_learning_rate) – supersedes any specification for the
Process
and/orSystem
to which the function’sowner
belongs (seelearning_rate
for details).params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
owner (Component) – component to which to assign the Function.
name (str : default see
name
) – specifies the name of the Function.prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the
PreferenceSet
for the Function (seeprefs
for details).
- activation_input¶
first item of
variable
; this is not used (it is implemented for compatibility with other LearningFunctions).- Type
1d array
- activation_output¶
second item of
variable
; contains a single “prediction” or “action” value as one of its elements, the others of which are zero.- Type
1d array
- error_signal¶
third item of
variable
; contains a single scalar value, specifying the error associated with the non-zero item inactivation_output
.- Type
1d array
- learning_rate¶
the learning rate used by the function. If specified, it supersedes any learning_rate specified for the
Process
and/orSystem
to which the function’sowner
belongs. If it isNone
, then thelearning_rate
specified for the Process to which theowner
belongs is used; and, if that isNone
, then thelearning_rate
for the System to which it belongs is used. If all areNone
, then thedefault_learning_rate
is used.- Type
float
- default_learning_rate¶
the value used for the
learning_rate
if it is not otherwise specified.- Type
float
- function¶
the function that computes the weight change matrix, and returns that along with the
error_signal
received.- Type
function
- prefs¶
the
PreferenceSet
for the Function (seeprefs
for details).- Type
PreferenceSet or specification dict : default Function.classPreferences
- _validate_variable(variable, context=None)¶
Validate variable and return validated variable
Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:
VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]
- Perform top-level type validation of variable against the self.class_defaults.variable;
if the type is OK, the value is returned (which should be used by the function)
This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is
True
(which it is by default)- IMPLEMENTATION NOTES:
future versions should add hierarchical/recursive content (e.g., range) checking
add request/target pattern?? (as per _validate_params) and return validated variable?
- Parameters
variable – (anything other than a dictionary) - variable to be validated:
context – (str)
- Return variable
validated variable
- _validate_params(request_set, target_set=None, context=None)¶
Validate learning_rate
- _function(variable=None, context=None, params=None, **kwargs)¶
- Parameters
variable (List or 2d np.array [length 3 in axis 0] : default class_defaults.variable) –
must have three items that are (in order):
activation_input
(not used);activation_output
(1d array with only one non-zero value);error_signal
(1d array with a single scalar element);
(see note above).
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
- Returns
error array – Both 1d arrays are the same, with a single non-zero error term (see note above).
- Return type
List[1d array, 1d array]
- class psyneulink.core.components.functions.learningfunctions.BayesGLM(default_variable=None, mu_0=None, sigma_0=None, gamma_shape_0=None, gamma_size_0=None, params=None, owner=None, seed=None, prefs=None)¶
Use Bayesian linear regression to find means and distributions of weights that predict dependent variable(s).
function
uses a normal linear model:\[dependent\ variable(s) = predictor(s)\ \Theta + \epsilon,\]with predictor(s) in
variable
[0] and dependent variable(s) invariable
[1], and a normal-gamma prior distribution of weights (\(\Theta\)), to update the weight distribution parametersmu_n
,Lambda_n
,gamma_shape_n
, andgamma_size_n
, and returns an array of prediction weights sampled from the multivariate normal-gamma distribution [based on Falk Lieder’s BayesianGLM.m, adapted for Python by Yotam Sagiv and for PsyNeuLink by Jon Cohen; useful reference:Hint
The mu_0 or sigma_0 arguments of the consructor can be used in place of default_variable to define the size of the predictors array and, correspondingly, the weights array returned by the function (see Parameters below).
- Parameters
default_variable (3d array : default None) – first item of axis 0 must be a 2d array with one or more 1d arrays to use as predictor vectors, one for each sample to be fit; second item must be a 2d array of equal length to the first item, with a 1d array containing a scalar that is the dependent (to-be-predicted) value for the corresponding sample in the first item. If
None
is specified, but either mu_0 or sigma_0 is specified, then the they are used to determine the shape of `variable <BayesGLM.variable>`. If neither **mu_0 nor sigma_0 are specified, then the shape ofvariable
is determined by the first call to itsfunction
, as aremu_prior
,sigma_prior
,gamma_shape_prior
andgamma_size_prior
.mu_0 (int, float or 1d array : default 0) – specifies initial value of
mu_prior
(the prior for the mean of the distribution for the prediction weights returned by the function). If a scalar is specified, the same value will be used for all elements ofmu_prior
; if it is an array, it must be the same length as the predictor array(s) in axis 0 of default_variable. If default_variable is not specified, the specification for mu_0 is used to determine the shape ofvariable
andsigma_prior
.sigma_0 (int, float or 1d array : default 0) – specifies initial value of
sigma_prior
(the prior for the variance of the distribution for the prediction weights returned by the function). If a scalar is specified, the same value will be used for all elements ofLambda_prior
; if it is an array, it must be the same length as the predictor array(s) in axis 0 of default_variable. If neither default_variable nor mu_0 is specified, the specification for sigma_0 is used to determine their shapes.gamma_shape_0 (int or float : default 1) – specifies the shape of the gamma distribution from which samples of the weights are drawn (see documentation for numpy.random.gamma.
gamma_size_0 (int or float : default 1) –
specifies the size of the gamma distribution from which samples of the weights are drawn (see documentation for numpy.random.gamma.
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
owner (Component) – component to which to assign the Function.
name (str : default see
name
) – specifies the name of the Function.prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the
PreferenceSet
for the Function (seeprefs
for details).
- variable¶
samples used to update parameters of prediction weight distributions. variable[0] is a 2d array of predictor vectors, all of the same length; variable[1] is a 2d array of scalar dependent variables, one for each predictor vector.
- Type
3d array
- mu_0¶
determines the initial prior(s) for the means of the distributions of the prediction weights; if it is a scalar, that value is assigned as the priors for all means.
- Type
int, float or 2d array
- mu_prior¶
current priors for the means of the distributions of the predictions weights.
- Type
2d array
- mu_n¶
current means for the distributions of the prediction weights.
- Type
2d array
- sigma_0¶
value used to determine the initial prior(s) for the variances of the distributions of the prediction weights; if it is a scalar, that value is assigned as the priors for all variances.
- Type
int, float or 2d array
- Lambda_prior¶
current priors for the variances of the distributions of the predictions weights.
- Type
2d array
- Lambda_n¶
current variances for the distributions of the prediction weights.
- Type
2d array
- gamma_shape_0¶
determines the initial value used for the shape parameter of the gamma distribution used to sample the prediction weights.
- Type
int or float
- gamma_shape_prior¶
current prior for the shape parameter of the gamma distribution used to sample the prediction weights.
- Type
int or float
- gamma_shape_n¶
current value of the shape parameter of the gamma distribution used to sample the prediction weights.
- Type
int or float
- gamma_size_0¶
determines the initial value used for the size parameter of the gamma distribution used to sample the prediction weights.
- Type
int or float
- gamma_size_prior¶
current prior for the size parameter of the gamma distribution used to sample the prediction weights.
- Type
int or float
- gamma_size_n¶
current value of the size parameter of the gamma distribution used to sample the prediction weights.
- Type
2d array with single scalar value
- random_state¶
private pseudorandom number generator
- Type
numpy.RandomState
- weights_sample¶
last sample of prediction weights drawn in call to
sample_weights
and returned byfunction
.- Type
1d array
- prefs¶
the
PreferenceSet
for the Function (seeprefs
for details).- Type
PreferenceSet or specification dict : default Function.classPreferences
- _handle_default_variable(default_variable=None, size=None)¶
Finds whether default_variable can be determined using default_variable and size arguments.
- Returns
a default variable if possible
None otherwise
- initialize_priors()¶
Set the prior parameters (
mu_prior
,Lamba_prior
,gamma_shape_prior
, andgamma_size_prior
) to their initial (_0) values, and assign current (_n) values to the priors
- reset(default_variable=None, context=None)¶
If the component’s execute method involves execution of an
IntegratorFunction
Function, this method effectively begins the function’s accumulation over again at the specified value, and may update related values on the component, depending on the component type. Otherwise, it simply reassigns the Component’s value based on its default_variable.
- _function(variable=None, context=None, params=None)¶
- Parameters
variable (2d or 3d array : default class_defaults.variable) –
- If it is a 2d array, the first item must be a 1d array of scalar predictors,
and the second must be a 1d array containing the dependent variable to be predicted by the predictors.
- If it is a 3d array, the first item in the outermost dimension must be a 2d array containing one or more
1d arrays of scalar predictors, and the second item must be a 2d array containing 1d arrays each of which contains a scalar dependent variable for the corresponding predictor vector.
params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
- Returns
sample weights – array of weights drawn from updated weight distributions.
- Return type
1d array
- sample_weights(gamma_shape_n, gamma_size_n, mu_n, Lambda_n, context)¶
Draw a sample of prediction weights from the distributions parameterized by
mu_n
,Lambda_n
,gamma_shape_n
, andgamma_size_n
.
- class psyneulink.core.components.functions.learningfunctions.BackPropagation(default_variable=None, activation_derivative_fct=Logistic().derivative, learning_rate=None, params=None, name=None, prefs=None)¶
Calculate and return a matrix of weight changes and weighted error signal from arrays of inputs, outputs and error terms.
function
calculates a matrix of weight changes using the backpropagation (Generalized Delta Rule) learning algorithm, computed as:\[weight\_change\_matrix = learning\_rate * activation\_input * \frac{\delta E}{\delta W}\]where
\[\frac{\delta E}{\delta W} = \frac{\delta E}{\delta A} * \frac{\delta A}{\delta W}\]is the derivative of the error_signal with respect to the weights, in which:
\(\frac{\delta E}{\delta A} = error\_matrix \bullet error\_signal\)
is the derivative of the error with respect to activation_output (i.e., the weighted contribution to the error_signal of each unit that receives activity from the weight matrix being learned), and
\(\frac{\delta A}{\delta W} = activation\_derivative\_fct(input=activation\_input,output=activation\_output)\)
is the derivative of the activation function responsible for generating activation_output at the point that generates each of its entries.
The values of
activation_input
,activation_output
anderror_signal
are specified as items of thevariable
both in the constructor for the BackPropagation Function, and in calls to itsfunction
. Althougherror_matrix
is not specified in the constructor, it is required as an argument of thefunction
; it is assumed that it’s value is determined in context at the time of execution (e.g., by a Learning Mechanism that uses the BackPropagation LearningFunction).The BackPropagation
function
returns the weight_change_matrix as well as theerror_signal
it receives weighted by the contribution made by each element ofactivation_output
as a function of theerror_matrix
\(\frac{\delta E}{\delta W}\).- Parameters
variable (List or 2d array [length 3 in axis 0] : default class_defaults.variable) – specifies a template for the three items provided as the variable in the call to the
function
(in order):activation_input
(1d array),activation_output
(1d array),error_signal
(1d array).activation_derivative_fct (Function or function) – specifies the derivative for the function of the Mechanism that generates
activation_output
.learning_rate (float : default default_learning_rate) – supersedes any specification for the
Process
and/orSystem
to which the function’sowner
belongs (seelearning_rate
for details).params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
owner (Component) – component to which to assign the Function.
name (str : default see
name
) – specifies the name of the Function.prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the
PreferenceSet
for the Function (seeprefs
for details).
- variable¶
contains the three values used as input to the
function
:activation_input
,activation_output
, anderror_signal
.- Type
2d array
- activation_input¶
the input to the matrix being modified; same as 1st item of
variable
.- Type
1d array
- activation_output¶
the output of the function for which the matrix being modified provides the input; same as 2nd item of
variable
.- Type
1d array
- activation_derivative_fct¶
the derivative for the function of the Mechanism that generates
activation_output
.- Type
Function or function
- error_signal¶
the error signal for the next matrix (layer above) in the learning pathway, or the error computed from the target (training signal) and the output of the last Mechanism in the sequence; same as 3rd item of
variable
.- Type
1d array
- error_matrix¶
matrix, the input of which is
activation_output
and the output of which is used to calculate theerror_signal
; if it is a ParameterPort, it refers to the MATRIX parameterPort of the MappingProjection being learned.- Type
2d array or ParameterPort
- learning_rate¶
the learning rate used by the function. If specified, it supersedes any learning_rate specified for the
process
and/orsystem
to which the function’sowner
belongs. If it isNone
, then the learning_rate specified for the process to which theowner
belongs is used; and, if that isNone
, then the learning_rate for the system to which it belongs is used. If all areNone
, then thedefault_learning_rate
is used.- Type
float
- default_learning_rate¶
the value used for the
learning_rate
if it is not otherwise specified.- Type
float
- loss_function¶
the operation to apply to the error signal before computing weight changes.
- Type
string : default ‘MSE’
- prefs¶
the
PreferenceSet
for the Function (seeprefs
for details).- Type
PreferenceSet or specification dict : default Function.classPreferences
- _validate_variable(variable, context=None)¶
Validate variable and return validated variable
Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:
VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]
- Perform top-level type validation of variable against the self.class_defaults.variable;
if the type is OK, the value is returned (which should be used by the function)
This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is
True
(which it is by default)- IMPLEMENTATION NOTES:
future versions should add hierarchical/recursive content (e.g., range) checking
add request/target pattern?? (as per _validate_params) and return validated variable?
- Parameters
variable – (anything other than a dictionary) - variable to be validated:
context – (str)
- Return variable
validated variable
- _validate_params(request_set, target_set=None, context=None)¶
Validate learning_rate and error_matrix params
error_matrix
argument must be one of the following2d list, np.ndarray or np.matrix
ParameterPort for one of the above
MappingProjection with a parameterPorts[MATRIX] for one of the above
Parse error_matrix specification and insure it is compatible with error_signal and activation_output
- Insure that the length of the error_signal matches the number of cols (receiver elements) of error_matrix
(since it will be dot-producted to generate the weighted error signal)
- Insure that length of activation_output matches the number of rows (sender elements) of error_matrix
(since it will be compared against the result of the dot product of the error_matrix and error_signal
- Note: error_matrix is left in the form in which it was specified so that, if it is a ParameterPort
or MappingProjection, its current value can be accessed at runtime (i.e., it can be used as a “pointer”)
- _function(variable=None, context=None, error_matrix=None, params=None, **kwargs)¶
Note
Both variable and error_matrix must be specified for the function to execute.
- Parameters
variable (List or 2d array [length 3 in axis 0]) – must have three items that are the values for (in order):
activation_input
(1d array),activation_output
(1d array),error_signal
(1d array).error_matrix (List, 2d array, np.matrix, ParameterPort, or MappingProjection) – matrix of weights that were used to generate the
error_signal
(3rd item ofvariable
fromactivation_output
; its dimensions must be the length ofactivation_output
(rows) x length oferror_signal
(cols).params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
- Returns
weight change matrix, weighted error signal – the modifications to make to the matrix,
error_signal
weighted by the contribution made by each element ofactivation_output
as a function oferror_matrix
.- Return type
List[2d array, 1d array]
- class psyneulink.core.components.functions.learningfunctions.TDLearning(default_variable=None, learning_rate=None, params=None, owner=None, prefs=None)¶
Implement temporal difference learning using the
Reinforcement
Function (seeReinforcement
for class details).- _validate_variable(variable, context=None)¶
Validate variable and return validated variable
Convert self.class_defaults.variable specification and variable (if specified) to list of 1D np.ndarrays:
VARIABLE SPECIFICATION: ENCODING: Simple value variable: 0 -> [array([0])] Single state array (vector) variable: [0, 1] -> [array([0, 1])] Multiple port variables, each with a single value variable: [[0], [0]] -> [array[0], array[0]]
- Perform top-level type validation of variable against the self.class_defaults.variable;
if the type is OK, the value is returned (which should be used by the function)
This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is
True
(which it is by default)- IMPLEMENTATION NOTES:
future versions should add hierarchical/recursive content (e.g., range) checking
add request/target pattern?? (as per _validate_params) and return validated variable?
- Parameters
variable – (anything other than a dictionary) - variable to be validated:
context – (str)
- Return variable
validated variable