# TransferFunctions¶

## Overview¶

Functions that transform their variable but maintain its shape.

All TransferFunctions have the following attributes:

• bounds: specifies the lower and upper limits of the result; if there are none, the attribute is set to None; if it has at least one bound, the attribute is set to a tuple specifying the lower and upper bounds, respectively, with None as the entry for no bound.
• multiplicative_param and additive_param: each of these is assigned the name of one of the function’s parameters and used by ModulatoryProjections to modulate the output of the TransferFunction’s function (see Modulatory Parameters).
class psyneulink.core.components.functions.transferfunctions.Linear(default_variable, slope=1.0, intercept=0.0, params=None, owner=None, name=None, prefs=None)

function returns linear transform of variable:

$slope * variable + intercept$

Note: default values for slope and intercept implement the IDENTITY_FUNCTION.

Parameters: default_variable (number or array : default class_defaults.variable) – specifies a template for the value to be transformed. slope (float : default 1.0) – specifies a value by which to multiply variable. intercept (float : default 0.0) – specifies a value to add to each element of variable after applying slope. params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. owner (Component) – component to which to assign the Function. name (str : default see name) – specifies the name of the Function. prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).
variable

number or array – contains value to be transformed.

slope

float – value by which each element of variable is multiplied before applying the intercept (if it is specified).

intercept

float – value added to each element of variable after applying the slope (if it is specified).

bounds

Tuple or None – determines the lower and upper limits of the result; if at least one bound is specified, the attribute is a tuple specifying the lower and upper bounds, respectively, with None as the entry for no bound.

owner

Componentcomponent to which the Function has been assigned.

name

str – the name of the Function; if it is not specified in the name argument of the constructor, a default is assigned by FunctionRegistry (see Naming for conventions used for default and duplicate names).

prefs

PreferenceSet or specification dict : Function.classPreferences – the PreferenceSet for function; if it is not specified in the prefs argument of the Function’s constructor, a default is assigned using classPreferences defined in __init__.py (see PreferenceSet for details).

function(variable=None, execution_id=None, params=None, context=None)
Parameters: variable (number or array : default class_defaults.variable) – a single value or array to be transformed. params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. linear transformation of variable number or array
derivative(input)

Derivative of function at input.

Parameters: input (number) – value of the input to the Linear transform at which derivative is to be taken. Slope of function number or array
class psyneulink.core.components.functions.transferfunctions.Exponential(default_variable, rate=1.0, bias=0.0, scale=1.0, offset=0.0, params=None, owner=None, name=None, prefs=None)

function returns exponential transform of variable:

$scale * e^{rate*variable+bias} + offset$

derivative returns the derivative of the Exponential:

$rate*input+bias$
Parameters: default_variable (number or array : default class_defaults.variable) – specifies a template for the value to be transformed. rate (float : default 1.0) – specifies a value by which to multiply variable before exponentiation. bias (float : default 0.0) – specifies a value to add to variable after multplying by rate and before exponentiation. scale (float : default 1.0) – specifies a value by which to multiply the exponentiated value of variable. offset (float : default 0.0) – specifies value to add to the exponentiated value of variable after multiplying by scale. params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. owner (Component) – component to which to assign the Function. name (str : default see name) – specifies the name of the Function. prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).
variable

number or array – contains value to be transformed.

rate

float – value by which variable is multiplied before exponentiation; assigned as MULTILICATIVE_PARAM of the Exponential Function.

bias

float – value added to variable after multiplying by rate and before exponentiation; assigned as ADDITIVE_PARAM of the Exponential Function.

scale

float – value by which the exponentiated value is multiplied.

offset

float – value added to exponentiated value after multiplying by scale.

bounds

(0, None)

owner

Componentcomponent to which the Function has been assigned.

name

str – the name of the Function; if it is not specified in the name argument of the constructor, a default is assigned by FunctionRegistry (see Naming for conventions used for default and duplicate names).

prefs

PreferenceSet or specification dict : Function.classPreferences – the PreferenceSet for function; if it is not specified in the prefs argument of the Function’s constructor, a default is assigned using classPreferences defined in __init__.py (see PreferenceSet for details).

function(variable=None, execution_id=None, params=None, context=None)
Parameters: variable (number or array : default class_defaults.variable) – a single value or array to be exponentiated. params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. Exponential transformation of variable number or array
derivative(input)
Parameters: input (number) – value of the input to the Exponential transform at which derivative is to be taken. of function at input. (Derivative) – derivative number or array
class psyneulink.core.components.functions.transferfunctions.Logistic(default_variable, gain=1.0, bias=0.0, x_0=0.0, offset=0.0, scale=1.0, params=None, owner=None, name=None, prefs=None)

function returns logistic transform of variable:

$\frac{1}{1 + e^{ - gain ( variable + bias - x_{0}) + offset}}$

(this is an offset and scaled version of the Tanh, which is centered on origin).

Note

The bias and x_0 arguments are identical, apart from opposite signs: bias is included to accomodate the convention in the machine learning community; x_0 is included to match the standard form of the Logistic Function (in which gain corresponds to the k parameter and scale corresponds to the L parameter).

derivative returns the derivative of the Logistic using its output:

$gain * scale * output * (1-output)$
Parameters: default_variable (number or array : default class_defaults.variable) – specifies a template for the value to be transformed. gain (float : default 1.0) – specifies value by which to multiply variable before logistic transformation bias (float : default 0.0) – specifies value to add to each element of variable before applying gain and before logistic transformation. This argument is identical to x_0, with the opposite sign. x_0 (float : default 0.0) – specifies value to subtract from each element of variable before applying gain and before logistic transformation. This argument is identical to bias, with the opposite sign. offset (float : default 0.0) – specifies value to add to each element of variable after applying gain but before logistic transformation. scale (float : default 0.0) – specifies value by which each element is multiplied after applying the logistic transformation. params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. owner (Component) – component to which to assign the Function. name (str : default see name) – specifies the name of the Function. prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).
variable

number or array – contains value to be transformed.

gain

float : default 1.0 – value by which each element of variable is multiplied before applying the bias (if it is specified).

bias

float : default 0.0 – value added to each element of variable before applying the gain (if it is specified). This attribute is identical to x_0, with the opposite sign.

x_0

float : default 0.0 – value subtracted from each element of variable before applying the gain (if it is specified). This attribute is identical to bias, with the opposite sign.

offset

float : default 0.0 – value to added to each element of variable after applying gain but before logistic transformation.

scale

float : default 0.0 – value by which each element is multiplied after applying the Logistic transform.

bounds

(0,1)

owner

Componentcomponent to which the Function has been assigned.

name

str – the name of the Function; if it is not specified in the name argument of the constructor, a default is assigned by FunctionRegistry (see Naming for conventions used for default and duplicate names).

prefs

PreferenceSet or specification dict : Function.classPreferences – the PreferenceSet for function; if it is not specified in the prefs argument of the Function’s constructor, a default is assigned using classPreferences defined in __init__.py (see PreferenceSet for details).

function(variable=None, execution_id=None, params=None, context=None)
Parameters: variable (number or array : default class_defaults.variable) – a single value or array to be transformed. params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. Logistic transformation of variable number or array
derivative(input=None, output=None)

Derivative of function at either input or output.

Either input or ouput must be specified. If output is not specified, it is computed from input. If both are specified, input is ignored unless paramValidationPref is set, in which case an error is generated if output does not correspond to function(input).

Parameters: input (number) – value of the input to the Logistic transform at which derivative is to be taken. output (number) – value of the output of the Logistic transform at which derivative is to be taken. Deriviative of logistic transform at output number or array
class psyneulink.core.components.functions.transferfunctions.Tanh(default_variable, gain=1.0, bias=0.0, x_0=0.0, offset=0.0, scale=1.0, params=None, owner=None, name=None, prefs=None)

function returns hyperbolic tangent of variable:

$\frac{1 - e^{-2(gain*(variable+bias-x\_0)+offset)}}{1 + e^{-2(gain*(variable+bias-x\_0)+offset)}}$

Note

The Logistic function is an offset and scaled version of this function. The parameters used here have the same meaning as those used for the Logistic Function.

derivative returns the derivative of the hyperbolic tangent at its input:

$\frac{gain*scale}{(\frac{1+e^{-2(gain*(variable+bias-x\_0)+offset)}}{2e^{-(gain*( variable+bias-x\_0)+offset)}})^2}$
Parameters: default_variable (number or array : default class_defaults.variable) – specifies template for the value to be transformed. gain (float : default 1.0) – specifies value by which to multiply variable before logistic transformation bias (float : default 0.0) – specifies value to add to each element of variable before applying gain and before logistic transformation. This argument is identical to x_0, with the opposite sign. x_0 (float : default 0.0) – specifies value to subtract from each element of variable before applying gain and before logistic transformation. This argument is identical to bias, with the opposite sign. offset (float : default 0.0) – specifies value to add to each element of variable after applying gain but before logistic transformation. scale (float : default 1.0) – specifies value by which to multiply each element after applying Tanh transform. params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. owner (Component) – component to which to assign the Function. name (str : default see name) – specifies the name of the Function. prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).
variable

number or array – contains value to be transformed.

gain

float : default 1.0 – value by which each element of variable is multiplied before applying the bias (if it is specified).

bias

float : default 0.0 – value added to each element of variable before applying the gain (if it is specified). This attribute is identical to x_0, with the opposite sign.

x_0

float : default 0.0 – value subtracted from each element of variable before applying the gain (if it is specified). This attribute is identical to bias, with the opposite sign.

offset

float : default 0.0 – value to added to each element of variable after applying gain but before logistic transformation.

scale

float : default 1.0 – value by which element is multiplied after applying Tanh transform.

bounds

(0,1)

owner

Componentcomponent to which the Function has been assigned.

name

str – the name of the Function; if it is not specified in the name argument of the constructor, a default is assigned by FunctionRegistry (see Naming for conventions used for default and duplicate names).

prefs

PreferenceSet or specification dict : Function.classPreferences – the PreferenceSet for function; if it is not specified in the prefs argument of the Function’s constructor, a default is assigned using classPreferences defined in __init__.py (see PreferenceSet for details).

function(variable=None, execution_id=None, params=None, context=None)
Parameters: variable (number or array : default class_defaults.variable) – a single value or array to be transformed. params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. hyperbolic tangent of variable number or array
derivative(input)

Derivative of function at input.

Parameters: input (number) – value of the input to the Tanh transform at which derivative is to be taken. derivative number or array
class psyneulink.core.components.functions.transferfunctions.ReLU(default_variable, gain=1.0, bias=0.0, leak=0.0, params=None, owner=None, name=None, prefs=None)

function returns rectified linear tranform of variable:

$x = gain*(variable - bias)$
$max(x, leak * x)$

Commonly used by ReLU units in neural networks.

derivative returns the derivative of of the rectified linear tranform at its input:

$gain\ if\ input > 0,\ gain*leak\ otherwise$
Parameters: default_variable (number or array : default class_defaults.variable) – specifies a template for the value to be transformed. gain (float : default 1.0) – specifies a value by which to multiply variable after bias is subtracted from it. bias (float : default 0.0) – specifies a value to subtract from each element of variable. leak (float : default 0.0) – specifies a scaling factor between 0 and 1 when (variable - bias) is lesser than or equal to 0. params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. owner (Component) – component to which to assign the Function. name (str : default see name) – specifies the name of the Function. prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).
variable

number or array – contains value to be transformed.

gain

float : default 1.0 – value by which to multiply variable after bias is subtracted from it.

bias

float : default 0.0 – value to subtract from each element of variable.

leak

float : default 0.0 – scaling factor between 0 and 1 when (variable - bias) is lesser than or equal to 0.

bounds

(None,None)

owner

Componentcomponent to which the Function has been assigned.

name

str – the name of the Function; if it is not specified in the name argument of the constructor, a default is assigned by FunctionRegistry (see Naming for conventions used for default and duplicate names).

prefs

PreferenceSet or specification dict : Function.classPreferences – the PreferenceSet for function; if it is not specified in the prefs argument of the Function’s constructor, a default is assigned using classPreferences defined in __init__.py (see PreferenceSet for details).

function(variable=None, execution_id=None, params=None, context=None)
Parameters: variable (number or array : default class_defaults.variable) – a single value or array to be transformed. params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. ReLU transformation of variable number or array
derivative(input)

Derivative of function at input.

Parameters: input (number) – value of the input to the ReLU transform at which derivative is to be taken. derivative number or array
class psyneulink.core.components.functions.transferfunctions.Gaussian(default_variable, standard_deviation=1.0, bias=0.0, scale=1.0, offset=0.0, params=None, owner=None, name=None, prefs=None)

function returns Gaussian transform of variable:

$scale*\frac{e^{-\frac{(varible-bias)^{2}}{2\sigma^{2}}}}{\sqrt{2\pi}\sigma}+offset$

where $$\sigma$$ = standard_deviation

Note

the value returned is deterministic (i.e., the value of the probability density function at variable), not a randomly chosen sample from the Gaussian distribution; for the latter, use GaussianDistort.

derivative returns derivative of the Gaussian transform of variable:

$\frac{-(variable-bias)*e^{-\frac{(variable-bias)^{2}}{2\sigma^{2}}}}{\sqrt{2\pi}\sigma^{3}}$
Parameters: default_variable (number or array : default class_defaults.variable) – specifies a template for the value used as the mean for the Guassian transform. standard_deviation (float : default 1.0) – specifies “width” of the Gaussian transform applied to each element of variable. bias (float : default 0.0) – value to add to each element of variable before applying Gaussian transform. offset (float : default 0.0) – value to add to each element after applying Gaussian transform and scale. scale (float : default 1.0) – value by which to multiply each element after applying Gaussian transform. params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. owner (Component) – component to which to assign the Function. name (str : default see name) – specifies the name of the Function. prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).
variable

number or array – value used as the mean of the Gaussian transform.

standard_deviation

float : default 1.0 – standard_deviation used for Gaussian transform.

bias

float : default 0.0 – value added to each element of variable before applying the Gaussian transform.

scale

float : default 0.0 – value by which each element is multiplied after applying the Gaussian transform.

offset

float : default 0.0 – value added to each element after applying the Gaussian transform and scale.

owner

Componentcomponent to which the Function has been assigned.

name

str – the name of the Function; if it is not specified in the name argument of the constructor, a default is assigned by FunctionRegistry (see Naming for conventions used for default and duplicate names).

prefs

PreferenceSet or specification dict : Function.classPreferences – the PreferenceSet for function; if it is not specified in the prefs argument of the Function’s constructor, a default is assigned using classPreferences defined in __init__.py (see PreferenceSet for details).

function(variable=None, execution_id=None, params=None, context=None)
Parameters: variable (number or array : default class_defaults.variable) – a single value or array to be transformed. params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. Gaussian transformation of variable number or array
derivative(input)

Derivative of function at input.

Parameters: input (number) – value of the input of the Gaussian transform at which derivative is to be taken. Derivative of Guassian of variable number or array
class psyneulink.core.components.functions.transferfunctions.SoftMax(default_variable, gain=1.0, output=ALL, params=None, owner=None, name=None, prefs=None)

SoftMax transform of variable

function returns SoftMax transform of variable:

$\frac{e^{gain * variable_i}}{\sum\limits^{len(variable)}e^{gain * variable}}$

filtered by ouptput specification (see The Softmax function and its derivative for a nice discussion).

derivative returns the derivative of the SoftMax. If OUTPUT_TYPE for the SoftMax is ALL, returns Jacobian matrix (derivative for each element of the output array with respect to each of the others):

$D_jS_i = S_i(\delta_{i,j} - S_j),\ where\ \delta_{i,j}=1\ if\ i=j\ and\ \delta_{i,j}=0\ if\ i≠j.$

If OUTPUT_TYPE is MAX_VAL or MAX_INDICATOR, returns 1d array of the derivatives of the maximum value with respect to the others (calculated as above). If OUTPUT_TYPE is PROB, raises an exception (since it is ambiguous as to which element would have been chosen by the SoftMax function)

Parameters: default_variable (1d array : default class_defaults.variable) – specifies a template for the value to be transformed. gain (float : default 1.0) – specifies a value by which to multiply variable before SoftMax transformation. output (ALL, MAX_VAL, MAX_INDICATOR, or PROB : default ALL) – specifies the format of array returned by function (see output for details). per_item (boolean : default True) – for 2d variables, determines whether the SoftMax function will be applied to the entire variable (per_item = False), or applied to each item in the variable separately (per_item = True). params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. owner (Component) – component to which to assign the Function. name (str : default see name) – specifies the name of the Function. prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).
variable

1d array – contains value to be transformed.

gain

float – value by which variable is multiplied before the SoftMax transformation; determines the “sharpness” of the distribution.

output

ALL, MAX_VAL, MAX_INDICATOR, or PROB – determines how the SoftMax-transformed values of the elements in variable are reported in the array returned by function:

• ALL: array of all SoftMax-transformed values (the default);
• MAX_VAL: SoftMax-transformed value for the element with the maximum such value, 0 for all others;
• MAX_INDICATOR: 1 for the element with the maximum SoftMax-transformed value, 0 for all others;
• PROB: probabilistically chosen element based on SoftMax-transformed values after setting the sum of values to 1 (i.e., their Luce Ratio), 0 for all others.
per_item

boolean : default True – for 2d variables, determines whether the SoftMax function will be applied to the entire variable (per_item = False), or applied to each item in the variable separately (per_item = True).

bounds

None if output == MAX_VAL, else (0,1) : default (0,1)

owner

Componentcomponent to which the Function has been assigned.

name

str – the name of the Function; if it is not specified in the name argument of the constructor, a default is assigned by FunctionRegistry (see Naming for conventions used for default and duplicate names).

prefs

PreferenceSet or specification dict : Function.classPreferences – the PreferenceSet for function; if it is not specified in the prefs argument of the Function’s constructor, a default is assigned using classPreferences defined in __init__.py (see PreferenceSet for details).

function(variable=None, execution_id=None, params=None, context=None)
Parameters: variable (1d array : default class_defaults.variable) – an array to be transformed. params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. SoftMax transformation of variable number or array
derivative(output)
Returns: derivative of values returned by SoftMax 1d or 2d array (depending on OUTPUT_TYPE of SoftMax)
class psyneulink.core.components.functions.transferfunctions.LinearMatrix(default_variable, matrix=None, params=None, owner=None, name=None, prefs=None)

Matrix transform of variable.

function returns dot product of variable with matrix:

$variable \bullet matrix$
Parameters: variable (list or 1d array : default class_defaults.variable) – specifies a template for the value to be transformed; length must equal the number of rows of matrix. matrix (number, list, 1d or 2d np.ndarray, np.matrix, function, or matrix keyword : default IDENTITY_MATRIX) – specifies matrix used to transform variable (see matrix for specification details). When LinearMatrix is the function of a projection: the matrix specification must be compatible with the variables of the sender and receiver a matrix keyword specification generates a matrix based on the sender and receiver shapes When LinearMatrix is instantiated on its own, or as the function of Mechanisms or States: the matrix specification must be compatible with the function’s own variable if matrix is not specified, a square identity matrix is generated based on the number of columns in variable matrix keywords are not valid matrix specifications bounds (None) – params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. owner (Component) – component to which to assign the Function. name (str : default see name) – specifies the name of the Function. prefs (PreferenceSet or specification dict : default Function.classPreferences) – specifies the PreferenceSet for the Function (see prefs for details).
variable

1d array – contains value to be transformed.

matrix

2d array – matrix used to transform variable. Can be specified as any of the following:

• number - used as the filler value for all elements of the matrix (call to np.fill);
• list of arrays, 2d array or np.matrix - assigned as the value of matrix;
• matrix keyword - see MatrixKeywords for list of options.

Rows correspond to elements of the input array (outer index), and columns correspond to elements of the output array (inner index).

owner

Componentcomponent to which the Function has been assigned.

name

str – the name of the Function; if it is not specified in the name argument of the constructor, a default is assigned by FunctionRegistry (see Naming for conventions used for default and duplicate names).

prefs

PreferenceSet or specification dict : Function.classPreferences – the PreferenceSet for function; if it is not specified in the prefs argument of the Function’s constructor, a default is assigned using classPreferences defined in __init__.py (see PreferenceSet for details).

instantiate_matrix(specification, context=None)

Implements matrix indicated by specification

Specification is derived from MATRIX param (passed to self.__init__ or self.function)

Specification (validated in _validate_params):
• single number (used to fill self.matrix)
• matrix keyword (see get_matrix)
• 2D list or np.ndarray of numbers
Return matrix: (2D list)
function(variable=None, execution_id=None, params=None, context=None)
Parameters: variable (list or 1d array) – array to be transformed; length must equal the number of rows of matrix. params (Dict[param keyword: param value] : default None) – a parameter dictionary that specifies the parameters for the function. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. dot product of variable and matrix – length of the array returned equals the number of columns of matrix. 1d array