KWTA¶

Overview¶

A KWTA is a subclass of RecurrentTransferMechanism that implements a k-winners-take-all (kWTA) constraint on the number of elements of the Mechanism’s variable that are above a specified threshold. The implementation is based on the one described in O’Reilly and Munakata, 2012.

Creating a KWTA¶

A KWTA Mechanism can be created directly by calling its constructor. The k_value, threshold, and ratio arguments can be used to specify the function of the KWTA Mechanism, and default to a condition in which half of the elements in the KWTA Mechanism’s variable (k_value = 0.5) are above 0 and half are below (threshold = 0), achieved using an intermediate degree of value displacement (ratio = 0.5).

Structure¶

The KWTA calculates an offset to apply to all elements of the Mechanism’s variable array so that it has a specified number of the elements that are at or above a specified threshold value. Typically, this constraint can be satisfied in a number of ways; how it is satisfied is determined by three parameters and two options of the KWTA:

• inhibition_only option – determines whether the offset applied to the elements of the KWTA’s variable is allowed to be positive (i.e., whether the KWTA can increase the value of any elements of its variable). If set to False, the KWTA will use any offset value determined by the ratio parameter from the scope determined by the average_based option (including positive offsets). If inhibition_only is True, then any positive offset selected is “clipped” at (i.e re-assigned a value of) 0. This ensures that the values of the elements of the KWTA’s variable are never increased.

In all other respects, a KWTA has the same attributes and is specified in the same way as a standard RecurrentTransferMechanism.

Execution¶

When a KTWA is executed, it first determines its variable as follows:

The modified variable is then passed to the KWTA’s function to determine its value.

Class Reference¶

class psyneulink.library.mechanisms.processing.transfer.kwta.KWTA(default_variable=None, size=None, function=Logistic, matrix=None, auto=None, hetero=None, initial_value=None, noise=0.0, integration_rate=1.0, k_value=0.5, threshold=0, ratio=0.5, average_based=False, inhibition_only=True, clip=None, params=None, name=None, prefs=None)

Subclass of RecurrentTransferMechanism that dynamically regulates its input relative to a given threshold.

Parameters: default_variable (number, list or np.ndarray : default Transfer_DEFAULT_BIAS) – specifies the input to the mechanism to use if none is provided in a call to its execute or run method; also serves as a template to specify the length of variable for function, and the primary OutputState of the mechanism. size (int, list or np.ndarray of ints) – specifies variable as array(s) of zeros if variable is not passed as an argument; if variable is specified, it takes precedence over the specification of size. As an example, the following mechanisms are equivalent: T1 = TransferMechanism(size = [3, 2]) T2 = TransferMechanism(default_variable = [[0, 0, 0], [0, 0]])  function (TransferFunction : default Linear) – specifies the function used to transform the input; can be Linear, Logistic, Exponential, or a custom function. matrix (list, np.ndarray, np.matrix, matrix keyword, or AutoAssociativeProjection : default FULL_CONNECTIVITY_MATRIX) – specifies the matrix to use for creating a recurrent AutoAssociativeProjection, or a AutoAssociativeProjection to use. If auto or hetero arguments are specified, the matrix argument will be ignored in favor of those arguments. auto (number, 1D array, or None : default None) – specifies matrix as a diagonal matrix with diagonal entries equal to auto, if auto is not None; If auto and hetero are both specified, then matrix is the sum of the two matrices from auto and hetero. For example, setting auto to 1 and hetero to -1 would set matrix to have a diagonal of 1 and all non-diagonal entries -1. if the matrix argument is specified, it will be overwritten by auto and/or hetero, if either is specified. auto can be specified as a 1D array with length equal to the size of the mechanism, if a non-uniform diagonal is desired. Can be modified by control. hetero (number, 2D array, or None : default None) – specifies matrix as a hollow matrix with all non-diagonal entries equal to hetero, if hetero is not None; If auto and hetero are both specified, then matrix is the sum of the two matrices from auto and hetero. For example, setting auto to 1 and hetero to -1 would set matrix to have a diagonal of 1 and all non-diagonal entries -1. if the matrix argument is specified, it will be overwritten by auto and/or hetero, if either is specified. hetero can be specified as a 2D array with dimensions equal to the matrix dimensions, if a non-uniform diagonal is desired. Can be modified by control. initial_value (value, list or np.ndarray : default Transfer_DEFAULT_BIAS) – specifies the starting value for time-averaged input (only relevant if integration_rate is not 1.0). noise (float or function : default 0.0) – a value added to the result of the function or to the result of integrator_function, depending on whether integrator_mode is True or False. See noise for more details. integration_rate (float : default 0.5) – the smoothing factor for exponential time averaging of input when integrator_mode is set to True result = (integration_rate * current input) + (1-integration_rate * result on previous time_step)  k_value (number : default 0.5) – specifies the proportion or number of the elements of variable that should be at or above the threshold. A value between 0 and 1 specifies the proportion of elements that should be at or above the threshold, while a positive integer specifies the number of values that should be at or above the threshold. A negative integer specifies the number of elements that should be below the threshold. threshold (number : default 0) – specifies the threshold at or above which the KTWA seeks to assign k_value elements of its variable. ratio (number : default 0.5) – specifies the offset used to adjust the elements of variable so that there are the number specified by k_value at or above the threshold; it must be a number from 0 to 1 (see ratio for additional information). average_based (boolean : default False) – specifies whether the average-based scaling is used to determine the scope of offsets (see average_based for additional information). inhibition_only (boolean : default True) – specifies whether positive offsets can be applied to the variable in an effort to achieve k_value elements at or above the threshold. If set to False, any offset is allowed, including positive offsets; if set to True, a positive offset will be re-assigned the value of 0 (see inhibition_only for additional information). clip (list [float, float] : default None (Optional)) – specifies the allowable range for the result of function the item in index 0 specifies the minimum allowable value of the result, and the item in index 1 specifies the maximum allowable value; any element of the result that exceeds the specified minimum or maximum value is set to the value of clip that it exceeds. params (Dict[param keyword: param value] : default None) – a parameter dictionary that can be used to specify the parameters for the mechanism, its function, and/or a custom function and its parameters. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor. name (str : default see name) – specifies the name of the KWTA Mechanism. prefs (PreferenceSet or specification dict : default Mechanism.classPreferences) – specifies the PreferenceSet for the KWTA Mechanism; see prefs for details. context (str : default componentType+INITIALIZING) – string used for contextualization of instantiation, hierarchical calls, executions, etc.
variable

value – the input to Mechanism’s function.

function

Function – the Function used to transform the input.

matrix

2d np.array – the matrix parameter of the recurrent_projection for the Mechanism.

recurrent_projection

AutoAssociativeProjection – an AutoAssociativeProjection that projects from the Mechanism’s primary OutputState back to its primary inputState.

initial_value

value, list or np.ndarray : Transfer_DEFAULT_BIAS – determines the starting value for time-averaged input (only relevant if integration_rate parameter is not 1.0).

noise

float or function : default 0.0 – When integrator_mode is set to True, noise is passed into the integrator_function. Otherwise, noise is added to the output of the function.

If noise is a list or array, it must be the same length as variable.

If noise is specified as a single float or function, while variable is a list or array, noise will be applied to each variable element. In the case of a noise function, this means that the function will be executed separately for each variable element.

Note

In order to generate random noise, we recommend selecting a probability distribution function (see Distribution Functions for details), which will generate a new noise value from its distribution on each execution. If noise is specified as a float or as a function with a fixed output, then the noise will simply be an offset that remains the same across all executions.

integration_rate

float : default 0.5 – the smoothing factor for exponential time averaging of input when integrator_mode is set to True:

result = (integration_rate * current input) + (1-integration_rate * result on previous time_step)

k_value

number – determines the number or proportion of elements of variable that should be above the threshold of the KWTA (see k_value for additional information).

threshold

number – determines the threshold at or above which the KTWA seeks to assign k_value elements of its variable.

ratio

number – determines the offset used to adjust the elements of variable so that there are k_value elements at or above the threshold (see ratio for additional information).

average_based

boolean : default False – determines the way in which the scope of offsets is determined, from which the one is selected that is applied to the elements of the variable (see average_based for additional information).

inhibition_only

boolean : default True – determines whether a positive offset is allowed; if it is True, then the value of the offset is “clipped” at (that is, any positive value is replaced by) 0. Otherwise, any offset is allowed (see inhibition_only for additional information).

clip

list [float, float] : default None (Optional) – specifies the allowable range for the result of function

the item in index 0 specifies the minimum allowable value of the result, and the item in index 1 specifies the maximum allowable value; any element of the result that exceeds the specified minimum or maximum value is set to

the value of clip that it exceeds.
integrator_function

When integrator_mode is set to True, the KWTA executes its integrator_function, which is the AdaptiveIntegrator. See AdaptiveIntegrator for more details on what it computes. Keep in mind that the integration_rate parameter of the KWTA corresponds to the rate of the KWTAIntegrator.

integrator_mode

When integrator_mode is set to True:

the variable of the mechanism is first passed into the following equation:

$value = previous\_value(1-smoothing\_factor) + variable \cdot smoothing\_factor + noise$

The result of the integrator function above is then passed into the mechanism's function. Note that on the first execution, initial_value sets previous_value.

When integrator_mode is set to False:

The variable of the mechanism is passed into the function of the mechanism. The mechanism’s integrator_function is skipped entirely, and all related arguments (noise, leak, initial_value, and time_step_size) are ignored.

previous_input

1d np.array of floats – the value of the input on the previous execution, including the value of recurrent_projection.

value

2d np.array [array(float64)] – result of executing function; same value as first item of output_values.

output_states

Dict[str, OutputState] – an OrderedDict with the following OutputStates:

• TRANSFER_RESULT, the value of which is the result of function;
• TRANSFER_MEAN, the value of which is the mean of the result;
• TRANSFER_VARIANCE, the value of which is the variance of the result;
• ENERGY, the value of which is the energy of the result, calculated using the Stability Function with the ENERGY metric;
• ENTROPY, the value of which is the entropy of the result, calculated using the Stability Function with the ENTROPY metric; note: this is only present if the mechanism’s function is bounded between 0 and 1 (e.g., the Logistic function).
output_values

List[array(float64), float, float] – a list with the following items:

• result of the function calculation (value of TRANSFER_RESULT OutputState);
• mean of the result (value of TRANSFER_MEAN OutputState)
• variance of the result (value of TRANSFER_VARIANCE OutputState);
• energy of the result (value of ENERGY OutputState);
• entropy of the result (if the ENTROPY OutputState is present).
name

str – the name of the KWTA Mechanism; if it is not specified in the name argument of the constructor, a default is assigned by MechanismRegistry (see Naming for conventions used for default and duplicate names).

prefs

PreferenceSet or specification dict – the PreferenceSet for the KWTA Mechanism; if it is not specified in the prefs argument of the constructor, a default is assigned using classPreferences defined in __init__.py (see PreferenceSet for details).

Returns: instance of KWTA KWTA