KWTA¶
Overview¶
A KWTA is a subclass of RecurrentTransferMechanism that implements a kwinnerstakeall (kWTA) constraint on the
number of elements of the Mechanism’s variable
that are above a specified threshold. The
implementation is based on the one described in O’Reilly and Munakata, 2012.
Creating a KWTA¶
A KWTA Mechanism can be created directly by calling its constructor. The k_value, threshold,
and ratio arguments can be used to specify the function of the KWTA Mechanism, and default to a condition
in which half of the elements in the KWTA Mechanism’s variable
(k_value = 0.5) are above 0 and
half are below (threshold = 0), achieved using an intermediate degree of value displacement (ratio = 0.5).
Structure¶
The KWTA calculates an offset to apply to all elements of the Mechanism’s variable
array so that it
has a specified number of the elements that are at or above a specified threshold value. Typically, this constraint
can be satisfied in a number of ways; how it is satisfied is determined by three parameters and two options of the
KWTA:
k_value
parameter – determines the number of elements of itsvariable
that should be at or above the specifiedthreshold
. A value between 0 and 1 specifies the proportion of elements that should be at or above thethreshold
, while a positive integer specifies the number of elements that should be at or above thethreshold
. A negative integer specifies the number of elements that should be below thethreshold
. Whether or not the exact specification is achieved depends on the settings of theaverage_based
andinhibition_only
options (see below).
threshold
parameter – determines the value at or above which the KTWA seeks to assignk_value
elements of itsvariable
.
ratio
parameter – determines how the offset applied to the elements of the KWTA’svariable
is selected from the scope of possible values; theratio
must be a number between 0 and 1. An offset is picked that is above the low end of the scope by a proportion of the scope equal to theratio
parameter. How the scope is calculated is determined by theaverage_based
option, as described below.
average_based
option – determines how the scope of values is calculated from which the offset applied to the elements of the KWTA’svariable
is selected; Ifaverage_based
isFalse
, the low end of the scope is the offset that sets the kth highest element exactly at at the threshold (that is, the smallest value that insures thatk_value
elements are at or above thethreshold
; the high end of the scope is the offset that sets the k+1th highest element exactly at the threshold (that is, the largest possible value, such that thek_value
elements but no more are above thethreshold
(i.e., the next one is exactly at it). With this setting, all values of offset within the scope generate exactlyk_value
elements at or above thethreshold
. Ifaverage_based
isTrue
, the low end of the scope is the offset that places the average of the elements with thek_value
highest values at thethreshold
, and the high end of the scope is the offset that places the average of the remaining elements at thethreshold
. In this case, the lowest values of offset within the scope may produce fewer thank_value
elements at or above thethreshold
, while the highest values within the scope may produce more. An offset is picked from the scope as specified by theratio
parameter (see above).Note
If the
average_based
option isFalse
(the default), the KWTA’svariable
is guaranteed to have exactlyk_value
elements at or above thethreshold
(that is, for any value of theratio
). However, ifaverage_based
isTrue
, this guarantee does not hold;variable
may have fewer thank_value
elements at or above thethreshold
(if theratio
is low), or more thank_value
(if theratio
is high).Although setting the
average_based
option toTrue
does not guarantee that exactlyk_value
elements will be above the threshold, the additional flexibility it affords in the Mechanism’svariable
attribute can be useful in some settings – for example, when training hidden layers in a multilayered network, which may require different numbers of elements to be above the specifiedthreshold
for different inputtarget pairings.
inhibition_only
option – determines whether the offset applied to the elements of the KWTA’svariable
is allowed to be positive (i.e., whether the KWTA can increase the value of any elements of itsvariable
). If set toFalse
, the KWTA will use any offset value determined by theratio
parameter from the scope determined by theaverage_based
option (including positive offsets). Ifinhibition_only
isTrue
, then any positive offset selected is “clipped” at (i.e reassigned a value of) 0. This ensures that the values of the elements of the KWTA’svariable
are never increased.
In all other respects, a KWTA has the same attributes and is specified in the same way as a standard RecurrentTransferMechanism.
Execution¶
When a KTWA is executed, it first determines its variable
as follows:
 First, like every RecurrentTransferMechanism, it combines the input it receives from its recurrent
AutoAssociativeProjection (see Recurrent_Transfer_Structure) with the input
from any other MappingProjections it receives, and assigns this to its
variable
attribute.
Then it modifies its
variable
, by calculating and assigning an offset to its elements, so that as close tok_value
elements as possible are at or above thethreshold
. The offset is determined by carrying out the following steps in each execution of the KTWA: calculate the scope of offsets that will satisfy the constraint; how this is done is determined by the
average_based
attribute (see above);
 constrain the offset to be 0 or negative if the
inhibition_only
option is set (see above;
 apply the offset to all elements of the
variable
.
 calculate the scope of offsets that will satisfy the constraint; how this is done is determined by the
The modified variable
is then passed to the KWTA’s function
to determine its
value
.
Class Reference¶

class
psyneulink.library.mechanisms.processing.transfer.kwta.
KWTA
(default_variable=None, size=None, function=Logistic, matrix=None, auto=None, hetero=None, initial_value=None, noise=0.0, smoothing_factor=1.0, k_value=0.5, threshold=0, ratio=0.5, average_based=False, inhibition_only=True, clip=None, params=None, name=None, prefs=None)¶ Subclass of RecurrentTransferMechanism that dynamically regulates its input relative to a given threshold.
Parameters:  default_variable (number, list or np.ndarray : default Transfer_DEFAULT_BIAS) – specifies the input to the mechanism to use if none is provided in a call to its
execute
orrun
method; also serves as a template to specify the length ofvariable
forfunction
, and the primary OutputState of the mechanism.  size (int, list or np.ndarray of ints) –
specifies variable as array(s) of zeros if variable is not passed as an argument; if variable is specified, it takes precedence over the specification of size. As an example, the following mechanisms are equivalent:
T1 = TransferMechanism(size = [3, 2]) T2 = TransferMechanism(default_variable = [[0, 0, 0], [0, 0]])
 function (TransferFunction : default Linear) – specifies the function used to transform the input; can be
Linear
,Logistic
,Exponential
, or a custom function.  matrix (list, np.ndarray, np.matrix, matrix keyword, or AutoAssociativeProjection : default FULL_CONNECTIVITY_MATRIX) – specifies the matrix to use for creating a recurrent AutoAssociativeProjection, or a AutoAssociativeProjection to use. If auto or hetero arguments are specified, the matrix argument will be ignored in favor of those arguments.
 auto (number, 1D array, or None : default None) – specifies matrix as a diagonal matrix with diagonal entries equal to auto, if auto is not None; If auto and hetero are both specified, then matrix is the sum of the two matrices from auto and hetero. For example, setting auto to 1 and hetero to 1 would set matrix to have a diagonal of 1 and all nondiagonal entries 1. if the matrix argument is specified, it will be overwritten by auto and/or hetero, if either is specified. auto can be specified as a 1D array with length equal to the size of the mechanism, if a nonuniform diagonal is desired. Can be modified by control.
 hetero (number, 2D array, or None : default None) – specifies matrix as a hollow matrix with all nondiagonal entries equal to hetero, if hetero is not None; If auto and hetero are both specified, then matrix is the sum of the two matrices from auto and hetero. For example, setting auto to 1 and hetero to 1 would set matrix to have a diagonal of 1 and all nondiagonal entries 1. if the matrix argument is specified, it will be overwritten by auto and/or hetero, if either is specified. hetero can be specified as a 2D array with dimensions equal to the matrix dimensions, if a nonuniform diagonal is desired. Can be modified by control.
 initial_value (value, list or np.ndarray : default Transfer_DEFAULT_BIAS) – specifies the starting value for timeaveraged input (only relevant if
smoothing_factor
is not 1.0).  noise (float or function : default 0.0) – specifies a stochasticallysampled value added to the result of the
function
: if it is a float, it must be in the interval [0,1] and is used to scale the variance of a zeromean Gaussian; if it is a function, it must return a scalar value.  smoothing_factor (float : default 0.5) –
the smoothing factor for exponential time averaging of input when
integrator_mode
is set to Trueresult = (smoothing_factor * current input) + (1smoothing_factor * result on previous time_step)
 k_value (number : default 0.5) – specifies the proportion or number of the elements of
variable
that should be at or above thethreshold
. A value between 0 and 1 specifies the proportion of elements that should be at or above thethreshold
, while a positive integer specifies the number of values that should be at or above thethreshold
. A negative integer specifies the number of elements that should be below thethreshold
.  threshold (number : default 0) – specifies the threshold at or above which the KTWA seeks to assign
k_value
elements of itsvariable
.  ratio (number : default 0.5) – specifies the offset used to adjust the elements of
variable
so that there are the number specified byk_value
at or above thethreshold
; it must be a number from 0 to 1 (see ratio for additional information).  average_based (boolean : default False) – specifies whether the averagebased scaling is used to determine the scope of offsets (see average_based for additional information).
 inhibition_only (boolean : default True) – specifies whether positive offsets can be applied to the
variable
in an effort to achievek_value
elements at or above thethreshold
. If set toFalse
, any offset is allowed, including positive offsets; if set toTrue
, a positive offset will be reassigned the value of 0 (see inhibition_only for additional information).  clip (Optional[Tuple[float, float]]) – specifies the allowable range for the result of
function
: the first item specifies the minimum allowable value of the result, and the second its maximum allowable value; any element of the result that exceeds the specified minimum or maximum value is set to the value ofclip
that it exceeds.  params (Dict[param keyword: param value] : default None) – a parameter dictionary that can be used to specify the parameters for the mechanism, its function, and/or a custom function and its parameters. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
 name (str : default see
name
) – specifies the name of the KWTA Mechanism.  prefs (PreferenceSet or specification dict : default Mechanism.classPreferences) – specifies the
PreferenceSet
for the KWTA Mechanism; seeprefs
for details.  context (str : default componentType+INITIALIZING) – string used for contextualization of instantiation, hierarchical calls, executions, etc.

function
¶ Function – the Function used to transform the input.

matrix
¶ 2d np.array – the
matrix
parameter of therecurrent_projection
for the Mechanism.

recurrent_projection
¶ AutoAssociativeProjection – an AutoAssociativeProjection that projects from the Mechanism’s primary OutputState back to its primary inputState.

initial_value
¶ value, list or np.ndarray : Transfer_DEFAULT_BIAS – determines the starting value for timeaveraged input (only relevant if
smoothing_factor
parameter is not 1.0).

noise
¶ float or function : default 0.0 – a stochasticallysampled value added to the output of the
function
: if it is a float, it must be in the interval [0,1] and is used to scale the variance of a zeromean Gaussian; if it is a function, it must return a scalar value.

smoothing_factor
¶ float : default 0.5 – the smoothing factor for exponential time averaging of input when
integrator_mode
is set to True:result = (smoothing_factor * current input) + (1smoothing_factor * result on previous time_step)

k_value
¶ number – determines the number or proportion of elements of
variable
that should be above thethreshold
of the KWTA (see k_value for additional information).

threshold
¶ number – determines the threshold at or above which the KTWA seeks to assign
k_value
elements of itsvariable
.

ratio
¶ number – determines the offset used to adjust the elements of
variable
so that there arek_value
elements at or above thethreshold
(see ratio for additional information).

average_based
¶ boolean : default False – determines the way in which the scope of offsets is determined, from which the one is selected that is applied to the elements of the
variable
(see average_based for additional information).

inhibition_only
¶ boolean : default True – determines whether a positive offset is allowed; if it is
True
, then the value of the offset is “clipped” at (that is, any positive value is replaced by) 0. Otherwise, any offset is allowed (see inhibition_only for additional information).

clip
¶ Tuple[float, float] – determines the allowable range of the result: the first value specifies the minimum allowable value and the second the maximum allowable value; any element of the result that exceeds minimum or maximum is set to the value of
clip
it exceeds. Iffunction
isLogistic
,clip
is set by default to (0,1).

previous_input
¶ 1d np.array of floats – the value of the input on the previous execution, including the value of
recurrent_projection
.

value
¶ 2d np.array [array(float64)] – result of executing
function
; same value as first item ofoutput_values
.

output_states
¶ Dict[str, OutputState] – an OrderedDict with the following OutputStates:
TRANSFER_RESULT
, thevalue
of which is the result offunction
;TRANSFER_MEAN
, thevalue
of which is the mean of the result;TRANSFER_VARIANCE
, thevalue
of which is the variance of the result;ENERGY
, thevalue
of which is the energy of the result, calculated using theStability
Function with the ENERGY metric;ENTROPY
, thevalue
of which is the entropy of the result, calculated using theStability
Function with the ENTROPY metric; note: this is only present if the mechanism’sfunction
is bounded between 0 and 1 (e.g., theLogistic
function).

output_values
¶ List[array(float64), float, float] – a list with the following items:
 result of the
function
calculation (value of TRANSFER_RESULT OutputState);  mean of the result (
value
of TRANSFER_MEAN OutputState)  variance of the result (
value
of TRANSFER_VARIANCE OutputState);  energy of the result (
value
of ENERGY OutputState);  entropy of the result (if the ENTROPY OutputState is present).
 result of the

name
¶ str – the name of the KWTA Mechanism; if it is not specified in the name argument of the constructor, a default is assigned by MechanismRegistry (see Naming for conventions used for default and duplicate names).

prefs
¶ PreferenceSet or specification dict – the
PreferenceSet
for the KWTA Mechanism; if it is not specified in the prefs argument of the constructor, a default is assigned usingclassPreferences
defined in __init__.py (see PreferenceSet for details).
Returns: instance of KWTA Return type: KWTA  default_variable (number, list or np.ndarray : default Transfer_DEFAULT_BIAS) – specifies the input to the mechanism to use if none is provided in a call to its