ObjectiveMechanism¶
Contents¶
Overview¶
An ObjectiveMechanism is a ProcessingMechanism that monitors the OutputPorts
of one or more other ProcessingMechanisms specified in its monitor attribute,
and evaluates them using its function. The result of the evaluation is placed in the
ObjectiveMechanism’s OUTCOME (primary) OutputPort. ObjectiveMechanisms are typically used
closely with (and often created automatically by) ModulatoryMechanisms.
Creating an ObjectiveMechanism¶
ObjectiveMechanisms are often created automatically when other PsyNeuLink components are created (in particular,
ModulatoryMechanisms, such as LearningMechanisms and
ControlMechanisms). An ObjectiveMechanism can also be created directly by calling its
constructor. The primary attribute used to define an ObjectiveMechanism is its monitor
attribute, that is specified using the corresponding argument of its constructor as described below.
Monitor¶
The monitor argument of an ObjectiveMechanism’s constructor specifies the OutputPorts it monitors.
This takes the place of the input_ports argument used by most other forms of Mechanism, and is used
by the ObjectiveMechanism to create an InputPort for each OutputPort it monitors, along with a MappingProjection
from the OutputPort to that InputPort. The monitor argument takes a list of items that can
include any of the forms of specification used in a standard input_ports argument.
For the monitor argument, this is usually a list of OutputPorts to be monitored. However, as with a standard
input_ports argument, the monitor argument can include Mechanisms (in which case their primary OutputPort is used) or the InputPort(s) of other Mechanisms (in which case the
ObjectiveMechanism will be assigned Projections from all of the OutputPorts that project to the specified InputPort
– that is, it will shadow their inputs). Items in the monitor argument can also be
used to specify attributes of the InputPort and/or MappingProjection(s) to it, that the ObjectiveMechanism creates to
monitor the specified OutputPort. In general, the value of each specified OutputPort determines
the format of the variable of the InputPort that is created for it by the ObjectiveMechanism.
However, this can be overridden using the ObjectiveMechanism’s default_variable
or size attributes (see Mechanism InputPort specification), or by specifying a Projection from the OutputPort to the InputPort (see
Input Source Specification). If an item in the
monitor argument specifies an InputPort for the ObjectiveMechanism, but not the OutputPort to
be monitored, the InputPort is created but will be ignored until an OutputPort (and MappingProjection from it) are
specified for that InputPort.
The OutputPorts monitored by the ObjectiveMechanism are listed in its monitor attribute.
When an ObjectiveMechanism is created by a ControlMechanism, these may pass a set of OutputPorts to be monitored to
the ObjectiveMechanism. A ControlMechanism passes OutputPort specifications listed in its objective_mechanism
argument (see Objective Mechanism).
Structure¶
Input¶
An ObjectiveMechanism has one InputPort for each of the OutputPorts specified in its
monitor argument (see Monitor). Each InputPort receives a MappingProjection from the
corresponding OutputPort, the values of which are used by the ObjectiveMechanism’s
function to generate the value of its OUTCOME OutputPort. The InputPorts are listed in the ObjectiveMechanism’s input_ports attribute, and the monitored OutputPorts from which they receive projections are
listed in the same order its monitor attribute.
By default, the format of the variable for each InputPort is determined by the value of the monitored OutputPort(s) to which it corresponds. However, if either the
default_variable or size argument is specified in an Objective Mechanism’s constructor, or a variable is specified for an InputPort for one or more of the items in
its monitor argument, then that is used as the format for the corresponding InputPort(s). This can be used to
transform the value of a monitored OutputPort into different form for the variable of the InputPort (see the first example below).
If the weight and/or exponent is specified for ay item in the monitor argument of the ObjectiveMechanism’s
constructor, it is assigned to the corresponding InputPort. If the ObjectiveMechanism’s function implements a weights and/or exponents attribute, the values specified is assigned to
the corresponding attribute, and applied to the value of the InputPort before it is combined
with that of the other InputPorts to generate the ObjectiveMechanism’s output.
Function¶
The ObjectiveMechanism’s function uses the values of its input_ports to compute an objective (or “loss”) function, that is assigned as the value of its OUTCOME OutputPort. By default, it uses a LinearCombination function to sum the values of the values of
the items in its variable. However, by assigning values to the ‘weight
<InputPort.weight>` and/or ‘exponent <InputPort.exponent>` attributes of the corresponding InputPorts,
it can be configured to calculate differences, ratios, etc. (see example below). The function can also
be replaced with any CombinationFunction, or any python function that takes an 2d array as
its input (with a number of items in axis 0 equal to the number of the ObjectiveMechanism’s InputPorts), and generates
a 1d array as its result. If it implements weight and/or exponent attributes, those are assigned
from weight and exponent attributes of its input_ports (also listed in the monitor_weights_and_exponents attribute); otherwise, they are ignored.
Output¶
The primary OutputPort of an Objective mechanism is a 1d array, named OUTCOME, that is the
result of its function (as described above).
Execution¶
When an ObjectiveMechanism is executed, it updates its input_ports with the values of the OutputPorts listed in
its monitor attribute, and then uses its function
to evaluate these. The result is assigned as to its value attribute as the value of its
OUTCOME (primary) OutputPort.
Examples
Specifying the variable for the InputPorts of an ObjectiveMechanism
This can be useful in some situations, and there are several ways it can be done. For example, for
Reinforcement Learning, an ObjectiveMechanism is used to monitor the rewards by an action selection
Mechanism (and used by a LearningMechanism to improve those predictions). However, whereas the action selection
Mechanism generates a vector indicating the reward predicted by the selected action, the ObjectiveMechanism for RL
simply needs to know the magnitude of the reward predicted, irrespective of the action taken; that is, it
simply requires a single scalar value indicating the magnitude of the predicted reward. Thus, the vector of
action-related reward predictions needs to be condensed to a single predicted value for the ObjectiveMechanism. This
can be accomplished in several ways, that are illustrated in the examples below. In the first example,
a TransferMechanism with the SoftMax function (and the PROB as its output format) to select
an action and represent its reward prediction. This generates a vector with a single non-zero value, which designates
the predicted reward for the selected action. Because the output is a vector, by default the InputPort of the
ObjectiveMechanism created to monitor it will also be a vector. However, the ObjectiveMechanism requires this to be
a single value, that it can compare with the value of the reward Mechanism (monitoring the feedback provided by the
environment). In the example below, this is accomplished by using default_variable in the constructor of the
ObjectiveMechanism to force the InputPort for the ObjectiveMechanism to have a single value:
>>> import psyneulink as pnl
>>> my_action_select_mech = pnl.TransferMechanism(default_variable=[0, 0, 0],
... function=pnl.SoftMax(output=pnl.PROB),
... name='Action Selection Mech')
>>> my_reward_mech = pnl.TransferMechanism(default_variable=[0],
... name='Reward Mech')
>>> my_objective_mech = pnl.ObjectiveMechanism(default_variable=[[0],[0]],
... monitor=[my_action_select_mech, my_reward_mech])
Note that the OutputPorts for the my_action_selection and my_reward_mech are specified
in monitor. If that were the only specification, the InputPort created for my_action_select_mech
would be a vector of length 3. This is overridden by specifying default_variable as an array with two
single-value arrays (one corresponding to my_action_select_mech and the other to my_reward_mech). This forces
the InputPort for my_action_select_mech to have only a single element which, in turn, will cause a
MappingProjection to be created from my_action_select_mech to the ObjectiveMechanism’s InputPort using a
FULL_CONNECTIVITY_MATRIX (the one used for AUTO_ASSIGN_MATRIX when the sender and receiver have values of
different lengths). This produces the desired effect, since the action selected is the only non-zero value in the
output of my_action_select_mech, and so the FULL_CONNECTIVITY_MATRIX will combine it with zeros (the other values
in the vector), and so its value will be assigned as the value of the corresponding InputPort in the
ObjectiveMechanism.
An alternative would be to explicitly specify the variable attribute for the InputPort created
for my_action_select_mech using a InputPort specification dictionary in
the monitor argument of my_objective_mech, as follows:
>>> my_objective_mech = pnl.ObjectiveMechanism(monitor=[{pnl.MECHANISM: my_action_select_mech,
... pnl.VARIABLE: [0]},
... my_reward_mech])
Note that the VARIABLE entry here specifies the variable for the InputPort of the
ObjectiveMechanism created to receive a Projection from my_action_select_mech, and not my_action_select_mech
itself (see Input for a full explanation).
Another way to specify the variable for the InputPort of an ObjectiveMechanism is to
specify the Projections it receives from the OutputPort it monitors. The following example uses a tuple
specification to assign the matrix for the MappingProjection from
my_action_select_mech to the corresponding InputPort of my_objective_mech:
>>> import numpy as np
>>> my_objective_mech = pnl.ObjectiveMechanism(monitor=[(my_action_select_mech, np.ones((3,1))), my_reward_mech])
Since the matrix specified has three rows (for its inputs) and one col (for the output), it will take the length three
vector provided as the output of my_action_select_mech and combine its elements into a single value that is
provided to the InputPort of the ObjectiveMechanism.
A Connection tuple could also be used to specify the matrix, but this would require that additional entries (for the weight and exponent of the InputPort) which, in this case, are not necessary (see example below for how these can be used).
By default, an ObjectiveMechanism simply adds the values received by each of its InputPorts to generate its output. However, this too can be customized in a variety of ways.
The simplest way is to assign values to the weight and/or exponent
attributes of its InputPorts. This can be done by placing them in a tuple specification for the OutputPort that provides value for the InputPort. In the example
below, the ObjectiveMechanism used in the previous example is further customized to subtract the value of the action
selected from the value of the reward:
>>> my_objective_mech = pnl.ObjectiveMechanism(default_variable = [[0],[0]],
... monitor = [(my_action_select_mech, -1, 1), my_reward_mech])
This specifies that my_action_select_mech should be assigned a weight of -1 and an exponent of 1 when it is
submitted to the ObjectiveMechanism’s function. Notice that the exponent had to be
included, even though it is the default value; when a tuple is used, the weight and exponent values must both be
specified. Notice also that my_reward_mech does not use a tuple, so it will be assigned defaults for both the
weight and exponent parameters.
An ObjectiveMechanism can also be configured to monitor multiple OutputPorts of the same Mechanism. In the following example, an ObjectiveMechanism is configured to calculate the reward rate for a DDM Mechanism, by specifying OutputPorts for the DDM that report its response time and accuracy:
>>> my_decision_mech = pnl.DDM(output_ports=[pnl.RESPONSE_TIME,
... pnl.PROBABILITY_UPPER_THRESHOLD])
>>> my_objective_mech = pnl.ObjectiveMechanism(monitor=[
... my_reward_mech,
... my_decision_mech.output_ports[pnl.PROBABILITY_UPPER_THRESHOLD],
... (my_decision_mech.output_ports[pnl.RESPONSE_TIME], 1, -1)])
This specifies that the ObjectiveMechanism should multiply the value of my_reward_mech’s
primary OutputPort by the value of my_decision_mech’s
PROBABILITY_UPPER_THRESHOLD, and divide the result by my_decision_mech’s RESPONSE_TIME value. The two OutputPorts of my_decision_mech are referenced as items in the output_ports list of my_decision_mech. However, a 2-item (Port name, Mechanism) tuple can be used to reference them more simply, as follows:
>>> my_objective_mech = pnl.ObjectiveMechanism(monitor=[
... my_reward_mech,
... (pnl.PROBABILITY_UPPER_THRESHOLD, my_decision_mech),
... ((pnl.RESPONSE_TIME, my_decision_mech), 1, -1)])
Customizing the ObjectiveMechanism’s function
In the examples above, the weights and exponents assigned to the InputPorts are passed to the ObjectiveMechanism’s
function for use in combining their values. The same can be accomplished by
specifying the relevant parameter(s) of the function itself, as in the following example:
>>> my_objective_mech = pnl.ObjectiveMechanism(default_variable = [[0],[0]],
... monitor = [my_action_select_mech, my_reward_mech],
... function=pnl.LinearCombination(weights=[[-1], [1]]))
Here, the weights parameter of the LinearCombination function is specified directly,
with two values [-1] and [1] corresponding to the two items in monitor (and default_variable).
This will multiply the value from my_action_select_mech by -1 before adding it to (and thus subtracting it from)
the value of my_reward_mech. Notice that the weight for my_reward_mech had to be specified, even though it
is using the default value (1); whenever a weight and/or exponent parameter is specified, there must be an entry for
every item of the function’s variable. However, the exponents did not need to be
specified, as it is not used. However it, and the operation parameters of
LinearCombination can also be used to multiply and divide quantities.
Class Reference¶
- class psyneulink.core.components.mechanisms.processing.objectivemechanism.ObjectiveMechanism(monitor=None, default_variable=None, size=None, function=None, output_ports=None, params=None, name=None, prefs=None, **kwargs)¶
Subclass of ProcessingMechanism that evaluates the value(s) of one or more OutputPorts. See Mechanism for additional arguments and attributes.
- Parameters
monitor (List[OutputPort, ‘InputPort`, Mechanism, str, value, dict,
MonitoredOutputPortsOption] or dict) – specifies the OutputPorts, thevaluesof which will be monitored, and evaluated byfunction(see Monitor for details of specification).function (CombinationFunction, ObjectiveFunction, function or method : default LinearCombination) – specifies the function used to evaluate the values listed in
monitor<ObjectiveMechanism.monitor>` (seefunctionfor details).output_ports (list[OutputPort, value, str or dict] or dict[] : default [OUTCOME]) – specifies the OutputPorts for the Mechanism;
- monitor¶
determines the OutputPorts, the
valuesof which are monitored, and evaluated by the ObjectiveMechanism’sfunction. Each item in the list refers to an OutputPort containing the value to be monitored, with a MappingProjection from it to an corresponding InputPort listed in theinput_portsattributeNote
If any of the ObjectiveMechanism’s
input_portswere specified to shadow the InputPort of another Mechanism, and any of those shadowed InputPorts receives more than one Projection, then the list of monitored OutputPorts inmonitorwill be longer than the list of the ObjectiveMechanism’sinput_ports.- Type
ContentAddressableList[OutputPort]
- monitor_weights_and_exponents¶
each tuple in the list contains a weight and exponent associated with a corresponding InputPort listed in the ObjectiveMechanism’s
input_portsattribute; these are used by itsfunctionto parametrize the contribution that the values of each of the OuputStates monitored by the ObjectiveMechanism makes to its output (see Function)- Type
List[Tuple(float, float)]
- input_ports¶
contains the InputPorts of the ObjectiveMechanism, each of which receives a MappingProjection from the OutputPorts specified in its
monitorattribute.- Type
ContentAddressableList[InputPort]
- function¶
the function used to evaluate the values monitored by the ObjectiveMechanism. The function can be any CombinationFunction or a Python function that takes a 2d array with an arbitrary number of items or a number equal to the number of items in the ObjectiveMechanism’s variable (i.e., its number of input_ports) and returns a 1d array.
- Type
CombinationFunction, ObjectiveFunction, function, or method
- output_port¶
contains the primary OutputPort of the ObjectiveMechanism; the default is its OUTCOME OutputPort, the value of which is equal to the
valueattribute of the ObjectiveMechanism.- Type
- output_ports¶
by default, contains only the OUTCOME (primary) OutputPort of the ObjectiveMechanism.
- Type
ContentAddressableList[OutputPort]
- output_values¶
contains one item that is the value of the OUTCOME OutputPort.
- Type
2d np.array
- standard_output_ports¶
list of Standard OutputPort that includes the following in addition to the
standard_output_portsof a Mechanism:- OUTCOME1d np.array
the value of the objective or “loss” function computed by the ObjectiveMechanism’s
function
- Type
list[str]
- modulatory_mechanism¶
ModulatoryMechanism to which ObjectiveMechanism has been assigned, and to which one or more of its
output_portsproject. If that is a ControlMechanism, then the ObjectiveMechanism is assigned as itsobjective_mechanism; if it is a LearningMechanism, it is in itserror_sourcesattribute.- Type
None or ModulatoryMechanism
- _validate_params(request_set, target_set=None, context=None)¶
Validate role, monitor, amd input_ports arguments
- _instantiate_input_ports(monitor_specs=None, reference_value=None, context=None)¶
Instantiate InputPorts specified in input_ports argument of constructor or each OutputPort specified in monitor_specs
- Called during initialization as well as by _add_to_monitor(),
so must distinguish between initialization and adding to instantiated input_ports.
- During initialization, uses input_ports as specification of InputPorts to instantiate;
if none are specified, instantiates a default InputPort
- Otherwise, uses monitor_specs as specification of InputPorts to instantiate;
these will replace any existing InputPorts (including a default one)
- add_to_monitor(monitor_specs, context=None)¶
Instantiate OutputPorts to be monitored by the ObjectiveMechanism.
Used by other Components to add a Port or list of Ports to be monitored by the ObjectiveMechanism. The monitor_spec can be any of the following: - MonitoredOutputPortTuple - Mechanism; - OutputPort; - tuple specification; - Port specification dictionary; - list with any of the above. If the item is a Mechanism, its primary OutputPort is used.
- _instantiate_attributes_after_function(context=None)¶
Assign InputPort weights and exponents to ObjectiveMechanism’s function
- _instantiate_function_weights_and_exponents(context=None)¶
Assign weights and exponents to ObjectiveMechanism’s function if it has those attributes
For each, only make assignment if one or more entries in it has been assigned a value If any one value has been assigned, assign default value (1) to all other elements
- exception psyneulink.core.components.mechanisms.processing.objectivemechanism.ObjectiveMechanismError(message, component=None)¶