# ObjectiveMechanism¶

## Overview¶

An ObjectiveMechanism is a ProcessingMechanism that monitors the OutputPorts of one or more other ProcessingMechanisms specified in its monitor attribute, and evaluates them using its function. The result of the evaluation is placed in the ObjectiveMechanism’s OUTCOME (primary) OutputPort. ObjectiveMechanisms are typically used closely with (and often created automatically by) ModulatoryMechanisms.

## Creating an ObjectiveMechanism¶

ObjectiveMechanisms are often created automatically when other PsyNeuLink components are created (in particular, ModulatoryMechanisms, such as LearningMechanisms and ControlMechanisms). An ObjectiveMechanism can also be created directly by calling its constructor. The primary attribute used to define an ObjectiveMechanism is its monitor attribute, that is specified using the corresponding argument of its constructor as described below.

### Monitor¶

The monitor argument of an ObjectiveMechanism’s constructor specifies the OutputPorts it monitors. This takes the place of the input_ports argument used by most other forms of Mechanism, and is used by the ObjectiveMechanism to create an InputPort for each OutputPort it monitors, along with a MappingProjection from the OutputPort to that InputPort. The monitor argument takes a list of items that can include any of the forms of specification used in a standard input_ports argument. For the monitor argument, this is usually a list of OutputPorts to be monitored. However, as with a standard input_ports argument, the monitor argument can include Mechanisms (in which case their primary OutputPort is used) or the InputPort(s) of other Mechanisms (in which case the ObjectiveMechanism will be assigned Projections from all of the OutputPorts that project to the specified InputPort – that is, it will shadow their inputs). Items in the monitor argument can also be used to specify attributes of the InputPort and/or MappingProjection(s) to it, that the ObjectiveMechanism creates to monitor the specified OutputPort. In general, the value of each specified OutputPort determines the format of the variable of the InputPort that is created for it by the ObjectiveMechanism. However, this can be overridden using the ObjectiveMechanism’s default_variable or size attributes (see Mechanism InputPort specification), or by specifying a Projection from the OutputPort to the InputPort (see Input Source Specification). If an item in the monitor argument specifies an InputPort for the ObjectiveMechanism, but not the OutputPort to be monitored, the InputPort is created but will be ignored until an OutputPort (and MappingProjection from it) are specified for that InputPort.

The OutputPorts monitored by the ObjectiveMechanism are listed in its monitor attribute. When an ObjectiveMechanism is created by a ControlMechanism, these may pass a set of OutputPorts to be monitored to the ObjectiveMechanism. A ControlMechanism passes OutputPort specifications listed in its objective_mechanism argument (see Objective Mechanism).

## Structure¶

### Input¶

An ObjectiveMechanism has one InputPort for each of the OutputPorts specified in its monitor argument (see Monitor). Each InputPort receives a MappingProjection from the corresponding OutputPort, the values of which are used by the ObjectiveMechanism’s function to generate the value of its OUTCOME OutputPort. The InputPorts are listed in the ObjectiveMechanism’s input_ports attribute, and the monitored OutputPorts from which they receive projections are listed in the same order its monitor attribute.

By default, the format of the variable for each InputPort is determined by the value of the monitored OutputPort(s) to which it corresponds. However, if either the default_variable or size argument is specified in an Objective Mechanism’s constructor, or a variable is specified for an InputPort for one or more of the items in its monitor argument, then that is used as the format for the corresponding InputPort(s). This can be used to transform the value of a monitored OutputPort into different form for the variable of the InputPort (see the first example below).

If the weight and/or exponent is specified for ay item in the monitor argument of the ObjectiveMechanism’s constructor, it is assigned to the corresponding InputPort. If the ObjectiveMechanism’s function implements a weights and/or exponents attribute, the values specified is assigned to the corresponding attribute, and applied to the value of the InputPort before it is combined with that of the other InputPorts to generate the ObjectiveMechanism’s output.

### Function¶

The ObjectiveMechanism’s function uses the values of its input_ports to compute an objective (or “loss”) function, that is assigned as the value of its OUTCOME OutputPort. By default, it uses a LinearCombination function to sum the values of the values of the items in its variable. However, by assigning values to the ‘weight <InputPort.weight> and/or ‘exponent <InputPort.exponent> attributes of the corresponding InputPorts, it can be configured to calculate differences, ratios, etc. (see example below). The function can also be replaced with any CombinationFunction, or any python function that takes an 2d array as its input (with a number of items in axis 0 equal to the number of the ObjectiveMechanism’s InputPorts), and generates a 1d array as its result. If it implements weight and/or exponent attributes, those are assigned from weight and exponent attributes of its input_ports (also listed in the monitor_weights_and_exponents attribute); otherwise, they are ignored.

### Output¶

The primary OutputPort of an Objective mechanism is a 1d array, named OUTCOME, that is the result of its function (as described above).

## Execution¶

When an ObjectiveMechanism is executed, it updates its input_ports with the values of the OutputPorts listed in its monitor attribute, and then uses its function to evaluate these. The result is assigned as to its value attribute as the value of its OUTCOME (primary) OutputPort.

Examples

Specifying the variable for the InputPorts of an ObjectiveMechanism

This can be useful in some situations, and there are several ways it can be done. For example, for Reinforcement Learning, an ObjectiveMechanism is used to monitor the rewards by an action selection Mechanism (and used by a LearningMechanism to improve those predictions). However, whereas the action selection Mechanism generates a vector indicating the reward predicted by the selected action, the ObjectiveMechanism for RL simply needs to know the magnitude of the reward predicted, irrespective of the action taken; that is, it simply requires a single scalar value indicating the magnitude of the predicted reward. Thus, the vector of action-related reward predictions needs to be condensed to a single predicted value for the ObjectiveMechanism. This can be accomplished in several ways, that are illustrated in the examples below. In the first example, a TransferMechanism with the SoftMax function (and the PROB as its output format) to select an action and represent its reward prediction. This generates a vector with a single non-zero value, which designates the predicted reward for the selected action. Because the output is a vector, by default the InputPort of the ObjectiveMechanism created to monitor it will also be a vector. However, the ObjectiveMechanism requires this to be a single value, that it can compare with the value of the reward Mechanism (monitoring the feedback provided by the environment). In the example below, this is accomplished by using default_variable in the constructor of the ObjectiveMechanism to force the InputPort for the ObjectiveMechanism to have a single value:

>>> import psyneulink as pnl
>>> my_action_select_mech = pnl.TransferMechanism(default_variable=[0, 0, 0],
...                                               function=pnl.SoftMax(output=pnl.PROB),
...                                               name='Action Selection Mech')

>>> my_reward_mech = pnl.TransferMechanism(default_variable=[0],
...                                        name='Reward Mech')

>>> my_objective_mech = pnl.ObjectiveMechanism(default_variable=[[0],[0]],
...                                            monitor=[my_action_select_mech, my_reward_mech])


Note that the OutputPorts for the my_action_selection and my_reward_mech are specified in monitor. If that were the only specification, the InputPort created for my_action_select_mech would be a vector of length 3. This is overridden by specifying default_variable as an array with two single-value arrays (one corresponding to my_action_select_mech and the other to my_reward_mech). This forces the InputPort for my_action_select_mech to have only a single element which, in turn, will cause a MappingProjection to be created from my_action_select_mech to the ObjectiveMechanism’s InputPort using a FULL_CONNECTIVITY_MATRIX (the one used for AUTO_ASSIGN_MATRIX when the sender and receiver have values of different lengths). This produces the desired effect, since the action selected is the only non-zero value in the output of my_action_select_mech, and so the FULL_CONNECTIVITY_MATRIX will combine it with zeros (the other values in the vector), and so its value will be assigned as the value of the corresponding InputPort in the ObjectiveMechanism.

An alternative would be to explicitly specify the variable attribute for the InputPort created for my_action_select_mech using a InputPort specification dictionary in the monitor argument of my_objective_mech, as follows:

>>> my_objective_mech = pnl.ObjectiveMechanism(monitor=[{pnl.MECHANISM: my_action_select_mech,
...                                                      pnl.VARIABLE: [0]},
...                                                      my_reward_mech])


Note that the VARIABLE entry here specifies the variable for the InputPort of the ObjectiveMechanism created to receive a Projection from my_action_select_mech, and not my_action_select_mech itself (see Input for a full explanation).

Another way to specify the variable for the InputPort of an ObjectiveMechanism is to specify the Projections it receives from the OutputPort it monitors. The following example uses a tuple specification to assign the matrix for the MappingProjection from my_action_select_mech to the corresponding InputPort of my_objective_mech:

>>> import numpy as np
>>> my_objective_mech = pnl.ObjectiveMechanism(monitor=[(my_action_select_mech, np.ones((3,1))), my_reward_mech])


Since the matrix specified has three rows (for its inputs) and one col (for the output), it will take the length three vector provided as the output of my_action_select_mech and combine its elements into a single value that is provided to the InputPort of the ObjectiveMechanism.

A Connection tuple could also be used to specify the matrix, but this would require that additional entries (for the weight and exponent of the InputPort) which, in this case, are not necessary (see example below for how these can be used).

By default, an ObjectiveMechanism simply adds the values received by each of its InputPorts to generate its output. However, this too can be customized in a variety of ways.

The simplest way is to assign values to the weight and/or exponent attributes of its InputPorts. This can be done by placing them in a tuple specification for the OutputPort that provides value for the InputPort. In the example below, the ObjectiveMechanism used in the previous example is further customized to subtract the value of the action selected from the value of the reward:

>>> my_objective_mech = pnl.ObjectiveMechanism(default_variable = [[0],[0]],
...                                            monitor = [(my_action_select_mech, -1, 1), my_reward_mech])


This specifies that my_action_select_mech should be assigned a weight of -1 and an exponent of 1 when it is submitted to the ObjectiveMechanism’s function. Notice that the exponent had to be included, even though it is the default value; when a tuple is used, the weight and exponent values must both be specified. Notice also that my_reward_mech does not use a tuple, so it will be assigned defaults for both the weight and exponent parameters.

An ObjectiveMechanism can also be configured to monitor multiple OutputPorts of the same Mechanism. In the following example, an ObjectiveMechanism is configured to calculate the reward rate for a DDM Mechanism, by specifying OutputPorts for the DDM that report its response time and accuracy:

>>> my_decision_mech = pnl.DDM(output_ports=[pnl.RESPONSE_TIME,
...                                           pnl.PROBABILITY_UPPER_THRESHOLD])

>>> my_objective_mech = pnl.ObjectiveMechanism(monitor=[
...                                              my_reward_mech,
...                                              my_decision_mech.output_ports[pnl.PROBABILITY_UPPER_THRESHOLD],
...                                              (my_decision_mech.output_ports[pnl.RESPONSE_TIME], 1, -1)])


This specifies that the ObjectiveMechanism should multiply the value of my_reward_mech’s primary OutputPort by the value of my_decision_mech’s PROBABILITY_UPPER_THRESHOLD, and divide the result by my_decision_mech’s RESPONSE_TIME value. The two OutputPorts of my_decision_mech are referenced as items in the output_ports list of my_decision_mech. However, a 2-item (Port name, Mechanism) tuple can be used to reference them more simply, as follows:

>>> my_objective_mech = pnl.ObjectiveMechanism(monitor=[
...                                           my_reward_mech,
...                                           (pnl.PROBABILITY_UPPER_THRESHOLD, my_decision_mech),
...                                           ((pnl.RESPONSE_TIME, my_decision_mech), 1, -1)])


Customizing the ObjectiveMechanism’s function

In the examples above, the weights and exponents assigned to the InputPorts are passed to the ObjectiveMechanism’s function for use in combining their values. The same can be accomplished by specifying the relevant parameter(s) of the function itself, as in the following example:

>>> my_objective_mech = pnl.ObjectiveMechanism(default_variable = [[0],[0]],
...                                            monitor = [my_action_select_mech, my_reward_mech],
...                                            function=pnl.LinearCombination(weights=[[-1], [1]]))


Here, the weights parameter of the LinearCombination function is specified directly, with two values [-1] and [1] corresponding to the two items in monitor (and default_variable). This will multiply the value from my_action_select_mech by -1 before adding it to (and thus subtracting it from) the value of my_reward_mech. Notice that the weight for my_reward_mech had to be specified, even though it is using the default value (1); whenever a weight and/or exponent parameter is specified, there must be an entry for every item of the function’s variable. However, the exponents did not need to be specified, as it is not used. However it, and the operation parameters of LinearCombination can also be used to multiply and divide quantities.

## Class Reference¶

class psyneulink.core.components.mechanisms.processing.objectivemechanism.ObjectiveMechanism(monitor, function=LinearCombination, output_ports=OUTCOME)

Subclass of ProcessingMechanism that evaluates the value(s) of one or more OutputPorts. See Mechanism for additional arguments and attributes.

Parameters
• monitor (List[OutputPort, ‘InputPort, Mechanism, str, value, dict, MonitoredOutputPortsOption] or dict) – specifies the OutputPorts, the values of which will be monitored, and evaluated by function (see Monitor for details of specification).

• function (CombinationFunction, ObjectiveFunction, function or method : default LinearCombination) – specifies the function used to evaluate the values listed in monitor <ObjectiveMechanism.monitor> (see function for details).

• output_ports (list[OutputPort, value, str or dict] or dict[] : default [OUTCOME]) – specifies the OutputPorts for the Mechanism;

• role (LEARNING or CONTROL : default None) – specifies if the ObjectiveMechanism is being used for learning or control (see role for details).

monitor

determines the OutputPorts, the values of which are monitored, and evaluated by the ObjectiveMechanism’s function. Each item in the list refers to an OutputPort containing the value to be monitored, with a MappingProjection from it to an corresponding InputPort listed in the input_ports attribute

Note

If any of the ObjectiveMechanism’s input_ports were specified to shadow the InputPort of another Mechanism, and any of those shadowed InputPorts receives more than one Projection, then the list of monitored OutputPorts in monitor will be longer than the list of the ObjectiveMechanism’s input_ports.

Type

monitor_weights_and_exponents

each tuple in the list contains a weight and exponent associated with a corresponding InputPort listed in the ObjectiveMechanism’s input_ports attribute; these are used by its function to parametrize the contribution that the values of each of the OuputStates monitored by the ObjectiveMechanism makes to its output (see Function)

Type

List[Tuple(float, float)]

input_ports

contains the InputPorts of the ObjectiveMechanism, each of which receives a MappingProjection from the OutputPorts specified in its monitor attribute.

Type

function

the function used to evaluate the values monitored by the ObjectiveMechanism. The function can be any PsyNeuLink CombinationFunction or a Python function that takes a 2d array with an arbitrary number of items or a number equal to the number of items in the ObjectiveMechanism’s variable (i.e., its number of input_ports) and returns a 1d array.

Type

CombinationFunction, ObjectiveFunction, function, or method

role

specifies whether the ObjectiveMechanism is used for learning in a Composition (in conjunction with a Learning Mechanism), or for control in a Composition (in conjunction with a ControlMechanism).

Type

None, LEARNING or CONTROL

output_port

contains the primary OutputPort of the ObjectiveMechanism; the default is its OUTCOME OutputPort, the value of which is equal to the value attribute of the ObjectiveMechanism.

Type

OutputPort

output_ports

by default, contains only the OUTCOME (primary) OutputPort of the ObjectiveMechanism.

Type

output_values

contains one item that is the value of the OUTCOME OutputPort.

Type

2d np.array

standard_output_ports

list of Standard OutputPort that includes the following in addition to the standard_output_ports of a Mechanism:

OUTCOME1d np.array

the value of the objective or “loss” function computed by the ObjectiveMechanism’s function

Type

list[str]

_validate_params(request_set, target_set=None, context=None)

Validate role, monitor, amd input_ports arguments

_instantiate_input_ports(monitor_specs=None, reference_value=None, context=None)

Instantiate InputPorts specified in input_ports argument of constructor or each OutputPort specified in monitor_specs

Called during initialization as well as by _add_to_monitor(),

so must distinguish between initialization and adding to instantiated input_ports.

During initialization, uses input_ports as specification of InputPorts to instantiate;

if none are specified, instantiates a default InputPort

Otherwise, uses monitor_specs as specification of InputPorts to instantiate;

these will replace any existing InputPorts (including a default one)

add_to_monitor(monitor_specs, context=None)

Instantiate OutputPorts to be monitored by the ObjectiveMechanism.

Used by other Components to add a Port or list of Ports to be monitored by the ObjectiveMechanism. The monitor_spec can be any of the following: - MonitoredOutputPortTuple - Mechanism; - OutputPort; - tuple specification; - Port specification dictionary; - list with any of the above. If the item is a Mechanism, its primary OutputPort is used.

_instantiate_attributes_after_function(context=None)

_instantiate_function_weights_and_exponents(context=None)
exception psyneulink.core.components.mechanisms.processing.objectivemechanism.ObjectiveMechanismError(error_value)