EVCAuxiliary

Auxiliary functions for EVCControlMechanism.

class psyneulink.library.components.mechanisms.modulatory.control.evc.evcauxiliary.ValueFunction(function=None)

Calculate the EVC for a given performance outcome and set of costs.

ValueFunction takes as its arguments an outcome (a value representing the performance of a System) and list of costs (each reflecting the cost of a ControlSignal of the controller System.controller of that System), and returns an expected value of control (EVC) based on these (along with the outcome and combination of costs used to calculate the EVC).

ValueFunction is the default for an EVCControlMechanism’s value_function attribute, and it is called by ControlSignalGridSearch (the EVCControlMechanism’s default function).

The ValueFunction’s default function calculates the EVC using the result of the function <ObjectiveMechanism.function of the EVCControlMechanism’s objective_mechanism, and two auxiliary functions specified in corresponding attributes of the EVCControlMechanism: cost_function and combine_outcome_and_cost_function. The calculation of the EVC provided by ValueFunction can be modified by customizing or replacing any of these functions, or by replacing the ValueFunction’s function itself (in the EVCControlMechanism’s value_function attribute). Replacement functions must use the same format (number and type of items) for its arguments and return values (see note).

class psyneulink.library.components.mechanisms.modulatory.control.evc.evcauxiliary.ControlSignalGridSearch(default_variable=None, params=None, function=None, owner=None)

Conduct an exhaustive search of allocation polices and return the one with the maximum EVC.

This is the default function for an EVCControlMechanism. It identifies the control_allocation with the maximum EVC by a conducting a grid search over every possible control_allocation given the allocation_samples specified for each of its ControlSignals (i.e., the Cartesian product of the allocation values specified by the allocation_samples attribute of each ControlSignal). The full set of allocation policies is stored in the EVCControlMechanism’s control_signal_search_space attribute. The EVCControlMechanism’s evaluate method is used to simulate its system under each control_allocation in control_signal_search_space, calculate the EVC for each of those policies, and return the policy with the greatest EVC. By default, only the maximum EVC is saved and returned. However, setting the save_all_values_and_policies attribute to True saves each policy and its EVC for each simulation run (in the EVCControlMechanism’s EVC_policies and EVC_values attributes, respectively). The EVC is calculated for each policy by iterating over the following steps:

  • Select an control_allocation:

    draw a successive item from control_signal_search_space in each iteration, and assign each of its values as the allocation value for the corresponding ControlSignal for that simulation of the system.

  • Simulate performance:

    execute the system under the selected control_allocation using the EVCControlMechanism’s evaluate method, and the values of its prediction_mechanisms as the input to the corresponding ORIGIN Mechanisms of the system it controls; the values of all stateful attributes of ‘Components` in the System are re-initialized to the same value prior to each simulation, so that the results for each control_allocation are based on the same initial conditions. Each simulation includes execution of the EVCControlMechanism’s objective_mechanism, which provides the result to the EVCControlMechanism. If system.recordSimulationPref is True, the results of each simulation are appended to the simulation_results attribute of system.

  • Calculate the EVC:

    call the EVCControlMechanism’s value_function to calculate the EVC for the current iteration, using three values (see Function for additional details):


    • the result of the cost function, called by the value_function, that returns the cost for the control_allocation based on the current cost associated with each of its ControlSignals;

    • the result of the combine function, called by the value_function, that returns the EVC by subtracting the cost from the outcome
  • Save the values:

    if the save_all_values_and_policies attribute is True, save allocation policy in the EVCControlMechanism’s EVC_policies attribute, and its value is saved in the EVC_values attribute; otherwise, retain only maximum EVC value.

The ControlSignalGridSearch function returns the control_allocation that yielded the maximum EVC. Its operation can be modified by customizing or replacing any or all of the functions referred to above (also see Function).

class psyneulink.library.components.mechanisms.modulatory.control.evc.evcauxiliary.PredictionMechanism(default_variable=None, size=None, function=TIME_AVERAGE_INPUT, initial_value=None, window_size=1, filter_function=None, params=None, name=None, prefs=None)

Tracks the inputs to an ORIGIN Mechanism of the system controlled by an EVCControlMechanism, and used to generate the input for that ORIGIN Mechanism in a simulated run of that System.

Creating a PredictionMechanism

PredictionMechanisms are created automatically when an EVCControlMechanism is created, one for each ORIGIN Mechanism in the system for which the EVCControlMechanism is a controller. PredictionMechanisms should not be created on their own, as their execution requires tight integration with an EVCControlMechanism and the System to which it belongs, and they will not function properly if this is not insured.

Structure

The System to which a PredictionMechanism belongs is referenced in its system attribute, and the System’s ORIGIN Mechanism which the PredictionMechanism is associated is referenced in its origin_mechanism attribute. A PredictionMechanism has the same number of InputStates as its origin_mechanism, each of which receives a Projection from the same source as the corresponding InputState of its origin_mechanism (see Prediction Mechanisms); and it has one OutputState for each of its InputStates.

Function

The function of a PredictionMechanism records the input received by its origin_mechanism, and possibly transforms this in some way. Any function can be assigned, so long as it can take as its input a value with the same format as the variable of its origin_mechanism, and returns a similarly formatted value or a list of them. The result of a PredictionMechanism’s function is provided as the input to its origin_mechanism in each trial of a simulated run of its system (see EVCControlMechanism Execution). If a PredictionMechanism’s function returns more than one item, then the EVCControlMechanism runs as many simulations as there are items, using each as the input for one simulation, and computes the mean EVC over all of them. Therefore, the function of every PredictionMechanism associated with an EVCControlMechanism must return the same number of items.

In place of a function, the following keywords can be used to specify one of three standard configurations:

  • INPUT: uses an Linear Function to simply pass the input it receives as its output, possibly modified by the values specified in the PredictionMechanism’s rate and noise arguments. If those are specified, they are assigned as the Linear function’s slope and intercept parameters, respectively.
  • TIME_AVERAGE_INPUT: uses an AdaptiveIntegrator Function to compute an exponentially weighted time-average of the input to the PredictionMechanism; the PredictionMechanism’s rate and noise arguments can be used to specify the corresponding rate and noise parameters of the function. The function returns the time-averaged input as a single item.
  • AVERAGE_INPUTS: uses a Buffer Function to compute the average of the number of preceding inputs specified in the PredictionMechanism’s window_size argument. If the rate and/or noise arguments are specified, they are applied to each item in the list before it is averaged, as follows: \(item * rate + noise\).
  • INPUT_SEQUENCE: uses a Buffer Function to maintain a running record of preceding inputs, which are returned in a list. When the EVCControlMechanism runs a simulation, one trial is run using each item in the list as the input to the PredictionMechanism’s origin_mechanism. The window_size argument can be used to specify how many preceding inputs should be maintained in the record; if the number of preceding inputs exceeds the value of window_size, the oldest is deleted and the most recent one is added to the list. If window_size is not specified (None), a record of all preceding inputs is maintained and returned. The filter_function argument can be used to specify a function that filters the list (e.g., modifies and/or deletes items); the modified list is then returned by the PredictionMechanism’s function. If the rate and/or noise arguments are specified, they are applied to each item in the list, as follows: \(item * rate + noise\); note that since the list is maintained across trials in the actual run of the system, the effects of these parameters are cumulative over trials.

Execution

A PredictionMechanism is executed each time its system is run; however, it is not executed when that System is simulated (that is, it is run using the EVCControlMechanism’s evaluate method). Thus, its inputs are updated only once per actual run of the System, and each simulated run (i.e., the simulation for each control_allocation; see EVCControlMechanism Execution) uses the exact same set of inputs – the value of the PredictionMechanisms resulting from the last actual run of its system) – so that the results of the simulated run for each control_allocation can properly be compared. If the PredictionMechanisms for an EVCControlMechanism generate a list of inputs (e.g., using INPUT_SEQUENCE), then each simulated run involves several trials, each of which uses one item from the list of each PredictionMechanism as the input to its origin_mechanism; items in the list are used as inputs in the exact same sequence for the simulated run of every control_allocation so that, once again, proper comparisons can be made between policies.

Parameters:
  • default_variable (number, list or np.ndarray : default None) – specifies the input to the Mechanism to use if none is provided in a call to its execute or run method; format must match the variable for its origin_mechanism.
  • size (int, list or np.ndarray of ints) – see size.
  • function (function, INPUT, TIME_AVERAGE_INPUT, AVERAGE_INPUTS, or INPUT_SEQUENCE : default) –
  • *TIME_AVERAGE_INPUT* – specifies the function used to generate the input provided to its origin_mechanism; the function must take as its input a single value with the same format as the variable of its origin_mechanism, and must return a similarly formatted value or a list of them (see above for additional details).
  • initial_value (value, list or np.ndarray : default None) – specifies value used to initialize the PredictionMechanism’s value attribute; if None is specified, 0 is used if the value of the PredictionMechanism’s function is numeric, and an empty list is used if INPUT_SEQUENCE is specified.
  • window_size (int : default None) – specifies number of input values to maintain when INPUT_SEQUENCE option is used for function
  • filter_function (function : default None) – specifies a function that takes a list of values, each of which has the same format as the variable of the PredictionMechanism’s origin_mechanism, and returns a list of similarly formatted values, though not necessarily of the same length (see above for additional details).
  • params (Dict[param keyword: param value] : default None) – a parameter dictionary that can be used to specify the parameters for the Mechanism, its function, and/or a custom function and its parameters. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
  • name (str : default see name) – specifies the name of the TransferMechanism.
  • prefs (PreferenceSet or specification dict : default Mechanism.classPreferences) – specifies the PreferenceSet for the TransferMechanism; see prefs for details.
system

System – the System to which the PredictionMechanism belongs.

origin_mechanism

Mechanism – the ORIGIN Mechanism with which the PredictionMechanism is associated (in its system), and the input of which it tracks.

variable

2d np.array – the input received from the PredictionMechanism’s system, that is identical to the input received by its origin_mechanism.

function

Function – used to track the inputs to the PredictionMechanism’s origin_mechanism; the default is an AdaptiveIntegrator (see above for additional details).

value

3d np.array – result returned by the PredictionMechanism’s function, and provided as input to its origin_mechanism. The format conforms to that of a System’s run method: items in the outermost dimension (axis 0) correspond to the inputs for each trial of a simulation, each of which is a 2d np.array containing the input for each InputState of the Mechanisms.