EVCAuxiliary

Auxiliary functions for EVCControlMechanism.

class psyneulink.library.subsystems.evc.evcauxiliary.ValueFunction(function=None)

Calculate the EVC for a given performance outcome and set of costs.

ValueFunction takes as its arguments an outcome (a value representing the performance of a System) and list of costs (each reflecting the cost of a ControlSignal of the controller System.controller of that System), and returns an expected value of control (EVC) based on these (along with the outcome and aggregation of costs used to calculate the EVC).

ValueFunction is the default for an EVCControlMechanism’s value_function attribute, and it is called by ControlSignalGridSearch (the EVCControlMechanism’s default function).

The ValueFunction’s default function calculates the EVC using the result of the function <ObjectiveMechanism.function of the EVCControlMechanism’s objective_mechanism, and two auxiliary functions specified in corresponding attributes of the EVCControlMechanism: cost_function and combine_outcome_and_cost_function. The calculation of the EVC provided by ValueFunction can be modified by customizing or replacing any of these functions, or by replacing the ValueFunction’s function itself (in the EVCControlMechanism’s value_function attribute). Replacement functions must use the same format (number and type of items) for its arguments and return values (see note).

function(controller, outcome, costs)

Calculate the EVC as follows:

  • call the cost_function for the EVCControlMechanism specified in the controller argument, to combine the list of costs specified in the costs argument into a single cost value;
  • call the combine_outcome_and_cost_function for the EVCControlMechanism specified in the controller argument, to combine the value specified in the outcome argument with the value returned by the cost_function;
  • return the results in a three item tuple: (EVC, outcome and cost).
Parameters:
Returns:

(EVC, outcome, cost)

Return type:

Tuple(float, float, float)

class psyneulink.library.subsystems.evc.evcauxiliary.ControlSignalGridSearch(default_variable=None, params=None, function=None, owner=None)

Conduct an exhaustive search of allocation polices and return the one with the maximum EVC.

This is the default function for an EVCControlMechanism. It identifies the allocation_policy with the maximum EVC by a conducting a grid search over every possible allocation_policy given the allocation_samples specified for each of its ControlSignals (i.e., the Cartesian product of the allocation values specified by the allocation_samples attribute of each ControlSignal). The full set of allocation policies is stored in the EVCControlMechanism’s control_signal_search_space attribute. The EVCControlMechanism’s run_simulation method is used to simulate its system under each allocation_policy in control_signal_search_space, calculate the EVC for each of those policies, and return the policy with the greatest EVC. By default, only the maximum EVC is saved and returned. However, setting the save_all_values_and_policies attribute to True saves each policy and its EVC for each simulation run (in the EVCControlMechanism’s EVC_policies and EVC_values attributes, respectively). The EVC is calculated for each policy by iterating over the following steps:

  • Select an allocation_policy:

    draw a successive item from control_signal_search_space in each iteration, and assign each of its values as the allocation value for the corresponding ControlSignal for that simulation of the system.

  • Simulate performance:

    execute the system under the selected allocation_policy using the EVCControlMechanism’s run_simulation method, and the values of its prediction_mechanisms as the input to the corresponding ORIGIN Mechanisms of the system it controls; the values of all stateful attributes of ‘Components` in the System are re-initialized to the same value prior to each simulation, so that the results for each allocation_policy are based on the same initial conditions. Each simulation includes execution of the EVCControlMechanism’s objective_mechanism, which provides the result to the EVCControlMechanism. If system.recordSimulationPref is True, the results of each simulation are appended to the simulation_results attribute of system.

  • Calculate the EVC:

    call the EVCControlMechanism’s value_function to calculate the EVC for the current iteration, using three values (see Function for additional details):


    • the result of the cost function, called by the value_function, that returns the cost for the allocation_policy based on the current cost associated with each of its ControlSignals;

    • the result of the combine function, called by the value_function, that returns the EVC by subtracting the cost from the outcome
  • Save the values:

    if the save_all_values_and_policies attribute is True, save allocation policy in the EVCControlMechanism’s EVC_policies attribute, and its value is saved in the EVC_values attribute; otherwise, retain only maximum EVC value.

The ControlSignalGridSearch function returns the allocation_policy that yielded the maximum EVC. Its operation can be modified by customizing or replacing any or all of the functions referred to above (also see Function).

function(controller=None, variable=None, runtime_params=None, params=None, context=None)

Grid search combinations of control_signals in specified allocation ranges to find one that maximizes EVC

Note

  • runtime_params is used for self.__execute (that calculates the EVC for each call to System.execute); it is NOT used for System.execute – that uses the runtime_params provided for the Mechanisms in each Process.configuration

Return (2D np.array): value of outputState for each monitored state (in self.input_states) for EVC_max

class psyneulink.library.subsystems.evc.evcauxiliary.PredictionMechanism(default_variable=None, size=None, function=TIME_AVERAGE_INPUT, initial_value=None, window_size=1, filter_function=None, params=None, name=None, prefs=None)

Tracks the inputs to an ORIGIN Mechanism of the system controlled by an EVCControlMechanism, and used to generate the input for that ORIGIN Mechanism in a simulated run of that System.

Creating a PredictionMechanism

PredictionMechanisms are created automatically when an EVCControlMechanism is created, one for each ORIGIN Mechanism in the system for which the EVCControlMechanism is a controller. PredictionMechanisms should not be created on their own, as their execution requires tight integration with an EVCControlMechanism and the System to which it belongs, and they will not function properly if this is not insured.

Structure

The System to which a PredictionMechanism belongs is referenced in its system attribute, and the System’s ORIGIN Mechanism which the PredictionMechanism is associated is referenced in its origin_mechanism attribute. A PredictionMechanism has the same number of InputStates as its origin_mechanism, each of which receives a Projection from the same source as the corresponding InputState of its origin_mechanism (see Prediction Mechanisms); and it has one OutputState for each of its InputStates.

Function

The function of a PredictionMechanism records the input received by its origin_mechanism, and possibly transforms this in some way. Any function can be assigned, so long as it can take as its input a value with the same format as the variable of its origin_mechanism, and returns a similarly formatted value or a list of them. The result of a PredictionMechanism’s function is provided as the input to its origin_mechanism in each trial of a simulated run of its system (see EVCControlMechanism Execution). If a PredictionMechanism’s function returns more than one item, then the EVCControlMechanism runs as many simulations as there are items, using each as the input for one simulation, and computes the mean EVC over all of them. Therefore, the function of every PredictionMechanism associated with an EVCControlMechanism must return the same number of items.

In place of a function, the following keywords can be used to specify one of three standard configurations:

  • TIME_AVERAGE_INPUT: uses an AdaptiveIntegrator Function to compute an exponentially weighted time-average of the input to the PredictionMechanism; the PredictionMechanism’s rate and noise arguments can be used to specify the corresponding rate and noise parameters of the function. The function returns the time-averaged input as a single item.
  • AVERAGE_INPUTS: uses a Buffer Function to compute the average of the number of preceding inputs specified in the PredictionMechanism’s window_size argument. If the rate and/or noise arguments are specified, they are applied to each item in the list before it is averaged, as follows: \(item * rate + noise\).
  • INPUT_SEQUENCE: uses a Buffer Function to maintain a running record of preceding inputs, which are returned in a list. When the EVCControlMechanism runs a simulation, one trial is run using each item in the list as the input to the PredictionMechanism’s origin_mechanism. The window_size argument can be used to specify how many preceding inputs should be maintained in the record; if the number of preceding inputs exceeds the value of window_size, the oldest is deleted and the most recent one is added to the list. If window_size is not specified (None), a record of all preceding inputs is maintained and returned. The filter_function argument can be used to specify a function that filters the list (e.g., modifies and/or deletes items); the modified list is then returned by the PredictionMechanism’s function. If the rate and/or noise arguments are specified, they are applied to each item in the list, as follows: \(item * rate + noise\); note that since the list is maintained across trials in the actual run of the system, the effects of these parameters are cumulative over trials.

Execution

A PredictionMechanism is executed each time its system is run; however, it is not executed when that System is simulated (that is, it is run using the EVCControlMechanism’s run_simulation method). Thus, its inputs are updated only once per actual run of the System, and each simulated run (i.e., the simulation for each allocation_policy; see EVCControlMechanism Execution) uses the exact same set of inputs – the value of the PredictionMechanisms resulting from the last actual run of its system) – so that the results of the simulated run for each allocation_policy can properly be compared. If the PredictionMechanisms for an EVCControlMechanism generate a list of inputs (e.g., using INPUT_SEQUENCE), then each simulated run involves several trials, each of which uses one item from the list of each PredictionMechanism as the input to its origin_mechanism; items in the list are used as inputs in the exact same sequence for the simulated run of every allocation_policy so that, once again, proper comparisons can be made between policies.

Parameters:
  • default_variable (number, list or np.ndarray : default None) – specifies the input to the Mechanism to use if none is provided in a call to its execute or run method; format must match the variable for its origin_mechanism.
  • size (int, list or np.ndarray of ints) – see size.
  • function (function, TIME_AVERAGE_INPUT, AVERAGE_INPUTS, or INPUT_SEQUENCE : default TIME_AVERAGE_INPUT) – specifies the function used to generate the input provided to its origin_mechanism; the function must take as its input a single value with the same format as the variable of its origin_mechanism, and must return a similarly formatted value or a list of them (see above for additional details).
  • initial_value (value, list or np.ndarray : default None) – specifies value used to initialize the PredictionMechanism’s value attribute; if None is specified, 0 is used if the value of the PredictionMechanism’s function is numeric, and an empty list is used if INPUT_SEQUENCE is specified.
  • window_size (int : default None) – specifies number of input values to maintain when INPUT_SEQUENCE option is used for function
  • filter_function (function : default None) – specifies a function that takes a list of values, each of which has the same format as the variable of the PredictionMechanism’s origin_mechanism, and returns a list of similarly formatted values, though not necessarily of the same length (see above for additional details).
  • params (Dict[param keyword: param value] : default None) – a parameter dictionary that can be used to specify the parameters for the Mechanism, its function, and/or a custom function and its parameters. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
  • name (str : default see name) – specifies the name of the TransferMechanism.
  • prefs (PreferenceSet or specification dict : default Mechanism.classPreferences) – specifies the PreferenceSet for the TransferMechanism; see prefs for details.
system

System – the System to which the PredictionMechanism belongs.

origin_mechanism

Mechanism – the ORIGIN Mechanism with which the PredictionMechanism is associated (in its system), and the input of which it tracks.

variable

2d np.array – the input received from the PredictionMechanism’s system, that is identical to the input received by its origin_mechanism.

function

Function – used to track the inputs to the PredictionMechanism’s origin_mechanism; the default is an AdaptiveIntegrator (see above for additional details).

value

3d np.array – result returned by the PredictionMechanism’s function, and provided as input to its origin_mechanism. The format conforms to that of a System’s run method: items in the outermost dimension (axis 0) correspond to the inputs for each trial of a simulation, each of which is a 2d np.array containing the input for each InputState of the Mechanisms.