Auxiliary functions for EVCControlMechanism.

class psyneulink.library.subsystems.evc.evcauxiliary.ValueFunction(function=None)

Calculate the EVC for a given performance outcome and set of costs.

ValueFunction takes as its arguments an outcome (a value representing the performance of a System) and list of costs (each reflecting the cost of a ControlSignal of the controller System.controller of that System), and returns an expected value of control (EVC) based on these (along with the outcome and aggregation of costs used to calculate the EVC).

ValueFunction is the default for an EVCControlMechanism’s value_function attribute, and it is called by ControlSignalGridSearch (the EVCControlMechanism’s default function).

The ValueFunction’s default function calculates the EVC using the result of the function <ObjectiveMechanism.function of the EVCControlMechanism’s objective_mechanism, and two auxiliary functions specified in corresponding attributes of the EVCControlMechanism: cost_function and combine_outcome_and_cost_function. The calculation of the EVC provided by ValueFunction can be modified by customizing or replacing any of these functions, or by replacing the ValueFunction’s function itself (in the EVCControlMechanism’s value_function attribute). Replacement functions must use the same format (number and type of items) for its arguments and return values (see note).

function(controller, outcome, costs)

Calculate the EVC as follows:

  • call the cost_function for the EVCControlMechanism specified in the controller argument, to combine the list of costs specified in the costs argument into a single cost value;
  • call the combine_outcome_and_cost_function for the EVCControlMechanism specified in the controller argument, to combine the value specified in the outcome argument with the value returned by the cost_function;
  • return the results in a three item tuple: (EVC, outcome and cost).

(EVC, outcome, cost)

Return type:

Tuple(float, float, float)

class psyneulink.library.subsystems.evc.evcauxiliary.ControlSignalGridSearch(default_variable=None, params=None, function=None, owner=None, context=None)

Conduct an exhaustive search of allocation polices and return the one with the maximum EVC.

This is the default function for an EVCControlMechanism. It identifies the allocation_policy with the maximum EVC by a conducting a grid search over every possible allocation_policy given the allocation_samples specified for each of its ControlSignals (i.e., the Cartesian product of the allocation values specified by the allocation_samples attribute of each ControlSignal). The full set of allocation policies is stored in the EVCControlMechanism’s control_signal_search_space attribute. The EVCControlMechanism’s run_simulation method is used to simulate its system under each allocation_policy in control_signal_search_space, calculate the EVC for each of those policies, and return the policy with the greatest EVC. By default, only the maximum EVC is saved and returned. However, setting the save_all_values_and_policies attribute to True saves each policy and its EVC for each simulation run (in the EVCControlMechanism’s EVC_policies and EVC_values attributes, respectively). The EVC is calculated for each policy by iterating over the following steps:

  • Select an allocation_policy:

    draw a successive item from control_signal_search_space in each iteration, and assign each of its values as the allocation value for the corresponding ControlSignal for that simulation of the system.

  • Simulate performance:

    execute the system under the selected allocation_policy using the EVCControlMechanism’s run_simulation method, and the values of its prediction_mechanisms as the input to the system; these use the history of previous trials to generate an average expected input for each ORIGIN Mechanism of the system.

  • Calculate the EVC:

    call the EVCControlMechanism’s value_function to calculate the EVC for the current iteration, using three values (see Function for additional details):

    • the result of the cost function, called by the value_function, that returns the cost for the allocation_policy based on the current cost associated with each of its ControlSignals;

    • the result of the combine function, called by the value_function, that returns the EVC by subtracting the cost from the outcome
  • Save the values:

    if the save_all_values_and_policies attribute is True, save allocation policy in the EVCControlMechanism’s EVC_policies attribute, and its value is saved in the EVC_values attribute; otherwise, retain only maximum EVC value.

The ControlSignalGridSearch function returns the allocation_policy that yielded the maximum EVC. Its operation can be modified by customizing or replacing any or all of the functions referred to above (also see Function).

function(controller=None, variable=None, runtime_params=None, params=None, context=None)

Grid search combinations of control_signals in specified allocation ranges to find one that maximizes EVC

Note: * runtime_params is used for self.__execute (that calculates the EVC for each call to System.execute);

it is NOT used for System.execute – that uses the runtime_params provided for the Mechanisms in each

Return (2D np.array): value of outputState for each monitored state (in self.input_states) for EVC_max