Auxiliary functions for EVCControlMechanism.
Calculate the EVC for a given performance outcome and set of costs.
ValueFunction takes as its arguments an outcome (a value representing the performance of a System) and list of costs (each reflecting the
costof a ControlSignal of the
controller System.controllerof that System), and returns an expected value of control (EVC) based on these (along with the outcome and aggregation of costs used to calculate the EVC).
The ValueFunction’s default
functioncalculates the EVC using the result of the
function <ObjectiveMechanism.functionof the EVCControlMechanism’s
objective_mechanism, and two auxiliary functions specified in corresponding attributes of the EVCControlMechanism:
combine_outcome_and_cost_function. The calculation of the EVC provided by ValueFunction can be modified by customizing or replacing any of these functions, or by replacing the ValueFunction’s
functionitself (in the EVCControlMechanism’s
value_functionattribute). Replacement functions must use the same format (number and type of items) for its arguments and return values (see note).
Calculate the EVC as follows:
- call the
cost_functionfor the EVCControlMechanism specified in the controller argument, to combine the list of costs specified in the costs argument into a single cost value;
- call the
combine_outcome_and_cost_functionfor the EVCControlMechanism specified in the controller argument, to combine the value specified in the outcome argument with the value returned by the
- return the results in a three item tuple: (EVC, outcome and cost).
- controller (EVCControlMechanism) – the EVCControlMechanism for which the EVC is to be calculated; this is required so that the controller’s
combine_outcome_and_cost_functionfunctions can be called.
- outcome (value : default float) – should represent the outcome of performance of the System for which an
control_allocationis being evaluated.
- costs (list or array of values : default 1d np.array of floats) – each item should be the
costof one of the controller’s ControlSignals.
(EVC, outcome, cost)
Tuple(float, float, float)
- call the
Conduct an exhaustive search of allocation polices and return the one with the maximum EVC.
This is the default
functionfor an EVCControlMechanism. It identifies the
control_allocationwith the maximum EVC by a conducting a grid search over every possible
allocation_samplesspecified for each of its ControlSignals (i.e., the Cartesian product of the
allocationvalues specified by the
allocation_samplesattribute of each ControlSignal). The full set of allocation policies is stored in the EVCControlMechanism’s
control_signal_search_spaceattribute. The EVCControlMechanism’s
evaluatemethod is used to simulate its
control_signal_search_space, calculate the EVC for each of those policies, and return the policy with the greatest EVC. By default, only the maximum EVC is saved and returned. However, setting the
Truesaves each policy and its EVC for each simulation run (in the EVCControlMechanism’s
EVC_valuesattributes, respectively). The EVC is calculated for each policy by iterating over the following steps:
Select an control_allocation:
systemunder the selected
control_allocationusing the EVCControlMechanism’s
evaluatemethod, and the
values of its
prediction_mechanismsas the input to the corresponding
ORIGINMechanisms of the
systemit controls; the values of all stateful attributes of ‘Components` in the System are re-initialized to the same value prior to each simulation, so that the results for each
control_allocationare based on the same initial conditions. Each simulation includes execution of the EVCControlMechanism’s
objective_mechanism, which provides the result to the EVCControlMechanism. If
True, the results of each simulation are appended to the
Calculate the EVC:
call the EVCControlMechanism’s
value_functionto calculate the EVC for the current iteration, using three values (see Function for additional details):
- the EVCControlMechanism’s
input, which is the result of its
function) and provides an evaluation of the outcome of processing in the
systemunder the current
- the result of the
costfunction, called by the
value_function, that returns the cost for the
control_allocationbased on the current
costassociated with each of its ControlSignals;
- the result of the
combinefunction, called by the
value_function, that returns the EVC by subtracting the cost from the outcome
- the EVCControlMechanism’s
Save the values:
The ControlSignalGridSearch function returns the
control_allocationthat yielded the maximum EVC. Its operation can be modified by customizing or replacing any or all of the functions referred to above (also see Function).
Grid search combinations of control_signals in specified allocation ranges to find one that maximizes EVC
- Called by ControlSignalGridSearch.
- Call System.execute for each
- Store an array of values for output_states in
monitored_output_states(i.e., the input_states in
input_states) for each
_compute_EVCfor each control_allocation to calculate the EVC, identify the maximum, and assign to
control_allocation(outputState.values) corresponding to EVC_max.
- Set value for each control_signal (outputState.value) to the values in
- Return an control_allocation.
- runtime_params is used for self.__execute (that calculates the EVC for each call to System.execute); it is NOT used for System.execute – that uses the runtime_params provided for the Mechanisms in each Process.configuration
Return (2D np.array): value of outputState for each monitored state (in self.input_states) for EVC_max
Creating a PredictionMechanism
PredictionMechanisms are created automatically when an EVCControlMechanism is created, one for each
ORIGINMechanism in the
systemfor which the EVCControlMechanism is a
controller. PredictionMechanisms should not be created on their own, as their execution requires tight integration with an EVCControlMechanism and the System to which it belongs, and they will not function properly if this is not insured.
The System to which a PredictionMechanism belongs is referenced in its
systemattribute, and the System’s
ORIGINMechanism which the PredictionMechanism is associated is referenced in its
origin_mechanismattribute. A PredictionMechanism has the same number of InputStates as its
origin_mechanism, each of which receives a Projection from the same source as the corresponding InputState of its
origin_mechanism(see Prediction Mechanisms); and it has one OutputState for each of its InputStates.
functionof a PredictionMechanism records the input received by its
origin_mechanism, and possibly transforms this in some way. Any function can be assigned, so long as it can take as its input a value with the same format as the
origin_mechanism, and returns a similarly formatted value or a list of them. The result of a PredictionMechanism’s
functionis provided as the input to its
origin_mechanismin each trial of a simulated run of its
system(see EVCControlMechanism Execution). If a PredictionMechanism’s
functionreturns more than one item, then the EVCControlMechanism runs as many simulations as there are items, using each as the input for one simulation, and computes the mean EVC over all of them. Therefore, the
functionof every PredictionMechanism associated with an EVCControlMechanism must return the same number of items.
In place of a function, the following keywords can be used to specify one of three standard configurations:
- INPUT: uses an
LinearFunction to simply pass the input it receives as its output, possibly modified by the values specified in the PredictionMechanism’s rate and noise arguments. If those are specified, they are assigned as the
- TIME_AVERAGE_INPUT: uses an
AdaptiveIntegratorFunction to compute an exponentially weighted time-average of the input to the PredictionMechanism; the PredictionMechanism’s rate and noise arguments can be used to specify the corresponding
noiseparameters of the function. The function returns the time-averaged input as a single item.
- AVERAGE_INPUTS: uses a
BufferFunction to compute the average of the number of preceding inputs specified in the PredictionMechanism’s window_size argument. If the rate and/or noise arguments are specified, they are applied to each item in the list before it is averaged, as follows: \(item * rate + noise\).
- INPUT_SEQUENCE: uses a
BufferFunction to maintain a running record of preceding inputs, which are returned in a list. When the EVCControlMechanism runs a simulation, one trial is run using each item in the list as the input to the PredictionMechanism’s
origin_mechanism. The window_size argument can be used to specify how many preceding inputs should be maintained in the record; if the number of preceding inputs exceeds the value of window_size, the oldest is deleted and the most recent one is added to the list. If window_size is not specified (
None), a record of all preceding inputs is maintained and returned. The filter_function argument can be used to specify a function that filters the list (e.g., modifies and/or deletes items); the modified list is then returned by the PredictionMechanism’s
function. If the rate and/or noise arguments are specified, they are applied to each item in the list, as follows: \(item * rate + noise\); note that since the list is maintained across trials in the actual run of the
system, the effects of these parameters are cumulative over trials.
A PredictionMechanism is executed each time its
systemis run; however, it is not executed when that System is simulated (that is, it is run using the EVCControlMechanism’s
evaluatemethod). Thus, its inputs are updated only once per actual run of the System, and each simulated run (i.e., the simulation for each
control_allocation; see EVCControlMechanism Execution) uses the exact same set of inputs – the value of the PredictionMechanisms resulting from the last actual run of its
system) – so that the results of the simulated run for each
control_allocationcan properly be compared. If the PredictionMechanisms for an EVCControlMechanism generate a list of inputs (e.g., using INPUT_SEQUENCE), then each simulated run involves several trials, each of which uses one item from the list of each PredictionMechanism as the input to its
origin_mechanism; items in the list are used as inputs in the exact same sequence for the simulated run of every
control_allocationso that, once again, proper comparisons can be made between policies.
- default_variable (number, list or np.ndarray : default None) – specifies the input to the Mechanism to use if none is provided in a call to its
runmethod; format must match the
- size (int, list or np.ndarray of ints) – see
- function (function, INPUT, TIME_AVERAGE_INPUT, AVERAGE_INPUTS, or INPUT_SEQUENCE : default) –
- *TIME_AVERAGE_INPUT* – specifies the function used to generate the input provided to its
origin_mechanism; the function must take as its input a single value with the same format as the
origin_mechanism, and must return a similarly formatted value or a list of them (see above for additional details).
- initial_value (value, list or np.ndarray : default None) – specifies value used to initialize the PredictionMechanism’s
Noneis specified, 0 is used if the
valueof the PredictionMechanism’s
functionis numeric, and an empty list is used if INPUT_SEQUENCE is specified.
- window_size (int : default None) – specifies number of input values to maintain when INPUT_SEQUENCE option is used for
- filter_function (function : default None) – specifies a function that takes a list of values, each of which has the same format as the
variableof the PredictionMechanism’s
origin_mechanism, and returns a list of similarly formatted values, though not necessarily of the same length (see above for additional details).
- params (Dict[param keyword: param value] : default None) – a parameter dictionary that can be used to specify the parameters for
the Mechanism, its
function, and/or a custom function and its parameters. Values specified for parameters in the dictionary override any assigned to those parameters in arguments of the constructor.
- name (str : default see
name) – specifies the name of the TransferMechanism.
- prefs (PreferenceSet or specification dict : default Mechanism.classPreferences) – specifies the
PreferenceSetfor the TransferMechanism; see
System – the System to which the PredictionMechanism belongs.
3d np.array – result returned by the PredictionMechanism’s
function, and provided as input to its
origin_mechanism. The format conforms to that of a System’s
runmethod: items in the outermost dimension (axis 0) correspond to the inputs for each trial of a simulation, each of which is a 2d np.array containing the input for each InputState of the Mechanisms.
- INPUT: uses an