ParameterEstimationComposition¶

Contents¶

Overview

Data Fitting

Parameter Optimization

Supported Optimizers

Usage Examples

Class Reference

Overview¶

A ParameterEstimationComposition is a subclass of Composition that estimates specified parameters of a model Composition. It can fit the outputs of the model to empirical data (Data Fitting) by maximizing likelihood using kernel density estimation (KDE), or it can optimize a user-provided scalar objective_function (Parameter Optimization). In either case, when the ParameterEstimationComposition is run with a given set of inputs, it sets its optimized_parameter_values attribute to the parameter values that best satisfy the fitting or optimization objective. The results attribute is also set to the optimal parameter values. The arguments below configure a ParameterEstimationComposition for either Data Fitting or Parameter Optimization; later sections describe arguments specific to each mode.

model - specifies the Composition whose parameters are to be estimated.

Note

Neither the controller nor any of its associated arguments can be specified in the constructor for a ParameterEstimationComposition; this is constructed automatically using the arguments described below.

parameters - specifies the parameters of the model to be estimated. These are specified in a dict, in which the key of each entry specifies a parameter to estimate, and its value is a list values to sample for that parameter.

outcome_variables - specifies the OUTPUT Nodes of the model, the values of which are used to evaluate the fit of the different combinations of parameter values sampled. An important limitation of the PEC is that the outcome_variables must be a subset of the output ports of the model’s terminal Mechanism.

optimization_function - specifies the function used to search over the combinations of parameter values to be estimated. This is usually a PECOptimizationFunction, which supports SciPy differential evolution and Optuna samplers or studies. Passing "differential_evolution" is a shorthand for PECOptimizationFunction(method="differential_evolution").

num_estimates - specifies the number of independent samples that are estimated for a given combination of parameter values.

num_trials_per_estimate - specifies the number of trials executed when the model is run for each estimate of a combination of parameter values. Typically, this can be left unspecified and the model will be run until all trials of inputs are exhausted.

Data Fitting¶

The ParameterEstimationComposition can be used to find a set of parameters for the model such that, when it is run with a given set of inputs, its results best match (maximum likelihood) a specified set of empirical data. This requires that the data argument be specified:

data - specifies the data to which the outcome_variables are fit in the estimation process. They must be in a format that aligns with the specification of the outcome_variables. The parameter data should be a pandas DataFrame where each column corresponds to one of the outcome_variables. If one of the outcome variables should be treated as a categorical variable (e.g. a decision value in a two-alternative forced choice task modeled by a DDM), then it should be specified as a pandas Categorical variable.

objective_function - A function that computes the sum of the log likelihood of the data is automatically assigned for data fitting purposes and should not need to be specified. This function uses a kernel density estimation of the data to compute the likelihood of the data given the model. If you would like to use your own estimation of the likelihood, see Parameter Optimization below.

Warning

The objective_function argument should NOT be specified for data fitting; specifying both the data and objective_function arguments generates an error.

Parameter Optimization¶

The ParameterEstimationComposition can be used to find a set of parameters for the model such that, when it is run with a given set of inputs, its results either maximize or minimize the objective_function, as determined by the optimization_function. This requires that the objective_function argument be specified:

objective_function - specifies a function used to evaluate the values of the outcome_variables, according to which combinations of parameters are assessed; this must be a Callable that takes a 3d array as its only argument, the shape of which will be (num_trials, num_estimates, number of outcome_variables). The function should specify how to aggregate the value of each outcome_variable over num_estimates and/or num_trials if either is greater than 1.

Warning

The data argument should NOT be specified for parameter optimization; specifying both the objective_function and the data arguments generates an error.

Supported Optimizers¶

A ParameterEstimationComposition uses PECOptimizationFunction to search candidate parameter values. The search method is configured by the method argument of PECOptimizationFunction; it is separate from the objective_function that scores each candidate.

Supported PECOptimizationFunction methods include:

"differential_evolution" Differential evolution as implemented by SciPy.
Optuna sampler instances, such as optuna.samplers.RandomSampler(seed=0) or optuna.samplers.CmaEsSampler(seed=0).
Optuna sampler classes, such as optuna.samplers.TPESampler. In this form, keyword arguments for the sampler can be passed using optuna_kwargs.
Optuna Study objects.

The max_iterations argument specifies the number of generations for "differential_evolution" and the number of trials for Optuna methods. The direction argument controls whether the objective is minimized or maximized.

For Dask/SLURM execution of data-fitting runs, pass distributed=True to either ParameterEstimationComposition or PECOptimizationFunction; see Distributed Fitting.

Usage Examples¶

The examples below show the PEC-specific configuration. They assume that comp is the Composition being fit, decision is an output Mechanism, trial_inputs contains the inputs used to run the model, and data_to_fit is a pandas DataFrame with columns matching the requested outcome variables.

Data fitting with SciPy differential evolution:

import numpy as np
import psyneulink as pnl

pec = pnl.ParameterEstimationComposition(
    nodes=comp,
    parameters={("threshold", decision): np.linspace(0.1, 1.0, 10)},
    outcome_variables=[
        decision.output_ports[pnl.DECISION_OUTCOME],
        decision.output_ports[pnl.RESPONSE_TIME],
    ],
    data=data_to_fit,
    optimization_function=pnl.PECOptimizationFunction(
        method="differential_evolution",
        max_iterations=20,
    ),
    num_estimates=100,
    initial_seed=0,
    same_seed_for_all_parameter_combinations=True,
)
pec.run(inputs={comp: trial_inputs})

Parameter optimization with an Optuna sampler:

import numpy as np
import optuna
import psyneulink as pnl

def reward_rate(sim_data):
    return np.mean(sim_data[:, :, 0] / sim_data[:, :, 1])

pec = pnl.ParameterEstimationComposition(
    nodes=comp,
    parameters={("threshold", decision): np.linspace(0.1, 1.0, 10)},
    outcome_variables=[
        decision.output_ports[pnl.DECISION_OUTCOME],
        decision.output_ports[pnl.RESPONSE_TIME],
    ],
    objective_function=reward_rate,
    optimization_function=pnl.PECOptimizationFunction(
        method=optuna.samplers.TPESampler,
        optuna_kwargs={"seed": 0},
        max_iterations=50,
        direction="maximize",
    ),
    num_estimates=100,
)
pec.run(inputs={comp: trial_inputs})

Using an existing Optuna study:

import optuna
import psyneulink as pnl

study = optuna.create_study(
    sampler=optuna.samplers.RandomSampler(seed=0),
    direction="maximize",
)

optimizer = pnl.PECOptimizationFunction(
    method=study,
    max_iterations=50,
    direction="maximize",
)

The resulting optimizer can be supplied as the optimization_function argument to ParameterEstimationComposition.

Structure¶

ParameterEstimationComposition uses a PEC_OCM as its controller – a specialized subclass of OptimizationControlMechanism that intercepts inputs provided to the run method of the ParameterEstimationComposition, and assigns them directly to the state_feature_values of the PEC_OCM when it executes. By default, PEC_OCM sets comp_execution_mode to 'LLVM' for controller simulations; this can be overridden by assigning 'Python' or 'PTX' to the controller’s comp_execution_mode parameter.

Class Reference¶

class psyneulink.core.compositions.parameterestimationcomposition.ParameterEstimationComposition(parameters, outcome_variables, optimization_function, model=None, data=None, likelihood_include_mask=None, data_categorical_dims=None, objective_function=None, num_estimates=1, num_trials_per_estimate=None, initial_seed=None, same_seed_for_all_parameter_combinations=None, depends_on=None, name=None, context=None, distributed=False, distributed_options=None, **kwargs)¶

Subclass of Composition that estimates specified parameters either to fit the results of a Composition to a set of data or to optimize a specified function.

Automatically implements an OptimizationControlMechanism as its controller, that is constructed using arguments to the ParameterEstimationComposition’s constructor as described below.

The following arguments are those specific to ParmeterEstimationComposition; see Composition for additional arguments

Parameters:

parameters (dict) – specifies the parameters of the model to be estimated. These are specified in a dict, in which the key of each entry specifies a parameter to estimate, and its value is a list values to sample for that parameter.
depends_on (Optional[Mapping]) – A dictionary that specifies which parameters depend on a condition. The keys of the dictionary are the specified identically to the keys of the parameters dictionary. The values are a string that specifies a column in the data that the parameter depends on. The values of this column must be categorical. Each unique value will represent a condition and will result in a separate parameter being estimated for it. The number of unique values should be small because each unique value will result in a separate parameter being estimated.
outcome_variables (Union[list[Mechanism], Mechanism, list[OutputPort], OutputPort]) – specifies the OUTPUT Nodes of the model, the values of which are used to evaluate the fit of the different combinations of parameter values sampled. An important limitation of the PEC is that the outcome_variables must be a subset of the output ports of the model’s terminal Mechanism.
model (Optional[Composition]) – specifies an external Composition for which parameters are to be fit to data or optimized according to a specified objective_function.
data (Optional[DataFrame]) – specifies the data to which the outcome_variables are fit in the estimation process. They must be in a format that aligns with the specification of the outcome_variables. The parameter data should be a pandas DataFrame where each column corresponds to one of the outcome_variables. If one of the outcome variables should be treated as a categorical variable (e.g. a decision value in a two-alternative forced choice task modeled by a DDM), then it should be specified as a pandas Categorical variable.
data_categorical_dims (Union[Iterable] : default None) – specifies the dimensions of the data that are categorical. If a list of boolean values is provided, it is assumed to be a mask for the categorical data dimensions and must have the same length as columns in data. If it is an iterable of integers, it is assumed to be a list of the categorical dimensions indices. If it is None, all data dimensions are assumed to be continuous. Alternatively, if data is a pandas DataFrame, then the columns which have Category dtype are assumed to be categorical.
objective_function (ObjectiveFunction, function or method) – specifies the function used by optimization_function (see objective_function for additional information); the shape of its variable argument (i.e., its first positional argument) must be the same as an array containing the value of the OutputPort corresponding to each item specified in outcome_variables.
optimization_function (PECOptimizationFunction or str : default None) – specifies the function used to search over the combinations of parameter values to be estimated. This is usually a PECOptimizationFunction, which supports SciPy differential evolution and Optuna sampler instances, sampler classes, or studies. Passing "differential_evolution" is a shorthand for PECOptimizationFunction(method="differential_evolution").
num_estimates (int : default 1) – specifies the number of estimates made for a each combination of parameter values (see num_estimates for additional information); it is passed to the ParameterEstimationComposition’s controller to set its num_estimates Parameter.
num_trials_per_estimate (int : default None) – specifies an exact number of trials to execute for each run of the model when estimating each combination of parameter values (see num_trials_per_estimate for additional information).
initial_seed (int : default None) – specifies the seed used to initialize the random number generator at construction; it is passed to the ParameterEstimationComposition’s controller to set its initial_seed Parameter.
same_seed_for_all_parameter_combinations (bool : default False) – specifies whether the random number generator is re-initialized to the same value when estimating each combination of parameter values; it is passed to the ParameterEstimationComposition’s controller to set its same_seed_for_all_allocations Parameter.
distributed (bool : default False) – specifies whether to evaluate candidate parameterizations in parallel across a Dask cluster during fitting; forwarded to the PECOptimizationFunction (a PECOptimizationFunction passed explicitly as optimization_function with its own distributed setting takes precedence). See distributed and Distributed Fitting for details; requires the psyneulink[dask] extra and LLVM execution.
distributed_options (Mapping : default None) – specifies options for distributed fitting (used only when distributed is True); forwarded to the PECOptimizationFunction. Must include a "pec_factory" callable; see distributed_options and Distributed Fitting for the full set of keys.

model¶

identifies the Composition used for Data Fitting or Parameter Optimization. If the model argument of the ParameterEstimationComposition’s constructor is not specified, model returns the ParameterEstimationComposition itself.

Type:: Composition

parameters¶

determines the parameters of the model used for Data Fitting or Parameter Optimization (see control for additional details).

Type:: list[Parameters]

parameter_ranges_or_priors¶

determines the range of values evaluated for each parameter. These are assigned as the allocation_samples for the ControlSignal assigned to the ParameterEstimationComposition’s OptimizationControlMechanism corresponding to each of the specified parameters.

Type:: List[Union[Iterator, Function, ist or Value]

outcome_variables¶

determines the OUTPUT Nodes of the model, the values of which are either compared to the data when the ParameterEstimationComposition is used for Data Fitting, or evaluated by the ParameterEstimationComposition’s optimization_function when it is used for Parameter Optimization.

Type:: list[Composition Output Nodes]

data¶

determines the data to be fit by the model when the ParameterEstimationComposition is used for Data Fitting. These must be structured in form that aligns with the specified outcome_variables (see data for additional details). The data are passed to the optimizer used by optimization_function. Returns None if the model is being used for Parameter Optimization.

Type:: array

objective_function¶

determines the function used to evaluate the results of the model under each set of parameter values. It is passed to the ParameterEstimationComposition’s OptimizationControlMechanism as the function of its objective_mechanism, that is used to compute the net_outcome for the model each time it is run (see objective_function for additional details).

Type:: ObjectiveFunction, function or method

optimization_function¶

determines the function used to estimate the parameters of the model that either best fit the data when the ParameterEstimationComposition is used for Data Fitting, or that achieve some maximum or minimum value of the objective_function when the ParameterEstimationComposition is used for Parameter Optimization. This is assigned as the function of the ParameterEstimationComposition’s OptimizationControlMechanism.

Type:: OptimizationFunction

num_estimates¶

determines the number of estimates of the net_outcome of the model (i.e., number of calls to its evaluate method) for a given combination of parameter values (i.e., control_allocation) evaluated.

Type:: int

num_trials_per_estimate¶

imposes an exact number of trials to be executed in each run of model used to evaluate its net_outcome by a call to its OptimizationControlMechanism’s evaluate_agent_rep method. If it is None (the default), then either the number of inputs or the value specified for num_trials in the ParameterEstimationComposition’s run method used to determine the number of trials executed (see number of trials for additional information).

Note

The num_trials_per_estimate is distinct from the num_trials argument of the ParameterEstimationComposition’s run method. The latter determines how many full fits of the model are carried out (that is, how many times the ParameterEstimationComposition itself is run), whereas num_trials_per_estimate determines how many trials are run for a given combination of parameter values within each fit.

Type:: int or None

initial_seed¶

contains the seed used to initialize the random number generator at construction, that is stored on the ParameterEstimationComposition’s controller, and setting it sets the value of that Parameter (see initial_seed for additional details).

Type:: int or None

same_seed_for_all_parameter_combinations¶

contains the setting for determining whether the random number generator used to select seeds for each estimate of the model's net_outcome is re-initialized to the same value for each combination of parameter values evaluated. Its value is stored on the ParameterEstimationComposition’s controller, and setting it sets the value of that Parameter (see same_seed_for_all_allocations for additional details).

Type:: bool

optimized_parameter_values¶

contains the values of the parameters of the model that best fit the data when the ParameterEstimationComposition is used for Data Fitting, or that optimize performance of the model according to the objective_function when the ParameterEstimationComposition is used for Parameter Optimization. If parameter values are specified as ranges of values, then each item of optimized_parameter_values is the optimized value of the corresponding parameter. If parameter values are specified as priors, then each item of optimized_parameter_values is an array containing the values of the corresponding parameter the distribution of which were determined to be optimal.

Type:: list

optimal_value¶

contains the results returned by execution of agent_rep for the parameter values in optimized_parameter_values.

Type:: float

results¶

contains the output_values of the OUTPUT Nodes in the model for every TRIAL executed (see Composition.results for more details). If the ParameterEstimationComposition is used for Data Fitting, and parameter values are specified as ranges of values, then each item of results is an array of output_values (sampled over num_estimates) obtained for the single optimized combination of parameter values contained in the corresponding item of optimized_parameter_values. If parameter values are specified as priors, then each item of results is an array of output_values (sampled over num_estimates), each of which corresponds to a combination of parameter values that were used to generate those results; it is the distribution of those parameter values that were found to best fit the data.

Type:: list[list[list]]

class Parameters(owner, parent=None)¶

initial_seed¶

see input_specification

Default value:: None
Type:: int

same_seed_for_all_parameter_combinations¶

see input_specification

Default value:: False
Type:: bool

_validate_data()¶: Check if user supplied data to fit is valid for data fitting mode.

log_likelihood(*args, inputs=None, return_sim_data=False, context=None)¶

Compute the log-likelihood of the data given the specified parameters of the model.

Parameters:

*args – Positional args, one for each paramter of the model. These must correspond directly to the parameters that have been specified in the parameters argument of the constructor.
return_sim_data (bool) – If True, return a tuple containing the log-likelihood and the simulated data used to compute it.

Return type:

Union[float, tuple]

Returns:

The sum of the log-likelihoods of the data given the specified parameters of the model, or
(log_likelihood, sim_data) when return_sim_data is True.

_prepare_pec_inputs_for_simulation(inputs, context)¶: Normalize, cache, and assign inputs used by PEC optimization and likelihood simulations.

_complete_init_of_partially_initialized_nodes(context)¶

Attempt to complete initialization of aux_components for any nodes with: aux_components that were not previously compatible with Composition

exception psyneulink.core.compositions.parameterestimationcomposition.ParameterEstimationCompositionError(message, component=None)¶