# OptimizationFunctions¶

## Overview¶

Functions that return the sample of a variable yielding the optimized value of an objective_function.

class psyneulink.core.components.functions.optimizationfunctions.OptimizationFunction(default_variable=None, objective_function=lambda x: ..., search_function=lambda x: ..., search_space=[0], search_termination_function=lambda x, y, z: ..., save_samples=False, save_values=False, max_iterations=None, params=Nonse, owner=Nonse, prefs=None)

Provides an interface to subclasses and external optimization functions. The default function executes iteratively, generating samples from search_space using search_function, evaluating them using objective_function, and reporting the value of each using report_value until terminated by search_termination_function. Subclasses can override function to implement their own optimization function or call an external one.

Samples in search_space are assumed to be a list of one or more SampleIterator objects.

Default Optimization Procedure

When function is executed, it iterates over the following steps:

The current iteration numberris contained in iteration. Iteration continues until all values of search_space have been evaluated and/or search_termination_function returns True. The function returns:

Note

An OptimizationFunction or any of its subclasses can be created by calling its constructor. This provides runnable defaults for all of its arguments (see below). However these do not yield useful results, and are meant simply to allow the constructor of the OptimziationFunction to be used to specify some but not all of its parameters when specifying the OptimizationFunction in the constructor for another Component. For example, an OptimizationFunction may use for its objective_function or search_function a method of the Component to which it is being assigned; however, those methods will not yet be available, as the Component itself has not yet been constructed. This can be handled by calling the OptimizationFunction’s reset method after the Component has been instantiated, with a parameter specification dictionary with a key for each entry that is the name of a parameter and its value the value to be assigned to the parameter. This is done automatically for Mechanisms that take an ObjectiveFunction as their function (such as the OptimizationControlMechanism), but will require it be done explicitly for Components for which that is not the case. A warning is issued if defaults are used for the arguments of an OptimizationFunction or its subclasses; this can be suppressed by specifying the relevant argument(s) as NotImplemnted.

Parameters
variable

first sample evaluated by objective_function (i.e., one used to evaluate it in the first iteration of the optimization process).

Type

ndarray

objective_function

used to evaluate the sample in each iteration of the optimization process.

Type

function or method

search_function

used to select a sample evaluated by objective_function in each iteration of the optimization process. NotImplemented if the objective_function generates its own samples.

Type

function, method or None

search_space

used by search_function to generate samples evaluated by objective_function in each iteration of the optimization process. The number of SampleIterators in the list determines the dimensionality of each sample: in each iteration of the optimization process, each SampleIterator is called upon to provide the value for one of the dimensions of the sample.mNotImplemented if the objective_function generates its own samples. If it is required and not specified, the optimization process executes exactly once using the value passed as its variable parameter (see note).

Type

list or array of SampleIterators

search_termination_function

used to terminate iterations of the optimization process; if it is required and not specified, the optimization process executes exactly once (see note).

Type

function or method that returns a boolean value

iteration

the current iteration of the optimization process.

Type

int

max_iterations

specifies the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the last sample evaluated.

Type

int : default 1000

save_samples

determines whether or not to save the values of the samples used to evalute objective_function over all iterations of the optimization process.

Type

bool

save_values

determines whether or not to save and return the values of objective_function for samples evaluated in all iterations of the optimization process.

Type

bool

_validate_params(request_set, target_set=None, context=None)

Validate params and assign validated values to targets,

This performs top-level type validation of params

This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is True (which it is by default)

IMPLEMENTATION NOTES:
• future versions should add recursive and content (e.g., range) checking

• should method return validated param set?

Parameters
• validated (dict (target_set) - repository of params that have been) –

• validated

Return none

reset(default_variable=None, objective_function=None, search_function=None, search_termination_function=None, search_space=None, context=None)

Reset parameters of the OptimizationFunction

Parameters to be reset should be specified in a parameter specification dictionary, in which they key for each entry is the name of one of the following parameters, and its value is the value to be assigned to the parameter. The following parameters can be reset:

_function(variable=None, context=None, params=None, **kwargs)

Find the sample that yields the optimal value of objective_function.

See optimization process for details.

Returns

optimal sample, optimal value, saved_samples, saved_values – first array contains sample that yields the optimal value of the optimization process, and second array contains the value of objective_function for that sample. If save_samples is True, first list contains all the values sampled in the order they were evaluated; otherwise it is empty. If save_values is True, second list contains the values returned by objective_function for all the samples in the order they were evaluated; otherwise it is empty.

Return type

array, array, list, list

_report_value(new_value)

Report value returned by objective_function for sample.

Sample variable by following gradient with respect to the value of objective_function it generates, and return the sample that generates either the highest (**direction=*ASCENT*) or lowest (**direction=*DESCENT*) value.

Optimization Procedure

When function is executed, it iterates over the folowing steps:

The current iteration is contained in iteration. Iteration continues until convergence_criterion falls below convergence_threshold or the number of iterations exceeds max_iterations. The function returns the last sample evaluated by objective_function (presumed to be the optimal one), the value of the function, as well as lists that may contain all of the samples evaluated and their values, depending on whether save_samples and/or save_vales are True, respectively.

The gradient is evaluated by gradient_function, which should be the derivative of the objective_function with respect to variable at its current value: $$\frac{d(objective\_function(variable))}{d(variable)}$$. If the gradient_function* argument of the constructor is not specified, then an attempt is made to use Autograd’s <https://github.com/HIPS/autograd>_ grad <autograd.grad> method to generate gradient_function <GradientOptimization.gradient_function>. If that fails, an erorr occurs. The **search_space argument can be used to specify lower and/or upper bounds for each dimension of the sample; if the gradient causes a value of the sample to exceed a bound along a dimenson, the value of the bound is used for that dimension, unless/until the gradient shifts and causes it to return back within the bound.

Parameters
• default_variable (list or ndarray : default None) – specifies a template for (i.e., an example of the shape of) the samples used to evaluate the objective_function.

• objective_function (function or method) – specifies function used to evaluate variable in each iteration of the optimization process; it must be specified and it must return a scalar value.

• gradient_function (function) – specifies function used to compute the gradient in each iteration of the optimization process; if it is not specified, an attempt is made to compute it using autograd.grad.

• direction (ASCENT or DESCENT : default ASCENT) – specifies the direction of gradient optimization: if ASCENT, movement is attempted in the positive direction (i.e., “up” the gradient); if DESCENT, movement is attempted in the negative direction (i.e. “down” the gradient).

• step_size (int or float : default 1.0) – specifies the rate at which the variable is updated in each iteration of the optimization process; if annealing_function is specified, step_size specifies the intial value of step_size.

• search_space (list or array : default None) – specifies bounds of the samples used to evaluate objective_function along each dimension of variable; each item must be a list or tuple, or a SampleIterator that resolves to one. If the item has two elements, they are used as the lower and upper bounds respectively, and the lower must be less than the upper; None can be used in either place, in which case that bound is ignored. If an item has more than two elements, the min is used as the lower bound and the max is used as the upper bound; none of the elements can be None.

• annealing_function (function or method : default None) – specifies function used to adapt step_size in each iteration of the optimization process; must take accept two parameters — step_size and iteration, in that order — and return a scalar value, that is used for the next iteration of optimization.

• convergence_criterion (VARIABLE or VALUE : default VALUE) – specifies the parameter used to terminate the optimization process. VARIABLE: process terminates when the most recent sample differs from the previous one by less than convergence_threshold; VALUE: process terminates when the last value returned by objective_function differs from the previous one by less than convergence_threshold.

• convergence_threshold (int or float : default 0.001) – specifies the change in value of convergence_criterion below which the optimization process is terminated.

• max_iterations (int : default 1000) – specifies the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the last sample evaluated.

• save_samples (bool) – specifies whether or not to save and return all of the samples used to evaluate objective_function in the optimization process.

• save_values (bool) – specifies whether or not to save and return the values of objective_function for all samples evaluated in the optimization process

variable

sample used as the starting point for the optimization process (i.e., one used to evaluate objective_function in the first iteration).

Type

ndarray

objective_function

function used to evaluate variable in each iteration of the optimization process; it must be specified and it must return a scalar value.

Type

function or method

function used to compute the gradient in each iteration of the optimization process (see Gradient Calculation for details).

Type

function

direction

direction of gradient optimization: if ASCENT, movement is attempted in the positive direction (i.e., “up” the gradient); if DESCENT, movement is attempted in the negative direction (i.e. “down” the gradient).

Type

ASCENT or DESCENT

step_size

determines the rate at which the variable is updated in each iteration of the optimization process; if annealing_function is specified, step_size determines the initial value.

Type

int or float

search_space

contains tuples specifying bounds within which each dimension of variable is sampled, and used to evaluate objective_function in iterations of the optimization process.

Type

list or array

bounds

contains two 2d arrays; the 1st contains the lower bounds for each dimension of the sample (variable), and the 2nd the upper bound of each.

Type

tuple

annealing_function

function used to adapt step_size in each iteration of the optimization process; if None, no call is made and the same step_size is used in each iteration.

Type

function or method

iteration

the currention iteration of the optimization process.

Type

int

convergence_criterion

determines parameter used to terminate the optimization process. VARIABLE: process terminates when the most recent sample differs from the previous one by less than convergence_threshold; VALUE: process terminates when the last value returned by objective_function differs from the previous one by less than convergence_threshold.

Type

VARIABLE or VALUE

convergence_threshold

determines the change in value of convergence_criterion below which the optimization process is terminated.

Type

int or float

max_iterations

determines the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the last sample evaluated.

Type

int

save_samples

determines whether or not to save and return all of the samples used to evaluate objective_function in the optimization process.

Type

bool

save_values

determines whether or not to save and return the values of objective_function for all samples evaluated in the optimization process

Type

bool

_validate_params(request_set, target_set=None, context=None)

Validate params and assign validated values to targets,

This performs top-level type validation of params

This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is True (which it is by default)

IMPLEMENTATION NOTES:
• future versions should add recursive and content (e.g., range) checking

• should method return validated param set?

Parameters
• validated (dict (target_set) - repository of params that have been) –

• validated

Return none

reset(default_variable=None, objective_function=None, context=None, **kwargs)

Reset parameters of the OptimizationFunction

Parameters to be reset should be specified in a parameter specification dictionary, in which they key for each entry is the name of one of the following parameters, and its value is the value to be assigned to the parameter. The following parameters can be reset:

_function(variable=None, context=None, params=None, **kwargs)

Return the sample that yields the optimal value of objective_function, and possibly all samples evaluated and their corresponding values.

Optimal value is defined by direction: - if ASCENT, returns greatest value - if DESCENT, returns least value

Returns

optimal sample, optimal value, saved_samples, saved_values – first array contains sample that yields the highest or lowest value of objective_function, depending on direction, and the second array contains the value of the function for that sample. If save_samples is True, first list contains all the values sampled in the order they were evaluated; otherwise it is empty. If save_values is True, second list contains the values returned by objective_function for all the samples in the order they were evaluated; otherwise it is empty.

Return type

ndarray, list, list

class psyneulink.core.components.functions.optimizationfunctions.GridSearch(default_variable=None, objective_function=None, direction=MAXIMIZE, max_iterations=1000, save_samples=False, save_values=False, params=None, owner=None, prefs=None)

Search over all samples generated by search_space for the one that optimizes the value of objective_function.

Grid Search Procedure

When function is executed, it iterates over the folowing steps:

The current iteration is contained in iteration and the total number comprising the search_space <GridSearch.search_space>2 is contained in num_iterations). Iteration continues until all values in search_space have been evaluated (i.e., num_iterations is reached), or max_iterations is execeeded. The function returns the sample that yielded either the highest (if direction is MAXIMIZE) or lowest (if direction is MINIMIZE) value of the objective_function, along with the value for that sample, as well as lists containing all of the samples evaluated and their values if either save_samples or save_values is True, respectively.

Parameters
variable

first sample evaluated by objective_function (i.e., one used to evaluate it in the first iteration of the optimization process).

Type

ndarray

objective_function

function used to evaluate sample in each iteration of the optimization process.

Type

function or method

search_space

contains SampleIterators for generating samples evaluated by objective_function in iterations of the optimization process;

Type

list or array of Sampleiterators

grid

generates samples from the Cartesian product of SampleIterators in search_space.

Type

iterator

direction

determines the direction of optimization: if MAXIMIZE, the greatest value of objective_function is sought; if MINIMIZE, the least value is sought.

Type

MAXIMIZE or MINIMIZE : default MAXIMIZE

iteration

the currention iteration of the optimization process.

Type

int

num_iterations

number of iterations required to complete the entire grid search; equal to the produce of all the num attributes of the SampleIterators in the search_space.

Type

int

max_iterations

determines the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the optimal sample of those evaluated.

Type

int

save_samples

determines whether or not to save and return all samples generated from search_space and evaluated by the objective_function in the optimization process.

Type

True

save_values

determines whether or not to save and return the value of objective_function for all samples evaluated in the optimization process.

Type

bool

_validate_params(request_set, target_set=None, context=None)

Validate params and assign validated values to targets,

This performs top-level type validation of params

This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is True (which it is by default)

IMPLEMENTATION NOTES:
• future versions should add recursive and content (e.g., range) checking

• should method return validated param set?

Parameters
• validated (dict (target_set) - repository of params that have been) –

• validated

Return none

reset(search_space, context=None, **kwargs)

Assign size of search_space <GridSearch.search_space>

reset_grid()

Reset iterators in search_space <GridSearch.search_space>

_function(variable=None, context=None, params=None, **kwargs)

Return the sample that yields the optimal value of objective_function, and possibly all samples evaluated and their corresponding values.

Optimal value is defined by direction: - if MAXIMIZE, returns greatest value - if MINIMIZE, returns least value

Returns

optimal sample, optimal value, saved_samples, saved_values – first array contains sample that yields the highest or lowest value of objective_function, depending on direction, and the second array contains the value of the function for that sample. If save_samples is True, first list contains all the values sampled in the order they were evaluated; otherwise it is empty. If save_values is True, second list contains the values returned by objective_function for all the samples in the order they were evaluated; otherwise it is empty.

Return type

ndarray, list, list

_traverse_grid(variable, sample_num, context=None)

Get next sample from grid. This is assigned as the search_function of the OptimizationFunction.

_grid_complete(variable, value, iteration, context=None)

Return False when search of grid is complete This is assigned as the search_termination_function of the OptimizationFunction.

class psyneulink.core.components.functions.optimizationfunctions.GaussianProcess(default_variable=None, objective_function=None, search_space=None, direction=MAXIMIZE, max_iterations=1000, save_samples=False, save_values=False, params=None, owner=None, prefs=None)

Draw samples with dimensionality and bounds specified by search_space and return one that optimizes the value of objective_function.

Gaussian Process Procedure

The number of items (SampleIterators in search_space determines the dimensionality of each sample to evaluate by objective_function, with the start and stop attributes of each SampleIterator specifying the bounds for sampling along the corresponding dimension.

When function is executed, it iterates over the folowing steps:

The current iteration is contained in iteration. Iteration continues until [ FRED: FILL IN THE BLANK], or max_iterations is execeeded. The function returns the sample that yielded either the highest (if direction is MAXIMIZE) or lowest (if direction is MINIMIZE) value of the objective_function, along with the value for that sample, as well as lists containing all of the samples evaluated and their values if either save_samples or save_values is True, respectively.

Parameters
• default_variable (list or ndarray : default None) – specifies a template for (i.e., an example of the shape of) the samples used to evaluate the objective_function.

• objective_function (function or method) – specifies function used to evaluate sample in each iteration of the optimization process; it must be specified and must return a scalar value.

• search_space (list or array) – specifies bounds of the samples used to evaluate objective_function along each dimension of variable; each item must be a tuple the first element of which specifies the lower bound and the second of which specifies the upper bound.

• direction (MAXIMIZE or MINIMIZE : default MAXIMIZE) – specifies the direction of optimization: if MAXIMIZE, the highest value of objective_function is sought; if MINIMIZE, the lowest value is sought.

• max_iterations (int : default 1000) – specifies the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the optimal sample of those evaluated.

• save_samples (bool) – specifies whether or not to return all of the samples used to evaluate objective_function in the optimization process (i.e., a copy of the search_space.

• save_values (bool) – specifies whether or not to save and return the values of objective_function for all samples evaluated in the optimization process.

variable

template for sample evaluated by objective_function.

Type

ndarray

objective_function

function used to evaluate sample in each iteration of the optimization process.

Type

function or method

search_space

contains tuples specifying bounds within which each dimension of variable is sampled, and used to evaluate objective_function in iterations of the optimization process.

Type

list or array

direction

determines the direction of optimization: if MAXIMIZE, the greatest value of objective_function is sought; if MINIMIZE, the least value is sought.

Type

MAXIMIZE or MINIMIZE : default MAXIMIZE

iteration

the currention iteration of the optimization process.

Type

int

max_iterations

determines the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the optimal sample of those evaluated.

Type

int

save_samples

determines whether or not to save and return all samples evaluated by the objective_function in the optimization process (if the process completes, this should be identical to search_space.

Type

True

save_values

determines whether or not to save and return the value of objective_function for all samples evaluated in the optimization process.

Type

bool

_validate_params(request_set, target_set=None, context=None)

Validate params and assign validated values to targets,

This performs top-level type validation of params

This can be overridden by a subclass to perform more detailed checking (e.g., range, recursive, etc.) It is called only if the parameter_validation attribute is True (which it is by default)

IMPLEMENTATION NOTES:
• future versions should add recursive and content (e.g., range) checking

• should method return validated param set?

Parameters
• validated (dict (target_set) - repository of params that have been) –

• validated

Return none

_function(variable=None, context=None, params=None, **kwargs)

Return the sample that yields the optimal value of objective_function, and possibly all samples evaluated and their corresponding values.

Optimal value is defined by direction: - if MAXIMIZE, returns greatest value - if MINIMIZE, returns least value

Returns

optimal sample, optimal value, saved_samples, saved_values – first array contains sample that yields the highest or lowest value of objective_function, depending on direction, and the second array contains the value of the function for that sample. If save_samples is True, first list contains all the values sampled in the order they were evaluated; otherwise it is empty. If save_values is True, second list contains the values returned by objective_function for all the samples in the order they were evaluated; otherwise it is empty.

Return type

ndarray, list, list

_gaussian_process_sample(variable, sample_num, context=None)

Draw and return sample from search_space.

_gaussian_process_satisfied(variable, value, iteration, context=None)

Determine whether search should be terminated; return True if so, False if not.

Create an iterator that returns the next sample from a sequence on each call to next. (e.g., when next(<SampleIterator>) is called)

The pattern of the sequence depends on the specification, which may be a list, nparray, range, function, or a SampleSpec. Most of the patterns depend on the “current_step,” which is incremented on each iteration, and set to zero when the iterator is reset.

 Specification what happens on each iteration StopIteration condition list, nparray look up the item with index current_step list/array range, np.arange start + step*current_step range stop value is reached callable call callable iteration does not stop SampleSpec(start, stop, step) start + step*current_step current_step = num or value > stop SampleSpec(start, stop, num) start + step*current_step current_step = num or value > stop SampleSpec(function, num) call function current_step = num SampleSpec(function) call function iteration does not stop

Note

We recommend reserving the list/nparray option for cases in which the samples do not have a pattern that can be represented by a SampleSpec, or the number of samples is small. The list/nparray option requires all of the samples to be stored and looked up, while the SampleSpec options generate samples as needed.

Reset iterator to a specified item If called with None, resets to first item (if generator` is a list or deterministic function.