# OptimizationFunctions¶

## Overview¶

Functions that return the sample of a variable yielding the optimized value of an objective_function.

class psyneulink.core.components.functions.optimizationfunctions.OptimizationFunction( default_variable=None, objective_function=lambda x:0, search_function=lambda x:x, search_space=[0], search_termination_function=lambda x,y,z:True, save_samples=False, save_values=False, max_iterations=None, params=Nonse, owner=Nonse, prefs=None)

Provides an interface to subclasses and external optimization functions. The default function executes iteratively, generating samples from search_space using search_function, evaluating them using objective_function, and reporting the value of each using report_value until terminated by search_termination_function. Subclasses can override function to implement their own optimization function or call an external one.

Samples in search_space are assumed to be a list of one or more SampleIterator objects.

Default Optimization Procedure

When function is executed, it iterates over the following steps:

The current iteration numberris contained in iteration. Iteration continues until all values of search_space have been evaluated and/or search_termination_function returns True. The function returns:

Note

An OptimizationFunction or any of its subclasses can be created by calling its constructor. This provides runnable defaults for all of its arguments (see below). However these do not yield useful results, and are meant simply to allow the constructor of the OptimziationFunction to be used to specify some but not all of its parameters when specifying the OptimizationFunction in the constructor for another Component. For example, an OptimizationFunction may use for its objective_function or search_function a method of the Component to which it is being assigned; however, those methods will not yet be available, as the Component itself has not yet been constructed. This can be handled by calling the OptimizationFunction’s reinitialization method after the Component has been instantiated, with a parameter specification dictionary with a key for each entry that is the name of a parameter and its value the value to be assigned to the parameter. This is done automatically for Mechanisms that take an ObjectiveFunction as their function (such as the EVCControlMechanism, LVOCControlMechanism and ParamterEstimationControlMechanism), but will require it be done explicitly for Components for which that is not the case. A warning is issued if defaults are used for the arguments of an OptimizationFunction or its subclasses; this can be suppressed by specifying the relevant argument(s) as NotImplemnted.

Parameters: default_variable (list or ndarray : default None) – specifies a template for (i.e., an example of the shape of) the samples used to evaluate the objective_function. objective_function (function or method : default None) – specifies function used to evaluate sample in each iteration of the optimization process; if it is not specified, a default function is used that simply returns the value passed as its variable parameter (see note). search_function (function or method : default None) – specifies function used to select a sample for objective_function in each iteration of the optimization process. It must be specified if the objective_function does not generate samples on its own (e.g., as does GradientOptimization). If it is required and not specified, the optimization process executes exactly once using the value passed as its variable parameter (see note). search_space (list or array of SampleIterators : default None) – specifies iterators used by search_function to generate samples evaluated objective_function in each iteration of the optimization process. It must be specified if the objective_function does not generate samples on its own (e.g., as does GradientOptimization). If it is required and not specified, the optimization process executes exactly once using the value passed as its variable parameter (see note). search_termination_function (function or method : None) – specifies function used to terminate iterations of the optimization process. It must return a boolean value, and it must be specified if the objective_function is not overridden. If it is required and not specified, the optimization process executes exactly once (see note). save_samples (bool) – specifies whether or not to save and return the values of the samples used to evalute objective_function over all iterations of the optimization process. save_values (bool) – specifies whether or not to save and return the values of objective_function for samples evaluated in all iterations of the optimization process. max_iterations (int : default 1000) – specifies the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the last sample evaluated.
variable

ndarray – first sample evaluated by objective_function (i.e., one used to evaluate it in the first iteration of the optimization process).

objective_function

function or method – used to evaluate the sample in each iteration of the optimization process.

search_function

function, method or None – used to select a sample evaluated by objective_function in each iteration of the optimization process. NotImplemented if the objective_function generates its own samples.

search_space

list or array of SampleIterators – used by search_function to generate samples evaluated by objective_function in each iteration of the optimization process. The number of SampleIterators in the list determines the dimensionality of each sample: in each iteration of the optimization process, each SampleIterator is called upon to provide the value for one of the dimensions of the sample.mNotImplemented if the objective_function generates its own samples. If it is required and not specified, the optimization process executes exactly once using the value passed as its variable parameter (see note).

search_termination_function

function or method that returns a boolean value – used to terminate iterations of the optimization process; if it is required and not specified, the optimization process executes exactly once (see note).

iteration

int – the current iteration of the optimization process.

max_iterations

int : default 1000 – specifies the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the last sample evaluated.

save_samples

bool – determines whether or not to save the values of the samples used to evalute objective_function over all iterations of the optimization process.

save_values

bool – determines whether or not to save and return the values of objective_function for samples evaluated in all iterations of the optimization process.

reinitialize(*args, context=None)

Reinitialize parameters of the OptimizationFunction

Parameters to be reinitialized should be specified in a parameter specification dictionary, in which they key for each entry is the name of one of the following parameters, and its value is the value to be assigned to the parameter. The following parameters can be reinitialized:

class psyneulink.core.components.functions.optimizationfunctions.GradientOptimization(default_variable=None, objective_function=None, gradient_function=None, direction=ASCENT, search_space=None, step_size=1.0, annealing_function=None, convergence_criterion=VALUE, convergence_threshold=.001, max_iterations=1000, save_samples=False, save_values=False, params=None, owner=None, prefs=None)

Sample variable by following gradient with respect to the value of objective_function it generates, and return the sample that generates either the highest (**direction=*ASCENT*) or lowest (**direction=*DESCENT*) value.

Optimization Procedure

When function is executed, it iterates over the folowing steps:

The current iteration is contained in iteration. Iteration continues until convergence_criterion falls below convergence_threshold or the number of iterations exceeds max_iterations. The function returns the last sample evaluated by objective_function (presumed to be the optimal one), the value of the function, as well as lists that may contain all of the samples evaluated and their values, depending on whether save_samples and/or save_vales are True, respectively.

The gradient is evaluated by gradient_function, which should be the derivative of the objective_function with respect to variable at its current value: $$\frac{d(objective\_function(variable))}{d(variable)}$$. If the gradient_function* argument of the constructor is not specified, then an attempt is made to use Autograd’s <https://github.com/HIPS/autograd>_ grad <autograd.grad> method to generate gradient_function <GradientOptimization.gradient_function>. If that fails, an erorr occurs. The **search_space argument can be used to specify lower and/or upper bounds for each dimension of the sample; if the gradient causes a value of the sample to exceed a bound along a dimenson, the value of the bound is used for that dimension, unless/until the gradient shifts and causes it to return back within the bound.

Parameters: default_variable (list or ndarray : default None) – specifies a template for (i.e., an example of the shape of) the samples used to evaluate the objective_function. objective_function (function or method) – specifies function used to evaluate variable in each iteration of the optimization process; it must be specified and it must return a scalar value. gradient_function (function) – specifies function used to compute the gradient in each iteration of the optimization process; if it is not specified, an attempt is made to compute it using autograd.grad. direction (ASCENT or DESCENT : default ASCENT) – specifies the direction of gradient optimization: if ASCENT, movement is attempted in the positive direction (i.e., “up” the gradient); if DESCENT, movement is attempted in the negative direction (i.e. “down” the gradient). step_size (int or float : default 1.0) – specifies the rate at which the variable is updated in each iteration of the optimization process; if annealing_function is specified, step_size specifies the intial value of step_size. search_space (list or array : default None) – specifies bounds of the samples used to evaluate objective_function along each dimension of variable; each item must be a list or tuple, or a SampleIterator that resolves to one. If the item has two elements, they are used as the lower and upper bounds respectively, and the lower must be less than the upper; None can be used in either place, in which case that bound is ignored. If an item has more than two elements, the min is used as the lower bound and the max is used as the upper bound; none of the elements can be None. annealing_function (function or method : default None) – specifies function used to adapt step_size in each iteration of the optimization process; must take accept two parameters — step_size and iteration, in that order — and return a scalar value, that is used for the next iteration of optimization. convergence_criterion (VARIABLE or VALUE : default VALUE) – specifies the parameter used to terminate the optimization process. VARIABLE: process terminates when the most recent sample differs from the previous one by less than convergence_threshold; VALUE: process terminates when the last value returned by objective_function differs from the previous one by less than convergence_threshold. convergence_threshold (int or float : default 0.001) – specifies the change in value of convergence_criterion below which the optimization process is terminated. max_iterations (int : default 1000) – specifies the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the last sample evaluated. save_samples (bool) – specifies whether or not to save and return all of the samples used to evaluate objective_function in the optimization process. save_values (bool) – specifies whether or not to save and return the values of objective_function for all samples evaluated in the optimization process
variable

ndarray – sample used as the starting point for the optimization process (i.e., one used to evaluate objective_function in the first iteration).

objective_function

function or method – function used to evaluate variable in each iteration of the optimization process; it must be specified and it must return a scalar value.

gradient_function

function – function used to compute the gradient in each iteration of the optimization process (see Gradient Calculation for details).

direction

ASCENT or DESCENT – direction of gradient optimization: if ASCENT, movement is attempted in the positive direction (i.e., “up” the gradient); if DESCENT, movement is attempted in the negative direction (i.e. “down” the gradient).

step_size

int or float – determines the rate at which the variable is updated in each iteration of the optimization process; if annealing_function is specified, step_size determines the initial value.

search_space

list or array – contains tuples specifying bounds within which each dimension of variable is sampled, and used to evaluate objective_function in iterations of the optimization process.

bounds

tuple – contains two 2d arrays; the 1st contains the lower bounds for each dimension of the sample (variable), and the 2nd the upper bound of each.

annealing_function

function or method – function used to adapt step_size in each iteration of the optimization process; if None, no call is made and the same step_size is used in each iteration.

iteration

int – the currention iteration of the optimization process.

convergence_criterion

VARIABLE or VALUE – determines parameter used to terminate the optimization process. VARIABLE: process terminates when the most recent sample differs from the previous one by less than convergence_threshold; VALUE: process terminates when the last value returned by objective_function differs from the previous one by less than convergence_threshold.

convergence_threshold

int or float – determines the change in value of convergence_criterion below which the optimization process is terminated.

max_iterations

int – determines the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the last sample evaluated.

save_samples

bool – determines whether or not to save and return all of the samples used to evaluate objective_function in the optimization process.

save_values

bool – determines whether or not to save and return the values of objective_function for all samples evaluated in the optimization process

class psyneulink.core.components.functions.optimizationfunctions.GridSearch(default_variable=None, objective_function=None, direction=MAXIMIZE, max_iterations=1000, save_samples=False, save_values=False, params=None, owner=None, prefs=None)

Search over all samples generated by search_space for the one that optimizes the value of objective_function.

Grid Search Procedure

When function is executed, it iterates over the folowing steps:

The current iteration is contained in iteration and the total number comprising the search_space <GridSearch.search_space>2 is contained in num_iterations). Iteration continues until all values in search_space have been evaluated (i.e., num_iterations is reached), or max_iterations is execeeded. The function returns the sample that yielded either the highest (if direction is MAXIMIZE) or lowest (if direction is MINIMIZE) value of the objective_function, along with the value for that sample, as well as lists containing all of the samples evaluated and their values if either save_samples or save_values is True, respectively.

Parameters: default_variable (list or ndarray : default None) – specifies a template for (i.e., an example of the shape of) the samples used to evaluate the objective_function. objective_function (function or method) – specifies function used to evaluate sample in each iteration of the optimization process; it must be specified and must return a scalar value. search_space (list or array of SampleIterators) – specifies SampleIterators used to generate samples evaluated by objective_function; all of the iterators be finite (i.e., must have a num attribute; see SampleSpec for additional details). direction (MAXIMIZE or MINIMIZE : default MAXIMIZE) – specifies the direction of optimization: if MAXIMIZE, the highest value of objective_function is sought; if MINIMIZE, the lowest value is sought. max_iterations (int : default 1000) – specifies the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the optimal sample of those evaluated. save_samples (bool) – specifies whether or not to return all of the samples used to evaluate objective_function in the optimization process (i.e., a copy of the samples generated from the search_space. save_values (bool) – specifies whether or not to save and return the values of objective_function for all samples evaluated in the optimization process.
variable

ndarray – first sample evaluated by objective_function (i.e., one used to evaluate it in the first iteration of the optimization process).

objective_function

function or method – function used to evaluate sample in each iteration of the optimization process.

search_space

list or array of Sampleiterators – contains SampleIterators for generating samples evaluated by objective_function in iterations of the optimization process;

grid

iterator – generates samples from the Cartesian product of SampleIterators in search_space.

direction

MAXIMIZE or MINIMIZE : default MAXIMIZE – determines the direction of optimization: if MAXIMIZE, the greatest value of objective_function is sought; if MINIMIZE, the least value is sought.

iteration

int – the currention iteration of the optimization process.

num_iterations

int – number of iterations required to complete the entire grid search; equal to the produce of all the num attributes of the SampleIterators in the search_space.

max_iterations

int – determines the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the optimal sample of those evaluated.

save_samples

True – determines whether or not to save and return all samples generated from search_space and evaluated by the objective_function in the optimization process.

save_values

bool – determines whether or not to save and return the value of objective_function for all samples evaluated in the optimization process.

reinitialize(*args, context=None)

Assign size of search_space <GridSearch.search_space>

reset_grid()

Reset iterators in search_space <GridSearch.search_space>

class psyneulink.core.components.functions.optimizationfunctions.GaussianProcess(default_variable=None, objective_function=None, search_space=None, direction=MAXIMIZE, max_iterations=1000, save_samples=False, save_values=False, params=None, owner=None, prefs=None)

Draw samples with dimensionality and bounds specified by search_space and return one that optimizes the value of objective_function.

Gaussian Process Procedure

The number of items (SampleIterators in search_space determines the dimensionality of each sample to evaluate by objective_function, with the start and stop attributes of each SampleIterator specifying the bounds for sampling along the corresponding dimension.

When function is executed, it iterates over the folowing steps:

The current iteration is contained in iteration. Iteration continues until [ FRED: FILL IN THE BLANK], or max_iterations is execeeded. The function returns the sample that yielded either the highest (if direction is MAXIMIZE) or lowest (if direction is MINIMIZE) value of the objective_function, along with the value for that sample, as well as lists containing all of the samples evaluated and their values if either save_samples or save_values is True, respectively.

Parameters: default_variable (list or ndarray : default None) – specifies a template for (i.e., an example of the shape of) the samples used to evaluate the objective_function. objective_function (function or method) – specifies function used to evaluate sample in each iteration of the optimization process; it must be specified and must return a scalar value. search_space (list or array) – specifies bounds of the samples used to evaluate objective_function along each dimension of variable; each item must be a tuple the first element of which specifies the lower bound and the second of which specifies the upper bound. direction (MAXIMIZE or MINIMIZE : default MAXIMIZE) – specifies the direction of optimization: if MAXIMIZE, the highest value of objective_function is sought; if MINIMIZE, the lowest value is sought. max_iterations (int : default 1000) – specifies the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the optimal sample of those evaluated. save_samples (bool) – specifies whether or not to return all of the samples used to evaluate objective_function in the optimization process (i.e., a copy of the search_space. save_values (bool) – specifies whether or not to save and return the values of objective_function for all samples evaluated in the optimization process.
variable

ndarray – template for sample evaluated by objective_function.

objective_function

function or method – function used to evaluate sample in each iteration of the optimization process.

search_space

list or array – contains tuples specifying bounds within which each dimension of variable is sampled, and used to evaluate objective_function in iterations of the optimization process.

direction

MAXIMIZE or MINIMIZE : default MAXIMIZE – determines the direction of optimization: if MAXIMIZE, the greatest value of objective_function is sought; if MINIMIZE, the least value is sought.

iteration

int – the currention iteration of the optimization process.

max_iterations

int – determines the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the optimal sample of those evaluated.

save_samples

True – determines whether or not to save and return all samples evaluated by the objective_function in the optimization process (if the process completes, this should be identical to search_space.

save_values

bool – determines whether or not to save and return the value of objective_function for all samples evaluated in the optimization process.

class psyneulink.core.components.functions.optimizationfunctions.SampleIterator(specification)

Create an iterator that returns the next sample from a sequence on each call to next. (e.g., when next(<SampleIterator>) is called)

The pattern of the sequence depends on the specification, which may be a list, nparray, range, function, or a SampleSpec. Most of the patterns depend on the “current_step,” which is incremented on each iteration, and set to zero when the iterator is reset.

 Specification what happens on each iteration StopIteration condition list, nparray look up the item with index current_step list/array range, np.arange start + step*current_step range stop value is reached callable call callable iteration does not stop SampleSpec(start, stop, step) start + step*current_step current_step = num or value > stop SampleSpec(start, stop, num) start + step*current_step current_step = num or value > stop SampleSpec(function, num) call function current_step = num SampleSpec(function) call function iteration does not stop

Note

We recommend reserving the list/nparray option for cases in which the samples do not have a pattern that can be represented by a SampleSpec, or the number of samples is small. The list/nparray option requires all of the samples to be stored and looked up, while the SampleSpec options generate samples as needed.

reset(head=None)

Reset iterator to a specified item If called with None, resets to first item (if generator` is a list or deterministic function.