OptimizationFunctions¶
Overview¶
Functions that return the sample of a variable yielding the optimized value of an objective_function.

class
psyneulink.core.components.functions.optimizationfunctions.
OptimizationFunction
( default_variable=None, objective_function=lambda x:0, search_function=lambda x:x, search_space=[0], search_termination_function=lambda x,y,z:True, save_samples=False, save_values=False, max_iterations=None, params=Nonse, owner=Nonse, prefs=None)¶ Provides an interface to subclasses and external optimization functions. The default
function
executes iteratively, generating samples fromsearch_space
usingsearch_function
, evaluating them usingobjective_function
, and reporting the value of each usingreport_value
until terminated bysearch_termination_function
. Subclasses can overridefunction
to implement their own optimization function or call an external one.Samples in
search_space
are assumed to be a list of one or moreSampleIterator
objects.Default Optimization Procedure
When
function
is executed, it iterates over the following steps: get sample from
search_space
by callingsearch_function
 compute value of
objective_function
for the sample by callingobjective_function
;
 report value returned by
objective_function
for the sample by callingreport_value
;
 evaluate
search_termination_function
.
The current iteration numberris contained in
iteration
. Iteration continues until all values ofsearch_space
have been evaluated and/orsearch_termination_function
returnsTrue
. Thefunction
returns: the last sample evaluated (which may or may not be the optimal value, depending on the
objective_function
);  the value of
objective_function
associated with the last sample;  two lists, that may contain all of the samples evaluated and their values, depending on whether
save_samples
and/orsave_vales
areTrue
, respectively.
Note
An OptimizationFunction or any of its subclasses can be created by calling its constructor. This provides runnable defaults for all of its arguments (see below). However these do not yield useful results, and are meant simply to allow the constructor of the OptimziationFunction to be used to specify some but not all of its parameters when specifying the OptimizationFunction in the constructor for another Component. For example, an OptimizationFunction may use for its
objective_function
orsearch_function
a method of the Component to which it is being assigned; however, those methods will not yet be available, as the Component itself has not yet been constructed. This can be handled by calling the OptimizationFunction’sreinitialization
method after the Component has been instantiated, with a parameter specification dictionary with a key for each entry that is the name of a parameter and its value the value to be assigned to the parameter. This is done automatically for Mechanisms that take an ObjectiveFunction as theirfunction
(such as the EVCControlMechanism,LVOCControlMechanism
andParamterEstimationControlMechanism
), but will require it be done explicitly for Components for which that is not the case. A warning is issued if defaults are used for the arguments of an OptimizationFunction or its subclasses; this can be suppressed by specifying the relevant argument(s) asNotImplemnted
.Parameters:  default_variable (list or ndarray : default None) – specifies a template for (i.e., an example of the shape of) the samples used to evaluate the
objective_function
.  objective_function (function or method : default None) – specifies function used to evaluate sample in each iteration of the optimization process; if it is not specified, a default function is used that simply returns
the value passed as its
variable
parameter (see note).  search_function (function or method : default None) – specifies function used to select a sample for
objective_function
in each iteration of the optimization process. It must be specified if theobjective_function
does not generate samples on its own (e.g., as doesGradientOptimization
). If it is required and not specified, the optimization process executes exactly once using the value passed as itsvariable
parameter (see note).  search_space (list or array of SampleIterators : default None) – specifies iterators used by
search_function
to generate samples evaluatedobjective_function
in each iteration of the optimization process. It must be specified if theobjective_function
does not generate samples on its own (e.g., as doesGradientOptimization
). If it is required and not specified, the optimization process executes exactly once using the value passed as itsvariable
parameter (see note).  search_termination_function (function or method : None) – specifies function used to terminate iterations of the optimization process.
It must return a boolean value, and it must be specified if the
objective_function
is not overridden. If it is required and not specified, the optimization process executes exactly once (see note).  save_samples (bool) – specifies whether or not to save and return the values of the samples used to evalute
objective_function
over all iterations of the optimization process.  save_values (bool) – specifies whether or not to save and return the values of
objective_function
for samples evaluated in all iterations of the optimization process.  max_iterations (int : default 1000) – specifies the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the last sample evaluated.

variable
¶ ndarray – first sample evaluated by
objective_function
(i.e., one used to evaluate it in the first iteration of the optimization process).

objective_function
¶ function or method – used to evaluate the sample in each iteration of the optimization process.

search_function
¶ function, method or None – used to select a sample evaluated by
objective_function
in each iteration of the optimization process.NotImplemented
if theobjective_function
generates its own samples.

search_space
¶ list or array of
SampleIterators
– used bysearch_function
to generate samples evaluated byobjective_function
in each iteration of the optimization process. The number of SampleIterators in the list determines the dimensionality of each sample: in each iteration of the optimization process, each SampleIterator is called upon to provide the value for one of the dimensions of the sample.m`NotImplemented` if theobjective_function
generates its own samples. If it is required and not specified, the optimization process executes exactly once using the value passed as itsvariable
parameter (see note).

search_termination_function
¶ function or method that returns a boolean value – used to terminate iterations of the optimization process; if it is required and not specified, the optimization process executes exactly once (see note).

iteration
¶ int – the current iteration of the optimization process.

max_iterations
¶ int : default 1000 – specifies the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the last sample evaluated.

save_samples
¶ bool – determines whether or not to save the values of the samples used to evalute
objective_function
over all iterations of the optimization process.

save_values
¶ bool – determines whether or not to save and return the values of
objective_function
for samples evaluated in all iterations of the optimization process.

reinitialize
(*args, execution_id=None)¶ Reinitialize parameters of the OptimizationFunction
Parameters to be reinitialized should be specified in a parameter specification dictionary, in which they key for each entry is the name of one of the following parameters, and its value is the value to be assigned to the parameter. The following parameters can be reinitialized:
default_variable
objective_function
search_function
search_termination_function

function
(variable=None, execution_id=None, params=None, context=None, **kwargs)¶ Find the sample that yields the optimal value of
objective_function
.See optimization process for details.
Returns: optimal sample, optimal value, saved_samples, saved_values – first array contains sample that yields the optimal value of the optimization process, and second array contains the value of objective_function
for that sample. Ifsave_samples
isTrue
, first list contains all the values sampled in the order they were evaluated; otherwise it is empty. Ifsave_values
isTrue
, second list contains the values returned byobjective_function
for all the samples in the order they were evaluated; otherwise it is empty.Return type: array, array, list, list
 get sample from

class
psyneulink.core.components.functions.optimizationfunctions.
GradientOptimization
(default_variable=None, objective_function=None, gradient_function=None, direction=ASCENT, step=1.0, annealing_function=None, convergence_criterion=VALUE, convergence_threshold=.001, max_iterations=1000, save_samples=False, save_values=False, params=None, owner=None, prefs=None)¶ Sample variable by following gradient with respect to the value of
objective_function
it generates, and return the sample that generates either the highest (**direction=*ASCENT*) or lowest (**direction=*DESCENT*) value.Optimization Procedure
When
function
is executed, it iterates over the folowing steps: compute gradient using the
gradient_function
;
 adjust
variable
based on the gradient, in the specifieddirection
and by an amount specified bystep
and possiblyannealing_function
;
 compute value of
objective_function
using the adjusted value ofvariable
;
 adjust
step
usingannealing_function
, if specified, for use in the next iteration;
 evaluate
convergence_criterion
and test whether it is below theconvergence_threshold
.
The current iteration is contained in
iteration
. Iteration continues untilconvergence_criterion
falls belowconvergence_threshold
or the number of iterations exceedsmax_iterations
. Thefunction
returns the last sample evaluated byobjective_function
(presumed to be the optimal one), the value of the function, as well as lists that may contain all of the samples evaluated and their values, depending on whethersave_samples
and/orsave_vales
areTrue
, respectively.Gradient Calculation
The gradient is evaluated by
gradient_function
, which should be the derivative of theobjective_function
with respect tovariable
at its current value: \(\frac{d(objective\_function(variable))}{d(variable)}\). If the **gradient_function* argument of the constructor is not specified, then an attempt is made to use Autograd’sgrad
method to generategradient_function
. If that fails, a warning is issued, and gradients are not calculated.Parameters:  default_variable (list or ndarray : default None) – specifies a template for (i.e., an example of the shape of) the samples used to evaluate the
objective_function
.  objective_function (function or method) – specifies function used to evaluate
variable
in each iteration of the optimization process; it must be specified and it must return a scalar value.  gradient_function (function) – specifies function used to compute the gradient in each iteration of the optimization process; if it is not specified, an attempt is made to compute it using autograd.grad.
 direction (ASCENT or DESCENT : default ASCENT) – specifies the direction of gradient optimization: if ASCENT, movement is attempted in the positive direction (i.e., “up” the gradient); if DESCENT, movement is attempted in the negative direction (i.e. “down” the gradient).
 step (int or float : default 1.0) – specifies the rate at which the
variable
is updated in each iteration of the optimization process; ifannealing_function
is specified, step specifies the intial value ofstep
.  annealing_function (function or method : default None) – specifies function used to adapt
step
in each iteration of the optimization process; must take accept two parameters —step
and iteration, in that order — and return a scalar value, that is used for the next iteration of optimization.  convergence_criterion (VARIABLE or VALUE : default VALUE) – specifies the parameter used to terminate the optimization process.
VARIABLE: process terminates when the most recent sample differs from the previous one by less than
convergence_threshold
; VALUE: process terminates when the last value returned byobjective_function
differs from the previous one by less thanconvergence_threshold
.  convergence_threshold (int or float : default 0.001) – specifies the change in value of
convergence_criterion
below which the optimization process is terminated.  max_iterations (int : default 1000) – specifies the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the last sample evaluated.
 save_samples (bool) – specifies whether or not to save and return all of the samples used to evaluate
objective_function
in the optimization process.  save_values (bool) – specifies whether or not to save and return the values of
objective_function
for all samples evaluated in the optimization process

variable
¶ ndarray – sample used as the starting point for the optimization process (i.e., one used to evaluate
objective_function
in the first iteration).

objective_function
¶ function or method – function used to evaluate
variable
in each iteration of the optimization process; it must be specified and it must return a scalar value.

gradient_function
¶ function – function used to compute the gradient in each iteration of the optimization process (see Gradient Calculation for details).

direction
¶ ASCENT or DESCENT – direction of gradient optimization: if ASCENT, movement is attempted in the positive direction (i.e., “up” the gradient); if DESCENT, movement is attempted in the negative direction (i.e. “down” the gradient).

step
¶ int or float – determines the rate at which the
variable
is updated in each iteration of the optimization process; ifannealing_function
is specified,step
determines the initial value.

annealing_function
¶ function or method – function used to adapt
step
in each iteration of the optimization process; ifNone
, no call is made and the samestep
is used in each iteration.

iteration
¶ int – the currention iteration of the optimization process.

convergence_criterion
¶ VARIABLE or VALUE – determines parameter used to terminate the optimization process. VARIABLE: process terminates when the most recent sample differs from the previous one by less than
convergence_threshold
; VALUE: process terminates when the last value returned byobjective_function
differs from the previous one by less thanconvergence_threshold
.

convergence_threshold
¶ int or float – determines the change in value of
convergence_criterion
below which the optimization process is terminated.

max_iterations
¶ int – determines the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the last sample evaluated.

save_samples
¶ bool – determines whether or not to save and return all of the samples used to evaluate
objective_function
in the optimization process.

save_values
¶ bool – determines whether or not to save and return the values of
objective_function
for all samples evaluated in the optimization process

function
(variable=None, execution_id=None, params=None, context=None, **kwargs)¶ Return the sample that yields the optimal value of
objective_function
, and possibly all samples evaluated and their corresponding values.Optimal value is defined by
direction
:  if ASCENT, returns greatest value  if DESCENT, returns least valueReturns: optimal sample, optimal value, saved_samples, saved_values – first array contains sample that yields the highest or lowest value of objective_function
, depending ondirection
, and the second array contains the value of the function for that sample. Ifsave_samples
isTrue
, first list contains all the values sampled in the order they were evaluated; otherwise it is empty. Ifsave_values
isTrue
, second list contains the values returned byobjective_function
for all the samples in the order they were evaluated; otherwise it is empty.Return type: ndarray, list, list
 compute gradient using the

class
psyneulink.core.components.functions.optimizationfunctions.
GridSearch
(default_variable=None, objective_function=None, direction=MAXIMIZE, max_iterations=1000, save_samples=False, save_values=False, params=None, owner=None, prefs=None)¶ Search over all samples generated by
search_space
for the one that optimizes the value ofobjective_function
.Grid Search Procedure
When
function
is executed, it iterates over the folowing steps: get next sample from
search_space
;
 compute value of
objective_function
for that sample;
The current iteration is contained in
iteration
and the total number comprising thesearch_space <GridSearch.search_space>2
is contained innum_iterations
). Iteration continues until all values insearch_space
have been evaluated (i.e.,num_iterations
is reached), ormax_iterations
is execeeded. The function returns the sample that yielded either the highest (ifdirection
is MAXIMIZE) or lowest (ifdirection
is MINIMIZE) value of theobjective_function
, along with the value for that sample, as well as lists containing all of the samples evaluated and their values if eithersave_samples
orsave_values
isTrue
, respectively.Parameters:  default_variable (list or ndarray : default None) – specifies a template for (i.e., an example of the shape of) the samples used to evaluate the
objective_function
.  objective_function (function or method) – specifies function used to evaluate sample in each iteration of the optimization process; it must be specified and must return a scalar value.
 search_space (list or array of SampleIterators) – specifies
SampleIterators
used to generate samples evaluated byobjective_function
; all of the iterators be finite (i.e., must have anum
attribute; seeSampleSpec
for additional details).  direction (MAXIMIZE or MINIMIZE : default MAXIMIZE) – specifies the direction of optimization: if MAXIMIZE, the highest value of
objective_function
is sought; if MINIMIZE, the lowest value is sought.  max_iterations (int : default 1000) – specifies the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the optimal sample of those evaluated.
 save_samples (bool) – specifies whether or not to return all of the samples used to evaluate
objective_function
in the optimization process (i.e., a copy of the samples generated from thesearch_space
.  save_values (bool) – specifies whether or not to save and return the values of
objective_function
for all samples evaluated in the optimization process.

variable
¶ ndarray – first sample evaluated by
objective_function
(i.e., one used to evaluate it in the first iteration of the optimization process).

objective_function
¶ function or method – function used to evaluate sample in each iteration of the optimization process.

search_space
¶ list or array of Sampleiterators – contains
SampleIterators
for generating samples evaluated byobjective_function
in iterations of the optimization process;

grid
¶ iterator – generates samples from the Cartesian product of
SampleIterators in `search_space
.

direction
¶ MAXIMIZE or MINIMIZE : default MAXIMIZE – determines the direction of optimization: if MAXIMIZE, the greatest value of
objective_function
is sought; if MINIMIZE, the least value is sought.

iteration
¶ int – the currention iteration of the optimization process.

num_iterations
¶ int – number of iterations required to complete the entire grid search; equal to the produce of all the
num
attributes of theSampleIterators
in thesearch_space
.

max_iterations
¶ int – determines the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the optimal sample of those evaluated.

save_samples
¶ True – determines whether or not to save and return all samples generated from
search_space
and evaluated by theobjective_function
in the optimization process.

save_values
¶ bool – determines whether or not to save and return the value of
objective_function
for all samples evaluated in the optimization process.

function
(variable=None, execution_id=None, params=None, context=None, **kwargs)¶ Return the sample that yields the optimal value of
objective_function
, and possibly all samples evaluated and their corresponding values.Optimal value is defined by
direction
:  if MAXIMIZE, returns greatest value  if MINIMIZE, returns least valueReturns: optimal sample, optimal value, saved_samples, saved_values – first array contains sample that yields the highest or lowest value of objective_function
, depending ondirection
, and the second array contains the value of the function for that sample. Ifsave_samples
isTrue
, first list contains all the values sampled in the order they were evaluated; otherwise it is empty. Ifsave_values
isTrue
, second list contains the values returned byobjective_function
for all the samples in the order they were evaluated; otherwise it is empty.Return type: ndarray, list, list
 get next sample from

class
psyneulink.core.components.functions.optimizationfunctions.
GaussianProcess
(default_variable=None, objective_function=None, direction=MAXIMIZE, max_iterations=1000, save_samples=False, save_values=False, params=None, owner=None, prefs=None)¶ Draw samples with dimensionality and bounds specified by
search_space
and return one that optimizes the value ofobjective_function
.Gaussian Process Procedure
The number of items (
SampleIterators
insearch_space
determines the dimensionality of each sample to evaluate byobjective_function
, with thestart
andstop
attributes of eachSampleIterator
specifying the bounds for sampling along the corresponding dimension.When
function
is executed, it iterates over the folowing steps: draw sample along each dimension of
search_space
, within bounds specified bystart
andstop
attributes of eachSampleIterator
in thesearch_space
list.
 compute value of
objective_function
for that sample;
The current iteration is contained in
iteration
. Iteration continues until [ FRED: FILL IN THE BLANK], ormax_iterations
is execeeded. The function returns the sample that yielded either the highest (ifdirection
is MAXIMIZE) or lowest (ifdirection
is MINIMIZE) value of theobjective_function
, along with the value for that sample, as well as lists containing all of the samples evaluated and their values if eithersave_samples
orsave_values
isTrue
, respectively.Parameters:  default_variable (list or ndarray : default None) – specifies a template for (i.e., an example of the shape of) the samples used to evaluate the
objective_function
.  objective_function (function or method) – specifies function used to evaluate sample in each iteration of the optimization process; it must be specified and must return a scalar value.
 search_space (list or array) – specifies bounds of the samples used to evaluate
objective_function
along each dimension ofvariable
; each item must be a tuple the first element of which specifies the lower bound and the second of which specifies the upper bound.  direction (MAXIMIZE or MINIMIZE : default MAXIMIZE) – specifies the direction of optimization: if MAXIMIZE, the highest value of
objective_function
is sought; if MINIMIZE, the lowest value is sought.  max_iterations (int : default 1000) – specifies the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the optimal sample of those evaluated.
 save_samples (bool) – specifies whether or not to return all of the samples used to evaluate
objective_function
in the optimization process (i.e., a copy of thesearch_space
.  save_values (bool) – specifies whether or not to save and return the values of
objective_function
for all samples evaluated in the optimization process.

variable
¶ ndarray – template for sample evaluated by
objective_function
.

objective_function
¶ function or method – function used to evaluate sample in each iteration of the optimization process.

search_space
¶ list or array – contains tuples specifying bounds within which each dimension of
variable
is sampled, and used to evaluateobjective_function
in iterations of the optimization process.

direction
¶ MAXIMIZE or MINIMIZE : default MAXIMIZE – determines the direction of optimization: if MAXIMIZE, the greatest value of
objective_function
is sought; if MINIMIZE, the least value is sought.

iteration
¶ int – the currention iteration of the optimization process.

max_iterations
¶ int – determines the maximum number of times the optimization process is allowed to iterate; if exceeded, a warning is issued and the function returns the optimal sample of those evaluated.

save_samples
¶ True – determines whether or not to save and return all samples evaluated by the
objective_function
in the optimization process (if the process completes, this should be identical tosearch_space
.

save_values
¶ bool – determines whether or not to save and return the value of
objective_function
for all samples evaluated in the optimization process.

function
(variable=None, execution_id=None, params=None, context=None, **kwargs)¶ Return the sample that yields the optimal value of
objective_function
, and possibly all samples evaluated and their corresponding values.Optimal value is defined by
direction
:  if MAXIMIZE, returns greatest value  if MINIMIZE, returns least valueReturns: optimal sample, optimal value, saved_samples, saved_values – first array contains sample that yields the highest or lowest value of objective_function
, depending ondirection
, and the second array contains the value of the function for that sample. Ifsave_samples
isTrue
, first list contains all the values sampled in the order they were evaluated; otherwise it is empty. Ifsave_values
isTrue
, second list contains the values returned byobjective_function
for all the samples in the order they were evaluated; otherwise it is empty.Return type: ndarray, list, list
 draw sample along each dimension of

class
psyneulink.core.components.functions.optimizationfunctions.
SampleIterator
(specification)¶ Create an iterator that returns the next sample from a sequence on each call to
next
. (e.g., when next(<SampleIterator>) is called)The pattern of the sequence depends on the specification, which may be a list, nparray, range, function, or a SampleSpec. Most of the patterns depend on the “current_step,” which is incremented on each iteration, and set to zero when the iterator is reset.
Specification what happens on each iteration StopIteration condition list, nparray look up the item with index current_step list/array range, np.arange start + step*current_step range stop value is reached callable call callable iteration does not stop SampleSpec(start, stop, step) start + step*current_step current_step = num or value > stop SampleSpec(start, stop, num) start + step*current_step current_step = num or value > stop SampleSpec(function, num) call function current_step = num SampleSpec(function) call function iteration does not stop Note
We recommend reserving the list/nparray option for cases in which the samples do not have a pattern that can be represented by a SampleSpec, or the number of samples is small. The list/nparray option requires all of the samples to be stored and looked up, while the SampleSpec options generate samples as needed.

reset
(head=None)¶ Reset iterator to a specified item If called with None, resets to first item (if
generator
is a list or deterministic function.
