AutodiffComposition¶
Contents¶
Overview¶
AutodiffComposition is a subclass of Composition for constructing and training feedforward neural network either, using either direct compilation (to LLVM) or automatic conversion to PyTorch, both of which considerably accelerate training (by as much as three orders of magnitude) compared to the standard implementation of learning in a Composition. Although an AutodiffComposition is constructed and executed in much the same way as a standard Composition, it largely restricted to feedforward neural networks using supervised learning, and in particular the the backpropagation learning algorithm. although it can be used for some forms of unsupervised learning that are supported in PyTorch (e.g., self-organized maps).
Creating an AutodiffComposition¶
An AutodiffComposition can be created by calling its constructor, and then adding Components using
the standard Composition methods for doing so (e.g., add_node
,
add_projection
, add_linear_processing_pathway
, etc.). The constructor also includes a number of parameters that are
specific to the AutodiffComposition (see Class Reference for a list of these parameters,
and examples below). While an AutodiffComposition can generally be created using the
same methods as a standard Composition, there are a few restrictions that apply to its construction, summarized below.
Only one OutputPort per Node¶
The Nodes of an AutodiffComposition currently can have only one OutputPort, though that
can have more than one efferent
MappingProjection. Nodes can also have more than one
InputPort, that can receive more than one afferent `path_afferent
Projections.
No Modulatory Components¶
All of the Components in an AutodiffComposition must be able to be subjected to learning, which means that no ModulatoryMechanisms can be included in an AutodiffComposition. Specifically, this precludes any learning components, ControlMechanisms, or a controller.
Learning Components. An AutodiffComposition cannot include any learning components themselves (i.e., LearningMechanisms, LearningSignals, or LearningProjections <LearningProjection>`, nor the ComparatorMechanism
or ObjectiveMechanism used to compute the loss for learning). These are constructed
automatically when learning is executed in Python mode or LLVM mode, and PyTorch-compatible Components are constructed when it is executed in PyTorch mode.
Control Components. An AutodiffComposition also cannot include any ControlMechanisms or a controller. However, it can include Mechanisms that are subject to modulatory control (see Figure, and modulation) by ControlMechanisms outside the Composition, including the controller of a Composition within which the AutodiffComposition is nested. That is, an AutodiffComposition can be nested in a Composition that has such other Components (see Nested Execution and Modulation below).
No Bias Parameters¶
AutodiffComposition does not (currently) support the automatic construction of separate bias parameters.
Thus, when constructing a model using an AutodiffComposition that corresponds to one in PyTorch, the bias
parameter of PyTorch modules should be set
to False
. Trainable biases can be specified explicitly in an AutodiffComposition by including a
TransferMechanism that projects to the relevant Mechanism (i.e., implementing that layer of the network to
receive the biases) using a MappingProjection with a matrix
parameter that
implements a diagnoal matrix with values corresponding to the initial value of the biases.
Nesting¶
An AutodiffComposition can be nested inside another Composition for learning, and there can
be any level of such nestings. However, all of the nested Compositions must be AutodiffCompositions. Furthermore, all
nested Compositions use the learning_rate
specified for the outermost Composition,
whether this is specified in the call to its learn
method, its constructor, or its
default value is being used (see learning_rate
below for additional details).
Projections from Nodes in an immediately enclosing outer Composition to the input_CIM
of a nested Composition, and from its output_CIM
to Nodes
in the outer Composition are subject to learning; however those within the nested Composition itself (i.e.,
from its input_CIM to its INPUT Nodes and from its OUTPUT Nodes to its output_CIM) are not subject to learning,
as they serve simply as conduits of information between the outer Composition and the nested one.
Warning
Nested Compositions are supported for learning only in PyTorch mode, and will
cause an error if the learn
method of an AutodiffComposition is executed in
Python mode or LLVM mode.
No Post-construction Modification¶
Mechanisms or Projections should not be added to or deleted from an AutodiffComposition after it has been executed. Unlike an ordinary Composition, AutodiffComposition does not support this functionality.
Execution¶
An AutodiffComposition’s run
, execute
, and learn
methods are the same as for a Composition. However, the execution_mode in the learn
method has different effects than for a standard Composition, that determine whether it uses LLVM compilation or translation to PyTorch to execute learning.
These are each described in greater detail below, and summarized in this table
which provides a comparison of the different modes of execution for an AutodiffComposition and standard Composition.
PyTorch mode¶
This is the default for an AutodiffComposition, but, can be specified explicitly by setting **execution_mode =
ExecutionMode.PyTorch
in the learn
method (see example
in Basics and Primer). In this mode, the AutodiffComposition is automatically translated to a PyTorch model for learning. This is comparable in speed to LLVM compilation
, but provides greater flexiblity, including the ability to include nested
AutoDiffCompositions in learning. Although it is best suited for use with supervised learning, it can also be used for some forms of unsupervised learning that are supported in PyTorch (e.g., self-organized maps).
Note
While specifying
ExecutionMode.PyTorch
in thelearn
method of an AutodiffComposition causes it to use PyTorch for training, specifying this in therun
method causes it to be executed using the Python interpreter (and not PyTorch); this is so that any modulation can take effect during execution (see Nested Execution and Modulation below), which is not supported by PyTorch.Warning
Specifying
ExecutionMode.LLVM
orExecutionMode.PyTorch
in the learn() method of a standard Composition causes an error.
LLVM mode¶
This is specified by setting execution_mode = ExecutionMode.LLVMRun
in the learn
method
of an AutodiffCompositon. This provides the fastest performance, but is limited to supervised learning using the BackPropagation
algorithm. This can be run using standard forms of
loss, including mean squared error (MSE) and cross entropy, by specifying this in the loss_spec argument of
the constructor (see AutodiffComposition for additional details, and
Compilation Modes for more information about executing a Composition in compiled mode.
Note
Specifying
ExecutionMode.LLVMRUn
in either thelearn
andrun
methods of an AutodiffComposition causes it to (attempt to) use compiled execution in both cases; this is because LLVM compilation supports the use of modulation in PsyNeuLink models (as compared to PyTorch mode; see note below).
Python mode¶
An AutodiffComposition can also be run using the standard PsyNeuLink learning components. However, this cannot be used if the AutodiffComposition has any nested Compositions, irrespective of whether they are ordinary Compositions or AutodiffCompositions.
Nested Execution and Modulation¶
# FIX:
Like any other Composition, an AutodiffComposition may be nested inside another
(see example below). However, during learning, none of the internal
Components of the AutodiffComposition (e.g., intermediate layers of a neural network model) are accessible to the
other Components of the outer Composition, (e.g., as sources of information, or for modulation). However, when
it is executed using its run
method, then the AutodiffComposition functions like any other,
and all of its internal Components are accessible to other Components of the outer Composition. Thus, as long as access
to its internal Components is not needed during learning, an AutodiffComposition can be trained, and then used to
execute the trained Composition like any other.
Logging¶
Logging in AutodiffCompositions follows the same procedure as logging in a Composition. However, since an AutodiffComposition internally converts all of its Mechanisms either to LLVM or to an equivalent PyTorch model, then its inner components are not actually executed. This means that there is limited support for logging parameters of components inside an AutodiffComposition; Currently, the only supported parameters are:
Examples
The following is an example showing how to create a simple AutodiffComposition, specify its inputs and targets, and run it with learning enabled and disabled:
>>> import psyneulink as pnl
>>> # Set up PsyNeuLink Components
>>> my_mech_1 = pnl.TransferMechanism(function=pnl.Linear, size = 3)
>>> my_mech_2 = pnl.TransferMechanism(function=pnl.Linear, size = 2)
>>> my_projection = pnl.MappingProjection(matrix=np.random.randn(3,2),
... sender=my_mech_1,
... receiver=my_mech_2)
>>> # Create AutodiffComposition
>>> my_autodiff = pnl.AutodiffComposition()
>>> my_autodiff.add_node(my_mech_1)
>>> my_autodiff.add_node(my_mech_2)
>>> my_autodiff.add_projection(sender=my_mech_1, projection=my_projection, receiver=my_mech_2)
>>> # Specify inputs and targets
>>> my_inputs = {my_mech_1: [[1, 2, 3]]}
>>> my_targets = {my_mech_2: [[4, 5]]}
>>> input_dict = {"inputs": my_inputs, "targets": my_targets, "epochs": 2}
>>> # Run Composition in learnng mode
>>> my_autodiff.learn(inputs = input_dict)
>>> # Run Composition in test mode
>>> my_autodiff.run(inputs = input_dict['inputs'])
The following shows how the AutodiffComposition created in the previous example can be nested and run inside another Composition:
>>> # Create outer composition
>>> my_outer_composition = pnl.Composition()
>>> my_outer_composition.add_node(my_autodiff)
>>> # Specify dict containing inputs and targets for nested Composition
>>> training_input = {my_autodiff: input_dict}
>>> # Run in learning mode
>>> result1 = my_outer_composition.learn(inputs=training_input)
Class Reference¶
- class psyneulink.library.compositions.autodiffcomposition.AutodiffComposition(pathways=None, learning_rate=None, optimizer_type='sgd', weight_decay=0, loss_spec=Loss.MSE, disable_learning=False, refresh_losses=False, disable_cuda=True, cuda_index=None, force_no_retain_graph=False, name='autodiff_composition', **kwargs)¶
Subclass of Composition that trains models using either LLVM compilation or PyTorch; see and Composition for additional arguments and attributes.
- Parameters
learning_rate (float : default 0.001) – specifies the learning rate passed to the optimizer if none is specified in the
learn
method of the AutodiffComposition (seelearning_rate
for additional details).disable_learning (bool: default False) – specifies whether the AutodiffComposition should disable learning when run in
learning mode
.optimizer_type (str : default 'sgd') – the kind of optimizer used in training. The current options are ‘sgd’ or ‘adam’.
weight_decay (float : default 0) – specifies the L2 penalty (which discourages large weights) used by the optimizer.
loss_spec (Loss or PyTorch loss function : default Loss.MSE) – specifies the loss function for training; see
Loss
for arguments.
- optimizer¶
the optimizer used for training. Depends on the optimizer_type, learning_rate, and weight_decay arguments from initialization.
- Type
PyTorch optimizer function
- learning_rate¶
determines the learning_rate passed the optimizer, and is applied to all Projections in the AutodiffComposition that are
learnable
.Note
At present, the same learning rate is applied to all Components of an AutodiffComposition, irrespective of the
learning_rate
that may be specified for any individual Mechanisms or any nested Compositions; in the case of the latter, thelearning_rate
of the outermost AutodiffComposition is used, whether this is specified in the call to itslearn
method, its constructor, or its default value is being used.Hint
To disable updating of a particular MappingProjection in an AutodiffComposition, specify the learnable parameter of its constructor as
False
; this applies to MappingProjections at any level of nesting.- Type
float
- loss¶
the loss function used for training. Depends on the loss_spec argument from initialization.
- Type
PyTorch loss function
- losses¶
tracks the average loss after each weight update (i.e. each minibatch) during learning.
- Type
list of floats
- last_saved_weights¶
path for file to which weights were last saved.
- Type
path
- last_loaded_weights¶
path for file from which weights were last loaded.
- Type
path
- class PytorchCompositionWrapper(composition, device, outer_creator=None, context=None)¶
Wrapper for a Composition as a Pytorch Module Set up parameters of PyTorch model & information required for forward computation
Handle nested compositions (flattened in infer_backpropagation_learning_pathways): Deal with Projections into or out of a nested Composition as follows:
- [ OUTER ][ NESTED ][ OUTER ]
learnable// not learnable// not learnable// learnable//
- —> [Node] —-> [input_CIM] ~~~> [INPUT Node] —-> [OUTPUT Node] ~~~> [output_CIM] —-> [Node] —>
- sndr rcvr nested_rcvr nested_sndr sndr rcvr
^–projection–>^ ^—projection–>^ ^—-PytorchProjectionWrapper—->^ ^—-PytorchProjectionWrapper—->^
ENTRY EXIT
- nodes¶
- Type
List[PytorchMechanismWrapper]
- projections_map¶
keys are Projections in the Composition being wrapped, and keys are the ProjectionWrappers to which they are mapped (see above).
- Type
Dict[Projection, PytorchProjectionWrapper]
- _regenerate_paramlist()¶
Add Projection matrices to Pytorch Module’s parameter list
- execute_node(node, variable, context=None)¶
Execute node and store the result in the node’s value attribute Implemented as method (and includes context as arg) so that it can be overridden by subclasses of PytorchCompositionWrapper
- forward(inputs, context=None)¶
Forward method of the model for PyTorch and LLVM modes Returns a dictionary {output_node:value} of output values for the model
- Return type
dict
- pytorch_composition_wrapper_type¶
alias of
psyneulink.library.compositions.pytorchwrappers.PytorchCompositionWrapper
- class Parameters(owner, parent=None)¶
- infer_backpropagation_learning_pathways(execution_mode, context=None)¶
Create backpropapagation learning pathways for every Input Node –> Output Node pathway Flattens nested compositions:
only includes the Projections in outer Composition to/from the CIMs of the nested Composition (i.e., to input_CIMs and from output_CIMs) – the ones that should be learned;
excludes Projections from/to CIMs in the nested Composition (from input_CIMs and to output_CIMs), as those should remain identity Projections;
see
PytorchCompositionWrapper
for table of how Projections are handled and further details.
- _build_pytorch_representation(context=None, refresh=False)¶
Builds a Pytorch representation of the AutodiffComposition
- autodiff_training(inputs, targets, context=None, scheduler=None)¶
Perform learning/training on all input-target pairs received for given number of epochs
- _update_learning_parameters(context)¶
Carry out backpropagation learning for one or more trials Updates parameters (weights) based on trials run since last update. Uses Pytorch backward method to compute gradients and update weights
- _infer_input_nodes(input_dict)¶
Remove TARGET Nodes, and return dict with values of INPUT Nodes for single trial For nested Compositions, replace input to nested Composition with inputs to its INPUT Nodes For InuptPorts, replace with owner
- Returns
- Return type
A dict mapping INPUT Nodes -> input values for a single trial
- _infer_output_nodes(input_dict)¶
Remove INPUT Nodes, and return dict with values for TARGET Nodes
Get Inputs to TARGET Nodes and assign to dict mapping them to OUTPUT Nodes of Composition, which are used for computation of loss in autodiff_training().
- Returns
- Return type
A dict mapping TARGET Nodes -> target values corresponding to OUTPUT Nodes of Composition
- _parse_learning_spec(inputs, targets, execution_mode, context)¶
Converts learning inputs and targets to a standardized form
- Returns
dict
– Dict mapping mechanisms to values (with TargetMechanisms inferred from output nodes if needed)int
– Number of input sets in dict for each input node in the Composition
- learn(*args, **kwargs)¶
Runs the composition in learning mode - that is, any components with disable_learning False will be executed in learning mode. See Learning in a Composition for details.
- Parameters
inputs ({Node:list }) –
a dictionary containing a key-value pair for each Node (Mechanism or Composition) in the composition that receives inputs from the user. There are several equally valid ways that this dict can be structured:
For each pair, the key is the and the value is an input, the shape of which must match the Node’s default variable. This is identical to the input dict in the
run
method (see Input Dictionary for additional details).A dict with keys ‘inputs’, ‘targets’, and ‘epochs’. The
inputs
key stores a dict that is the same same structure as input specification (1) of learn. Thetargets
andepochs
keys should contain values of the same shape astargets
andepochs
.
targets ({Node:list }) – a dictionary containing a key-value pair for each Node in the Composition that receives target values as input to the Composition for training learning pathways. The key of each entry can be either the TARGET_MECHANISM for a learning pathway or the final Node in that Pathway, and the value is the target value used for that Node on each trial (see target inputs for additional details concerning the formatting of targets).
num_trials (int (default=None)) – typically, the Composition infers the number of trials to execute from the length of its input specification. However, num_trials can be used to enforce an exact number of trials to execute; if it is greater than there are inputs then inputs will be repeated (see Composition Inputs for additional information).
epochs (int (default=1)) – specifies the number of training epochs (that is, repetitions of the batched input set) to run with
learning_rate (float : default None) – specifies the learning_rate used by all learning pathways when the Composition’s learn method is called. This overrides the
learning_rate specified for any individual Pathways at construction, but only applies for the current execution of the learn method (see `Composition_Learning_Rate
for additional details).minibatch_size (int (default=1)) – specifies the size of the minibatches to use. The input trials will be batched and run, after which learning mechanisms with learning mode TRIAL will update weights
randomize_minibatch (bool (default=False)) – specifies whether the order of the input trials should be randomized on each epoch
patience (int or None (default=None)) – used for early stopping of training; If a model has more than
patience
bad consecutive epochs, thenlearn
will prematurely return. A bad epoch is determined by themin_delta
valuemin_delta (float (default=0)) – the minimum reduction in average loss that an epoch must provide in order to qualify as a ‘good’ epoch; Any reduction less than this value is considered to be a bad epoch. Used for early stopping of training, in combination with
patience
.scheduler (Scheduler) – the scheduler object that owns the conditions that will instruct the execution of the Composition If not specified, the Composition will use its automatically generated scheduler.
context – context will be set to self.default_execution_id if unspecified
call_before_minibatch (callable) – called before each minibatch is executed
call_after_minibatch (callable) – called after each minibatch is executed
report_output (ReportOutput : default ReportOutput.OFF) – specifies whether to show output of the Composition and its Nodes trial-by-trial as it is generated; see Output Reporting for additional details and
ReportOutput
for options.report_params (ReportParams : default ReportParams.OFF) – specifies whether to show values the Parameters of the Composition and its Nodes as part of the output report; see Output Reporting for additional details and
ReportParams
for options.report_progress (ReportProgress : default ReportProgress.OFF) – specifies whether to report progress of execution in real time; see Progress Reporting for additional details.
report_simulations (ReportSimulatons : default ReportSimulations.OFF) – specifies whether to show output and/or progress for simulations executed by the Composition’s controller; see Simulations for additional details.
report_to_devices (list(ReportDevices) : default ReportDevices.CONSOLE) – specifies where output and progress should be reported; see
Report_To_Device
for additional details andReportDevices
for options.
- Returns
the results of the last trial of training (list)
.. note:: – the results of the final epoch of training are stored in the Composition’s
learning_results
attribute.
- _get_execution_mode(execution_mode)¶
Parse execution_mode argument and return a valid execution mode for the learn() method Can be overridden by subclasses to change the permitted and/or default execution mode for learning
- execute(inputs=None, num_trials=None, minibatch_size=1, do_logging=False, scheduler=None, termination_processing=None, call_before_minibatch=None, call_after_minibatch=None, call_before_time_step=None, call_before_pass=None, call_after_time_step=None, call_after_pass=None, reset_stateful_functions_to=None, context=None, base_context=<psyneulink.core.globals.context.Context object>, clamp_input='soft_clamp', targets=None, runtime_params=None, execution_mode=ExecutionMode.PyTorch, skip_initialization=False, report_output=ReportOutput.OFF, report_params=ReportParams.OFF, report_progress=ReportProgress.OFF, report_simulations=ReportSimulations.OFF, report_to_devices=None, report=None, report_num=None)¶
Override to execute autodiff_training() in learning mode if execute_mode is not Python
- save(path=None, directory=None, filename=None, context=None)¶
Saves all weight matrices for all MappingProjections in the AutodiffComposition
- Parameters
path (Path, PosixPath or str : default None) – path specification; must be a legal path specification in the filesystem.
directory (str : default
current working directory
) – directory wherematrices
for all MappingProjections in the AutodiffComposition are saved.filename (str : default
<name of AutodiffComposition>_matrix_wts.pnl
) – filename in whichmatrices
for all MappingProjections in the AutodiffComposition are saved.note:: (.) – Matrices are saved in PyTorch state_dict format.
- Returns
- Return type
Path
- load(path=None, directory=None, filename=None, context=None)¶
Loads all weight matrices for all MappingProjections in the AutodiffComposition from file :type path:
PosixPath
:param path: Path for file in which MappingProjectionmatrices
are stored.This must be a legal PosixPath object; if it is specified directory and filename are ignored.
- Parameters
directory (str : default
current working directory
) – directory where MappingProjectionmatrices
are stored.filename (str : default
<name of AutodiffComposition>_matrix_wts.pnl
) – name of file in which MappingProjectionmatrices
are stored.note:: (.) –
Matrices must be stored in PyTorch state_dict format.
- parameters = <psyneulink.library.compositions.autodiffcomposition.AutodiffComposition.Parameters object> : ( execute_until_finished = Parameter( default_value=True delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=True mdf_name=None name='execute_until_finished' parse_spec=False pnl_internal=True port=None reference=False specify_none=False stateful=True structural=False user=True values={} ), execution_count = Parameter( default_value=0 delivery_condition=<LogCondition.OFF: 0> dependencies=None fallback_default=True function_arg=True history={} history_max_length=1 history_min_length=0 loggable=False mdf_name=None name='execution_count' parse_spec=False pnl_internal=True port=None read_only=True reference=False specify_none=False stateful=False structural=False user=True values={} ), has_initializers = Parameter( default_value=False delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=True mdf_name=None name='has_initializers' parse_spec=False pnl_internal=True port=None reference=False setter=<function _has_initializers_setter> specify_none=False stateful=True structural=False user=True values={} ), input_specification = Parameter( default_value=None delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=False mdf_name=None name='input_specification' parse_spec=False pnl_internal=True port=None reference=False specify_none=False stateful=False structural=False user=True values={} ), is_finished_flag = Parameter( default_value=True delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=False mdf_name=None name='is_finished_flag' parse_spec=False pnl_internal=False port=None reference=False specify_none=False stateful=True structural=False user=True values={} ), learning_rate = Parameter( default_value=0.001 delivery_condition=<LogCondition.OFF: 0> dependencies=None fallback_default=True function_arg=True history={} history_max_length=1 history_min_length=0 loggable=True mdf_name=None name='learning_rate' parse_spec=False pnl_internal=False port=None reference=False specify_none=False stateful=True structural=False user=True values={} ), learning_results = Parameter( default_value=[] delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=False mdf_name=None name='learning_results' parse_spec=False pnl_internal=True port=None reference=False specify_none=False stateful=True structural=False user=True values={} ), losses = Parameter( default_value=[] delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=True mdf_name=None name='losses' parse_spec=False pnl_internal=False port=None reference=False specify_none=False stateful=True structural=False user=True values={} ), max_executions_before_finished = Parameter( default_value=1000 delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=True mdf_name=None name='max_executions_before_finished' parse_spec=False pnl_internal=True port=None reference=False specify_none=False stateful=True structural=False user=True values={} ), num_executions = Parameter( default_value=Time(run: 0, trial: 0, pass: 0, time_step: 0) delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=False mdf_name=None name='num_executions' parse_spec=False pnl_internal=True port=None read_only=True reference=False specify_none=False stateful=True structural=False user=True values={} ), num_executions_before_finished = Parameter( default_value=0 delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=True mdf_name=None name='num_executions_before_finished' parse_spec=False pnl_internal=True port=None read_only=True reference=False specify_none=False stateful=True structural=False user=True values={} ), optimizer = Parameter( default_value=None delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=True mdf_name=None name='optimizer' parse_spec=False pnl_internal=False port=None reference=False specify_none=False stateful=True structural=False user=True values={} ), pytorch_representation = Parameter( default_value=None delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=True mdf_name=None name='pytorch_representation' parse_spec=False pnl_internal=False port=None reference=False specify_none=False stateful=True structural=False user=True values={} ), results = Parameter( default_value=[] delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=False mdf_name=None name='results' parse_spec=False pnl_internal=True port=None reference=False specify_none=False stateful=True structural=False user=True values={} ), retain_old_simulation_data = Parameter( default_value=False delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=False mdf_name=None name='retain_old_simulation_data' parse_spec=False pnl_internal=True port=None reference=False specify_none=False stateful=False structural=False user=True values={} ), simulation_results = Parameter( default_value=[] delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=False mdf_name=None name='simulation_results' parse_spec=False pnl_internal=True port=None reference=False specify_none=False stateful=True structural=False user=True values={} ), tracked_loss = Parameter( default_value=None delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=True mdf_name=None name='tracked_loss' parse_spec=False pnl_internal=True port=None reference=False specify_none=False stateful=True structural=False user=True values={} ), tracked_loss_count = Parameter( default_value=0 delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=True mdf_name=None name='tracked_loss_count' parse_spec=False pnl_internal=True port=None reference=False specify_none=False stateful=True structural=False user=True values={} ), trial_losses = Parameter( default_value=[] delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=True mdf_name=None name='trial_losses' parse_spec=False pnl_internal=False port=None reference=False specify_none=False stateful=True structural=False user=True values={} ), value = Parameter( default_value=array([0]) delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=True mdf_name=None name='value' parse_spec=False pnl_internal=True port=None read_only=True reference=False specify_none=False stateful=True structural=False user=True values={} ), variable = Parameter( constructor_argument='default_variable' default_value=array([0]) delivery_condition=<LogCondition.OFF: 0> dependencies=None function_arg=True history={} history_max_length=1 history_min_length=0 loggable=True mdf_name=None name='variable' parse_spec=False pnl_internal=True port=None read_only=True reference=False specify_none=False stateful=True structural=False user=True values={} ), )¶