Table of Contents

AutodiffComposition

Contents

Overview

Warning

As of PsyNeuLink 0.7.5, the API for using AutodiffCompositions has been slightly changed! Please see this link for more details!

AutodiffComposition is a subclass of Composition used to train feedforward neural network models through integration with PyTorch, a popular machine learning library, which executes considerably more quickly than using the standard implementation of learning in a Composition, using its learning methods. An AutodiffComposition is configured and run similarly to a standard Composition, with some exceptions that are described below.

Creating an AutodiffComposition

An AutodiffComposition can be created by calling its constructor, and then adding Components using the standard Composition methods for doing so. The constructor also includes an number of parameters that are specific to the AutodiffComposition. See the <class reference AutodiffComposition> for a list of these parameters.

Warning

Mechanisms or Projections should not be added to or deleted from an AutodiffComposition after it has been run for the first time. Unlike an ordinary Composition, AutodiffComposition does not support this functionality.

Warning

When comparing models built in PyTorch to those using AutodiffComposition, the bias parameter of PyTorch modules should be set to False, as AutodiffComposition does not currently support trainable biases.

Execution

An AutodiffComposition’s run, execute, and learn methods are the same as for a Composition.

The following is an example showing how to create a simple AutodiffComposition, specify its inputs and targets, and run it with learning enabled and disabled.

>>> import psyneulink as pnl
>>> # Set up PsyNeuLink Components
>>> my_mech_1 = pnl.TransferMechanism(function=pnl.Linear, size = 3)
>>> my_mech_2 = pnl.TransferMechanism(function=pnl.Linear, size = 2)
>>> my_projection = pnl.MappingProjection(matrix=np.random.randn(3,2),
...                     sender=my_mech_1,
...                     receiver=my_mech_2)
>>> # Create AutodiffComposition
>>> my_autodiff = pnl.AutodiffComposition()
>>> my_autodiff.add_node(my_mech_1)
>>> my_autodiff.add_node(my_mech_2)
>>> my_autodiff.add_projection(sender=my_mech_1, projection=my_projection, receiver=my_mech_2)
>>> # Specify inputs and targets
>>> my_inputs = {my_mech_1: [[1, 2, 3]]}
>>> my_targets = {my_mech_2: [[4, 5]]}
>>> input_dict = {"inputs": my_inputs, "targets": my_targets, "epochs": 2}
>>> # Run Composition in learnng mode
>>> my_autodiff.learn(inputs = input_dict)
>>> # Run Composition in test mode
>>> my_autodiff.run(inputs = input_dict['inputs'])

Logging

Logging in AutodiffCompositions follows the same procedure as logging in a Composition. However, since an AutodiffComposition internally converts all of its mechanisms to an equivalent PyTorch model, then its inner components are not actually executed. This means that there is limited support for logging parameters of components inside an AutodiffComposition; Currently, the only supported parameters are:

  1. the matrix parameter of Projections

  2. the value parameter of its inner components

Nested Execution

Like any other Composition, an AutodiffComposition may be nested inside another.

The following shows how the AutodiffComposition created in the previous example can be nested and run inside another Composition:

>>> # Create outer composition
>>> my_outer_composition = pnl.Composition()
>>> my_outer_composition.add_node(my_autodiff)
>>> # Specify dict containing inputs and targets for nested Composition
>>> training_input = {my_autodiff: input_dict}
>>> # Run in learning mode
>>> result1 = my_outer_composition.learn(inputs=training_input)

Class Reference

class psyneulink.library.compositions.autodiffcomposition.AutodiffComposition(learning_rate=None, optimizer_type='sgd', weight_decay=0, loss_spec='mse', disable_learning=False, refresh_losses=False, disable_cuda=True, cuda_index=None, force_no_retain_graph=False, pathways=None, name='autodiff_composition')

Subclass of Composition that trains models using PyTorch. See Composition for additional arguments and attributes.

Parameters

  • learning_rate (float : default 0.001) – the learning rate, which is passed to the optimizer.

  • disable_learning (bool: default False) – specifies whether the AutodiffComposition should disable learning when run in learning mode.

  • optimizer_type (str : default 'sgd') – the kind of optimizer used in training. The current options are ‘sgd’ or ‘adam’.

  • weight_decay (float : default 0) – specifies the L2 penalty (which discourages large weights) used by the optimizer.

  • loss_spec (str or PyTorch loss function : default 'mse') – specifies the loss function for training. The current string options are ‘mse’ (the default), ‘crossentropy’, ‘l1’, ‘nll’, ‘poissonnll’, and ‘kldiv’. Any PyTorch loss function can work here, such as ones from https://pytorch.org/docs/stable/nn.html#loss-functions

  • losses (list of floats) – tracks the average for each weight update (i.e. each minibatch)

  • optimizer (PyTorch optimizer function) – the optimizer used for training. Depends on the optimizer_type, learning_rate, and weight_decay arguments from initialization.

  • loss (PyTorch loss function) – the loss function used for training. Depends on the loss_spec argument from initialization.

_update_learning_parameters(context)

Updates parameters based on trials ran since last update.

_infer_output_nodes(nodes)

Maps targets onto target mechanisms (as needed by learning)

Returns

Return type

A dict mapping TargetMechanisms -> target values

_infer_input_nodes(nodes)

Maps targets onto target mechanisms (as needed by learning)

Returns

Return type

A dict mapping TargetMechanisms -> target values

learn(*args, **kwargs)

Runs the composition in learning mode - that is, any components with disable_learning False will be executed in learning mode. See Learning in a Composition for details.

Parameters

  • inputs ({Node:list }) –

    a dictionary containing a key-value pair for each Node (Mechanism or Composition) in the composition that receives inputs from the user. There are several equally valid ways that this dict can be structured:

    1. For each pair, the key is the and the value is an input, the shape of which must match the Node’s default variable. This is identical to the input dict in the run method (see Input Dictionary for additional details).

    2. A dict with keys ‘inputs’, ‘targets’, and ‘epochs’. The inputs key stores a dict that is the same same structure as input specification (1) of learn. The targets and epochs keys should contain values of the same shape as targets and epochs.

  • targets ({Node:list }) – a dictionary containing a key-value pair for each Node in the Composition that receives target values as input to the Composition for training learning pathways. The key of each entry can be either the TARGET_MECHANISM for a learning pathway or the final Node in that Pathway, and the value is the target value used for that Node on each trial (see target inputs for additional details concerning the formatting of targets).

  • num_trials (int (default=None)) – typically, the Composition infers the number of trials to execute from the length of its input specification. However, num_trials can be used to enforce an exact number of trials to execute; if it is greater than there are inputs then inputs will be repeated (see Composition Inputs for additional information).

  • epochs (int (default=1)) – specifies the number of training epochs (that is, repetitions of the batched input set) to run with

  • minibatch_size (int (default=1)) – specifies the size of the minibatches to use. The input trials will be batched and ran, after which learning mechanisms with learning mode TRIAL will update weights

  • randomize_minibatch (bool (default=False)) – specifies whether the order of the input trials should be randomized on each epoch

  • patience (int or None (default=None)) – used for early stopping of training; If a model has more than patience bad consecutive epochs, then learn will prematurely return. A bad epoch is determined by the min_delta value

  • min_delta (float (default=0)) – the minimum reduction in average loss that an epoch must provide in order to qualify as a ‘good’ epoch; Any reduction less than this value is considered to be a bad epoch. Used for early stopping of training, in combination with patience.

  • scheduler (Scheduler) – the scheduler object that owns the conditions that will instruct the execution of the Composition If not specified, the Composition will use its automatically generated scheduler.

  • context – context will be set to self.default_execution_id if unspecified

  • call_before_minibatch (callable) – called before each minibatch is executed

  • call_after_minibatch (callable) – called after each minibatch is executed

  • report_output (ReportOutput : default ReportOutput.OFF) – specifies whether to show output of the Composition and its Nodes trial-by-trial as it is generated; see Output Reporting for additional details and ReportOutput for options.

  • report_params (ReportParams : default ReportParams.OFF) – specifies whether to show values the Parameters of the Composition and its Nodes as part of the output report; see Output Reporting for additional details and ReportParams for options.

  • report_progress (ReportProgress : default ReportProgress.OFF) – specifies whether to report progress of execution in real time; see Progress Reporting for additional details.

  • report_simulations (ReportSimulatons : default ReportSimulations.OFF) – specifies whether to show output and/or progress for simulations executed by the Composition’s controller; see Simulations for additional details.

  • report_to_devices (list(ReportDevices) : default ReportDevices.CONSOLE) – specifies where output and progress should be reported; see Report_To_Devices for additional details and ReportDevices for options.

Returns

the results of the final epoch of training

Return type

list

execute(inputs=None, num_trials=None, minibatch_size=1, do_logging=False, scheduler=None, termination_processing=None, call_before_minibatch=None, call_after_minibatch=None, call_before_time_step=None, call_before_pass=None, call_after_time_step=None, call_after_pass=None, reset_stateful_functions_to=None, context=None, base_context=<psyneulink.core.globals.context.Context object>, clamp_input='soft_clamp', targets=None, runtime_params=None, execution_mode=ExecutionMode.Python, skip_initialization=False, report_output=ReportOutput.OFF, report_params=ReportParams.OFF, report_progress=ReportProgress.OFF, report_simulations=ReportSimulations.OFF, report_to_devices=None, report=None, report_num=None)

Passes inputs to any Nodes receiving inputs directly from the user (via the “inputs” argument) then coordinates with the Scheduler to execute sets of Nodes that are eligible to execute until termination conditions are met.

Parameters

  • inputs ({ Node: list } : default None) – a dictionary containing a key-value pair for each Node in the Composition that receives inputs from the user. For each pair, the key is the Node (a Mechanism or Composition) and the value is an input, the shape of which must match the Node’s default variable. If inputs is not specified, the default_variable for each INPUT Node is used as its input (see Input Formats for additional details).

  • clamp_input (SOFT_CLAMP : default SOFT_CLAMP) –

  • runtime_params (Dict[Node: Dict[Parameter: Tuple(Value, Condition)]] : default None) – specifies alternate parameter values to be used only during this EXECUTION when the specified Condition is met (see Runtime Parameters for more details and examples of valid dictionaries).

  • skip_initialization (: default False) –

  • scheduler (Scheduler : default None) – the scheduler object that owns the conditions that will instruct the execution of the Composition If not specified, the Composition will use its automatically generated scheduler.

  • context (execution_id : default default_execution_id) – execution context in which the Composition will be executed.

  • base_context (execution_id : Context(execution_id=None)) – the context corresponding to the execution context from which this execution will be initialized, if values currently do not exist for context.

  • call_before_time_step (callable : default None) – called before each TIME_STEP is executed passed the current context (but it is not necessary for your callable to take).

  • call_after_time_step (callable : default None) – called after each TIME_STEP is executed passed the current context (but it is not necessary for your callable to take).

  • call_before_pass (callable : default None) – called before each PASS is executed passed the current context (but it is not necessary for your callable to take).

  • call_after_pass (callable : default None) – called after each PASS is executed passed the current context (but it is not necessary for your callable to take).

  • execution_mode (enum.Enum[Auto|LLVM|LLVMexec|Python|PTXExec] : default Python) – specifies whether to run using the Python interpreter or a compiled mode. see execution_mode argument of run method for additional details.

  • report_output (ReportOutput : default ReportOutput.OFF) – specifies whether to show output of the Composition and its Nodes for the execution; see Output Reporting for additional details and ReportOutput for options.

  • report_params (ReportParams : default ReportParams.OFF) – specifies whether to show values the Parameters of the Composition and its Nodes for the execution; see Output Reporting for additional details and ReportParams for options.

  • report_progress (ReportProgress : default ReportProgress.OFF) – specifies whether to report progress of the execution; see Progress Reporting for additional details.

  • report_simulations (ReportSimulations : default ReportSimulations.OFF) – specifies whether to show output and/or progress for simulations executed by the Composition’s controller; see Simulations for additional details.

  • report_to_devices (list(ReportDevices) : default ReportDevices.CONSOLE) – specifies where output and progress should be reported; see Report_To_Devices for additional details and ReportDevices for options.

Returns

  • output_values (List)

  • These are the values of the Composition’s output_CIM.output_ports, excluding those the source of which

  • are from a (potentially nested) Node with NodeRole.PROBE in its enclosing Composition.