AutodiffComposition

Related

• Learning in a Composition

• BIAS Nodes

Contents

• Overview

• Creating an AutodiffComposition

  • `AutodiffComposition_Configuring_Learning

  • Learning Pathways

    • Sample

    • Target

    • LossMechanism

    • Specifying sample-target pairs

  • Learning Rates

  • Exchanging Parameters with Pytorch Modules

  • AutodiffComposition Restrictions

• Structure

  • Learning Components

    • Loss Mechanism

    • TARGET_MECHANISM

  • Pytorch Representation

  • Nesting

• Execution

  • PyTorch mode

    • Additional Optimizations Steps

    • Synchronization of PsyNeuLink Values with PyTorch

    • Saving Pytorch Training Data

  • LLVM mode

  • Python mode

  • Logging

• Examples

• Class Reference

Overview

AutodiffComposition is a subclass of Composition for constructing and training neural networks using PyTorch and, in some cases, direct compilation using LLVM. These can considerably accelerate training, by as much as three orders of magnitude compared to Python mode used by a standard Composition. An AutodiffComposition is constructed and executed in the same way as a standard Composition, though it provides additional functionality, including:

• use of internal target signals for training;

• training of nested Compositions.

• training of recurrent neural networks (RNNs, e.g., GRUComposition);

• training of external (episodic) memory structures (e.g., EMComposition);

In addition to supporting supervised learning using the backpropagation learning algorithm, it also supports some forms of unsupervised learning that are possible in PyTorch (e.g., self-organized maps).

Creating an AutodiffComposition

An AutodiffComposition is created in the same way as a standard Composition, with the following differences:

• learning pathways are configured by specifing pairs of samples and targets (or “teachers”), each of which is a Mechanism or the OutputPort of one and the values of which are used to compute the loss on each trial of training (see below for details of specification);

• the constructor includes a number of additional arguments that are specific to the AutodiffComposition;

• there are some restrictions that apply to its construction;

• an AutodiffComposition’s pytorch_representation is used to execute it in PyTorch, which is constructed when its learn() method is called (see Pytorch Representation for additional details).

A learning Pathway in an AutodiffComposition, as in a Composition, is a Pathway that contains one or more learnable MappingProjections – that is, in which the learnable attribute of the MappingProjection is set to True. Unlike a Composition, however, the SAMPLE_MECHANISM (“student”) and TARGET_MECHANISM (“teacher”) for each learning Pathway can be specified in the targets argument of the AutodiffComposition’s constructor, as described below. If these are not specified, then these are configured automatically as for a Composition, by assigning the OUTPUT Node as the SAMPLE_MECHANISM for every pathway that has at least one learnable MappingProjection, and aautomatically constructing a corresponding TARGET_MECHANISM, the input for which is provided in the inputs or targets argument of the learn() method, and used to train that learning Pathway.

Configuring Learning Pathways

A learning Pathway can be configured by specifying either a sample-target target pair – or a LossMechanism that specifies these as its SAMPLE and TARGET InputPorts – in the targets argument of the AutodiffComposition’s constructor. These Components are described below, followed by the ways in which they can be specified in the constructor’s targets argument. However, a few important rules apply:

• a Mechanism or OutputPort specified as a sample must follow at least one learnable MappingProjection in its Pathway;

• every learnable MappingProjection must be followed somewhere by a sample;

• if a sample is followed somewhere in a Pathway by a learnable MappingProjection, a warning will be issued.

• a sample can have only one target, though a target can be used to train more than one sample.

• the target for a sample cannot be in the same Pathway as that sample.

Note

Since pathways can overlap (converge and/or diverge; e.g., see figure), a learnable MappingProjection may influenced by the training of several samples (see note below).

Note

Because a learning Pathway must have at least one learnable MappingProjection, a Pathway with a single Mechanism (i.e., a SINGLETON Node) is not learnable. This differs from configuration in Pytorch, in which a single torch.nn.Module can be trained since it is automatically assigned parameters (based on its input dimensionality) at construction; this can be thought of as equivalent to – and can be replicated in PsyNeulink by – constructing a leaning Pathway with a single learnable MappingProjection from an input Node to a Mechanism that that corresponds to (i.e., implements the same function as) the torch.nn.Module being trained. In other words, in PsyNeulink, the equivalent of a module’s parameters must be constructed explicity in the form of a learnable afferent MappingProjection which, in turn, requires a node that sends that Projection to the Mechanism.

The technical reason that a pathway with only a SINGLETON Node cannot be trained is that its afferent and efferent MappingProjections are from the input_CIM and to the output_CIM of the Composition to which it belongs. Such MappingProjections (i.e., from an input_CIM to its INPUT Nodes nor those from its OUTPUT Nodes to its output_CIM) are not learnable; they serve simply as conduits of information between the Composition and either the Composition within which it is nested, or the “outside world.”

Sample

This generates the value being trained (sometimes referred to as the “student”). It is the OutputPort of a SAMPLE_MECHANISM in a learning Pathway, that can be assigned anywhere in an AutodiffComposition, or in one nested within it, subject to the rules outlined above. The value of the sample is trained using the value of the target with which it is paired, or by values specified in the targets argument of the learn() method (see below). A sample can be assigned only a single target, though a target can be assigned to multiple samples. The SAMPLE_MECHANISMs of an AutodiffComposition are assigned the NodeRole SAMPLE and are listed in its sample_mechanisms attribute.

Note

Although a sample can be assigned only one target, it can participate in (i.e., be an intermediate Node) in other learning pathways, in which case the error signal it receives from its target will be combined with those that are transmitted to it from any other learning pathways in which it participates when the gradients are calcuated by the AutodiffComposition’s backward method.

Target

This provides the value (sometimes referred to as a “teacher”) used to train the sample with which it is paired. It is the value of the OutputPort of a specified TARGET_MECHANISM. Any ProcessingMechanism (or the OutputPort of one) in an AutodiffComposition (or one nested within it) can be specified as a target, so long as it is not in the same pathway as the sample it trains. This allows the value of one pathway to be used to train another. Such TARGET_MECHANISMs are assigned the NodeRole TARGET_INTERNAL, and are listed in the AutodiffComposition’s target_internal_mechanisms attribute as well as its target_mechanisms attribute.

Alternatively, the kewyord TARGET can be used to specify the target for a sample in the targets argument of of the AutodiffComposition’s constructor, which allows external values provided in the targets (or inputs) argument of the learn() method to be used to train the Pathway (see Target Inputs for learning). In that case, a TARGET_MECHANISM is automtically constructed for the sample, to receive the external input when learning is executed, and the values (assigned as inputs to the that TARGET_MECHANISM) must be provided in the targets (or inputs) argument of the learn() when it is called (see Target Inputs for learning). If no sample-target pairs are specified in the targets argument of the AutodiffComposition’s constructor, then a TARGET_MECHANISM is automatically constructed for each OUTPUT Node in the Composition, which serves as its sample. Automtically constructed TARGET_MECHANISMs are always INPUT Nodes that are assigned the NodeRole TARGET_INPUT as well as INPUT, and receive their values from the targets (or inputs) argument of the learn() method. These are listed in its target_input_mechanisms attribute of the AutodiffComposition, as well as its target_mehanisms attribute.

The table below provides a summary of the options for specifying samples and their targets.

Hint

The same target can be used to train more than one sample.

Warning

If an internal source (i.e., a ProcessingMechanism) is specified for the target of a sample in the targets argument of the AutodiffComposition’s constructor, then there should NOT be an entry for that sample-target pair in the targets argument of the learn() method; the presence of one will raise an error.

Conversely, any sample paired with the keyword TARGET in the targets argument of the AutodiffComposition’s constructor (specifying the use of external training signals) MUST appear in the targets argument of the learn() method, paired with one or more values to be used for training that sample during learning (see Target Inputs for information specifying these).

LossMechanism

This calculates the loss for the current values of a sample and target. If the LossMechanism is specified explicity (see below), it uses the form of Loss specified in either the loss or function argument of its constructor; in this case then its sample and target must also be specified in the corresponding arguments of the constructor. If a LossMechanism is not specified explicity for a sample-target pair, one is automatically constructed for them, and uses the Loss specified by the loss_spec of the AutodiffComposition.

Specifying sample-target pairs

This is done in the targets argument of the AutodiffComposition’s constructor, using any of the forms of specification listed below. If any sample-target pairs are specified, only those are used; if none are specified, then all OUTPUT Nodes of the AutodiffComposition are used as samples, and corresponding TARGET_MECHANISMs are automatically constructed to receive the target values specified for each in the targets argument of the AutodiffComposition’s learn() method when it is called (see Target inputs for learning).

• tuple: (<sample>, <target or TARGET>), where sample and target are each a ProcessingMechanism or the OutputPort of one, and the tuple specifies a sample-target pair.

• LossMechanism: the sample and target arguments of the LossMechanism's constructor must be specified; its loss argument can also be used to specify a form of Loss; if none is specified, then the loss is determined by the AutodiffComposition’s loss_spec Parameter.

• list: any combination of the above;

• dict: {sample: <target or TARGET} where sample and target are each a ProcessingMechanism or the OutputPort of one, and each entry specifies a sample-target pair.

Note

If samples and targets are specified in the targets argument of the AutodiffComposition’s constructor for some but not all of the learnable pathways (i.e. ones with learnable Projections), a warning is issued listing the learnable pathways that lack learning components (and, in particular, a LossMechanism); if this is not corrected, an error is raised when the learn() method is called.

	constructor(targets)	learn(targets)	Assignments
Composition	N/A	{SAMPLE_MECHANISM: target value}	all OUTPUT Nodes assigned as SAMPLE_MECHANISMs and assigned NodeRole.SAMPLE all TARGET_MECHANISMs constructed automatically and assigned NodeRole.TARGET_INPUT
Autodiff with dict containing:	SAMPLE_MECHANISM: TARGET_MECHANISM ———– and/or ————— SAMPLE_MECHANISM: TARGET	N/A	TARGET_MECHANISM assigned NodeRole.TARGET_INTERNAL
		{SAMPLE_MECHANISM: target value}	TARGET_MECHANISM constructed automatically and assigned NodeRole.TARGET_INPUT
Autodiff with no targets argument specified	None	{SAMPLE_MECHANISM: target value}	all OUTPUT Nodes assigned as SAMPLE_MECHANISMS assigned NodeRole.SAMPLE all TARGET_MECHANISMs constructed automatically and assigned NodeRole.TARGET_INPUT

Learning Rates

The learning argument of the constructor and/or the learn method can be used to specify a learning_rate for an entire AutodiffComposition, ones nested within it, and/or individual MappingProjections (see Learning Rate for details of specification, and the table for which specifications take prcedence over others). Learning_rates specified for individual MappingProjections are passed to the corresponding parameters of the AutodiffComposition’s pytorch_representation when it is executed. Specifications made in the constructor for the AutodiffComposition are used as the default learning_rates for all executions of the learn; specifications made in the call to the learn() method override any made in the constructor, but are used only for that execution. A warning is issued if a learning_rate is specified for a Projection with a learnable attribute set to False, and an error is generated if the Projection is associated with a PyTorch Parameter that is not learnable. See Learning Rate for additional information about specifying learning_rates, including how the learning_rate is determined for Projections that are not expliclity specified.

Hint

To disable learning for a particular MappingProjection in an AutodiffComposition, assign False either to the learnable argument in its constructor, or in an entry of a dict used to specify the learning_rate argument of the AutodiffComposition’s constructor or its learn() method (see Learning Rate); this applies to MappingProjections at any level of nesting.

Exchanging Parameters with Pytorch Modules

The AutodiffComposition’s copy_torch_param_to_projection_matrix and copy_projection_matrix_to_torch_param methods can be used to exchange weight matrices between the parameters of a PyTorch module and the matrix Parameter of a MappingProjection in the AutodiffComposition. Pytorch Parameters can be referenced either by the Parameter object itself, or by the module and either the name or index of the Parameter in the module’s state_dict or parameter list, respectively.Slices of PyTorch Parameters can also be used, for cases in which the matrix of a Project corresponds to only a subpart of the PyTorch Parameter (e.g., for GRUComposition). Both methods return the item assigned.

Warning

PsyNeuLink matrix Parameters are transposed with respect to PyTorch parameters. This is managed automatically by the copy methods noted above, but must be taken into account if either is accessed and/or copied to the other by any other means.

AutodiffComposition Restrictions

Control Components. An AutodiffComposition can contain ControlMechanisms or a controller, that will operate normally when it’s run() method is called in both Python mode and PyTorch mode. However, at present, these are not supported for learning in PyTorch mode; a warning is issued and these are ignored when the learn() method is called with execution_mode = ExecutionMode.PyTorch. Accomodation of control during learning in PyTorch mode will be implemented in a future version.

PsyNeuLink Learning Components. An AutodiffComposition cannot include any learning components themselves (i.e., LearningMechanisms, LearningSignals, or LearningProjections, nor the ComparatorMechanism or ObjectiveMechanism used to compute the loss for learning). These are constructed automatically when learning is executed in Python mode or LLVM mode, and PyTorch-compatible Components are constructed when it is executed in PyTorch mode.

No Bias Parameters. AutodiffComposition does not (currently) support the automatic construction of separate bias parameters. Thus, when constructing the PyTorch version of an AutodiffComposition, the bias parameter of any PyTorch modules are set to False. However, biases can be implemented using BIAS Nodes.

No Post-construction Modification. Mechanisms or Projections should not be added to or deleted from an AutodiffComposition after it has been executed. Unlike an ordinary Composition, AutodiffComposition does not support this functionality.

Post-construction modification is currently not possible because the pytorch_representation is constructed at the time the AutodiffComposition is first constructed, and can’t be modified after that. This will be fixed in a future version.

Structure

Learning Components

The following learning components are constructed for an AutodiffComposition for use in PyTorch mode, that are listed in its learning_components attribute:

Loss Mechanism

This computes the loss for a given pathway, using its sample, target, and assigned form of loss. It receives MappingProjections from sample and target Mechanisms, each of which is non-learnable and assigned an IDENTITY_MATRIX. If the LossMechanism was generated automatically (see LossMechanism), it uses the loss_spec specified for the AutodiffComposition; if it was specified explicity, it uses the form of Loss specified in the loss argument of its constructor, or the PyTorch loss function specified in the function argument of its constructor.

The LossMechanism of an AutodiffComposition is comparable to the ComparatorMechanism (of which it is a sublcass) used as the OBJECTIVE MECHANISM to compute the error for learning in a standard Composition.

The tensor that the LossMechanism receives from its target is detached prior to its use in computing the loss, in order to prevent gradient propagation to the target Mechanism, which may be in its own learning pathway.

SAMPLE_MECHANISM

A SAMPLE_MECHANISM generates, in a designated OutputPort (or its primary OutputPort if one is not specified) the value that is trained through learning to be as close to the value provided by the TARGET_MECHANISM as possible. In an AutodiffComposition, unlike a standard Composition, this can be any ProcessingMechanism (or the OutputPort of one), subject to the restrictions outlined above. See SAMPLE_MECHANISM for additional information.

TARGET_MECHANISM

A TARGET_MECHANISM provides the target value to the LossMechanism used to train a SAMPLE_MECHANISM. There are two types of TARGET_MECHANISM: TARGET_INPUT, specified in the targets argument of the AutodiffComposition’s constructor using the keyword TARGET; and TARGET_INTERNAL, specified as a ProcessingMechanism or the OutputPort of one (see Specifying sample-target pairs). Each type is described below. All of the TARGET_MECHANISMs of an AutodiffComposition are listed in its target_mechanisms attibute, and included in its learning_components attribute.

TARGET_INPUT Mechanisms. These are automatically constructed for each sample specified with the keywor TARGET in the targets argument of the AutodiffComposition’s constructor, along with a Projection from its OutputPort to the TARGET InputPort of the LossMechanism constructed for that sample-target pair. The TARGET_MECHANISM is assigned the NodeRole TARGET_INPUT, and receives the target value from the targets (or inputs) argument of the learn() (see Target inputs for learning). If no targets argument is specified in the AutodiffComposition’s constructor, a TARGET_INPUT Mechanism is constructed for every OUTPUT Node of the AutodiffComposition that belongs to a pathway with at least one learnable Projection. The TARGET_INPUT Mechanisms of an AutodiffComposition are listed in its target_input_mechanisms attribute.

TARGET_INTERNAL Mechanisms. A Projection is automatically constructed from each of these specified in the targets argument of the AutodiffComposition’s constructor, to the the TARGET InputPort of the LossMechanism constructed for that sample-target pair. The TARGET_MECHANISM is assigned the NodeRole TARGET_INTERNAL, and its OutputPort provides the target value used to train the corresponing sample. The TARGET_INPUT Mechanisms of an AutodiffComposition are listed in its target_internal_mechanisms attribute.

Pytorch Representation

An AutodiffComposition uses a pytorch_representation to execute learning when its learn() method is called in Pytorch mode. This is comprised of an outer PytorchCompositionWrapper for the AutodiffComposition, that itself is comprised of PytorchMechanismWrappers and PytorchProjectionWrappers for the Composition’s Mechanisms and Projections, and PytorchCompositionWrappers for any AutodCompositions that are nested within it. Although the pytorch_representation maintains the hierarchical structure of any nested Compositions, when it is executed it “flattens” this, incorporating the nodes of any nested AutodiffCompositions into the top level. This can be shown graphically using the AutodiffComposition’s show_graph method, as described below.

The pytorch_representation is constructed automtically when the learn() method of AutodiffComposition is executed in PyTorch mode (the default), and is used to execute it in PyTorch. It is also constructed when the show_graph method is called with its show_pytorch argument set to True, which generates a graphic display of the pytorch_representation. As noted above, this shows the “flattened” version of the AutodiffComposition (if it has any nested AutodiffCompositions within it) that will execute in PyTorch, with direct Projections between Nodes at different levels of nesting. This also shows any LossMechanisms and TARGET_MECHANISMs that have been automatically constructed (see LossMechanism and Target, respectively). Furthermore, note that no control-related components are shown. Finally, Projections that are excluded from gradient calculations are shown with dotted arrows; dotted arrows are also used to show the flow of the training signal from a LossMechanism to the SAMPLE_MECHANISM for which it calculates the loss.

Note

Calling show_graph with show_pytorch=True is sufficient to show the learning components used for Pytorch mode. Using both show_pytorch and show_learning together is redundant, and will issue a warning. Using show_learning=True alone will show the standard learning Components used for learning in Python mode, but may cause an error if the AutodiffComposition has any nested AutodiffCompositions (see note below).

An AutodiffComposition’s _build_pytorch_representation method can be called to force construction of the pytorch_representation before the `learn_method

Nesting

An AutodiffComposition can be nested inside another Composition for learning, and there can be any number of such nestings. However, all of the nested Compositions must be AutodiffCompositions. As noted above, the AutodiffComposition is “flattened” when its pytorch_representation is used for learning in PyTorch mode; this can be seen by calling the AutodiffComposition’s show_graph method with show_pytorch=True.

Warning

When show_graph is called for an AutodiffComposition with a nested Composition, an error is raised, as standard learning (using Python mode cannot be used; instead, use show_graph(show_pytorch=True) to display the structure of the AutodiffComposition that will executed when PyTorch mode is used for learning.

Even though it is flattened, Projections between Nodes at different levels of nesting can still occur if they are specified for learning. The learning_rate for nested Compositions is inherited from the enclosing Composition unless it is set individually (see Learning Rate for a full discussion of how learning rates and precedence of assignment; see Enabling Learning for enabling and disabling learning in nested Compositions).

Projections from Nodes in an immediately enclosing outer Composition to the input_CIM of a nested Composition, and from its output_CIM to Nodes in the outer Composition are subject to learning; however those within the nested Composition itself (i.e., from its input_CIM to its INPUT Nodes and from its OUTPUT Nodes to its output_CIM) are not subject to learning, as they serve simply as conduits of information between the outer Composition and the nested one.

Warning

Nested Compositions are supported for learning only in PyTorch mode, and cause an error if the learn method of an AutodiffComposition is executed in Python mode or LLVM mode.

Execution

An AutodiffComposition’s run and learn methods are the same as for a Composition. However, the execution_mode argument has different effects than for a standard Composition.

For run(), execution occurs in Python mode by default and if either ExecutionMode.Python or ExecutionMode.PyTorch are specified explicitly (see note below); LLVM compilation is attempted if one of the ExecutionMode.LLVM modes is specified.

For learn(), PyTorch mode is used by default, which uses the pytorch_representation for execution. Python execution and LLVM Compilation can be specified explicity (using ExecutionMode.Python or ExecutionMode.LLVMRun, respectively), but restrictions apply. Each mode of exeuction is described in greater detail below, and summarized in this table, which provides a comparison of the different modes of execution for an AutodiffComposition and standard Composition.

PyTorch mode

This is the default mode for learning of an AutodiffComposition, but can also be specified explicitly by setting execution_mode = ExecutionMode.PyTorch in the learn() method (see example in Basics and Primer). In this mode, the AutodiffComposition’s pytorch_representation is used for learning, which is about three orders of magntidue faster than Python mode, and provides additional funtionality (see above). Although it is best suited for use with supervised learning, it can also be used for some forms of unsupervised learning that are supported in PyTorch (e.g., self-organized maps).

Note

While specifying ExecutionMode.PyTorch in the learn method of an AutodiffComposition causes it to use PyTorch for training, specifying this in the run method causes it to be executed in Python mode (i.e., using the Python interpreter, and not PyTorch); this is so that any modulation can take effect during execution, which is not supported by PyTorch (see Control Components above).

Warning

• Specifying ExecutionMode.LLVMRun or ExecutionMode.PyTorch in the learn() method of a standard Composition raises an error.

Execution Sequence

When PyTorch is used for learning, the AutodiffComposition’s pytorch_representation is executed, which is used to implement each optimization_step of the learning process, by calling the relevant forward, backward, and optimizer_step methods of Pytorch used to implement learning; each optimization_step carries out the following operations:

• execute the AutodiffComposition’s forward method for each stimulus in the minibatch – the number of which is specified by the value of minibatch_size – to generate the values used to compute the Losses for each stimulus;

• aggregate the losses across all stimuli in the minibatch, which is then passed to the AutdoiffComposition’s backward method to compute the gradients and corresponding weight changes for all learnable parameters in the AutodiffComposition;

• copy the Node values generated in the forward pass and changes to parameters generated in the backward pass and optimizer step of the pytorch_representation to the corresponding Mechanisms’ variables and/or values, and learnable Projections’ matrices) of the AutodiffComposition as specified, which can be after each optimizer step, or at the end of the MINIBATCH or EPOCH (see below), but always at the end of the RUN (i.e., call to learn()).

Which nodes are executed in each optimization step, and which parameters are included in the gradient calculation can be further customized as described below.

Additional Optimizations Steps

optimizations_per_minibatch: By default, a single optimization_step is carried out for all of the stimuli in a minibatch. However, as long as there is only one stimulus in a minibatch (i.e., minibatch_size==1), then multiple optimization_steps can be specified for each stimulus, using the optimizations_per_minibatch argument of the AutodiffComposition’s constructor (to specify the default number) or its learn() method (to specify it for just that execution). Specifying optimizations_per_minibatch > 1 can be similar to, but is not the same as increasing the learning_rate (see note) and, when used with execute_in_additional_optimizations can produce important differences, as described below.

execute_in_additional_optimizations: this can be used to specify which Nodes are executed in which additional optimization_steps (i.e., after the first) when more than one optimization_step is specified. This can be used to implement a form of “online replay” (or backprop-to-activity procedure) in which a particular part of the model is given extra optimization_steps to quickly search for a pattern of activity over a subset of Nodes in response to the stimulus that is useful for some downstream purpose (see EGO Model for an example). The execute_in_additional_optimizations argument can be specified in either the AutodiffComposition’s constructor (to sepcify a default value) or its learn() method (which applies to only that execution). It is specified as a dict, each key of which is a Node in the AutodiffComposition or one nested within it, and its value is of the following:

None or True: execute in all additional optimizations ;

False or EXCLUDE: exclude from execution during optimization_steps after the first; this is useful primarly when a nested Composition is specified but nodes within it should be excluded (e.g., see note below);

FIRST, LAST, ALL or range: include in only the first, last, all, or a specified set of additional optimization steps.

Note

If an AutodiffComposition is specified as a key, then all Nodes within that AutodiffComposition and any nested within it are included, except for any explicitly excluded.

Synchronization of PsyNeuLink Values with PyTorch

By default, the outputs (of the modules) and parameters (connection weights) generated in Pytorch during execution of an AutodiffComposition’s learn() method (using its pytorch_representation) are copied to the corresponding Mechanisms and Projections of the AutodiffComposition itself at the end of each run. However, this can be cusotmized, selectively for Mechanism variables or values, Projection matrices, and/or the Composition results, to occur after each optimization_step, minibatch, trial, training epoch, full run, or not at all. This can be specified using following arguments of either the AutodiffComposition’s constructor or learn() method:

• synch_projection_matrices_with_torch : OPTIMIZATION_STEP, MINIBATCH, EPOCH or RUN

• synch_node_variables_with_torch : OPTIMIZATION_STEP, TRIAL, MINIBATCH, EPOCH, RUN or None

• synch_node_values_with_torch : OPTIMIZATION_STEP, MINIBATCH, EPOCH or RUN

• synch_results_with_torch : OPTIMIZATION_STEP, MINIBATCH, EPOCH or RUN

Note

Copying more frequently keeps the PsyNeuLink components more closely synchronized with the corresponding Pytorch elements of the pytorch_representation during learning, which can be useful for debugging and/or monitoring the learning process in Pytorch; but can slow performance.

Saving Pytorch Training Data

By default, the samples, targets, and losses are stored for the last stimulus of each MINIBATCH. However, this can be customized to occur for each OPTIMIZATION_STEP, EPOCH, RUN, or not at all (using None) by specifying one of these values for the following Parameters, using the corresponding argument in either the AutodiffComposition’s constructor (to specify the default vaue) or its learn() method (to specify the value used for that execution): retain_torch_sample_values, retain_torch_targets, or retain_torch_losses.

Python mode

An AutodiffComposition can also be run using the standard PsyNeuLink learning components. However, this cannot be used if the AutodiffComposition has any nested Compositions, irrespective of whether they are ordinary Compositions or AutodiffCompositions; nor can it be used to specify internal targets.

LLVM mode

This is specified by setting execution_mode = ExecutionMode.LLVMRun in the learn method of an AutodiffComposition. This provides the fastest performance, but is limited to supervised learning using the BackPropagation algorithm, and does not support learning of `nested

Compositions <Composition_Nested>` nor subclasses of AutodiffComposition that rely on PyTorch (e.g., GRUComposition and EMComposition) – PyTorch mode should be used for these.

LLVMRun can be used with standard forms of loss, including mean squared error (MSE) and cross entropy, by specifying this in the loss_spec argument of the constructor (see AutodiffComposition for additional details, and Compilation Modes for more information about executing a Composition in compiled mode.

Note

Specifying ExecutionMode.LLVMRun in either the learn and run methods of an AutodiffComposition causes it to (attempt to) use compiled execution in both cases; this is because LLVM compilation supports the use of modulation in PsyNeuLink models (as compared to PyTorch mode; see note below).

Logging

Logging in AutodiffCompositions follows the same procedure as logging in a Composition. However, since an AutodiffComposition internally converts all of its Mechanisms either to an equivalent PyTorch module (or to LLVM in LLVM mode), then its inner components are not actually executed. This means that there is limited support for logging parameters of components inside an AutodiffComposition; Currently, the only supported parameters are the:

• matrix parameter of MappingProjection;

• value parameter of its Mechanisms.

Examples

The following is an example showing how to create a simple AutodiffComposition, specify its inputs and targets, and run it with learning enabled and disabled:

>>> import psyneulink as pnl
>>> # Set up PsyNeuLink Components
>>> my_mech_1 = pnl.TransferMechanism(function=pnl.Linear, input_shapes = 3)
>>> my_mech_2 = pnl.TransferMechanism(function=pnl.Linear, input_shapes = 2)
>>> my_projection = pnl.MappingProjection(matrix=np.random.randn(3,2),
...                     sender=my_mech_1,
...                     receiver=my_mech_2)
>>> # Create AutodiffComposition
>>> my_autodiff = pnl.AutodiffComposition()
>>> my_autodiff.add_node(my_mech_1)
>>> my_autodiff.add_node(my_mech_2)
>>> my_autodiff.add_projection(sender=my_mech_1, projection=my_projection, receiver=my_mech_2)
>>> # Specify inputs and targets
>>> my_inputs = {my_mech_1: [[1, 2, 3]]}
>>> my_targets = {my_mech_2: [[4, 5]]}
>>> input_dict = {"inputs": my_inputs, "targets": my_targets, "epochs": 2}
>>> # Run Composition in learnng mode
>>> my_autodiff.learn(inputs = input_dict)
>>> # Run Composition in test mode
>>> my_autodiff.run(inputs = input_dict['inputs'])

The following shows how the AutodiffComposition created in the previous example can be nested and run inside another Composition:

>>> # Create outer composition
>>> my_outer_composition = pnl.Composition()
>>> my_outer_composition.add_node(my_autodiff)
>>> # Specify dict containing inputs and targets for nested Composition
>>> training_input = {my_autodiff: input_dict}
>>> # Run in learning mode
>>> result1 = my_outer_composition.learn(inputs=training_input)

Class Reference

class psyneulink.library.compositions.autodiffcomposition.AutodiffComposition(pathways=None, optimizer_type='sgd', loss_spec=Loss.MSE, targets=None, weight_decay=0.0, learning_rate=0.001, enable_learning=True, execute_in_additional_optimizations=None, force_no_retain_graph=False, refresh_losses=False, synch_projection_matrices_with_torch=LearningScale.RUN, synch_node_variables_with_torch=None, synch_node_values_with_torch=LearningScale.RUN, synch_results_with_torch=LearningScale.RUN, retain_torch_sample_values=LearningScale.MINIBATCH, retain_torch_targets=LearningScale.MINIBATCH, retain_torch_losses=LearningScale.MINIBATCH, device=None, disable_cuda=True, cuda_index=None, full_sequence_mode=False, name='autodiff_composition', **kwargs)

AutodiffComposition( optimizer_type=’sgd’, loss_spec=Loss.MSE, targets=None, weight_decay=0, enable_learning=True, learning_rate=0.001, execute_in_additional_optimizations=None synch_projection_matrices_with_torch=RUN, synch_node_variables_with_torch=None, synch_node_values_with_torch=RUN, synch_results_with_torch=RUN, retain_torch_sample_values=MINIBATCH, retain_torch_targets=MINIBATCH, retain_torch_losses=MINIBATCH, device=CPU

)

Subclass of Composition that trains models using either LLVM compilation or PyTorch; see and Composition for additional arguments and attributes. See Composition for additional arguments to constructor.

Parameters:

• optimizer_type (str : default 'sgd') – the kind of optimizer used in training. The current options are ‘sgd’ or ‘adam’.

• loss_spec (Loss or PyTorch loss function : default Loss.MSE) – specifies the default loss function for training; see Loss for arguments; any specifications in targets override this default.

• targets (LossMechanism, tuple, list or dict : default None) – specifies the target(s) used for training the model; see AutodiffComposition_Target_Specification for details of specification, and `targets <AutodiffComposition.targets for additional information).

• weight_decay (float : default 0) – specifies the L2 penalty (which discourages large weights) used by the optimizer.

• enable_learning (bool: default True) – specifies whether the AutodiffComposition should enable learning when run in learning mode (see Enabling Learning for additional details).

• learning_rate (float, int, bool or dict : default 0.001) – specifies the learning rate(s) passed to the optimizer; overridden by any specified in the learn method of the AutodiffComposition; if a dict is used, and it does not contain an entry for DEFAULT_LEARNING_RATE, the default indicated above is used (see learning_rate (see `AutodiffComposition_Learning_Rate and Learning Rate for additional details).

• execute_in_additional_optimizations (dict{Node: [<bool | EXCLUDE | (Parameter, value)]} (default None)) – specifies which Nodes of the AutodiffComposition should be included in the forward pass for any additional optimization steps after the first (see AutodiffComposition_Optimization_Steps for fuller explanation and additional details of specification).

• synch_projection_matrices_with_torch (LearningScale : default RUN) – specifies the default for the AutodiffComposition for when to copy Pytorch parameters to PsyNeuLink Projection matrices (connection weights), which can be overridden by specifying the synch_projection_matrices_with_torch argument in the learn method (see LearningScale for information about settings, an Synchronization of PsyNeuLink Values with PyTorch for additional details).

• synch_node_variables_with_torch (LearningScale : default None) – specifies the default for the AutodiffComposition for when to copy the current input to Pytorch nodes to the PsyNeuLink variable of the corresponding PsyNeuLink Nodes, which can be overridden by specifying the synch_node_variables_with_torch argument in the learn method (see LearningScale for information about settings, and Synchronization of PsyNeuLink Values with PyTorch for additional details).

• synch_node_values_with_torch (LearningScale : default RUN) – specifies the default for the AutodiffComposition for when to copy the current output of Pytorch nodes to the PsyNeuLink value attribute of the corresponding PsyNeuLink nodes, which can be overridden by specifying the synch_node_values_with_torch argument in the learn method (see LearningScale for information about settings, and Synchronization of PsyNeuLink Values with PyTorch for additional details).

• synch_results_with_torch (LearningScale : default RUN) – specifies the default for the AutodiffComposition for when to copy the outputs of the Pytorch model to the AutodiffComposition’s results attribute, which can be overridden by specifying the synch_results_with_torch argument in the learn method. Note that this differs from retain_torch_sample_values, which specifies the frequency at which the outputs of the PyTorch model are tracked, all of which are stored in the AutodiffComposition’s torch_sample_values attribute at the end of the run (see LearningScale for information about settings, an Synchronization of PsyNeuLink Values with PyTorch for additional details).

• retain_torch_sample_values (LearningScale : default MINIBATCH) – specifies the default for the AutodiffComposition for the scale at which the outputs of the Pytorch model are tracked, all of which are stored in the AutodiffComposition’s torch_sample_values attribute at the end of the run; this can be overridden by specifying the retain_torch_sample_values argument in the learn method. Note that this differs from synch_results_with_torch, which specifies the frequency with which values are copied to the AutodiffComposition’s results attribute (see retain_torch_sample_values for additional details).

• retain_torch_targets (LearningScale : default MINIBATCH) – specifies the default for the AutodiffComposition for when to copy the targets used for training the Pytorch model to the AutodiffComposition’s torch_targets attribute, which can be overridden by specifying the retain_torch_targets argument in the learn method (see retain_torch_targets for additional details).

• retain_torch_losses (LearningScale : default MINIBATCH) – specifies the default for the AutodiffComposition for the scale at which the losses of the Pytorch model are tracked, all of which are stored in the AutodiffComposition’s torch_losses attribute at the end of the run (see retain_torch_losses for additional details).

• device (torch.device : default device-dependent) – specifies the device on which the model is run. If None, the device is set to ‘cuda’ if available, then ‘mps`, otherwise ‘cpu’.

pytorch_representation

represents the PyTorch model of the AutodiffComposition, which is created when the AutodiffComposition is run in PyTorch mode.

Type:

PytorchCompositionWrapper

optimizer

the optimizer used for training. Depends on the optimizer_type, learning_rate, and weight_decay arguments from initialization.

Type:

PyTorch optimizer function

loss_spec

the loss function used for training. Depends on the loss_spec argument from initialization.

Type:

PyTorch loss function

loss_mechanisms

each LossMechanism computes the loss for the output of the Node from which it recieves its SAMPLE input (the “student” Node) by comparing it to the output of the Node from which it receives its TARGET input (the “teacher” Node), using the specified loss function; see Target for additional details.

Type:

list of LossMechanisms

learning_rate

determines the default learning_rate passed the optimizer, that is applied to all Projections in the AutodiffComposition that are learnable, and for which individual rates have not been specified (see Learning Rates for additional details).

Type:

float or bool

targets

dictionary of {sample:target} specifiations, used to specify the TARGET_MECHANISM for each SAMPLE_MECHANISM in an AutodiffComposition (see TARGET_MECHANISM for details).

Type:

dict

sample_mechanisms

list of all SAMPLE_MECHANISMs in the AutodiffComposition.

Type:

list of SAMPLE_MECHANISMs

target_mechanisms

list of all TARGET_MECHANISMs in the AutodiffComposition.

Type:

list of TARGET_MECHANISMs

target_input_mechanisms

list of the TARGET_MECHANISMs in the AutodiffComposition assigned the NodeRole TARGET_INPUT (see AutodiffComposition_Structure_TARGET_INPUT for details)

Type:

list of TARGET_MECHANISMs

target_internal_mechanisms

list of the TARGET_MECHANISMs in the AutodiffComposition assigned the NodeRole TARGET_INTERNAL (see AutodiffComposition_Structure_TARGET_INTERNAL for details)

Type:

list of TARGET_MECHANISMs

execute_in_additional_optimizations

determines which Nodes of the AutodiffComposition should be included in the forward pass for any additional optimization steps after the first (see AutodiffComposition_Optimization_Steps for additional information).

Type:

dict{Node:[(Parameter, value)]}

synch_projection_matrices_with_torch

determines when to copy PyTorch parameters to PsyNeuLink Projection matrices (connection weights) if this is not specified in the call to learn (see Synchronization of PsyNeuLink Values with PyTorch for additional details).

Type:

OPTIMIZATION_STEP, MINIBATCH, EPOCH or RUN

synch_node_variables_with_torch

determines when to copy the current input to Pytorch functions to the PsyNeuLink variable attribute of the corresponding PsyNeuLink Nodes, if this is not specified in the call to learn (see Synchronization of PsyNeuLink Values with PyTorch for additional details)

Type:

OPTIMIZATION_STEP, TRIAL, MINIBATCH, EPOCH, RUN or None

synch_node_values_with_torch

determines when to copy the current output of Pytorch functions to the PsyNeuLink value attribute of the corresponding PsyNeuLink Nodes, if this is not specified in the call to learn (see Synchronization of PsyNeuLink Values with PyTorch for additional details).

Type:

OPTIMIZATION_STEP, MINIBATCH, EPOCH or RUN

synch_results_with_torch

determines when to copy the current outputs of Pytorch nodes to the PsyNeuLink results attribute of an AutodiffComposition if this is not specified in the call to learn (see Synchronization of PsyNeuLink Values with PyTorch for additional details).

Type:

OPTIMIZATION_STEP, TRIAL, MINIBATCH, EPOCH or RUN

retain_torch_sample_values

determines the scale at which the outputs of the Pytorch model are tracked, all of which are stored in the AutodiffComposition’s results attribute at the end of the run if this is not specified in the call to learn (see LearningScale for information about settings).

Type:

OPTIMIZATION_STEP, MINIBATCH, EPOCH, RUN or None

retain_torch_targets

determines the scale at which the targets used for training the Pytorch model are tracked, all of which are stored in the AutodiffComposition’s targets attribute at the end of the run if this is not specified in the call to learn (see LearningScale for information about settings).

Type:

OPTIMIZATION_STEP, TRIAL, MINIBATCH, EPOCH, RUN or None

retain_torch_losses

determines the scale at which the losses of the Pytorch model are tracked, all of which are stored in the AutodiffComposition’s torch_losses attribute at the end of the run if this is nota specified in the call to learn (see LearningScale for information about settings).

Type:

OPTIMIZATION_STEP, MINIBATCH, EPOCH, RUN or None

torch_parameters

list of PyTorch named_parameters() for pytorch_representation of AutodiffComposition.

Type:

List[Tuple[str, torch.nn.parameter]]

torch_sample_values

stores the outputs (converted to np arrays) of the Pytorch model trained during learning, at the frequency specified by retain_torch_sample_values if it is set to MINIBATCH, EPOCH, or RUN; see retain_torch_sample_values for additional details.

Type:

List[ndarray]

torch_targets

stores the targets used for training the Pytorch model during learning at the frequency specified by retain_torch_targets if it is set to MINIBATCH, EPOCH, or RUN; see retain_torch_targets for additional details.

Type:

List[ndarray]

torch_losses

stores the average loss after each weight update (i.e. each minibatch) during learning, at the frequency specified by retain_torch_sample_values if it is set to MINIBATCH, EPOCH, or RUN; see retain_torch_losses for additional details.

Type:

list of floats

last_saved_weights

path for file to which weights were last saved.

Type:

path

last_loaded_weights

path for file from which weights were last loaded.

Type:

path

device

the device on which the model is run.

Type:

torch.device

full_sequence_mode

Whether to run the underlying Composition in full sequence mode or not. In full sequence mode, each element of an input sequence for a trial is processed in a separate time step. This is needed only if there are sequential dependencies between the mechanisms of the compositions. Note, if the composition contains GRU compositions wrappers full sequence mode is not needed (and should be avoided to improve efficiency) because the composition wrapper itself handles the sequential dependencies between the mechanisms of the GRU composition.

Type:

bool : default False

class PytorchMechanismWrapper(mechanism, composition, outer_creator, component_idx, use, dtype, device, subclass_specifies_function=False, context=None, base_context=None)

Wrapper for a Mechanism in a PytorchCompositionWrapper These comprise nodes of the PytorchCompositionWrapper, and generally correspond to functions in a Pytorch model.

mechanism

the PsyNeuLink Mechanism being wrapped.

Type:

Mechanism

composition

the AutodiffComposition to which the Mechanism being wrapped belongs (and for which the PytorchCompositionWrapper – to which the PytorchMechanismWrapper belongs – is the pytorch_representation).

Type:

AutodiffComposition

afferents

list of PytorchProjectionWrapper objects that project to the PytorchMechanismWrapper.

Type:

List[PytorchProjectionWrapper]

input

most recent input to the PytorchMechanismWrapper.

Type:

torch.Tensor

function

Pytorch version of the Mechanism’s function assigned in its __init__.

Type:

_gen_pytorch_fct

integrator_function

Pytorch version of the Mechanism’s integrator_function assigned in its __init__ if Mechanism has an integrator_function; this assumes the Mechanism also has an integrator_mode attribute that is used to determine whether to execute the integrator_function first, and use its result as the input to its function.

Type:

_gen_pytorch_fct

output

most recent output of the PytorchMechanismWrapper.

Type:

torch.Tensor

efferents

list of PytorchProjectionWrapper objects that project from the PytorchMechanismWrapper.

Type:

List[PytorchProjectionWrapper]

exclude_from_gradient_calc

prevents a node from being included in the Pytorch gradient calculation by execluding it in calls to Autodiff.autodiff_backward(); entered in PytorchCompositionWrapper._nodes_to_execute_after_gradient_calc as a key, and the current variable that it uses for execution at the end of CompositionRuner._batch_input().

• AFTER: the node is executed on every optimization step, after all gradient updates have been done;

• LAST: if Composition.optimizations_per_minibatch is greater than 1, the node is executed only after the last optimization step

• BEFORE: not currently supported

Type:

bool or str[BEFORE | AFTER | LAST]: False

_use

designates the uses of the Mechanism, specified by the following keywords (see PytorchCompositionWrapper docstring for additional details):

• LEARNING: inputs and function Parameters) are used for actual execution of the corresponding Pytorch Module;

• SYNCH: used to store results of executing a Pytorch module that are then transferred to the value Parameter of the PytorchMechanismWrappers mechanism;

• SHOW_PYTORCH: Mechanism is included when the AutoDiffCompositions show_graph method to used with the show_pytorch option to display its pytorch_representation; if it is not specified, the Mechanism is not displayed when the AutoDiffCompositions show_graph method is called, even if the show_pytorch option is specified.

Type:

list[LEARNING, SYNCH]

add_afferent(afferent)

Add ProjectionWrapper for afferent to MechanismWrapper. For use in call to collect_afferents

add_efferent(efferent)

Add ProjectionWrapper for efferent from MechanismWrapper. Implemented for completeness; not currently used

collect_afferents(batch_size, port=None, inputs=None)

Return afferent projections for input_port(s) of the Mechanism If there is only one input_port, return the sum of its afferents (for those in Composition) If there are multiple input_ports, return a tensor (or list of tensors if input ports are ragged) of shape:

(batch, input_port, projection, …)

Where the ellipsis represent 1 or more dimensions for the values of the projected afferent.

FIX: AUGMENT THIS TO SUPPORT InputPort’s function

execute(variable, optimization_num, synch_with_pnl_options, sequence_lengths, context=None)

Execute Mechanism’s _gen_pytorch version of function on variable. Enforce result to be 2d, and assign to self.output

Return type:

Tensor

execute_function(function, variable, fct_has_mult_args=False)

Execute _gen_pytorch_fct on variable, enforce result to be 2d, and return it. If fct_has_mult_args is True, treat each item in variable as an arg to the function If False, compute function for each item in variable and return results in a list

set_pnl_variable_and_values(set_variable=False, set_value=True, context=None)

Set the state of the PytorchMechanismWrapper’s Mechanism Note: if execute_mech=True requires that variable=True

pytorch_composition_wrapper_type

alias of PytorchCompositionWrapper

pytorch_mechanism_wrapper_type

alias of PytorchMechanismWrapper

infer_backpropagation_learning_pathways(execution_mode, context=None, base_context=None)

Return type:

list

Create backpropagation learning pathways for every INPUT Node –> OUTPUT Node pathway Pathways are constructed in _get_pytorch_backprop_pathways()

Flattens nested compositions:

• only includes the Projections in outer Composition to/from the CIMs of the nested Composition (i.e., to input_CIMs and from output_CIMs) – the ones that should be learned;

• excludes Projections from/to CIMs in the nested Composition (from input_CIMs and to output_CIMs), as those should remain identity Projections;

see PytorchCompositionWrapper for table of how Projections are handled and further details.

For Python mode:

• calls add_backpropagation_learning_pathway() for each identified pathway which also creates TARGET_MECHANISMs for TERMINAL Nodes in each pathway

For PyTorch mode:

• if targets are specified in the AutodiffComposition constructor,

  LossMechanisms and MappingProjections are constructed for them;

• otherwise, TERMINAL Nodes of each pathway are used to construct LossMechanisms and TARGET_MECHANISMs with associated MappingProjections) to allow targets to be specified in inputs argument of learn().

• the above allow: - trial-by-trial losses to be kept aligned with inputs in batch / minibatch construction - losses to be tracked for logging (as mechs of a Composition)

For both:

• check that no LossMechanisms have been added to the AutodiffComposition on their own

  (i.e., outside of the targets argument of the constructor)

Return list of LossMechanisms and TARGET_MECHANISMs

_get_pytorch_backprop_pathways(context)

Get backpropagation pathways for all INPUT Nodes of AutodiffComposition Return a list of all pathways

Return type:

list

_mech_is_receiver_in_learnable_pathway(mech_output_port, visited=None)

Return True if mech receives a Projection from any pathway that has at least one learnable Projection

Return type:

bool

_mech_is_sender_in_learnable_pathway(sender)

Return True if sender sends a Projection to any pathway that ends in a LossMechanism.

Return type:

bool

_check_if_sample_is_in_learnable_pathway(sample_port, target_spec=None, loss_mech=None, constructed_target_mechs=None, action=None)

Take specified action if sample_port’s owner has no afferent pathways with any learnable Projections. - target_spec argument is used to determine error_message; - if no action is specified, return True or False

Return type:

bool

_check_if_target_is_in_sample_pathway(sample_port, target_port, pathways, context)

Determine if target appears before the sample in any pathway. Returns True if target appears before sample in any pathway, False otherwise.

_instantiate_loss_components(pathways, context, base_context)

Instantiate sample:target pairs, LossMechanisms, and any TARGET_MECHANISMs needed

Overivew: - Use any specifications in self.targets (from targets arg of AutodiffComposition constructor)

to identify sample-target pairs, and constuct LossMechanisms and any needed TARGET_MECHANISMs

• If there are no specifications in self.targets, then use OUTPUT Nodes of pathways as samples

  and construct TARGET_MECHANISMs for each.

Procedure: 1) Handle specifications from constructor (in self.targets) in call to _instantiate_constructor_targets_args():

• identifies sample-target pairs:

  • places them in self.sample_port_to_target_port_map

  • returns them as first item, placed in loss_mech_specs

• creates TARGET_MECHANISMs (that receive external input) for any targets specified using TARGET keyword - returns them as second item, placed in target_mechs

2. If there are no constructor specifications, then call _instantiate_default_targets():

   • assigns all OUTPUT Nodes of pathways as samples and TARGET_MECHANISMs as targets

     this allows:

     • external targets to be specified in learn() in the same way as for other execution_modes:

       learn(targets = {<OUTPUT Node> : <value>}) -> TARGET_MECHANISM (mapping is done in _map_external_target_values_to_target_nodes()

     • trial-by-trial losses to be kept aligned with inputs in batch / minibatch construction

     • losses to be tracked for logging (as mechs of a Composition)

     • places them in self.sample_port_to_target_port_map

     • returns them as first item, placed in loss_mech_specs

   • creates any TARGET_MECHANISMs that have not yet been constructed

     • returns them as second item, placed in target_mechs

3. Validate loss_mech_specs

4. Use loss_mechs and target_mechs to instantiate LossMechanisms in call to _instantiate_loss_mechanisms(): - constructs self.loss_mechs_map: {<LossMechanism>: (sample OutputPort, target OutputPort)} - adds LossMechanisms to AutodiffComposition

5. Exclude LossMechanisms and TARGET_MECHANISMs from OUTPUT role and suppress warnings about role assignments

_check_for_errant_loss_mechs(error_type)

Check if there are any “free-standing” LossMechanisms in any pathways This should only be specified in the targets argument of an AutodiffComposition

_instantiate_constructor_targets_args(pathways, context, base_context)

Instantiate targets specified by user in targets argument of AutodiffComposition constructor - These may be in

• target attribute of an explicitly specified LossMechanism

• a (sample:target) tuple

• a list containing tuples and/or LossMechanisms

• or dict of {sample:target} pairs

where:

sample = OutputPort or ProcessingMechanism, target = OutputPort, ProcessingMechanism, or TARGET keyword

• Identify all samples and assign NodeRole.SAMPLE to them

• Instantiate TARGET_MECHANISMs for any targets specified as TARGET, and assign NodeRole.TARGET_INPUT

• Update self._sample_target_pairs (with SAMPLE and TARGET Mechanisms and OutputPorts)

_instantiate_default_targets(pathways, context, base_context)

Construct default TARGET_MECHANISMs (since none were specified in targets arg of constructor Current default is to treat all OUTPUT Nodes as samples, and assign them TARGET_MECHANISMs IMPLEMENTATION NOTE:

This is to support legacy behavior, in which targets are not specified explicitly

• Only add TARGET_MECHANISMs if not already present in self.sample_port_to_target_port_map.values(),

  to avoid duplication in multiple calls, including from command line (see test_xor_training_identicalness_standard_composition_vs_PyTorch_and_LLVM for example)

• Update self.sample_port_to_target_port_map with construted TARGET_MECHANISMs

• Add constructed TARGET_MECHANISMs to AutodiffComposition with NodeRole.TARGET_INPUT and NodeRole.INPUT

Return list of loss_mech_specs ((sample OutputPort, targetOutputPort) tuples) and constructed TARGET_MECHANISMs

Return type:

Tuple[List, List]

_validate_loss_mech_specs(loss_mech_specs, context)

Validate specifications used to construct LossMechanism in _instantiate_loss_components

Return type:

Tuple[List, List]

_instantiate_loss_mechanisms(loss_mech_specs, context, base_context)

Construct and/or add LossMechanisms (and their MappingProjections) to AutodiffComposition - loss_mech_specs is a list with (sample OutputPort, target OutputPort) tuples and/or LossMechanisms - If item is a (sample OutputPort, target OutputPort) tuple construct LossMechanism with:

LossMechanism.input_port[SAMPLE] and LossMechanism.sample = sample OutputPort LossMechanism.input_port[TARGET] and LossMechanism.target = target OutputPort LossMechanism.loss = self.loss_spec

• Add LossMechanisms to AutodiffComposition, with NodeRole.LEARNING_OBJECTIVE

• Assign self.loss_mechs_map as {<LossMechanism>: (sample OutputPort, target OutputPort)}

Return list of constructed LossMechanisms

Return type:

list

_get_samples_dict(execution_mode=<ExecutionMode.Python: 0>, context=None, base_context=None)

Override to ensure that any SAMPLE Nodes specified in targets argument of constructor were found.

Return type:

dict

get_target_input_mechs(execution_mode=<ExecutionMode.PyTorch: 1>, context=None, base_context=None)

Override to call infer_backpropagation_learning_pathways This instantiates any TARGET_MECHANISMs specified in targets argument of the constructor.

Return type:

list

get_target_internal_mechs(execution_mode=<ExecutionMode.PyTorch: 1>, context=None, base_context=None)

Return list of TARGET_MECHANISMS with NodeRole.TARGET_INTERNAL This instantiates any TARGET_MECHANISMs specified in targets argument of the constructor.

Return type:

list

compute_loss(targets, pytorch_rep, context)

Compute loss for each trial Can be overridden to use direct/dedicated/customized computation of loss by subclasses. IMPLEMENTATION NOTE:

targets arg is included for overrides; LossMechanism uses its target input directly

compute_loss_using_loss_mechanisms(targets, pytorch_rep, context)

Compute loss after execution of autodiff_forward() Use values of LossMechanism(s) that computed loss for each pathway

_compute_loss_using_standalone_function_and_values_of_output_nodes(targets, pytorch_rep, context)

Compute loss using values of OUTPUT Nodes as samples Loss is computed using a single standalone loss function for all sampe-target pairs IMPLEMENTATION NOTE:

this is legacy code that may be restored for use in the future, though would need to be revised/validated

_get_autodiff_target_node_input_values(input_dict)

Return dict with input values for TARGET_MECHANISMs Get inputs to TARGET_MECHANISMs used for computation of loss in autodiff_forward(). Use input_dict to get input values for TARGET_MECHANISMs that are INPUT Nodes of the AutodiffComposition, If a TARGET_MECHANISM is not an INPUT Node, it is assumed to be an internal target as is ignored,

as those are assumed to be executed in autodiff_forward()

Return type:

A dict mapping TARGET_MECHANISMs -> target values

_map_external_target_values_to_target_nodes(target_specs, execution_mode)

Map target values to target mechanisms (as needed by learning)

Return type:

dict

Returns:

• dict – Dict mapping TargetMechanisms -> target values

_parse_learn_targets_specs(inputs, targets, execution_mode, context, base_context)

Override to handle targets arguments in construtor and learn() that are specific to AutodiffComposition Integrate target specifications from constructor (in self.targets) with those in targets argument of learn():

handled in override of _aggregate_and_filter_sample_target_specs()

Deal with nested Compositions

handled in return from override of this method

_parse_constructor_targets_specs()

Parse sample-target specifications from targets of constructor in self._constructor_target_specs Standardize format of entries as {sample.output_port: target.output_port or ‘TARGET’) Register samples and targets from LossMechanism specs in loss_mechs_map Note: specs have been validated in _validate_targets() for autodiffcomposition.parameters.targets

_validate_constructor_targets_specs()

Handle erroneous SAMPLE specs in targets argument of constructor - Handle redundant specifications and any conflicts among them

(done in _handle_redundant_sample_target_specs())

• Check for SAMPLE or TARGET specs NOT in the Composition

Notes: - These are done here and not on Composition, since that does not support specification of SAMPLES

(there they are assigned automatically as the OUTPUT Nodes of the Composition)

• The only checks here are for the validity of specifications in the targets argument of the constructor

  (at time of construction); compatibilty with specfications in the targets argument of learn() are handled in _validate_sample_target_specs_from_learn()

_validate_sample_target_specs_from_learn(learn_specs, name, allow_None_for_target)

Compare learn_specs with constructor specs for SAMPLEs and TARGETs Issue error for: - missing entries in learn() or ones with a non-numeric value

for SAMPLEs specified with the keyword ‘TARGET’ in the constructor

• any specifications for sample-target pairs specified with an internal TARGET_MECHANISM in the constructor

Notes: - validation of SAMPLES happens in _validate_constructor_targets_specs() - for every SAMPLE that has a value = TARGET and source = “autodiff_constructor”

there should be another entry for that SAMPLE that has value == numeric and source = {inputs, inputs[TARGETS or targets}

• the total number should = the number of SAMPLE Nodes in the Composition:

  if too many: bad specs if too few, error (see below)

Return type:

dict

_handle_redundant_sample_target_specs()

Override to include specs in targets arg of constructor

_handle_conflicting_sample_target_specs(specs_with_mismatching_values)

Override to handle conflict between sample specs and/or values from constructor and learn() Handle conflicts between different target values specified for:

same SAMPLE Nodes specified in constructor using different references (e.g., mech vs. mech.output_port) SAMPLE in constructor vs. learn() (e.g., Node in constructor vs. numeric value in learn())

_identify_output_nodes(context)

Recursively call all nested AutodiffCompositions to assign TARGET_MECHANISMs for learning

Return type:

list

set_weights(pnl_proj, weights, context=None)

Set weights for specified Projection.

learn(*args, execute_in_additional_optimizations=None, synch_projection_matrices_with_torch=NotImplemented, synch_node_variables_with_torch=NotImplemented, synch_node_values_with_torch=NotImplemented, synch_results_with_torch=NotImplemented, retain_torch_sample_values=NotImplemented, retain_torch_targets=NotImplemented, retain_torch_losses=NotImplemented, context=None, base_context=<psyneulink.core.globals.context.Context object>, skip_initialization=False, **kwargs)

Override to handle synch and retain args; see Composition.run for additional arguments and details.

Parameters:

• learning_rate (float, int, bool or dict : default 0.001) – specifies the learning rate(s) passed to the optimizer, that overrides any learning_rate specifications made in AutodiffComposition constructor and/or individual MappingProjections. If a value is specified, it overrides the default learning rate for the Composition, and is used as the default learning rate for all MappingProjections in the Composition (and any nested within it) that do not have a specific learning_rate specified in their constructor. A dict can be used to specify MappingProjection-specific learning_rate(s); if it contains a DEFAULT_LEARNING_RATE entry, that is used in the same was as specifing numeric value; if the dict does not contain a DEFAULT_LEARNING_RATE entry, then the default indicated above is used for all MappingProjections in the Composition, and MappingProjections in any nested Compositions use their default learning_rate (see AutodiffComposition_Learning_Rate and Learning Rate for additional details).

• execute_in_additional_optimizations (dict{Node:[(Parameter, value)]} (default None)) – specifies which Nodes of the AutodiffComposition should be included in the forward pass for any additional optimization steps after the first; this overrides any specifications made in the execute_in_additional_optimizations argument of the AutodiffComposition’s constructor (see AutodiffComposition_Optimization_Steps for fuller explanation and details of specification).

• synch_projection_matrices_with_torch (SynchRetainArg : Default NotImplemented) – overrides specification(s) made in Autodiff constructor; see synch_projection_matrices_with_torch for additional details.

• synch_node_variables_with_torch (SynchRetainArg : Default NotImplemented) – overrides specification(s) made in Autodiff constructor; see synch_node_variables_with_torch for additional details.

• synch_node_values_with_torch (SynchRetainArg : Default NotImplemented) – overrides specification(s) made in Autodiff constructor; see synch_node_values_with_torch for additional details.

• synch_results_with_torch (SynchRetainArg : Default NotImplemented) – overrides specification(s) made in Autodiff constructor; see synch_results_with_torch for additional details.

• retain_torch_sample_values (SynchRetainArg : Default NotImplemented) – overrides specification(s) made in Autodiff constructor; see retain_torch_sample_values for additional details.

• retain_torch_targets (SynchRetainArg : Default NotImplemented) – overrides specification(s) made in Autodiff constructor; see retain_torch_targets for additional details.

• retain_torch_losses (SynchRetainArg : Default NotImplemented) – overrides specification(s) made in Autodiff constructor; see retain_torch_losses for additional details.

Return type:

list

execute(inputs=None, num_trials=None, minibatch_size=1, optimizations_per_minibatch=1, optimization_num=None, do_logging=False, scheduler=None, termination_processing=None, call_before_minibatch=None, call_after_minibatch=None, call_before_time_step=None, call_before_pass=None, call_after_time_step=None, call_after_pass=None, reset_stateful_functions_to=None, context=None, base_context=<psyneulink.core.globals.context.Context object>, clamp_input='soft_clamp', targets=None, optimizer_params=None, runtime_params=None, execution_mode=<ExecutionMode.PyTorch: 1>, skip_initialization=False, synch_with_pnl_options=None, retain_in_pnl_options=None, report_output=ReportOutput.OFF, report_params=ReportParams.OFF, report_progress=ReportProgress.OFF, report_simulations=ReportSimulations.OFF, report_to_devices=None, report=None, report_num=None)

Override to execute autodiff_forward() in learning mode if execute_mode is not Python

Return type:

ndarray

run(*args, execution_mode=<ExecutionMode.Python: 0>, synch_projection_matrices_with_torch=NotImplemented, synch_node_variables_with_torch=NotImplemented, synch_node_values_with_torch=NotImplemented, synch_results_with_torch=NotImplemented, retain_torch_sample_values=NotImplemented, retain_torch_targets=NotImplemented, retain_torch_losses=NotImplemented, batched_results=False, context=None, base_context=<psyneulink.core.globals.context.Context object>, **kwargs)

Override to handle synch and retain args if called directly from run() rather than learn() Note: defaults for synch and retain args are NotImplemented, so that the user can specify None if they want

to locally override the default values for the AutodiffComposition (see parse_synch_and_retain_args() for details). This is distinct from the user assigning the Parameter default_values(s), which is done in the AutodiffComposition constructor and handled by the Parameter._specify_none attribute.

save(path=None, directory=None, filename=None, context=None)

Saves all weight matrices for all MappingProjections in the AutodiffComposition

Parameters:

• path (Path, PosixPath or str : default None) – path specification; must be a legal path specification in the filesystem.

• directory (str : default current working directory) – directory where matrices for all MappingProjections in the AutodiffComposition are saved.

• filename (str : default <name of AutodiffComposition>_matrix_wts.pnl) – filename in which matrices for all MappingProjections in the AutodiffComposition are saved.

• note:: (..) – Matrices are saved in PyTorch state_dict format.

Return type:

Path

load(path=None, directory=None, filename=None, context=None, weights_only=False)

Loads all weight matrices for all MappingProjections in the AutodiffComposition from file :type path: PosixPath :param path: Path for file in which MappingProjection matrices are stored.

This must be a legal PosixPath object; if it is specified directory and filename are ignored.

Parameters:

• directory (str : default current working directory) – directory where MappingProjection matrices are stored.

• filename (str : default <name of AutodiffComposition>_matrix_wts.pnl) – name of file in which MappingProjection matrices are stored.

• note:: (..) –

  Matrices must be stored in PyTorch state_dict format.

copy_torch_param_to_projection_matrix(projection, torch_param, torch_module=None, torch_slice=None, validate=True, context=None)

Assign torch Parameter to matrix Parameter of specified MappingProjection. Return torch_param as the np.ndarray assigned to matrix Parameter of projection.

Parameters:

• projection (str or MappingProjection) – specifies MappingProjection to which the torch_param is assigned as its matrix Parameter; if specified as a str, it must be the name of a MappingProjection in the AutodiffComposition.

• torch_param (torch.nn.Parameter, str or int) – specifies torch_param to assign to the matrix Parameter of projection; if it is a torch.nn.Parameter or torch.Tensor, then the torch_module argument does not need to be specified; if specified as a str or int, it must be the name of a torch Parameter (used to access it in the state_dict) or its index (used to access it in the parameterlist) of the torch_module argument, which must be also specified.

• torch_module (torch.nn.Module : default None) – specifies a torch.nn.Module containing torch_param assigned to the`matrix<MappingProjection.matrix>` Parameter of projection; this does not need to be specified if torch_param is a torch.nn.Parameter or torch.Tensor, but must be specified if torch_param is a str or int.

• torch_slice (slice : default None) –

  specifies a slice of torch_param to assign to the matrix Parameter

  of projection; if it is not specified, the entire tensor of torch_param is used.

  Warning

  torch_slice should not be specified if the specification of torch_param already takes this into account.

• validate (bool : default True) –

  specifies whether to validate the projection and torch_param arguments; setting it to False results in more efficient processing if this method is called frequently; however, invalid arguments will raise standard Python exceptions rather than more informative AutodiffComposition errors, and unexpected results may go unnoticed.

  Warning

  if validate is False, for efficiency: projection must be a MappingProjection, torch_param must be a torch.Tensor, and both torch_module and torch_slice are ignored.

• context (Context or None : default most recent Context) – specifies context to use for the value of Projection.matrix; if it is not provided, then a default Context is constructed using the name of the AutodiffComposition as the execution_id, commensurate with the one used bydefault for its execution.

Return type:

ndarray

copy_projection_matrix_to_torch_param(projection, torch_param, torch_module=None, torch_slice=None, validate=True, context=None)

Assign the matrix Parameter of a MappingProjection to a Pytorch Parameter.

Return torch.Tensor assigned to torch_param

Parameters:

• projection (str or MappingProjection) – specifies MappingProjection, the matrix of which is assigned torch_param; if specified as a str, it must be the name of a MappingProjection in the AutodiffComposition.

• torch_param (torch.nn.Parameter, str or int) – specifies torch Parameter to which the matrix of the Projection is assigned; if it is a torch.nn.Parameter or torch.Tensor, then the torch_module argument does not need to be specified; if specified as a str or int, it must be the name of a torch Parameter (used to access it in the state_dict) or its index (used to access it in the parameterlist) of the torch_module argument, which must be also specified.

• torch_module (torch.nn.Module : default None) – specifies a torch.nn.Module containing torch_param to which the projection’s matrix Parameter is assigned; this does not need to be specified if torch_param is a torch.nn.Parameter or torch.Tensor, but must be specified if torch_param is a str or int.

• torch_slice (slice : default None) –

  specifies a slice of torch_param to assign to the matrix Parameter

  of projection; if it is not specified, the entire tensor of torch_param is used.

  Warning

  torch_slice should not be specified if the specification of torch_param already takes this into account.

• validate (bool : default True) –

  specifies whether to validate the projection and torch_param arguments; setting it to False results in more efficient processing if this method is called frequently; however, invalid arguments then raise standard Python exceptions rather than more informative AutodiffComposition errors, and unexpected results may go unnoticed.

  Warning

  if validate is False, for efficiency: projection must be a MappingProjection, torch_param must be a torch.Tensor, and both torch_module and torch_slice are ignored.

• context (Context or None : default most recent Context) – specifies context to use for the value of Projection.matrix; if it is not provided, then a default Context is constructed using the name of the AutodiffComposition as the execution_id, commensurate with the one used bydefault for its execution.

Return type:

Tensor

_validate_torch_param_and_projection(torch_param, torch_module, torch_slice, projection_spec)

Validate torch and projection arguments for copying between PyTorch and AutodiffComposition. Return tuple of torch.Tensor and MappingProjection.

Return type:

tuple

show_graph(*args, **kwargs)

Override to use PytorchShowGraph if show_pytorch is True

property num_learnable_pathways

Return number of unique learnable pathways in the AutodiffComposition Learnable pathways are ones that end in a non-loss Node and contain at least one learnableMappingProjection; Unique learnable pathways are defined as those that have different sets of learnable MappingProjections. NOTE: THis method is used to insure that all learnable pathways are assigned a TARGET_MECHANISM and LossMechanism.

property target_mechanisms

Override to call infer_backpropagation_learning_pathways This instantiates any TARGET_MECHANISMs specified in targets argument of the constructor.

property target_input_mechanisms

Override to call infer_backpropagation_learning_pathways This instantiates any TARGET_MECHANISMs specified in targets argument of the constructor.

property target_internal_mechanisms

Return list of TARGET_MECHANISMs with NodeRole.TARGET_INTERNAL This instantiates any TARGET_MECHANISMs specified in targets argument of the constructor.

property torch_parameters

Return Pytorch Parameters for pytorch_representation of AutodiffComposition

property _dependent_components

Returns: Components that must have values in a given Context for this Component to execute in that Context