ops – Some Common Ops and extra Ops stuff

This file contains auxiliary Ops, used during the compilation phase and Ops building class (FromFunctionOp) and decorator (wrap_py()) that help make new Ops more rapidly.

class pytensor.compile.ops.DeepCopyOp

c_code(node, name, inames, onames, sub)

Return the C implementation of an Op.

Returns C code that does the computation associated to this Op, given names for the inputs and outputs.

Parameters:

• node (Apply instance) – The node for which we are compiling the current C code. The same Op may be used in more than one node.

• name (str) – A name that is automatically assigned and guaranteed to be unique.

• inputs (list of strings) – There is a string for each input of the function, and the string is the name of a C variable pointing to that input. The type of the variable depends on the declared type of the input. There is a corresponding python variable that can be accessed by prepending "py_" to the name in the list.

• outputs (list of strings) – Each string is the name of a C variable where the Op should store its output. The type depends on the declared type of the output. There is a corresponding Python variable that can be accessed by prepending "py_" to the name in the list. In some cases the outputs will be preallocated and the value of the variable may be pre-filled. The value for an unallocated output is type-dependent.

• sub (dict of strings) – Extra symbols defined in CLinker sub symbols (such as 'fail').

c_code_cache_version()

Return a tuple of integers indicating the version of this Op.

An empty tuple indicates an “unversioned” Op that will not be cached between processes.

The cache mechanism may erase cached modules that have been superseded by newer versions. See ModuleCache for details.

See also

c_code_cache_version_apply

make_node(x)

Construct an Apply node that represent the application of this operation to the given inputs.

This must be implemented by sub-classes.

Returns:

node – The constructed Apply node.

Return type:

Apply

perform(node, args, outs)

Calculate the function on the inputs and put the variables in the output storage.

Parameters:

• node – The symbolic Apply node that represents this computation.

• inputs – Immutable sequence of non-symbolic/numeric inputs. These are the values of each Variable in node.inputs.

• output_storage – List of mutable single-element lists (do not change the length of these lists). Each sub-list corresponds to value of each Variable in node.outputs. The primary purpose of this method is to set the values of these sub-lists.

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a NumPy ndarray with the right number of dimensions and the correct dtype. Its shape and stride pattern can be arbitrary. It is not guaranteed that such pre-set values were produced by a previous call to this Op.perform(); they could’ve been allocated by another Op’s perform method. An Op is free to reuse output_storage as it sees fit, or to discard it and allocate new memory.

class pytensor.compile.ops.FromFunctionOp(fn, itypes, otypes, infer_shape)

Build a basic PyTensor Op around a function.

Since the resulting Op is very basic and is missing most of the optional functionalities, some optimizations may not apply. If you want to help, you can supply an infer_shape function that computes the shapes of the output given the shapes of the inputs.

Also the gradient is undefined in the resulting op and PyTensor will raise an error if you attempt to get the gradient of a graph containing this op.

perform(node, inputs, outputs)

Calculate the function on the inputs and put the variables in the output storage.

Parameters:

• node – The symbolic Apply node that represents this computation.

• inputs – Immutable sequence of non-symbolic/numeric inputs. These are the values of each Variable in node.inputs.

• output_storage – List of mutable single-element lists (do not change the length of these lists). Each sub-list corresponds to value of each Variable in node.outputs. The primary purpose of this method is to set the values of these sub-lists.

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a NumPy ndarray with the right number of dimensions and the correct dtype. Its shape and stride pattern can be arbitrary. It is not guaranteed that such pre-set values were produced by a previous call to this Op.perform(); they could’ve been allocated by another Op’s perform method. An Op is free to reuse output_storage as it sees fit, or to discard it and allocate new memory.

class pytensor.compile.ops.TypeCastingOp

Op that performs a graph-level type cast operation, but has no effect computation-wise (identity function).

c_code(node, nodename, inp, out, sub)

Return the C implementation of an Op.

Returns C code that does the computation associated to this Op, given names for the inputs and outputs.

Parameters:

• node (Apply instance) – The node for which we are compiling the current C code. The same Op may be used in more than one node.

• name (str) – A name that is automatically assigned and guaranteed to be unique.

• inputs (list of strings) – There is a string for each input of the function, and the string is the name of a C variable pointing to that input. The type of the variable depends on the declared type of the input. There is a corresponding python variable that can be accessed by prepending "py_" to the name in the list.

• outputs (list of strings) – Each string is the name of a C variable where the Op should store its output. The type depends on the declared type of the output. There is a corresponding Python variable that can be accessed by prepending "py_" to the name in the list. In some cases the outputs will be preallocated and the value of the variable may be pre-filled. The value for an unallocated output is type-dependent.

• sub (dict of strings) – Extra symbols defined in CLinker sub symbols (such as 'fail').

c_code_cache_version()

Return a tuple of integers indicating the version of this Op.

An empty tuple indicates an “unversioned” Op that will not be cached between processes.

The cache mechanism may erase cached modules that have been superseded by newer versions. See ModuleCache for details.

See also

c_code_cache_version_apply

perform(node, inputs, outputs_storage)

Calculate the function on the inputs and put the variables in the output storage.

Parameters:

• node – The symbolic Apply node that represents this computation.

• inputs – Immutable sequence of non-symbolic/numeric inputs. These are the values of each Variable in node.inputs.

• output_storage – List of mutable single-element lists (do not change the length of these lists). Each sub-list corresponds to value of each Variable in node.outputs. The primary purpose of this method is to set the values of these sub-lists.

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a NumPy ndarray with the right number of dimensions and the correct dtype. Its shape and stride pattern can be arbitrary. It is not guaranteed that such pre-set values were produced by a previous call to this Op.perform(); they could’ve been allocated by another Op’s perform method. An Op is free to reuse output_storage as it sees fit, or to discard it and allocate new memory.

view_map = {0: [0]}

A dict that maps output indices to the input indices of which they are a view.

Examples

view_map = {0: [1]} # first output is a view of second input
view_map = {1: [0]} # second output is a view of first input

class pytensor.compile.ops.ViewOp

Returns an inplace view of the input. Used internally by PyTensor.

make_node(x)

Construct an Apply node that represent the application of this operation to the given inputs.

This must be implemented by sub-classes.

Returns:

node – The constructed Apply node.

Return type:

Apply

pullback(args, outputs, g_outs)

Construct a graph for the vector-Jacobian product (pullback).

Given a function \(f\) implemented by this Op with inputs \(x\) and outputs \(y = f(x)\), the pullback computes \(\bar{x} = \bar{y}^T J\) where \(J\) is the Jacobian \(\frac{\partial f}{\partial x}\) and \(\bar{y}\) are the cotangent vectors (upstream gradients).

This is the core method for reverse-mode automatic differentiation.

If the output is not differentiable with respect to an input, return a variable of type DisconnectedType for that input. If the gradient is not implemented for some input, return a variable of type NullType (see pytensor.gradient.grad_not_implemented() and pytensor.gradient.grad_undefined()).

Parameters:

• inputs (Sequence[Variable]) – The input variables of the Apply node using this Op.

• outputs (Sequence[Variable]) – The output variables of the Apply node using this Op.

• cotangents (Sequence[Variable]) – The cotangent vectors (gradients w.r.t. each output).

Returns:

input_cotangents – The cotangent vectors w.r.t. each input. One Variable per input.

Return type:

list of Variable

pytensor.compile.ops.register_deep_copy_op_c_code(typ, code, version=())

Tell DeepCopyOp how to generate C code for an PyTensor Type.

Parameters:

• typ (PyTensor type) – It must be the PyTensor class itself and not an instance of the class.

• code (C code) – Deep copies the PyTensor type ‘typ’. Use %(iname)s and %(oname)s for the input and output C variable names respectively.

• version – A number indicating the version of the code, for cache.

pytensor.compile.ops.register_view_op_c_code(type, code, version=())

Tell ViewOp how to generate C code for an PyTensor Type.

Parameters:

• type (PyTensor type) – It must be the PyTensor class itself and not an instance of the class.

• code (C code) – Returns a view for the PyTensor type ‘type’. Use %(iname)s and %(oname)s for the input and output C variable names respectively.

• version – A number indicating the version of the code, for cache.

pytensor.compile.ops.wrap_py(itypes, otypes, infer_shape=None)

Decorator that converts a function into a basic PyTensor op that will call the supplied function as its implementation.

It takes an optional infer_shape parameter that should be a callable with this signature:

def infer_shape(fgraph, node, input_shapes):

… return output_shapes

Here input_shapes and output_shapes are lists of tuples that represent the shape of the corresponding inputs/outputs.

This should not be used when performance is a concern since the very basic nature of the resulting Op may interfere with certain graph optimizations.

Examples

@wrap_py(itypes=[pytensor.tensor.fmatrix, pytensor.tensor.fmatrix],

otypes=[pytensor.tensor.fmatrix])

def numpy_dot(a, b):

return numpy.dot(a, b)