`tensor.elemwise` – Tensor Elemwise#

class pytensor.tensor.elemwise.CAReduce(scalar_op, axis=None, dtype=None, acc_dtype=None, upcast_discrete_output=False)[source]#

Reduces a scalar operation along specified axes.

The scalar op should be both commutative and associative.

CAReduce = Commutative Associative Reduce.

The output will have the same shape as the input minus the reduced dimensions. It will contain the variable of accumulating all values over the reduced dimensions using the specified scalar Op.

Notes

CAReduce(add)  # sum (ie, acts like the numpy sum operation)
CAReduce(mul)  # product
CAReduce(maximum)  # max
CAReduce(minimum)  # min
CAReduce(or_)  # any # not lazy
CAReduce(and_)  # all # not lazy
CAReduce(xor)  # a bit at 1 tell that there was an odd number of
# bit at that position that where 1. 0 it was an
# even number ...

In order to (eventually) optimize memory usage patterns, CAReduce makes zero guarantees on the order in which it iterates over the dimensions and the elements of the array(s). Therefore, to ensure consistent variables, the scalar operation represented by the reduction must be both commutative and associative (eg add, multiply, maximum, binary or/and/xor - but not subtract, divide or power).

c_code(node, name, inames, onames, sub)[source]#

Return the C implementation of an Op.

Returns C code that does the computation associated to this Op, given names for the inputs and outputs.

Parameters:

node (Apply instance) – The node for which we are compiling the current C code. The same Op may be used in more than one node.
name (str) – A name that is automatically assigned and guaranteed to be unique.
inputs (list of strings) – There is a string for each input of the function, and the string is the name of a C variable pointing to that input. The type of the variable depends on the declared type of the input. There is a corresponding python variable that can be accessed by prepending "py_" to the name in the list.
outputs (list of strings) – Each string is the name of a C variable where the Op should store its output. The type depends on the declared type of the output. There is a corresponding Python variable that can be accessed by prepending "py_" to the name in the list. In some cases the outputs will be preallocated and the value of the variable may be pre-filled. The value for an unallocated output is type-dependent.
sub (dict of strings) – Extra symbols defined in CLinker sub symbols (such as 'fail').

c_code_cache_version_apply(node)[source]#

Return a tuple of integers indicating the version of this Op.

An empty tuple indicates an “unversioned” Op that will not be cached between processes.

The cache mechanism may erase cached modules that have been superseded by newer versions. See ModuleCache for details.

See also

c_code_cache_version

Notes

This function overrides c_code_cache_version unless it explicitly calls c_code_cache_version. The default implementation simply calls c_code_cache_version and ignores the node argument.

c_headers(**kwargs)[source]#

Return a list of header files required by code returned by this class.

These strings will be prefixed with #include and inserted at the beginning of the C source code.

Strings in this list that start neither with < nor " will be enclosed in double-quotes.

Examples

def c_headers(self, **kwargs):
    return ["<iostream>", "<math.h>", "/full/path/to/header.h"]

make_node(input)[source]#

Construct an Apply node that represent the application of this operation to the given inputs.

This must be implemented by sub-classes.

Returns:: node – The constructed Apply node.
Return type:: Apply

perform(node, inp, out)[source]#

Calculate the function on the inputs and put the variables in the output storage.

Parameters:

node – The symbolic Apply node that represents this computation.
inputs – Immutable sequence of non-symbolic/numeric inputs. These are the values of each Variable in node.inputs.
output_storage – List of mutable single-element lists (do not change the length of these lists). Each sub-list corresponds to value of each Variable in node.outputs. The primary purpose of this method is to set the values of these sub-lists.

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a NumPy ndarray with the right number of dimensions and the correct dtype. Its shape and stride pattern can be arbitrary. It is not guaranteed that such pre-set values were produced by a previous call to this Op.perform(); they could’ve been allocated by another Op’s perform method. An Op is free to reuse output_storage as it sees fit, or to discard it and allocate new memory.

class pytensor.tensor.elemwise.DimShuffle(*, input_ndim, new_order)[source]#

Allows to reorder the dimensions of a tensor or insert or remove broadcastable dimensions.

In the following examples, ‘x’ means that we insert a broadcastable dimension and a numerical index represents the dimension of the same rank in the tensor passed to perform.

Parameters:

input_ndim – The expected number of dimension of the input
new_order – A list representing the relationship between the input’s dimensions and the output’s dimensions. Each element of the list can either be an index or ‘x’. Indices must be encoded as python integers, not pytensor symbolic integers. Missing indexes correspond to drop dimensions.

Notes

If j = new_order[i] is an index, the output’s ith dimension will be the input’s jth dimension. If new_order[i] is x, the output’s ith dimension will be 1 and broadcast operations will be allowed to do broadcasting over that dimension.

If input.type.shape[i] != 1 then i must be found in new_order. Broadcastable dimensions, on the other hand, can be discarded.

DimShuffle(input_ndim=3, new_order=["x", 2, "x", 0, 1])

This Op will only work on 3d tensors. The first dimension of the output will be broadcastable, then we will have the third dimension of the input tensor as the second of the resulting tensor, etc. If the tensor has shape (20, 30, 40), the resulting tensor will have dimensions (1, 40, 1, 20, 30). (AxBxC tensor is mapped to 1xCx1xAxB tensor)

DimShuffle(input_ndim=2, new_order=[1])

This Op will only work on 2d tensors with the first dimension broadcastable. The second dimension of the input tensor will be the first dimension of the resulting tensor. If the tensor has shape (1, 20), the resulting tensor will have shape (20, ).

Examples

DimShuffle(input_ndim=0, new_order=["x"])  # make a 0d (scalar) into a 1d vector
DimShuffle(input_ndim=2, new_order=[0, 1])  # identity
DimShuffle(input_ndim=2, new_order=[1, 0])  # transposition
# Make a row out of a 1d vector (N to 1xN)
DimShuffle(input_ndim=1, new_order=["x", 0])
# Make a colum out of a 1d vector (N to Nx1)
DimShuffle(input_ndim=1, new_order=[0, "x"])
DimShuffle(input_ndim=3, new_order=[2, 0, 1])  # AxBxC to CxAxB
DimShuffle(input_ndim=2, new_order=[0, "x", 1])  # AxB to Ax1xB
DimShuffle(input_ndim=2, new_order=[1, "x", 0])  # AxB to Bx1xA

Notes

The python implementation of this Op combines numpy.transpose for reordering of the dimensions and numpy.reshape for subtracting and adding broadcastable dimensions.

make_node(inp)[source]#

Construct an Apply node that represent the application of this operation to the given inputs.

This must be implemented by sub-classes.

Returns:: node – The constructed Apply node.
Return type:: Apply

perform(node, inp, out)[source]#

Calculate the function on the inputs and put the variables in the output storage.

Parameters:

node – The symbolic Apply node that represents this computation.
inputs – Immutable sequence of non-symbolic/numeric inputs. These are the values of each Variable in node.inputs.
output_storage – List of mutable single-element lists (do not change the length of these lists). Each sub-list corresponds to value of each Variable in node.outputs. The primary purpose of this method is to set the values of these sub-lists.

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a NumPy ndarray with the right number of dimensions and the correct dtype. Its shape and stride pattern can be arbitrary. It is not guaranteed that such pre-set values were produced by a previous call to this Op.perform(); they could’ve been allocated by another Op’s perform method. An Op is free to reuse output_storage as it sees fit, or to discard it and allocate new memory.

pullback(inp, outputs, grads)[source]#

Construct a graph for the vector-Jacobian product (pullback).

Given a function \(f\) implemented by this Op with inputs \(x\) and outputs \(y = f(x)\), the pullback computes \(\bar{x} = \bar{y}^T J\) where \(J\) is the Jacobian \(\frac{\partial f}{\partial x}\) and \(\bar{y}\) are the cotangent vectors (upstream gradients).

This is the core method for reverse-mode automatic differentiation.

If the output is not differentiable with respect to an input, return a variable of type DisconnectedType for that input. If the gradient is not implemented for some input, return a variable of type NullType (see pytensor.gradient.grad_not_implemented() and pytensor.gradient.grad_undefined()).

Parameters:

inputs (Sequence[Variable]) – The input variables of the Apply node using this Op.
outputs (Sequence[Variable]) – The output variables of the Apply node using this Op.
cotangents (Sequence[Variable]) – The cotangent vectors (gradients w.r.t. each output).

Returns:

input_cotangents – The cotangent vectors w.r.t. each input. One Variable per input.

Return type:

list of Variable

pushforward(inputs, outputs, tangents)[source]#

Construct a graph for the Jacobian-vector product (pushforward).

Given a function \(f\) implemented by this Op with inputs \(x\) and outputs \(y = f(x)\), the pushforward computes \(\dot{y} = J \dot{x}\) where \(J\) is the Jacobian \(\frac{\partial f}{\partial x}\) and \(\dot{x}\) are the tangent vectors.

This is the core method for forward-mode automatic differentiation.

If an output is not differentiable with respect to any input, return a variable of type DisconnectedType for that output. Unlike the legacy R_op method, pushforward must never use None to indicate disconnected outputs.

Parameters:

inputs (Sequence[Variable]) – The input variables of the Apply node using this Op.
outputs (Sequence[Variable]) – The output variables of the Apply node using this Op.
tangents (Sequence[Variable]) – The tangent vectors. One per input. A variable of DisconnectedType indicates that the corresponding input is not being differentiated.

Returns:

output_tangents – The tangent vectors w.r.t. each output. One Variable per output.

Return type:

list of Variable

view_map = {0: [0]}[source]#

A dict that maps output indices to the input indices of which they are a view.

Examples

view_map = {0: [1]} # first output is a view of second input
view_map = {1: [0]} # second output is a view of first input

class pytensor.tensor.elemwise.DimShufflePrinter[source]#

process(r, pstate)[source]#: Construct a string representation for a Variable.

class pytensor.tensor.elemwise.Elemwise(scalar_op, inplace_pattern=None, name=None, nfunc_spec=None, openmp=None)[source]#

Generalizes a scalar Op to tensors.

All the inputs must have the same number of dimensions. When the Op is performed, for each dimension, each input’s size for that dimension must be the same. As a special case, it can also be one but only if the input’s broadcastable flag is True for that dimension. In that case, the tensor is (virtually) replicated along that dimension to match the size of the others.

The dtypes of the outputs mirror those of the scalar Op that is being generalized to tensors. In particular, if the calculations for an output are done in-place on an input, the output type must be the same as the corresponding input type (see the doc of ScalarOp to get help about controlling the output type)

Notes

-Elemwise(add): represents + on tensors x + y -Elemwise(add, {0 : 0}): represents the += operation x += y -Elemwise(add, {0 : 1}): represents += on the second argument y += x -Elemwise(mul)(np.random.random((10, 5)), np.random.random((1, 5))): the second input is completed along the first dimension to match the first input -Elemwise(true_div)(np.random.random(10, 5), np.random.random(10, 1)): same but along the second dimension -Elemwise(int_div)(np.random.random((1, 5)), np.random.random((10, 1))): the output has size (10, 5). -Elemwise(log)(np.random.random((3, 4, 5)))

c_code(node, nodename, inames, onames, sub)[source]#

Return the C implementation of an Op.

Returns C code that does the computation associated to this Op, given names for the inputs and outputs.

Parameters:

node (Apply instance) – The node for which we are compiling the current C code. The same Op may be used in more than one node.
name (str) – A name that is automatically assigned and guaranteed to be unique.
inputs (list of strings) – There is a string for each input of the function, and the string is the name of a C variable pointing to that input. The type of the variable depends on the declared type of the input. There is a corresponding python variable that can be accessed by prepending "py_" to the name in the list.
outputs (list of strings) – Each string is the name of a C variable where the Op should store its output. The type depends on the declared type of the output. There is a corresponding Python variable that can be accessed by prepending "py_" to the name in the list. In some cases the outputs will be preallocated and the value of the variable may be pre-filled. The value for an unallocated output is type-dependent.
sub (dict of strings) – Extra symbols defined in CLinker sub symbols (such as 'fail').

c_code_cache_version_apply(node)[source]#

Return a tuple of integers indicating the version of this Op.

An empty tuple indicates an “unversioned” Op that will not be cached between processes.

The cache mechanism may erase cached modules that have been superseded by newer versions. See ModuleCache for details.

See also

c_code_cache_version

Notes

This function overrides c_code_cache_version unless it explicitly calls c_code_cache_version. The default implementation simply calls c_code_cache_version and ignores the node argument.

c_header_dirs(**kwargs)[source]#

Return a list of header search paths required by code returned by this class.

Provides search paths for headers, in addition to those in any relevant environment variables.

Note

For Unix compilers, these are the things that get -I prefixed in the compiler command line arguments.

Examples

def c_header_dirs(self, **kwargs):
    return ["/usr/local/include", "/opt/weirdpath/src/include"]

c_headers(**kwargs)[source]#: Return the header file name "omp.h" if OpenMP is supported.

c_support_code(**kwargs)[source]#

Return utility code for use by a Variable or Op.

This is included at global scope prior to the rest of the code for this class.

Question: How many times will this support code be emitted for a graph with many instances of the same type?

Return type:: str

c_support_code_apply(node, nodename)[source]#

Return Apply-specialized utility code for use by an Op that will be inserted at global scope.

Parameters:

node (Apply) – The node in the graph being compiled.
name (str) – A string or number that serves to uniquely identify this node. Symbol names defined by this support code should include the name, so that they can be called from the CLinkerOp.c_code(), and so that they do not cause name collisions.

Notes

This function is called in addition to CLinkerObject.c_support_code() and will supplement whatever is returned from there.

get_output_info(*inputs)[source]#: Return the outputs dtype and broadcastable pattern and the dimshuffled inputs.

make_node(*inputs)[source]#: If the inputs have different number of dimensions, their shape is left-completed to the greatest number of dimensions with 1s using DimShuffle.

perform(node, inputs, output_storage)[source]#

Calculate the function on the inputs and put the variables in the output storage.

Parameters:

node – The symbolic Apply node that represents this computation.
inputs – Immutable sequence of non-symbolic/numeric inputs. These are the values of each Variable in node.inputs.
output_storage – List of mutable single-element lists (do not change the length of these lists). Each sub-list corresponds to value of each Variable in node.outputs. The primary purpose of this method is to set the values of these sub-lists.

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a NumPy ndarray with the right number of dimensions and the correct dtype. Its shape and stride pattern can be arbitrary. It is not guaranteed that such pre-set values were produced by a previous call to this Op.perform(); they could’ve been allocated by another Op’s perform method. An Op is free to reuse output_storage as it sees fit, or to discard it and allocate new memory.

prepare_node(node, storage_map, compute_map, impl)[source]#

Make any special modifications that the Op needs before doing Op.make_thunk().

This can modify the node inplace and should return nothing.

It can be called multiple time with different impl values.

Warning

It is the Op’s responsibility to not re-prepare the node when it isn’t good to do so.

pullback(inputs, outs, ograds)[source]#

Construct a graph for the vector-Jacobian product (pullback).

Given a function \(f\) implemented by this Op with inputs \(x\) and outputs \(y = f(x)\), the pullback computes \(\bar{x} = \bar{y}^T J\) where \(J\) is the Jacobian \(\frac{\partial f}{\partial x}\) and \(\bar{y}\) are the cotangent vectors (upstream gradients).

This is the core method for reverse-mode automatic differentiation.

If the output is not differentiable with respect to an input, return a variable of type DisconnectedType for that input. If the gradient is not implemented for some input, return a variable of type NullType (see pytensor.gradient.grad_not_implemented() and pytensor.gradient.grad_undefined()).

Parameters:

inputs (Sequence[Variable]) – The input variables of the Apply node using this Op.
outputs (Sequence[Variable]) – The output variables of the Apply node using this Op.
cotangents (Sequence[Variable]) – The cotangent vectors (gradients w.r.t. each output).

Returns:

input_cotangents – The cotangent vectors w.r.t. each input. One Variable per input.

Return type:

list of Variable

pushforward(inputs, outputs, tangents)[source]#

Construct a graph for the Jacobian-vector product (pushforward).

Given a function \(f\) implemented by this Op with inputs \(x\) and outputs \(y = f(x)\), the pushforward computes \(\dot{y} = J \dot{x}\) where \(J\) is the Jacobian \(\frac{\partial f}{\partial x}\) and \(\dot{x}\) are the tangent vectors.

This is the core method for forward-mode automatic differentiation.

If an output is not differentiable with respect to any input, return a variable of type DisconnectedType for that output. Unlike the legacy R_op method, pushforward must never use None to indicate disconnected outputs.

Parameters:

inputs (Sequence[Variable]) – The input variables of the Apply node using this Op.
outputs (Sequence[Variable]) – The output variables of the Apply node using this Op.
tangents (Sequence[Variable]) – The tangent vectors. One per input. A variable of DisconnectedType indicates that the corresponding input is not being differentiated.

Returns:

output_tangents – The tangent vectors w.r.t. each output. One Variable per output.

Return type:

list of Variable

pytensor.tensor.elemwise.get_normalized_batch_axes(core_axes, core_ndim, batch_ndim)[source]#

Compute batch axes for a batched operation, from the core input ndim and axes.

e.g., sum(matrix, axis=None) -> sum(tensor4, axis=(2, 3)) batch_axes(None, 2, 4) -> (2, 3)

e.g., sum(matrix, axis=0) -> sum(tensor4, axis=(2,)) batch_axes(0, 2, 4) -> (2,)

e.g., sum(tensor3, axis=(0, -1)) -> sum(tensor4, axis=(1, 3)) batch_axes((0, -1), 3, 4) -> (1, 3)

pytensor.tensor.elemwise.scalar_elemwise(*symbol, nfunc=None, nin=None, nout=None, symbolname=None)[source]#

Replace a symbol definition with an Elemwise-wrapped version of the corresponding scalar Op.

If it is not None, the nfunc argument should be a string such that getattr(numpy, nfunc) implements a vectorized version of the Elemwise operation. nin is the number of inputs expected by that function, and nout is the number of destination inputs it takes. That is, the function should take nin + nout inputs. nout == 0 means that the numpy function does not take a NumPy array argument to put its result in.

tensor.elemwise – Tensor Elemwise#

`tensor.elemwise` – Tensor Elemwise#