Description
linalg.norm
was added to the specification in gh-20 and subsequently placed in a linalg
extension in gh-182.
Signature
linalg.norm(x, /, *, axis=None, keepdims=False, ord=None)
Background
The current API is derived from the comparison of API signatures across array libraries. The initial intent was to codify existing practice among array libraries to ensure API consistency; less attention was given to whether the norm
API was sound/desirable to begin with.
Problem
norm
is easily one of the most complex APIs in the specification. Depending on the dimensionality of x
and the values of the function's keyword arguments, many different behaviors are possible. For example,
-
when
axis
is-
None
andx
is- one-dimensional: vector norm.
- two-dimensional: matrix norm.
- n-dimensional: vector norm over flattened array.
-
an
int
, then specifies dimension along which to compute the vector norm. -
a 2-tuple, then specifies the batch dimensions for computing the matrix norm.
-
Furthermore, the ord
keyword is constrained based on whether one is computing a matrix or vector norm (and thus, the dimensionality of x
and the value of axis
). For example,
- the
ord
valuesfro
ornuc
are strictly matrix norms and thus invalid when provided vectors. - when
ord
is anint
value other than+-1
and+-2
, matrix norms are not currently supported.
Consequently, from an implementation perspective, disentangling valid combinations can be challenging and portends brittle and hard-to-maintain code.
Lastly, the overloading of behavior does not allow for performing batch vector norm operations. One can only batch matrix operations.
Proposal
Torch has introduced separate APIs for computing vector and matrix norms.
-
torch.linalg.vector_norm(A, ord=2, dim=None, keepdim=False, *, dtype=None, out=None) → Tensor
-
torch.linalg.matrix_norm(A, ord='fro', dim=(-2, -1), keepdim=False, *, dtype=None, out=None) → Tensor
For vector_norm
, ord
is restricted to only applicable vector norms and dim
is allowed to be an n-tuple to specify dimensions over which to perform batch operations (e.g., to compute the vector norms over the last two dimensions).
For matrix_norm
, ord
is restricted to only applicable matrix norms and dim
is limited to a 2-tuple to specify batch dimensions (by default, the last two dimensions). This API is similar to other linear algebra APIs in the specification which perform batch operations.
This proposal seeks to follow Torch's lead by more clearly delineating behavior and splitting norm operations into separate APIs. Hence, this proposal seeks to
- remove
linalg.norm
- add
linalg.vector_norm
- add
linalg.matrix_norm
Accordingly, using the same keyword argument names as norm
, the added functions would have the following signatures:
linalg.vector_norm(x, /, *, axis=None, keepdims=False, ord=2)
with the default ord
being the L2-norm (Euclidean norm).
linalg.matrix_norm(x, /, *, axis=(-2,-1), keepdims=False, ord='fro')
with the default ord
being the Frobenius norm over the last two dimensions.
Questions
- Is this proposal desirable?
- For
vector_norm
, should theaxis
default beNone
or the last dimension? IfNone
, would followlinalg.norm
and compute the vector norm over the flattened array. - For
vector_norm
, shouldaxis
be allowed to be ann-tuple
? The Torch implementation does not seem to disallow this, but not clear the use case for, e.g., specifying three dimensions over which to compute a batched vector norm.