Gradient of tf.reduce_sum(_, axis = _) throws an exception

[mishun]

Hi!

This example (it creates 2x2 matrix, uses tf.reduce_sum on it and tries to differentiate the result):

open Tensorflow
open type Tensorflow.Binding

[<EntryPoint>]
let main argv =
    tf.compat.v1.disable_eager_execution()
    let x = tf.placeholder(tf.float64, shape = TensorShape(2))
    let sq = tf.matmul(tf.reshape(x, shape = TensorShape(2, 1)), tf.reshape(x, shape = TensorShape(1, 2))) // Make 2z2 matrix
    let y = tf.reduce_sum(sq, axis = 0)
    let g = tf.gradients(y, x)
    begin
        use session = tf.Session()
        let res = session.run(g.[0], [| FeedItem(x, [| -1.0 ; 1.0 |]) |])
        printfn "result = %O" res
    end
    0

causes following exception:

Unhandled exception. System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.NotImplementedException
   at Tensorflow.Gradients.math_grad._SumGrad(Operation op, Tensor[] grads)
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.RuntimeType.InvokeMember(String name, BindingFlags bindingFlags, Binder binder, Object target, Object[] providedArgs, ParameterModifier[] modifiers, CultureInfo culture, String[] namedParams)
   at Tensorflow.ops.<>c__DisplayClass1_1.<RegisterFromAssembly>b__3(Operation oper, Tensor[] out_grads)
   at Tensorflow.gradients_util._MaybeCompile(String scope, Operation op, Tensor[] out_grads, Action func, Func`3 grad_fn)
   at Tensorflow.gradients_util.<>c__DisplayClass0_2.<_GradientsHelper>b__6(NameScope scope1)
   at Tensorflow.Binding.tf_with[T](T py, Action`1 action)
   at Tensorflow.gradients_util.<>c__DisplayClass0_0.<_GradientsHelper>b__0(NameScope scope)
   at Tensorflow.Binding.tf_with[T](T py, Action`1 action)
   at Tensorflow.gradients_util._GradientsHelper(Tensor[] ys, Tensor[] xs, Tensor[] grad_ys, String name, Boolean colocate_gradients_with_ops, Boolean gate_gradients, Int32 aggregation_method, Tensor[] stop_gradients, Graph src_graph)
   at Tensorflow.tensorflow.gradients(Tensor ys, Tensor xs, Tensor[] grad_ys, String name, Boolean colocate_gradients_with_ops, Boolean gate_gradients, Nullable`1 aggregation_method, Tensor[] stop_gradients)
   at Program.main(String[] argv) in C:\Users\user\source\repos\TF-Test\Program.fs:line 10

[mishun]

I tried to look into fixing that and, if I understand correctly, these lines are culprit:

TensorFlow.NET/src/TensorFlowNET.Core/Gradients/math_grad.cs

Lines 530 to 533 in 6adcfae

	else if (!input_0_shape.Contains(-1) && !tf.Context.executing_eagerly())
	{
	throw new NotImplementedException("");
	}

Now I don't understand how _SumGrad works. Isn't is supposed to look at keep_dims here:

TensorFlow.NET/src/TensorFlowNET.Core/Gradients/math_grad.cs

Lines 514 to 515 in 6adcfae

	var new_shape = constant_op.constant(range(0, rank).Select(x => 1).ToArray(), dtype: TF_DataType.TF_INT32);
	grad = array_ops.reshape(grad, new_shape);

and here:

TensorFlow.NET/src/TensorFlowNET.Core/Gradients/math_grad.cs

Lines 519 to 520 in 6adcfae

	var new_shape = range(rank).Select(x => 1).ToArray();
	grad = array_ops.reshape(grad, new_shape);

? Why is -1 here:

TensorFlow.NET/src/TensorFlowNET.Core/Gradients/math_grad.cs

Line 530 in 6adcfae

else if (!input_0_shape.Contains(-1) && !tf.Context.executing_eagerly())

? And why these things:

TensorFlow.NET/src/TensorFlowNET.Core/Gradients/math_grad.cs

Line 549 in 6adcfae

return new Tensor[] { gen_array_ops.broadcast_to(grad, input_shape), null };

TensorFlow.NET/src/TensorFlowNET.Core/Gradients/math_grad.cs

Line 558 in 6adcfae

return new Tensor[] { gen_array_ops.tile(grad, tile_scaling), null };

do something different from each other?

[Oceania2018]

Can you update with latest version?
If the issue still exist, please reopen this issue and PR a C# unit test to repreduce.

[mishun]

I haven't checked it yet, but since nothing has changed in

TensorFlow.NET/src/TensorFlowNET.Core/Gradients/math_grad.cs

Line 497 in 6d21358

public static Tensor[] _SumGrad(Operation op, Tensor[] grads)

for pretty long time I'm reasonably sure that it still doesn't work. Equivalent part in tensorflow's code suggests that there's some optimization there that just isn't implemented:
https://github.com/tensorflow/tensorflow/blob/21cc3eb8393991e312a2c2de397508cb297364d6/tensorflow/python/ops/math_grad.py#L149
On the other hand, just leaving non-optimized version is probably better than throwing an exception.

Could you remind me where unit tests for non-eager differentiation go? (I PR'ed a couple tests there a couple years ago, but forgot since)

[Oceania2018]

@mishun You can PR a test in this file.
There are commits what you have ever PRed.

[mishun]

Test case added: #917

[mishun]

Looks like I have no rights to reopen this one. Should I create new issue?