Open
Description
I have written a custom training loop with a mask operation (tf.multiply). Unfortunately I get an error when calculating gradients.
Here are the ops recorded on tape:
| Name | Value | Type | |
|---|---|---|---|
| tf.call(); | |||
| ▶ | [0] | {ReadVariableOp, inputs: 23} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [1] | {MatMul, inputs: 1997,2004} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [2] | {ReadVariableOp, inputs: 29} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [3] | {Identity, inputs: 2006} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [4] | {BiasAdd, inputs: 2005,2007} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [5] | {Relu, inputs: 2008} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [6] | {ReadVariableOp, inputs: 47} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [7] | {MatMul, inputs: 2009,2010} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [8] | {ReadVariableOp, inputs: 53} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [9] | {Identity, inputs: 2012} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [10] | {BiasAdd, inputs: 2011,2013} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [11] | {Relu, inputs: 2014} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [12] | {ReadVariableOp, inputs: 71} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [13] | {MatMul, inputs: 2015,2016} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [14] | {ReadVariableOp, inputs: 77} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [15] | {Identity, inputs: 2018} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [16] | {BiasAdd, inputs: 2017,2019} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| tf.Multiply(); | |||
| ▶ | [17] | {Pack, inputs: 2020} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [18] | {Mul, inputs: 2065,2002} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [19] | {StridedSlice, inputs: 2066,2067,2068,2069} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| tf.reduce_sum(); | |||
| ▶ | [20] | {Sum, inputs: 2070,2126} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| lossFunction.Call(); | |||
| ▶ | [21] | {SquaredDifference, inputs: 2127,2150} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [22] | {Mean, inputs: 2151,2152} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [23] | {Mul, inputs: 2153,2154} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [24] | {Sum, inputs: 2155,2160} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [25] | {Sum, inputs: 2161,2168} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
| ▶ | [26] | {DivNoNan, inputs: 2169,2163} | Tensorflow.Gradients.OpTapeEntry<Tensorflow.BackwardFunction, Tensorflow.Gradients.TapeTensor> |
And the calling tape.gradient() results in this exception:
System.Collections.Generic.KeyNotFoundException
HResult=0x80131577
Message=The given key 'Pack' was not present in the dictionary.
Source=System.Private.CoreLib
StackTrace:
at System.ThrowHelper.ThrowKeyNotFoundException[T](T key)
at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
at Tensorflow.Eager.EagerRunner.<>c__DisplayClass5_0.<GetGradientFunction>b__0(Tensor[] output_grads, Int64[] unneeded_gradients)
at Tensorflow.Gradients.Tape.CallBackwardFunction(BackwardFunction backward_function, List`1 unneeded_gradients, List`1 output_gradients)
at Tensorflow.Gradients.Tape.ComputeGradient(Int64[] target_tensor_ids, Int64[] source_tensor_ids, UnorderedMap`2 sources_that_are_targets, Tensor[] output_gradients)
at Tensorflow.Gradients.GradientTape.gradient(Tensor target, IEnumerable`1 sources)