[BUG Report]: System.NullReferenceException: 'Object reference not set to an instance of an object.'

[GadgetNutt]

Description

Trying to setup a LSTM in VB.NET, .NET6, VS2022. Granted I am a newbie, but I've tried various ways and I just keep getting this error on the lines indicated here:
inputLayer and lstmLayerFw are set, are not nothing. inputLayer has the shape info it as set in the line above.
SequenceLength = 60
InputFeatures = 17

Protected Overrides Sub BuildModel()
    Dim num_hidden = 64
    Dim layers = keras.layers

    Dim inputLayer = layers.Input(New Integer() {SequenceLength, InputFeatures}, dtype:=TF_DataType.TF_FLOAT)

    ' Define forward LSTM layer
    Dim lstmLayerFw = New LSTM(New LSTMArgs With {.Units = num_hidden})
    Dim lstmOutputFw = lstmLayerFw.Apply(inputLayer)  <----

    ' Define backward LSTM layer
    Dim inputLayerReversed = ReverseTensor(inputLayer)
    Dim lstmLayerBw = New LSTM(New LSTMArgs With {.Units = num_hidden})
    Dim lstmOutputBw = lstmLayerBw.Apply(inputLayerReversed)   <----

Reproduction Steps

See description.

Known Workarounds

None known.

Configuration and Other Information

Windows 11
VS2022
.NET6
TF.NET 0.100.4
TensorFlow.Keras 0.10.4
SciSharp.TensorFlow.Redist 2.11.2

[GadgetNutt]

I think this might be related to a package conflict.

Here are the imports I'm doing:
Imports Tensorflow
Imports Tensorflow.Keras
Imports Tensorflow.Keras.ArgsDefinition
Imports Tensorflow.Keras.Engine
Imports Tensorflow.NumPy
Imports Tensorflow.Binding
Imports Tensorflow.KerasApi
Imports Tensorflow.ApiDef.Types
Imports Tensorflow.Keras.Losses
Imports Tensorflow.Keras.Callback

I got a different error when calling it with tf.keras. prefix:
System.NotImplementedException: '_tensordot_reshape'

[GadgetNutt]

Found it.

    Dim inputLayer = layers.Input(New Integer() {SequenceLength, InputFeatures}, 256,, dtype:=TF_DataType.TF_FLOAT)

Batch size must be set. Should there perhaps be a better exception error so that it's easier to determine this?

[AsakusaRinne]

Hi, thank you for your good catch. In version 0.100.4 the RNN and LSTM has not been totally implemented. @Wanglongzhi2001 is now working on it. If you're finding a way to loading a LSTM model from tensorflow python, #1024 (comment) may help you. If defining keras model of LSTM in tf.net is what you want, please have a wait and we'll include this feature in the next release. :)

[GadgetNutt]

Thanks. I have another update to this. Adding the batch worked for the .Dense implementation, but not for the .LSTM. I have no idea at this point how to identify what is missing in initializing it, but at the moment this doesn't work:

  Dim inputLayer = layers.Input(New Integer() {SequenceLength, InputFeatures}, 256,, dtype:=TF_DataType.TF_FLOAT)
    Dim lstmLayer = layers.LSTM(64, activation:=keras.activations.Relu, return_sequences:=True)
    Dim lstmOutput = lstmLayer.Apply(inputLayer)
    
    System.NullReferenceException: 'Object reference not set to an instance of an object.'

But this does:
Dim inputLayer = layers.Input(New Integer() {SequenceLength, InputFeatures}, 256,, dtype:=TF_DataType.TF_FLOAT)
Dim denseLayer = layers.Dense(64, activation:=keras.activations.Relu)
Dim denseOutput = denseLayer.Apply(inputLayer)

And this doesn't:
Dim inputLayer = layers.Input(New Integer() {SequenceLength, InputFeatures},,, dtype:=TF_DataType.TF_FLOAT)
Dim denseLayer = layers.Dense(64, activation:=keras.activations.Relu)
Dim denseOutput = denseLayer.Apply(inputLayer)

[AsakusaRinne]

Thank you a lot for reporting us it. This is a BUG due to the missing of an implementation. I've fixed it in #1039. 😊

[GadgetNutt]

Hi, thank you for your good catch. In version 0.100.4 the RNN and LSTM has not been totally implemented. @Wanglongzhi2001 is now working on it. If you're finding a way to loading a LSTM model from tensorflow python, #1024 (comment) may help you. If defining keras model of LSTM in tf.net is what you want, please have a wait and we'll include this feature in the next release. :)

Out of curiousity, when is the next release?

[AsakusaRinne]

I think we'll make a release v0.100.5 before publishing v1.0.0. The v0.100.5 will include RNN and LSTM of keras. @Wanglongzhi2001 Any ideas about when these two features will be completed?

[Wanglongzhi2001]

I think we'll make a release v0.100.5 before publishing v1.0.0. The v0.100.5 will include RNN and LSTM of keras. @Wanglongzhi2001 Any ideas about when these two features will be completed?

I will hurry up and finish it in these days.

[GadgetNutt]

Incidentally, this error still seems to be happening in 10.5

Here is a test code for VB.NET

Imports System
Imports Tensorflow
Imports Tensorflow.Keras.Engine
Imports Tensorflow.NumPy
Imports Tensorflow.KerasApi
Imports Tensorflow.Keras.Losses
Imports Tensorflow.Keras.Callbacks

Module Program
Sub Main(args As String())
Dim layers = keras.layers

    Dim inputLayer = layers.Input(New Integer() {10, 5}, dtype:=TF_DataType.TF_FLOAT)
    Console.WriteLine("Input Layer:")
    Console.WriteLine(inputLayer.ToString())

    Dim lstmLayer1 = layers.LSTM(512, return_sequences:=True)
    Console.WriteLine("LSTM Layer:")
    Console.WriteLine(lstmLayer1.ToString())

    Try
        Dim lstmOutput1 = lstmLayer1.Apply(inputLayer)
        Console.WriteLine("Output from LSTM layer:")
        Console.WriteLine(lstmOutput1.ToString())
    Catch ex As Exception
        Console.WriteLine("Error encountered:")
        Console.WriteLine(ex.ToString())
    End Try

    Console.WriteLine("Press any key to exit...")
    Console.ReadKey()
End Sub

End Module


Results in:

System.NullReferenceException
HResult=0x80004003
Message=Object reference not set to an instance of an object.
Source=Tensorflow.Keras
StackTrace:
at Tensorflow.Keras.Layers.Rnn.RNN.build(KerasShapesWrapper input_shape)
at Tensorflow.Keras.Engine.Layer.MaybeBuild(Tensors inputs)
at Tensorflow.Keras.Engine.Layer.FunctionalConstructionCall(Tensors inputs)
at TensorFlowConsoleTest.Program.Main(String[] args) in D:\OneDrive........\TensorFlowConsoleTest\Program.vb:line 22

[Wanglongzhi2001]

I'm sorry, but the RNN and LSTM features are still under development, please be patient for a while, I will let you know as soon as the development is complete.

[GadgetNutt]

I think we'll make a release v0.100.5 before publishing v1.0.0. The v0.100.5 will include RNN and LSTM of keras. @Wanglongzhi2001 Any ideas about when these two features will be completed?

I will hurry up and finish it in these days.

Sorry, I was going by this comment that suggested it was going to be included in 10.5.

[Wanglongzhi2001]

I think we'll make a release v0.100.5 before publishing v1.0.0. The v0.100.5 will include RNN and LSTM of keras. @Wanglongzhi2001 Any ideas about when these two features will be completed?

I will hurry up and finish it in these days.

Sorry, I was going by this comment that suggested it was going to be included in 10.5.
Thank you for your attention to TensorFlow.NET, because RNN is a relatively large feature, so if it is completed, it should be released in v1.0.0.