unable to load saved_model.pb from a custom trained model from t2 model zoo: Invalid GraphDef

[funkysandman]

I have trained an object detection model from the tf2 model zoo. I have exported it from a check point to produce a saved model. This leaves me with a saved_model.pb file. When I attempt to load this model using the sample code for object detection, I hit an error at graph.import(pbfilie) : Tensorflow.InvalidArgumentError: 'Invalid GraphDef'

version of Tensorflow.Net is v 0.30.0

zoo model is EfficientDet-d0

[mdable2]

I am also having the same issue. Here are the details around my problem and what I have tried:

I have trained a model in Python using Tensorflow 2.3 and the Tensorflow Object Detection API. I save the model using the SavedModel format that gives me a .pb file. From there I am trying to use that graph in Tensorflow.NET. To do so I am trying to mimic one of the TensorFlow.NET examples (DetectInMobilenet.cs).

The example starts with Run():

        public bool Run()
        {
            tf.compat.v1.disable_eager_execution();

            Predict();

            return true;
        }

        public Graph ImportGraph()
        {
            var graph = new Graph().as_default();
            graph.Import(Path.Join(modelDir, pbFile));

            return graph;
        }

        public void Predict()
        {
            // read in the input image
            var imgArr = ReadTensorFromImageFile(Path.Join(imageDir, picFile));

            var graph = ImportGraph();

            using (var sess = tf.Session(graph))
            {
                Tensor tensorNum = graph.OperationByName("num_detections");
                Tensor tensorBoxes = graph.OperationByName("detection_boxes");
                Tensor tensorScores = graph.OperationByName("detection_scores");
                Tensor tensorClasses = graph.OperationByName("detection_classes");
                Tensor imgTensor = graph.OperationByName("image_tensor");
                var outTensorArr = new Tensor[] { tensorNum, tensorBoxes, tensorScores, tensorClasses };

                var results = sess.run(outTensorArr, new FeedItem(imgTensor, imgArr));

                BuildOutputImage(results);
            }
        }

This works fine. So then I try to implement this in my solution.

Here is what I have tried so far:

tf.compat.v1.disable_eager_execution();
var graph = new Graph().as_default();
graph.Import(modelPath);

This gives this error: Tensorflow.InvalidArgumentError: 'Invalid GraphDef'. After Googling around, it looks like the format that graph.Import() is looking for is a frozen graph. But since I am using TF 2.3, it no longer supports saving in the frozen format and instead uses the SavedModel format.

Then I tried:

tf_with(Session.LoadFromSavedModel(modelPath), sess =>
			{
				Tensor tensorNum = sess.graph.OperationByName("num_detections");
				Tensor tensorBoxes = sess.graph.OperationByName("detection_boxes");
				Tensor tensorScores = sess.graph.OperationByName("detection_scores");
				Tensor tensorClasses = sess.graph.OperationByName("detection_classes");
				Tensor imgTensor = sess.graph.OperationByName("image_tensor");
				var outTensorArr = new Tensor[] { tensorNum, tensorBoxes, tensorScores, tensorClasses };

				var results = sess.run(outTensorArr, new FeedItem(imgTensor, image));
				return results;
			});

And this loads the model fine, but then says that this line Tensor tensorNum = sess.graph.OperationByName("num_detections"); is throwing an error saying that "num_detections" is not found in the graph.

Thanks!

[xsoheilalizadeh]

I have the same issue when I load a trained model through ML.NET which uses Tensorflow.NET 20.1.

The returned status is TF_INVALID_ARGUMENT.

TensorFlow.NET/src/TensorFlowNET.Core/Graphs/Graph.Import.cs

Line 58 in ba49010

status.Check(true);

An unhandled exception of type 'System.FormatException' occurred in Microsoft.ML.TensorFlow.dll: 'Tensorflow exception triggered while loading model.'
 Inner exceptions found, see $exception in variables window for more details.
 Innermost exception 	 Tensorflow.InvalidArgumentError : Invalid GraphDef
   at Tensorflow.Status.Check(Boolean throwException)
   at Tensorflow.Graph.Import(Byte[] bytes, String prefix)
   at Tensorflow.Graph.Import(String file_path, String prefix)
   at Microsoft.ML.TensorFlow.TensorFlowUtils.LoadTFSessionByModelFilePath(IExceptionContext ectx, String modelFile, Boolean metaGraph)

[Oceania2018]

@xsoheilalizadeh Could you PR a runnable unit test include model file, so we can investigate the issue.

[xsoheilalizadeh]

https://github.com/xsoheilalizadeh/TensorflowIssue
@Oceania2018

[xsoheilalizadeh]

@Oceania2018 Do you have any updates?

[Oceania2018]

@xsoheilalizadeh Are you sure whether it's a valid pb?
I can't read it even in python

import tensorflow as tf
from tensorflow.python.platform import gfile
GRAPH_PB_PATH = './data/saved_model.pb'
with tf.compat.v1.Session() as sess:
   print("load graph")
   with tf.io.gfile.GFile(GRAPH_PB_PATH,'rb') as f:
       graph_def = tf.compat.v1.GraphDef()
       graph_def.ParseFromString(f.read())
   sess.graph.as_default()
   tf.import_graph_def(graph_def, name='')
   graph_nodes=[n for n in graph_def.node]
   names = []
   for t in graph_nodes:
      names.append(t.name)
   print(names)

[xsoheilalizadeh]

It's trained by TensorFlow 2.x, I already load it with tensorflow.js

[ButterMeWaffle]

Is there any update on this issue? I too am experiencing it and even after managing to create a frozen graph I cannot get model to load

[Exitare]

Having the same issue too.

Tensorflow.InvalidArgumentError: Invalid GraphDef

[bbhxwl]

me too

[captainst]

The .pb file created using object detection api 2 is a saved_model format, which enables eager mode. I don't think that you can load it using the conventional method.
For reference, opencv/opencv#19257, tensorflow/models#8966