example of loading a tensorflow model

[machuck]

Hi, do we have an example of loading a tensorflow model some where in the docs already? If not yet, can you provide one? thanks

[ludwigschubert]

We do not yet, and we will add one. Thank you for your patience!

[machuck]

thanks, looking forward to!

[JegZheng]

Thanks for reply. I just notice that model should be from moralex@'s modelzoo when using optvis. Is there any differences between the models in modelzoo and our trained models? BTW, could you please provide a link for moralex@'s modelzoo? Thanks.

[ludwigschubert]

Hey everyone (@machuck, @Wursthub, @emptyewer, @JegZheng)!
Writing up a guide has proven more challenging than I expected. If you want to help out and have the time you can take a look at this WIP notebook and leave any comments you have:

https://drive.google.com/file/d/1PPzeZi5sBN2YRlBmKsdvZPbfYtZI-pHl

Thanks for your continued patience! :-)

[tul-urte]

Hi Ludwig,

I am successfully training using TensorFlow 1.8.0 on the standard SSD MobileNet V1 model config file, and exporting graphs using the object_detection script "export_inference_graph.py". I'm working in continuous mode whereby I collect and classify new images every day, feed back some of them to the Ground Truth data set, and then train again (once a week).

So I'm very keen and interested to use Lucid to explore the output models and to inform the "feedback-to-ground-truth" step in a more scientific way. Obviously it's amazingly cool too.

So I jumped in, tried the tutorial notebook, and it worked perfectly. But when I switched to one of my own exported graphs I got errors. What did I expect - I've managed to get this far without really understanding what's going on with Tensorflow graphs.

Eventually I understood that the "input" and "layer" choices are critical. There are so many different errors that occur if these are poor choices, largely due to the "non-backpropagatability" of operations in the graph between the designated "input" and "layer". I was massively helped by the example on this site: https://github.com/Hvass-Labs/TensorFlow-Tutorials/blob/master/14_DeepDream.ipynb

What works for me is:

# Lucid Example

input_name = 'Preprocessor/sub'
layer = "FeatureExtractor/MobilenetV1/Conv2d_13_pointwise_2_Conv2d_5_3x3_s2_128/Relu6"

# therefore pick some index between 0..127
index = 105


import os
import lucid.optvis.objectives as objectives
import lucid.optvis.param as param
import lucid.optvis.render as render
import lucid.optvis.transform as transform
from lucid.modelzoo.vision_base import Model

# locations
graph_dir = "/XENOPHON/nutsack/graphs/nutsack_11_v8_510x510_02_150k/"
graph_name = "inference_graph.pb"
labels_name = "object-detection.pbtxt"

graph_path = os.path.join( graph_dir, graph_name )
labels_path = os.path.join( graph_dir, labels_name )

# zoo-like model
class SSDMobilenetV1(Model):
    def __init__(self, graph_path, labels_path, input_name, image_shape=None, image_value_range=None ):        
        self.model_path = graph_path
        self.labels_path = labels_path
        self.input_name = input_name
        self.image_shape = image_shape
        self.image_value_range = image_value_range
        super().__init__()

scale = 300
image_shape = [ 510, 510, 3 ]
image_value_range = (-1, 1)

param_f = lambda: param.image( scale, fft=True, decorrelate=True )

def get_objective( layer, index=0 ):
    return objectives.channel( layer , index )

transforms = [
    transform.pad(16),
    transform.jitter(32),
    transform.random_scale([n/100. for n in range(80, 120)]),
#     transform.random_rotate(range(-10,10) + range(-5,5) + 10*range(-2,2)),
    transform.jitter(2)
]

model = SSDMobilenetV1( graph_path, labels_path, input_name, image_shape, image_value_range )
model.load_graphdef()

_ = render.render_vis( model, get_objective( layer, index ), transforms=transforms, param_f=param_f, thresholds=(scale,) )

[ludwigschubert]

Hi @tul-urte,
thanks for your patience and I'm sorry you're running into this much trouble. When we started writing lucid there was not yet a standard for how to save graphs, so there's much confusion around the topic.

Regarding your concrete issue:

Model classes use the attribute input_name to know which tensor to feed the input image in your model. That name comes from your original model definition or may be set in an export script, etc. From the name of the files you're referencing I'm guessing it originates at this line! Looks like the right name, so what's wrong?

In your current code you create a new placeholder that's not connected at all to your original model's loaded graph. That's why you get the "No gradients…" Value Error. You shouldn't have to modify the create_input function. Instead, ensure you know the name of your model's input and use it in your Model subclass.

Use the code in the linked notebook (for node in graph_def.node: …) to look at the names of the nodes in your graph. Maybe you're just missing a prefix.

Hope this helped! Let me know whether it did or did not. :-)

[qihongl]

Thanks for this thread! It would be awesome if this package interfaces with generic Tensorflow models easily.

[ilamanov]

Hi @ludwigschubert ,

I am getting a
ValueError: No gradients provided for any variable, check your graph for ops that do not support gradients, between variables ["<tf.Variable 'Variable:0' shape=(1, 28, 28, 1) dtype=float32_ref>"] and loss Tensor("Neg:0", shape=(), dtype=float32)
when I try to visualize a channel using
vis = render.render_vis(nasnet, objectives.channel("conv2/weights", 2), param_f).

You mentioned that this can be caused by the placeholder being disconnected from the graph. However, I did not modify the create_input function and if you look at the graph https://drive.google.com/file/d/1_6e3YhDzwlYNPScImDGZeAXBTmgANb5V/view?usp=sharing
the placeholder called "Variable" (lower left) seems to be connected to the loss named "Neg" (lower right) through node (weights) which comes from conv2/weights. Can you suggest a fix for this please? Thank you!

Modelzoo metadata that I used:

class Net(Model):
    model_path = 'pre_trained_model/frozen_model.pb'
    image_shape = [28, 28, 1]
    image_value_range = (0, 1)
    input_name = 'Reshape'

[ludwigschubert]

@namnov you use input_name = 'Reshape', which seems to result in that disconnected Placeholder -> Reshape sub-graph within "import". input_name should reference the place you feed in images/want to get images out of. I can't know if the lower half of your graph is "just" preprocessing… if it isn't, than "Variable" seems like the input_name you want.

Please let me know if this helped, I may have a follow up question.