Random thoughts on modelzoo

[colah]

• I'd really like to have a convenient way to evaluate a model once without going through the whole rigamarole of setting up a graph.

  • Maybe model.isolated_eval(layer, input).
  • Use cases to consider -- ask Chris for example notebooks
  • Most common scenario: want to get activations for a single thing.
  • Second most common scenario: we want to get a gradient. Not sure what a super convenient framework for that would look like. Think about attribution workloads here.

• We need to get model.labels back. Done!

• It might make sense to attach expensive, precomputed assets to models. For example feature visualization spritemaps.

  • Maybe model.precomputed.vis_spritemaps["mixed4d"] gives me a url?

[ludwigschubert]

• We need model.interesting_tensors (name tbd) if we want to automatically create visualizations

[tul-urte]

In "modelzoo/slim_models", the various models each provide a hard-coded list of layers.

E.g. MobilenetV2.py

MobilenetV2_10_slim.layers = _layers_from_list_of_dicts(MobilenetV2_10_slim, [
...
  {'tags': ['conv'], 'name': 'MobilenetV2/expanded_conv_15/add', 'depth': 160},
  {'tags': ['dense'], 'name': 'MobilenetV2/Predictions/Softmax', 'depth': 1001},
]

Is it possible to explain the reason these layers were selected?
Is it intending to be a list of the only layers in the graph worth visualizing?

Currently, I list all nodes in a graph (that I train myself) as text and then construct my own selections of layers to visualize. But this is tedious, and, of course, some nodes are inappropriate.

So I would appreciate a function that, given a graph node, generates a corresponding layer object (e.g.: as above) for every node, so I can filter a list of them for visualization. This is an admission of my own ignorance because I don't know how to get the depth from a node (nor tags, ['dense'] ?).

In general, I'm interested what makes a node "good" for visualization, and what makes one "bad".

[colah]

That's a great question, @tul-urte!

The present list is created by a simple heuristic I wrote to find "interesting" nodes. It isn't perfect, but hopefully makes it easier for people to run lots of visualizations across models without thinking too much. I think it's basically looking for ReLu nodes, concat nodes, and add nodes (for resnets) and maybe trimming down some nodes in the resnet case.

There's some better (but very messy) code in lucid/scratch/pretty_graph/ that we're planning to clean up and reuse.