Welcome to GLOM-TensorFlow Discussions!

[Rishit-dagli]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[ParnianA]

Hi Rashit. Nice implementation of GLOM. I am wondering since the model outputs levels, and not not the target, how can one train the weights?

[Rishit-dagli]

Hi @ParnianA ,

Thanks!
Training GLOM is quite an interesting question. As I understand from the paper an obvious way to train this system might be to show it images that have missing patches; similar to what we do for BERT, show it sentences with missing words and BERT fills those up or maybe generalizing this how a lot of other language models work.

So, you would show it an image and at every location, it would be interacting with neighbor locations. After some time you would expect it to fill in parts of the image that were missing. While backpropagation you would want it to learn the hierarchical description of the objects.

You could also make this more effective and better at forming islands using contrastive learning (specifically one example with the benefits of using this the paper shares is about SimCLR). Your net would be making a prediction for the embedding at an adjacent level in the next layer. To train these nets you would average predictions with similar representation in nearby locations (specifically in the paper using tension weighted average). Then what you want the bottom up and top down net to try and agree and we try to make their individual predictions with the general consensus and other locations and we train these nets to agree. This would make it better at forming islands

[ParnianA]

Thank you Rishit for the explanation. Let's say I want to train GLOM for image denoising. I noticed that your model works with input images and not placeholders. How can you train it using the "fit" attribute?

[Rishit-dagli]

I think it would be pretty straightforward to write a custom training loop for image completion. However, I am not clear about using the contrastive learning approach, I also in one of Geoffery Hinton's talk (this timestamp) where he simply says the contrastive learning approach hasn't got working yet.