08 PyTorch Paper Replicating - question about the visualization of flattened feature map

[dzy212]

Hi,

In section 4.4 Flattening the patch embedding with torch.nn.Flatten()

The flattened feature map that was visualized was created by indexing on the embedding dimension:
single_flattened_feature_map = image_out_of_conv_flattened_reshaped[ : , : , 0]

Which results in the shape: (1, 196)

I thought that a single feature map should have had a dimension of (1, 768), as each patch is flattened to P x P x C, and we have 196 of them.

Is this correct? Is what is being visualized a single pixel across all the patches?
Many thanks

[mrdbourke]

Hi @dzy212,

Each patch is embedded into an embedding size of 768.

The actual patches of the image start as 14x14 but are then flattened to 196 -> embedded to 768.

My interpretation is the feature map they talk about is the features from the image patches (rather than the embedding itself).

I'm getting this from green box below, which is taken from the paper: