load model步骤缺少模型

[zuichu35]

load models

checkpoint_path = "LinWeizheDragon/PreFLMR_ViT-G"
image_processor_name = "laion/CLIP-ViT-bigG-14-laion2B-39B-b160k"

大模型刚入门，根据您的方法复现，并未提到这两个从哪里来

[zuichu35]

还有一个问题是，如果需要复现结果是需要下载所有IMAGE吗 ，感觉加起来有1T+了

[LinWeizheDragon]

模型会自动从huggingface 下载，连不上的话用hf mirrors镜像网站替换就可以

[LinWeizheDragon]

还有一个问题是，如果需要复现结果是需要下载所有IMAGE吗 ，感觉加起来有1T+了

复现结果肯定需要至少把测试集下载下来

[zuichu35]

模型会自动从huggingface 下载，连不上的话用hf mirrors镜像网站替换就可以
以下两部分分别为code和error
import os
os.environ['HF_ENDPOINT'] = 'https://hf-mirror.com'
import torch
import pandas as pd
import numpy as np
from torchvision.transforms import ToPILImage
from transformers import AutoImageProcessor

from flmr import index_custom_collection
from flmr import FLMRQueryEncoderTokenizer, FLMRContextEncoderTokenizer, FLMRModelForRetrieval

load models

checkpoint_path = "LinWeizheDragon/PreFLMR_ViT-G"
image_processor_name = "laion/CLIP-ViT-bigG-14-laion2B-39B-b160k"

query_tokenizer = FLMRQueryEncoderTokenizer.from_pretrained(checkpoint_path, subfolder="query_tokenizer")
context_tokenizer = FLMRContextEncoderTokenizer.from_pretrained(
checkpoint_path, subfolder="context_tokenizer"
)

model = FLMRModelForRetrieval.from_pretrained(
checkpoint_path,
query_tokenizer=query_tokenizer,
context_tokenizer=context_tokenizer,
)
image_processor = AutoImageProcessor.from_pretrained(image_processor_name)

——————————————————————————————————————————

AttributeError Traceback (most recent call last)
Cell In[1], line 18
15 checkpoint_path = "LinWeizheDragon/PreFLMR_ViT-G"
16 image_processor_name = "laion/CLIP-ViT-bigG-14-laion2B-39B-b160k"
---> 18 query_tokenizer = FLMRQueryEncoderTokenizer.from_pretrained(checkpoint_path, subfolder="query_tokenizer")
19 context_tokenizer = FLMRContextEncoderTokenizer.from_pretrained(
20 checkpoint_path, subfolder="context_tokenizer"
21 )
23 model = FLMRModelForRetrieval.from_pretrained(
24 checkpoint_path,
25 query_tokenizer=query_tokenizer,
26 context_tokenizer=context_tokenizer,
27 )

File /mnt/data/wangli/FLMR/FLMR/flmr/models/flmr/tokenization_flmr.py:429, in FLMRQueryEncoderTokenizer.from_pretrained(cls, text_config, *args, **kwargs)
422 @classmethod
423 def from_pretrained(
424 cls,
(...)
427 **kwargs,
428 ):
--> 429 if text_config.text_encoder_base_model == "bert-base-uncased":
430 return FLMRBertQueryEncoderTokenizer.from_pretrained(*args, **kwargs)
431 else:

AttributeError: 'NoneType' object has no attribute 'text_encoder_base_model'

[zuichu35]

OH我去看了你的源码，发现你用的都是用的类似于https://huggingface.co/LinWeizheDragon/PreFLMR_ViT-L的链接

[zuichu35]

ImportError: cannot import name 'AdamW' from 'transformers' (c:\Users\zuichu.conda\envs\FLMR\lib\site-packages\transformers_init_.py)
还有就是可能需要指定以下transformer的版本

[zuichu35]

模型会自动从huggingface 下载，连不上的话用hf mirrors镜像网站替换就可以
以下两部分分别为code和error
import os
os.environ['HF_ENDPOINT'] = 'https://hf-mirror.com'
import torch
import pandas as pd
import numpy as np
from torchvision.transforms import ToPILImage
from transformers import AutoImageProcessor

from flmr import index_custom_collection from flmr import FLMRQueryEncoderTokenizer, FLMRContextEncoderTokenizer, FLMRModelForRetrieval

load models

checkpoint_path = "LinWeizheDragon/PreFLMR_ViT-G" image_processor_name = "laion/CLIP-ViT-bigG-14-laion2B-39B-b160k"

query_tokenizer = FLMRQueryEncoderTokenizer.from_pretrained(checkpoint_path, subfolder="query_tokenizer") context_tokenizer = FLMRContextEncoderTokenizer.from_pretrained( checkpoint_path, subfolder="context_tokenizer" )

model = FLMRModelForRetrieval.from_pretrained( checkpoint_path, query_tokenizer=query_tokenizer, context_tokenizer=context_tokenizer, ) image_processor = AutoImageProcessor.from_pretrained(image_processor_name)

——————————————————————————————————————————

AttributeError Traceback (most recent call last) Cell In[1], line 18 15 checkpoint_path = "LinWeizheDragon/PreFLMR_ViT-G" 16 image_processor_name = "laion/CLIP-ViT-bigG-14-laion2B-39B-b160k" ---> 18 query_tokenizer = FLMRQueryEncoderTokenizer.from_pretrained(checkpoint_path, subfolder="query_tokenizer") 19 context_tokenizer = FLMRContextEncoderTokenizer.from_pretrained( 20 checkpoint_path, subfolder="context_tokenizer" 21 ) 23 model = FLMRModelForRetrieval.from_pretrained( 24 checkpoint_path, 25 query_tokenizer=query_tokenizer, 26 context_tokenizer=context_tokenizer, 27 )

File /mnt/data/wangli/FLMR/FLMR/flmr/models/flmr/tokenization_flmr.py:429, in FLMRQueryEncoderTokenizer.from_pretrained(cls, text_config, *args, **kwargs) 422 @classmethod 423 def from_pretrained( 424 cls, (...) 427 **kwargs, 428 ): --> 429 if text_config.text_encoder_base_model == "bert-base-uncased": 430 return FLMRBertQueryEncoderTokenizer.from_pretrained(*args, **kwargs) 431 else:

AttributeError: 'NoneType' object has no attribute 'text_encoder_base_model'

这个问题一直解决不了，换了可以直连hf的电脑还是

tokenization_flmr.py:429, in FLMRQueryEncoderTokenizer.from_pretrained(cls, text_config, *args, **kwargs)
422 @classmethod
423 def from_pretrained(
424 cls,
(...)
427 **kwargs,
428 ):
--> 429 if text_config.text_encoder_base_model == "bert-base-uncased":
430 return FLMRBertQueryEncoderTokenizer.from_pretrained(*args, **kwargs)
431 else:

AttributeError: 'NoneType' object has no attribute 'text_encoder_base_model'

[JingbiaoMei]

模型会自动从huggingface 下载，连不上的话用hf mirrors镜像网站替换就可以
以下两部分分别为code和error
import os
os.environ['HF_ENDPOINT'] = 'https://hf-mirror.com'
import torch
import pandas as pd
import numpy as np
from torchvision.transforms import ToPILImage
from transformers import AutoImageProcessor

from flmr import index_custom_collection from flmr import FLMRQueryEncoderTokenizer, FLMRContextEncoderTokenizer, FLMRModelForRetrieval

load models

checkpoint_path = "LinWeizheDragon/PreFLMR_ViT-G" image_processor_name = "laion/CLIP-ViT-bigG-14-laion2B-39B-b160k"
query_tokenizer = FLMRQueryEncoderTokenizer.from_pretrained(checkpoint_path, subfolder="query_tokenizer") context_tokenizer = FLMRContextEncoderTokenizer.from_pretrained( checkpoint_path, subfolder="context_tokenizer" )
model = FLMRModelForRetrieval.from_pretrained( checkpoint_path, query_tokenizer=query_tokenizer, context_tokenizer=context_tokenizer, ) image_processor = AutoImageProcessor.from_pretrained(image_processor_name)

——————————————————————————————————————————

AttributeError Traceback (most recent call last) Cell In[1], line 18 15 checkpoint_path = "LinWeizheDragon/PreFLMR_ViT-G" 16 image_processor_name = "laion/CLIP-ViT-bigG-14-laion2B-39B-b160k" ---> 18 query_tokenizer = FLMRQueryEncoderTokenizer.from_pretrained(checkpoint_path, subfolder="query_tokenizer") 19 context_tokenizer = FLMRContextEncoderTokenizer.from_pretrained( 20 checkpoint_path, subfolder="context_tokenizer" 21 ) 23 model = FLMRModelForRetrieval.from_pretrained( 24 checkpoint_path, 25 query_tokenizer=query_tokenizer, 26 context_tokenizer=context_tokenizer, 27 )
File /mnt/data/wangli/FLMR/FLMR/flmr/models/flmr/tokenization_flmr.py:429, in FLMRQueryEncoderTokenizer.from_pretrained(cls, text_config, *args, **kwargs) 422 @classmethod 423 def from_pretrained( 424 cls, (...) 427 **kwargs, 428 ): --> 429 if text_config.text_encoder_base_model == "bert-base-uncased": 430 return FLMRBertQueryEncoderTokenizer.from_pretrained(*args, **kwargs) 431 else:
AttributeError: 'NoneType' object has no attribute 'text_encoder_base_model'

这个问题一直解决不了，换了可以直连hf的电脑还是

tokenization_flmr.py:429, in FLMRQueryEncoderTokenizer.from_pretrained(cls, text_config, *args, **kwargs) 422 @classmethod 423 def from_pretrained( 424 cls, (...) 427 **kwargs, 428 ): --> 429 if text_config.text_encoder_base_model == "bert-base-uncased": 430 return FLMRBertQueryEncoderTokenizer.from_pretrained(*args, **kwargs) 431 else:

AttributeError: 'NoneType' object has no attribute 'text_encoder_base_model'

The error is due to using the FLMR script for loading the PreFLMR model. For running the PreFLMR model, please refer to examples/example_use_preflmr.py

from flmr import (
    FLMRModelForRetrieval,
    FLMRQueryEncoderTokenizer,
    FLMRContextEncoderTokenizer,
    FLMRConfig,
)
ckpt = "LinWeizheDragon/PreFLMR_ViT-G"
processor = "laion/CLIP-ViT-bigG-14-laion2B-39B-b160k"

flmr_config = FLMRConfig.from_pretrained(ckpt)
query_tokenizer = FLMRQueryEncoderTokenizer.from_pretrained(ckpt,
                                                                text_config=flmr_config.text_config,
                                                                subfolder="query_tokenizer")
context_tokenizer = FLMRContextEncoderTokenizer.from_pretrained(ckpt,
                                                                text_config=flmr_config.text_config,
                                                                subfolder="context_tokenizer")

flmr_model = FLMRModelForRetrieval.from_pretrained(
    ckpt,
    query_tokenizer=query_tokenizer,
    context_tokenizer=context_tokenizer,
)
image_processor = AutoImageProcessor.from_pretrained(processor)

This should fix your error.

[zuichu35]

The error is due to using the FLMR script for loading the PreFLMR model. For running the PreFLMR model, please refer to examples/example_use_preflmr.py
Thank you for your help, but the step and code are totally following your tutorial named "Index a custom document collection
Load pre-trained models" #https://github.com/LinWeizheDragon/FLMR?tab=readme-ov-file#index-a-custom-document-collection

[JingbiaoMei]

We have recently updated the code to support FLMR-CN model, but we did not change the README file. We have now made changes to the README file to reflect this.