Cache only has 0 layers, attempted to access layer with index 0

[xqhejie]

I have tried various versions of transformers (4.36.0, 4.40.0, 4.47.0, etc.), but I still encounter errors.Your assistance is greatly needed, thank you.

12/17 15:12:24 - mmengine - INFO - before_train in EvaluateChatHook.
The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results.
get_max_cache() is deprecated for all Cache classes. Use get_max_cache_shape() instead. Calling get_max_cache() will raise error from v4.48
Traceback (most recent call last):
File "/autodl-fs/data/hjxtuner/xtuner/xtuner/tools/train.py", line 342, in
main()
File "/autodl-fs/data/hjxtuner/xtuner/xtuner/tools/train.py", line 338, in main
runner.train()
File "/root/miniconda3/envs/xtuner/lib/python3.10/site-packages/mmengine/runner/_flexible_runner.py", line 1200, in train
model = self.train_loop.run() # type: ignore
File "/root/miniconda3/envs/xtuner/lib/python3.10/site-packages/mmengine/runner/loops.py", line 270, in run
self.runner.call_hook('before_train')
File "/root/miniconda3/envs/xtuner/lib/python3.10/site-packages/mmengine/runner/_flexible_runner.py", line 1271, in call_hook
getattr(hook, fn_name)(self, **kwargs)
File "/autodl-fs/data/hjxtuner/xtuner/xtuner/engine/hooks/evaluate_chat_hook.py", line 221, in before_train
self._generate_samples(runner, max_new_tokens=50)
File "/autodl-fs/data/hjxtuner/xtuner/xtuner/engine/hooks/evaluate_chat_hook.py", line 210, in _generate_samples
self._eval_language(runner, model, device, max_new_tokens,
File "/autodl-fs/data/hjxtuner/xtuner/xtuner/engine/hooks/evaluate_chat_hook.py", line 175, in _eval_language
generation_output = model.generate(
File "/root/miniconda3/envs/xtuner/lib/python3.10/site-packages/peft/peft_model.py", line 1838, in generate
outputs = self.base_model.generate(*args, **kwargs)
File "/root/miniconda3/envs/xtuner/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/root/miniconda3/envs/xtuner/lib/python3.10/site-packages/transformers/generation/utils.py", line 2252, in generate
result = self._sample(
File "/root/miniconda3/envs/xtuner/lib/python3.10/site-packages/transformers/generation/utils.py", line 3251, in _sample
outputs = self(**model_inputs, return_dict=True)
File "/root/miniconda3/envs/xtuner/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/miniconda3/envs/xtuner/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1568, in _call_impl
result = forward_call(*args, **kwargs)
File "/root/.cache/huggingface/modules/transformers_modules/internlm2-chat-1_8b/modeling_internlm2.py", line 1212, in forward
outputs = self.model(
File "/root/miniconda3/envs/xtuner/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/miniconda3/envs/xtuner/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1568, in _call_impl
result = forward_call(*args, **kwargs)
File "/root/.cache/huggingface/modules/transformers_modules/internlm2-chat-1_8b/modeling_internlm2.py", line 1008, in forward
layer_outputs = decoder_layer(
File "/root/miniconda3/envs/xtuner/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/miniconda3/envs/xtuner/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1568, in _call_impl
result = forward_call(*args, **kwargs)
File "/root/.cache/huggingface/modules/transformers_modules/internlm2-chat-1_8b/modeling_internlm2.py", line 742, in forward
hidden_states, self_attn_weights, present_key_value = self.attention(
File "/root/miniconda3/envs/xtuner/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/miniconda3/envs/xtuner/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1568, in _call_impl
result = forward_call(*args, **kwargs)
File "/autodl-fs/data/hjxtuner/xtuner/xtuner/model/modules/dispatch/internlm2.py", line 222, in internlm2_attn_forward
kv_seq_len += past_key_value[0].shape[-2]
File "/root/miniconda3/envs/xtuner/lib/python3.10/site-packages/transformers/cache_utils.py", line 390, in getitem
raise KeyError(f"Cache only has {len(self)} layers, attempted to access layer with index {layer_idx}")
KeyError: 'Cache only has 0 layers, attempted to access layer with index 0'