Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KeyError: 'loss_cls' #9

Open
Feeeeeel opened this issue Nov 17, 2024 · 5 comments
Open

KeyError: 'loss_cls' #9

Feeeeeel opened this issue Nov 17, 2024 · 5 comments

Comments

@Feeeeeel
Copy link

when i run the command：python tools/train.py configs/visdrone_step1_base.py
it got an error：

11/17 14:19:53 - mmengine - INFO - Iter(train) [ 450/80000] lr: 8.9990e-03 eta: 17:22:39 time: 0.7945 data_time: 0.0204 memory: 17784 loss: 0.9282 sup_loss_rpn_cls: 0.2866 sup_loss_rpn_bbox: 0.0894 sup_loss_cls: 0.3180 sup_acc: 95.3125 sup_loss_bbox: 0.1006 unsup_loss_rpn_cls: 0.0831 unsup_loss_rpn_bbox: 0.0000 unsup_loss_cls: 0.0505 unsup_acc: 100.0000 unsup_loss_bbox: 0.0000
File "tools/train.py", line 122, in
main()
File "tools/train.py", line 118, in main
runner.train()
File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1777, in train
model = self.train_loop.run() # type: ignore
File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmengine/runner/loops.py", line 289, in run
self.run_iter(data_batch)
File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmengine/runner/loops.py", line 313, in run_iter
outputs = self.runner.model.train_step(
File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 114, in train_step
losses = self._run_forward(data, mode='loss') # type: ignore
File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 361, in _run_forward
results = self(**data, mode=mode)
File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmdet/models/detectors/base.py", line 92, in forward
return self.loss(inputs, data_samples)
File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmdet/models/detectors/semi_base.py", line 87, in loss
losses.update(**self.loss_by_pseudo_instances(
File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmdet/models/detectors/soft_teacher.py", line 78, in loss_by_pseudo_instances
losses.update(**self.rcnn_cls_loss_by_pseudo_instances(
File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmdet/models/detectors/soft_teacher.py", line 255, in rcnn_cls_loss_by_pseudo_instances
losses['loss_cls'] = losses['loss_cls'] * len(
KeyError: 'loss_cls'

Can you solve this problem？Thanks！
@UnchangedOriInten
Copy link

You can modify the learning rate to make it smaller, but it will affect the detection accuracy later on. It is recommended to continue training with the original configuration.

@lili196
Copy link

lili196 commented Jan 15, 2025

I meet the same error,do you solve it?I want to konw how to solve it.Thanks

@lizzy8587
Copy link
Collaborator

lizzy8587 commented Feb 8, 2025 •
edited
Loading

when i run the command：python tools/train.py configs/visdrone_step1_base.py it got an error：

11/17 14:19:53 - mmengine - INFO - Iter(train) [ 450/80000] lr: 8.9990e-03 eta: 17:22:39 time: 0.7945 data_time: 0.0204 memory: 17784 loss: 0.9282 sup_loss_rpn_cls: 0.2866 sup_loss_rpn_bbox: 0.0894 sup_loss_cls: 0.3180 sup_acc: 95.3125 sup_loss_bbox: 0.1006 unsup_loss_rpn_cls: 0.0831 unsup_loss_rpn_bbox: 0.0000 unsup_loss_cls: 0.0505 unsup_acc: 100.0000 unsup_loss_bbox: 0.0000 File "tools/train.py", line 122, in main() File "tools/train.py", line 118, in main runner.train() File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1777, in train model = self.train_loop.run() # type: ignore File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmengine/runner/loops.py", line 289, in run self.run_iter(data_batch) File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmengine/runner/loops.py", line 313, in run_iter outputs = self.runner.model.train_step( File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 114, in train_step losses = self._run_forward(data, mode='loss') # type: ignore File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 361, in _run_forward results = self(**data, mode=mode) File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmdet/models/detectors/base.py", line 92, in forward return self.loss(inputs, data_samples) File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmdet/models/detectors/semi_base.py", line 87, in loss losses.update(**self.loss_by_pseudo_instances( File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmdet/models/detectors/soft_teacher.py", line 78, in loss_by_pseudo_instances losses.update(**self.rcnn_cls_loss_by_pseudo_instances( File "/home/zwh/.conda/envs/castdet/lib/python3.8/site-packages/mmdet/models/detectors/soft_teacher.py", line 255, in rcnn_cls_loss_by_pseudo_instances losses['loss_cls'] = losses['loss_cls'] * len( KeyError: 'loss_cls'

Can you solve this problem？Thanks！

  • You can try slightly reducing the learning rate or increasing the batch size.
  • Alternatively, the rcnn_cls_loss_by_pseudo_instances function has been rewritten in castdet/castdet.py. You can follow this function to rewrite it in mmdet/models/detectors/soft_teacher.py.

@lili196
Copy link

lili196 commented Feb 8, 2025 via email

@Andrew0425
Copy link

after i modify the $rcnn_cls_loss_by_pseudo_instances$ function in the $soft_teacher.py$, it goes on training while the loss and accuracy goes gradually to nan/100

how can i solve this problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@Feeeeeel @lizzy8587 @Andrew0425 @UnchangedOriInten @lili196