Skip to content

ONNX inference support in predict() and better export naming #905

@farukalamai

Description

@farukalamai

Search before asking

  • I have searched the RF-DETR issues and found no similar feature requests.

Description

Two improvements to the ONNX workflow:

1. Export naming — Currently model.export() always saves as inference_model.onnx regardless of variant. Exporting RFDETRNano and RFDETRMedium to the same directory overwrites the file. The filename should include the variant name (e.g. rfdetr-nano.onnx, rfdetr-medium.onnx).

2. ONNX predict — After exporting a model to ONNX, there's no built-in way to run inference with it. Users have to handle preprocessing, ONNX Runtime session management, and postprocessing on their own. predict() should work with ONNX files directly, returning the same sv.Detections output as the PyTorch path.

Proposed workflow:

# Export — filename includes variant
model = RFDETRMedium()
model.export(output_dir="output")  # → output/rfdetr-medium.onnx

# Inference with ONNX — same API as PyTorch
model = RFDETRMedium(weights="output/rfdetr-medium.onnx")
detections = model.predict("image.jpg", threshold=0.5)

Use case

Anyone deploying RF-DETR with ONNX Runtime currently has to write their own inference code with duplicate preprocessing and postprocessing logic. This makes it harder to move from training to deployment.

With built-in ONNX predict:

  • Same predict() API works for both PyTorch and ONNX models
  • Returns sv.Detections regardless of backend
  • Enables lighter deployment without the full PyTorch dependency
  • Lays the groundwork for future backend support (TensorRT, OpenVINO)

Additional

No response

Are you willing to submit a PR?

  • Yes I'd like to help by submitting a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions