fix: cv2.error: OpenCV(4.13.0)

[intel-ohara]

Summary

When I tried to run a policy (SmolVLA), runtime failed with the error below.

File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/inference/preprocessors/smolvla.py", line 105, in _resize_with_pad
resized_hwc = cv2.resize(hwc, (resized_width, resized_height), interpolation=cv2.INTER_LINEAR)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cv2.error: OpenCV(4.13.0) /io/opencv/modules/imgproc/src/resize.cpp:3845: error: (-215:Assertion failed) !dsize.empty() in function 'resize'

In smolvla.py, images are expected be (batch_size, channel, height, width), but it looks (batch_size, height, width, channel) in my environment.

Steps to reproduce

1. create uv venv environment
2. install physicalai in the env
3. run python test_inference.py

# test_inference.py
from physicalai.capture import UVCCamera
from physicalai.capture import RealSenseCamera
from physicalai.inference import InferenceModel
from physicalai.runtime import PolicyRuntime, SyncExecution
from physicalai.robot import SO101

#policy = InferenceModel("./SmolVLA/exports/openvino", device="GPU")
policy = InferenceModel("./SmolVLA/exports/openvino")

cameras = {
    "bird view": UVCCamera(device="/dev/video0", width=960, height=720, fps=30), # v4l2-ctl --list-devices
    "side view": RealSenseCamera(serial_number="013222072639", width=640, height=480, fps=30),
}

robot = SO101(port="/dev/ttyACM1", calibration="./Bob.json", role="follower")
#robot.connect()

runtime = PolicyRuntime(
    fps=30,
    robot=robot,
    model=policy,
    cameras=cameras,
    execution=SyncExecution(),
)

with runtime:
    runtime.run(duration_s=60)

Logs and traceback

Traceback (most recent call last):
  File "/home/intel/openvino-physical-ai/test_inference.py", line 27, in <module>
    runtime.run(duration_s=60)
  File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/runtime/runtime.py", line 313, in run
    self._warmup_with_retry()
  File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/runtime/runtime.py", line 586, in _warmup_with_retry
    self._execution.warmup(sample_obs)
  File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/runtime/execution.py", line 107, in warmup
    actions = self._model.predict_action_chunk(sample_observation)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/inference/model.py", line 253, in predict_action_chunk
    outputs = self(observation)
              ^^^^^^^^^^^^^^^^^
  File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/inference/model.py", line 203, in __call__
    inputs = preprocessor(inputs)
             ^^^^^^^^^^^^^^^^^^^^
  File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/inference/preprocessors/smolvla.py", line 69, in __call__
    resized_img = self._resize_with_pad(img, *self.image_resolution, pad_value=0)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/inference/preprocessors/smolvla.py", line 105, in _resize_with_pad
    resized_hwc = cv2.resize(hwc, (resized_width, resized_height), interpolation=cv2.INTER_LINEAR)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cv2.error: OpenCV(4.13.0) /io/opencv/modules/imgproc/src/resize.cpp:3845: error: (-215:Assertion failed) !dsize.empty() in function 'resize'

Environment

• OS: Ubuntu 24.04.4
• Python: 3.12.3
• physicalai: 0.1.1

Additional context

No response

[intel-ohara]

Move forward with modifying two lines anyway..

In smolvla.py
L89

# cur_height, cur_width = img.shape[2:]
cur_height, cur_width = img.shape[1:3]

L99

# hwc = np.transpose(img[i], (1, 2, 0))
hwc = img[i]

[intel-ohara]

When I try to run ACT model, I experience the same error in resize.py.