Skip to content

Documentation: PolicyRuntime in Runtime API Reference #176

Description

@intel-ohara

The document doesn't mention, but 'task' seems necessary.

When I run the script,

from physicalai.capture import UVCCamera
from physicalai.capture import RealSenseCamera
from physicalai.inference import InferenceModel
from physicalai.runtime import PolicyRuntime, SyncExecution
from physicalai.robot import SO101

#policy = InferenceModel("./SmolVLA/exports/openvino", device="GPU")
policy = InferenceModel("./SmolVLA/exports/openvino")

cameras = {
#    "bird view": UVCCamera(device="/dev/video32", width=960, height=720, fps=30), # v4l2-ctl --list-devices
    "bird view": UVCCamera(device="/dev/video40", width=1280, height=720, fps=30), # v4l2-ctl --list-devices
#    "side view": RealSenseCamera(serial_number="013222072639", width=640, height=480, fps=30),
}

robot = SO101(port="/dev/ttyACM1", calibration="./Bob.json", role="follower")
#robot.connect()

runtime = PolicyRuntime(
    fps=30,
    robot=robot,
    model=policy,
    cameras=cameras,
    execution=SyncExecution(),
#    task='pick pen and put it on cup',
)

with runtime:
    runtime.run(duration_s=60)

I got the error below.

Traceback (most recent call last):
File "/home/intel/openvino-physical-ai/test_inference.py", line 29, in
runtime.run(duration_s=60)
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/runtime/runtime.py", line 313, in run
self._warmup_with_retry()
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/runtime/runtime.py", line 590, in _warmup_with_retry
self._execution.warmup(sample_obs)
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/runtime/execution.py", line 107, in warmup
actions = self._model.predict_action_chunk(sample_observation)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/inference/model.py", line 253, in predict_action_chunk
outputs = self(observation)
^^^^^^^^^^^^^^^^^
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/inference/model.py", line 203, in call
inputs = preprocessor(inputs)
^^^^^^^^^^^^^^^^^^^^
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/inference/preprocessors/new_line.py", line 33, in call
batch_tasks = inputs[TASK]
~~~~~~^^^^^^

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions