Live stream of OpenArm using my own lib XoQ :)

[haixuanTao]

Made a quick robot virtualization library called XoQ making it possible to connect the OpenArm from any OS including macOS, but also the Web ( for listening mainly writing is a bit slow )

The livestream is available here: https://huggingface.co/spaces/haixuantao/openarm

The robot is probably not moving and it's still a test but the idea is that in the future we can have live demo of policy and remote teleoperation and data collection.

To get started:

Run the server on the linux machine:

git clone https://github.com/haixuanTao/xoq
cd xoq
cargo run --release --bin can-server --features "iroh,can" -- can0:fd  can1:fd  can2:fd  can3:fd --moq-relay https://cdn.1ms.ai

and then run lerobot with:

pip install xoq-can
lerobot-teleoperate \
    --robot.type=openarm_follower \
    --robot.port=<PUBLIC_KEY_FOLLOWER> \
    --robot.side=right \
    --robot.id=my_follower \
    --teleop.type=openarm_leader \
    --teleop.port=<PUBLIC_KEY_LEADER>

And then optionally I have a realsense server running at the same time

live3.mp4

hotdog.mp4

[kou]

Wow! Thanks for sharing it!

This uses the following architecture, right?

flowchart TD
    macOS-->Linux
    Linux-->leader["OpenArm (leader)"]
    Linux-->follower["OpenArm (follower)"]
    leader["OpenArm (leader)"]-->Linux
    follower["OpenArm (follower)"]-->Linux
    Linux-->macOS
    Linux-->hf["Hugging Face"]
    hf["Hugging Face"]-->Linux

Loading

[haixuanTao]

There's no server on Huggingface.

Everything is a web app.

But the connection can be native as well as peer to peer.

[haixuanTao]

Sorry I had to stop the livestream because we had a small leak in my level not even joking. Going to try to get it back up today!

[kou]

Thanks for your correction! BTW, where does Relay work? On macOS? On Linux? Or other host (cdn.1ms.ai)?

[haixuanTao]

So this is purely for web visualization and it's on a geodns cloud.

[kou]

I see!