Control robot that move according to music beats?

[corey]

Hello Guys,

we are working on a project with robot playing a groove box and a keyboard for so music. Kinda like DJ adding layers to music. We are having trouble asking the robot to follow the beats. (Kinda like Marc Rebillet's funky music). For example we want robot move from A to B with V40, it's too slow, but we make it move with V50 it's too quick.

May I ask if anybody have experiences with this or point me to some direction with coding?

We are using an ABB 6700 robot with IRC5

Thank you!!!

[Johannes @ Robots in Architecture]

Hello,

We did a surprising amount of musical projects, like this one (https://www.youtube.com/watch?v=he5iY_X-E5U—the video is out of sync often, but it played nicely and in real-time), and also performative projects (e.g., the one Karl posted at https://www.youtube.com/watch?v=C0kUZhFH4hc).

But unfortunately, there is no easy solution; for the e-guitar project, we developed a kind of custom notation system that allowed a non-programmer to time and fine-tune the sound in Grasshopper (followed by a lot of trial and error to get the timing right, but GH nicely supported such an iterative process). On the KUKA side there is a system for time-based programming called ready2animate, but it also has got its limitations and won't help you on an ABB robot.

For that performance (https://creativerobotics.at/underbody/), we used RSI (hard-realtime control on KUKA robots, i.e. one position every 4ms) to map the movement in real-time to the robots. I assume that ABB has a similar system. But hard-realtime can be a pain to deal with.

Hopefully, these pointers will also be useful for an ABB-based application.

Best,
Johannes

[corey]

Thank you for the prompt reply, Johanne! Helpful as always! I see. It's not an easy solution. I will just try to use the groove box without management of beats. and use the beat time for adjusting as simple rythm then!
Here is a rough draft. haha. Thanks again!

[Johannes @ Robots in Architecture]

Nice! For that project, we went the other way and distributed the beat between four robots: https://creativerobotics.at/ars-electronica-festival-2020/
But yes, if it's an individual installation, it's probably easier to simply finetune the speed, etc., manually than to come up with a fully integrated solution.

Best,
Johannes

[corey]

That looks extremely interesting! Wish i had more robots! I like your another project in ars electronica with the robot mimicking human motion! Anyway thank you Johannes again for your helpful insight.