Attaching a Camera to a Robotic Arm and Checking the Video Feed (YOLO)
管理人Introduction
A while back, I tried attaching a camera to the robotic arm I purchased on Amazon and set up for operation. The goal was to check the field of view and visibility while using YOLO for real-time object detection. I felt that the power supply capacity or torque was slightly insufficient.
▼I am using the robotic arm from this article. It seems the price has gone up since I bought it.
Shopping: Trying Out a Robotic Arm (5-axis and Gripper)
Info This article is translated from Japanese to English. Introduction In this post, I tried out a robotic arm I bought on Amazon. My original goal was to get …
趣味的ロボット研究所 
ポチップ
▼Previous articles are here:
Node-REDを使ってみる その４（MQTT通信、ロボットアーム）
はじめに 今回は以前Amazonで購入したロボットアームを、Node-REDでMQTT通信を利用して制御できるようにしてみました。 これまではボタンで制御していたのですが、ネ…
Trying Out Unreal Engine 5 Part 6 (Robotic Arm and Blueprints)
Info This article is translated from Japanese to English. Introduction In this post, I tried creating something like a robotic arm in Unreal Engine 5 (UE5). It…
Extending the Cables
As of my previous article, the cables were too short and getting caught, so I extended them using QI connectors and wire.
▼I used a large set of QI connectors purchased from Amazon.
ポチップ
Since I removed the end effector to attach the camera this time, only four servo motors are used. However, I prepared extension cables for all six original servos.
Since a 3x6-pin QI connector housing wasn't available, I made a 2x6-pin for power and a 1x6-pin for signals.
▼If bundled properly, it should be quite compact. The signal wires could probably be even thinner.
Now I can move the arm without worrying about cable length.
Trying it Out
I checked the video feed from the "hand-eye" camera. I connected the following USB camera to my PC for verification.
▼I use this USB camera.
ポチップ
▼Here is the video feed from the camera:
I realized that the field of view was narrower than expected, and the shaking of the arm significantly affected the video. Occasionally, the servos would lose power and the arm would suddenly collapse, likely due to insufficient power supply capacity or lack of torque. I'll need to address these issues.
I also tried object detection using YOLO.
▼The settings for GPU processing are covered in the following article:
Object Detection with YOLO Part 4 (GPU Setup, CUDA 12.6)
Info This article is translated from Japanese to English. Introduction In this post, I tried object detection with YOLO using a GPU.According to the Ultralytic…
▼Detection is working using a pre-trained model.
▼There are some false positives, but it is detecting various objects.
The video was edited later to show both the bird's-eye view and the hand-eye camera, so there is a slight time lag in the display. However, looking at the bird's-eye view, you can see that detection is happening almost in real-time.
Objects that haven't been trained cannot be detected, so depending on the target, I would need to create a custom dataset.
▼I usually use CVAT for this.
Annotation with CVAT (YOLO v8, Object Detection)
Info This article is translated from Japanese to English. Introduction In this post, I tried performing annotation using CVAT (Computer Vision Annotation Tool)…
Finally
This time, I simply tested the camera feed and object detection. I need to think about how to actually control the arm based on these detections. For example, if I wanted to track an object, I would need to control the arm so that the target stays in the center of the screen. Since the video blurs and shakes while the arm is moving, that will likely impact the detection accuracy as well.
Since then, I have designed and built a new 5-axis robotic arm. I'm planning to equip it with a camera and enable remote operation.
▼This is the new arm. I will introduce it in another article.
