The camera's purpose is to detect the pink balls and the trash can. It can't do any complicated tasks like identifying objects and such. Doing it this way is simple. The camera captures frame after frame, for about 30 per second. And then analyze them using some complex algorithm, and then spit out/output those frames back with an outline of the object I want to detect.

The implementation was fairly straightforward, but there are some downsides to this approach. The first downside is that if the balls were of different colors, I would have to re-adjust the program, which can take a lot of time. And the next problem is that the program can't identify the objects. So, if any object has the same color as the ball, let's say a pink pencil, then the program would consider the pencil as a ball.

I added more code to the script. When the rover hits the walls or any obstacles that are in its way, it will try to rotate around and move to an empty area, and it will keep doing that over and over again until it sees the pink balls.

It was straightforward, just send a command to move forward, and it will move forward, send a command backward, and it will move backward. But the one problem I was having was there's a slight movement to the right or to the left when I tried to move forward. This is a problem because it would have to realign itself multiple times to be in position. So that it can lower the robotic arm to pick up the ball accurately. Otherwise, if it’s not in position, the arm could possibly push the ball aside or miss picking it up completely.

I needed the arm to make some basic motions, like picking up the ball, bring the arm back to position so that the rover can move around freely and doesn't obstruct or collide with any obstacles. And then drop the ball onto the trash bin.

I got the robotic arm's job to pick up objects, in this case, pink paper balls, and then drop them in the trash bin whenever appropriate. The motors can rotate at any angle, 180 degrees, 90 degrees, or whatever. The problem I was having was how do I attach this arm and the camera on top of the rover? So that they would not fall off or move out of position, which is very critical for when it tries to pick the ball up. So what I did was, used duck tape to tape both of them on top of the rover, and it worked. But if I command it to move really fast, the tape would not be able to hold it down.

I now needed to combine them all together into one. It was pretty time-consuming and setting up the environment too. Since the robot requires a lot of juice, I had to keep it plugged into the outlet. I tried using batteries, but they are strong enough to keep it alive, which limits the ability of the robot to move far away distances. And there are also attachment issues, like I mentioned before, I had to duct tape the arm and camera onto the rover. As a result, I had to config the rover and the arm in such a way that it moves as slowly as possible so that it doesn’t yank the wire out of the outlet. But on the good side, I can stop it in time if anything happens. There were occasions when the arm missed picking up the ball. So I had to stop it and adjust some values.