Blogg

During Security Fest 2018 we decided to build a competition the only way we know how: by over-engineering it! We started by buying a ball maze. We also recently saw that 8Bitdo had released their new SN30 controller and we wanted an excuse to buy one … maybe we can control the ball maze with it? So of course, we had to auto-maze the game and play it using a controller!

Ball mazeing for dummies

Controlling

We broke out the 3D-printer, modified some stepper stands (creative commons, made by adamyelland download drawing), bought some servos (Luxorparts S3003) and got to soldering! We attached the whole thing to a Raspberry Pi 3 which in turn steered the servos and listened to the NS30 over Bluetooth (using python, following this guide).

At first, we wanted to use the gyro function to steer how much the servos should move but for some reason we never got it to work…

So, we moved on to using the sticks. Left stick for up/down, right one for left/right. It was not the most natural way to steer so the first couple of times playing most struggled slightly, just as intended! The last piece of our over-engineering was how to time each race from start to either finish or failure.

Detecting the ball and measure time and length of the game

We opted for a webcam that kept track of the ball as it moved through the maze, giving points for each zone cleared and ending the game when it lost sight of the ball. Recognizing objects using computer vision is a somewhat mystified process. Similar to how a person who is not a programmer perceives how applications are made, even a programmer may find that they don’t have a clue on what actually happens to “recognize” things using a camera and some code.

One way of doing video/picture analysis is using machine learning, for example Googles TensorFlow. You take a picture (or video stream), feed it to a “magical tool”, and out comes data saying what it is, and perhaps also where it is on the picture.

We went for a more special purpose solution using OpenCV.

OpenCV is an open source computer vision library, written mostly in C++, it effectively allows you, as a developer, to modify an image as much as it lets you measure things. The steps we took was:

• Modify: Capturing the source picture and shrinking it to a smaller resolution.
• Modify: Removing all the Blue and Green in a picture, keeping only the Redness.
• Modify: Replacing a range of redness with a single, solid, red color.
• Measure: Get all matching pixels where neighboring pixels are also red.
• Measure: Finding the biggest source of red by applying regular programming arithmetic.

While the example above is intentionally convoluted, the reality is that this is how we detected the ball in the maze. :-)