Infrared Methods for Guitar ROI Detection
Our previous attempt to segment the guitar strings area (via edge methods) was not that successful, especialy due to computational cost and pipeline complexity. Here we report a better strategy, this time facilitating the process by modifying somehow the appearance of both the instrument and the musician hands.
It is worth mentioning, again, that most of the algorithms we will mention are implemented at the OpenCV Computer Vision library.
Let us speak about the hardware. We are using an infrared camera to capture the scene, which is properly illuminated with infrared light. The following picture shows the camera surrounded by 4 infrared light sources.
Special markers are attached to the guitar in order to easily locate the instrument. Such markers are made with a material which has good reflexive properties.
For the fingers, special reflexive gloves dress the middle phalanges.
So, after a thresholding operation, this is what we get:
Using the contour detection algorithm and contour data structure provided by OpenCV, guitar and finger markers can be separated
With 4 known points in the scene, and with the knowledge about its relations with the guitar geometry, a perspective transform can be applied in order put the guitar where we want in the image and identify the Region of Interest.
The presented pipeline is very simple and robust, provided the guitar doesn't move fast. A small problem is that the perspective transform is an expensive operation, so the image size we are using to get “real-time” feedback is of 320x240 px.