Patrick Bader's Blog a blog about software development


Master Thesis Presentation [2nd Update]

This time a posting in German since the presentation will be held in German:
Am Dienstag den 14.09.2010 werde ich eine Präsentation über meine Master Thesis mit dem Titel: "GPU-unterstützte Bildverarbeitung und Bilderkennung
im Kontext einer Multi-Touch-Anwendung" halten.
Der Vortrag findet an der Hochschule der Medien in Stuttgart statt.
Beginn ist 10:30 Uhr und der Vortrag wird ca. 45 Minuten dauern.
[Update]Das ganze findet in Raum 135 statt.[/Update]
Interessierte sind herzlich eingeladen teilzunehmen.

[2. Update]Der Vortrag ist als Videostream hier abrufbar.[/2. Update]


it’s about time…

I finally finished my Master's Thesis about GPU based image processing in the context of a multi-touch application. As I wrote in older posts I have built a prototype of an LCD based multi-touch screen. The PlayStation Eye camera is used to track blobs and fiducials using IR light and the images from the cam are processed on the GPU to extract positions, IDs, etc. For the implementation which will possibly be published in a later post, I used OpenCL, which allows programming on various heterogenous hardware. The visualization is done using solely OpenGL 3.2 Core Profile.

The thesis in in german and can be downloaded here. Feel free to read and comment it.


Blob Detection Reloaded

Today, I finally got the 50 Osram SFH 4350 LEDs I ordered. These shine at 850 nm, a wavelength at which the PS3 Eye is quite sensitive. The first thing to do with these LEDs was to test them somehow. Since I haven't got an IR bandpass filter for the cam (I ordered one from ebay), I had to test them without the filter.

So how did I run the test? A few days ago I ordered a sample of some Plexiglass EndLighten, which is available here. It's a special glass that shines diffuse when light shines from the side into the sheet. The final setup was simple: I connected 8 LEDs in row with a 1 ohm resistor and powered the setting with 12 Volts, the LEDs were attached to the Plexiglass and and everything was filmed with the cam.

A Screenshot with blobs detected on my hands is below, the other blobs in the surface come from a screw clamp.


stay tune for some more news in the near future 😉


Upgrading PS3 Eye Camera and Blob Detection

As noted in my last posting, the images of the modified Eye camera were quite blurry. That was the case because the IR filter was missing and so the camera got out of focus. I was asking myself for the reason, since the filter is not a lense at all. Glass has a different refraction index than air resulting in diagonal rays becoming offset. I tried to visualize the effect in the following picture:


More on that can be found on wikipedia.

I have finally managed to correct this issue. I had to find a replacement for the filter. It had to be of glass or a similar material with the same width as the filter. Quite tough to find a piece of glass or plastic with a depth of about a millimeter. Suprisingly the cap of a CD case did the job as a filter replacement. I first sawed a piece off and afterwards filed it into shape.

The room was quite dark but the results are  better than without the replacement, I think:

PS3 Eye Capture with IR filter replacement

PS3 Eye Capture with IR filter replacement

Now the second part of this posting, blob detection. Blob detection is about finding bright or dark spots in an image. I used some kind of global flood fill algorithm to find the blobs. Sadly I was not able to programm this part in the pixel shader so I implemented an algorithm on the CPU. On my laptop CPU power is fairly restricted so it took me a few attempts to do blob detection at 60 fps. The algorithm is a scanline algorithm which searches for horizontal lines of bright pixels. After the complete image has been scanned, one or more connected lines become a blob. The blob center is the barycenter of the pixels, at the moment without regard of their brightness. I have to test this algorithm for robustness, but can already give you a first impression:

A first test of blob detection. The red dots are centers of detected blobs.

A first test of blob detection. The red dots are centers of detected blobs.

Hope I could help anyone with the camera focus problem. Maybe I release the source code of the project when it is out of experimental stage, or to say it straight: When it's a bit less messy. 😉

Tagged as: No Comments

Edge detection with PS3 Eye cam

I recently bought a Playstation Eye camera because I would like to get my hands on multitouch surfaces. At first, I disassembled my camera according to Peter Kirn's instruction videos to remove the IR filter. I was quite lucky since I bought the cam at Saturn in Germany and did not know that there are two versions of the cam. The filter of the one mostly sold in the USA can be removed easily, as seen in the video. The other version cannot be modified that way, so the complete lens has to be exchanged.
With the help of Alex Popovich's Driver which nicely works on Windows 7 64 bit edition and two days of learning some Direct3D programming, I was finally able to render the captured images on screen. After some experimenting with HLSL I had my very first realtime edge detector running.
The detection is quite simple, only a simple 3x3 filter matrix is applied to each frame. Here is the matrix:

0.0,   -1.0,    0.0,
F=  -1.0,    4.0,   -1.0,
0.0,   -1.0,    0.0

And some threshold operations afterwards leads to these results:

Screenshot without edge detection

Screenshot with edge detection enabled

As you can see, the picture is quite blurry which has a negative impact on corner detection. That is because of the missing IR filter, now I am unable to set the focus right, maybe I can put a piece of glass into the lens to correct the error.