MirrorFun


Idea
You go to the screen and see your mirrored image which moves accordingly to your motions directions.
The effect resembles in some kind  slit-scan effect, but which depends (is controlled by) the motions of user.
The project works in two modes - grid mode (Fig. 1), where user controls grid on the screen, and mirror mode (Fig. 2), where used controls its mirrored image.

Exhibitions and other uses
The project was exposed on many exhibitions of various kinds by myself, and I know it was used some enthusiasts for their own purposes.

Connection to other projects
The MirrorFun project is a further development of RubberMirror project, where I keep idea of  showing “changed image” from web camera, but instead “rubbering” the part of the image, I “rubbering” the whole image, and use for this new technical idea - “optical flow as driving force of interactive image”.

Keywords 
Hook’s law, optical flow


Technology
The image captured from web camera is processed by OpenCV algorithm, which computes optical flow, that is moving of the pixels when comparing previous and current camera’s frame. The optical flow is used as a force for moving grid of points, which are placed regurarly on the screen and “connected by a rubber sticks”.
As a result, we get set of points, which starts moving when there is some motion in the image, and slowly freezing (due friction) when there are no motion.
Next, we draw the points as a grid (grid mode), or use points for drawing quads with inout image as a texture (mirror mode).

Implementation
Project is written on openFrameworks with using ofxOpenCV and ofxIniSettings addons.

1. https://sourceforge.net/projects/perevalov-tech/files/2010/MirrorFun/

2. https://cloud.mail.ru/public/GQDZ/fhRDwsciL

Running requirements: Windows, webcamera.

Instructions: If it not startied, please install vcredist located in install-win folder.  
Press SPACE to switch between grid and mirror modes.
Press ESC to close the app.

Included files contains binaries for Windows, but you can rebuilt the project for Mac OS: install openFrameworks, then use Project Generator included in openFrameworks for generating project files and build the project in XCode.


History
In 2010, a some clothing store asked me to create an playful interactive installation which shows persons on a big TV such that when people walking near it, the image on TV began to fluctuate like a water. I made it, but finally it’s appear that “fluid mirror” is more attractive thatn fluctuated fixes image. The project was never used in that store, but resulted MirrorFun project was my first “seccessful” openFrameworks projects, which was mentioned on several forums and by some bloggers of that time.

Note: Actually, I published video on this project as “YouTube answer video” on another’s guy video on using Optical Flow, and in this way video get unexpectedly many views and was noted by several blogs/forums on creative coding. Currently such “youtube answer” option is disabled by YouTube.

It was the first time when I use optical flow algorithm from OpenCV library.

Note: In first versions I use Lucas-Canade method, but in latest versions use Farneback algorithm, because it’s more robust.

It was a big breakthrough for my further interactive projects, because optical flow works very stable with web cameras (it was before Kinect), much more stable than computing background and analyzing silhuettes (what’s very simple with depth cameras).



Fig. 1. Mirror fun (grid mode)


Fig. 2. Mirror fun (mirror mode)



Fig. 3. Mirror fun scheme (mirror mode)









Video 1. MirrorFun (grid mode)



Video 2. MirrorFun (mirror mode)



Video 3. MirrorFun modification - controlling fixed image


Video 4. Using MirrofFun’s image modification for gaming


DENIS PEREVALOV, EKATERINBURG, RUSSIA.