One of the most popular software for VJs
Modul8 is a MacOS X application designed for real time video mixing and compositing. It has been designed for VJs and live performers.
Features:
Real Time Video Mixer and a user interface for real time- Is thought for real time video.
Is based on a layer metaphor, each change can be seen immediately in the composition.
Each media is a layer that can be moved, resizaed an rotated.
Has 7 outputs plus one for the user interface, you can determine what region of the composition is sent to which projector or screen.
Is posible extend the system incorporating new modules, writing in Python script.
Is an accessible on-line library more of them free.
Modul8 is a proprietary software.
Wednesday, June 23, 2010
Wednesday, June 16, 2010
Differents structured light techniques
Structured light is the process of projecting a known pattern of pixels on to a scene, the displacement of the stripes allows for an exact retrieval of the 3D coordinates of any details on the object's surface (depth and surface information).
A structured-light 3D scanner is a device for measuring the three-dimensional shape of an object using projected light patterns and a camera system.
This paper presents a comprehensive survey on coded structured light techniques.
The patterns are specially designed so that codewords are assigned to a set of pixels. Every coded pixel has its own codeword, so there is a direct mapping from the codewords to the corresponding coordinates of the pixel in the pattern.The codewords are simply numbers, which are mapped in the pattern by using grey levels, color or geometrical representations.
Shows pattern projection techniques classified in three groups according to their coding strategy:
Time-multiplexing strategy
A set of patterns are projected onto the measuring surface, the codeword for a given pixel is formed by the sequence of illuminance values for that pixel across the projected patterns. The codification is called temporal because the bits of the codewords are multiplexed in time.
This kind of pattern can achieve high accuracy in the measurements. This is due to two factors:
1- since multiple patterns are projected, the codeword basis tends to be small (usually binary) and therefore a small set of primitives is used, being easily distinguishable among each other;
2- a coarse-to-fine paradigm is followed, since the position of a pixel is being encoded more precisely while the patterns are successively projected.
They clasify the different techniques based on time-multiplexing as:
a) techniques based on binary codes: a sequence of binary patterns is used in order to generate binary codewords
b) techniques based on n-ary codes: a basis of n primitives is used to generate the codewords
c) Gray code combined with phase shifting: the same pattern is projected several times, shifting it in a certain direction in order to increase resolution
d) hybrid techniques: combination of time-multiplexing and neighborhood strategies
Spatial neighborhood
Tend to concentrate all the coding scheme in a unique pattern
clasification of techniques:
a) strategies based on non-formal codification: the neighborhoods are generated intuitively
b) strategies based on De Bruijn sequences: the neighborhoods are defined using pseudorandom sequences
c) strategies based on M-arrays: extension of the pseudorandom theory to the 2-D case
Direct codification
The entire codeword for a given point is contained in a unique pixel. In order to achieve this, it is necessary to use either a large range of color values or introduce periodicity.In theory, a high resolution of 3D information can be obtained.The sensitivity to noise is very high because the "distance" between "codewords", i.e. the colors used, is nearly zero.Moreover, the imaged colors depend not only on the projected colors, but also on the intrinsic color of the measuring surface.
They discuss of two groups of methods:
a) codification based on grey levels: a spectrum of grey levels is used to encode the points of the pattern
b) codification based on color: these techniques take advantage of a large spectrum of colors.
After clasification they implement some methods, compare them and present their resuls.
A structured-light 3D scanner is a device for measuring the three-dimensional shape of an object using projected light patterns and a camera system.
This paper presents a comprehensive survey on coded structured light techniques.
The patterns are specially designed so that codewords are assigned to a set of pixels. Every coded pixel has its own codeword, so there is a direct mapping from the codewords to the corresponding coordinates of the pixel in the pattern.The codewords are simply numbers, which are mapped in the pattern by using grey levels, color or geometrical representations.
Shows pattern projection techniques classified in three groups according to their coding strategy:
- time-multiplexing - generate the codewords by projecting a sequence of patterns along time, so the structure of every pattern can be very simple
- neighborhood codification - represents the codewords in a unique pattern
- direct codification - define a codeword for every pixel, which is equal to its grey level or color
Time-multiplexing strategy
A set of patterns are projected onto the measuring surface, the codeword for a given pixel is formed by the sequence of illuminance values for that pixel across the projected patterns. The codification is called temporal because the bits of the codewords are multiplexed in time.
This kind of pattern can achieve high accuracy in the measurements. This is due to two factors:
1- since multiple patterns are projected, the codeword basis tends to be small (usually binary) and therefore a small set of primitives is used, being easily distinguishable among each other;
2- a coarse-to-fine paradigm is followed, since the position of a pixel is being encoded more precisely while the patterns are successively projected.
They clasify the different techniques based on time-multiplexing as:
a) techniques based on binary codes: a sequence of binary patterns is used in order to generate binary codewords
b) techniques based on n-ary codes: a basis of n primitives is used to generate the codewords
c) Gray code combined with phase shifting: the same pattern is projected several times, shifting it in a certain direction in order to increase resolution
d) hybrid techniques: combination of time-multiplexing and neighborhood strategies
Spatial neighborhood
Tend to concentrate all the coding scheme in a unique pattern
clasification of techniques:
a) strategies based on non-formal codification: the neighborhoods are generated intuitively
b) strategies based on De Bruijn sequences: the neighborhoods are defined using pseudorandom sequences
c) strategies based on M-arrays: extension of the pseudorandom theory to the 2-D case
Direct codification
The entire codeword for a given point is contained in a unique pixel. In order to achieve this, it is necessary to use either a large range of color values or introduce periodicity.In theory, a high resolution of 3D information can be obtained.The sensitivity to noise is very high because the "distance" between "codewords", i.e. the colors used, is nearly zero.Moreover, the imaged colors depend not only on the projected colors, but also on the intrinsic color of the measuring surface.
They discuss of two groups of methods:
a) codification based on grey levels: a spectrum of grey levels is used to encode the points of the pattern
b) codification based on color: these techniques take advantage of a large spectrum of colors.
After clasification they implement some methods, compare them and present their resuls.
Sunday, May 30, 2010
Structured light - Continuity vs Discontinuity
A third experiment with structured light was performed trying to scan a simple and very used scene in the video-mapping field: a group of primitive shapes.
We faced a lot of problems with this apparently simple scene and that made us think that structured light wasnt suitable for discontinuous shapes, so we built a complex but continuous geometry by wrapping the scene with a piece of cloth. The SL-applet had no trouble generating the 3d geometry.
After getting these results we asked the creator of the applet for confirmation on our early conclusion and we got a response. It was true that the experiment performed wasnt suitable for discontinuous shapes because it uses phase-shifting scanning based on the principle of propagating depth values across a surface. So if two surfaces are disconnected, it cannot determine how they are related depth-wise.
However, it wasn't true that SL is inappropriate for discontinuous shapes. Other pattern codifications and algorithms not so based on real-time can return better results for these type of scenes.
This is what we'll be working on the next weeks.
Photos of the scenes - discontinuous and continuous
We faced a lot of problems with this apparently simple scene and that made us think that structured light wasnt suitable for discontinuous shapes, so we built a complex but continuous geometry by wrapping the scene with a piece of cloth. The SL-applet had no trouble generating the 3d geometry.
After getting these results we asked the creator of the applet for confirmation on our early conclusion and we got a response. It was true that the experiment performed wasnt suitable for discontinuous shapes because it uses phase-shifting scanning based on the principle of propagating depth values across a surface. So if two surfaces are disconnected, it cannot determine how they are related depth-wise.
However, it wasn't true that SL is inappropriate for discontinuous shapes. Other pattern codifications and algorithms not so based on real-time can return better results for these type of scenes.
This is what we'll be working on the next weeks.
Photos of the scenes - discontinuous and continuous
Thursday, May 20, 2010
Second scanning
On this second experiment with structured light we scanned a simple scene: two perpendicular walls.
A lot of video mapping applications like this one focus on objects with planar faces like boxes so the idea of this experiment was to test structured light with these type of objects.
The geometry was fairly well retrieved with minor tweakings. The pictures were only cropped around the target zone and the Z scale and Z skew were adjusted in the SL applet.
Here are the pictures taken and a snapshot of the retrieved cloud of points.
A lot of video mapping applications like this one focus on objects with planar faces like boxes so the idea of this experiment was to test structured light with these type of objects.
The geometry was fairly well retrieved with minor tweakings. The pictures were only cropped around the target zone and the Z scale and Z skew were adjusted in the SL applet.
Here are the pictures taken and a snapshot of the retrieved cloud of points.
Wednesday, May 19, 2010
Saturday, May 15, 2010
First tests
Last Saturday we could finally get a projector and started making the first experiments of mapping and scanning.
Manual mapping
The goal of this experiment was to project two different images on two perpendicular walls (a corner of the room). The projector was not aligned with any of these walls so any planar projection resulted in deformed images. With a 3D software two textured planes were created simulating the target planes. Then the corner vertices were manually adjusted until the projected images looked undeformed on the walls.
Remark: The deformed planes in the 3D software didn't represent the shape of the walls but a planar deformed shape that matches the projection. That lead us to think that we could accomplish the goal of project over real 3D structures using simplified 2D representations of that 3D object.
This first mapping was manually performed. Programs like modul8 map the images to the geometry manually.
In others like vvvv the definition of the geometry is needed to generate the virtual model. Then this virtual model is adjusted with the real one.
This last automatic approach is what we want to achieve.
3D Scanning
The second experiment was scanning objects using the structured light technique.
The goal was to obtain three photos projecting the three-phase patterns seen in the link above. These photos and the mentioned patterns are then processed by the application to generate a cloud of points that represents the retrieved 3D geometry.
Many configurations were tested using different camera positions and different distances from the projector to the target object. We also started using an standard web camera but we were having very bad resolution in the captured images and we had problems focusing the scanned object/person (it was too "far" in the image with the corresponding loss of detail).
The best results were obtained using a photo camera of 5 mega pixels and optical zoom x12 and with the camera field of view similar to the projector's. Also the pictures were cropped with an image editor so the applet focuses only on the target object.
The calibration of the projector seemed to be very important so the pattern projected looks as sharp as possible. That, summed up to the fact that we had to use a high resolution photo camera instead of a low resolution web camera, lead us to think that the structured light algorithm requires images with a mid-to-high resolution
Another conclusion, taken directly from the obtained results, is that the objects to scan shouldn't have glossy surfaces because the patterns are lost on those areas.
Taking into consideration the conclusions expressed above, we tend to think we will have problems implementing a real time solution of 3D reconstruction using standard market cameras.
See here the obtained pictures of the reconstructed Javier's (jefa) head.
Recognized 2.5D shape:
Manual mapping
The goal of this experiment was to project two different images on two perpendicular walls (a corner of the room). The projector was not aligned with any of these walls so any planar projection resulted in deformed images. With a 3D software two textured planes were created simulating the target planes. Then the corner vertices were manually adjusted until the projected images looked undeformed on the walls.
Remark: The deformed planes in the 3D software didn't represent the shape of the walls but a planar deformed shape that matches the projection. That lead us to think that we could accomplish the goal of project over real 3D structures using simplified 2D representations of that 3D object.
This first mapping was manually performed. Programs like modul8 map the images to the geometry manually.
In others like vvvv the definition of the geometry is needed to generate the virtual model. Then this virtual model is adjusted with the real one.
This last automatic approach is what we want to achieve.
3D Scanning
The second experiment was scanning objects using the structured light technique.
The goal was to obtain three photos projecting the three-phase patterns seen in the link above. These photos and the mentioned patterns are then processed by the application to generate a cloud of points that represents the retrieved 3D geometry.
Many configurations were tested using different camera positions and different distances from the projector to the target object. We also started using an standard web camera but we were having very bad resolution in the captured images and we had problems focusing the scanned object/person (it was too "far" in the image with the corresponding loss of detail).
The best results were obtained using a photo camera of 5 mega pixels and optical zoom x12 and with the camera field of view similar to the projector's. Also the pictures were cropped with an image editor so the applet focuses only on the target object.
The calibration of the projector seemed to be very important so the pattern projected looks as sharp as possible. That, summed up to the fact that we had to use a high resolution photo camera instead of a low resolution web camera, lead us to think that the structured light algorithm requires images with a mid-to-high resolution
Another conclusion, taken directly from the obtained results, is that the objects to scan shouldn't have glossy surfaces because the patterns are lost on those areas.
Taking into consideration the conclusions expressed above, we tend to think we will have problems implementing a real time solution of 3D reconstruction using standard market cameras.
See here the obtained pictures of the reconstructed Javier's (jefa) head.
Recognized 2.5D shape:
Tuesday, May 11, 2010
Dynamic Projection Environments for Immersive Visualization
This paper present a system for dynamic projection.
the projected surfaces are large screens (in human-body scale), each one has wheels that allow move it easyly. When the projection surfaces are moved in real-time, the application re-calculates the visualization on the fly.
They use a technique they call projection keyframing to provide continuity on moving surfaces
while waiting for simulations to complete.
The system allows multiple users to participate interactively with each other and the visualization application.
Initial target application for the system was interactive
architectural lighting visualization, they give a simulated environment to architects and
clients to evaluate the natural and artificial lighting of a proposed architectural design.
The distributed system allows for:
- Projector keyframing - a technique to impart slow applications with a dynamic, responsive feel
- Tracking projection surfaces of known geometry with simple IR-based LED markers, and
- A distributed rendering system which can be extended to drive an arbitrary number of projectors.
The system steps:
- The system use a single camara to obtain images of the scene
- determine the projection surface geometry whit the information of the camera
- use a gigabit-Ethernet connected camera to detect the LEDs sensor (ubicated in the top of each
screen)
- dinamic projection using 10 projectors
Aplications:
- architectural visualization
- explore volumetric data by defining cross-sections
- general purpose user-interace elements
see a video with examples here
the projected surfaces are large screens (in human-body scale), each one has wheels that allow move it easyly. When the projection surfaces are moved in real-time, the application re-calculates the visualization on the fly.
They use a technique they call projection keyframing to provide continuity on moving surfaces
while waiting for simulations to complete.
The system allows multiple users to participate interactively with each other and the visualization application.
Initial target application for the system was interactive
architectural lighting visualization, they give a simulated environment to architects and
clients to evaluate the natural and artificial lighting of a proposed architectural design.
The distributed system allows for:
- Projector keyframing - a technique to impart slow applications with a dynamic, responsive feel
- Tracking projection surfaces of known geometry with simple IR-based LED markers, and
- A distributed rendering system which can be extended to drive an arbitrary number of projectors.
The system steps:
- The system use a single camara to obtain images of the scene
- determine the projection surface geometry whit the information of the camera
- use a gigabit-Ethernet connected camera to detect the LEDs sensor (ubicated in the top of each
screen)
- dinamic projection using 10 projectors
Aplications:
- architectural visualization
- explore volumetric data by defining cross-sections
- general purpose user-interace elements
see a video with examples here
Subscribe to:
Posts (Atom)