Motion Capture & Computer Vision@AIS-lab


Copyright notice: Most of the papers linked to this page are protected by copyright. It is the responsability of people who download papers to be sure they have the right to do it.

Three are the main domains in which AIS-Lab has contributed:
- Tracking with cameras: a full system for tracking surgical instruments has been realized in an industrial project. Tracking of overlapping objects has also been developed.
- Calibration solutions have been deviced, using points or conics. Simplified calibration procedures for the new Kinect have also been developed.
- Scanning Systems based on cameras have been developed.

Micro-Motion capture. The system is based on a four cameras firewire architecture. It provides in real time at 25Hz the 6dof of up to different surgical instruments (a 3min, 256Mbyte video, can be viewed from here), while it is manouvred by the surgeon (12Mbyte video, click here).
The instruments are markered with spherical white markers on the axes of their tips, also when moving fast (12Mbyte video, click here).
Thanks to dedicated image processing algorithms, its performance is superior to commercial systems now on the market.
The two main features are:
A). Detection of the markers with an innovative circle fitting algorithm, which is not sensitive to partial occlusions or overlapping (to read more, click here).
2. The reconstruction of the axial rotation using only spherical markers on the axis, painting a line on the surface of at least one of it (to read more, click here TO BE SUBMITTED).

This methodology has been patented.

Tracking. A new approach to the reconstruction of 3D trajectories of dense marker sets has been developed. Key element is the use of multiple passes to reconstruct the spatio-temporal structure of the movement with high reliability. First the tracking procedure computes a coarse structure of the motion, which is then recursively refined disambiguating difficult classification of the markers. The tracking procedure is based on integrating the temporal dimension in the matching process, by analyzing strings instead of points to derive more robust matches. Strings are analyzed using smoothness, n-focal constraints, and fitting of a skeleton to derive a proper matching. An innovative augmented-reality like interface greatly simplifies the labeling task. Lastly, a proper value for the critical parameters is automatically derived. Results on real data show that the system is able to produce a robust and largely complete set of trajectories, which greatly minimize the time required by post-processing editing sessions. A preliminary report has been published in the proc. of the first IEEE conference on 3DPVT (2002). Some results can be downloaded: escape (1,975k), fall_run (1,966k), run (472kbyte), roll (1,310k) and one trial with more than 150 markers tracked (1,702k).
Augmented reality. We have developed a novel simple procedure to compute the focal length of a camera. The method is based on zooming in and out only a single point. The same approach allows computing the principal point when only two points are available on a pair of images surveyed with a different focal length. Experimental results show that the method is as accurate as classical full calibration methods. Moreover, its application to augmented reality produces more accurate results than those obtained when the simple pin-hole model is considered as it can be seen comparing the two lower panels: on the right (pin-hole model) the virtual and real balls are significantly shifted while in the left panel (proposed method) the alignement has almost no error. Results have been published in 2006 on Pattern Recognition.

Self-Calibration. Calibration is carried out surveying only a rigid bar carrying two markers on its extremities, moved inside the working volume. The external parameters are estimated through the epipolar geometry up to a scale factor which is determined from the true length of the bar. The focal lengths are determined using the properties of the absolute conic in the projective space. The principal points are computed through a non-linear minimization carried out through an evolutionary optimization (2001 IEEE Trans. Evol. Computation). The accuracy of the method compares favorably with that obtained through classical approaches based on control points of known 3D coordinates. It requires 20 seconds on a PC, Pentium II. More can be learnt from the 2000 Pattern Recognition paper.
A portable 3D scanner . We pioneered the low-cost, high accuracy 3D scanners with a dedicated vision system described in the 1998 paper on IEEE Computer & Graphics. The system was further developed and a more complete report can be found on 2000 IEEE Trans. I&M. The description of a method able to construct in real-time multi-resolution meshes is described in a 2005 IEEE Trans. I&M paper. Besides, methods for constructing 2.5 multi-resolution meshes and to reduce and filter point clouds have been developed. More recently we have developed an on-line verion of the algorithm that has been published on IEEE Trans. on Neural Networks; a video of the algorithm behavior can be downloaded from here.
Mesh retiling with colour attributes. It allows to produce a 3D mesh which satisfies both a given geometry and colour error. The novel philosophy is to unify colour and texture into the colour field with a per-vertex representation. The algorithm re-tiles the mesh obtained by a geometry simplification algorithm, by adding new vertexes and triangles until a given colour error is achieved over all the mesh. The method can work with any geometry simplification algorithm and allows the user to get control on the desired LOD in geometry space and the desired visual degradation in colour space. Results show how, at the expense of few more polygons, the final appearance of the model can be greatly improved in a very short time. The approach presented here is quite general: it can be easily extnded to incorporate diffuse colour, but it can also be applied to other kinds of fields (e.g. normal maps, displacement maps). Moreover, it is particularly suitable to progressive mesh transmission, where the basic mesh is transmitted first with its per-vertex colours, and in a second stage, the position and colour value of the faces added in the refinement stage have to be sent. You can learn more from the 2001 Computer & Graphics paper.

The left mesh is constituted of 2,505 triangles, the mesh on the right of 4,307. Click here for a picture of higher resolution. Two different views of the re-tiled model.
 

Profilometry. The control of the quality of rubber tyres and of its cut is achieved through a real-time scsnning system based on a laser, a camera, and a dedicated processing software. The system is able to reliably spot defects of the canyons and to compute when the tyre has to be cut by the machine (you can download an informal presentation here).
Classification of defects on a production line. A prototype system based on boosting and on robust feature extraction has been developed to classify in real-time the defect detected on a production line.
Panoramic images. Epipolar geometry and a careful choice of reference points has allowed to construct a handy method to create panoramic images from skewed photographies.

Last update 31.10.2012

 For more information click here