Research Projects & Grants
2018-2012. Institute of Robotics and Information and Communication Technologies, Universitat de València
H2020, European Commission
Erasmus + Programme, European Commission. 2016-1-PL01-KA202-026102
Erasmus + Programme, European Commission. 2016-1-ES-EPPKA2-SSA
Sistema de Realidad Aumentada para Auto-Inspección de Edificios Energéticamente Eficientes (SIRAE)
Ministerio de Economía y Competitividad. RTC-2015-4203-7
Sistema inteligente de clasificación de nubes de puntos 3D para la industria pesquera
IUNNA - Skyline Optical Applications
AR.Key: Augmented Reality Applied to training in key competences
Comisión de las Comunidades Europeas. Proyecto Leonardo da Vinci. 2013-1-ES1-LEO05-66235
ARCW: Health and Safety Procedures for Curtain Walls with the Use of Augmented
Comisión de las Comunidades Europeas. Proyecto Leonardo da Vinci. 2013-1-PL1-LEO05-37525
Simulación y tratamiento guiado por imagen en radioterapia (PRECISION) / Simulation and image-guided treatment in radiotherapy (PRECISION)
NCPP - Programa Nacional de Cooperación Público-Privada -subprograma INNPACTO del Ministerio de Economía y Competitividad
Desarrollo de un protitipo de visión artificial para asistencia a personas con discapacidad visual
Reconstrucción 3D tutilizando múltiples patrones de proyección (sub-contratación FP7-SYDDARTA)
Instituto tecnológico de óptica, color e imagen (AIDO)
Postdoctoral Grant Juan de la Cierva
Ministerio de Economía y Competitividad
2012-2011. Instituto Tecnológico de Óptica, Color e Imagen (AIDO)
SYstem for Digitization and Diagnosis in ART Applications (SYDDARTA)
VII Framework Programme
Subcontracting for the project: Technologies for the hotel of the future (THOFU)
CDTI - Centro para el Desarrollo Tecnológico Industrial (CDTI)
Ministerio de Ciencia e Innovación, Programa CENIT
Development of integrated platform management processes in the value chain of the optical industry trade (RETINA)
CDTI - Centro para el Desarrollo Tecnológico Industrial (CDTI)
Fondos Europeos de Desarrollo Regional (FEDER)
Quality control system and classification of grapes in arrival at the wine cellar (ENOVISION)
BARNIGRADO SERVICIOS PARA EL ANÁLISIS DEL GRADO S.L.
2011-2008. Grupo de Investigación en Fotogrametría y Láser Escáner (GIFLE) / Environmental and Heritage Assessment Research Group (ENHEA), Universitat Politècnica de València
Thermography and 3D Documentation of Djin Block number 9 in Petra
Agencia Española para la Cooperación INternacional y el Desarrollo (AECID)
Design and implementation of programs for the interpretation of the heritage for tourist guidance
Agencia Española para la Cooperación Internacional y el Desarrollo (AECID)
Technical assistance to the project Virtual Street Walker
Trabajos Catastrales S. A.
Development of the photogrammetric software ORTOFOTOUAV, assistance and quality control with a UAV (Unmanned Aerial Vehicle) system, with the application for photogrammetric projects, construction and infrastructure exploitation
Aula de Natura per als Visitants de la Marjal de Gandia
Transformación Agraria, S.A.
Design of a recreational management plan for the archaeological site Castellet de Bernabé
Consorcio de Museos de la Comunidad Valenciana
2008-2004. Laboratorio de Luz, Universitat Politècnica de València / 2006. Interaction & Entertainment Research Centre, Nanyang University of Singapore. Mixed Reality Lab, National University of Singapore / 2005. Mixed Reality Laboratory, University of Nottingham
Augmented vision: visual tracking systems for interactive installations in the arts
Ministerio de Ciencia e Innovación, Dirección General de Investigación y Gestión del Plan Nacional de I+D+i.
Grant Formación de Personal Investigador (FPI)
Ministerio de Educación y Ciencia
Laboratorio de Luz, Departamento de Pintura de la Facultad de Bellas Artes de San Fernando, Universitat Politècnica de València.
2004-2002. Departament of Geodesy, Cartography and Photogrammetry, Universitat Politècnica de València
Grant Colaboración para Proyectos de Innovación Docente (PIE) Departamento de Matemática Aplicada, Universitat Politècnica de València.
Estudio y Diagnosis de Manifestaciones Patológicas de la Piedra y Proyecto de Ejecución y Dirección de las Obras de Limpieza y Conservación de la Lonja de los Mercaderes de Valencia
Forum UNESCO, Universitat Politècnica de València.
2002. Institute of Photogrammetry and Remote Sensing, Vienna Universit of Technology
Georeferencing of GPS/INS-supported airborne multispectral scanner data
Grant LEONARDO DA VINCI
Institute of Photogrammetry and Remote Sensing (IPF). Vienna University of Technology (TU-Wien)
Master in Surveying and Geoinformation
Vienna University of Technology (TU-Wien).
Master in Surveying and Geoinformation
Vienna University of Technology (TU-Wien).
SILKNOW, weaving our past into the future
27 Nov 2018
Jornada Oportunidades a la Financiación de las Industrias Culturales y Creativas, AMETIC, eNEM
Interactive visutalization of tangible and intangible Heritage
3 July 2019
DataBeersVLC, International edition organized within the Summer School on Data Science at the University of València.
⚡4th speaker! @CristinaP2012— DataBeers VLC (@databeersvlc) July 3, 2019
from @IRTIC_UV presents last results of @silknow_eu project in @DataUv
⏩ "Interactive #visualization of tangible and intangible #heritage"#DataScience #etseuv#DatabeersVLC@ETSEUV @FdEconomiaUV @FILTRADCOMUV pic.twitter.com/SHH2PyKmHy
Lobby Hour: El Papel de la Ciencia en el Parlamento
17 Sept 2020
Evento organizado por la asociación APRI. Particpan: Asociación Ciencia en el Parlamento, Cotec y FECYT.
Some Research Outcomes
The geometric calibration of projectors is a demanding task in many areas related to computer vision, virtual reality or augmented reality, to name some. Up to date, different methods have been proposed to retrieve the intrinsic and extrinsic parameters of projectors. During the last twenty years, researchers have used cameras as means to calibrate projectors in order to automatize the process. However, this might add: 1) complexity in terms of mathematical formulation; 2) restrictions in terms of camera locations relative to projectors; and 3) additional errors (those due to the camera calibration itself). Most of these camera-based methods make use of planar homographies and others require an extended calibration process (for both the camera and the projector). We have developed an approach that combines a Direct Transformation Method (DLT) with projected Augmented Reality (AR) to perform an interactive calibration of projectors without the need of cameras. The method is based on non-coplanar points and 2D/3D correspondences, which are interactively established. Intrinsic and extrinsic calibration is achieved in a single step, making use of the DLT equations.
MIME is a mixed-space tri-collaborative system. It considers three different types of users (in different locations) according to the level of immersion in the system, who can interact simultaneously – what we call a tri-collaboration. For the three types, we provide a solution to navigate, point at objects/locations and make annotations, while users are able to see a virtual representation of the rest of users. Additionally, the total number of users that can simultaneously interact with the system is only restricted by the available hardware, i.e., various users of the same type can be simultaneously connected to the system.
Research on diverse algorithms to obtain 3D accurate models using structured light techniques. Working with methods able to obtain a phase image after projecting and processing one or some images with sinusoidal stripes. In particular, to obtain the so-called phase images, the candidate has implemented the techniques of Phase Shifting, Multi Phase Shifting, Fourier Transform Profilometry, and a combination of some of them. Implementation of two unwrapping algorithms to get the unwrapped or absolute phase. To deal with the problem of phase-to-height conversion, she is working with different techniques.
Design and implementation of an automatic procedure to auto-calibrate a projector-camera system. The system is calibrated by using the method of Direct Linear Transformation. The 3D calibration pattern follows a cylindrical shape, and only two images are needed as input to the algorithm. It takes only few seconds to calibrate both devices. Once calibrated, the system acts as a stereoscopic system and objects can be digitalized following the principles of spatial ray intersection. Portalés has also implemented different image mappings (orthogonal projection, wallpaper or perspective) to project images without deformation into cylindrical surfaces. An extension of this method for any kind of 3D projection surfaces is being implemented. This method is being used for driving simulations where a virtual reality world is projected on cylindrical screens, but it can be used in other contexts where the projected image needs to be fitted to a specific 3D surface or when the projection needs to be recalculated in real time and shown to the user according to its point of view.
Implementation of a method to calibrate a camera-projector system , which is suited for monochromatic cameras and projectors. The method makes use of planar calibration patterns, is fully automatic, easy to use and fast. Additionally, no initial information regarding the camera interior orientation is needed, and both sensors are fully calibrated during the process.
Artwork chemical deterioration assessment, 2012
Implementation of a methodology to deal with chemical artwork deterioration by analyzing data from a hyperspectral imaging system. The system consists of a digital light projector (DLP) and two cameras: a camera working in the visible range (400 to 720 nm) and providing information each 10 nm with a Liquid Crystal Tunable Filter (LCTF); and a camera working in the infrared region (900 to 2500 nm) and providing information each 10 nm with an Acousto-Optic Tunable Filter (AOTF). To assess chemical deterioration, a database was built with the spectral information of deteriorated pigments (from ultra-violet radiation, from relative humidity/temperature and from pollutants). Data was pre-treated to remove non-lineariteis due to scatter from particulates in the samples. Different classification methods were assessed. Classification results reached 95% of success for 30 classes and considering only 10 dimensions. This research is part of the project SYDDARTA.
Detection and quantification of stems and other objects, 2012
Design, construction and software implementation of an optical system to automatically detect, classify and quantify grape stems and other objects from images acquired during the grape harvest on arrival to the wine cellar, in order to assess a preliminary quality control of the collected grains. A non-destructive technology is used by means of computer vision, and the developed approach is based on the combination of a spectral and spatial analysis. This research is part of the project ENOVISION.
Gender and age classification from face images, 2011
Gender and age classification from face images (considering a total of 8 classification groups). To that purpose, first a database was created by considering ages from 15 to 80 and including women and men. First of all, faces were detected on images and an alignment and scaling were applied after detecting the pupils. Then, images were normalized and equalized. The average face and the eigenfaces were computed by applying Principal Components Analysis (PCA). Then, a total of nine classification methods were considered, including Support Vector Machines (SVM). This research is part of the project RETINA (AIDO).
People verification from 3D data, 2011
Preliminary study to choose an appropriated technology and approach to deal with people verification by using 3D data. To that purpose, a comparative study with using different devices was carried out: stereoscopy, structured light, Kinect and Nub3D.
Design of a low-cost laser-based profilometer to measure the indent of (glasses) frames, 2011
Design, calibration and software implementation of a low-cost laser-based optical profilometer, which makes use of commercially available devices such as a CCD camera, a line laser and stages. The outcomes of the 3D reconstruction of an inspected piece were compared to those obtained with a commercial system, which resulted in discrepancies below 0.2 mm for the 84% of the data, considering a working volume of 150x50x50 mm.
Integration of terrestrial laser scanning and imagery, 2009
Design and implementation of a methodology to accurately derive 3D photo models by means of terrestrial laser scanning (TLS) and images from a visible and a thermal camera. TLS provides 3D point clouds as well as rough photo models that can substantially be improved draping texture with external imagery. The ancient monument tomb Djin Block No. 9 in Petra was used to compare texture results delivered by the camera of the terrestrial laser scanner versus an external off-the-shelf digital camera. During the process, image resection and calibration of non-metric single imagery was tackled making use of a digital surface model. The developed approach is suitable for off-the-self digital cameras and can cope with single images without overlap. Results were satisfactory and showed that it is possible, on the one hand, to enhance photo models draping external imagery onto the 3D model, and on the other, to take advantage of single non-metric images to record cultural heritage. This research was funded by the project “Termografía y Documentación 3D del Djin Block número 9 en Petra”.
Classification of thermal images for moisture assessment on façades, 2009
It is well known that the content of water does affect materials behavior and may cause important damages to buildings, including structural deterioration. Thermography is a non-intrusive imaging technique that allows the inspection of different materials based on their surface temperatures. Within an appropriate thermal analysis, moisture and other structural damages can be detected.
Research dealing with multitemporal thermal analysis to detect moisture on buildings, as well as other alternations. Two different approaches were tested to determine the effectiveness on the moisture surveys, one simple based on image differences, and another more sophisticated based on principal components analysis (PCA). The results were tested at the front door of the Arenberg Castle (Leuven, Belgium). Some moisture maps were derived and used to determine the reliability of either the computation of image differences or the PCA to carry out semi-automatically moisture surveys on architectural façades. The latter method was proven to be superior when using thermal imagery acquired at different times.
AR-Inmersive Cinema is a novel AR application where augmented users are inserted into lineal video sketches. “History” is usually seen as too-boring to learn by young people. In this sense, we find essential to present important historical moments in a fashion way, in such manner that it would be able to engage users. This application is achieved within two technologies: the first one is a cinema filming with authors that interpret a sketch; the second part concerns to the AR and is managed within the ARToolKit libraries (HITLab, 2007). Users see themselves virtually transformed in one of the actors thanks to the paradigm of magic mirror (Giner Martínez and Portalés Ricart, 2005). To increase engagement, actors at the video sketches do talk to the user, looking to its position. Both technologies are visually integrated within a combination of spatial aligned projections.
Ciclo hidrológico, 2008
Hydrological Cycle is a Flash-based multimodal interactive application, as different stimuli are integrated: it is visual and sonic. In addition, user interaction is achieved thanks to a tangible screen. A total of six stages are considered as being representative of the Marjal de Gandia: evapo-transpiration, precipitation, runoff, ground water supply, natural spring and pumping. In the last stage, it is shown the marine intrusion in the natural aquifer due to the human action. Visually, the application is composed of a combination of images (including aerial ones), vector graphics and short animations. This combination makes it more attractive and highly dynamic. Reproduced sound accompanies the visual actions; for instance, when it is raining the falling water drops are heard; the sky is illuminated and a thunder sounds; when there is calm, a sound of birds is displayed.
Riego tradicional, 2008
Traditional Irrigation is a Flash based game that has the goal to teach people the way traditional irrigation was carried out at the Marjal. Users have to open and/or close some kind of irrigation doors in order to properly irrigate one of the fields. In addition, users have to find the optimum combination of opened and closed doors in order to use only the necessary water to irrigate the field. When users are ready, they should press the button of irrigation, and then the program show one of the three possible results: (1) well done! You managed to water the field and you did it very efficiently; (2) fine! You managed to water the field but you have used more water than the necessary; (3) try again, you haven’t managed to water the field. This interactive application is visualized on a tangible screen and is integrated in the museum in the middle of a 3x3 m panel showing an orthogonal aerial image. The screen shows also the orthogonal aerial image having continuity within that of the panel.
Castellet de Bernabé, 2007
En este trabajo se realiza una reconstrucción 3D del Castellet de Bernabé (Lliria). Para ello es necesario la ayuda de estudios arqueológicos, puesto que actualmente sólo se conserva parte de esta construcción Íbera.
AR-Jazz is an augmented reality application designed to visualize sound and movements in live jazz performances. The augmented scenario is achieved within the program Max MSP Jitter, with an integrated inertial sensor and a microphone. As a display, a cinema screen is used. The application was first shown at the SedaJazz Festival 2007 in Valencia (Spain). In first place, a workshop was made in order the musicians to become familiar with the application. Secondly, a live performance was shown.
Augmented teaching, 2007
Augmented Teaching is a new proposal for the e-learning through the Augmented Reality (AR) technology. We have developed two possible applications based on the same idea. The first one is mainly built for youth and children, where a theatre environment is established to live performing a tale. In this so called AR-theatre, children are the actors and the AR is used for the customs. The second one is thought as a complementary tool for history learning for all ages, where the teacher can ‘manipulate’ 3D-virtual ancient tools and show them to the pupils, with the additional advantage that virtual objects are shown with a real scale and related to the human being.
Live LEGO House (LLH) is an interactive Augmented Reality (AR) environment to explore coexistence and multicultural factors through gaming. The system consists basically of a physical/real house built with the LEGO blocks, which is enriched with different non-physical/virtual multimedia files (sounds, videos and 3D animations), insisting in social, cultural and pedagogical issues that the AR technology can bring for children. Tactile, visual and auditory stimuli are present at the game.
Unoo shows a changing image of the reality that affects one of the most important aspects for the construction of the subject, the own identity, that is adulterated, suggesting a set of new possibilities, fruit of the union of the reflected image of the user, along with other forms. Therefore, the user is augmented, becoming a hybrid between the real and the virtual, acquiring so many appearances or identities as different mirrors observe him. When the user moves, he discovers that the time of reaction of his new identity is immediate; he finds himself in front of a mirror that maintains the time of reaction but where his own image has changed, so he interprets and assimilates quickly that what the mirror projects is his new identity: ‘I continue being I, but I recognize myself through the mirror like a different being. This can condition temporarily my behaviour, when trying to move and to act according to the new acquired identity’. These words remind us the reflections of Ascott (1998) saying that each individual is made of a lot of individuals, and that all of we are beings mediated and augmented by the computers.
In Unoo three different identity changes are proposed through three different virtual mirrors that try to investigate in the following aspects:
Emotional Mirror (EM)
Referred to the psychological identity, the true I against the apparent one, the one that is shown to the others. As García Cortés (1997) says, ‘people disguise themselves, hide and do metamorphosis, use artifices, concealments and masks to question their identity, to reinvent it and to give a plural image of their existence’. This duality between the internal emotional state and the expressed emotional state, of being and pretending, is shown in Unoo by means of a game of masks that reveal different emotional states as the mask denies as much as it affirms (Lévi-Strauss 1982).
The head of the users appears as traditional Greek masks, with their factions changing slowly from the happiness to the sadness, from the surprise to the indifference, etc. The hands of them show some texts related to the masks, as for example: ‘in the mask the reality and desire agree, the being and wanting to be’, or ‘when the shape, the face, is only transformation, the mask is the mirror of the soul’ (Villanueva 1995). These texts are divided in such a way, that each user has a part of the whole. Then a game is proposed, in which the different users have to collaborate in order to find the whole sentences by joining their hands.
Stereotype Mirror (SM)
Referred to the physical identity, the physical I pretend or invented, designed to the personal pleasure and for the others. This mirror is based on the posthuman concept (Cruz Sánchez et. al. 2004): on one hand, the posthuman body constitutes a body reconstructed thanks to the aesthetic surgery; on the other hand, a protein body is built by means of the incorporation to our body of diverse devices.
In this mirror the user is shown with a cartoon head that remind to some TV fashion dolls. The factions of these heads appear and disappear, accordingly to the proposed posthuman reconstruction. The hands of the users are related to some words regarding to this mirror, as for example: surgery, diet, fashion, body-pump, piercing, or tattoo. These texts are arranged in some kind of rotating spheres, so users have to pay attention in order to read them, meaning that not always we realize about the imposition of these changes.
Multicultural Mirror (MM)
After being analyzed the identity of the individuals inside the society, this mirror is referred to the social identity, i.e., the sum of individuals within a society. Here it is where the search of the personal identity becomes more evident, in contraposition to the social plurality. Here is where the individual try to find something that identifies him and distinguishes from the others, being part of a cosmopolite society.
Therefore, in this mirror the head of the users appear as a composition of lots of people of different age, sex and race. The 3D model consists of a cube. On each of the cube’s faces, a person portrait is displayed. At the same time, this portrait is segmented into 3x3 squares that are constantly changing into another’s person portrait. Therefore, the result is a portrait composed of different portraits. Their hands show some words related to this mirror, as for example: cosmopolite, plural, racial, woman or sex. The words rotate in a kind of arranged spiral, meaning that we are all part of the same chain.
The following video was recorded in the exhibition Obsevatori 2009. Other virtual characters where modelled for the event.
Notbook AR, 2005
Notbook AR es una instalación artística basada en un dispositivo multiusuario de narración multimedia, realizada utilizando tecnología de realidad aumentada (AR). Propone al espectador explorar los caminos que forman distintas lecturas interrelacionadas, jugando con las diversas pistas que le ofrecen las imágenes 3D y el sonido.
La instalación se desarrolla en un espacio cuadrangular vacío, donde los usuarios se sitúan llevando en sus manos dos tablet PC equipadas, cada una de ellas, con: webCam USB2, sensor inercial y auriculares.
Hot images, 2005
Hot Images is an artistic mixed reality application that deals with the relation between human beings and city environments, thus proposing a novel cartography and navigation tool for the city. Within the virtual recreated environment of the Hot Images, the inert city is closed to the human field, transforming the cold aspect of the city into the hotness of the close friends and relatives. The user is carrying a laptop where a 3D virtual model of the city is presented within a blue color. Some images of past events related to the user are hidden in the city, and s/he has to find them. As the user gets closer to the images locations, the environment changes to red color. Once the user arrives to a location, s/he can catch some images that will appear mapped on some of the buildings’ facades, thus changing the aspect of the city. The location of the user is achieved with a combination of a GPS and an inertial sensor. The application is built with the Max/MSP Jitter software.
El túnel mágico, 2005
El Túnel Mágico es una aplicación de realidad aumentada basada en las librerías de ARToolKit; estas librerías son de libre distribución y se han utilizado en numerosas aplicaciones. La mayoría de estas aplicaciones tratan de mezclar mundos y/o escenarios nuevos, insertando elementos virtuales dentro de la realidad que nos rodea. El Túnel Mágico consta de una serie de pantallas a modo de espejo con las que diversos usuarios pueden interactuar al verse transformados en un personaje pseudo-virtual. Estas pantallas están dispuestas linealmente, a modo de túnel, y el usuario se puede mover por el escenario y así verse reflejado en las distintas pantallas, cada una de ellas aportándole una nueva identidad. En nuestro sistema, tratamos de relativizar alguna de las dualidades o estructuras binarias que conllevan este tipo de aplicaciones en términos de percepción, como por ejemplo:
1. Relación usuario/objeto: el usuario es parte del objeto.
2. Dentro/fuera: incorporación del usuario en el espacio de representación.
3. Referencia al ‘estadio del espejo’, cambiando la identidad del usuario.
Para ello se invierte la disposición espacial de algunos elementos básicos:
1. Normalmente el usuario de aplicaciones de realidad Aumentada establece una identificación entre el display y su visión, identificación incrementada por la localización física de las gafas o HMD (Head Mounted Display) a la altura de sus ojos. En este ensayo el ojo maquinal, la cámara DV está alejado del sujeto.
2. También normalmente el usuario equipado con las gafas observa la(s) marca(s) de realidad aumentada como objeto exterior. En el ensayo la marca forma parte del sujeto.
3. La imagen proyectada funciona a modo de espejo, cuyo "reflejo" es visible a este usuario y otros, la experiencia más que multiusuario es compartida.
Este ensayo se aproxima de forma genérica a la contraposición: "ser es percibir (esse es percipere) / ser es ser percibido (esse est percipi)", incidiendo también en cuestiones de identidad, avatar y juego. El Túnel Mágico es un proyecto en continuo proceso, en el cual pretendemos crear un sistema propio de realidad aumentada que, principalmente, se puede encuadrar dentro de las áreas de diversión, narración y exposición, no descartando en un futuro desarrollarlo para su uso en la educación o, incluso, para fines terapéuticos.
La aplicación Urbanmix, nos posibilita este concepto a escala urbana y en exteriores usando un sistema propio de realidad aumentada, que se basa en modelos 3D y el programa Max/MSP Jitter de edición de video en tiempo real. A través de Urbanmix, podemos acceder a una visión de la ciudad o espacio en el que nos encontremos, ampliada con algunos edificios o monumentos provenientes de otras. Con ello, y utilizando escalas 1:1 en el sistema, podemos realizar comparaciones reales fácilmente y así tomar verdadera conciencia de la magnitud de los edificios que visualizamos.
Ante la gran dimensión de algunos de los edificios que podemos introducir en el sistema, es necesario ubicarlos a una distancia lo suficientemente grande como para que el sistema pueda tomar las referencias necesarias para ubicarlo correctamente y su visualización continúe siendo creíble; por la falta de espacio en las ciudades lo normal es que algunos edificios reales se superpongan generalmente a los virtuales, por una superposición de canales de video, con lo que nos encontramos con un problema de oclusión de formas reales a virtuales que no se corresponde con lo habitual en estos sistemas, oclusión de formas virtuales a reales. El problema se resuelve con una mascara tridimensional de video que actualiza sus datos en tiempo real, que permite al usuario moverse a su antojo dentro del área acotada en donde se desarrolle la aplicación, y continuar viendo la inmersión del edificio o monumento escogido en el espacio precargado desde cualquier punto de vista.
Back to the 70’s is an augmented reality application designed to deliver the Vera University campus back to the city centre as it was 30 years ago. Within this, the user is immersed in a new hypothetical environment where the campus buildings are seen together with some historical ones belonging to Valencia City. The application is built within the Max/MSP software, and the occlusion problem has been solved via a conjunction of 3D objects acting as subtracting models. User pose in real time is achieved via a combination of a visual and an inertial tracker.
Augmented user, 2004
'The Augmented User' is a new Augmented Reality (AR) system based on the ARToolKit and MRXToolKit technologies. We have developed an innovative system in which the user is visible in the AR scenario and indispensable part of the final result: he/she is in fact the carrier of the markers that are recognized by the system allowing his/her own transformation; this opens the possibilities of the users that are immersed in an AR system, being at the same time the augmented matter and the spectator. We also introduce two interesting applications for our system: the 'changing personalities' game and the 'image carrousel' performance, as simple examples of the multiple applications that our system actually have. The collaborative part is also proposed, which makes the applications even more interesting.