Error! Reference source not found. refers to the process that

mediates the visual reality and that may insert virtual objects

into the visual stream)

For the final geometrical registration, the following algorithm was employed, according to

Fig. 2 and Fig. 4 to modify the scenegraph virtual parts as shown in Fig. 3:

1. Retrieve the camera image

2. Run the feature tracker on this image

3. Extract ModelView and Projection camera matrices

4. modify the combined camera matrix according to user authoring scaling/position controls

(mapped virtual trackball mouse metaphor operation)

5. apply the combined/adjusted camera matrix to scenegraph renderer

6. move Occluder geometries as root nodes in the scenegraph with GL-DEPTH test set to

OFF

7. set the background image acquired from the camera in a 2D projected screen textured

quad (thus since background image applied as a texture it is hardware accelerated and independent

of window resolution, as hardware extension for non power of 2 textures was

employed)

8. modify the Main Scene root node according to user preference for further global positioning

and scaling

9. execute draw threads on the whole scenegraph

Thus with the above simple and fast algorithm we were able to both stage our fiducial MR

characters and scene Occluders without any performance drop in frame rate.

4 MR ILLUMINATION GPU BASED MODEL

In Mann & Fung 1998, a “Visual Filter” on top of the R (real-actual) scene is defined as a reality

mediator. In Fung et al 2002, various ‘Reality Mediators’ are described such as thermal

sensor vision, or a diminished mediator who acts by removing parts of the image of the real environment.

In this work (Fig. 4) we propose to apply such Visual Filters independently on both

the R and V parts of the mediated reality spectrum, as well as to their joined common image

space, in the forms of GPU shader programs.

In the following list the main steps of the algorithm are defined:

1. Mediate the Virtual component with an IBL shader

2. Mediate the Real component with an image space processing shader

Mediated Reality

vs.

Stylized Mediated Reality

Real

(Video Image)

Virtual

(Synthetic image)

GPU

GPU

GPU

Real

(Video Image)

Virtual

(Synthetic image)

Visual filter

Scenegraph view

Main

Camera

Rotation,

Translation

Matrices

Virtual world (augmenta-

Real

Real

Oc- Virtual

Back-

Virtual

Camera

space

3. Transform and register both virtual and real in a common image space by a “Render to

Texture” operation

4. Further mediate the combined integrated common image space with a stroke-based NPR