Testing variables: Representational and Abstract

Experimenting with Adobe After Effects software, I have been theorising the ramifications of the different styles of graphics included in the virtual. Through experimentation, analysis and valuation, I have deduced that the creation of the hypersphere (the co-dimensional performance space) is affected by another variable: realistic representation and abstract graphical objects. The world of the virtual – synthetic, made of text, mathematics; not a representation of the real, but no less part of the real. Steven Parrella writes: ‘The division between real and virtual, an operative division, is illusory. The virtual is an extension of ourselves into a manufactured and constructed space. It is not a separate space but an extrusion of being. The virtual recreates the specific and local conditions of our bodies and projects. Virtuality represences our work through the technotranslation (coding) of our actions’ (Parrella 1997: 64).

Representation of the real, on the other hand, whilst maintaining its attachment to the real ontologically, is all the while twice removed from the real.

As a result of such theorisation, I hypothesise that the inclusion and oscillation between the varying degrees of representation and the virtual will impact on the virtuality of the hypersphere.

So, as you can see, I propose that the more representational (closer to the real) the graphics, the more augmentation within the nexus of interconnections within the hypersphere. The more abstract (closer to the virtual) the graphics, the more virtuality within the hypersphere.

Using literary sources and artistic idea generation, I have tested the results of each variable using Elleström’s model for media analysis, to determine what medial components affect the ontology of the hypersphere.

Observing Change

To determine the transformation properties of the media at play, I have analysed the results from my perspective, using reflection in and on action to determine results. To begin, I observed the material, sensorial, spatiotemporal and semiotic values of the hypersphere including realistic visual graphics. Particular attention was directed towards the spatiotemporal, as I hypothesised that it would be that modality that would be most greatly affected by this variable. I then repeated this process for the more abstract elements of the visual graphics. The results of this test do not rule out one variable over the other; but merely observes the changes in the ontology of the hypersphere as a result. As such, each variable does not result in failure to produce the co-dimensional hypersphere, but consequently changes its position amongst the virtuality continuum.

---

Realistic representation

---

Material

The hypersphere consists of both static materials and a fixed sequence of moving images.

Sensorial

The image is primarily perceived visually, but contains associational sense-memory data. I indirectly feel the tactile qualities of the space through sense-memory data. In this way, the hypersphere maintains its material dimension and attachment to the real. The movement of light within the hypersphere seems to affect the tactile qualities of the hypersphere; as the pixels adjust the visual data thrown out by the hypersphere, the indirect tactile qualities of the real illuminated by the virtual consequently change. In these lighting conditions, it is difficult to discern between the visual data of the real and the virtual in isolation (a crucial component of the ontologically whole hypersphere), and as such the sense-data of the real and the virtual, in being mutually affective, lose their independence; in that we, as perceiving human agents, can no longer discern between the real and the virtual any longer. Crucially, the tactile qualities of the hypersphere in this representational experiment draw their sense-memory data from the real world. In this way, our sense-memory is awakened through our previous tactile contact with such surfaces. We associate the perceived texture quality of the image with our memory of similar surfaces.

Spatiotemporal

The virtual, in its spatial perception of 3-dimensional depth, is embedded into the 3-dimensional real space due to the lack of light illuminating the real space. The perception of depth is now reinforced by the actual depth of the space, which thus creates a meta or hyper textual contamination between the real and the virtual. The real lends actual depth to the virtual, and the virtual lends a perceived, augmented depth to the real. Such an extension of space into the hypersphere complicates the church’s real spatial co-ordinates and Euclidean space. The hypersphere exists within it’s own dimension, and through the isolation of visual sense-data to the hypersphere alone, it is thus deterritorialised. Crucially, it appears and disappears in a number of perceived locations that take their spatial construction from that experienced in the real world. Some images are drawn from the real world itself, through photograph (including those of the church), however some are created using graphical rendering to mimic the real world. In this way, representation can include images that are purely virtual, in that they are entirely synthetic. Spatially, however, images that take their spatial construction from the real world are remediated, translated and constructed from the virtual. But, due to the hyper nature of the hypersphere, in being something above and beyond the virtual and the real, in it’s mutually affecting properties and shared ontology, the virtual can no longer exist as a separate entity. As such, they are deposited back into the real, but cannot be regarded as real because it is deterritorialised, extended and hybridised. 

Temporally, notions of clock time are represented through perceived lighting conditions: the representation of a night sky, for example. These temporal flows interact with, converge and diverge with the atemporality of the hypersphere as a sequence of moving images; images which do not belong to any GPS location, but which tie themselves to the GPS location of the real in being situated there. The notion that the hypersphere is located in the spatiotemporal framework of the church is complicated by the hybridisation of the hypersphere’s spatiotemporal framework. Images from unknown places in the world that were captured in the past and presented in the present, now intertwine with the space of the real, in the present. But because the real is no longer an isolated component of the hypersphere, these images exist within themselves; they are atemporal; their ontology is that of something above, beyond, in excess of the real and the virtual. Within such atemporality, any linear notion of time no longer applies. Spacetime interconnects through a nexus of disappearing trajectories that span past, present and future. As we take the leap from linear, clock time to the next, atemporal moment, our experience of time is no longer that of a continuous present, but a continuous future. All space and time inside the hypersphere appears, disappears, converges, diverges, interacts, flows, shifts and resonates as something other, beyond and above. 

Semiotic

The use of realistic representation seems to bring forth elements of the real, from various times and spaces, and fuse them with the real church space. Whilst this is a simulation, the effects of the church’s spatiotemporal and ontological transformation within the hypersphere is nonetheless real. Whilst the hypersphere is something above and beyond that which is real and virtual, our reading of the signs within the hypersphere are related to those which exist within the real and as such cannot the real via semiotic association. We will always take meaning from the realm of the real world, and as such the hypersphere will always retain a semiotic connection with the real. 

---

Abtract virtuality

---

Material

The hypersphere consists of both static materials and a fixed sequence of moving images.

Sensorial

The image is primarily perceived visually, but contains associational sense-memory data. I indirectly feel the tactile qualities of the space through sense-memory data. In this way, the visual data of abstract graphics still retain the visual texture and indirect tactile qualities of the real components that it illuminates. As light moves to display various abstract flows of information, the texture of the real space is relocated within the luminosity of virtual, abstract space and vice versa, and as such the sense-data of the real and the virtual, in being mutually affective, loses its independence; in that we, as perceiving human agents, can no longer discern between the real and the virtual any longer. In contrast to the representational experiment, we can no longer associate the indirect tactile qualities of the hypersphere with sense-memory data, because the abstract does not exist as a tactile quality of the real; it is but a manufactured and constructed extension of ourselves into virtual space.

Spatiotemporal

The virtual, in its spatial perception of 3-dimensional depth, is embedded into the 3-dimensional real space due to the lack of light illuminating the real space. The perception of depth is now reinforced by the actual depth of the space, which thus creates a meta or hyper textual contamination between the real and the virtual. The real lends actual depth to the virtual, and the virtual lends a perceived, augmented depth to the real. Such an extension of space into the hypersphere complicates the church’s real spatial co-ordinates and Euclidean space. The hypersphere exists within it’s own dimension, and through the isolation of visual sense-data to the hypersphere alone, it is thus deterritorialised. Crucially, in contrast to the representational experiment, the abstract holds no origin in GPS location or clock time; it is wholly synthetic. The ties to the real, here, are through the hypersphere as a co-dimension, rather than through the virtual as representation of the real. This relationship exists through the hypersphere’s relationship to the real, in that it embodies elements of the real in it’s complex web of spacetime in a shared ontological dimension. The spatial construction of these graphical elements are something other; and as such they place the hypersphere further away from reality on the mixed reality continuum than that of representation. This dimension seems to have no Euclidean boundaries, no fixed GPS position, and thus removes the experiencer from the world of representation (the real) they have left behind. 

Temporally, the abstract graphics contribute to the atemporality of the hypersphere quite significantly. In being synthetically removed from the spatiotemporal framework of the real (insofar as they are non-representational), they exist as myopic visions of an atemporal dreamworld; they do not represent clock time and exist purely as virtual time. These temporal flows interact with, converge and diverge with the atemporality of the hypersphere as a sequence of moving images; images which do not belong to any GPS location, but which tie themselves to the GPS location of the real in being situated there. The notion that the hypersphere is located in the spatiotemporal framework of the church is complicated by the hybridisation of the hypersphere’s spatiotemporal framework. Images from the virtual world now intertwine with the space of the real, in the present. But because the real is no longer an isolated component of the hypersphere, these images exist within themselves; they are atemporal; their ontology is that of something above, beyond, in excess of the real and the virtual. In contrast to representation, abstract graphical objects go further in this direction; they detract the hypersphere further away from reality than the reality of appearances in the representation experiment.

Semiotic

The use of abstract graphical elements as metaphors for reality (or rather, the reality beneath appearances) seems to bring forth elements of the other, as if from nowhere, and fuse them with the real church’s spacetime. Whilst this is a simulation, the effects of the church’s spatiotemporal and ontological transformation within the hypersphere is nonetheless real. The hypersphere, now further beyond that which we consider to be real in this abstraction, gives it’s signs from the world of the mystical, the dream-like, the synthetic and manufactured, and in this way, is associated as being separate from the real. Whilst this is a false distinction, signs here, seem to originate from ambiguity, ephemerality, metaphor, poetics and symbol. 

---

Conclusion

As hypothesised, the inclusion of abstract graphics spatiotemporally and semiotically shift the hypersphere closer to the virtual end of the mixed reality continuum. Representation, in reproducing the image of external reality through remediation and translation, is closer to augmentation than the virtual, in that they draw their spatiotemporal and semiotic construction from our experience in the real world of appearances. In being hyper, these connections to the real are hybridised with a nexus of shifting interconnections within a co-dimensional experience, and as such are never truly tied to the real world of appearances. Such is the mutually interdependent and affecting nature of the hypersphere. Abstraction, in acting as metaphor for reality; in being a synthetic extrusion of our being and our reality, is made of text, symbol, and is closer to the inner realm of poetics, experience and metaphor; a domain that further transcends spacetime and the reality of appearances. As Parrella points out, ‘[w]e explore the unknown through a superpositioning of the known’ (1997).

Testing variables: geometry

Using my findings from the light experiment, I am now going to experiment with another variable: geometry. These experiments will take place at night, so that the creation of the hypersphere can take place.

Observing Change

Whilst the light experiment focused on the change in medial behaviour through the subjugation of the hypersphere, this experiment observes the changes that take place within the hypersphere as a whole. Whilst, I propose, that such changes will duly affect the modalities of the real and the virtual if I were to analyse them in isolation, this experiment will draw its findings from the hypersphere as a singular ontological co-dimension. Using Elleström’s model of media modalities as a framework of analysis, I observed the modalities of the hypersphere in two situations. In the first, the real and the virtual were brought together geometrically; i.e., the spatial co-ordinates of the real and the virtual were overlaid so that both co-ordinates aligned. In the second, the co-ordinates of the virtual were misaligned with the real. In this way, I can extrapolate what it is exactly about geometric alignment that allows for the creation of the hypersphere.

---

Geometrically misaligned hypersphere* (or, is it?)

---

 

Material

The hypersphere consists of both static materials and a fixed sequence of moving images.

Sensorial

The virtual, with it’s visual perception of 3-dimensional depth, can be seen as an overlay on-top of the spatial co-ordinates of the real space. The perception of depth within the virtual can be seen within it’s textual framework, and thus the hypertextual sphere cannot be created. Instead, the virtual illuminates areas of the real that do not correlate to the representation of those areas as seen in the virtual. In this way, the real is subordinate to the virtual; it is incomplete. The virtual, which can be seen as a complete image, dominates the problematic, and as such, the subjugation of textual sensory information between the real and the virtual does not occur.

Spatiotemporal

As a result of the virtual’s inability to illuminate the whole image of the real, as well as the division of dimensionality between the two, the real is now placed within the virtual. The virtual, exists in and of itself , and would behave the same way if presented within its traditional interface (e.g. a screen). The real, by contrast, is now situated within the ontology of the virtual. It appears to do so, paradoxically, with the ability to retain it’s own ontology. Let me break this down further. The church space exists within 4 dimensions (height, width, depth and time), and continues to do so regardless of whether the virtual illuminates said space or not. In this way, it retains its own ontology continually (as long as there are human agents to perceive such dimensions). The virtual, again, exists within its own ontology. However, the primary sensory stimulant is light, which brings to view both the real and the virtual. With this being the primary source of perception for each spatiotemporal framework, the real exists largely to perception through the virtual. The virtual is the dominator here; it dictates how much of the space we see. Temporally, the clock time of the real is now situated within the virtual, but because the two are not mutually affective spatially, they cannot be temporally.

Semiotic

The signs of both the real and the virtual, although identical, are both shown separately. Separate in that the signs of the real are either read as incomplete, or not at all. The virtual signs dominate, here, and thus signs are not read from the hypersphere, but from the real and the virtual.

---

Geometrically aligned hypersphere (yup, I think it is!)

---

Material

The hypersphere consists of both static materials and a fixed sequence of moving images.

Sensorial

It is perceived visually, but contains associational sense-memory data. The virtual lights up the areas of the real of which it depicts, achieving a satisfying correlation and successful contamination of textual sense-giving data. Both images are seen to be complete.

Spatiotemporal

The real and the virtual, now giving the full amount of sense-data possible due to geometric alignment, are now placed within each other; i.e., they are both mutually affecting one another and creating a new, extended spatiotemporal dimension. The perception of depth is now reinforced by the actual depth of the space, which thus creates a meta or hyper textual contamination between the real and the virtual.The real lends actual depth to the virtual, and the virtual lends a perceived, augmented depth to the real. This perception, however, is limited to what is being lit by the projector, which is the whole real image, or the whole as it correlates to the virtual image. Because of these amendments to each, they both now behave differently because of geometric alignment, and thus contribute towards the formulation of the hyper within the hypersphere.

Semiotic

The identical nature of the iconic sings within both the real and the virtual draws attention to the medial interface of the virtual. The real, however, being located spatiotemporally within the hypersphere, is no longer read. The hypersphere gives its signs; not the real; not the virtual. Within the hypersphere are the same iconic, symbolic and indexical signs included in both the real and the virtual, but they not converge as communicate via the same medial interface: the hypersphere. Similarly to both the real and the virtual, the hypersphere still submits to darkness as a sign.

---

Conclusion

The materiality, as with my previous experiment, was unaffected by any change in geometric alignment. Sensorially, how we perceive is unaffected, but what we see, i.e., what parts of the real, are. Spatiotemporally, geometric misalignment produces an overlay of virtual over real, but paradoxically, the real is now situated within the real, whilst retaining its own ontology. Geometric alignment introduces a new dimension of space; one that conforms to neither the real nor the virtual. This is because the real and the virtual are illuminated by the projector together, and so the real only appears to us via the luminosity of the virtual, and the virtual only appears to us within the real (which is now located within the virtual). This paradox affirms the hyper nature of the hypersphere; that is can no longer exist as the converge of ontologically separate real and virtual, but as something other, beyond, above. Semiotically, the alignment of the real and the virtual within the hypersphere allows for a whole reading of signs, and signs are now being given by the hypersphere, and not by the real and the virtual.

---

Testing variables: light

Methodologically, this project engages with a process of practice-as-research. That is, I am using the processes of creation to generate knowledge. This involves the creation of a framework through which to scrutinise the critical context of projection-based mixed-reality performance.

I began with a hypothesis: that the integration of projection-based graphical media into a physical, real-world environment has the potential to create a hybrid structure of space and time, as well as integrate itself into the real-world environment in such a way that both the real and the virtual become interdependent.

As such, I have sought to garner evidence to back up this theoretical claim. My methodology has thus far taken place as follows:

From an initial gathering of resources, mostly literary, I formulated a hypothesis that duly informed the methodological processes used to generate knowledge, have made valuations of such processes (in-action and on-actions), to refine and determine what the framework for testing this hypothesis actually is in a performative outcome.

I have realised that the process of creation is difficult to monitor without the concretisation of a set of variables, so that my hypothesis can ‘be tested not just for success, but against failure, otherwise it will remain just a statement of faith’ (O’Toole 2006: 52). Whilst I have been engaging with this methodology throughout the duration of the project, my findings have been presented in a conceptual manner; i.e. I have not made explicit what variables have determined such conclusions. I am now going to reflect on my methodology to deduce what the variables are at this stage in the research process.

Observing Change

In engaging with experimental research, my task is to observe a change in behaviour, to determine the transformational properties of the media at play within the hypersphere. This observation takes place from the maker’s perspective, and as such valuations have been drawn from reflections in-action (in vivo findings, which are being drawn from field-notes) and reflections on-action. Using Lars Elleström’s four modalities of media as a framework for enquiry (sensorial, semiotic, material, spatiotemporal),  I observed the modalities of both the real and the virtual in isolation, and will summarise my findings below. I chose Elleström’s model through which to measure the medial abilities for hybrid time and space and ontological oneness because Elleström’s model presents ‘a theoretical framework that explains and describes how media are related to each other: what they have in common, in what ways they differ and how these differences are bridges over by intermediality’ (Elleström 2010: 12). By working with intermediality, Elleström’s model is the most insightful in determining the medial differences of the real and the virtual under varying conditions.

Next, I observed the modalities of the hypersphere, which will also be summarised below, and have measured the change in behaviour to determine the affecting properties of this mixed-reality environment. In this case, I have conducted my analysis of medial modalities using light as a variable. In this way, I can determine what conditions of light are required to achieve a hybridised spatial and temporal co-dimensional performance.

---

Let me begin with my articulation of a simple test. In the church, I conducted the same experiment twice in varying lighting conditions. In order to isolate the variables in this test to lighting conditions alone, I controlled several aspects of the experiments. Firstly, the experiments were conducted at the same time, to control the differences in natural light that occur at different times of day. Diffusion of light throughout the church, however, was unavoidable. Secondly, the projected surface was the same material: stone. Imperfections in each surface varied, and as a result varying levels of diffusion occurred on a micro level. And thirdly, the focus of the projector was adjusted so that each image was in focus.

I found out that proximity and angle of projector affect the luminosity of the projected image:

Projector position: Close to projection surface

When placed close to the projection surface, the light of the projector shines brightest. In this position, however, the spatial coverage of the projector over real space is not sufficient.

Further away from projection surface

Further away this time, the luminosity of the images decreased.

Far away from projected surface

At this distance, the coverage of the projector is sufficient, but luminosity has decreased further. In this respect, the luminosity of the projector cannot be controlled any further.

Conclusion:

Due to the spatial and practical requirements of the projector’s position, the consequent luminous affect of projector position is now a fixed property. Consequently, any variances in the projector’s ability to hybridise space and time and ontologically fuse the virtual with the real space could not be refined or controlled via projector position. As such, the variable to be measured in this experiment is the natural lighting conditions of the space. To begin, I measured the medial modalities of the same image in daylight (mid-afternoon in summer, when the sun is high and luminosity at one of its strongest), and then at night, a good deal after sunset, when there is no natural sunlight at all.

---

The virtual

---

Daylight

My analysis of the medial modality of the virtual is contained to its original interface: the computer screen. These graphics have been created on a laptop computer, and observing them in isolation involves their display through their original format. In both tests, the brightness control on the laptop was the same.

Material

‘The material modality can thus be defined as the latent corporeal interface of the medium’ (.: 17).

The material interface of the virtual consists of a 2-dimensional surface of changing images, generated by pixel-based light, which is in-turn programmed by the computer’s hard and software.

Sensorial

‘The sensorial modality is the physical and mental acts of perceiving the present interface of the medium through the sense faculties’ (.: 17).

The brightness is controlled by LED’s in the computer’s material interface, which allows for the generation of sense-data to be perceived by our eyes. In daylight, the image is strong and visible. Our sense of sensorial memory indirectly creates a sense of what the scene depicted (semiotically) on-screen feels like, what it might smell like, what the environment might sound like. We position ourselves in relation to the space the virtual scene depicts: we sit before it, but we cannot reach out and touch what our inner sense experience; we can only touch the interface. In this respect, the virtual window exists in sense-data only by way of the laptop’s materiality: we see the LED lights, the assimilation of pixel-based light in varying patterns that are received by the eye.

Spatiotemporal

‘The spatiotemporal modality of media covers the structuring of the sensorial perception of sense-data of the material interface into experiences and conceptions of space and time’ (.: 18).

The virtual consists of four dimensions: width, height, depth and time. The fourth dimension, time, is not incorporated in its physical manifestation; that is, through time, the computer screen does not change. Whilst this can be widely accepted as the case, it is the manifestations of light as a waveform that changes, and thus our sensorial cognition of the virtual changes through time as the light causes the image to materialise (to the senses). The medial progression takes place over a fixed sequentiality.
Spatially, the virtual achieves it’s 3-dimensionality from our perception and interpretation of said space. The materiality of the screen consists of two dimensions: height and width, but it is through perception that the illusion of depth is created.
Temporally, the interface is realised in time but does not have a temporal interface. Virtual space, however, comes into play here. We can interpret the space around the window; visualise the surrounding space with our inner senses. The representation of the virtual takes place over time, but represented in the media is virtual space. We cannot, however, hear anything. In this respect, our sense of virtual space and time is limited to our visual perception of it, and not our aural immersion in space over time.

Semiotic

‘[M]eaning must be understood as the product of a perceiving and conceiving subject situated in social circumstances’ (.: 21)

What our inner-senses see is a window – our memories (personal, cultural) of similar windows, our experiences of windows, which in turn position us in relation to the window spatiotemporally – are all generated from the materiality of the laptop (which, in itself, contains a web of semiotic references). By receiving the sense-data of the light emitted by the laptop, we associate these senses with our experiences of a window in space and time. The image depicted is iconic, in that we perceive the window to be a part of the wider, natural world we perceive in waking life in spatiotemporal structures. The window signifies hope; I associate it with notions of new possibilities.

---

Night

Material

The material interface of the virtual consists of a 2-dimensional surface of changing images, generated by pixel-based light, which is in-turn programmed by the computer’s hard and software. This does not change during nighttime, when lighting conditions impair our vision. Regardless of whether we can see the laptop or not, it’s ontology is not affected by light.

Sensorial

The brightness is controlled by LED’s in the computer’s material interface, which allows for the generation of sense-data to be perceived by our eyes. At nighttime, the image is still strong and visible. This time, however, our visual sense of the wider world around us now fades from our view. The virtual dominates our eyesight. We can still perceive the wider world around us through our other sense-gathering organs, but the virtual, at night, dominates our sensorial attention.

Spatiotemporal

Interestingly, there appears to be change in our perception of virtual time and space when observed at nighttime. Materially, the spatiotemporal stays the same. Perceptually, the sense of depth created in the image stays the same, in that the image itself does not alter the image which allows for the perception of depth to be created. Virtually, however, our sense of real time somewhat affects our inner perception of virtual time. Observing the window at night, I could not help but imagine a sense of virtual night time and space. The sense-data of the virtual was not altered, but my perception of the virtual was largely influenced by my position in real space and time. Once this change had been observed, I realised that my sense of virtual time and space was also affected by day time. In this way, virtual time and space seems to run parallel with real time in that my perception of virtual time is affected by my immersion in real time and space. 

Semiotic

The window, in the most basic of semiotic interpretations, still signifies a window in it’s purest sense. However, the semiotic value of the absence of light was the main observation in the semiotic perception of the virtual. That is, through augmentation of real-world lighting conditions, my perception of the virtual (made possible through cognition and perception in a real-world environment) was radically different because of the real-world conditions in which it was presented. The window seemed to take on a gothic character as I associated the stained-glass windows with gothic literature and other such connotations. In this way, it was difficult to separate my semiotic reading of the virtual from the conditions in which it was presented. Likewise, the reading of the window in daylight was much the same.

---

Conclusion

Of the four media modalities Elleström outlines, I observed changes in three: the sensorial, the spatiotemporal and the semiotic. This change is due to our perception of the virtual media; the elements of media that belong to perception were varied and those that belong to the material observed no changes. The sensorial recorded the smallest of changes, notably because of the sensorial’s reliance of sense-data (which, in-turn, is reliant on materiality), which stayed the same. The absence of natural light, however, made the visual element of the virtual dominate, due to the absence of real-world visual objects in my visual cognition of the virtual. Whilst the spatiotemporal aspects of the media’s presentation over time and in space recorded no change, virtual time, which belongs to perception, noted a change due to the absence of representation clock time in the sense-data of the media; thus my sense of virtual time was somewhat influenced by real time. Space, in this respect, was affected due to the lighting conditions of both day and night. Finally, the semiotics of day and night in isolation fed into my semiotic reading of the virtual. In this way, the spatiotemporal conditions of presentation (the variable) carries with it semiotic readings that are difficult to ignore. The presentation of the virtual is reliant on the real to present itself: it is impossible to observe the virtual’s ontology outside of the ontology of our perception. In this way, it is inextricably linked to the real, but crucially, it’s ontology can still be observed as separate.

---

The real

---

Daylight

Material

The inside of the church has a ‘3-dimensional, solid and static material interface’ (.: 23).

Sensorial

It is perceived visually, through smell and it also has tactile qualities. The space also opens up a realm of sense memory, particularly in smell, in a complex web of perceived and conceived sensations.

Spatiotemporal

The church has three dimensions: height, width and depth, which are made available to my perceptions via the above cognitive functions. I can sense where I am in relation to the space around me because of the lighting conditions of the space. Whilst the presentation of the church takes place over time and in space, we can only experience the church’s temporality in our perception of it. Whilst the church had a beginning, and will certainly come to an end, the spatial form remains intact throughout our experience of it through time. Therefore, the church’s temporality exists in virtual time; our perception of real space and time is linked with a sense of virtual space and time in our perception – we imagine the church’s journey through time, past, present and future – time exists here as ‘an interpretive aspect of what the medium represents’ (.: 21).

Semiotic

In this modality, the space throws out a plethora of rich and varied semiotic readings: symbol, index and icon. Symbolically, the space is filled with religious iconography; from the altar, to one of the most widely recognised signs in the world, the christian cross. Indexical signs such as scratches in stone, or the burnt-out candles fill the space, linking to a series of forgotten narratives that are inextricably linked to the spatiotemporal. Iconically, stained-glass depictions of biblical tales fill the windows of the space, representing people and the places from a space and time of the past.

---

Night

Material

The inside of the church has a ‘3-dimensional, solid and static material interface’ (.: 23). This does not change when the natural lighting conditions have changed in nighttime.

Sensorial

The space is now only perceived partially in comparison to the daytime experiment – through smell and through touch. The space still opens up a realm of sense memory, particularly in smell, in a complex web of perceived and conceived sensations, but acknowledgement of the space’s materiality through vision can only take place through sense memory.

Spatiotemporal

At night, the church’s three dimensions remain the same. Perceptually, however, my visual cognition of the space is impaired, and consequently the space is limited mainly to tactile responses, which cannot determine my positioning within the whole space. I can feel the wall around me, but I cannot simultaneously feel the wall across the room, and therefore my positioning in space is lost. My perception of virtual time shifted somewhat due to the clock time of this experiment; my perception of virtual time included an interpretation of the church’s many midnight mass ceremonies; I began to interpret the spaces in night time that the church had previously occupied. Similarly to the virtual, it seems that, in this case, my spatiotemporal virtual perception of media is altered by my spatiotemporal positioning in reality.

Semiotic

Crucially, the semiotic reading of the space is limited by my visual cognition of it; if I cannot see the cross or the altar, I cannot interpret their signs. Whilst I can see outlines of forms, they are hard to discern. In this way, the absence of light overpowers any semiotic reading of the space, and as such, the space submits to darkness as a sign.

---

Conclusion

Of the four media modalities, change was once again registered in three of four: sensorial, spatiotemporal and semiotic. Again, these changes were registered in the perception of media. The materiality of the church registered no change, however, due to the change in sense-data availability, our perception of that materiality was limited to interpretation and sense memory. Sensorially, at night I could not see the space, but could still smell and touch it. As a result, my spatial positioning within the space was somewhat limited to tactile responses, which thus limited my sense of location to my immediate surroundings. Temporally, my perception of virtual time was affected by my positioning in reality. Semiotically, the space could only be read in daytime, whilst at night it submitted to a reading of darkness as a sign.

---

The hypersphere

---

Daylight

Material

The hypersphere consists of both static materials and a fixed sequence of moving images.

Sensorial

It is perceived, at present, visually, but contains associational sense-memory data. I indirectly feel the tactile qualities of the space. The moving images, I propose, will invariably feel the same to touch. The moving light images have a similar luminosity to the surrounding environment, in that they do not draw the eye’s attention, and parts of the virtual could easily be missed.

Spatiotemporal

The virtual, in its spatial perception of 3-dimensional depth, is now embedded into the 3-dimensional real space. The perception of depth is now reinforced by the actual depth of the space, which thus creates a meta or hyper textual contamination between the real and the virtual. The real lends actual depth to the virtual, and the virtual lends a perceived, augmented depth to the real.
Temporally, the virtual’s absence of clock time (that is, within it’s temporal representation, and not presentation) is filled with the clock time of the space. This positions the virtual within the temporal framework of the real. The virtual time of real space is unaffected by the addition of the virtual; I can imagine the church’s journey through history without the presence of the virtual. In this way, the addition of the virtual affects clock time only. The virtual’s absence of clock time in representation, introduces disjunctive temporal frameworks in my perception of clock time. As the virtual moves through virtual time, the real now moves through virtual time also. As with the above spatial contamination, both ontologically separate temporalities conjugate in a co-dimensional realm.

Semiotic

The hypersphere introduces a whole new realm of semiotic interpretations once both the real and the virtual converge in a single totality. The virtual, in isolation, denoted a church window, because of it’s iconic sign. Now, with the addition of the real space, the placement of the virtual within the real seems to draw attention to the virtuality of the virtual; that is, its ontology. Because both the real and the virtual signify the same thing, attention is now drawn to the other modalities: their materiality and ontology, their sense-giving data, their spatiotemporality. The hypersphere makes their medial interface more explicit, as they are brought together. Crucially, however, because of the spatiotemporal conjugation of both real and virtual, the two now share the same ontology, and therefore representational signs are drawn from the same sphere.

---

Night

Material

The hypersphere consists of both static materials and a fixed sequence of moving images. The materiality has not been affected by absence of light.

Sensorial

It is perceived, at present, visually, but contains associational sense-memory data. I indirectly feel the tactile qualities of the space. The moving images, I propose, will invariably feel the same to touch. The moving light images are now bolder and brighter, with a greater luminosity. The effect of this on my attention is noticeable; the image draws my attention to what is being lit by the virtual and only that. My eyes do not dance around the space, they are drawn directly to where the virtual is.

Spatiotemporal

The virtual, in its spatial perception of 3-dimensional depth, is now embedded into the 3-dimensional real space. The perception of depth is now reinforced by the actual depth of the space, which thus creates a meta or hyper textual contamination between the real and the virtual. The real lends actual depth to the virtual, and the virtual lends a perceived, augmented depth to the real. This perception, however, is limited to what is being lit by the projector. This takes my sensorial attention away from the rest of the space and its materiality, and repositions real space within virtual space. The difference between day and night here, however, is that the positioning of virtual space within real space takes place from a perspective that is selected by the mutually affecting hypersphere. That is, that the interface of the real only exists within the virtual by luminosity, and the virtual within the real (which is now located within the virtual). This paradox affirms my hypothesis that within the hypersphere, the multiple acts of representation are not heterogenous, but co-dimensional. In this respect, it is clear how we can refer to the hypersphere as hyper: other, beyond, above. 

Because there is no longer any clear division between media (spatially) at night, the virtual’s absence of clock time comes to the fore. Whilst the hypersphere is presented in clock time, it’s representational, virtual time exists within the atemporality of the virtual. Because, spatially, the real is now located within the virtual, the real, by extension, now sits within the atemporality of the virtual.
Unlike the changes recorded from day to night in the virtual experiment, the hypersphere is not as influenced by clock time as the virtual was in isolation. This now seems due to the co-dimensionality that darkness has now sensorially afforded.

Semiotic

As during the day, the identical nature of the iconic sings within both the real and the virtual draws attention to the medial interface of the virtual. The real, however, being located spatiotemporally within the hypersphere, is no longer read. The hypersphere gives its signs; not the real; not the virtual. Within the hypersphere are the same iconic, symbolic and indexical signs included in both the real and the virtual, but they not converge as communicate via the same medial interface: the hypersphere. Similarly to both the real and the virtual, the hypersphere still submits to darkness as a sign.

---

Conclusion

Once again, the materiality of the hypersphere was unaffected by the absence of natural light. Sensorially, the absence of light shuts our attention off from the visual sense-data of the real, reinforcing the co-dimensionality of the hypersphere. Spatially, the absence of light draws attention away from the entire space and introduces a new dimension of space; one that conforms to neither the real nor the virtual. This is because the real and the virtual are illuminated by the projector together, and so the real only appears to us via the luminosity of the virtual, and the virtual only appears to us within the real (which is now located within the virtual). This paradox affirms the hyper nature of the hypersphere; that is can no longer exist as the converge of ontologically separate real and virtual, but as something other, beyond, above. This notion is continued temporally. The hypersphere’s virtual time is not influenced by clock time in this way, as was the virtual’s virtual time in the previous experiment, because of the atemporality of the real now. In this way, clock time now somewhat exists simply as a perceived notion of virtual time: we acknowledge it’s existence, but within the hypersphere, it ceases to exist. Semiotically, signs are drawn from the hypersphere at night. Unlike the day, signs are not drawn from the real and virtual modalities.

---

Summary:

The relationship between the hypersphere and natural light

As a result of my findings, it is clear to see that the creation of the hypersphere as a newly-created ontological entity which eliminates the oppositional ontological frameworks of the real and the virtual as separate media, could only take place at night. This is largely down to the sensorial differences between day and night; being largely a visual medium, the absence of light reduces the sense-giving data of the space. The virtual, which sensorially drew my eye’s attention more at night, allows the real to enter into it, contaminate it, and create a co-dimensional performance space. As a result, I have chosen to show the performance at night. In day, the ontological difference between the real and the virtual would have been too great, and the interdependency of both the real and the virtual would be limited in their spatiotemporality.
Darkness allows the hypersphere to dominate the problematic; in this way, neither the real nor the virtual do. Whilst time was linked to light for the real, for the virtual, this was not the case. It existed separate from clock time. The hypersphere, being atemporal, by extension also exists separate to clock time. As a result, clock time ceases to exist in the hypersphere. For the hypersphere, however, time is linked to light because without light, we are sensorially blind. We cannot see space, it does not travel as a visual entity through time. Light is linked to time for the hypersphere.

Defining ‘hyper’

As I have realised, the hypersphere as a medium is a transformation tool, one which fuses onto the space around it and refuses to let go; in becoming interdependent with the real, one becomes ontologically immersed within the hypersphere.

I spoke in a previous blog about place as practiced space, about how our actions and our presence define the environments we are situated in, and how the hypersphere, being an ontological spatio-temporal extension of such space, becomes practiced in much the same way. The focus here is on immediacy, not a distance between media and immersant. The virtual shares the same reality with the space and alters it, a space in which the immersant, due to the technological alteration of the projector, can navigate due to the shared ontology of the real and the virtual. As a consequence, our presence – that is, our psychological, perceptual and cognitive condition in space and time – is now involved and attached to the medial stimuli of the hypersphere (Mestre, 2005).

In this respect, it is clear that the hypersphere – in its interdependency, its immediacy, its ontological oneness, its alteration of reality – seeks to create an environment that is whole; that is, one which does not make the process of mediation explicit or available for deconstruction; it does not isolate one’s presence within any constituent part. Let me explain this using Bolter and Grusin’s (1999) articulation of hypermedia.

Using the words of William J. Mitchell (1994), Bolter and Grusin begin by suggesting that hypermedia ‘privileges fragmentation, interdeterminacy, and heterogeneity” (8). They go on to suggest that:

‘The logic of hypermediacy acknowledges multiple acts of representation and makes them visible. Where immediacy suggests a unified visual space, contemporary hypermediacy offers a heterogeneous space, in which representation is conceived of not as a window on to the world, but rather as “windowed” itself – with windows that open on to other representations or other media. The logic of hypermediacy multiplies the signs of mediation and in this way tries to reproduce the rich sensorium of human experience’ (Bolter and Grusin 1999: 33-4) 

If heterogeneity is one of the defining feature of the hypermedium, then how does the hypersohere fit into this framework? If both dimensions, the real and the virtual, are to share the same ontology, then how can projection mapping, in integrating into the physical environment, be heterogeneous? Granted, the constituent parts of the hypersphere are heterogeneous, but once they come together in performance, their diverse characteristics are no longer made available for us to observe.

I like their use of the word ‘patchwork’ (31), which suggests that the composite elements of the hypersphere, the forces that bring such an event into existence, have been stitched together to create a final image. Whilst this is unavoidably the case, a patchwork creates an image from which one can determine each separate material; one can see the stitches, can un-stitch them, can observe one pattern singularly. In the case of the hypersphere, as articulated above, this cannot be done. The transformational affect of the digital media at play is such that the spatio-temporal framework into which the real usually rests is no longer singular and separate as a visible or tangible layer; it is acknowledged as an element of the hypersphere, but it travels along with the virtual in such a way that is is incomplete without it.

In keeping with the concept of the piece, it is now appropriate to think of the hypersphere as a co-dimensional performance space. This extends the notions of co-dimensionality in the theme; the revelation, or illumination, of the ‘other’ – the other dimension, the indemonstrable, the area in which the spiritual, whatever that may be, resides. The virtual is the other, it is, in representational light waves, in ones and noughts, the abstract metaphor for all of these notions of the metaphysical. The hypersphere suggests a co-habitation of metaphysical and physical. The real and the virtual are ontologically fused. We are immersed in the hypersphere. Our presence travels along multiple frameworks of space and time, through an altered reality. Hyper: over; beyond; above. Beyond the physical, in excess of the real, above what we perceive to be normal in space and time. It induces sublime confrontations with media saturated environments. But crucially, the saturation of media in this case is such that the erotics of such media – interface, function- are ontologically integrated into the real space, to the point where the environment is media, and media is environment. 

---

 

BOLTER, J. D. and R. GRUSIN (1999) Remediation: Understanding New Media. 4th ed. Cambridge, Massachusetts: The MIT Press. 

MESTRE, D. R. (2005) ‘Immersion & Presence’:  http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=91D6605D114DF846EA5E1744A04DB8D9?doi=10.1.1.96.1276&rep=rep1&type=pdf 

MITCHELL, W. J. (1995) The City of Bits: Space, Place, and the Infobahn. Cambridge, Massachusetts: The MIT Press. 

 

3-D simulation: realising the Hypersphere

Using Giannachi’s formulation of the hypersurface as the framework from which I have built my practice-as-research methodology, I am now beginning to question the idea of the ‘surface’ as a 2-dimensional space.

Beginning this project, I had stated in my proposal that I would ‘explore and understand the properties of projector-based light and the geometric relationship between the two-dimensional pixels produced by the projector and the three-dimensional co-ordinates in the real space illuminated by those pixels’. This relationship, as stated in my proposal, allows the creation of hybridised and extended notions of space and time, and is thus appropriately termed ‘mixed reality’ performance. Mixed-reality, however, encompasses a wide range of digital performance practice, and its effects can be observed without integrating the virtual and the real through the hypersurface. But to articulate the interdependent relationship between the 2-d pixels (virtual) and the 3-d space (real) in the form of projection mapping, there are a number of pre-requisites that, through experimentation in the space, I have begun to articulate. At this point, I have proposed that this occurrence is dependent on a number of properties, including the simulation of the photo-realistic and the use of representation in the virtual, which have been discussed in the post prior to this. Here, I examine how the first pre-requisite, that of spatial co-ordination, complicates the idea of the surface, and how light relocates the real within the virtual.

Firstly, the overlay of the real and virtual must align with one another geometrically. That is, ‘[g]raphical objects, even if rendered in a high visual quality, would have to be integrated into the real environment in a consistent way’ (Bimber and Raskar 2005: 5). This is crucial in ensuring that ‘neither the materiality nor the image dominates the problematic’ (Parrella 1998: 13). As I have experienced, if the image is not aligned correctly to the spatial co-ordinates of the space, the image dominates the problematic and becomes an isolated component of the work. Domination, in this respect, was achieved by the image due to the lighting conditions of the space. The projector, in releasing the dominant source of light, was thus illuminating the spatially misaligned co-ordinates of the space, which meant that the virtual dictated what part of the real the eye could see. In this way, the real was reliant on the virtual to reveal itself, but because one could only see parts of a constituent image, it felt incomplete and inferior in its meaning than the virtual, which shone out in brilliant totality.

What Bimber and Raskar refer to as consistency, however, I have come to realise does not refer to lighting conditions. To try and be consistent in lighting conditions within the virtual is to simulate the countless properties of natural light, which is completely and fundamentally different to electrically generated light in how it is perceived by the eye. It is the illumination of the real by virtual light that relocates the real in the luminosity of virtual space, and creates an interdependent relationship between the two. Geometrically, however, one can map the co-ordinates of the space in one’s virtual construction of said space. For example, if I project a virtual reconstruction of the window and correctly align it’s co-ordinates with the real window and integrate that into the environment, I am illuminating the whole image: both real and virtual. What Parella refers to as a ‘co-presence of both material and image upon an architectural surface/membrane/subtrate’ (1998: 13) here can be further understood in practice. The interdependency of each is realised, and the creation of a hybrid reality comes to the fore.

However, the idea of a surface, here, loses it’s potency in practice. Whilst the 3-dimensions of real space have been overlaid with 2-dimensional pixels produced by the projector, the hypersurface, realised to it’s fullest potential via correct spatial alignment, actually simulates the 3-dimensional construction of virtual space. That is, the hypersurface is thus a hybrid of real 3-dimensional space and the simulation of 3-dimensional virtual space. This world, whilst being displayed via 2-d pixels, constructs a 3-dimensional world, that is perceived phenomenologically in it’s 3-dimensions in our observation of it.  Our presence seeps through into the virtual; psychologically, perceptually, and cognitively, we immerse ourselves in the virtual. Marcos Novak points out that the hypersurface is ‘an effort to see surface not as an Aristotelian delimiter of space but as the portal between worlds through which subjectivity emerges’ (1998: 88). Subjectivity, here, refers to our experience of such immersion; ‘the subjective experience of being in an environment, even when one is physically situated in another’ (Witmer and Singer 1998: 225). As a consequence, the idea that we, as beings immersed in 3-dimensional reality, can somehow immerse ourselves within, that is, inside, a surface, becomes nonsensical.

As the space of the virtual aligns itself with and illuminates the real, and thus situates the real within the virtual, the virtual must embody the 3 dimensions of the real, and further complicate those dimensions by repositioning the laws unto which those dimensions usually exist into a lawless realm of play and possibility. The window now stretches into infinity; it defies gravity; it exists in ones and noughts, in abstract entities, in representation. I have been realising this notion through a representation of 3-d space in creating 2-d motion graphics: the crumbling of the church into the void, the tumult of the big bang explosion, the undulation of church space in liquid-like motion, all of these to further reinforce the nature of the virtual as a 2-d space that is transformed into a 3-d space by our perception of it. The articulation of a sphere, as a 3-dimensional realm that allows such possibilities to exist, becomes semiotically appropriate in terms of both immersion, and the interdependency of real and virtual, as suggested in the name of this project: Through the Hypersurface. It is through  the hypersurface that we emerge within the virtual, which has transformed real space into a hybridised plane of possibility and multiplicity, in which traditional perceptions of time and space become increasingly complicated. In keeping with Novak, the relationship of real and virtual in the hypersphere suggests a portal between worlds. But here I must stress that in travelling through such a portal, we do not leave the previous (real) world behind: it stays with us, it fuses itself with the virtual world. In this respect, it is appropriate to suggest that the hypersphere is the opening of the virtual world into the real world. In this way, we can further understand Parella’s suggestion that ‘[t]he term hypersurface is not a concept that contains meaning, but is an event; one with a material dimension’ (1998: 10). The notion of an event further reinstates the hypersphere within the temporal, where it is fundamentally and irrevocably situated. Furthermore, we can now realise, through practice, what Steve Dixon means when he writes that: ‘Hypersurfaces are places of exchange, fleeting intertextual strata in which dialectical opposites interact and continuously contaminate one another. As part of the real, they are bound to materiality: as image, film or photography, they are both intertextual and metatextual, and belong to the sphere of discourse’ (Dixon 2005: 99).

---

DIXON, S. (2007) Digital Performance. Cambridge: The MIT Press.

PARRELLA, S.(1998) Hypersurface Architecture. Bognor Regis: John Wiley & Sons Ltd.

WITMER, B.G. and SINGER, M.J. ‘Measuring Presence in Virtual Environments: A Presence Questionnaire’, Presence: Teleoperators an Virtual Environments(MIT), 7:3, June 1998, pp. 225-240.

Realism and representation

In negotiating the balance between the real and the virtual, I am now considering the use of representational graphics and photo-realistic graphical elements within the work. To consider this, I am using Giannachi’s articulation of the hypersurface to further my research.

As written in my previous post, ‘The Hypersurface': If the hypersurface is both real and representation, but also liminal, this would imply that the hypersurface can act in a number of ways. Firstly, one could propose that the hypersurface might translate properties of one reality into another, for example, representation (projection) might map the spatial co-ordinates of a space and thus become a virtual representation of the space. Secondly, one might suggest that real (physical space) has the potential to embody qualities of virtual space, and thirdly, one could imagine the potential for each space to fuse together and thus mutually co-depend on one another.

In this way, I can begin to map the varying spatial and temporal relationships between the two different variations of the virtual: the realistic and the representational. Let’s begin with the realistic.

When I have taken a photograph of the 3-dimensional space, I have remediated the reality of the space into that of the photograph, which now exists as a 2-dimensional photo-realistic translation of that space. Using pixel-based light, the photograph depicts the space as it appears to the lens of the camera. This mediated replica of the space embodies the light-reflecting qualities of each surface in pixels, which, when projected onto such surface with geometric and spatial accuracy, overlays the 3-d space with a 2-d virtual construction of that space. In this way, the light thrown out by the projector (somewhat) replicates that of the mediated (photographed) space and is thus closely related to that space. Geometrically (the mediated, 2-d space is the same size as the 3-d, real space) and spatially (the 2-d space appears quite similar to the 3-d space to the naked eye, insofar as it is a photograph of that space, which has selected various photons from that space to absorb into the camera’s lens and project back into that space). Temporally, the virtual overlays the place in time at which the photograph was taken on top of real time. Depending on the nature of animation of that photograph, the virtual allows that temporal structure to fluctuate and morph. But, because of the spatial and geometric closeness between photograph and real space, the temporal structure of the real space fluctuates and morphs also. The integration of graphical object into real-world environment here, is somewhat greater than that of the representation.

By representational, here, I mean the avoidance of realism, for example, a drawing of a tree (and not a photograph of it). The graphics I have thus far created are mostly representational, becoming abstract textures that depict the space as a ‘myopic vision of a dream world’ (Paul 2003: 127). Representations allude to the real space by association, they are metaphors of the space, they are abstract entities. Whilst this approach is appropriate in creating a mixed-reality with a virtual temporal and spatial framework, (one that exists outside of the real, as with the religious, spiritual and philosophical idea of the other, God or other beings), I have realised that the representational cannot exist in the projector-based light without projector-based realism. The representational, in isolation, does not integrate into the real-world environment as effectively as the realistic, but without the representational augmentation of the realistic (the augmentation of realistic elements by digital software), the projector can no longer mix the reality of the space. In this way, the two depend upon each other.

With this in mind, I have realised the progression of the performance. I want to begin with projections that appear somewhat photo-realistic, to complicate the clock-time of  the church, and apply augmentation of the real-space in which it exists. To do this, I will begin with a projected nighttime environment that depicts the space outside. This space will mix elements of representation with photo-realism, because I want the effect to appear dream-like, but not wholly removed from realism in terms of detail and aesthetic, like in this example:

 (click to enlarge)

To achieve this desired effect, I will need to observe the light and shadow structure of a single light-source cast onto a stony surface, in order to replicate the quality of the moon’s light shining onto the window ledges.

Once I have introduced this complication of clock-time, I want to begin to augment the space with greater representational elements, such as the graphical depiction of the wall cracking and disappearing into a black-hole-like void:

Overlaid with some purely representational elements, like abstract light:

As you can see, my aim is to begin by mixing the reality of the space slightly at first, with a distortion of clock-time and real space, that are still grounded in the laws of physics and our sense of what reality is. I then want to transition into the more sublime elements of the performance with greater representational, dream-like depictions of the space, which then make their journey into the hypersphere – the other, and break all laws of physics and begin to construct a reality of their own. Time and space warps and suspends itself here, the virtual space embodied elements of the real space but has partly departed from that space. The real space now embodies the virtual space, and is this dream-like world. My plan now, with this in mind, is to oscillate between these two modes of projector-based spatial realisation, to produce a mixed-reality projector-based performance that closely examines the interdependent relationship between the real and the virtual.

Experiments 2

After testing the spatial co-ordinates of my designs in the space last week, I was keen to test the lighting conditions of the church at night. Me and fellow Banger (the wonderful co-artistic director of A Bang in the Void) Laura began the journey at around half 10. At night, the church adopts a completely different feel. The limited amount of lighting in the old space instantly made me feel uneasy, with the many ghost stories, spooky films and subconscious associations with dark, old buildings rising into my mind. But I could begin to imagine the gothic and otherworldly atmosphere that the piece would fit nicely into. Being in the space at that time has now shaped the aesthetic direction of the graphics I am creating. We began projecting at around quarter to eleven, at which time the orange glow of the church lights flood into the space. The results were less than desriable:

Whilst the white outlines of the window are clearly visible in the darkness of the church, my plans to project onto the stained-glass window were then complicated by the light pollution of the street lights. Here, pixel-based light would not appear as bright to the human eye due to the infiltration of electrical light outside, and thus the ability for projector-based light to meet the real and create an interdependent relationship with that real space in an augmented and mixed-reality fusion was then compromised. This light constituted too large a portion of the real-virtual relationship, and so the performance would not ‘establish multiple and coexisting modes of performance often spanning real and virtual environments’ (Benford and Giannachi 2011: 6).

However, once the bells struck eleven times at eleven o’clock, I was delighted when the outside lights turned off. This seemed like the perfect cue to start the performance, using the sounds of the bells that begin the soundtrack to the graphics as a gateway to this new hypersphere of mixed realities.

This was the ensuing result. I was pleased to note that the windows provided the translucent luminosity I had hoped they would. The projector-based light, when beamed onto the translucent surface, is partly absorbed by that surface. The rest of the light continues to beam straight through the glass, away from the visible perspective of the human eye in the space (Bimber and Raskar, 2005). When beaming a solid area of pixels at 100% opacity onto this translucent surface, the resulting aesthetic is one akin to the natural translucent quality of the windows in natural light. With this insight, I realised that I could project simulations or photographs of the windows onto the windows, and then distort those graphics to achieve an increased level of integration between the real and virtual, than say, a solid coloured area of pixel-based light. I suspect that the resulting photo-realistic projection of the window would appear as though the window has been illuminated by an artificial light. In this way, I must acknowledge the existence of this artificial light in the virtual. One way of doing this might be to continue to use a spotlight effect as a way of illuminating different areas of the window, overlaying photographs of that area onto the real area. I will experiment, however, with various techniques on this point, also trying solid coloured pixels, as seen in the bottom photo above. I hypothesise that the former will achieve optimum results, due to the synthesis between the spatial area depicted in the 2-dimensional surface of the pixel (which has translated properties of the 3-dimensional space) and the 3-dimensional space itself (which, once augmented, translates properties of the virtual space) (Giannachi, 2004).

I am aware, however, that in the same way that the orange light created a power imbalance between real and virtual, the white light of the graphics now appear somewhat removed from the space. Whilst, geometrically, they map the architecture, the white lines are visible simply as floating lines against a dense black background. I realised that I must provide some kind of lighting to illuminate other areas of the space, so that the white lines are associated with the space visible through projector illumination. This remains my next task: to negotiate the balance between the real and the virtual through various lighting techniques.

---

BENFORD, S. and GIANNACHI, G. (2011) Performing Mixed Reality. Massachusetts: MIT Press.

BIMBER, O. and RASKAR, R. (2005) Spatial Augmented Reality: Merging Real and Virtual Worlds. Wellesley, MA: A K Peters, Ltd.

GIANNACHI, G. (2004) Virtual Theatres: An Introduction. London: Routledge.

Experiments in the space…

Above are some screenshot examples of the kind of graphics I have created to be overlaid in the space. I used Adobe After Effects CS5 for this, consulting this book:

and various online tutorials for help. I am new to After Effects, so part of the practice in this research project is learning this software. The software acts as the gateway into my research findings: without an effective use of motion graphics, the research will be limited in its findings. If I can learn the processes that will enable me to experiment with how the space can appear as though it is morphing through mixed-reality, then the project’s outcomes will be much more reliable and worth investigating.

As part of practice, I have utilised a literature search and video tutorials as a method to mimic, refine and explore the practical processes involved in creating the visuals needed for the project. My next step, in creating a short piece of graphics, was to experiment with how these demonstrated my aims and objectives by projecting them in the space. I have chosen this particular area in the church for a number of reasons, some practical, some artistic and some theoretical. Firstly, I was somewhat limited by power supply, length of extension leads, appropriate surfaces on which to place the projector (which proved to be a difficult task if I wanted the projected area to cover the area I had designed graphics for), light (will the orange glow of a streetlamp infiltrate the space?), and a space big enough (and safe enough) for an audience to stand. This was my first step. I then, from this deduction, articulated my theoretical concerns (which, in some instances, were mutually dependent on the practicalities of space, for example, light): is the area going to be dark enough for the projector to be visible, and create a series of lighting conditions? Artistically, I wondered whether the area embodied enough of the church’s architectural and artistic features for me to augment, and whether choosing a corner of the church would neglect the rest of the space.

In this first experiment, however, I was limited by the light conditions, as I could only access the church during the day. I am allowed access at night, but circumstances in this instance meant that I couldn’t conduct the experiment then, so I have had to shift my focus away from lighting conditions and towards geometric mapping of the features of the space. Determining this early on, I deduced, is essential for me to create graphics that will be geometrically accurate. If I can map the co-ordinates of the space early on, I can move on to experimenting with light and shadow rendering techniques, which will culminate in a further experiment within the site, which will take place at night.

So I set up the projector, inserted the graphics into MadMapper, and experimented with configurations to best fit the co-ordinates of the space. Crucially, I discovered that my graphics were flawed. I had created them over the top of a photograph I had taken of the given space, but the photograph was taken at an angle and was thus skewed. I presumed that MadMapper would rectify such a warped perspective, but asking the software to firstly adjust the perspective of the graphics and then the skewed perspective of the projector in the space was simply too much to ask. I could never reach a satisfying result: if I moved one corner, it would throw off the balance of the rest of the graphics:

In Spatial Augmented Reality: Merging Real and Virtual Worlds (2005), Bimber and Raskar propose three geometric rendering components: projector model, display portal and user location (96). Projector model outlines how the projector geometrically projects light outward, display portal, or display surface, is the spatial qualities of the projected surface, and user location outlines the location of the projector in the real-world co-ordinate system. If I can align the perspective of the graphics with that of the projector model, MadMapper can consequently adjust the co-ordinates to align themselves with the display portal as a result of the user location.

Result: I must, in Photoshop, adjust the perspective of the photograph to create a flat perspective, and experiment with this result using the same user location. If satisfying results are not reached, experiment with other user locations. Once I do this, it remains for me to analyse how the meeting of the real and the virtual ‘interact and continuously contaminate one another’ (Giannachi 2004: 99). The abstract and virtual qualities of the white lines served originally to highlight the structural qualities of the space in the dark. I wanted to explore how the real space might embody qualities of the virtual lines, how the lines clearly embody qualities of the real space and how each thus co-depend on one another. In order to experiment with notions of disjunctive time, I suspect that I must experiment with more realistic graphics, to allow the virtual to firstly embody the temporal framework of the space, and to consequently reappropriate it.

---

 

BIMBER, O. and RASKAR, R. (2005) Spatial Augmented Reality: Merging Real and Virtual Worlds. Wellesley, MA: A K Peters, Ltd.

GIANNACHI, G. (2004) Virtual Theatres: An Introduction. London: Routledge.

MEYER, C. and MEYER, T. (2010) Creating Motion Graphics with After Effects: Essential and Advanced Techniques. 5th ed. Oxford: Focal Press.

The site of St. Nicholas

Here are some pictures I have taken of the site I have chosen for this research project: St. Nicholas Church, Leicester.

St. Nicholas is the oldest church in Leicester, dating back 1200 years, and sits next to Jewry Wall, the largest Roman structure of its kind still intact in England. Underneath the church’s foundations, some fifty inches below the floor, indicate that there was a colonnade east of Jewry Wall.

Parts of the church date back to Saxon 900AD, although it wasn’t the first church to stand on the site (the first incorporating Jewry Wall as part of its architecture, which is why that part of the wall still stands today). Through into Norman times, the church saw several additions, notably the (now partly restored) Norman tower, a rebuilding of the original Saxon tower. In the Middle Ages, the Early English arcade was built. In the 16th century, the timber-framed porch was added, which was restored in 1975. Renovation continued through to the 19th and 20th centuries, seeing the addition of a typically Victorian archway in the church.

Walking around St. Nicholas, it is easy to see the many layers of history embedded into one place; from the small saxon windows to the 21st century kitchen, one sees how, over the years, the place has adapted to each time in history, yet still retained the impression of each period that preceded it. St. Nicholas feels like a meeting-point of many locations and times, all interceding in multiple layers; where Roman worshipper may converse with a visitor born in 1991; where, if you look, you can imagine the candlelit vestiges of century-old worship, or if you put your ear against the stone, you hear the faint traces of hymns once sung.

It is apt, therefore, for me to stage this research enquiry into mixed-reality performance in such a place that already has the qualities of a mixed-reality, without artistic augmentation of any type. For me to further deconstruct and reappropriate the complex mesh of location and time seems to add to the church’s sense of multiplicity. When I first visited the site, it was pleasing for me to see the sheer scale of adornment dedicated to people’s beliefs over many hundreds of years, and it seemed to me that despite what people have believed – Krishna, Yahweh, Allah, Buddha or Jesus Christ – they all seem to be pointing in the same direction: toward the unknown, the Other, and each are given human form in a vast fractal of beautiful representations in architecture, art and scripture. With this assumption in mind, I aim to gratefully give thanks to one very tiny example of the Other given form in St. Nicholas, and to show how one such example need not be the authoritative example of spiritual representation. It remains for me to explore how, through deconstruction of the church as a site, we may lift some of the negative (and perhaps fearful) feelings we have towards the church (in general) in regard to the heinous acts certain individuals have committed in the name of their beliefs. Is it possible for me to inspire a transcendental and spiritual encounter, by pointing towards a place that has traditionally been the only place to do so, but, paradoxically, in doing so, to suggest that such a place need not be the only place to do so?

 

 

Space as practiced place

Using philosopher Michel de Certeau’s reflections on the relationship between ‘place’ and ‘space’, I will be analysing the virtual’s role within this paradigm. Specifically, de Certeau’s notion of space as a practiced place, i.e.  realised by spatial practices, to deduce whether, by extension, the virtual becomes a practiced place, and whether the interdependent relationship of real and virtual, introduces new relationships between space and place.

Place, de Certeau asserts, in his 1984 text The Practice of Everyday Life, is realised through a system of spatial practices. Giving an example, he proposes that: ‘the street geometrically defined by urban planning is transformed into a space by walkers’ (de Certeau 1984: 117). He further suggests that place must adhere to a specific set of rules, that ensures the order of place, e.g. two places cannot exist within the same location (place). Spatial practices, however, may give rise to multiple expressions of a place, thus allowing several spaces to exist within a single place; people may walk, talk, sit and jump in the same place. He suggests:

in relation to place, space is like the word when it is spoken, that is, when it is caught in the ambiguity of an actualization, transformed into a term dependent upon many different conventions, situated as the act of a present (or of a time), and modified by the transformations caused by successive contexts. In contradistinction to the place, it has thus none of the univocity or stability of a “proper”. (.: 117)

In this respect, space is thus open to transformation through practice; it need not succumb to order and rigidity, it cannot be fixed. Space implies movement, or transformation. Audience members realise a space through a multitude of individual spatial practices, further complicating the place through transitory acts. Additionally, Nick Kaye suggests in Site-Specific Art (2000), that space ‘is realised in a practice which can never rest in the order it implies’. Thus, spatial practices imply an absence from place, ‘in their inability to realise the order and stability of the proper’ (6). In consequence, Kaye suggests an antithesis between space and the presence of place/location, just as representation presents itself as a removal from the object it represents; it removes itself from the presence of that object.

So what does this mean for a space augmented with spatial projection-cased mixed-reality performance? It certainly suggests that, as previously discussed, such practice has the ability to introduce new spatio-temporal frameworks to the place. This is due to the spatial practice of the virtual, it’s being an act within the place. The removal of this act from place, however, needs to be questioned. If, as suggested, spatial projection-based mixed-reality performance contains within it the parameters to hybridise the spatio-temporal frameworks of the place (the real) with the space (the virtual), is this really a complete removal from place? Does this mode of performance problematise space’s removal from place, if the place itself becomes practiced?

Or at least, it seems to be. If the level of augmentation is geometrically and optically accurate enough, the projection allows the space to become deconstructed – it optically merges with the place, and produces effects convincing enough to have us (sometimes) believe it has ontologically merged with the place also. This hybridisation introduces an element of interdependence, it would seem, between place and space. The place still adheres to its ordering system, but introduces elements of space’s multiplicity, and has us believe that place truly can defy the ordering system into which it is placed. The removal, however, has not completely disappeared. For it is precisely this removal, the inability for the projection to realise the ordering system of place, that allows us to deconstruct that very ordering system, and introduce new realities and temporal structures.

Furthermore, the immersive nature of such mixed-reality performance, even suggests that my audience, by extension, become hyridised with and inextricably implicated within place. Their practices, in this instance – their immersion in the mixed-reality performance – can never truly be antithetical with place. There appears to be another practice emerging from their involvement in the piece, namely the path (or trajectory) each individual chooses to pursue and experience in the performance. The trajectories each individual follow through the hypersurface are somewhat limited by the pre-determined nature of the piece (i.e. not interactive or open-ended), but each individual’s transformative trajectory, or encounter with the Other, will be deeply personal and subjective. In this sense, spatial practises refuse to adhere to the stability of place. Perhaps the place becomes a location where multiplicity can exist, where the audience can traverse through the hypersurface and thus question what is real and not real, where they are, where they have come from, and where they are going. Perhaps the transformative power of the hypersurface extends beyond spatio-temporal, and crosses into experience, which, as with spatial practices, refuses to be singular.

---

DE CERTEAU, M. (1984) The Practice of Everyday Life. Berkeley: University of California Press.

KAYE, N. (2000) Site-Specific Art: Performance, Place and Documentation. Oxon: Routledge.