Terminology Analysis of Digital Games, part 8

Continued from part 7.

CYBERKINAESTHESIA (cont.):

Previously, I wrote about the adaptation needed for algorithmic coupling between a player's body and their virtual avatar/point of view in a digital game.

Mark Hansen Bodies in Code (2006)

Sylvie Bissonnette posits that our body image and body schema are keys to understanding the biofeedback system we enter when we interface with a digital game (2019 pp. 209-13), echoing Mark Hansen's "body-in-code" theories (Hansen Bodies in Code (2006)).

Philosopher Shaun Gallagher wrote to clarify the meanings behind body image and body schema.

BODY IMAGE: is the mental construct of one's own body. It consists of at least three parts:

1. One's perceptual experience of their body.
2. One's conceptual understanding of the body in general (includes mythical and/or scientific knowledge).
3. One's emotional attitude toward their body. (Gallagher "Body Schema and Intentionality" The Body and the Self (1995) pg. 226)

It is "a conscious image or representation... and appears to be something in-itself, differentiated from its environment" (Gallagher "Body Image and Body Schema: A Conceptual Clarification" The Journal of Mind and Behavior (1986) vol. 7 no. 4 pg. 541). When first 

BODY SCHEMA: is the nonconscious control of body posture in order to facilitate perception. It is "pre-personal, functions holistically, and is not something in-itself apart from its environment" ((1986) pg. 541). It involves "an extraintentional operation carried out prior to or outside of intentional awareness" ((1995) pg. 228).

Body image and schema are the conscious and nonconscious (or "prenoetic," as Gallagher calls it ((1995) pg. 226)) factors of the bodily experience. As Gallagher describes, body image allows one to consciously raise their hand with their perceptual attention focused on that action, but in that movement, the body schema will enact certain postural adjustments of the body that serve to maintain balance that are not under conscious control ((1995) pg. 229). The body image is a conscious perception of the body as an object while the body schema is the preintentional performance of the body, organized in relation to its environment.

Parts of the environment, such as tools, may be incorporated into the body schema. "The carpenter's hammer becomes an operative extension of the carpenter's hand" ((1986) pg. 548). After becoming skilled with a tool, the object becomes part of the body schema, even though it exceeds the body image. The tool is not part of what we consciously understand as part of our body, yet it becomes implemented into the body's sensorimotor and perception systems.

Philosopher Maurice Merleau-Ponty describes a blind man using a cane to navigate as an example of extending the body through an external object:

The cane is no longer an object that the blind man would perceive, it has become an instrument with which he perceives. It is an appendage of the body, or an extension of the bodily synthesis (Merleau-Ponty Phenomenology of Perception 1945, trans. 2012 pg. 154).

Gregory Bateson uses a similar example to explain that there is not point where the body's "self" ends and the integrated tool begins:

Consider a blind man with a stick. Where does the blind man's self begin? At the tip of the stick? At the handle of the stick? Or at some point halfway up the stick? These questions are nonsense, because the stick is a pathway along which differences are transmitted under transformation, so that to draw a delimiting line across this pathway is to cut off a part of the systemic circuit which determines the blind man's locomotion (Bateson Steps to an Ecology of Mind 1972 pg. 324).

[Update 1 Dec 2022] 

Philosopher Andy Clark describes this grafting of environmental objects to the body schema as a "fluid mesh:" 

What we are witnessing... [is] the remarkable capacity of the human brain to learn new modes of controlling action and to rapidly reach a point where such control is so easy and fluent that all we experience is a fluid, apparently unmediated mesh between will and motion... The expert car driver, golfer, tennis player, or video games player... have reached a point where aspects of the apparatus (the clutch pedal, the racket) become transparent in use (Clark Natural-Born Cyborgs 2003 pg. 120).

[End update 1 Dec 2022]

[Update 5 Dec 2022] 

N. Katherine Hayles refers to augmentation to the body schema as "proprioceptive coherence" and ties it directly into the formation of virtual subjectivity:

Proprioception is the sense that tells us where the boundaries of our body are. Associated with inner-ear mechanisms and internal nerve endings, it makes us feel that we inhabit our bodies from the inside. Proprioceptive coherence, a term used by phenomenologists, refers to how these boundaries are formed through a combination of physiological feedback loops and habitual usage. An experienced tennis player, for example, frequently feels proprioceptive coherence with the racquet, experiencing it as if it were an extension of her arm. In much the same way, an experienced computer user feels proprioceptive coherence with the keyboard, experiencing the screen surface as a space into which her subjectivity can flow (Hayles "The Condition of Virtuality" The Digital Dialectic 1999 pg. 88)

[End update 5 Dec 2022]

Mark Hansen follows Merleau-Ponty's phenomenological concept of embodiment (and body image/body schema) in that it allows for technical extensions of human movement and bodily habit (Hansen (2006) pg. 39). This forms the basis of Hansen's "body-in-code" theory of integrating with digital media to utilize the excess of body schema over body image and create a body "submitted to and constituted by an unavoidable and empowering technical deterritorialization -- a body whose embodiment is realized, and can only be realized, in conjunction with technics" ((2006) pg. 20 (emphasis in original) and Bissonnette 2019 pg. 210). This same body-environment coupling with the body schema is how we integrate with our avatars in virtual environments, what Bissonnette calls algorithmic coupling (2019 pg. 214).

Andreas Gregersen and Torben Grodal draw upon Gallagher's and Merleau-Ponty's concepts for their own similar theory that "video games may lead to a sense of extended embodiment and sense of agency that lies somewhere between the two poles of schema and image -- it is an embodied awareness in the moment of action, a kind of body image in action -- where one experiences both agency and ownership of virtual entities" (Gregersen and Grodal "Embodiment and Interface" The Video Game Theory Reader 2 2009 pg. 67 (emphasis original)). They note that the effect is, of course, temporary, that "once the player stops acting in relation to the game system and pays conscious attention to his or her own embodiment, this effect subsides in favor of a more regular body image" (Ibid.).

To be continued...

Terminology Analysis of Digital Games, part 7

Continued from part 6.

CYBERKINAESTHESIA (cont.): 

In the last post, I reviewed how Silvie Bissonnette posits that a digital game player enters an algorithmic coupling of their physical body to their digital avatar (Affect and Embodied Meaning in Animation (2019) pg. 203). The player's sensorimotor skills adapt to the mappings between the physical and logical interface and even complex actions within the virtual environment can be enacted, seemingly without even thinking.

That coupling can take practice and work on the part of even an experienced game player, as each game has its own distinct schema that requires time for adaptation. Game scholar Soraya Murray describes the disorientation she sometimes feels before she has fully adapted to a game's spatiality:

"That brief disorientation I experience could be thought of as a mere behavioral quirk, or a moment of indecision, but I think it points to something else: a momentary disjuncture between the spatiality presented by the game and my personal spatial orientation. In that moment, I graft my sense of spatiality onto that of the game. But that means there exists a difference between the two. That difference appears as an ideological gap that must be bridged, in order for me to give sense to the space as a player who finds herself 'out of place' in a social construction inconsistent with her own. That gap has been an object of discomfort, a turbulence in the otherwise smooth transition of mapping one's self onto the technological space of a game" ("Coda: Disoriented in the Field of Play," Video Games and Spatiality in American Studies (2022) pg. 275).

Sudnow Pilgrim in the Microworld (1983)

Author David Sudnow wrote the pioneering early work Pilgrim in the Microworld (1983) in order to record his strategies and experiences adapting his mind and body to the virtual space of Breakout (1978) for Atari 2600. Early in his experience with the game, Sudnow describes adapting to the mapping between the its physical and logical interfaces:

"At first it felt like my eyes told my fingers where to go. But in time I knew the smooth rotating hand motions were assisting the look in turn, eyes and fingers in a two-way partnership... So too with sight reading music at the piano for instance, where you never look ahead of what you can grasp and your hands’ own sense of their location therefore instructs the gaze where to regard the score. So too again with typing from a text, where if your eyes move in front of where your fingers are, you’ll likely make an error, and thus hands and gaze maintain a delicate rhythmic alignment. And so too here, you’d have to sustain a pulse to organize the simultaneous work of visually and tactilely grasping the ball, your hands helping your look help your hands make the shot" (1983 pp. 40-41).

In each of these comparisons, the user's perception (reading sheet music or text) must align with the inputs the user is giving to a machine (playing piano or typing) in order to produce the desired result (play music or copy the original text). In a digital game, the user's perception is their avatar or their point of view on the environment (the virtual "self" (Ellis "Nature and Origins of Virtual Environments: A Bibliographical Essay" (1991))) and the user must adapt to the mappings between that perception and the inputs they give to the game machine.

[Update 3 Dec 2022]

Researcher Ulf Wilhelmsson calls this connection between human and virtual environment the "Game Ego presence."

"The Game Ego is a bodily based function that enacts a point of being within a game environment through a tactile motor/kinesthetic link. Computer and video games typically allow the game player to establish a virtual proprioceptive chain based on sight, hearing and tactile motor action adding up to a tactile motor link and kinesthesia, i.e. a sensory awareness of the position of the body within the game environment. In turn this may result in a strong performative experience of interaction, interactabillity and being. The player does not only see and hear but is enacting a point of being" (Wilhelmsson "Game Ego Presence in Video and Computer Games" Extending Experiences 2008 pg. 61).

The Game Ego's focus is the player avatar, or the point-of-view self of Stephen Ellis' definition of a virtual environment. It is a connection through an embodied interface, a proprioceptive bond that leads to cyberkinaesthesia.

"The player incorporates the Game Ego function, which serve[s] as an instrument for controlling the game environment. The exertion of control is an extension of the player's sensory motor system via a tactile motor/kinesthetic link. The end outcome of this control is not only the controlled and perceived motion on the screen but also, and more important, the experience of locomotion within an environment" (Ibid. emphasis added).

[End update 3 Dec 2022]

To be continued...

Terminology Analysis BEYOND Coming of Age in Second Life, part 6

Continued from part 5.

As this series of blog posts continues, it becomes less and less about the terminology in Boellstorff's Coming of Age in Second Life.

CYBERKINAESTHESIA (cont.): 

Continuing to look at Silvie Bissonnette's Affect and Embodied Meaning in Animation (2019), she states that players enter an "algorithmic coupling" of the player's body with their avatar (2019 pg. 203). They achieve perceptual symbiosis with their avatar, but must establish a form of algorithmic synchronism with the interface and physical mastery of different mappings in order to achieve success in a game with more complex moves (pg. 203).