What Doesn't Change Across Form Factors in Interaction Design

Users.

Users don’t change.

There are no new ideas in the combination of words below, you create the ideas and meaning as a viewer and user of this document.

Although the concept of ‘the user’ is perhaps the only thing that doesn’t change across form factors when considering user experience design, the users, their selves, do change. Of course we know this is true because we are reactionary and adaptive organisms existing in a constantly changing environment. Our interests, capabilities, dependencies, and expectations are in flux with our surroundings. We are all users, and only through an understanding of ourselves, do we uncover our understanding of others.

There are, however, certain properties of the human mental construct that remain constant enough to create around and, with reasonable certainty, predict a user’s response to the imposed stimulation. This in fact drives to a founding stone in this essay that when creating for users, we are invoking a simple action-reaction process not unlike that at the biological or sub-atomic level. We have the concepts of a receptor and stimulator (probabilistically more physical) and the concepts of an impulse operator and reactionary value (probabilistically more energetic).

Considering first the receptor-stimulator construct, we must appreciate the problem of designing for user experience as visual communication problem solving; dependent on a language as alive as the object that inherits it, but also consisting of ‘atomic’ building blocks that form the inputs for predictive calculations. These basic shapes of the visual language act exactly as the phonemes of verbal language and are universally recognized and ‘spoken’ across all cultures and demographics. An example here would be a circle- it may be called many things, but it is always a circle and recognized, or received by, its never-ending roundness. This roundness then acts as a vehicle for non-specific user reactions such as comfort, safety, and time-dilation.

So there are receptors of the human mental architecture that don’t change across humans at the most basic level or over time (much)- if they do change over time, they change isotropically. There are also stimulators, the building blocks of the visual language, that do not change at all.

We can liken this phenomenon to that of our taste receptors where we have the same receptors for sweet and sour; it is our cultural learning and development that alters these receptors making them more or less affinitive for a particular taste, yet the existence of the receptors is the same. The stimulators, the chemical combinations that invoke sensation, are constant because of the electrochemical laws that govern their shape in space.

Naturally following the pattern of our surrounding universe, elements of the visual language 

are drawn together, interact, and create higher order structures and groupings invoking patterns and sequences catalyzing new reactions in viewers, or users. Not only are new shapes instantiated, but also new spaces- the encapsulating envelopes and framing devices of the visual language- the negative space is altered, and new meaning is realized. As we consider form factors, we must take into account this space surrounding the object and not just the object itself.

Let us define what we mean by form factor in this context before going forward.

A form factor is that container for an interface, for the construct of the interface, and for the user of the interface.

With this definition, we understand the device and the user are both form factors that must be accounted for in design. So we are not just thinking in terms of pixel dimensions or metric units, but also in terms of human intelligence, cultural affiliation, age, or any other number of human properties. While this broad definition of a form factor may seem to intimidate or retard the genesis of design, it is actually more useful than aversive because it forces us to rely on those fundamental human properties that are universal, and in this way things are created that appeal to the greatest audience with greater assurance of their impact and potential reaction.

We also see that the form factor is the space in which the user experience takes place- it is an empty container for processing, interaction, and design. The form factor is not the limiter or catalyst of design problems of user experience as it is typically approached. The major cause of problems in user experience design is the lack of user (human) understanding at the fundamental level during the design phase.

As the word ‘interface’ was mentioned, let’s define that before going forward as

An interface is a constructor that maps a set of inherited functions onto any set of given data to allow for impulse, action, and potential cognition.

So the interface is that active glue which binds the form factors of design, of space, and of the user. We can understand this programmatically as the functional constructor bringing variables and methods into an active or kinetic process. The interface architecture does not exist without the three form factors, yet its elements (elements of the visual language) do exist independent of design and the user, but not of space.

If we now take a big picture perspective of this lecture so far, we will see that all users have fundamental receptors, and therefore, predictable responses to visual stimuli. That user experience design is a process of visual problem solving relying on the same principles of reception and reaction as those on the biological and sub-atomic levels. That form factors include the users as well as the used objects and interfaces.

Let’s continue down the path of understanding the user then to realize specifically those things that don’t change across form factors. From a visual communication problem solving strategy, we can break designing for the user down into concepts of energy and in this way continue to steer toward the foundational and fundamental elements that govern user experience.

We can begin with the potential energy side of the equation or the design planning phase. The visual problem is solved by accounting for universal human traits and conforming the design to draw out (enhance) or push back (de-emphasize) the responses originating from those traits. Let’s take for example the human trait of curiosity. Curiosity is biologically an open receptor for electrochemical change with a high amount of potential for energy change and according to physical law, there is a tendency toward equilibrium with an increase in entropy. (This is the action potential). When the receptor is covered, it undergoes a conformational change and this change, paired with a series of electrochemical responses, indicates the process of learning at the macroscopic level. (A protein folds according to the naturally allowable folds and the system remembers or learns its abilities). At a higher perspective level, the receptor’s host learns this process of stimulation and physical change comes from an external source and the paths of certain stimuli are attributed to a system of rewards and punishment. (Cause and effect are attributed to this process as a method of understanding and predicting the future).

This is a universal process shared inter-culturally amongst all humans and we must search out those other unchanging traits when designing across form factors.

In another language of study we can view this process in terms of linear algebra and the state vectors of quantum mechanics. Let’s pretend we are in the human basis | h > and we create a very simple equation that equates the operation of an interface on a human user to produce some sort of impulse as an eigenvalue. The equation looks like: 

human basis, | h > : I | h > - i | h > = 0

In its simplicity we see the human basis as a state vector which is unchanging and we emphasize the likeness of all humans under specific visual stimulation. So we design our operator ( I ), the impulse of the interface, to produce a measurable and predictable reaction if we understand the unshifting state vector.

If we tend now toward the kinetic energy side of the equation, the design and manufacturing process itself, the visual problem across form factors can be solved in numerous ways. Essentially these methods all employ the use of reshaping information to ‘fit’ the form factor. This is done spatially and temporally. Spatially, the technique is essentially to bifurcate the data from one form factor and reconstruct it to fit the alternate form factor. During this process, information can be lost, the architectural elements can be critically broken and the interface’s ability to allow for potential cognition as we have defined it can be jeopardized. 

This precipitates an issue at the pinnacle of user experience design, especially across form factors, that design is not about haphazardly placing shapes within a frame, it is about composing a message with utmost sensitivity to user cognition. It is also not about squishing and smushing information to fit various form factors- each form factor, whether human or object or interface has its own intrinsic properties that convey the presented information differently. For example, a tablet is held differently than a phone, and therefore will be interacted with differently. The direct translation of visual data from one form factor to another by reshaping it should be approached with exceptional care. 

The most impactful and effective visual design is nearly identical across form factors- it does not depend on the spatial constraints of the display, only on the unchanging fundamental properties of the user. And, here lies the difficulty of this problem; a difficulty that energizes those that undertake the search for its solution.

Temporally, information is reshaped by understanding the user’s ‘read’ of the presented message. (Note that the reading action is necessarily time-dependent). This is not only done by the local placement of objects within an interface, but also by employing those elements of design that again call to the human receptors of visual processing. Namely contrast, composition, and conservation of energy for the user. Through contrast, importance is established- high contrast elements are read first while low contrast elements fade into ‘second reads’ or beyond or are not read at all. Composition is the layout of the interface and it is clear that the human form factor will read a composition differently based on cultural learning. However, this can be tailored through the use of contrast. Conservation of energy is a principle followed by the rest of the universe and we are no exception. We will read information with a default minimal effort and increase or decrease this effort based on our interest in the subject matter. The majority of failures in translations of visual information across form factors occur because at least one of these principles is not respected.

Metaphor and Metonymy in Mobile Interface Design

Essay Catalyst

The use of metaphor and metonym are crucial to user interface design, especially for the mobile landscape, because we are investigating those signs, signifiers, and sequences that relate to some fundamental and deep-rooted structure of how humans interact with their visual environment; a visual environment that has been augmented with portable electronic devices.

Abstract

Tremendous amounts of research and effort are being focused on the design (drawing) and

development (programming) of user interfaces for consumer products and digestion of electronic information. This makes sense because the interface, the interactive space where a human manipulates and connects with electronic data, can establish an emotive connection between the human user and their device that has numerous effects in the business, cultural, and pedagogical spheres. Yet most of the information collected and published lacks reference to the fundamental aspects of human interpretation of imagery- there is little discussion of how metaphor and metonymy elicit emotive and cognitive effects within the human user. Additionally, the usage of fundamental design principles can easily be pointed to in the creation of any user interface, but again, a discussion of why certain imagery resonates with humans and how that can be manipulated is lacking outside studio lunch breaks and classrooms.

So-called Introduction (after Lucy Lippard)

This essay (as an attempt) discusses the deficiency of ‘art speak’ in the realm of user interface design where a conversation of human relation to metaphor and metonymy seems a good starting point for mending human relations to fine art principles with human relations to computer interactions. Through an understanding of the relationship humans have with metaphor and metonym, I believe we become empowered to create more effective user interfaces for the purposes of learning, entertainment, and advancing human cognition. The mobile form factor is chosen because it is arguable that our mobile devices are increasingly the most important tools for carrying out our daily tasks.

This work is inspired by my own interest in user interface design, specifically for mobile products (software and hardware), and discussions led by Charles Gaines, a Los Angeles based artist and teacher at the California Institute of the Arts. I am taking the definition of essay as ‘attempt’ and therefore do not intend this essay to be a totalization of the topic presented; I would rather it exist as a spoke in a discursive wheel of critical thought on art, design, and human interfaces.

The information presented here is not new- a remarkable amount of information regarding

metaphor, metonymy, interface design, cognition, and linguistics has been developed over

centuries. The unique aspect of this contribution is the combination of these ideas. As this is not intended to be a formal attempt (I would call it pseudo-academic), and the amount of relevant literature available through search is great, citations are minimal, the language is at times personal, and any feedback can be directed to james{[at]}{firstnamelastname}.com.

Attempt

The Setup- Historical Importance of the Metaphor/Metonym Process

Metaphor, as utilized by the Greeks, was typically confined to the analysis and construction of poetry. For the Greeks, poetics were separate from the arts, which consisted of crafts- unlike our culturally agreed upon definition of the arts today which includes much more than masonry or metal fabrication. Interestingly, the poetic, which I’ll define as an elemental representation of human emotion, persists in the contemporary critical discussions of art. When presented in the physical world, the poetic element is an interface between one human’s emotive experience and another human’s potential cognition. The ink on the page is the interface between humans. Of course, we do not need to stick with the Western reference of the Greeks, as other even more ancient cultures such as the Japanese or Egyptians, also heavily employed metaphor for the purposes of learning, task achievement, and emotional experience. Even further back, we can excavate the use of drawing in the caves at Chauvet. These indications of the metaphor and metonym arising so early

in human development, and the persistence of their usage, serve to underscore the importance of understanding how these tropes function in society in our moment in the mobile landscape.

Quick Cognitive Linguistic Review of Metaphor/Metonym

Theoretical considerations of metaphor and metonymy developed largely through the rise of cognitive linguistics and structuralist theory. Ferdinand de Saussure’s work Course in General Linguistics is often cited as the first revelation of a dichotomy between metaphor and metonymy.  Saussure introduces the vocabulary of langue and parole and immediately a distinction is made between the use of grammar (langue) and the spoken use of words for communication purposes (parole). For us, we can interpret from Saussure that metaphor relates to langue and metonym relates to parole. Plotting langue and parole on two separate axes, Saussure refers to the vertical langue axis as the axis of simultaneity and the horizontal parole axis as the axis of successions.  Synchrony, a term evoking the shared language rules and constructions of individuals from the same language group, is embedded into the axis of simultaneity. Within the axis of successions, diachrony, or the evolving property of language, naturally resides due its time-related dependency.  

Another important player in the metaphor/metonym investigation is Roman Jakobson who published his findings in "Two Aspects of Language and Two Types of Aphasic Disturbances," which details Jakobson’s account of two aphasiac disorders named “similarity disorder” and “contiguity disorder.” Where the similarity disorder deals with a dependence on syntactic context, Jakobson’s contiguity disorder describes a condition of language that is independent of syntax that essentially arrives in communication as a “word heap.” Essentially, word choice selections fall into the metaphorical category and the combination of words (syntax) can be categorized as part of the metonym camp.

The distinction between metaphor and metonym, and its proposed binary structure, are important for user interface design because this construct separates the elemental pieces of information that elicit user cognition (the metaphors) from the time-based sequencing of metaphor presentation (the metonym). Through a separation of the pieces creating the whole, it becomes easier to critique and analyze the entire interface. Breaking down the complex form into smaller, easier-to-analyze, elements is a process we undertake constantly, so it makes sense that if we want to better understand human interface design, we can naturally apply this task at the deep-rooted metaphor/metonym level. It also becomes easier to apply fundamental design principles (such as contrast or shape) as a second layer of construction to the individual pieces and help unify the interface- if each element is specifically designed (controlled), there is design (control) across the interface.

We can also appreciate the metaphor element in a visual interface as part of a system, a universal and fundamental biological system within all humans, which functions on the basis of the interpretation of signs through visual reception. All humans with the ability to visualize simultaneously have the ability to interpret metaphor- regardless of the fact that the interpretation could be different. On the other hand, metonymy by definition, is not a universal system- its effective use relies on a culturally agreed upon meaning based on a sequence of metaphors. This is critical to the development of user interfaces for people of different cultures; and the collapsing distance of communication and commerce worldwide heightens this criticality.

Breaking Down the Metaphor and Metonym

If we follow the teachings of Saussure, we are also introduced to the temporal and spatial facets of metaphor and metonym. This is appealing to interface design, and especially user experience design, because of the interest in the time and space based aspects of interaction. An interface can be analyzed not only in terms of navigation from one element to another on a screen, possibly employing Hick’s or Fitts’s Laws for testing, but also by the time elapsed during the cognition of individual metaphors presented. Using this principle, one could imagine mapping the initial designs of an interface based on the time it takes to understand the metonymic structure of metaphors- the design would appear as numbers in place of buttons, graphics, or other visual user interface elements.

Here, at perhaps the deepest and most fundamental level of design, the level of the metaphor and metonym, it is possible to begin construction of an interface that is highly designed with superior control over the presentation of elements and anticipated cognition of a user. At this level, by beginning the design process with the metaphor/metonym structure outright, there is extraordinary potential to achieve the greatest emotive relationship between and amongst a human user, their mobile tool, and the designer (company or individual). In terms of capitalism (or hyper capitalism) and the corporate environment, customer satisfaction, brand loyalty, and the bottom line can be maximized. In terms of human development, there is the potential to maximize learning, interpretation, and task achievement. The processes of mobile users, including conducting business, communicating with others, enjoying entertainment, and fulfilling personal satisfaction through information gathering and memory casting, can all be heightened when design occurs at the metaphor/metonym level.

Of course, this process is already employed- icons represent other things, the layout of an

application may represent the layout of another more familiar interface. Since we understand our world through metaphor/metonym, it’s obvious that we would reconstruct other visual worlds by the same process. However, there is a lack of documentation of the use of metaphor/metonym in user interface design and user experience literature and an almost unconscious use of the binary in design and development. We tend to take the usage of the power of metaphors and metonyms for granted.

In most user experience literature, there is a scientific approach to understanding visual

interpretation; and inherently in any scientific examination, there is study bias arising from several factors (in the study environment, in sampling [amazon mechanical turk], user tasking, participant compensation). Individual designer bias (a personal and culturally developed property) exists in the interpretation of a user’s cognitive abilities during interface creation. All of these approaches toward interface design utilize metaphor and metonym, but utilize them without deep recognition. The researchers and designers are aware of metaphor and metonym in the interface under investigation, but fail to account for the metaphor/metonym baggage that every human being carries. Without this accounting, there is an assumption of a ‘totalization’ of human-device interaction and intensive critical discourse on interface design is lacking. Less inferential feedback and user testing is necessary by way of a critical analysis of the metaphor/metonym dichotomy in place.

Following Through the Attempt

There is an emphasis on design in economics, politics, and social relations. In mobile interface design there is a quickening collapse between reality and the metaphor for the user. Therefore, it is crucial, in our moment, to understand and appreciate the use of the metaphor/metonym dichotomy in the future design and development of interfaces across the disciplines of programmers, designers, and researchers working in the field.

For Further Reading and Reference Search These Out:

Google Publications User Experience, Marshall McLuhan, Charles Gaines, Roman Jakobson, Ferdinand de Saussure, Immanuel Kant