Design, Adaptation and Convention: The Emergence of Higher Order Graphical Representations Nicolas Fay (nfay@atr.jp) ATR Media Information Science Labs, 2-2-2 Hikaridai, Keihanna Science City, Kyoto 619-0228, Japan Simon Garrod (simon@psy.gla.ac.uk) Tracy MacLeod (tracym@psy.gla.ac.uk) Department of Psychology, University of Glasgow, Glasgow, G12 8QB, Scotland John Lee (j.lee@ed.ac.uk) Jon Oberlander (j.oberlander@ed.ac.uk) HCRC, University of Edinburgh, 2 Buccleuch Place, Edinburgh, EH8 9LW, Scotland Abstract example (Figure 1), we argue that conventions are culturally evolved higher order cognitions. Over several thousand years the original Chinese character that represents mountain (left) has evolved into its current, less complex, form (right). We argue that this change is not arbitrary; it is a result of global coordination that took place over time and space, culminating in a refined, conventional form that promotes rapid communication with reduced effort. This is an example of an evolutionary process where the whole is greater than the sum of its parts. To study the development of graphical conventions we had members of a simulated community play a series of graphical interaction games with partners drawn from the same pool (Experiment 1). Once the community was established, a conventional graphical referring scheme emerged that facilitated high levels of semantic coordination, with reduced communicative effort. Next, a forced choice reaction time study (Experiment 2) demonstrated that the graphical conventions developed in the simulated community offer distinct processing advantages when compared with those developed by isolated pairs (i.e. participants who always interact with the same partner). This is interpreted as evidence that the graphical conventions that evolve within a closed community constitute higher order cognitions, the whole being greater than the sum of its parts. Figure 1. The changing form of the Chinese character that represents mountain (Vacarri & Vacarri, 1961; cited in Arbib, in press) Background Vygotsky (1981) claims that higher order cognition is a product of social interaction, that novel structures emerge as a consequence of interpersonal, as opposed to intrapersonal, communication. Hutchins (1995) shares this view, arguing that higher order cognition is a cultural product, a consequence of interaction (human- environment and human-human) that is distributed across time and space. According to Hutchins, higher order cognitions emerge from “an adaptive process that accumulates partial solutions to frequently encountered problems” (p.354). Lewis (1969, 1975) defines conventions in a related way, as arising from situations where a community faces the recurrent problem of coordination. If we agree that conventions are cultural products, should we accept that they represent higher order cognitions? Using Chinese characters as an Having partners collaborate on a graphical referential communication task, Fay, Garrod, Lee and Oberlander (2003) studied the influence of interaction upon representational form. The task requires pairs of participants to graphically communicate a series of recurring concepts. Figure 2 illustrates the changing representation of the concept ‘Clint Eastwood’ over 6 games, where partners’ drawing and identifying roles alternated from game to game. What is initially a designed, iconic representation of Clint Eastwood, develops, through a process of adaptation and entrainment, into a simplified, symbolic form (an arrow pointing East). Although there are obvious similarities between this process and the evolution of Chinese characters, the derived representation of Clint Eastwood does not constitute a convention in Lewis’ terms. According to Lewis, a convention must be common knowledge within the
There's a saying that 'a picture says a thousand words'. Research supports the notion that pictures have benefits over words. Smith and McGee (1980) found that people extract meaning more quickly from pictures than from words. This observation goes some way to explaining the prolific use of signs in modern society. Road signs provide a case in point, taking advantage of the limited exposure time drivers are afforded (especially those driving at speed).
This thesis explores the effects of group size on gesture communication. Signs in general change, in the kind of information they convey and the way in which they do so, and changes depend on interactive communication. For instance, speech is like dialogue in smaller groups but like monologue in larger groups. It was predicted that gestures would be influenced in a similar way by group size. In line with predictions, communication in groups of 5 was like dialogue whereas in groups of 8 it was like monologue. This was evident from the types of gesture that occurred with more beat and deictic gestures being produced in groups of 5. Iconic gesture production was comparable across group size but as predicted gestures were more complex in groups of 8. This was also the case for social gestures. Findings fit with dialogue models of communication and in particular the Alignment Model. Also in line with this model, group members aligned on gesture production and form.
It has been suggested that iconic graphical signs evolve into symbolic graphical signs through repeated usage. This article reports a series of interactive graphical communication experiments using a 'pictionary' task to establish the conditions under which the evolution might occur. Experiment 1 rules out a simple repetition based account in favor of an account that requires feedback and interaction between communicators. Experiment 2 shows how the degree of interaction affects the evolution of signs according to a process of grounding. Experiment 3 confirms the prediction that those not involved directly in the interaction have trouble interpreting the graphical signs produced in Experiment 1. On the basis of these results, this article argues that icons evolve into symbols as a consequence of the systematic shift in the locus of information from the sign to the users' memory of the sign's usage supported by an interactive grounding process.
To study the development of graphical conventions we had members of a simulated community play a series of graphical interaction games with partners drawn from the same pool (Experiment 1). Once the community was established, a conventional graphical referring scheme emerged that facilitated high levels of semantic coordination, with reduced communicative effort. Next, a forced choice reaction time study (Experiment 2) demonstrated that the graphical conventions developed in the simulated community offer distinct processing advantages when compared with those developed by isolated pairs (i.e. participants who always interact with the same partner). This is interpreted as evidence that the graphical conventions that evolve within a closed community constitute higher order cognitions, the whole being greater than the sum of its parts.