JCPSLP Vol 21 No 3 2019

multiple modalities as being relevant when people use additional, non-talk-based systems for communicating (Pierce et al., 2019). Another way of thinking about modalities could start from the sensory and motor media through which people convey meaning. For instance, Stivers and Sidnell (2005) draw a contrast between the vocal/aural modality and the visuospatial modality . (Touch also provides a distinctive modality that may be exploited for communication, as do features of the material environment and objects in it.) Within each modality, there are a number of channels . In the vocal modality, there are ‘lexico-syntactic’ and ‘prosodic’ channels, each of which carries distinctive semiotic potential. The visuospatial modality is more diverse, and includes channels for body positioning, gaze, facial expression, and gesture. For people with typical speech, language, and hearing, the semiotic tasks associated with vocal modality tend to be symbolic, whereas those associated with the visuospatial modality tend to be more iconic (Stivers and Sidnell, 2005). 1 The temporalities and sensory presentations of these modalities are also contrastive. The vocal modality is discontinuous, conveyed in short bursts with clear starting and ending points. On the other hand (so to speak!), the channels of the visuospatial modalities are persistently available for people who are co-present (and co-located). This means that the movements, postures, and positioning of peoples’ bodies offers a continuous basis for communication; both with and without the discontinuous signals conveyed via the vocal modality. As one might expect, in typical communication, the vocal modality is used for many communicative functions, with turn-taking, sequences, and repair grounded in talking. However, there are distinctive and recurrent communicative functions that are achieved using the channels of the visuospatial modality. I will now highlight just a handful of examples. At a coarser level, body positioning and overall body movement is central for managing participation status. That is, the way people position their head, trunk, arms, hands, etc., in space (and relative to others) provides important signals for how people are understanding the communication situation and carrying out their role in it (see, e.g., Mondada, 2013). For instance, someone turning their head but not their trunk may indicate temporary involvement in one communication situation, while demonstrating a continuing commitment to another one (Schegloff, 2003). At a finer level, speaker and listener gaze are important resources for managing turn-taking and sequences. For example, both with and without explicit indications in the vocal modality, speaker gaze can signal which person should speak next (e.g., Lerner, 2003) or when a distinctive part of the ongoing communication situation is coming to a close (e.g., Rossano, 2013). Gestures and facial expressions are similarly essential for creating communicative actions. For instance, points and other hand gestures are important for requests, facilitating the listener’s understanding of the upcoming vocal request, and increasing its likelihood of success (e.g., Keisanen & Rauniomaa, 2012). Similarly, facial expressions can be used to project an upcoming evaluative action, and may solicit congruent communicative acts from other people (e.g., Ruusuvuori & Peräkylä, 2009). It is also worth considering just how these modalities come to be integrated with one another. For the most part, people will design their conduct across different modalities so that they cohere into meanings suited to their communicative acts and moments. Put another way, people will employ talk, gaze, body positioning,

facial expression, etc., that is complimentary; both in the sense that they are pulling in the same communicative direction, and in the sense that they carry distinctive mean-making burdens. 2 For instance, saying ‘can you pass me that’, in combination with purposeful gaze and pointing – and undertaken at a suitable communicative moment – will add up into a request for the sauce (or whatever else) from a particular person. So, although the vocal modality (and lexico-syntactic channel) may be in the foreground, communicative acts are best thought of as “multimodal gestalts” (see Mondada, 2019). This means that people communicate using combinations of meaning-making resources across multiple modalities, with each modality combining to become something more than the sum of their parts. In addition, the simultaneity of modalities and their channels makes possible multi-activity (Mondada, 2019). For example, at the dinner table, spirited conversation may be carried out via the vocal modality while the very same people simultaneously coordinate the passing of dishes and condiments via another modality. This simple example highlights yet another important dimension for a technical notion of multimodal communication: the possibly that different modalities are simultaneously accomplishing multiple communicative functions across multiple communicative activities. What’s next for multimodal communication? Multimodality is a pervasive feature of co-present human communication, regardless of the language(s) people speak, the cultures they belong to, the communicative abilities they have to employ, or the practical activities they are engaged in. Our profession’s idea of multimodal communication has been useful for highlighting the communication needs and rights of people with communication disorders in general, and complex communication needs in particular. As a next step, I have argued that we have some complicated thinking ahead to improve the ideas behind multimodal communication (see also Pierce et al., 2019). In summary, this would involve establishing theoretically and empirically motivated approaches to: 1) modalities; 2) relationships between modalities; 3) the communicative functions of modalities; and, 4) the relationships between modalities and practical activities. Embracing the depth and pervasiveness of multimodality in communication will enable us to better capture how communication disorders impact everyday life; especially when people employ atypical configurations of modalities and channels. With this renewed perspective, we can then implement better, and more targeted, assessment, intervention, and advocacy. 1 Of course, languages like Auslan, tactile sign language, whistled registers of languages, etc., demonstrate that there are a variety of ways that the symbolic functions conventionally associated with lexico-syntactic channel of the vocal modality (i.e., words, phrases, and sentences) can be redistributed to alternative modalities and channels. 2 By contrast, some communicative acts (e.g., sarcasm and jokes) are designed to leverage contrasts between modalities (e.g., delivering a ‘serious’ facial expression alongside “non- serious” talk). References Barnes, S., & Bloch, S. (2019). Why is measuring communication difficult? A critical review of current speech

133

JCPSLP Volume 21, Number 3 2019

www.speechpathologyaustralia.org.au