Ideophones, expressiveness and grammatical integration

Ideophones —vivid sensory words found in many of the world’s languages— are often described as having little or no morphosyntax. That simple statement conceals an interesting puzzle. It is not often that we can define a word class across languages in terms of its syntax (or lack thereof). After all, most major types of word classes show intriguing patterns of cross-linguistic variation. There is no particular reason to expect that the morphosyntactic position or degree of embedding of, say, noun-like or verb-like words will be similar across unrelated languages. Indeed that is why typologists define comparative concepts primarily by reference to semantic rather than grammatical or morphosyntactic properties (Croft 2003; Haspelmath 2007).  Continue reading

Sound symbolism boosts novel word learning: comparing ideophones and adjectives

We have a new paper out. It’s actually been available since February in an online-first version, but for those of us who love page numbers and dead trees, the journal has now printed it in its August issue on pages 1274-1281. Citation:

Lockwood, G., Dingemanse, M., & Hagoort, P. (2016). Sound-symbolism boosts novel word learning (PDF). Journal of Experimental Psychology: Learning, Memory, and Cognition, 42(8), 1274-1281. doi:10.1037/xlm0000235

This is another one for which we’ve made available the stimuli —word lists and sound files— through OSF, contributing to our mission to make research from our lab replicable. Also, we have since replicated the results in a follow-up study where we also took EEG and individual difference measures.

I provide a quick summary in 2×3 points below. For a write-up that’s much more fun and has great illustrations, check out Gwilym Lockwood’s Sound symbolism boosts novel word learning: the MS Paint version. Continue reading

What sound symbolism can and cannot do: new paper in Language

What sound symbolism can and cannot doWe have a new paper out in Language:

Dingemanse, Mark, Will Schuerman, Eva Reinisch, Sylvia Tufvesson, and Holger Mitterer. 2016. “What Sound Symbolism Can and Cannot Do: Testing the Iconicity of Ideophones from Five Languages.” Language 92 (2): e117–33. doi:10.1353/lan.2016.0034

The basic finding is this: people are sensitive to the meaning of ideophones they’ve never heard, even when they are produced out of context by a computer voice in a difficult forced choice task. Yet there is also reason for caution: the effect is not nearly as strong as what people have found for pseudowords like bouba and kiki.

As we note in the introduction, “there appears to be a tendency to either underplay or exaggerate the significance of iconicity in the study of language and mind”. In this paper we chart a middle way between these extremes. Here’s a quick summary in 3×3 points:

What we did:

  1. Sound symbolism (iconicity in spoken language) is usually studied using hand-crafted pseudowords in binary forced choice experiments (think bouba and kiki, as reviewed here), but there are three problems with such experimental designs: (i) they run the risk of inflating effect sizes, (ii) it is unclear how they relate to natural languages, and (iii) they usually don’t control for prosody.
  2. We designed a study to tackle these problems by (i) adjusting the binary choice task to be more realistic and harder, (ii) using real words and meanings from natural languages, and (iii) teasing apart prosody and segmental features. Essentially, we bring linguistic insights to bear on the psychological study of sound symbolism.
  3. We take 203 ideophones —lexical sound-symbolic words— from 5 languages and 5 semantic domains and present them to 80 participants in 4 versions: (i) full original recording, (ii) full speech synthesized version, (iii) prosody-only condition and (iv) phonemes-only condition. The versions help us control for variation due to different speakers and help us examine the contributions of prosody and segmental features.

What we found:

  1. People can choose the correct translation of ideophones at a level significantly above chance. So ideophones in Japanese, Korean, Semai, Ewe and Siwu are not fully arbitrary, as is normally assumed of words; they contain iconic cues that even people who don’t speak these languages can pick up.
  2. Sound ideophones are easiest to guess, but the other semantic domains (movement, texture, color/visual appearance, and shape) come out significantly above chance as well. However, the effect is much more modest than most bouba/kiki studies: in the best versions, people score 57.2% on average (where 50% would be chance level) — quite different from the 95% that has sometimes been claimed for pseudoword studies.
  3. Performance for the original and resynthesised stimuli is indistinguishable, so our speech synthesis method works. Performance is significantly better for the full versions (i-ii) than for the reduced versions (iii-iv), so both prosody and phonemes contribute to the effect (and neither alone is sufficient).

What we conclude:

  1. Findings based on pseudowords like bouba/kiki cannot be automatically translated into claims about natural languages. Ideophones combine iconicity and arbitrariness, and lexical iconicity in ideophones is best characterised as a weak bias, which is supported by multimodal performances in actual use and which may be amplified in cultural evolution (cf our TiCS paper).
  2. Prosody is just as important as segmental information in supporting iconic interpretations (as predicted here). Prior work, which has rarely controlled for prosody, likely overestimates the role of speech sounds and underestimates the role of intonation, duration and other prosodic cues.
  3. Speech synthesis offers a viable way to achieve experimental control in the study of sound symbolism. To stimulate its wider use we’re making available all stimulus materials, including the diphone synthesis source files we used to create them. Get them at MUSE or OSF.
Here’s the abstract:

Sound symbolism is a phenomenon with broad relevance to the study of language and mind, but there has been a disconnect between its investigations in linguistics and psychology. This study tests the sound-symbolic potential of ideophones—words described as iconic—in an experimental task that improves over prior work in terms of ecological validity and experimental control. We presented 203 ideophones from five languages to eighty-two Dutch listeners in a binary-choice task, in four versions: original recording, full diphone resynthesis, segments-only resynthesis, and prosody-only resynthesis. Listeners guessed the meaning of all four versions above chance, confirming the iconicity of ideophones and showing the viability of speech synthesis as a way of controlling for segmental and suprasegmental properties in experimental studies of sound symbolism. The success rate was more modest than prior studies using pseudowords like bouba/kiki, implying that assumptions based on such words cannot simply be transferred to natural languages. Prosody and segments together drive the effect: neither alone is sufficient, showing that segments and prosody work together as cues supporting iconic interpretations. The findings cast doubt on attempts to ascribe iconic meanings to segments alone and support a view of ideophones as words that combine arbitrariness and iconicity. We discuss the implications for theory and methods in the empirical study of sound symbolism and iconicity.

Description and depiction

Depiction is a technical term used in psychology, philosophy, and art history, but less so in linguistics. One of my claims about ideophones is that they are best understood (typically, canonically, with the customary provisos about the fluid categories of natural language) as depictive words. Do we really need a term like that? Can’t we make do with good old ‘expressive’ or ‘vivid‘? No, I say. But what then is this elusive concept of depiction? How can we tell that something is intended as a depiction? What is depiction, such that we can distinguish it from other modes of representation? In answering these questions, it is useful to make a distinction between the signals that frame something as a depiction on the one hand, and the aspects of depiction as a mode of representation on the other hand. But let’s start with something more basic: words and images.

Words versus images

There is a domain where distinct modes of representation are very important and very clear: that of visual representations, in particular written words versus images. Take the following slide (from one of my presentations), with on the left a sentence involving le soleil “the sun” and on the right an image of the sun. (Bonus points for those who see whose handwriting that is.) The claim here is a simple one: that the left and the right side involve two distinct modes of representation.

Description and depiction

Description and depiction (Mark Dingemanse)

I’m guessing most readers will agree with at least this basic distinction. What we call this distinction doesn’t matter a lot, but I call it, following the literature, “description” versus “depiction”. More important than the labels are the ways in which the phenomena differ. On the slide above, a first opposition is between propositional vs. imagistic. This is in a way restating the same point (see Kosslyn 1980 for more details), so I’ll focus on the remaining three. There are three key differences that help us to deductively distinguish depictions from descriptions: descriptions differ from depictions (1) in terms of symbol system they use (using discrete symbols vs. using gradient markings), (2) in terms of form-meaning mappings (basically arbitrary vs. basically iconic), (3) in terms of how we interpret them (“decode” to interpret vs. “imagine” to interpret).

Words as images

Words can't describe! (toothpaste for dinner)

Words can’t describe! (toothpaste for dinner)

Without even having started worrying about ideophones, this is a distinction that we need to make. It’s a distinction that is motivated on independent grounds. Words are different from images in their mode of representation, and we find a number of clear differences between the descriptive and depictive modes of representation.

The next step of my argument consists simply in noting that precisely those three key ways in which descriptions differ from depictions (in the visual mode) are also ways in which plain words differ from ideophones (in spoken language). Ideophones tend to use more gradient and discrete symbol systems; their form-meaning mappings tend to be more iconic than arbitrary; and to interpret them, we “imagine” more than we “decode” (Dingemanse 2011, 2012). Of course these differences are not absolute; in my thesis I have pointed out, for instance, that convention (and hence decoding) also plays an important role in depiction. After all ideophones are conventionalised words and not creative formations — most of the time.

When I say that ideophones are depictions, this is a statement about their mode of representation. It is more about a way some thing can be than about the thing itself. Take the sun. That’s a pretty good ‘thing’. Now take the two representations of the sun shown above. Even those representations are tangible ‘things’, in the sense that we can talk about them, point at them, isolate them (as I do on the slide above). But the way in which those two representations differ, that is their mode of representation. It is about the way something can represent something else. That is the sense in which I call an image of the sun and an ideophone in spoken language a depiction.  Continue reading

Advances in the cross-linguistic study of ideophones

Just out: a review of ideophone research by yours truly, titled Advances in the cross-linguistic study of ideophones, published in Language and Linguistics Compass. This article focuses on some recent developments in ideophone research. Some of the things it offers include a cross-linguistically viable definition of ideophones; an argument for why ideophones are structurally marked; a review of how ideophones depict sensory imagery; some tools for developing a semantic typology of ideophones; an implicational hierarchy of ideophone systems; and an argument for studying ideophones in actual use rather than just in narratives.

The review doesn’t say much about the morphosyntactic typology of ideophones because there will be another publication coming out on precisely that topic (‘Expressiveness and system integration‘, in STUF – Language Typology and Universals); and it doesn’t say much about ideophones and gesture because there is a chapter on that in my thesis (as well as forthcoming publications).

What’s wrong with “vivid”? The evolution of a definition

A "vivid" butterflyIdeophones, like so many things in life, are easy to identify but hard to define. Many researchers have grumbled about the shortcomings of Doke’s descriptive characterization of ideophones (see discussion here), but few have attempted to formulate an alternative. For better or worse, I did, but it took me a few iterations to arrive at something that I felt worked well enough to be useful in cross-linguistic research. One of these iterations featured the word “vivid”, but I’ve since dropped it. Why? Continue reading

New issue of SemiotiX

Just out: a new issue of SemiotiX, the e-journal on all things semiotic edited by prof. Paul Bouissac. Among other things, it features a guest column by yours truly.

Ever felt sceptical about the supposed iconicity of ideophones like Siwu kananaa ‘silent’ or Japanese iya iya ‘with a heavy heart’? Isn’t it similar to the cute, but mistaken story that all Chinese characters are like little pictograms? And what have these two things got to do with the paintings of Van Gogh, Duchamp, and Mondriaan? Read all about it.

Van Gogh, Almond Blossoms; Duchamp, Nu descendant un escalier; Mondriaan, Victory Boogie Woogie

  1. Dingemanse, Mark. 2012. “Coerced Iconicity in Writing and Speech.” SemiotiX XN-8 (September).

Phonosemantics, Chinese characters, and coerced iconicity

Update: SemiotiX issue XN-8 features a revised and expanded version of this essay.

The light descending (from the sun, moon and stars.) To be watched as component in ideograms indicating spirits, rites, ceremonies.The linguistic blogosphere featured some posts recently on the topic of phonosymbolism, phonosemantics, and Chinese characters. It started with a post by Victor Mair over at Language Log, outlining several approaches to “etymologizing” Chinese characters. A follow-up by David Branner highlighted some of the problems with simplistic notions of phonosymbolism. Here I add some texture to the conversation by discussing the views of Ezra Pound, making a comparison to form-meaning mappings in ideophones, and introducing the notion of coerced iconicity. Continue reading

Daniel Tammet invents his own Siwu ideophone

I loved Daniel Tammet’s second book Embracing The Wide Sky (2009). In his own words, Embracing The Wide Sky is “a personal and scientific exploration of how the brain works and the differences and similarities between savant and non-savant minds”. It surveys work from psychology and linguistics and even indirectly (okay, very indirectly) features my work on Siwu ideophones. Continue reading