Why article-level metrics are better than JIF if you value talent over privilege

Posted on by
1

I’ve been caught up in a few debates recently about Recognition and Rewards, a series of initiatives in the Netherlands to diversify the ways in which we recognize and reward talent in academia. One flashpoint was the publication of an open letter signed by ~170 senior scientists (mostly from medical and engineering professions), itself written in response to two developments. First, the 2019 shift towards a “narrative CV” format in grant applications for the Dutch Research Council (NWO), as part of which applicants are asked to show evidence of the excellence, originality and impact of their work using article-level metrics instead of journal level metrics like the Journal Impact Factor (JIF). Second, the recent announcement of Utrecht University (signatory of the Declaration on Research Assessment) to abandon the JIF in its hiring and promotion processes (see coverage).

Why funders in search of talent are ditching the JIF

Some background will be useful. The decision to not use JIF for the evaluation of researchers and their work is evidence-based. There is a lot of work in bibliometry and beyond showing that a 2-year average of a skewed citation distribution is an imperfect measure of journal quality, a driver for perverse incentives, and a misleading proxy for the quality of individual papers. Indeed Clarivate itself, the for-profit provider of the JIF metric, has this to say about it: “In the case of academic evaluation for tenure, it is inappropriate to use a journal-level metric as a proxy measure for individual researchers, institutions, or articles”.

Despite this evidence, JIFs have long been widely used across the sciences not just as a way to tell librarians which journals are making waves (= what they were designed for) but also as a quick heuristic to judge the merits of work appearing in them or people publishing in them. As they say, ‘Don’t judge a book by its cover, but do judge scientific work by its JIF’. There is a considerable halo-effect attached to JIFs, whereby an article that ends up in a high IF journal (whether by sheer brilliance or simply knowing the right editor, or both) is treated, unread, with a level of veneration normally reserved for Wunderkinder. Usually this is done by people totally oblivious to network effects, gatekeeping and institutional biases.

It appears that the decision to explicitly outlaw the use of JIFs now has people coming out of the woodwork to protest. The first letter (and also another one by a number of junior medical scientists) is aimed specifically at the prohibition against using the JIF, which is (incorrectly) framed as a ban on all quantification. The feeling is that this deprives us of a valuable (if inexact) metric that has long been used as a quick heuristic of the ‘value’ or ‘quality’ of work.

‘Halo? What halo?’

Raymond Poot, main author of the first letter, strongly believes that the JIF, even if inexact, should not be ditched. Saying, “Let’s talk JIF”, he provides this diagram of citation distributions in support:

The diagram compares the citation distributions of Nature and PLOS ONE (an open access megajournal). Poot’s argument, if I understand it well, is that even if Nature’s JIF is skewed by a few highly cited papers, the median number of citations is still higher, at 13, than the median number of cites that PLOS ONE papers receive (which looks like 1). As Poot says in reference to an earlier tweet of mine on the halo-effect, ‘Halo? What halo?’.

We want to identify and reward good work wherever it appears

We’ll get to that halo. First things first. We’re talking about whether using the JIF (a journal’s 2-year citation average) is a good idea if you want to identify and reward good individual work. And especially whether using the JIF is better or worse than using article-level metrics. Another assumption: we care about top science so we would like to identify good work by talented people wherever it appears. Analogy: we want to scout everywhere, not just at the fancy private school where privileges and network can obscure diverse and original talent.

Let’s assume the figure represents the citation distributions reasonably well (I’m going to ignore the obvious folly of taking an average of a very skewed and clearly not unimodal distribution). Where is the JIF halo? Right in front of you, where it says, for publication numbers, “in thousands for PLOS ONE”. Publication volume differs by an order of magnitude. This diagram hides that by heavily compressing the PLOS distribution, which is never good practice for responsible visualization, so let’s fix that. We’ll lose exact numbers (they’re hard to get) but the difference is large enough for this to work whatever the numbers.

The enormous difference in sheer volume means that an OA megajournal is likely to have quite a few papers with more cites than the Nature median — high impact work that we would miss entirely if we focused only on the JIF. The flip side is where we find the halo effect: there are, in any given year, hundreds of Nature papers that underperform quite a bit relative to the IF (indeed half of them underperform relative to the median). This —the skewed distributions for both the megajournal and the glamour journal— shows why it is a bad idea to ascribe properties to individual papers based on how other papers published under the same flag have been cited.

On average, my paper is better than yours

“But still, surely on average Nature papers are…” — besides the point. I would rather give a talent grant to the bright student who made their way through the public school system (an over-performing PLOS paper) than to the one who dangles at the bottom of the distribution of their privileged private school (an underperforming Nature paper). Identifying talent on the basis of JIF instead of content or impact is like handing out bonus points to private school essays in the central exam. “But on average those elite schools do tend to do better don’t they?” Unsurprisingly, they do, and if you think such differences are meaningful or worth further reinforcement , it’s worth reading some more sociology, starting perhaps with the diversity-innovation paradox.

There are other issues with our hyperfocus on glamour journals. These journals like to publish good work but they also apply some highly subjective filters (selecting for ‘broad appeal’ or ‘groundbreaking’ research — phrases that will sound familiar from the desk-rejects that statistically speaking many readers from academia will have seen). Nature prides itself on an 8% acceptance rate, the same chances that we rightly call a lottery when it concerns grant proposals. Being overly selective inevitably means that you’ll miss out on top performers. One recent study concluded that this kind of gate-keeping often leads us to miss highly impactful ideas and research:

However, hindsight reveals numerous questionable gatekeeping decisions. Of the 808 eventually published articles in our dataset, our three focal journals rejected many highly cited manuscripts, including the 14 most popular; roughly the top 2 percent. Of those 14 articles, 12 were desk-rejected. This finding raises concerns regarding whether peer review is ill-suited to recognize and gestate the most impactful ideas and research.

Gatekeepers of course also introduce their own networks, preferences and biases with regards to the disciplines, topics, affiliations, and genders they’re more likely to favour. In this context, Nature has acknowledged the sexism of how its editorial boards are constituted, and as the New York Times wrote last year, the publishing process at top journals is “deeply insular, often hinging on personal connections between journal editors and the researchers from whom they solicit and receive manuscripts”.

From smoke and mirrors to actual article-level impact

“But doesn’t my Nature paper count for anything?” I sure hope it does. And the neat thing is, under the new call for evidence-based CVs you can provide evidence and arguments instead of relying on marketing or association fallacies. Do show us what’s so brilliant and original about it. Do tell us about your contributions to the team, about the applications of your work in industry, and about the relative citation ratio of your paper. Indeed, such article-level metrics are explicitly encouraged as a direct indicator of impact and originality. To spell it out: a PLOS ONE paper that makes it to the 1% most cited papers of its field is more telling than, say, a Nature paper of the same age that has managed to accrue a meagre 30 cites. An evidence-based CV format can show this equally for any type of scientific output, without distracting readers with the smoke and mirrors of the JIF.

Scientists are people, and people are easily fooled by marketing. That is going to be the case whether we mention the JIF or not. (The concerned medical scientists writing the letters know full well that most grant reviewers will know the “top” journals and make inferences accordingly.) The purpose of outlawing the JIF is essentially a nudge, designed to make evaluators reflect on this practice, and inviting them to look beyond the packaging to the content and its actual impact. I can only see this as an improvement — if the goal is to identify truly excellent, original and impactful work. Content over silly bean counts. True impact over halo effects.

If you want to find actual impact, look beyond the JIF

I have focused so far on PLOS ONE and Nature because that’s the example provided by Raymond Poot. However, arguably these are two extremes in a very varied publishing landscape. Most people will seek more specialised venues or go for other multidisciplinary journals. But the basic argument easily generalizes. Most journals’ citation distributions will overlap more than those of PLOS ONE and Nature. For instance, let’s take three multidisciplinary journals titrated along the JIF ranks: Science Advances (14.1), PNAS (11.1), and Scientific Reports (4.4). Set up by Nature to capture some of the market share of OA megajournals, Scientific Reports is obviously less artificially selective than the other two. And yet its sheer publication volume means that a larger number of high impact papers appear in Scientific Reports than in PNAS and Science Advances combined! This means, again, that if you want to find high impact work and you’re just looking at high IF journals, you’re missing out.

Trying to find good or impactful work on the basis of the JIF is like searching for your keys under the streetlight because that’s where the light is. Without it, we stand a better chance of identifying truly groundbreaking work across the board — and fostering diversity and innovation in the process.

Caveats. I’ve used article-level citations as a measure of impact here because they most directly relate to the statistically illiterate but widespread use of the JIF to make inferences about individual work or individual researchers. However, citations come with a time lag, are subject to known biases against underrepresented minorities, and are only one of multiple possible measures of originality, reach and impact. Of course, to the extent that you think this makes actual citations problematic as indicators of article-level impact or importance, it means the JIF is even more problematic.

WOCAL10 workshop: Centering pragmatic phenomena on the margins

Posted on by
Reply

With the tenth World Congress of African Linguistics around the corner (June 7-12, 2021), let me draw your attention to a workshop we are organizing: Centering pragmatic phenomena on the margins in African languages. Convened by Felix Ameka and Mark Dingemanse, this workshop gathers researchers from at least 8 African universities and from around the world for a report on the latest development in this exciting research area. Workshop abstract:

In pragmatics, as in linguistics in general, various expressive devices that are indispensable in communication have been left on the margins as being non-conventional, non-lexical or non-verbal. This includes a range of interjections, particles, response cries, calls and conversational gestures but also bodily conduct such as sighs, sniffs, coughs, and winks. Despite their ubiquity in everyday interaction, many of these devices are thought of as extra-linguistic or paralinguistic and have consequently been mostly ignored in theoretical and empirical linguistic work. There is a realization on the rise in the language sciences that grappling with these devices holds the key to an understanding of language, the unique feature of the human species. In this workshop, we focus on the interactional uses of linguistic elements, or more broadly semiotic resources, that are traditionally thought of as extra-grammatical, non-lexical, or para-linguistic based on linguistic practices and norms in African communities of practice, with a view to moving them from the margins to the centre in African and general linguistics.

Felix Ameka & mark dingemanse, convenors

The workshop takes place online as part of WOCAL, for which registration is required. Registration is free for participants from the Global South and €50 for others. The fees are used to support the inclusiveness and diversity of the overall programme, including technical support, subtitling, live captioning, sign language interpretation and other measures.

Here’s the programme of our workshop:

Monday June 7, 16:00-18:00 — PRAG 1 🔗

1600-1630Intro | From liminal signs to phatic interjections
Mark Dingemanse
1630-1700Human-animal communication in the Iraqw language
Chrispina Alphonce
1700-1730Human-to-domesticated animal communication in Zargulla
Azeb Amha
1730-1800Masking emotions: a study of Hausa women’s expression of ‘kunya’
Aisha Umar Adamu

Break & possibility for socializing — and then PRAG 2 🔗

1900-1930Emotive interjections in Maasai (Arusa)
Michael Karani, Alexander Andrason
1930-2000Ideophones, interjections and particles: their forms and uses in Dompo
Ester Manu-Barfo
2000-2030Some utterance particles in Amharic conversations
Mulugeta Seyoum
2030-2100Pragmatic borrowing: the case of Tafi, a Ghana-Togo Mountain language
Mercy Bobuafor

Tuesday June 8, 16:00-18:00 — PRAG 3 🔗

1600-1630Forms and functions of backchanneling in Ruruuli and Luganda
Margaret Zellers, Mandy Lorenzen, Saudah Namyalo, Alena Witzlack-Makarevich
1630-1700Clicks as areal verbal gestures in the Lake Chad Basin and beyond
Florian Lionnet
1700-1730Moving clicks and other non-word vocalisations to the centre
Felix Ameka
1730-1800Final discussion All workshop participants (chaired by convenors)

You can download the workshop programme along with all abstracts here:

WOCAL10 Prag programme and abstractsDownload

The sound of rain, softly falling (Tucker Childs, 1948-2021)

Posted on by
Reply

News just reached me that we have lost a dear colleague and one of the people responsible for introducing the world of linguistics to African ideophones: George Tucker Childs, 1948-2021.

Tucker was a cheerful presence in the field of African linguistics and a towering figure in the subfield that he and I had in common, ideophone studies. His groundbreaking PhD dissertation on Kisi in 1988 was chock-full of these sparkling words evocative of sensory imagery, and the topic would never lose his interest. He was one of the pioneers of the sociolinguistic study of ideophones and his 1994 review of African ideophones remains one of the most cited chapters of a famed volume on Sound Symbolism. One of his last academic publications was a chapter in the 2019 proceedings of an international workshop on ideophones in which he, characteristically, combined acute fieldwork-based observations with perceptive questions for future research.

We met several times over the years and corresponded quite a bit, sometimes about new work, sometimes about the future of African linguistics and how to ensure better representation of its diversity. “This is an issue I wrestle with all the time, how best to encourage young [authors], especially African, to submit”, he wrote to me in his capacity as the editor of one of the specialist journals in the field. In Tucker we have lost an adventurous colleague driven by a sense of wonder and by a passion for the documentation and revitalization of endangered languages.

Tucker’s emails often started with a salutation that included a description of his location and the weather conditions — which, when they came from Portland, often meant rain. I enclose a rendition by artist Joanna Taylor of an evocative Kisi ideophone that appears in Tucker’s PhD thesis: bíààà ‘the sound of rain, softly falling’. Words are a poor substitute for human contact, but I wish his loved ones the serenity evoked by the sound of nourishing rain.

Note: If you want to write your condolences or share your memories of Tucker, his family set up a special website here.

Bibliography of Tucker Child’s ideophone-related publications (see Google Scholar)

  • Childs, G. Tucker. 1988a. The phonology and morphology of Kisi. University of California, Berkeley. (PhD dissertation.)
  • Childs, G. Tucker. 1988b. The phonology of Kisi ideophones. Journal of African Languages and Linguistics 10(2). 165–190. (doi:10.1515/jall.1988.10.2.165)
  • Childs, G. Tucker. 1989. Where do ideophones come from? Studies in the Linguistic Sciences 19(2). 55–76.
  • Childs, G. Tucker. 1994a. African Ideophones. In Hinton, Leanne & Nichols, Johanna & Ohala, John J. (eds.), Sound Symbolism, 178–204. Cambridge: Cambridge University Press.
  • Childs, G. Tucker. 1994b. Expressiveness in contact situations: the fate of African ideophones. Journal of Pidgin and Creole languages 9(2). 257–282.
  • Childs, G. Tucker. 1996. Where have all the ideophones gone? The death of a word category in Zulu. Toronto Working Papers in Linguistics 15. 81–103.
  • Childs, G. Tucker. 2001. Research on Ideophones, Whither Hence? The Need for a Social Theory of Ideophones. In Voeltz, F. K. Erhard & Kilian-Hatz, Christa (eds.), Ideophones, 63–73. Amsterdam: John Benjamins.
  • Childs, G. Tucker. 2003. An Introduction to African Languages. Amsterdam: John Benjamins.
  • Childs, G. Tucker. 2014. Constraints on violating constraints: How languages reconcile the twin dicta of “Be different” and “Be recognizably language.” Pragmatics and Society 5(3). 341–354. (doi:10.1075/ps.5.3.02chi)
  • Childs, G. Tucker. 2018. Forty-plus years before the mast: My experiences as a field linguist. In Sarvasy, Hannah & Forker, Diana (eds.), Word Hunters: Field linguists on fieldwork. John Benjamins Publishing Company.
  • Childs, G. Tucker. 2019. Ideophones as a measure of multilingualism. In Akita, Kimi & Pardeshi, Prashant (eds.), Iconicity in Language and Literature, vol. 16, 303–322. Amsterdam: John Benjamins Publishing Company. (doi:10.1075/ill.16.13tuc)

APA but without auto-sorting of in-text citations: easy CSL fix

Posted on by
Reply

For better or worse, APA is one of the most widely used citation styles in the cognitive sciences. One aspect of it that always bugs me is that it prescribes alphabetical sorting of in-text citations. I’m not talking about the bibliography; of course that should be alphabetical. I’m talking about the order of names when you cite multiple sources in one citation statement, as in “(Harris 1952; Chomsky 1957)” — or, as APA would have it, “(Chomsky 1957; Harris 1952)”.

My own strong preference is to have the order determined by priority and relevance. For instance, if I were writing about transformations in early generative syntax, I might want to cite both Chomsky and Harris, but I feel it would be useful to cite them in chronological order. Of course if I wanted to highlight Chomsky’s original contributions I could also do sth. like “(Chomsky 1957; and see Harris 1952 for a precursor)” — but the point is, in neither case would I want the ordering to be determined by a meaningless style prescription. The order of in-text citations is meaningful.

Now, if you’re using Zotero like me, you can already manually drag citations in any order you want. But still the APA default will hit you every now and then. Fortunately, this is really easy to fix in the CSL style, so I’m using a version of APA in which this is fixed. You can use my file, which is based on APA 7, currently the latest version.

I’m posting this for my future self as much as for others, so let me just note the utterly trivial single change you need to make in CSL terms. All you need to do is find the <citation> block and remove the <sort> statement inside it. That’s it. You can do this on your own system or in the online CSL code editor; or if you are more comfortable with the Visual Editor, you can also do it there. Save your adjusted style under a custom name to use it in Zotero.

  • before: select “sort” and click “-” to delete node
  • after: sorting is now as you want it (=custom)

Team science is slow science

Posted on by
2

With Times Higher Education writing about citation gaming and hyperprolific authors (surely not unrelated) I hope we can save some of our attention for what Uta Frith and others have called slow science. On that note, consider this: Team science is (often) slow science.

Recently two team science projects I’ve been involved in since the early 2010s resulted in publications: a book on recruitments, edited by Simeon Floyd, Giovanni Rossi en Nick Enfield; and a paper on sequence organization led by Kobin Kendrick. Some of the first results for both projects were presented at the 2014 International Conference on Conversation Analysis (ICCA) at UCLA, and it is so great to see them out now.

The Recruitments volume address the question of how we use language to get others to do things. It presents the results of collaborative research on 8 languages around the world and is published as an open access book by Language Science Press, linguistics’ most visionary open access publisher. This volume has a long history, and we’ve written about the background and methods of the project for the ROLSI blog. In a thread on Twitter, Nick Enfield sets out some of the key findings:

The sequence organization paper studies a basic aspect of how social action is organized in everyday language use in 12 languages around the world. It is published in Journal of Pragmatics, and first author Kobin Kendrick sets out the main findings in his own Twitter thread:

Our team work on repair —as part of Nick Enfield’s ERC grant— was similarly systematic and slow-paced; the special issue of Open Linguistics we edited is in many ways the sister to the recruitments book now out. From the start in 2010 it took us several years of intensive work before the first publications started coming out. The recruitment and sequence organization projects, which got off the ground a little later, had the additional challenge of an increasingly distributed team of collaborators (to the point that no one currently has the same affilation they had when the projects started).

This kind of systematic comparative work, which takes years to carry out and bring to fruition, is perhaps the antithesis of the hyperprolific output valued by bean counters. In this lies both a risk and a reward. The risk is that contributions can take years to become visible, which is especially tricky for early career researchers. The reward is that results tend to be solid and substantial. We need institutions & funders that don’t reduce us to output counts, and instead help us manage the risks and reap the long-term rewards of team science and slow science.

Notes

This post originated as a Twitter thread. The Frith paper mentioned in the first paragraph is:

New personal site

Posted on by
Reply

Almost 13 years ago, in 2007, this blog started as a sub-site on my personal web page. It soon took over most of my online presence and I moved it to its own domain. Now that I blog much less regularly, and have moved institutions, it’s useful again to have a personal academic web page. So I made one: markdingemanse.net.

This is also fitting because I have, over the past decade, developed a line of research on social interaction that doesn’t really fit what I’ve mostly blogged about on The Ideophone (which is topics around iconicity, ideophones, perception, and the senses). I will probably always keep working on ideophones, and I may well keep blogging here on various topics; but it was high time to have a web presence separate from this that more fully represents my scholarship and science communication work.

I used to have a really useful page at the MPI for Psycholinguistics, but an institutional move to Radboud University and a site redesign make the publication list there a little harder to navigate than it used to be. The neat thing is that Zotero (with ZotPress) makes it really easy to display full publication lists on my new site, even organised by topic:

Give it a gander, a glimpse or a glance — markdingemanse.net!

Liminal signs

Posted on by
Reply

I have a new paper out as part of a special issue filled to the brim with things on the border of language if not beyond it. There are seven empirical articles on response cries, “moans”, clicks, sighs, sniffs, & whistles, flanked by an intro (by editors Leelo Keevallik and Richard Ogden) and a commentary (by me). It was truly a privilege to sit down and spend time with this collection of papers to write a commentary; and quite the challenge to formulate a coherent take on phenomena so diverse in form and function, and so neglected in the language sciences.

Why are these things neglected? As I note in my commentary, there are at least three reasons: we’ve not been able to capture them until recently; some quarters of linguistics have been actively disinterested in them; but most intriguingly, they may be designed to be overlooked, or at least overlookable.

One challenge I set myself was to come up with a characterisation of these items that doesn’t focus on what they are not. “Non”-labels like non-lexical, non-linguistics, non-conventional, non-phonemic, non-committal et cetera buy into the framing that these things are not language, and imply that they have no qualities of their own worth mentioning. However, there is at least one thing that unites them: their in-betweenness. Are they lexical or not? Conventional or not? Phonemic or not? Intentional or not? They seem to skirt these issues — and derive interactional utility from that very ambiguity. Hence: liminal signs.

Many liminal signs originate in bodily conduct with non-interactional functions: sighing, sniffing, moaning, etc. This lends them an air of plausible deniablity and makes them off the record. It also makes them awesome cases of exaptation and ritualisation. Speaking of which: when Darwin wrote about whistles and clicks, he had to rely on anecdotal reports from around the world. The papers in this issue showcase the power of sequential analysis to bring to light the workings of liminal signs in interaction.

Inspired by Harvey Sacks, the commentary also aims to highlight the methodological and conceptual contributions of this special issue — from transcriptional innovations like >.nh< to interdisciplinary connections. As Sacks wrote:

[I]t would be nice if things were ripe so that any question you wanted to ask, you could ask. But there are all sorts of problems that we know in the history of any field that can’t be asked at a given time. They don’t have the technology, they don’t have the conceptual apparatus, etc. We just have to live with that, and find what we can ask and what we can handle.”

(Spring 1966 Lecture, in Sacks, 1992, vol. I:427)

The papers in this issue are part of a wave of new research into multimodal talk-in-interaction that is making remarkable progress in just what the study of talk-in-interaction can handle.

Looking for something to read? Dip into this special issue and prepare to have your sense of the boundaries of language subtly shifted — one sniff, click, or whistle at a time. My commentary (short and open access) is here:

Dingemanse, M. (2020). Between Sound and Speech: Liminal Signs in Interaction. Research on Language and Social Interaction, 53(1), 188–196. doi: 10.1080/08351813.2020.1712967

Farewell, Mr. Ideophone: William J. Samarin (1926-2020)

Posted on by
Reply
William John Samarin (photo University of Toronto)

I note with sadness that William J. Samarin has passed away in Toronto on January 16, 2020 at the age of 93. An all too short obituary notes that he was “known for his work on the language of religion and on two Central African languages: Sango and Gbeya”.

In linguistics, Samarin was of course also known for his extensive work on ideophones, playful and evocative words with sensory meanings. Only a few years after his Berkeley PhD he published a short and visionary paper on “African ideophones” (1966) that foreshadowed many of the themes that would occupy him in the next decades. A string of empirical and theoretical papers followed that brought new élan to the study of ideophones by dramatically extending the methodological toolbox and the kinds of topics studied, from lexical semantics to sociolinguistic variation, and from semantic typology to the use of ideophones in insults.

I have worked on the topic of ideophones a little over a decade now, and Samarin was always there in the background. He was there in the form of his formidable oeuvre, but also through active correspondence we kept up until halfway 2018. In this blog I want to share some personal recollections as well as some unpublished notes by Samarin about how he came to the study of ideophones.

Incidentally, we didn’t start off very well. In early 2010, when I first wrote to him as a wide-eyed grad student sharing a half-baked draft of a paper, he wrote back with stern (and justified) advice:

You see that if were your supervisor, I would be giving you a hard time about your generalizations. … Make sure that you are being as hard on yourself as you are (or might be) on others. (Samarin, personal communication, March 2010)

Several of my early interactions with Samarin were like this, and his bluntness was fairly intimidating to a PhD student in love with ideophones. Our exchanges led me to seriously rethink my rhetorical approach, placing more emphasis on theoretical foundations and methodological choices, and being as gentle and constructive as possible — in line with his advice to be “as hard on yourself as you are (or might be) on others”. This is why the acknowledgements of my PhD thesis note that “Samarin in particular has been highly sceptical at one point, and helpfully so”.

In late 2011, I sent him a hard copy of my thesis, a 400 page tome that he received in good spirits. This marked a change in our interactions, as he started to treat me more like a peer than a clueless grad student. In a message acknowledging receipt of the thesis, he fondly recalled how he used to be called “Mr Ideophone” at Leiden University, where he spent part of his sabbatical in 1966-1967:

Considering myself to be one of the pioneers in the study of ideophones (Jan Voorhoeve used to call me Mr Ideophone!), I am so pleased that they finally are getting the attention they deserve. They are the dramatic aspect of everyday speech, and speech should not be reduced to formulas and diagrams. (Samarin, personal communication, October 2011)

In later years, I would send drafts and papers to him knowing that they would get a tough but fair reading; and I would get the occasional email from him asking to look up an academic article not available in his library. His criticism remained as blunt and direct as ever, which made his rare notes of appreciation all the more precious.1

Samarin on ideophones

In one of our exchanges I asked Bill how he got involved in the study of ideophones. He responded, “since you asked me how I got on to studying  ideophones I decided to write a bit of autobiography for my archives.” I don’t know whether this bit of autobiography actually appears in his archives, so I share it here for posterity:

My serious study of ideophones arose from the fact that grammarians were not taking them seriously in African languages. They were even trivialized. This puzzled me because I found that they were used frequently in everyday discourse in all kinds of circumstances in the Gbaya (Gbeya) language which I began to analyze and learn in February 1954. Some of them I heard rather often, others rarely, but I could not ignore them if I wanted to speak the language in the same way Gbayas in northwestern Ubangi-Shari spoke it. I was using the language all day long, almost to the exclusion of Sango, in the Bossangoa district, most of whose population spoke mutually intelligible varieties of Gbaya. … Besides, they were curious words (like kpiti kpiti, with high tones) and hard to define.

But it was after I had written my grammar of the language in 1961 that I undertook to study them as a worthy topic in African linguistics. Naturally, the first thing was to read what had been said about them. This meant perusing grammars. Fortunately, I was a visiting professor at the University of Leiden in 1966-1967. There were plenty of grammars there, also at School of African Studies in London and at the Hartford Seminary Foundation, where I was teaching. (Henry Alan Gleason Jr had been librarian there, making an effort to acquire literature for graduate study in linguistics.)

Following our departure from the Central African Republic in 1960 I made several trips back for further work on Sango. These gave me the opportunity to sneak in some systematic study of Gbaya ideophones, like the one where I tape-recorded descriptions of someone making a clay pot in Sango and in Gbaya. I had more opportunity for study in Leiden, where my assistant was a Gbaya young married man. And in December 1972 my wife and I spent two weeks in the village of Bowai once again working on ideophones.

By this time my focus was on trying to demonstrate that Gbaya ideophones were authentic words that could (and should) be entered in a dictionary, not ephemeral and spontaneous idiolectal creations. And by this time one was able to analyze data with a computer, at that time with punched cards. For a while, therefore, I was working on the origin and development of Sango with my right hand and ideophones with my left one. A fire destroyed my computer data at the university, but there are many tape recordings in my archives at the University of Toronto. (Samarin, personal communication, January 2016)

The papers Samarin published in this period include important methodological contributions (Samarin 1967, 1970a, 1971a), a wide-ranging piece on expressive language (Samarin 1970b), and a comprehensive literature review of work on ideophones in Bantu (Samarin 1971b). I have built on Samarin’s work in several of my papers, but I don’t think a comprehensive appraisal of his methodological and theoretical to the study of ideophones is available. That is beyond the scope of this blog, however.

Rewards beyond words

Samarin and I were last in touch in 2018, when I wrote to him with a note of appreciation about his 1998 autobiographical essay (Samarin 1998). That essay contains the following gem which seemed to me entirely typical of Samarin’s poetic sensibilities and attention to detail:

If you have seen the full moon rising out of the deep sands that surround
Timbuctoo dwarfing the sky as well as earth in its clarity and brilliance while you are drinking mint tea with some Tamachek-speaking ‘Blue People,’ you will have experienced some rewards beyond words and sharing. If you are sensitive to such beauty, of course. It is given to us who study language to have rewarding experiences, sometimes of simple pleasure, sometimes of ‘spiritual’ if not of almost transcendental significance.

I have just teased a young girl going the opposite way by remarking that whereas she had a parasol to protect herself from the sun, what could I do without one. About fifteen feet away from me she stops and says, ‘Kà ga mu ma’, and I am overwhelmed with information and sensations: I hear the first word in a construction where I wouldn’t have expected it; I notice that she does not use the determinant ‘ni’ with the meaning ‘it;’ I enjoy the precise stepping up of pitch from low to mid to high and the abrupt falling to low again as she tells me, with no twinkle of coquetry on her lips, but with the spontaneous generosity of a well-reared African child: ‘So come take it.’ This is an imperishable and complex vignette. It illustrates the reward of being able to talk Sango and use it appropriately with another human being. (Samarin 1998:27)

I wrote to Bill to say I was touched by this vignette — it is such an eloquent representation of that quintessential fieldworkers’ feeling of belonging. It captures something very deep and real about the privilege of taking part in other linguistic and social worlds. It also brings out the always-on analytical mindset of the fieldworker, for whom being in the moment is always puncuated by meta-observations. Field work, for me, is very much about that liminal state between ‘other’ and ‘insider’, never fully one or the other, yet enough of both to feel oddly detached-yet-grounded.

In writing back, Bill shared another biographical fact that few people may know: his involvement as a linguistics expert in an International Criminal Court case about atrocities in Bangui (his expert testimony concerned the possibility of recognizing the Congolese origin of the perpetrators on the basis of their accents). He ended his message, characteristically, with a note of appreciation about field work that will resonate strongly with many linguists and anthropologists.

It was kind of you to comment on my professional memoir. I especially was pleased by your having perceived the emotion I had in recalling that experience with the little girl, which is repeated every time I recall it. She responded to my lighthearted remark with maturity, self-confidence, kindness, and trust, a lot more than many adults would have done. I should have interrupted my walk back home to go with her in the opposite direction to continue with a conversation.

You put your finger on the feeling of “belonging.” That’s what brought tears when I was testifying before the International Criminal Court in The Hague in 2011. (A Congolese general was being tried for what his soldiers did in Bangui.) The love I have for Central Africans welled up in me. … Field work gives us the opportunity to be enriched and blessed in many ways. (Samarin, personal communication, April 2018)

Besides working on ideophones, Samarin made important contributions to the fields of contact linguistics, field linguistics, and the study of glossolalia. I hope someone more qualified than me will write a comprehensive academic obituary. Here, I have just highlighted some of his pioneering contributions to the study of ideophones, which his work helped make not merely respectable but also exciting and relevant to the broader language sciences.

Samarin prided himself in being nicknamed Mr. Ideophone by Jan Voorhoeve in the 1960s. His lasting intellectual legacy may be that he helped prepare the field for contributions by a much wider range of scholars, so that today there is no longer a single “Mr” or “Ms” or “Mx” Ideophone, but a broad network of diverse researchers working together. Farewell, Mr. Ideophone!

References cited

A good amount of Samarin’s work is available in the University of Toronto’s T-SPACE repository. In 2018, Samarin sent me a overview of his papers, presentations, and research projects which I will publish in a separate post as it provides a good overview of his work from his own point of view. Here are the papers cited above:

  • Samarin, W. J. (1965). Perspective on African ideophones. African Studies, 24(2), 117–121.
  • Samarin, W. J. (1967). Determining the meaning of ideophones. Journal of West African Languages, 4(2), 35–41.
  • Samarin, W. J. (1970a). Field procedures in ideophone research. Journal of African Languages, 9(1), 27–30.
  • Samarin, W. J. (1970b). Inventory and choice in expressive language. Word, 26, 153–169.
  • Samarin, W. J. (1971a). Measuring variation in the use of Gbeya ideophones. Annales de L’Université d’Abidjan, Ser. H, 2, 483–488.
  • Samarin, W. J. (1971b). Survey of Bantu ideophones. African Language Studies, 12, 130–168.
  • Samarin, W. J. (1998). C’est passionnant d’être passionné. In E. F. K. Koerner (Ed.), First person singular III: Autobiographies by North American scholars in the language sciences (pp. 187–226). Amsterdam/Philadelphia: John Benjamins.
  1. About my Glossa review, he wrote: “It was thoughtful of you to inform me of the publication in ‘Glossa’ of your “chronological narrative” about ideophones. But you are being too modest: the essay is much more than that; it’s a ‘white paper’ or template for the study of this phenomenon that you so clearly describe from different points of view. It’s as if you were holding a handful of ore in you palm that contained a lot of gold. (…) Carry on with your good work. Bill.” [↩]

Playful iconicity: Having fun with words

Posted on by
Reply

What do words like waddle, slobber, tingle, oink, and zigzag have in common? These words sound funny, but they are also iconic, with forms that resemble aspects of their meanings. In a new paper we investigate the link between funniness and iconicity in 70,000 English words.

“This is play”

The starting point is a theory about metacommunication: some words (or signs) are more striking than others in terms of their form, which means they draw more attention to themselves and signal “this is play”. We think this explains the first finding of the paper: words that people rate as highly funny are also often rated as highly iconic.

Relations between funniness and iconicity after controlling for word frequency, in: A words with human ratings; B words with human funniness ratings and imputed iconicity ratings; C words for which we only have imputed ratings.

To test how general this finding is we developed a way to predict funniness and iconicity ratings for new words. Based on semantic relationships between millions of English words, we trained an algorithm to predict the iconicity (or funniness) of words that have already been rated by people, and then asked that algorithm to predict iconicity (or funniness) for new words.

For example, say the new word is ‘waggle’. First the algorithm learned that ‘waggle’ occurs in similar contexts to ‘wiggle’ and ‘wobble’. Then it learned that ‘wiggle’ and ‘wobble’ were rated as highly iconic by participants. As a result, it predicts that ‘waggle’ will be highly iconic too. Applying this method to ~70,000 words, we find that the relation between funniness and iconicity holds even for predicted ratings.

But what is it about some words that makes them both funny and iconic? Analysing the words that people rated as most funny and iconic, we found a number of recurring features: complex sequences of sounds at the start like str- and cl- or at the end like -nk and -mp, and an ending -le in verbs that contributes an element of movement and playfulness (as in ‘waggle’ and ‘wobble’).

These structural features, we propose, act as metacommunicative signals that help words stand out as playful, performative, and even poetic. They occur disproportionally in highly rated words. When we combined these cues in an overall structural markedness score, we found structural markedness predicts the iconicity and funniness ratings much better than other measures.

The relation between structural markedness and A funniness ratings, B iconicity ratings, and C funniness and iconicity together. Each dot represents 14 or 15 words. Solid lines and shading represent a loess function of cumulative markedness with 95% confidence intervals. Other lines show relative prevalence of complex onsets, codas, and verbal diminutives.

So our three main findings are:

  1. Words that are rated as highly iconic also tend to be rated as highly funny (in the few thousand words for which we have such ratings)
  2. This relation holds even in for ratings predicted based on semantic relationships (in ~65.000 words for which we have done this)
  3. The highly rated words tend to have special forms: they sound different from other words, which invites people to treat them as playful and performative

Making sense of apparent exceptions

We also found some other things. First, funniness and iconicity ratings do not always go hand in hand. There are highly iconic words like ‘roar’ and ‘scratch’ that people don’t feel are funny because they have to do with negative events. There are also words that are rated as very funny like ‘blonde’ and ‘buttocks’ mainly because they tend to be used in jokes; these are not rated as iconic and they are not relevant for our theory.

Another thing we found is that human ratings are far from perfect. As it turns out, for the data we used, the people who rated words for how much they “sound like what they mean” gave high ratings to words like ‘whoosh’ (where the sound of the word resembles aspects of its meaning) but also to words like ‘bedroom’ (which are built by combining meaningful parts).

Only words of the first type are really iconic; the others are merely analysable. Our theory holds only for the first, which means that the 10-15% of analysable words with high iconicity ratings are probably diluting the effects we find. Indeed, when we control for this issue by looking only at words of one piece, the relation between iconicity and funniness comes out a little stronger.

We included this analysis not just to show the subtleties of the effects, but also because we believe lexical ratings (whether done by people or by machines) should never be taken at face value. Now that there are so many types of ratings available, it’s tempting to just throw together a bunch of them and have a look at correlations. But to avoid cherry-picking or reporting false positives, it is important to start with a theoretical question, and to always control the findings with other methods.

Having fun with linguistics

While the study is based on English, its questions are inspired by work on ideophones, highly evocative words found in many languages around the world. And the theory put forward in the paper is general enough to help account for many other examples of playful language described in the literature, and to guide future investigations of the relation between playfulness and iconicity in spoken and signed languages.

Our study also contributes to broadening the perspective of linguistics. While anecdotal reports about perceptions of funniness and iconicity abound, our study is the first to investigate this relation on a large scale in English, and perhaps in any language. That this hasn’t been done before is partly because linguistics has long preferred to focus on “serious” matters. However, we argue that there is nothing frivolous about studying playful language.

Cybernetician Gregory Bateson argued that the very notion of play represents a fundamental transition in the evolution of communication. This is because play requires a form of metacommunication, a way of saying “What I do now is special”. Human language has perfected such forms of metacommunication, and in our paper we trace its influence in the very texture of the lexicon.

To enable others to build on our work we’ve made sure it is open science all the way: all primary data as well as our new predicted iconicity and funniness ratings are publicly available. We also share the Python code for our prediction algorithm and the R code for all of the analyses and figures. And last but not least, the paper itself is also published open access.

  • All data and code is in our GitHub repository
  • Dingemanse, M., & Thompson, B. (2020). Playful iconicity: Structural markedness underlies the relation between funniness and iconicity. Language and Cognition, 1-22. doi:10.1017/langcog.2019.49

Een week @NL_Wetenschap

Posted on by
Reply

In voorjaar 2019 mocht ik een week twitteren in naam der wetenschap voor het wissel-account @NL_Wetenschap (10-17 februari 2019). Omdat het account steeds rouleert van wetenschapper naar wetenschapper en omdat Twitter natuurlijk altijd in beweging blijft is het moeilijk om achteraf een goede indruk te krijgen van zo’n week. Daarom hier een overzichtje in blog-vorm.

De tweets van die week werden meer dan 250.000 keer gezien, wat lang niet gek is voor een account dat op dat moment een paar duizend volgens had. Het leverde ook veel interacties op, zowel met geïnteresseerde leken als met collega-wetenschappers uit andere disciplines. Ik werd op 15 februari ook geïnterviewd door Steven Smit in het programma NPO Focus Wetenschap op NPO Radio 1.

Hieronder een paar highlights. Veel van mijn tweets vormden ‘draadjes’, een soort aaneenschakeling van korte berichten die samen een mini-essay vormen. Dat zorgt voor samenhang en voor behapbare stukken. Klik op de datum of een foto in een individuele tweet om het draadje als geheel te lezen.

Taalkunde: alfa, beta, gamma? All of the above.

Eén van de aantrekkelijkste dingen van mijn vakgebied is dat het zich bevindt op het kruisvlak van zoveel wetenschapsgebieden. Daarover schreef ik twee draadjes. Het eerste begint met een beroemd diagram uit een rapport van toen de cognitiewetenschappen net in bloei waren gekomen.

In het tweede ga ik kritisch in op de populaire, maar misleidende alfa versus beta-indeling. Die werkt van geen meter voor taal of voor taalwetenschappers, en hier leg ik uit waarom de werkelijkheid veel te interessant is om in tweeën te delen.

Alice in Taalland

Elke werkdag plaatste ik een draadje met inzichten over taalwetenschap aan de hand van Alice in Wonderland, dat ik op dat moment aan mijn dochter aan het voorlezen was. N.B. de profielfoto van de tweet is dus de huidige tweeter, niet ik.

Alice in Taalland #1: over taal en sociale interactie
Alice in Taalland #2: over klanksymboliek
Alice in Taalland #3: over ongeschreven regels van de taal
Alice in Taalland #4: over betekenis, taalgebruik en pragmatiek
Alice in Taalland #5: over vlechtwoorden en waantaal

Reacties

Er kwamen mooie reacties en discussies.

En had Bart Braun een gouden tip over het Taverne-amendement in antwoord op een opmerking over het delen van publicaties: