No mind is an island (after John Donne). In a new piece, we make the case for putting interaction at the heart of cognition. This represents a figure-ground reversal for the cognitive sciences, which traditionally have focused on single minds. Read the piece on the journal website, head directly to the PDF, or see the simple accompanying website.
Perhaps only those who haven’t read Bakhtin can call themselves true Bakhtinians: the ideas have to reach you and influence you through a polyphony of other texts and people.
NRC vraagt zich af of wetenschappers hun werk blijven delen op twitter en vindt op twitter maar liefst 7 fervent twitterende wetenschappers die desgevraagd bevestigen nog op twitter te blijven.
Twitter is inderdaad van belang geweest voor de wetenschap, maar het lijkt vooral de journalistiek te zijn die nog aan het twitterinfuus ligt. Het geweldige collectief WO in Actie ontleende een deel van haar slagkracht aan sociale media; des te verbazender dat NRC onvermeld laat dat Robeyn’s trouwe kompanen Remco Breukers en Rens Bod al lang en breed twexit zijn. Ook over naar mastodon zijn bekende wetenschappers als Ionica Smeets, Daniël Lakens, Iris van Rooij, en Marc van Oostendorp — stuk voor stuk goed voor 8k-80k twittervolgers, maar de krant wist ze ineens niet meer te vinden, want ja, niet op twitter.
Grote delen van twitterend academia zijn kortom al lang elders. Ze schrijven, podcasten, bloggen, en tooten in alle vrijheid en openheid, zonder het knagende gevoel cosponsor te zijn van een povere miljardair die bevestiging zoekt bij bruinrechts. Groepsblogs als Neerlandistiek, Stuk Rood Vlees, Astroblogs en Bij Nader Inzien floreren. Podcasts zijn niet van de lucht en mooie initiatieven als Nemo Kennislink, de MuseumJeugdUniversiteit en de IMC Weekendschool brengen een groter publiek in aanraking met wetenschap en onderzoek.
Kortom, als het gaat om kanalen voor kennisdeling en opinievorming heeft het medialandschap buiten de twitterbubbel er lange tijd niet zo florissant bijgelegen als nu. Kom ook buitenspelen!
Sketches, visualizations and other forms of externalizing cognition play a prominent role in the work of just about any scientist. It’s why we love using blackboards, whiteboards, notebooks and scraps of paper. Many folks who had the privilege of working the late Pieter Muysken fondly remember his habit of grabbing any old piece of paper that came to hand, scribbling while talking, then handing it over to you.
Since the summer of 2021 I have owned a Remarkable, and it has become an essential part of my scientific workflow because it seamlessly bridges this physical form of thinking with the digital world of drafts, files and emails. I rarely rave about tools (to each their own, etc.) but this is one of those that has changed my habits for the better in several ways: I’ve been reading more, taking more notes, writing more, and also doodling and sketching more. As a cognitive scientist I would describe it as a distraction-free piece of technology with just the right affordances for powerful forms of extended cognition (it is probably no coincidence that it was recommended to me by fellow traveller Sébastien Lerique, whose interests range from embodied rationality to interaction).
One of ways in which the Remarkable has changed my workflow and my collaborations is that it is much easier to sketch a basic idea for a visualization and share it digitally. We use this during brainstorms to produce first impressions or visualize hypotheses. Often such a rough sketch then functions as a placeholder in a draft until we’ve made an actual version based on our data.
The above example from a recent paper with Andreas Liesenfeld shows this process: first my rough sketch of what the plot might look like, which fuels our discussion and helps me to express how to transform our source data in R. Then a ggplot version I made in R that preserves the key idea and adds some bells and whistles like loess lines and colour.
I want to credit my collaborator Andreas Liesenfeld for pushing me to do more of this visual-first way of thinking. One of the things Andreas often asks when brainstorming about a new paper is: “okay but what’s the visual?”. Thinking early about compelling visualizations has made our papers more tightly integrated hybrids of text and visuals than they might otherwise have been. For instance, our ACL paper has 7 figures, approximately one to a page, that support the arguments, help organize the flow, and generally make for a nicer reading experience.
Sketches can also be useful to work out conceptual frameworks. In a recent collaboration with Raphaela Heesen, Marlen Fröhlich, Christine Sievers and Marieke Woensdregt we spent a lot of time talking about ways to characterize various types of communicative “redoings” across species. A key insight was that the variety of terms used in different literatures (eg. primatology vs. human interaction) could actually be linked by looking more closely at the sequential structure of communicative moves. I sent off a quick Saturday morning doodle to my collaborators, and ultimately we published a polished version of it in our paper on communicative redoings across species (PDF here).
Finally, sketches are useful to express ideas and hypotheses visually even before the data is in. For instance, in current work with Bonnie McLean and Michael Dunn we’re thinking a lot about transmission biases and how they influence cultural evolution over time. Bonnie’s dataset looks at biases and rates of change in how concepts relate to phonemic features. It’s helped me to express my thinking on this visually, and I can’t wait to see what Bonnie ultimately comes up with. (This visualization is inspired in part by something I read about parallax in Nick Sousanis’ amazing book Unflattening.)
This is not a review of the Remarkable — just a reflection on how it’s changed my academic life for the better. Every device has pros and cons. For instance, I don’t particularly love the overpriced stylus (‘Marker plus’) or how they sell Connect subscriptions for slightly better syncing options — though you should be aware you don’t need a subscription to do any of the things I’ve described in this post. And on the other hand, I absolutely do love the litheness of this device, the just-right friction when writing, and the fact that it has no backlight. The design in general strikes me as a perfect embodiment of that philosopher Ivan Illich has called ‘convivial tools’: tech that is sophisticated yet also responsibly limited in ways that support human flourishing. Anyway, there’s a good remarkable subreddit if you’re in the market for a device like this.
Note. Remarkable has a referral program that gives you a $40 (or equivalent) discount if you use this link to purchase one. If you like the device and keep it, that would also mean I earn $40, which I would use to treat my team to fancy coffee and cakes!
Over two years ago I wrote about the unstoppable tide of uninformation that follows the rise of large language models. With ChatGPT and other models bringing large-scale text generation to the masses, I want to register a dystopian prediction.
Of course OpenAI and other purveyors of stochastic parrots are keeping the receipts of what they generate (perhaps full copies of generated output, perhaps clever forms of watermarking or hashing). They are doing so for two reasons. First, to mitigate the (partly inevitable) problem of information pollution. With the web forming a large part of the training data for large language models you don’t want these things to feed on their own uninformation. Or at least I hope they’re sensible enough to want to avoid that.
But the second reason is to enable a new form of monetization. Flood the zone with bullshit (or facilitate others doing so), then offer paid services to detect said bullshit. (I use bullshit as a technical term for text produced without commitment to truth values; see Frankfurt 2009.) It’s guaranteed to work because as I wrote, the market forces are in place and they will be relentless.
Universities will pay for it to check student essays, as certification is more important than education. Large publishers will likely want it as part of their plagiarism checks. Communication agencies will want to claim they offer certified original human-curated content (while making extra money with a cheaper tier of LLM-supported services, undercutting their own business). Google and other behemoths with an interest in high quality information will have to pay to keep their search indexes relatively LLM-free and fight the inevitable rise of search engine optimized uninformation.
Meanwhile, academics will be antiquarian dealers of that scarce good of human-curated information, slowly and painstakingly produced. My hope is that they will devote at least some of their time to what Ivan Illich called counterfoil research:
Present research is overwhelmingly concentrated in two directions: research and development for breakthroughs to the better production of better wares and general systems analysis concerned with protecting [hu]man[ity] for further consumption. Future research ought to lead in the opposite direction; let us call it counterfoil research. Counterfoil research also has two major tasks: to provide guidelines for detecting the incipient stages of murderous logic in a tool; and to devise tools and tool-systems that optimize the balance of life, thereby maximizing liberty for all.
Illich, Tools for Conviviality, p. 92
DALL-E, a new image generation system by OpenAI, does impressive visualizations of biased datasets. I like how the first example that OpenAI used to present DALL-E to the world is a meme-like koala dunking a baseball leading into an array of old white men — representing at one blow the past and future of representation and generation.
It’s easy to be impressed by cherry-picked examples of DALL•E 2 output, but if the training data is web-scraped image+text data (of course it is) the ethical questions and consequences should command much more of our attention, as argued here by Abeba Birhane and Vinay Uday Prabhu.
Suave imagery makes it easy to miss what #dalle2 really excels at: automating bias. Consider what DALL•E 2 produces for the prompt “a data scientist creating artificial general intelligence”:
When the male bias was pointed out to AI lead developer Boris Power, he countered that “it generates a woman if you ask for a woman”. Ah yes, what more could we ask for? The irony is so thicc on this one that we should be happy to have ample #dalle2 generated techbros to roll eyes at. It inspired me to make a meme. Feel free to use this meme to express your utter delight at the dexterousness of DALL-E, cream of the crop of image generation!
It is not surprising that glamour magazines like Cosmopolitan, self-appointed suppliers of suave imagery, are the first to fall for the gimmicks of image generation. As its editor Karen Cheng found out after thousands of tries, it generates a woman if you ask for “a female astronaut with an athletic feminine body walking with swagger” (Figure 3).
I also love this triptych because of the evidence of human curation in the editor’s tweet (“after thousands of options, none felt quite right…”) — and the glib erasure of exactly that curation in the subtitle of the magazine cover: “and it only took 20 seconds to make”.
The erasure of human labour holds for just about every stage of the processing-to-production pipeline of today’s image generation models: from data collection to output curation. Believing in the magic of AI can only happen because of this systematic erasure.
Based on a thread originally tweeted by @dingemansemark@scholar.social (@DingemanseMark) on April 7, 2022.
Part of the struggle of writing in a non-native language is that it can be hard to intuit the strength of one’s writing. Perhaps this is why it is especially gratifying when generous readers lift out precisely those lines that {it?} took hard work to streamline — belated thanks!
Interestingly, the German translation for Tech Review needed double the amount of words for the same point: “Ein Mehr an Präzision bedeutet manchmal ein Weniger an Freiheit.” I’m still wondering whether that makes it more precise or less.
Reading Suchman’s classic Human-machine reconfigurations: plans and situated actions, I am impressed by what I’m reading on the performative and interactional achievement of the construction of gothic cathedrals, as studied by David Turnbull. In brief, the intriguing point is that no blueprints or technical drawings or even sketches are known to have existed for any of the early modern gothic cathedrals, like that of Chartres. Instead, Turnbull proposes, their construction was massively iterative and interactional, requiring —he says— three main ingredients: “talk, tradition, templates”. Each of these well-summarized by Suchman. This sounds like an account worth reading; indeed perhaps also worth emulating or building on. In the context of the language sciences, an analogue readily suggests itself. Aren’t languages rather like cathedrals — immense, cumulative, complex outcomes of iterative human practice?
Okay nice. At such a point you can go (at least) two ways. You can take the analogy and run with it, taking Turnbull’s nicely alliterative triad and asking, what are “talk, traditions, and templates” for the case of language? It would be a nice enough paper. The benefit would be that you make it recognizably similar and so if the earlier analysis made an impact, perhaps some of its success may rub off on yours. The risk is that you’re buying into a triadic structure devised for a particular rhetorical purpose in the context of one particular scientific project.
The second way is to ‘go meta’ and ask, if this triad is a useful device to neatly and mnemonically explain something as complex as gothic cathedrals, what is the kind of rhetorical structure we need to make a point that is as compelling as this (in both form and content) for the domain we are interested in (say, language)? See, and I like that second move a lot more. Because you’ve learnt from someone else’s work, but on a fairly abstract level, without necessarily reifying the particular distinctions or terms they brought to bear on their phenomenon.
While writing these notes I realise that I in my reading and reviewing practice, I also tend to judge scientific work on these grounds (among others). Does it work with (‘apply’) reified distinctions in an unexamined way, or does it go a level up and truly build on others’ work? Does it treat citations perfunctorily and take frameworks as given, or does it reveal deep reading and critical engagement with the subject matter? The second approach, to me, is not only more interesting — it is also more likely to be novel, to hold water, to make a real contribution.
Been reading this paper by @blasi_lang @JoHenrich @EvangeliaAdamou Kemmerer & @asifa_majid and can recommend it — Figure 1 is likely to end up in many lecture slides http://doi.org/10.1016/j.tics.2022.09.015
Naturally I was interested in what the paper says about conversation. The claim about indirectness in Yoruba and other languages is sourced to a very nice piece by Felix Ameka and Marina Terkourafi.
The paper also devotes some attention to the importance of linguistic diversity in computer science and NLP — a key theme in the new language diversity track at #acl2022nlp, where another paper by Blasi and colleagues stood out. (The relevance of cross-linguistically diverse corpora for NLP was also a focus in this ACL paper of ours, where we argue such data is crucial for diversity-aware modelling of dialogue and conversational AI.
I do have a nitpick about Blasi &al’s backchannel claim. They note many languages have minimal forms (citing a study of ours that provides evidence on this for 32 languages) and add, “However, listeners of Ruruuli … repeat whole words said by the speaker” — seeming to imply they rarely produce such minimal forms and (tend to) repeat words instead. Or at least I’m guessing that would be most people’s reading of this claim.
The source given for this idea is Zellers 2021. However, this actually paints a very different picture: in fact, ~87% of relevant utterances (1325 out of 1517) do consist of minimal forms like the ‘nonlexical’ hmm and the ‘short lexical’ eeh ‘yes’, against <9% featuring repetition, as seen in this table from Zellers:
I don’t think anyone has done the relevant comparison for other languages yet, but it seems safe to say that Ruruuli/Lunyala does in fact mostly use “the minimal mm-hmm”, and that repetition, while certainly worthwhile of more research, is one of the minority strategies for backchanneling in the language.
Despite this shortcoming, the relevance of cross-linguistic diversity in this domain can be supported by a different observation: the relative frequency and points of occurrence of ‘backchannels’ do seem to differ across languages — as shown in our ACL paper for English versus Korean. And the work on repetition is fascinating in itself — it is certainly possible that repetition is used in a wider range of interactional practices in some languages, with possible effects on transmission & lg structure as suggested in work by Sonja Gipper.
Originally tweeted by @dingemansemark@scholar.social (@DingemanseMark) on October 17, 2022.
A serendipitous wormhole into #EMCA history. I picked up Sudnow’s piano course online and diligently work through the lessons. Guess what he says some time into the audio-recorded version of his 1988 Chicago weekend seminar (see lines 7-11)
[Chicago, 1988. Audio recording of David Sudnow’s weekend seminar]We learn too quickly and cannot afford to contaminate a movement by making a mistake.
People who type a lot have had this experience. You type a word and you make a mistake.
I have been involved, uh of late, in: a great deal of correspondence in connection with uh a deceased friend’s archives of scholarly work and what should be done with that and his name is Harvey. And about two months ago or three months ago when the correspondence started I made a mistake when I ( ) taped his name once and I wrote H A S R V E Y, >jst a mistake<.
I must’ve written his name uh two hundred times in the last few months in connection with all the letters and the various things they were doing. Every single time I do that I get H A S R V E Y and I have to go back and correct the S. I put it in the one time and my hands learned a new way of spelling Harvey. I call ‘m Harvey but my hands call ‘m Hasrvey.
And they learned it that one time. Right then and there, the old Harvey got replaced and a new Harvey, spelled H A S R V E Y got put in. So we learn very fast.
Folks who know #EMCA history will notice this is right at the height of the activity of the Harvey Sacks Memorial Association, when Sudnow, Jefferson, Schegloff, and others were exchanging letters on Sacks’ Nachlass, intellectual priority in CA, and so on
We have here a rare first person record of the activity that Gail Jefferson obliquely referred to in her acknowledgement to the posthumously published Sacks lectures (“With thanks to David Sudnow who kick-started the editing process when it had stalled”), and much more explicitly in an 1988 letter (paraphrased in Button et al. 2022).
Historical interest aside, I like how the telling demonstrates Sudnow’s gift for first-person observation — a powerful combination of ethnomethodology and phenomenology that is also on display in his books, Pilgrim in the Microworld and Ways of the Hand #EMCA
Originally tweeted by @dingemansemark@scholar.social (@DingemanseMark) on October 6, 2022.