It’s easy to forget amidst a rising tide of synthetic text, but language is not actually about strings of words, and language scientists would do well not to chain themselves to models that presume so. For apt and timely commentary we turn to Bronislaw Malinowski who wrote:
there is a series of phenomena of great importance which cannot possibly be recorded by questioning or computing documents, but have to be observed in their full actuality. Let us call them the inponderabilia of actual life.
In follow-up work, Malinowski has critiqued the unexamined use of decontextualised strings of words as a proxy for Meaning:
To define Meaning, to explain the essential grammatical and lexical characters of language on the material furnished by the study of [written records], is nothing short of preposterous in the light of our argument. Yet it would be hardly an exaggeration to say that 99 per cent of all linguistic work has been inspired by the study of dead languages or at best of written records torn completely out of any context of situation.
Malinowski did not write this on his substack, in an op-ed in the New York Times, or in a preprint. He spent time doing primary fieldwork, lived with people whose language he learned, and based on close observation of language in everyday use came to an informed critique of his contemporaries’ extreme reliance on strings of text.
He did all this over a century ago, and yet here we are, running in circles around stochastic text generators. Makes me think of something Wittgenstein wrote in another context, for a similar problem: “A picture held us captive. And we could not get outside it, for it lay in our language and language seemed to repeat it to us.”
Malinowski, B. (1922). Argonauts Of The Western Pacific. London: Routledge & Kegan Paul.
Malinowski, B. (1923). The problem of meaning in [underdescribed*] languages. In C. K. Ogden & Richards (Eds.), The meaning of meaning (pp. 296–336). London: Kegan Paul.
* I write [underdescribed] where Malinowski had ‘primitive’ to draw attention to the following: Malinowski wrote at a time when scientific racism meant that “modern” or “civilized” languages were habitually contrasted with “primitive” or “savage” ones — even as his own work helped demolish that distinction and showed the primacy of language use in everyday life across societies.
We don’t generally see PhD dissertations as an exciting genre to read, and that is wholly our loss. As the publishing landscape of academia is fast being homogenised, the thesis is one of the last places where we have a chance to see the unalloyed brilliance of up and coming researchers. Let me show you using three examples of remarkable theses I have come across in the past years.
Unflattening by Nick Sousanis
I didn’t even know it was even possible to do a PhD dissertation in graphic novel style. And yet here we are. This is a mindblowing work that (my colleagues can attest) I keep raving about. From the back blurb:
Nick Sousanis defies conventional forms of scholarly discourse to offer readers both a stunning work of graphic art and a serious inquiry into the ways humans construct knowledge. Unflattening is an insurrection against the fixed viewpoint. Weaving together diverse ways of seeing drawn from science, philosophy, art, literature, and mythology, it uses the collage-like capacity of comics to show that perception is always an active process of incorporating and reevaluating different vantage points.
The title, “Unflattening” reads as a reference to the famous mathematical allegory by Abbott, Flatland: the two-dimensional world where one day A. Square is visited by a being from another dimension. Just as A. Square learns to “unflatten” his world as he gets to know the three-dimensional Sphere visiting him, so the readers of Sousanis’ work are invited to venture out of their conceptual comfort zones and explore a multiplicity of perspectives.
(Update: Sousanis notes: “I came up with the word Unflattening early on, and in my planning, figured I should use Flatland – originally at the end, but moved up to front…”)
The published version of this work is a book to have lying around on your desk. It’s always a pleasure to leaf through and there’s always something new to discover.
The equidistribution of … number fields, by Piper Harron
The full title of this work is “The Equidistribution of Lattice Shapes of Rings of Integers of Cubic, Quartic, and Quintic Number Fields: an Artist’s Rendering”, and if this makes you glaze over, you should skip right to the prologue where it says:
Respected research math is dominated by men of a certain attitude. Even allowing for individual variation, there is still a tendency towards an oppressive atmosphere, which is carefully maintained and even championed by those who find it conducive to success. As any good grad student would do, I tried to fit in, mathematically. I absorbed the atmosphere and took attitudes to heart. I was miserable, and on the verge of failure. The problem was not individuals, but a system of self-preservation that, from the outside, feels like a long string of betrayals, some big, some small, perpetrated by your only support system. When I physically removed myself from the situation, I did not know where I was or what to do. First thought: FREEDOM!!!! Second thought: but what about the others like me, who don’t do math the “right way” but could still greatly contribute to the community? I combined those two thoughts and started from zero on my thesis. What resulted was a thesis written for those who do not feel that they are encouraged to be themselves
Harron 2016:1
The whole thing is written in an irreverent, cool and clear writing style, smoothly segueing from “laysplaining” sections aimed at a general audience to more technical parts aimed at mathematicians. No wonder that Harron’s thesis went viral, winning her coverage on Mathbabe, Scientific American and even getting its own Wikipedia article. Even for readers without the requisite background in mathematics, this thesis is a wonder to behold and a joy to read.
Swarmpunk by Jonny Saunders
Swarmpunk! The name alone is a reason to check it out — and be sure to have a look at the corresponding website. There are many things to admire about this thesis. Content-wise, it is hard to summarise (the mark of a good thesis to my mind). Here’s the author’s own attempt:
Drawing from decades of digital infrastructural history within and beyond science, I will sketch a path by which we might build systems that empower, rather than control us. I will argue a better future for science is not utopian, nor solely dependent on funding and administrative agencies, but something we can organize ourselves. Woven together with my work in ill-defined phonetic categories and distributed experimental systems, I have written a love letter to the power of swarms: how by embracing heterogeneity and rough consensus we might make science more boisterous, creative, and human.
I love the wide-ranging nature of this work all the more because the swarm metaphor does pull things together in a compelling way. Who would have thought that the neuroscience of phonetic categorization in mice could come together with a technical description of peer to peer protocols and trenchant critiques of academia’s infrastructural woes. And yet it all makes sense.
The sheer breadth of topics covered evokes Bruno Latour’s sprawling Actor Network Theory framework and also shares a lot of insights with current approaches to collective intelligence (e.g., all intelligence is collective intelligence). What’s also impressive about this thesis is the way it combines a hard-hitting critique of the current state of academic workflows with concrete tools and constructive proposals to make things better. This makes it a truly excellent example of what Ivan Illich calls counterfoil research.
And then I haven’t even discussed the type-setting (in lovely Tufte-style), the anarchist easter eggs (like figure captions that say “taken without permission”) and the sharp-witted writing throughout. Read this work and let it broaden your conceptions of how to build a better academia.
Real life is gnarly
Here’s another thing I love about these three theses: they all show something of the making process. They don’t present a fully polished surface; they have rough edges and that is entirely okay (especially for a thesis — but I think this holds in general for the scientific process).
Sousanis’ Unflattening was revised for publication with Harvard University Press, so I imagine some of the loose ends have been tied up; but it includes, at the end, a set of sketches that document the very beginning of the work, dated to April 14, 2011. Why don’t more theses have this?
First idea map for Unflattening, April 14, 2011 (Sousanis 2015:194)
Meanwhile, for Harron, working on the thesis had to find a place next to other momentous life events. In the Acknowledgements, there is a parenthetical aside “I would thank the children, but frankly, they’ve been no help.”. One reason for this is found on the opening page of Chapter 5: a cartoon of the author working on “THESIS (draft)” followed by a panel with an inset where the author says “OW”, and contractions start…
Saunders’ Swarmpunk moves from work on phonetic categorization in mice to an unfinished chapter of musings on the topic of language games, and from there even more abruptly to work on a software infrastructure for scientific experimentation and collaboration (check out Autopilot). Saunders explains:
I had intended to finish my dissertation with an experiment that was the next logical conclusion of the mouse model of phonetics, doing longitudinal mesoscopic calcium imaging of auditory cortex as the phonemes were being learned in order to model the changes in network activity. (…) It comes out of chronological order in the spring of 2021, after my work with Autopilot and a covid-induced awakening of the possibility of public engineering with the People’s Ventilator Project[72]. I was restless and not ready to return to basic research while the world was still so broken, and so it was abandoned in favor of the last piece on digital infrastructure. Accordingly, it ends relatively abruptly, without satisfying conclusion. I include it here in its unfinished form, roughly edited, warts and all, as something I intend to pick up perhaps one day when basic science is more possible.
And yet the end product is amazing. Perhaps precisely because of the abrupt transitions, it invites the reader to fill in the gaps, to create those conceptual linkages, to contribute the cognitive work that makes all of science a collective enterprise.
More importantly, it reminds us that we are humans first and foremost, and science is a human enterprise in which the neat plans we may make can be greatly affected by real life events — as they should.
A plea for serendipity
Dissertations like these may be rare. But they exist. And by existing, they show that true creativity is still alive in academia. For me personally, this kind of work strengthens my convinction that as supervisors, we need to be serious about ideas and open-minded about form and content. (Perhaps this is why I have always liked my PhD student’s Gwilym Lockwood’s mad MS-Paint skills — and the Harron-like irreverent style of the opening pages of his 2017 PhD.)
Today’s PhD projects tend to be considerably less free-form than even 5 or 10 years ago. Today we ask graduate students to write full-blown project proposals, force them to specify their planning in gantt charts, and herd them through series of checkpoints. I understand why this is done, and that it is helpful to some. But I cannot ignore that it is also a machinery for control, arising from management processes built towards verification, certification, and standardization. It is, at base, a stark form of risk-aversion.
But here is the thing about science: it thrives on serendipity. If all risks had been averted and all processes duly followed, it is unlikely that we would have seen these three mindblowing dissertations. Here’s to an academia with room for serendipity!
Harron, P. A. (2016). The Equidistribution of Lattice Shapes of Rings of Integers of Cubic, Quartic, and Quintic Number Fields: An Artist’s Rendering (PhD thesis, Princeton University) https://www.theliberatedmathematician.com/thesis/
Saunders, J. L. (2022). Swarmpunk: Rough Consensus and Running Code in Brains, Machines and Society (PhD dissertation, University of Oregon) https://jon-e.net/dissertation/
Sousanis, N. (2015). Unflattening. Cambridge, Massachusetts: Harvard University Press.
No mind is an island (after John Donne). In a new piece, we make the case for putting interaction at the heart of cognition. This represents a figure-ground reversal for the cognitive sciences, which traditionally have focused on single minds. Read the piece on the journal website, head directly to the PDF, or see the simple accompanying website.
Perhaps only those who haven’t read Bakhtin can call themselves true Bakhtinians: the ideas have to reach you and influence you through a polyphony of other texts and people.
NRC vraagt zich af of wetenschappers hun werk blijven delen op twitter en vindt op twitter maar liefst 7 fervent twitterende wetenschappers die desgevraagd bevestigen nog op twitter te blijven.
Twitter is inderdaad van belang geweest voor de wetenschap, maar het lijkt vooral de journalistiek te zijn die nog aan het twitterinfuus ligt. Het geweldige collectief WO in Actie ontleende een deel van haar slagkracht aan sociale media; des te verbazender dat NRC onvermeld laat dat Robeyn’s trouwe kompanen Remco Breukers en Rens Bod al lang en breed twexit zijn. Ook over naar mastodon zijn bekende wetenschappers als Ionica Smeets, Daniël Lakens, Iris van Rooij, en Marc van Oostendorp — stuk voor stuk goed voor 8k-80k twittervolgers, maar de krant wist ze ineens niet meer te vinden, want ja, niet op twitter.
Grote delen van twitterend academia zijn kortom al lang elders. Ze schrijven, podcasten, bloggen, en tooten in alle vrijheid en openheid, zonder het knagende gevoel cosponsor te zijn van een povere miljardair die bevestiging zoekt bij bruinrechts. Groepsblogs als Neerlandistiek, Stuk Rood Vlees, Astroblogs en Bij Nader Inzien floreren. Podcasts zijn niet van de lucht en mooie initiatieven als Nemo Kennislink, de MuseumJeugdUniversiteit en de IMC Weekendschool brengen een groter publiek in aanraking met wetenschap en onderzoek.
Kortom, als het gaat om kanalen voor kennisdeling en opinievorming heeft het medialandschap buiten de twitterbubbel er lange tijd niet zo florissant bijgelegen als nu. Kom ook buitenspelen!
Sketches, visualizations and other forms of externalizing cognition play a prominent role in the work of just about any scientist. It’s why we love using blackboards, whiteboards, notebooks and scraps of paper. Many folks who had the privilege of working the late Pieter Muysken fondly remember his habit of grabbing any old piece of paper that came to hand, scribbling while talking, then handing it over to you.
Since the summer of 2021 I have owned a Remarkable, and it has become an essential part of my scientific workflow because it seamlessly bridges this physical form of thinking with the digital world of drafts, files and emails. I rarely rave about tools (to each their own, etc.) but this is one of those that has changed my habits for the better in several ways: I’ve been reading more, taking more notes, writing more, and also doodling and sketching more. As a cognitive scientist I would describe it as a distraction-free piece of technology with just the right affordances for powerful forms of extended cognition (it is probably no coincidence that it was recommended to me by fellow traveller Sébastien Lerique, whose interests range from embodied rationality to interaction).
One of ways in which the Remarkable has changed my workflow and my collaborations is that it is much easier to sketch a basic idea for a visualization and share it digitally. We use this during brainstorms to produce first impressions or visualize hypotheses. Often such a rough sketch then functions as a placeholder in a draft until we’ve made an actual version based on our data.
Left: first sketch made on Remarkable; Right: version published in osf.io/cwvbe
The above example from a recent paper with Andreas Liesenfeld shows this process: first my rough sketch of what the plot might look like, which fuels our discussion and helps me to express how to transform our source data in R. Then a ggplot version I made in R that preserves the key idea and adds some bells and whistles like loess lines and colour.
I want to credit my collaborator Andreas Liesenfeld for pushing me to do more of this visual-first way of thinking. One of the things Andreas often asks when brainstorming about a new paper is: “okay but what’s the visual?”. Thinking early about compelling visualizations has made our papers more tightly integrated hybrids of text and visuals than they might otherwise have been. For instance, our ACL paper has 7 figures, approximately one to a page, that support the arguments, help organize the flow, and generally make for a nicer reading experience.
Conceptual frameworks
Sketches can also be useful to work out conceptual frameworks. In a recent collaboration with Raphaela Heesen, Marlen Fröhlich, Christine Sievers and Marieke Woensdregt we spent a lot of time talking about ways to characterize various types of communicative “redoings” across species. A key insight was that the variety of terms used in different literatures (eg. primatology vs. human interaction) could actually be linked by looking more closely at the sequential structure of communicative moves. I sent off a quick Saturday morning doodle to my collaborators, and ultimately we published a polished version of it in our paper on communicative redoings across species (PDF here).
Left: sketch of a conceptual framework for studying redoings across species. Right: published version following deliberation among collaborators and multiple smaller adjustments.
Finally, sketches are useful to express ideas and hypotheses visually even before the data is in. For instance, in current work with Bonnie McLean and Michael Dunn we’re thinking a lot about transmission biases and how they influence cultural evolution over time. Bonnie’s dataset looks at biases and rates of change in how concepts relate to phonemic features. It’s helped me to express my thinking on this visually, and I can’t wait to see what Bonnie ultimately comes up with. (This visualization is inspired in part by something I read about parallax in Nick Sousanis’ amazing book Unflattening.)
Not a review
This is not a review of the Remarkable — just a reflection on how it’s changed my academic life for the better. Every device has pros and cons. For instance, I don’t particularly love the overpriced stylus (‘Marker plus’) or how they sell Connect subscriptions for slightly better syncing options — though you should be aware you don’t need a subscription to do any of the things I’ve described in this post. And on the other hand, I absolutely do love the litheness of this device, the just-right friction when writing, and the fact that it has no backlight. The design in general strikes me as a perfect embodiment of that philosopher Ivan Illich has called ‘convivial tools’: tech that is sophisticated yet also responsibly limited in ways that support human flourishing. Anyway, there’s a good remarkable subreddit if you’re in the market for a device like this.
Note. Remarkable has a referral program that gives you a $40 (or equivalent) discount if you use this link to purchase one. If you like the device and keep it, that would also mean I earn $40, which I would use to treat my team to fancy coffee and cakes!
Over two years ago I wrote about the unstoppable tide of uninformation that follows the rise of large language models. With ChatGPT and other models bringing large-scale text generation to the masses, I want to register a dystopian prediction.
Of course OpenAI and other purveyors of stochastic parrots are keeping the receipts of what they generate (perhaps full copies of generated output, perhaps clever forms of watermarking or hashing). They are doing so for two reasons. First, to mitigate the (partly inevitable) problem of information pollution. With the web forming a large part of the training data for large language models you don’t want these things to feed on their own uninformation. Or at least I hope they’re sensible enough to want to avoid that.
But the second reason is to enable a new form of monetization. Flood the zone with bullshit (or facilitate others doing so), then offer paid services to detect said bullshit. (I use bullshit as a technical term for text produced without commitment to truth values; see Frankfurt 2009.) It’s guaranteed to work because as I wrote, the market forces are in place and they will be relentless.
Universities will pay for it to check student essays, as certification is more important than education. Large publishers will likely want it as part of their plagiarism checks. Communication agencies will want to claim they offer certified original human-curated content (while making extra money with a cheaper tier of LLM-supported services, undercutting their own business). Google and other behemoths with an interest in high quality information will have to pay to keep their search indexes relatively LLM-free and fight the inevitable rise of search engine optimized uninformation.
Meanwhile, academics will be antiquarian dealers of that scarce good of human-curated information, slowly and painstakingly produced. My hope is that they will devote at least some of their time to what Ivan Illich called counterfoil research:
Present research is overwhelmingly concentrated in two directions: research and development for breakthroughs to the better production of better wares and general systems analysis concerned with protecting [hu]man[ity] for further consumption. Future research ought to lead in the opposite direction; let us call it counterfoil research. Counterfoil research also has two major tasks: to provide guidelines for detecting the incipient stages of murderous logic in a tool; and to devise tools and tool-systems that optimize the balance of life, thereby maximizing liberty for all.
Illich, Tools for Conviviality, p. 92
Frankfurt, H. G. (2009). On Bullshit. In On Bullshit. Princeton University Press. doi: 10.1515/9781400826537
Illich, I. (1973). Tools for conviviality. London: Calder and Boyars.
Reading Suchman’s classic Human-machine reconfigurations: plans and situated actions, I am impressed by her description David Turnbull’s work on the construction of gothic cathedrals. In brief, the intriguing point is that no blueprints or technical drawings or even sketches are known to have existed for any of the early modern gothic cathedrals, like that of Chartres. Instead, Turnbull proposes, their construction was massively iterative and interactional, requiring —he says— three main ingredients: “talk, tradition, templates”. This sounds like an account worth reading; indeed perhaps also worth emulating or building on. In the context of the language sciences, an analogue readily suggests itself. Aren’t languages rather like cathedrals — immense, cumulative, complex outcomes of iterative human practice?
Okay nice. At such a point you can go (at least) two ways. You can take the analogy and run with it, taking Turnbull’s nicely alliterative triad and asking, what are “talk, traditions, and templates” for the case of language? It would be a nice enough paper. The benefit would be that you make it recognizably similar and so if the earlier analysis made an impact, perhaps some of its success may rub off on yours. The risk is that you’re buying into a triadic structure devised for a particular rhetorical purpose in the context of one particular scientific project.
Going meta
The second way is to ‘go meta’ and ask, if this triad is a useful device to neatly and mnemonically explain something as complex as gothic cathedrals, what is the kind of rhetorical structure we need to make a point that is as compelling as this (in both form and content) for the domain we are interested in (say, language)? See, and I like that second move a lot more. Because you’ve learnt from someone else’s work, but on a fairly abstract level, without necessarily reifying the particular distinctions or terms they brought to bear on their phenomenon.
While writing these notes I realise that I in my reading and reviewing practice, I also tend to judge scientific work on these grounds (among others). Does it work with (‘apply’) reified distinctions in an unexamined way, or does it go a level up and truly build on others’ work? Does it treat citations perfunctorily and take frameworks as given, or does it reveal deep reading and critical engagement with the subject matter? The second approach, to me, is not only more interesting — it is also more likely to be novel, to hold water, to make a real contribution.
Further readign
Gould, S. J. (1997). The exaptive excellence of spandrels as a term and prototype. Proceedings of the National Academy of Sciences, 94(20), 10750–10755. doi: 10.1073/pnas.94.20.10750
Suchman, L. A. (2007). Human-machine reconfigurations: Plans and situated actions (2nd ed). Cambridge ; New York: Cambridge University Press.
Turnbull, D. (1993). The Ad Hoc Collective Work of Building Gothic Cathedrals with Templates, String, and Geometry. Science, Technology, & Human Values, 18(3), 315–340. doi: 10.1177/016224399301800304
Naturally I was interested in what the paper says about conversation. The claim about indirectness in Yoruba and other languages is sourced to a very nice piece by Felix Ameka and Marina Terkourafi.
The paper also devotes some attention to the importance of linguistic diversity in computer science and NLP — a key theme in the new language diversity track at #acl2022nlp, where another paper by Blasi and colleagues stood out. (The relevance of cross-linguistically diverse corpora for NLP was also a focus in this ACL paper of ours, where we argue such data is crucial for diversity-aware modelling of dialogue and conversational AI.
I do have a nitpick about Blasi &al’s backchannel claim. They note many languages have minimal forms (citing a study of ours that provides evidence on this for 32 languages) and add, “However, listeners of Ruruuli … repeat whole words said by the speaker” — seeming to imply they rarely produce such minimal forms and (tend to) repeat words instead. Or at least I’m guessing that would be most people’s reading of this claim.
The source given for this idea is Zellers 2021. However, this actually paints a very different picture: in fact, ~87% of relevant utterances (1325 out of 1517) do consist of minimal forms like the ‘nonlexical’ hmm and the ‘short lexical’ eeh ‘yes’, against <9% featuring repetition, as seen in this table from Zellers:
I don’t think anyone has done the relevant comparison for other languages yet, but it seems safe to say that Ruruuli/Lunyala does in fact mostly use “the minimal mm-hmm”, and that repetition, while certainly worthwhile of more research, is one of the minority strategies for backchanneling in the language.
Despite this shortcoming, the relevance of cross-linguistic diversity in this domain can be supported by a different observation: the relative frequency and points of occurrence of ‘backchannels’ do seem to differ across languages — as shown in our ACL paper for English versus Korean. And the work on repetition is fascinating in itself — it is certainly possible that repetition is used in a wider range of interactional practices in some languages, with possible effects on transmission & lg structure as suggested in work by Sonja Gipper.
A serendipitous wormhole into #EMCA history. I picked up Sudnow’s piano course online and diligently work through the lessons. Guess what he says some time into the audio-recorded version of his 1988 Chicago weekend seminar (see lines 7-11)
[Chicago, 1988. Audio recording of David Sudnow’s weekend seminar]
We learn too quickly and cannot afford to contaminate a movement by making a mistake.
People who type a lot have had this experience. You type a word and you make a mistake.
I have been involved, uh of late, in: a great deal of correspondence in connection with uh a deceased friend’s archives of scholarly work and what should be done with that and his name is Harvey. And about two months ago or three months ago when the correspondence started I made a mistake when I ( ) taped his name once and I wrote H A S R V E Y, >jst a mistake<.
I must’ve written his name uh two hundred times in the last few months in connection with all the letters and the various things they were doing. Every single time I do that I get H A S R V E Y and I have to go back and correct the S. I put it in the one time and my hands learned a new way of spelling Harvey. I call ‘m Harvey but my hands call ‘m Hasrvey.
And they learned it that one time. Right then and there, the old Harvey got replaced and a new Harvey, spelled H A S R V E Y got put in. So we learn very fast.
Folks who know #EMCA history will notice this is right at the height of the activity of the Harvey Sacks Memorial Association, when Sudnow, Jefferson, Schegloff, and others were exchanging letters on Sacks’ Nachlass, intellectual priority in CA, and so on
We have here a rare first person record of the activity that Gail Jefferson obliquely referred to in her acknowledgement to the posthumously published Sacks lectures (“With thanks to David Sudnow who kick-started the editing process when it had stalled”), and much more explicitly in an 1988 letter (paraphrased in Button et al. 2022).
Historical interest aside, I like how the telling demonstrates Sudnow’s gift for first-person observation — a powerful combination of ethnomethodology and phenomenology that is also on display in his books, Pilgrim in the Microworld and Ways of the Hand #EMCA
