Duplicate detection is one of the things any serious reference manager should offer. Zotero users have been clamouring for it since the early days. There are basically two ways to implement it: as a preflight check, warning the user when they are about to add a potential duplicate; and as an after the fact scan, which enables users to weed duplicate items from their library.

The most recent version of Zotero takes the second route: a posthoc duplicate detection mechanism. Though definitely better than nothing, and with an elegant merging solution, the interface is still far from perfect and yields a lot of false positives, making it somewhat difficult to use. Besides, it is slow, because it tries to compare everything with everything, which amounts to a huge amount of operations even in moderately sized libraries. Although it is good to have at least something, what seems to me have been overlooked is that prevention is better than cure, and that a quick check before adding new items to the library would help users a lot. 

It appears that the Zotero developers have too much on their plate to think about such niceties, but as Avram Lyon has pointed out, this could be done as a Zotero plugin, and anyone interested in a little challenge could wade in. (I think you need to be literate in XUL and JS, and probably SQLite too. A sample Zotero plugin here. Also see this thread for some helpful comments by Zotero developer Dan Stillman on the best way to implement it.)

What kind of form should a user interface for this take? I would propose something like the following:

Do you really want to add this item? It looks like it already exists in your library.

Smith, Joe. 2010. How to avoid duplicate entries. Ms., Amsterdam.

[Button A: Cancel and go to similar item.] [Button B: Add anyway.]

The interface should be fast and reliable. I propose the following basic workflow: Upon adding a new item, check a low number of strategically chosen fields and assign a duplicate score according to some simple rules, similar to spam rating systems. If duplicate score exceeds x, bring up the interface I propose above. (The variable x and the weight of individual rules could be made customizable but there is no need in a first version — as they say, release early, release often.) My proposal of fields to use, ranked by descending weight:

1. DOI
2. author last name
3. title not case-sensitive (only first n words?)
4. year
5. publication
6. page numbers

DOI is hit or miss, so good; but not all items will have a DOI. Author last name + Title + Year probably should receive a combined weight that is the same or higher as DOI. Given the importance of the first four, perhaps 5 and 6 have little added value.

Coders and XUL/Javascript wizzkids looking for a challenge, here is your chance to make a difference! If you wrote a Zotero plugin doing this, it would find wide and immediate adoption by a significant portion of Zotero users, and eternal glory will be yours.

Here’s a quick tip for Zotero users who like to do their browsing in Chrome or Safari: you can install “Zotero Connectors” that will make Zotero recognize references in Chrome and Safari just like in Firefox. The Zotero developers are working on a standalone version, but these connectors can already talk to your Zotero library in Firefox. So if you, say, find yourself going to Chrome for its speed and nice interface, you can simply connect it with Zotero and use Firefox to host your local Zotero library until Zotero Standalone comes along.

Follow these steps:

1. Make sure you have the latest version of Zotero (3.0 Beta currently)
2. Install the Chrome or Safari connector for Zotero in the browser of your choice.
3. Start Firefox and type about:config in the address bar. Within the options, search for “zotero”, locate the extensions.zotero.httpServer.enabled option and double-click to enable it. (See kb article here.)
4. Restart Firefox as well as your other browser to get them linked up properly.
5. Enjoy the goods of Zotero translators in Chrome or Safari! (Remember that Firefox has to be open for Chrome to recognise and save the reference.)

Quick Q&A

Huh? When was this cool feature added?

Work on the Standalone version started about a year ago. The connectors have been developed for use with Standalone, but in a streak of insight, the developers also allowed communication with the Firefox version of Zotero. This was never announced, which is why I’m devoting a post to this well kept secret!

Can I now install Zotero in Chrome or Safari?

No, this post describes a way to get Zotero support in Chrome or Safari. You still need to have your main Zotero library in Firefox. If you are adventurous, you can try the Standalone Alpha or Beta version and connect it in the same way.

What are Zotero translators?

Translators enable Zotero to sense when you’re on a site displaying bibliographic information that you can import into your library. Thousands of sites are supported. Translators allow one-click saving of references in your library; if a PDF is available, they even download it for you and automatically attach it to the entry in your library!

What if it doesn’t work?

Be sure that Firefox is open, otherwise Chrome won’t be able to sense and save items. Try the troubleshooting translator issues page. If that doesn’t work, you can ask for help in the Zotero forums, where a lot of helpful people hang out.

I keep forgetting the kind of simple edits that are so trivial to make in CSL styles. Here I catalogue a few, for my own benefit and hopefully also useful to others. The occasion is a festive one —one paper in press and another accepted with minor revisions!— and I’m not going to let the fun be spoiled by the tedious job of making minute changes to the referencing styles. Where did the editors and reviewers find all this time anyway? One of them manually added spaces before the page numbers in all of my forty+ in-text references! I refuse to do that kind of monkey-work. Instead, I’m simply going to edit the nearest CSL style and woosh, the whole document will be fine.

The Zotero documentation offers the basic information on how to edit CSL styles and how to get your new style into Zotero; here I assume that you’ve read that. (The lowdown: use the reference test pane to see the XML code of an existing style. Make it your own. Save it under a different name. Drag it onto Firefox to add it to Zotero.) Before I give the code snippets, it is probably useful to briefly outline what a CSL style looks like so that the code looks less terrifying.

Basic anatomy of a CSL style

A CSL style is simply a file that you can edit in any text editor consisting of a number of different ‘blocks’ of information. The information is couched in terms of a relatively simple but powerful XML-based metalanguage called “Citation Style Language” or CSL for short. Every CSL style basically looks as follows. First, an info block providing the metadata for the style (author, name, url, type); then a lot of macros defining the building blocks; a citation block that determines what citations look like in your document; and finally a bibliography block.

<?xml version="1.0" encoding="UTF-8"?>
<style xmlns="http://purl.org/net/xbiblio/csl" class="in-text" xml:lang="en">
  <info>
	here goes the metadata
  </info>
	here go all the macros
  <citation>
	this block defines what a citation looks like
  </citation>
  <bibliography>
	this block defines what the bibliography looks like
  </bibliography>
</style>

Initials vs. full first names

Some styles have initials, others have full first names. Some of the first have initials followed by a period, others without. Personally, I think it is not the best thing to abbreviate information that is crucial for disambiguation, but this is what does it:

initialize-with="."

The above form gives you “Gombrich, E.” Removing the period (i.e. having initialize-with="") gives you “Gombrich, E”. Removing the option as a whole gives you “Gombrich, Ernst”. Note that in some styles, this setting is included in several macros (e.g. editor as well as author) so you might have to adjust it in several places.

Multiple entries by the same author

Some well-known styles (for example the Chicago Manual of Style) substitute subsequent recurrences of an author with three em-dashes. The highlighted line below is what does it:

<bibliography>
      ...
    <option name="subsequent-author-substitute" value="———"/>
      ...
</bibliography>

If you don’t want that happening, simply remove this option from the bibliography block.

Issue numbers

Some editors don’t like issue numbers. In the Chicago style, the issue number comes out of the locators macro, and it is generated by the following line:

<text variable="issue" prefix=", no. "/>

Deleting that line gets rid of the issue number.

Page number prefix for in-text citations

For page numbers in in-text citations, some styles want (Doe 2010:5), others (Doe 2010, 5) and yet others (Doe 2010: 5) (note the space before the page number in the latter case). This is controlled by the group delimiter in the citation block towards the end of the CSL style.

<citation>
      ...
      (basic options)
      ...
    <layout prefix="(" suffix=")" delimiter="; ">
      <group delimiter=": ">
      ...
      (contributors, date, locators)
      ...
      </group>
    </layout>
  </citation>

delimiter=": " yields (Doe 2010: 5), while delimiter=", " yields (Doe 2010, 5), etc.

Line spacing

Some styles, like APA, have a setting for line-spacing built in. For APA it is "2" (i.e. double), but actually that is only the APA style for submitting manuscripts — for camera-ready copy, most publishers simply want normal line-spacing. Changing this is simple. Search for "line-spacing" and either set it to "0" or remove attribute+value altogether. You want to do the latter if you would like the bibliography to inherit the line-spacing settings of your document.

Want more?

Do you want more information, or need to make further customizations? The Zotero documentation has a page giving a CSL syntax summary. The common options given on that page are probably most useful to start with. Also, Rintze Zelle has written a very nice CSL 1.0 primer. And don't forget you can always ask for help in the Zotero forums. Loads of volunteer supporters there know more about CSL than I do.

Not all linguists may be aware of this, but since 2007 there has been a Unified Style Sheet for publications in our field, developed by the editors of a number of linguistic journals, including Language. (Oddly enough, just which journals besides Language joined in the effort remains unclear.) There is not much centralized information available about this style, but we have the 2007 specifications (PDF) and a page endorsing the style at the official LSA website.

As Stephen Anderson writes on the Linguist List,

Use of this style is encouraged, and if it is widely adopted, that could considerably facilitate the preparation of manuscripts. In support of that, it would be useful to have software support for it in popular document preparation systems.

I’m happy to report that Zotero now supports the unified style through the powerful open format CSL. If you have Zotero, you can simply install the style right away. Zotero of course already supported lots of other citation styles, including the widely used APA and MLA as well as styles for specific journals like Language (install) and the Journal of Pragmatics (install).

I haven’t been able to find a definitive list of the journals that have adapted the Unified Style Sheet for Linguistics, but some examples of journals using and/or endorsing it are Language itself, Semantics & Pragmatics, and the Journal of English Linguistics. Also, the LDLT conference series at SOAS, London is using it in its proceedings. Does anyone know of more journals?

It’s here. Zotero 1.5 beta. The new version comes with built-in synchronization, exports to more than 1100 citation styles, and supports browsing your library online (see below). Zotero is now better than EndNote on all fronts. Here’s a quick overview of the most important features:

• Synchronization. Automatically keep your library in sync across different PCs. If you have access to WebDAV storage, synching can also include your attachments.
• Automatic backup. A copy of your library is stored safely on the synchronization server.
• More than 1100 CSL citation styles. The style repository has grown immensely due to community efforts. Zotero styles are built on the powerful open source Citation Style Language (CSL), an XML dialect.
• Support for EndNote styles. Thousands of EndNote .ens styles can now be used for citation formatting. These styles are available to licensed users of EndNote.
• Rich text notes. Formatting can now be applied to notes with a WYSIWYG editor.
• Automatic detection of PDF metadata. Another much requested feature. Not yet bulletproof because it depends on the information available in your PDF and the repository used to look it up, but a great step forward.
• Shared collections. Easily share and build collections with colleagues.

All of this built on open source technologies and standards, which means that your data is not locked up in proprietary software at the mercy of profit driven companies.

New website features

Browse your Zotero library online [click for fullsize]

Meanwhile, the Zotero website has seen a major revamping, the most important new feature being the ability to browse your library online. Other features are more geared towards social networking activities: users now can have an online Zotero profile, can follow other Zotero users, and can build an online CV.

If you’re still stuck on EndNote, check out making the switch to Zotero, or see my review and comparison from last year. Questions? There are lots of helpful and friendly people hanging out in the Zotero forums. You can also post them below.

Zotero is getting better and better. In a while, version 1.5 will bring synchronization, online backup of your library, +1100 CSL citation styles, and PDF metadata extraction (for the daring, a sync preview version is available). But even in its current incarnation Zotero is easily one of the best bibliographic managers out there. Here are twelve tips and tricks that help you to get the most out of it.

1. Drag files from the web right into your library
   Got a reference in your library, but no PDF? Or saved an item from a repository which doesn’t provide a fulltext version? Do a quick search for the title on Google Scholar — it is good at finding PDFs on author’s webpages. If you find one, just drag the link from the page onto the reference in your library. Zotero stores and attaches the PDF for you.
2. Enter a series of items by duplicating a template
   Adding a series of related references to your library? Start with one item for which you fill in the fields that are the same for all items (e.g. editors, book title, year, publisher, place) and duplicate it (Right-click > Duplicate item). Then fill in the particularities.
3. Quick Copy a citation using Ctrl+Alt+C or drag and drop
   Sending a PDF to a colleague, or mentioning a reference somewhere? Quickly copy the citation by selecting the reference and pressing Ctrl+Alt+C (Command+Alt+C on the Mac), or simply drag it from Zotero onto any edit window (for example a new email). The default output style can be specified under Preferences > Export; the shortcut key can be customized under Shortcut keys.
4. Have Zotero index your PDFs
   Zotero can index your PDF attachments and make them fully searchable, turning your library from a mere linked catalogue into a Google Books of sorts. The option is turned off by default because it relies on an external open source program (pdf2txt) which is not distributed with Zotero. However, Zotero can automatically install it and enable fulltext indexing: simply go to Preferences > Search and click on the ‘Check for installer’ button. For more info see pdf fulltext indexing in the Zotero documentation.
5. Start quicksearch with ” to trigger advanced search
   By default, Zotero starts searching when you put the first few characters in the Search box. In a large library with fulltext indexing enabled, this can be tiresome (you wanted to look for “statistical methods”, but Zotero locks down searching for “st”). To avoid this, simply start your search with ” (double quote) to have Zotero wait until you finish typing and hit enter.
6. Press Ctrl to find out in which collections an item is
   Looking at an item in your library and wondering whether you already categorized it? Press Control and Zotero will highlight the collections in which it is contained.
7. Relocate your Zotero folder to a more sensible place
   The default place for the Zotero database and attachments in right in your Firefox profile, which isn’t the easiest to locate whichever OS you are on. Go to Preferences > Advanced to customize the storage location. You can place it in a folder that is included in your regular backup schedule or put it on a portable drive so that your library always travels with you (tip: combine it with Firefox Portable, which you can even use without administrator rights).
8. Keep track of recent additions using a saved search
   Often you add new items without worrying about tagging or putting them in collections. Click Advanced search, select “Dated Added” > “is in the last” > X “days/months” and fill in the desired period; then save the search. This gives you a dynamically updated overview of your latest additions, so that you can go back to them and do the categorization and tagging work when it suits you.
9. Tag multiple items at once
   Want to tag multiple items at once? Select them, make sure the tag selector is visible in the left pane, and drag them onto the tag you want to use. The tag will be applied to all items.
10. Tag incomplete items to find them back and fix them later
    Sometimes you know an item has incomplete metadata (e.g. missing page numbers or publisher), but you don’t have the time to fix it right away. Make it a habit to tag such items (“needs metadata”) when you see them. Now you can find them and fix them whenever you have some time to kill.
11. Use a separate folder for files to be ingested
    Someone gives you a bunch of PDFs to read; or you download a paper somewhere without having the metadata handy. Make it a habit to save such files in a subfolder /new/ in your Zotero folder. Then once in a while go through that folder. Do a quick search for the title on your favourite repository, grab the metadata, and then drag the PDF from your filemanager onto the reference in Zotero. Much better than having those loose PDFs scattered all over your hard drive (or in your mailbox!) — and it helps you keep track of your reading history too.
12. Display a timeline to visualize your bibliography
    Not a feature you’ll use everyday, but a neat one nonetheless: Zotero can display your library, or portions of it, on a timeline. Select a group of references, a tag, or a collection and click ‘Create timeline’ (in the Gear menu). This gives you an overview of the items in time. Now you have to ask yourself: is the recency bias due to your reading habits or is it really true that most of the research was done in the last twenty years? (Probably a bit of both.)

Questions or suggestions? Leave a comment.

Exciting news for Zotero users: synchronization has arrived. After some months of closed beta-testing, a public Sync Preview version was released recently. This means that Zotero users can now automatically synchronize their libraries across computers and even across platforms.

Although there are still some minor wrinkles, the sync functionality works perfectly fine and there are some exciting new features, including the possibility to import thousands of Endnote styles. With the import functionality comes a handy style manager, another step towards an elegant, shared, and open source solution to citation styling. That’s two killer features in one release — impressive work by the Zotero folks.

Also note the following:

Before Zotero 1.5 ships, we will add functionality to allow users to synchronize attachments to their own servers or other storage space (and we’ll also provide a hosted storage solution for all Zotero users). [forum post by Sean Takats]

Do keep in mind that the current preview is a preliminary version intended for public testing; do not expect it to be bug-free. Always make a backup copy of your full Zotero folder and try the Sync Preview in a new profile (step-by-step instructions on the sync preview page). Easier yet, download Firefox 3 Portable and try out Zotero Sync Preview 1.5 on a copy of your library without risking data loss or profile mixups. If your workflow is fine without synchronization, my advice is to avoid the growing pains of the preview version and wait until the release of the official 1.5 version, which should follow within a few months.

Not sure what Zotero is? Check the website or read my review of it.

If you’ve read yesterday’s post (Zotero, an Endnote alternative) or come across Zotero elsewhere, you may have been wondering about its name. I believe most Anglophones pronounce the word [ˌzɔˈtɛɹoʊ], but the term itself actually derives from the Albanian verb zotëro-j [zɔtərɔj] ‘master, acquire’. The final -j marks the 1st person indicative (the regular citation form for Albanian verbs); in the imperative, we would get the bare verb root zotëro [zɔtərɔ]. Such subtleties did not figure in the initial baptismal act though, as we learn from the following transcript of a podcast featuring the people behind Zotero:

The web being what it is, I just quickly googled and found an English-Albanian dictionary and typed a bunch of our keywords that we associated with the project and when I typed in ‘learning’, uhm… one of the variants was ‘to learn something extremely well, that is to master or acquire a skill in learning’ was “zotëroj” [pronounced [ˌzɒˈtuəɹʏdʲ] by DC, MD] (laughs), which we have shortened, we took of the -j at the end which is more of a ‘y’-sound and uh we took off the umlaut …
(Dan Cohen, Library Geeks Podcast 5, 22:48—25:15)

It’s that simple. And for good reason: essentially, want you need in branding is a name that sticks but at the same time is not too common; if it makes some sense (as ‘Zotero’ does), that’s even better. The main reason for choosing an Albanian word was thus quite simply to minimize namespace competition. It could have been any other language — in the podcast, Cohen mentions Maori; Hawaiian is another popular one (wikiwiki), and Bantu languages do well too (cf. Ubuntu, a trendy Linux distribution).

Will It Brand?

Well, not really any other language of course — a quick glance over the newest web 2.0 names shows that the preferred languages for this kind of stuff seem to be those with simple phonotactics, a preference for open syllables, a basic 5 vowel system, and not-too-outlandish consonant inventories. So at least in the Zotero case, Siwu is out of luck with suã ‘learn’ (nasal vowel penalty); as is Tamashek with əlmæd ‘learn, acquire’ (muddy vowels and a voiced coda, tsk); as is Ibibio with kpéép ‘learn, acquire’ (a labio-velar stop, for petes sake!); readers are no doubt able to come up with better examples.

Fortunately, these need not be fatal problems. Dan Cohen’s account shows that if it doesn’t fit, we can always make it fit; just chop off needless morphology and diacritics and you’re good to go. Now Albanian, hitherto an obscure 6 million speaker language making up it’s own branch of Indo-European, enjoys celebrity status as the language that endowed the Next-Generation Research Tool with a worthy name. Come to think of it, who would not like to sacrifice some orthographic blunt for publicity’s sake? Suddenly all those woefully inadequate orthographies we linguists have been cursing at are beginning to make sense! Next time the underspecified orthography drives you nuts again, find a product in need of a name and monetize your despair. I’ve heard naming consultants easily make twice as much as linguists.

P.S. A great resource on naming is Nancy Friedman’s Away With Words, which I found via the posting on Web 2.0 names referenced above.

I wasn’t planning to make this a software weblog, but I’ll make an exception for Zotero because I think fellow researchers will find it an interesting tool. Zotero [ˌzɔˈtɛɹoʊ] is a free piece of software that lives in your browser, helping you to ‘collect, manage and cite your research sources’ in all sorts of beautiful ways. It bills itself as The Next-Generation Research Tool, and in this post I’ll try to explain why I think that’s true. The background to this posting is that I made the move from Endnote to Zotero two months ago — and I have never since considered going back.

It all started when I upgraded from Endnote 7 to Endnote X to get Unicode support. Endnote X included Endnote Web, a web-based implementation that looked interesting. I had some difficulty getting the two to work together, and when I finally did, there were drawbacks that made me look out for an alternative. A Google search led me to Zotero, which was a breeze to install. I could simply import my Endnote library and started a testdrive. Within minutes I was totally hooked. The Zotero interface offered everything I had been missing in Endnote and then some. What makes Zotero so good?

Seamless integration with online research

First of all, Zotero answers the needs of researchers in the digital age. The rise of online repositories like JSTOR, ProQuest, SpringerLink, and Google Scholar has caused a shift in our research habits; we spend more time browsing virtual libraries, and less time hanging around in physical ones. Zotero seamlessly integrates with this online experience by automating the wearisome labour of saving references and by offering many ways to manage and enrich the data thus collected. All from within the web browser.

With Zotero, adding a reference from an online repository to your database takes exactly one click. If an abstract is available (as in such repositories as SpringerLink and CSA), it will be saved with the record; if you have fulltext access to the repository, Zotero will moreover save a local copy of the PDF. But Zotero doesn’t just grab references from repositories; it can also save a snapshot of any web page and make it available for citation. This comes in handy when you want to make sure you have a copy of the page for future reference. Web pages saved in this way can be viewed offline, and Zotero even lets you annotate these snapshots just as you would do on paper. Unlike your paper scribbles, all of your Zotero annotations and notes are searchable.

Reference management, the intuitive way

Secondly, Zotero is a fully fledged reference manager with an intuitive interface. For those of you using Endnote —as I did until two months ago— this combination may sound otherworldly. Basically, Zotero takes that huge flat list of references that stares you in the face when you start up Endnote and transforms it into a neat, manageable and searchable personal library. No more trying to get your mind around the intricacies of Showing and Hiding references in Endnote — instead, use collections, tags and saveable searches to take control of your library.

Zotero enables you to organize references into collections. References can be in more than one collection and collections can have subcollections. References can be related to each other so that you can easily link reviews or chapters of a book to the book itself. Advanced search capabilities enable you to do fulltext searches in your library or in portions of it (yes, it can index your PDFs too!). Searches can be saved; if saved, they are dynamically updated so that any new items fitting the search criteria are found too. Zotero also brings the power of tagging to your reference database, enabling fine-grained categorization as well as easy category intersection. And of course, Zotero can import and export citations in all major formats, making it perfectly easy to switch back to Endnote, or more to the point, to share citations with colleagues.

Word processor integration

Not only is Zotero a great reference manager, it integrates with your favourite word processor too. It supports both Microsoft Word and OpenOffice/NeoOffice through plugins (here’s how), adding a bunch of buttons to your word processor which help you insert and edit citations and bibliographies. In a preference window you can specify the citation style you want to use. The most commonly used styles are already available, and the list of supported styles continues to grow. XML-savy users can add and distribute their own citation styles.

Another thing worth mentioning is that Zotero will run on Mac and Linux just as fine as on Windows. This is because Zotero is built on the Firefox extension framework. (One implication of this is that Zotero will only run on Firefox 2 and not, for example, on Internet Explorer. If you do not already use Firefox, this is as good a reason as any to switch.) While we’re at it, some of you may also like to know that under the hood, Zotero uses the sqlite database engine. If you are not sure what that means, the most important thing to remember is that Zotero is based on open source technologies which, by their very nature, will continue to be freely available in the future (unlike proprietary software, the development and maintenance of which is driven by monetary interests).

That’s just the start of it

In this post I have focused on those features of Zotero that made me decide to switch. But I have not nearly exhausted everything it does. Some things I find particularly cool are the fact that it can export to HTML with embedded COinS metadata, allowing other Zotero users to surf by and capture the references (more info); its ability to generate reports, consisting of a list of selected references together with tags and notes you added (example); the option to visualize your references on a timeline (screenshots); and the possibility to build a bibliography simply by dragging and dropping references from Zotero into any text edit box (which is how I usually cite on The Ideophone).

If you want to give Zotero a try (remember it’s free!), here are some useful resources:

• A review of Zotero by Scott McLemee at Inside Higher Education
• Zotero.org, the page from where you can install Zotero and find detailed instructions for use
• Documentation, including nice introductory screencasts
• The Zotero forums, a place where helpful people hang around to answer your questions
• A podcast on Zotero presented by Chris Strauber, Reference & Web Services Librarian at Wofford College

Thanks for reading — comments welcome!