Automatic Concordancing

The first session I attended at BibleTech was by Neil Rees from the United Bible Societies. Neil was demonstrating Paratext, a widely-used tool for doing Bible translation, and the Concordance Builder that works in Paratext.

Neil encouraged all of us who are involved in Bible translation to get onto Paratext. Paratext has 2000+ registered users and is being used on around 1000 Bible translation projects worldwide. So I went to the Paratext website and registered for an account. I'm still waiting to hear back. (It looks like Paratext is only available to people who can verify that they are involved in Bible translation in some fashion.)

The bulk of the talk concerned Concordance Builder. Neil reminded us that an intelligently edited concordance is an important tool for users of the Bible, to help them find concepts and topics by key terms. Now, it's certainly possible to provide an exhaustive concordance of every word mechanically, but this is much less useful for a number of reasons, including (1) too many hits, (2) no grouping of related word forms, and (3) no distinguishing between different meanings of the same word. Besides, it's impossible to print such a concordance in the back of a Bible. And even if your primary delivery is digital, a mechanically-generated concordance is much harder to use than one which has been intelligently edited.

So a concordance needs to be edited, and that's where all the time is spent in concordance-making. But this is where Concordance Builder really shines: (1) It aids grouping by word-stem; (2) it automatically generates lists of references for a given word or stem; (3) it comes with model concordances and statistical glossing functions to aid in creating these lists of references; (3) the generated list of references for each term can be edited; (4) the overall term list can also be edited, so as to exclude "structure" words like "the"; (5) it enables the inclusion of cross-references and alias terms; (6) it automates the process of selecting the portion of the sentence for display, based on column width; (7) it produces XML output that can be imported into typesetting software such as Adobe InDesign.

Using Concordance Builder, Neil says that an excellent, edited concordance for a new translation can be created in a few days or weeks rather than months or years.

More information about the theoretical basis for automatic concordance building can be found in an academic paper by the presenter, Neil Reese, and his colleague, J. D. Riding, entitled "Automatic Concordance Creation for Texts in Any Language." In addition, J. D. Riding has published a paper entitled "Statistical Glossing - Language Independent Analysis in Bible Translation", which is about the process of mapping one language to another, in order to give a head start at concordance building. More information about Concordance Builder and related topics is available at Neil Reese's and J. D. Riding's academic website.

For those who are involved in Bible translation work and making concordances, I have not seen a better toolset; and because it outputs XML and has been integrated into InDesign, it can be used in any Bible publisher's workflow.