PanLex: Research

The scientific and engineering research that enabled the PanLex project began in 02006. The questions addressed by this research include:

• When lexical translations for a panlingual database are obtained from diverse resources (dictionaries, glossaries, etc.), how can such translations be discovered, analyzed, harmonized, attributed, evaluated, licensed, and exposed? [1, 10, 14]
• Given a massive network of lexemes (dictionary words and phrases) and meanings, what automatic methods are best at choosing among translation alternatives, and how can they be executed fast enough for practical use? [7, 8, 12, 13]
• How can panlingual lexical translation support translingual communication and information transfer? [2, 3, 4, 5, 6, 9, 11, 13]
• Is an inventory of basic concepts empirically derivable from lexicographic translation data? [4, 8, 12, 16]

So far, research has found that inference algorithms applied to the PanLex data can approach the goal of panlingual lexical translation more fully than any single resource alone [7, 8]. It has also been shown that individuals forced to compose messages only as sequences of PanLex words can in large part understand each other’s intent across languages [5]. Other research produced a demonstration search engine that expanded the number, internationality, and relevance of images delivered to requesters, especially those not knowing the most widely used languages [2, 3, 4].

With its growing coverage of the world’s lexicons, researchers are also beginning to exploit PanLex as a resource [15].

References:

[1] Timothy Baldwin, Jonathan Pool, and Susan M. Colowick, “PanLex and LEXTRACT: Translating all Words of all Languages of the World”, 02010.

[2] Janara Christensen, Mausam, and Oren Etzioni, “A Rose is a Roos is a Ruusu: Querying Translations for Web Image Search”, 02009.

[3] Susan Colowick, “Multilingual Search with PanImages”, 02008.

[4] Oren Etzioni, Kobi Reiter, Stephen Soderland, and Marcus Sammer, “Lexical Translation with Application to Image Search on the Web”, 02007.

[5] Katherine Everitt, Christopher Lim, Oren Etzioni, Jonathan Pool, Susan Colowick, and Stephen Soderland, “Evaluating Lemmatic Communication”, 02010.

[6] Christopher Lim, “Panlingual Translator”, 02009.

[7] Mausam, Stephen Soderland, Oren Etzioni, Daniel S. Weld, Michael Skinner, and Jeff Bilmes, “Compiling a Massive, Multilingual Dictionary via Probabilistic Inference”, 02009.

[8] Mausam, Stephen Soderland, Oren Etzioni, Daniel S. Weld, Kobi Reiter, Michael Skinner, and Jeff Bilmes, “Panlingual Lexical Translation via Probabilistic Inference”, 02010.

[9] Jonathan Pool, “Panlingual Globalization”, 02010.

[10] Jonathan Pool, Sourcing in PanLex, 02012.

[11] Marcus Sammer, Kobi Reiter, Stephen Soderland, Katrin Kirchhoff, and Oren Etzioni, “Ambiguity Reduction for Machine Translation: Human-Computer Collaboration”, 02006.

[12] Marcus Sammer and Stephen Soderland, “Building a Sense-Distinguished Multilingual Lexicon from Monolingual Corpora and Bilingual Lexicons”, 02007.

[13] Stephen Soderland, Christopher Lim, Mausam, Bo Qin, Oren Etzioni, and Jonathan Pool, “Lemmatic Machine Translation”, 02009.

[14] Patrick Westphal, Claus Stadler, and Jonathan Pool, “Countering language attrition with PanLex and the Web of Data”, 02015.

[15] Francesca Gola, “An analysis of translation divergence patterns using PanLex translation pairs”, 02012.

[16] Jonathan Pool, PanLex Empirical Concepticons, 02014.