English-French Document Alignment Based on Keywords and Statistical Translation

This publication doesn't include Faculty of Arts. It includes Faculty of Informatics. Official publication website can be found on muni.cz.

Authors

MEDVEĎ Marek KOVÁŘ Vojtěch JAKUBÍČEK Miloš

Type Article in Proceedings
Conference Proceedings of the First Conference on Machine Translation, Volume 2: Shared Task Papers
MU Faculty or unit

Faculty of Informatics

Citation
Field Informatics
Keywords bilingual document alignment
Description In this paper we present our approach to the Bilingual Document Alignment Task (WMT16), where the main goal was to reach the best recall on extracting aligned pages within the provided data. Our approach consists of tree main parts: data preprocessing, keyword extraction and text pairs scoring based on keyword matching. For text preprocessing we use the TreeTagger pipeline that contains the Unitok tool (Michelfeit et al., 2014) for tokenization and the TreeTagger morphological analyzer (Schmid, 1994). After keywords extraction from the texts according TF-IDF scoring our system searches for comparable English-French pairs. Using a statistical dictionary created from a large English-French parallel corpus, the system is able to find comaparable documents. At the end this procedure is combined with the baseline algorithm and best one-to-one pairing is selected. The result reaches 91.6% recall on provided training data. After a deep error analysis (see section 5) the recall reached 97.4%.
Related projects: