arTenTen: a new, vast corpus for Arabic

Warning

This publication doesn't include Faculty of Arts. It includes Faculty of Informatics. Official publication website can be found on muni.cz.

Authors	BELINKOV Yonatan HABASH Nizar KILGARRIFF Adam ORDAN Noam ROTH Ryan SUCHOMEL Vít
Year of publication	2013
Type	Article in Proceedings
Conference	Proceedings of WACL’2 Second Workshop on Arabic Corpus Linguistics
MU Faculty or unit	Faculty of Informatics
Citation
web	Webové stránky workshopu Sborník abstraktů
Keywords	Arabic corpus; Arabic Corpus Linguistics; MADA; Arabic Gigaword; Modern Standard Arabic
Description	We present arTenTen, a web crawled corpus of Arabic, gathered in 2012, and a member of the TenTen Corpus Family (Jakubíček et al 2013). arTenTen comprises 5.8 billion words. It has been carefully cleaned, including duplicate removal, using the JusText and Onion tools (Pomikalek 2011). We are currently (May 2013) in the process of tokenising, lemmatising and part-of-speech tagging arTenTen with the leading MADA tool version 3.2 (Habash and Rambow 2005; Habash et al. 2009). Once arTenTen is fully encoded, we will compare it with Arabic Gigaword and an earlier web-crawled corpus (Sharoff 2006). We also plan to explore arTenTen’s composition in relation to Modern Standard Arabic and the dialects, using, amongst other things, Buckwalter and Parkinson’s Frequency Dictionary (2011) and the keywords method presented in (Kilgarriff 2012).
Related projects:	Projekt LINDAT-Clarin - Vybudování a provoz českého uzlu pan-evropské infrastruktury pro výzkum