Evert S (2008)
Publication Language: English
Publication Type: Conference contribution, Conference Contribution
Publication year: 2008
City/Town: Marrakech, Morocco
Conference Proceedings Title: Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC 2008)
URI: http://purl.org/stefan.evert/PUB/Evert2008_NCleaner.pdf
Originally conceived as a ”naive” baseline experiment using traditional n-gram language models as classifiers, the NCleaner system has turned out to be a fast and lightweight tool for cleaning Web pages with state-of-the-art accuracy (based on results from the CLEANEVAL competition held in 2007). Despite its simplicity, the algorithm achieves a significant improvement over the baseline (i.e. plain, uncleaned text dumps), trading off recall for substantially higher precision. NCleaner is available as an open-source software package. It is pre-configured for English Web pages, but can adapted to other languages with minimal amounts of manually cleaned training data. Since NCleaner does not make use of HTML structure, it can also be applied to existing Web corpora that are only available in plain text format, with a minor loss in classfication accuracy.
APA:
Evert, S. (2008). A lightweight and efficient tool for cleaning Web pages. In Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC 2008). Marrakech, Morocco.
MLA:
Evert, Stephanie. "A lightweight and efficient tool for cleaning Web pages." Proceedings of the Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC 2008) Marrakech, Morocco, 2008.
BibTeX: Download