On the Reconstruction of Text Phylogeny Trees: Evaluation and Analysis of Textual Relationships

PLoS One. 2016 Dec 19;11(12):e0167822. doi: 10.1371/journal.pone.0167822. eCollection 2016.

Abstract

Over the history of mankind, textual records change. Sometimes due to mistakes during transcription, sometimes on purpose, as a way to rewrite facts and reinterpret history. There are several classical cases, such as the logarithmic tables, and the transmission of antique and medieval scholarship. Today, text documents are largely edited and redistributed on the Web. Articles on news portals and collaborative platforms (such as Wikipedia), source code, posts on social networks, and even scientific publications or literary works are some examples in which textual content can be subject to changes in an evolutionary process. In this scenario, given a set of near-duplicate documents, it is worthwhile to find which one is the original and the history of changes that created the whole set. Such functionality would have immediate applications on news tracking services, detection of plagiarism, textual criticism, and copyright enforcement, for instance. However, this is not an easy task, as textual features pointing to the documents' evolutionary direction may not be evident and are often dataset dependent. Moreover, side information, such as time stamps, are neither always available nor reliable. In this paper, we propose a framework for reliably reconstructing text phylogeny trees, and seamlessly exploring new approaches on a wide range of scenarios of text reusage. We employ and evaluate distinct combinations of dissimilarity measures and reconstruction strategies within the proposed framework, and evaluate each approach with extensive experiments, including a set of artificial near-duplicate documents with known phylogeny, and from documents collected from Wikipedia, whose modifications were made by Internet users. We also present results from qualitative experiments in two different applications: text plagiarism and reconstruction of evolutionary trees for manuscripts (stemmatology).

MeSH terms

  • Databases, Factual
  • Internet
  • Plagiarism
  • Publications / standards*
  • Software

Grants and funding

The authors received funding from: Conselho Nacional de Desenvolvimento Científico e Tecnológico - CNPQ - National Counsel of Technological and Scientific Development (http://cnpq.br/), CNPq #483370/2013-4 ZD, CNPq #304472/2015-8 AR; Fundação de Amparo à Pesquisa do Estado de São Paulo - FAPESP (http://www.fapesp.br/en/), #2014/19401-8 ZD, #2013/08293-7 ZD, #2014/13433-5 GM, #2014/03535-5 MO, #2015/19222-9 AR; and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (http://www.capes.gov.br) / COFECUB Program, #831/15 ZD, #Deep Eyes Project AR. In addition, part of this material is based on research sponsored by DARPA and Air Force Research Laboratory (AFRL) under agreement number FA8750-16-2-0173. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.