Hierarchical Tree-structured Knowledge Graph For Academic Insight Survey
^††thanks: *This work was supported by JSPS KAKENHI Grant Number JP20H04295.

In recent years, there has been significant progress in the digital environment for academic papers. Databases of academic papers, such as S2orc[1] and Unarxiv[2], have been continuously improving. The advent of large language models (LLM) like Chatgpt have opened up new possibilities for utilizing these extensive data sources in NLP tasks, including question-answering systems and automatic summarization for academic papers. This advancement enables researchers to conduct research surveys more efficiently using research tools and resources, such as ChatPDF and SciSpace, which have been developed based on ChatGPT and specifically tailored for academic papers understanding[4]. Additionally, knowledge graphs (KG) built on academic papers, such as Connected Papers[3], make it more convenient for researchers to locate desired papers through keyword search. However, for researchers who are new to an academic field or who are exploring a specific research topic, it is difficult to quickly observe, understand, and explore research directions based on large repositories of academic articles solely by simply reading academic papers without a clear target[5]. This inefficient survey approach hampers the discovery of research direction and innovation. Therefore, relying solely on paper reading comprehension support software and existing paper recommendation systems may make it difficult to achieve the goal of gaining research insights. Providing an overview of possible research directions and branches within multiple papers on a specific topic may greatly facilitate a more efficient exploration of research topics. This process involves the challenging task of conducting an in-depth exploration of keywords, inheritance, relevance, and their applicability within specific research directions. To address these issues, the objective of this study is to create a knowledge graph for insights survey assistants from multiple academic papers on a specific research topic. It presents a tree structure that shows:

(1) From the origin of the research task, expand the citation inheritance and relevance associations . Here, ‘Inheritance’ refers to the route of research evolution from the origin (history) of a research direction to the current stage, which consists of at least two or more academic papers. ‘Relevance’ refers to the path composed of multiple academic papers that aim to solve similar academic issues, which is formed by the relevance chain. Unlike ‘Inheritance’ depended on citation connection, relevance tends to link the similarity of existing academic issues to discover the research direction.

(2) Explore the relevance chain within similar research tasks to demonstrate relevant research points. It provides potential valuable assistance to researchers in understanding, exploring, and gaining insights into research topics. Additionally, it guides researchers in quickly identifying topic clues and directions for further research expansion. This study contributes to an in-depth exploration and visualization of research topics in the following ways:

1. Based on the existing academic paper repository, we performed secondary development to fit the requirements of the research insight survey. We integrate a dataset that includes higher-dimensional features, which reflect the citation structure among papers and enable insights content segmentation.

2. To explore the connections of academic issue relevance between pairs of papers, we utilize machine learning technology to build a training model for classifying the insight content into ‘Issue finding’ and ‘Issue resolved’.

3. Based on the (1) and 2. , we establish a KG in a hierarchical tree structure. This KG serves as a guide for novice researchers exploring research topics to achieve efficient insight surveys.

Research survey is a comprehensive investigation conducted to gain an overview of a research field or topic. Novice researchers need to comprehensively understand the knowledge structure in their field through research surveys. The development of KG can benefit for novice researchers in efficiently comprehending and exploring research topics. As previous studies on the KG generated from academic papers, Xu et al. constructed a PubMed knowledge graph that includes meta information such as bio-entities and authors[6] and Martha et al. proposed the Tree of Science (ToS), which recommends articles based on their position in the graph of citation[7]. However, for novice researchers, relying solely on meta or citation information is insufficient to provide enough information about the research content in that field to gain the overview concept map of the research branch. Deeper data mining from the academic paper is still required to refine the academic knowledge graph, which may provide high-quality guidance for the research survey. The knowledge graphs that involve internal information of academic papers include Chan et al. proposed representation ontology for a four-space integrated KG (background, objectives, solutions, and findings) using NLP technology, as well as Tu et al. proposed Semantic Knowledge Graph (SKG) that integrates semantic concepts to represent the corpus[8][9]. They both classify the content of papers into entity concepts and associated research key-points in their knowledge graphs. However, the association paths of complex concept maps can be cumbersome, making it difficult for novice researchers to accurately locate academic papers in a specific research direction, which may result in limited insight into the research branch. Thus, extracting insightful content from the paper is crucial to guide researchers in conducting effective surveys. Hayashi et al. introduce disentangled paper summarization, where models generate both contribution and context summaries simultaneously to hint at the contribution overview of a specific research direction[10]. This type of summary may also link the contributions of multiple papers to perform a research survey of novelty in that research topic. However, the absence of summaries related to ‘issues that could be improved’ makes it challenging for researchers to trace the evolution history in a particular research branch, such as the relevance of ‘existing issues’ and ‘issues need to be resolved.’ The above studies have yet to involve the intuitive and concise expression of the evolution trajectory and the implicit connection between multiple papers. Therefore, We are committed to mining the inheritance relationships among papers and exploring the relevant connections between their mentioned issues. We focus on expanding knowledge mining in the field of insight survey support, which aims to enable researchers to gain insights into research directions quickly. To achieve this, we propose a hierarchical tree-structured KG based on the insight content of academic papers that reflects the inheritance relationships and relevance chain among academic papers.

In this work, we will use ‘HotpotQA’ as an example, which is a well-known dataset in the field of NLP[11]. The Ideal KG reflecting Inheritance insight and relevance insight survey of ‘HotpotQA’ topic are shown in Fig.1-2. The configuration of the KG is shown in Table I. In Fig.1, the branches:
p1(Dataset)-> p2(Reading Comprehension)
-> p4(Evaluation) and
p1(Dataset)-> p3(Benchmark)-> p7(Retrieval) represent two research directions extended by the core paper of the ‘HotpotQA dataset’. This graph can help researchers infer the elements of research inheritance in the ‘HotpotQA’ topic. For instance, P1 introduced the ‘HotpotQA dataset’, P2 utilized this dataset for reading comprehension, and P4 developed evaluation metrics based on reading comprehension. In Figure 2, the path of ‘Multi-hop’ branch:
p1 -> Network -> p2 -> Single hop -> p4
and the path of ‘Retrieval’ branch:
p1 -> Retrieval -> p3 -> Iteratively -> p7
are extensions of the ‘Multi-hop’ and ‘Retrieval’ research tasks in the ‘HotpotQA’ topic. This tree structure included multiple paths can help researchers infer the correlation factors between subtasks in the ‘HotpotQA’ topic. For example, in the ‘Multi-hop’ subtask, survey paper P1 provides a list of specific tasks, P2 extends it to the network establishment level, and P3 incorporates single-hop methods.

This study developed two types of hierarchical tree-structured knowledge graphs called inheritance tree and relevance tree to support insight surveys for research beginners. The process consists of four stages: data processing, insight sentence extraction, hierarchical tree construction, and KG visualization. First, we conducted high-dimensional secondary development of the S2orc dataset to create an insight survey dataset that includes citation information and insight content. Then, we extracted sentences from the insight survey dataset that express insight viewpoints on Issue finding’ and ’Issue resolved’ using machine learning techniques. These sentences were parsed and used to construct a relevance matrix. Finally, based on the citation information and relevance matrix, we generated two types of hierarchical tree structures: Inheritance and Relevance tree. The generated KG demonstrates their rationality, indicating that they can provide key-information to assist researchers in gaining insights into the directions of the research topic. The knowledge graph also exhibits interpretability and potential for further development. For the future expansion and improvement of this study, the following points are proposed:

(1) Incorporate Additional Text for Text Similarity and Relevance Chain Computation: Currently, this study only utilizes the content from the ‘conclusion’, ‘discussion’, and ‘limitation’ sections. However, the sections ‘abstract’ and ‘introduction,’ specifically the part discussing previous issues, contain insight elements related to ‘issue need to be solved in previous research’ and the ‘objective in this study’. Therefore, integrating these texts for high-dimensional training can optimize the content in the knowledge graph.

(2) We objectively evaluated the classification accuracy of the ’Issue Finding’ and ’Issue Resolved’ viewpoints. However, a future challenge is incorporating researchers’ subjective evaluations into the generated knowledge graph.