ModelWriter: Text & Model-Synchronized Document Engineering Platform

The complexity of software systems in safety critical domains (e.g. avionics and automotive) has significantly increased over the years. Development of such systems requires various phases which result in several artifacts (e.g., requirements documents, architecture models and test cases). In this context, traceability [1, 2] not only establishes and maintains consistency between these artifacts but also helps guarantee that each requirement is fulfilled by the source code and test cases properly cover all requirements, a very important objective in safety critical systems and the standards they need to comply with DO-178C (Software Considerations in Airborne Systems and Equipment Certification) [3] and ISO-26262 (Road Vehicles - Functional Safety) [4]. As a result, the engineers have to establish and maintain several types of traces, having different semantics, between and within various development artifacts.

Traceability is a quality concern that helps users understand each and every steps in the development or even the end to end life cycle of a product. Its implementation is highly contextual as the key artifacts produced or used along a process differ depending on the product. We want to provide a framework for users to specify which artifacts they want to precisely identify and monitor and what is the meaning for trace links between these artifacts.

The considered Artifacts represented in our context by trace locations might be of different levels of granularity, ranging from a complete document or model to fragments of text or code. Focusing on documents and text, both the structure and the content might be used to reason about traceability.

To this end, ModelWriter platform provides a generic traceability analysis applicable to Text & Model artifacts. Trace locations can be fragments of text, elements of an architectural model, and parts of program codes. Traces are relations between trace locations. ModelWriter platform allows axiomatization of these relations and reasoning about them, i.e. supporting traceability analysis for different types of artifacts.

In this paper, we focus on demonstrating the features of ModelWriter platform for the traceability analysis applied to technical documentation. A particular challenge in this use case is to take into account the meaning of natural language. We integrate techniques from Natural Language Processing (NLP) and Automated Reasoning to reason both about the meaning and about the structure of text. We use techniques from semantic parsing to assign formal meaning representations to NL text. We then use techniques from theorem proving and model building to infer traceability relations between text fragments (here SIDPs), to check consistency and to ensure completeness.

We illustrate the workings of the ModelWriter platform based on a set of System Installation Design Principles (SIDP) used by Airbus to ensure the correctness of aircraft design. A SIDP rule is actually a kind of system installation requirement, that is a description of system properties which should be fulfilled. In this usecase, SIDPs are trace locations and there are five types of trace links defined between trace locations, namely contains, refines, conflicts, equals, and requires. In the following, we informally give the meaning of the trace-types.

Rule $r_{1}$ contains Rule $r_{2}\dots r_{n}$ if $r_{2}\dots r_{n}$ are parts of the whole $r_{1}$ (part-whole hierarchy). The contained rule is a sub-rule of containing rule. Rule $r_{1}$ refines another Rule $r_{2}$ if $r_{1}$ is derived from $r_{2}$ by adding more details to its properties. The refined rule can be seen as an abstraction of the detailed rules. In Fig. 1 contains and refines traces are illustrated. Each box represents a property of the corresponding rule.

Rule $r_{1}$ conflicts with Rule $r_{2}$ if the fulfillment of $r_{1}$ excludes the fulfillment of $r_{2}$ and vice versa. The existence of a conflict trace indicates an inconsistency between two rules. Rule $r_{1}$ equals to Rule $r_{2}$ if $r_{1}$ states exactly the same properties with their constraints with $r_{2}$ and vice versa. Rule $r_{1}$ requires Rule $r_{2}$ if $r_{1}$ is fulfilled only when $r_{2}$ is fulfilled. The required rule can be seen as a pre-condition for the requiring rule. In the following Fig. 2 conflicts, equals and requires traces are illustrated.

Given a set of SIDPs, the ModelWriter platform can be used to check completeness and consistency as follows. First, SIDPs are parsed and assigned Description Logic formulae representing their meaning (cf. Section III-A). Second, traces are either manually specified by the end user or can be inferred using semantic parsing and DL theorem proving (cf. Section III-B). Third, new traces can be inferred upon existing ones using Relational Logic (cf. Section III-C) and Model Finding (cf. Section III-D). Importantly, the inference of trace links allows for the detection of missing or inconsistent SIDPs.

Table I illustrates this process. Given the SIDPs $r_{1}$-$r_{6}$, conflicts and refines trace links are first inferred using semantic parsing and the Hermit theorem prover [5] (DL lines in the table).

For example, the DL formulae obtained by parsing sentences $r_{5}$ and $r_{6}$ conflict with each other because the underlying ontology to which these axioms are added specifies that concepts “hydraulic area” and “fuel tank” are disjoint. Similarly, the axiom obtained for the sentence $r_{2}$ refines the axiom obtained for $r_{4}$ because the ontology specifies that “Bracket” is a sub concept of “Adhesive bonded bracket”. In Fig. 3, Table I is represented as a digraph model in which the nodes represent trace-locations, i.e. SIDP rules listed in the table and edges represents traces. A red edge specifically corresponds to the trace inferred using semantic parsing and DL theorem proving. The black one is an example trace, $refines(r_{3},r_{6})$ created by the user manually.

Later, additional trace links are inferred using Relational Model Finding (RL lines in the Table I and dashed blue edges on Fig. 3). For instance, as part of the trace semantics of this use case, according to the axiom schema (3) formalized in Section III-C where a, b and c are artifact elements, if a refines, requires oder contains b, while b conflicts with c, then a also conflicts with c. In this way, ModelWriter  generates conflicts traces such that combination of $conflicts(r_{5},r_{6})$ and $requires(r_{4},r_{5})$ makes $conflicts(r_{6},r_{4})$; on the other hand, according to axiom schema (1) described in Section III-C, the combination of $refines(r_{2},r_{4})$ and $requires(r_{4},r_{5})$ generates $requires(r_{2},r_{5})$ corresponding to the patterns shown in Fig. 4.

Finally, in this example, DL-based reasoning process inferred only one conflicts trace using the meaning of the sentences, i.e. $r_{5}$ conflicts with $r_{6}$ whereas the ModelWriter detects three more conflicts traces using the meaning of trace types by means of RL-based reasoning on top of DL-based reasoning. As a result, it can be seen that not only $r_{5}$ and $r_{6}$ but also $r_{4}$, $r_{1}$, and $r_{2}$ are inconsistent.

We now describe the four main modules making up the ModelWriter platform. Section III-A introduces the semantic parser, i.e., the module that converts text to Description Logic formulae. Section III-B explains how the Hermit reasoner can be used to detect refines, conflicts and equals trace links between text fragments (here, SIDPs). Section III-C shows how Alloy formalism [6] can be customized to axiomatize trace types and semantics. Finally, Section III-D explains how the KodKod model finder [7] is used to infer new trace links between SIDPs to detect the inconsistent SIDPs.

We evaluate Semantic Parsing approach of ModelWriter on a dataset of 960 SIDPs provided by Airbus which demonstrate (i) that the approach is robust (97.50% of the SIDPs can be parsed) and (ii) that DL axioms assigned to full parses are very likely to be correct in 96% of the cases. Regarding inference phase, since we observed that DL-based reasoning is relatively faster than the SAT-based reasoning in the context of SIDP case, we only focus on the Tarski module to evaluate the performance of ModelWriter approach. Table II shows the solving results of three configurations of the formal trace specification running with Alloy Analyzer [6], KodKod[7] and Z3[14]. Minisat[15] SAT Solver is chosen for both Alloy (alloy4.2-2015-02-22.jar) and KodKod (Kodkod 2.1) solvers. From Alloy to SMT solver translation for these cases, we employ the translation method proposed by El Ghazi et.al. [16] and the problems are encoded in SMT-LIB [17] syntax which is fed into Z3 solver. Transitive closure and integer arithmetic are not used in these use cases to fairly benchmark the results with the SMT solver. In SMT-LIB, the logic is set for Equality Logic with Uninterpreted Functions (UF).

Evaluation results are obtained on a machine, that runs 64 bit debian linux operating system with 8 GB of memory and 2.90GHz Intel i7-3520M CPU. Solving times are indicated in milliseconds. The best results are obtained by the direct use of KodKod API since to find satisfiable models, KodKod allows us to configure lower and upper bounds for the solution space employing different pre-processing techniques such as slicing, incremental upper bounds and unrolling transitive closures. The evaluation shows that our tool is practical and beneficial in industrial settings to specify trace semantics for automated trace reasoning. We plan to conduct more case studies to better evaluate the practical utility and usability of the platform.

Many existing works on semantic parsing describe the task of obtaining axiomatic representation of natural language sentences. However, they suffer from two main limitations: (i) use of controlled languages such as Attempto Controlled English[18] (e.g. [19, 20]) and/or (ii) inability to deduce complex axioms involving logical connectives, role restrictions and other expressive features of OWL (e.g. [21, 22]), as noted in [23]. In contrast, we work on human authored real-world text (Airbus SIDPs) and produce complex OWL axioms involving the following DL constructs: $\top$ (the most general concept), disjunction, conjunction, negation, role inverse, universal and existential restrictions. Moreover, we extended the scope of our application by deducing traces among the semantic parse outputs. Such traces were then used as baseline input to Tarski platform which could infer additional traces propagating over the whole system.

Similarly, several approaches and tools have been proposed for automated trace reasoning using the trace semantics [24, 25, 26, 27, 28, 29, 30, 31]. These approaches employ a predefined set of trace types and their corresponding semantics. For instance, Goknil et al. [29] provide a tool for inferencing and consistency checking of traces between requirements using a set of trace types and their formal semantics. Similarly, Egyed and Grünbacher [25] propose a trace generation approach. They do not allow the user to introduce new trace types and their semantics for automated reasoning. In the development of complex systems, it is required to enable the adoption of various trace types, and herewith automated reasoning using their semantics. Tarski module of ModelWriter allows the user to interactively define new trace types with their semantics to be used in automated reasoning about traces.

We presented an integrated platform for automatically mapping natural language text to trace types and performing further inferencing on those traces. Starting with the semantic parser module, we showed how complex axioms could be derived to represent text coming from real world use case. We identified the traces among the parse outputs and fed it to the Tarski tool. The Tarski tool, in turn, allowed users to specify configurable trace semantics for various forms of automated trace reasoning such as inferencing and consistency checking. The key characteristics of our tool are (1) automatic identification of traces existing in texts using semantic parsing (2) allowing user to define new trace types and their semantics which can be later configured, (3) deducing new traces based on the traces which the user has already specified, and (4) identifying traces whose existence causes a contradiction.

This work is conducted within ModelWriter project[32] labeled by the European Union’s EUREKA Cluster programme ITEA and partially supported by the Scientific and Technological Research Council of Turkey under project #9140014, #9150181 and Industry and Digital Affairs of France, Directorate-General for Enterprise under contract #142930204. The authors would like to acknowledge networking support by European Cooperation in Science and Technology Action IC1404 ”Multi-Paradigm Modelling for Cyber-Physical Systems”.