Showing 1–50 of 62 results for author: Mendez, D

Towards Automated Continuous Security Compliance

Authors: Florian Angermeir, Jannik Fischbach, Fabiola Moyón, Daniel Mendez

Abstract: Context: Continuous Software Engineering is increasingly adopted in highly regulated domains, raising the need for continuous compliance. Adherence to especially security regulations -- a major concern in highly regulated domains -- renders Continuous Security Compliance of high relevance to industry and research. Problem: One key barrier to adopting continuous software engineering in the indust… ▽ More Context: Continuous Software Engineering is increasingly adopted in highly regulated domains, raising the need for continuous compliance. Adherence to especially security regulations -- a major concern in highly regulated domains -- renders Continuous Security Compliance of high relevance to industry and research. Problem: One key barrier to adopting continuous software engineering in the industry is the resource-intensive and error-prone nature of traditional manual security compliance activities. Automation promises to be advantageous. However, continuous security compliance is under-researched, precluding an effective adoption. Contribution: We have initiated a long-term research project with our industry partner to address these issues. In this manuscript, we make three contributions: (1) We provide a precise definition of the term continuous security compliance aligning with the state-of-art, (2) elaborate a preliminary overview of challenges in the field of automated continuous security compliance through a tertiary literature study, and (3) present a research roadmap to address those challenges via automated continuous security compliance. △ Less

Submitted 1 August, 2024; v1 submitted 31 July, 2024; originally announced July 2024.

Teaching Survey Research in Software Engineering

Authors: Marcos Kalinowski, Allysson Allex Araújo, Daniel Mendez

Abstract: In this chapter, we provide advice on how to effectively teach survey research based on lessons learned from several international teaching experiences on the topic and from conducting large-scale surveys published at various scientific conferences and journals. First, we provide teachers with a potential syllabus for teaching survey research, including learning objectives, lectures, and examples… ▽ More In this chapter, we provide advice on how to effectively teach survey research based on lessons learned from several international teaching experiences on the topic and from conducting large-scale surveys published at various scientific conferences and journals. First, we provide teachers with a potential syllabus for teaching survey research, including learning objectives, lectures, and examples of practical assignments. Thereafter, we provide actionable advice on how to teach the topics related to each learning objective, including survey design, sampling, data collection, statistical and qualitative analysis, threats to validity and reliability, and ethical considerations. The chapter is complemented by online teaching resources, including slides covering an entire course. △ Less

Submitted 30 July, 2024; originally announced July 2024.

Industrial Practices of Requirements Engineering for ML-Enabled Systems in Brazil

Authors: Antonio Pedro Santos Alves, Marcos Kalinowski, Daniel Mendez, Hugo Villamizar, Kelly Azevedo, Tatiana Escovedo, Helio Lopes

Abstract: [Context] In Brazil, 41% of companies use machine learning (ML) to some extent. However, several challenges have been reported when engineering ML-enabled systems, including unrealistic customer expectations and vagueness in ML problem specifications. Literature suggests that Requirements Engineering (RE) practices and tools may help to alleviate these issues, yet there is insufficient understandi… ▽ More [Context] In Brazil, 41% of companies use machine learning (ML) to some extent. However, several challenges have been reported when engineering ML-enabled systems, including unrealistic customer expectations and vagueness in ML problem specifications. Literature suggests that Requirements Engineering (RE) practices and tools may help to alleviate these issues, yet there is insufficient understanding of RE's practical application and its perception among practitioners. [Goal] This study aims to investigate the application of RE in developing ML-enabled systems in Brazil, creating an overview of current practices, perceptions, and problems in the Brazilian industry. [Method] To this end, we extracted and analyzed data from an international survey focused on ML-enabled systems, concentrating specifically on responses from practitioners based in Brazil. We analyzed RE-related answers gathered from 72 practitioners involved in data-driven projects. We conducted quantitative statistical analyses on contemporary practices using bootstrapping with confidence intervals and qualitative studies on the reported problems involving open and axial coding procedures. [Results] Our findings highlight distinct RE implementation aspects in Brazil's ML projects. For instance, (i) RE-related tasks are predominantly conducted by data scientists; (ii) the most common techniques for eliciting requirements are interviews and workshop meetings; (iii) there is a prevalence of interactive notebooks in requirements documentation; (iv) practitioners report problems that include a poor understanding of the problem to solve and the business domain, low customer engagement, and difficulties managing stakeholders expectations. [Conclusion] These results provide an understanding of RE-related practices in the Brazilian ML industry, helping to guide research toward improving the maturity of RE for ML-enabled systems. △ Less

Submitted 22 July, 2024; originally announced July 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2310.06726

Measuring Information Diffusion in Code Review at Spotify

Authors: Michael Dorner, Daniel Mendez, Ehsan Zabardast, Nicole Valdez, Marcin Floryan

Abstract: Background: As a core practice in software engineering, the nature of code review has been frequently subject to research. Prior exploratory studies found that code review, the discussion around a code change among humans, forms a communication network that enables its participants to exchange and spread information. Although popular in software engineering, there is no confirmatory research corro… ▽ More Background: As a core practice in software engineering, the nature of code review has been frequently subject to research. Prior exploratory studies found that code review, the discussion around a code change among humans, forms a communication network that enables its participants to exchange and spread information. Although popular in software engineering, there is no confirmatory research corroborating this theory and the actual extent of information diffusion in code review is not well understood. Objective: In this registered report, we propose an observational study to measure information diffusion in code review to test the theory of code review as communication network. Method: We approximate the information diffusion in code review through the frequency and the similarity between (1) human participants, (2) affected components, and (3) involved teams of linked code reviews. The measurements approximating the information diffusion in code review serve as a foundation for falsifying the theory of code review as communication network. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Naming the Pain in Machine Learning-Enabled Systems Engineering

Authors: Marcos Kalinowski, Daniel Mendez, Görkem Giray, Antonio Pedro Santos Alves, Kelly Azevedo, Tatiana Escovedo, Hugo Villamizar, Helio Lopes, Teresa Baldassarre, Stefan Wagner, Stefan Biffl, Jürgen Musil, Michael Felderer, Niklas Lavesson, Tony Gorschek

Abstract: Context: Machine learning (ML)-enabled systems are being increasingly adopted by companies aiming to enhance their products and operational processes. Objective: This paper aims to deliver a comprehensive overview of the current status quo of engineering ML-enabled systems and lay the foundation to steer practically relevant and problem-driven academic research. Method: We conducted an internation… ▽ More Context: Machine learning (ML)-enabled systems are being increasingly adopted by companies aiming to enhance their products and operational processes. Objective: This paper aims to deliver a comprehensive overview of the current status quo of engineering ML-enabled systems and lay the foundation to steer practically relevant and problem-driven academic research. Method: We conducted an international survey to collect insights from practitioners on the current practices and problems in engineering ML-enabled systems. We received 188 complete responses from 25 countries. We conducted quantitative statistical analyses on contemporary practices using bootstrapping with confidence intervals and qualitative analyses on the reported problems using open and axial coding procedures. Results: Our survey results reinforce and extend existing empirical evidence on engineering ML-enabled systems, providing additional insights into typical ML-enabled systems project contexts, the perceived relevance and complexity of ML life cycle phases, and current practices related to problem understanding, model deployment, and model monitoring. Furthermore, the qualitative analysis provides a detailed map of the problems practitioners face within each ML life cycle phase and the problems causing overall project failure. Conclusions: The results contribute to a better understanding of the status quo and problems in practical environments. We advocate for the further adaptation and dissemination of software engineering practices to enhance the engineering of ML-enabled systems. △ Less

Submitted 20 May, 2024; originally announced June 2024.

Comments: arXiv admin note: text overlap with arXiv:2310.06726

Requirements Quality Research Artifacts: Recovery, Analysis, and Management Guideline

Authors: Julian Frattini, Lloyd Montgomery, Davide Fucci, Michael Unterkalmsteiner, Daniel Mendez, Jannik Fischbach

Abstract: Requirements quality research, which is dedicated to assessing and improving the quality of requirements specifications, is dependent on research artifacts like data sets (containing information about quality defects) and implementations (automatically detecting and removing these defects). However, recent research exposed that the majority of these research artifacts have become unavailable or ha… ▽ More Requirements quality research, which is dedicated to assessing and improving the quality of requirements specifications, is dependent on research artifacts like data sets (containing information about quality defects) and implementations (automatically detecting and removing these defects). However, recent research exposed that the majority of these research artifacts have become unavailable or have never been disclosed, which inhibits progress in the research domain. In this work, we aim to improve the availability of research artifacts in requirements quality research. To this end, we (1) extend an artifact recovery initiative, (2) empirically evaluate the reasons for artifact unavailability using Bayesian data analysis, and (3) compile a concise guideline for open science artifact disclosure. Our results include 10 recovered data sets and 7 recovered implementations, empirical support for artifact availability improving over time and the positive effect of public hosting services, and a pragmatic artifact management guideline open for community comments. With this work, we hope to encourage and support adherence to open science principles and improve the availability of research artifacts for the requirements research quality community. △ Less

Submitted 3 June, 2024; originally announced June 2024.

Measuring the Fitness-for-Purpose of Requirements: An initial Model of Activities and Attributes

Authors: Julian Frattini, Jannik Fischbach, Davide Fucci, Michael Unterkalmsteiner, Daniel Mendez

Abstract: Requirements engineering aims to fulfill a purpose, i.e., inform subsequent software development activities about stakeholders' needs and constraints that must be met by the system under development. The quality of requirements artifacts and processes is determined by how fit for this purpose they are, i.e., how they impact activities affected by them. However, research on requirements quality lac… ▽ More Requirements engineering aims to fulfill a purpose, i.e., inform subsequent software development activities about stakeholders' needs and constraints that must be met by the system under development. The quality of requirements artifacts and processes is determined by how fit for this purpose they are, i.e., how they impact activities affected by them. However, research on requirements quality lacks a comprehensive overview of these activities and how to measure them. In this paper, we specify the research endeavor addressing this gap and propose an initial model of requirements-affected activities and their attributes. We construct a model from three distinct data sources, including both literature and empirical data. The results yield an initial model containing 24 activities and 16 attributes quantifying these activities. Our long-term goal is to develop evidence-based decision support on how to optimize the fitness for purpose of the RE phase to best support the subsequent, affected software development process. We do so by measuring the effect that requirements artifacts and processes have on the attributes of these activities. With the contribution at hand, we invite the research community to critically discuss our research roadmap and support the further evolution of the model. △ Less

Submitted 16 May, 2024; originally announced May 2024.

Practices, Challenges, and Opportunities When Inferring Requirements From Regulations in the FinTech Sector - An Industrial Study

Authors: Parisa Elahidoost, Daniel Mendez, Michael Unterkalmsteiner, Jannik Fischbach, Christian Feiler, Jonathan Streit

Abstract: [Context and motivation]: Understanding and interpreting regulatory norms and inferring software requirements from them is a critical step towards regulatory compliance, a matter of significant importance in various industrial sectors. [Question/ problem]: However, interpreting regulations still largely depends on individual legal expertise and experience within the respective domain, with little… ▽ More [Context and motivation]: Understanding and interpreting regulatory norms and inferring software requirements from them is a critical step towards regulatory compliance, a matter of significant importance in various industrial sectors. [Question/ problem]: However, interpreting regulations still largely depends on individual legal expertise and experience within the respective domain, with little to no systematic methodologies and supportive tools to guide this practice. In fact, research in this area is too often detached from practitioners' experiences, rendering the proposed solutions not transferable to industrial practice. As we argue, one reason is that we still lack a profound understanding of industry- and domain-specific practices and challenges. [Principal ideas/ results]: We aim to close this gap and provide such an investigation at the example of the banking and insurance domain. We conduct an industrial multi-case study as part of a long-term academia-industry collaboration with a medium-sized software development and renovation company. We explore contemporary industrial practices and challenges when inferring requirements from regulations to support more problem-driven research. Our study investigates the complexities of requirement engineering in regulatory contexts, pinpointing various issues and discussing them in detail. We highlight the gathered insights and the practical challenges encountered and suggest avenues for future research. [Contribution]: Our contribution is a comprehensive case study focused on the FinTech domain, offering a detailed understanding of the specific needs within this sector. We have identified key practices for managing regulatory requirements in software development, and have pinpointed several challenges. We conclude by offering a set of recommendations for future problem-driven research directions. △ Less

Submitted 5 May, 2024; originally announced May 2024.

On Developing an Artifact-based Approach to Regulatory Requirements Engineering

Authors: Oleksandr Kosenkov, Michael Unterkalmsteiner, Jannik Fischbach, Daniel Mendez, Davide Fucci, Tony Gorschek

Abstract: Context: Regulatory acts are a challenging source when eliciting, interpreting, and analyzing requirements. Requirements engineers often need to involve legal experts who, however, may often not be available. This raises the need for approaches to regulatory Requirements Engineering (RE) covering and integrating both legal and engineering perspectives. Problem: Regulatory RE approaches need to c… ▽ More Context: Regulatory acts are a challenging source when eliciting, interpreting, and analyzing requirements. Requirements engineers often need to involve legal experts who, however, may often not be available. This raises the need for approaches to regulatory Requirements Engineering (RE) covering and integrating both legal and engineering perspectives. Problem: Regulatory RE approaches need to capture and reflect both the elementary concepts and relationships from a legal perspective and their seamless transition to concepts used to specify software requirements. No existing approach considers explicating and managing legal domain knowledge and engineering-legal coordination. Method: We conducted focus group sessions with legal researchers to identify the core challenges to establishing a regulatory RE approach. Based on our findings, we developed a candidate solution and conducted a first conceptual validation to assess its feasibility. Results: We introduce the first version of our Artifact Model for Regulatory Requirements Engineering (AM4RRE) and its conceptual foundation. It provides a blueprint for applying legal (modelling) concepts and well-established RE concepts. Our initial results suggest that artifact-centric RE can be applied to managing legal domain knowledge and engineering-legal coordination. Conclusions: The focus groups that served as a basis for building our model and the results from the expert validation both strengthen our confidence that we already provide a valuable basis for systematically integrating legal concepts into RE. This overcomes contemporary challenges to regulatory RE and serves as a basis for exposure to critical discussions in the community before continuing with the development of tool-supported extensions and large-scale empirical evaluations in practice. △ Less

Submitted 1 May, 2024; originally announced May 2024.

Comments: The paper was accepted to the 14th International Model-Driven Requirements Engineering (MoDRE) workshop co-located with the 32nd IEEE International Requirements Engineering Conference (RE 2024) in Reykjavik, Iceland

Towards an Approach to Pattern-based Domain-Specific Requirements Engineering

Authors: T. Chuprina, D. Méndez, V. Nigam, M. Reich, A. Schweiger

Abstract: Requirements specification patterns have received much attention as they promise to guide the structured specification of natural language requirements. By using them, the intention is to reduce quality problems related to requirements artifacts. Patterns may need to vary in their syntax (e.g. domain details/ parameter incorporation) and semantics according to the particularities of the applicatio… ▽ More Requirements specification patterns have received much attention as they promise to guide the structured specification of natural language requirements. By using them, the intention is to reduce quality problems related to requirements artifacts. Patterns may need to vary in their syntax (e.g. domain details/ parameter incorporation) and semantics according to the particularities of the application domain. However, pattern-based approaches, such as EARS, are designed domain-independently to facilitate their wide adoption across several domains. Little is yet known about how to adopt the principle idea of pattern-based requirements engineering to cover domain-specificity in requirements engineering and, ideally, integrate requirements engineering activities into quality assurance tasks. In this paper, we propose the Pattern-based Domain-specific Requirements Engineering Approach for the specification of functional and performance requirements in a holistic manner. This approach emerges from an academia-industry collaboration and is our first attempt to frame an approach which allows for analyzing domain knowledge and incorporating it into the requirements engineering process enabling automated checks for requirements quality assurance and computer-aided support for system verification. Our contribution is two-fold: First, we present a solution to pattern-based domain-specific requirements engineering and its exemplary integration into quality assurance techniques. Second, we showcase a proof of concept using a tool implementation for the domain of flight controllers for Unmanned Aerial Vehicles. Both shall allow us to outline next steps in our research agenda and foster discussions in this direction. △ Less

Submitted 26 April, 2024; originally announced April 2024.

Comments: 6 pages with 3 figures

ACM Class: D.2.1

NLP4RE Tools: Classification, Overview, and Management

Authors: Julian Frattini, Michael Unterkalmsteiner, Davide Fucci, Daniel Mendez

Abstract: Tools constitute an essential contribution to natural language processing for requirements engineering (NLP4RE) research. They are executable instruments that make research usable and applicable in practice. In this chapter, we first introduce a systematic classification of NLP4RE tools to improve the understanding of their types and properties. Then, we extend an existing overview with a systemat… ▽ More Tools constitute an essential contribution to natural language processing for requirements engineering (NLP4RE) research. They are executable instruments that make research usable and applicable in practice. In this chapter, we first introduce a systematic classification of NLP4RE tools to improve the understanding of their types and properties. Then, we extend an existing overview with a systematic summary of 126 NLP4RE tools published between April 2019 and June 2023 to ease reuse and evolution of existing tools. Finally, we provide instructions on how to create, maintain, and disseminate NLP4RE tools to support a more rigorous management and dissemination. △ Less

Submitted 11 March, 2024; originally announced March 2024.

Comments: The final version of the chapter will eventually appear in a book titled "Natural Language Processing for Requirements Engineering", edited by Alessio Ferrari and Gouri Ginde and published by Springer Nature Group

A Second Look at the Impact of Passive Voice Requirements on Domain Modeling: Bayesian Reanalysis of an Experiment

Authors: Julian Frattini, Davide Fucci, Richard Torkar, Daniel Mendez

Abstract: The quality of requirements specifications may impact subsequent, dependent software engineering (SE) activities. However, empirical evidence of this impact remains scarce and too often superficial as studies abstract from the phenomena under investigation too much. Two of these abstractions are caused by the lack of frameworks for causal inference and frequentist methods which reduce complex data… ▽ More The quality of requirements specifications may impact subsequent, dependent software engineering (SE) activities. However, empirical evidence of this impact remains scarce and too often superficial as studies abstract from the phenomena under investigation too much. Two of these abstractions are caused by the lack of frameworks for causal inference and frequentist methods which reduce complex data to binary results. In this study, we aim to demonstrate (1) the use of a causal framework and (2) contrast frequentist methods with more sophisticated Bayesian statistics for causal inference. To this end, we reanalyze the only known controlled experiment investigating the impact of passive voice on the subsequent activity of domain modeling. We follow a framework for statistical causal inference and employ Bayesian data analysis methods to re-investigate the hypotheses of the original study. Our results reveal that the effects observed by the original authors turned out to be much less significant than previously assumed. This study supports the recent call to action in SE research to adopt Bayesian data analysis, including causal frameworks and Bayesian statistics, for more sophisticated causal inference. △ Less

Submitted 16 February, 2024; originally announced February 2024.

Comments: Published at the first International Workshop on Methodological Issues with Empirical Studies in Software Engineering (WSESE '24)

ML-Enabled Systems Model Deployment and Monitoring: Status Quo and Problems

Authors: Eduardo Zimelewicz, Marcos Kalinowski, Daniel Mendez, Görkem Giray, Antonio Pedro Santos Alves, Niklas Lavesson, Kelly Azevedo, Hugo Villamizar, Tatiana Escovedo, Helio Lopes, Stefan Biffl, Juergen Musil, Michael Felderer, Stefan Wagner, Teresa Baldassarre, Tony Gorschek

Abstract: [Context] Systems incorporating Machine Learning (ML) models, often called ML-enabled systems, have become commonplace. However, empirical evidence on how ML-enabled systems are engineered in practice is still limited, especially for activities surrounding ML model dissemination. [Goal] We investigate contemporary industrial practices and problems related to ML model dissemination, focusing on the… ▽ More [Context] Systems incorporating Machine Learning (ML) models, often called ML-enabled systems, have become commonplace. However, empirical evidence on how ML-enabled systems are engineered in practice is still limited, especially for activities surrounding ML model dissemination. [Goal] We investigate contemporary industrial practices and problems related to ML model dissemination, focusing on the model deployment and the monitoring of ML life cycle phases. [Method] We conducted an international survey to gather practitioner insights on how ML-enabled systems are engineered. We gathered a total of 188 complete responses from 25 countries. We analyze the status quo and problems reported for the model deployment and monitoring phases. We analyzed contemporary practices using bootstrapping with confidence intervals and conducted qualitative analyses on the reported problems applying open and axial coding procedures. [Results] Practitioners perceive the model deployment and monitoring phases as relevant and difficult. With respect to model deployment, models are typically deployed as separate services, with limited adoption of MLOps principles. Reported problems include difficulties in designing the architecture of the infrastructure for production deployment and legacy application integration. Concerning model monitoring, many models in production are not monitored. The main monitored aspects are inputs, outputs, and decisions. Reported problems involve the absence of monitoring practices, the need to create custom monitoring tools, and the selection of suitable metrics. [Conclusion] Our results help provide a better understanding of the adopted practices and problems in practice and support guiding ML deployment and monitoring research in a problem-driven manner. △ Less

Submitted 7 February, 2024; originally announced February 2024.

Comments: arXiv admin note: text overlap with arXiv:2310.06726

Industrial Challenges in Secure Continuous Development

Authors: Fabiola Moyón, Florian Angermeir, Daniel Mendez

Abstract: The intersection between security and continuous software engineering has been of great interest since the early years of the agile development movement, and it remains relevant as software development processes are more frequently guided by agility and the adoption of DevOps. Several authors have contributed studies about the framing of secure agile development and secure DevOps, motivating acade… ▽ More The intersection between security and continuous software engineering has been of great interest since the early years of the agile development movement, and it remains relevant as software development processes are more frequently guided by agility and the adoption of DevOps. Several authors have contributed studies about the framing of secure agile development and secure DevOps, motivating academic contributions to methods and practices, but also discussions around benefits and challenges. Especially the challenges captured also our interest since, for the last few years, we are conducting research on secure continuous software engineering from a more applied, practical perspective with the overarching aim to introduce solutions that can be adopted at scale. The short positioning at hands summarizes a relevant part of our endeavors in which we validated challenges with several practitioners of different roles. More than framing a set of challenges, we conclude by presenting four key research directions we identified for practitioners and researchers to delineate future work. △ Less

Submitted 12 January, 2024; originally announced January 2024.

Applying Bayesian Data Analysis for Causal Inference about Requirements Quality: A Controlled Experiment

Authors: Julian Frattini, Davide Fucci, Richard Torkar, Lloyd Montgomery, Michael Unterkalmsteiner, Jannik Fischbach, Daniel Mendez

Abstract: It is commonly accepted that the quality of requirements specifications impacts subsequent software engineering activities. However, we still lack empirical evidence to support organizations in deciding whether their requirements are good enough or impede subsequent activities. We aim to contribute empirical evidence to the effect that requirements quality defects have on a software engineering ac… ▽ More It is commonly accepted that the quality of requirements specifications impacts subsequent software engineering activities. However, we still lack empirical evidence to support organizations in deciding whether their requirements are good enough or impede subsequent activities. We aim to contribute empirical evidence to the effect that requirements quality defects have on a software engineering activity that depends on this requirement. We conduct a controlled experiment in which 25 participants from industry and university generate domain models from four natural language requirements containing different quality defects. We evaluate the resulting models using both frequentist and Bayesian data analysis. Contrary to our expectations, our results show that the use of passive voice only has a minor impact on the resulting domain models. The use of ambiguous pronouns, however, shows a strong effect on various properties of the resulting domain models. Most notably, ambiguous pronouns lead to incorrect associations in domain models. Despite being equally advised against by literature and frequentist methods, the Bayesian data analysis shows that the two investigated quality defects have vastly different impacts on software engineering activities and, hence, deserve different levels of attention. Our employed method can be further utilized by researchers to improve reliable, detailed empirical evidence on requirements quality. △ Less

Submitted 1 July, 2024; v1 submitted 2 January, 2024; originally announced January 2024.

Automatic extraction of cause-effect-relations from requirements artifacts

Authors: Julian Frattini, Maximilian Junker, Michael Unterkalmsteiner, Daniel Mendez

Abstract: Background: The detection and extraction of causality from natural language sentences have shown great potential in various fields of application. The field of requirements engineering is eligible for multiple reasons: (1) requirements artifacts are primarily written in natural language, (2) causal sentences convey essential context about the subject of requirements, and (3) extracted and formaliz… ▽ More Background: The detection and extraction of causality from natural language sentences have shown great potential in various fields of application. The field of requirements engineering is eligible for multiple reasons: (1) requirements artifacts are primarily written in natural language, (2) causal sentences convey essential context about the subject of requirements, and (3) extracted and formalized causality relations are usable for a (semi-)automatic translation into further artifacts, such as test cases. Objective: We aim at understanding the value of interactive causality extraction based on syntactic criteria for the context of requirements engineering. Method: We developed a prototype of a system for automatic causality extraction and evaluate it by applying it to a set of publicly available requirements artifacts, determining whether the automatic extraction reduces the manual effort of requirements formalization. Result: During the evaluation we analyzed 4457 natural language sentences from 18 requirements documents, 558 of which were causal (12.52%). The best evaluation of a requirements document provided an automatic extraction of 48.57% cause-effect graphs on average, which demonstrates the feasibility of the approach. Limitation: The feasibility of the approach has been proven in theory but lacks exploration of being scaled up for practical use. Evaluating the applicability of the automatic causality extraction for a requirements engineer is left for future research. Conclusion: A syntactic approach for causality extraction is viable for the context of requirements engineering and can aid a pipeline towards an automatic generation of further artifacts from requirements artifacts. △ Less

Submitted 12 December, 2023; originally announced December 2023.

Comments: ASE '20: Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering

Describing Globally Distributed Software Architectures for Tax Compliance

Authors: Michael Dorner, Oliver Treidler, Tom-Eric Kunz, Ehsan Zabardast, Daniel Mendez, Darja Šmite, Maximilian Capraro, Krzysztof Wnuk

Abstract: Background: The company-internal reuse of software components owned by organizational units in different countries constitutes an implicit licensing across borders, which is taxable. This makes tax authorities a less known stakeholder in software architectures. Objective: Therefore, we investigate how software companies can describe the implicit license structure of their globally distributed soft… ▽ More Background: The company-internal reuse of software components owned by organizational units in different countries constitutes an implicit licensing across borders, which is taxable. This makes tax authorities a less known stakeholder in software architectures. Objective: Therefore, we investigate how software companies can describe the implicit license structure of their globally distributed software architectures to tax authorities. Method: We develop a viewpoint that frames the concerns of tax authorities, use this viewpoint to construct a view of a large-scale microservice architecture of a multinational enterprise, and evaluate the resulting software architecture description with a panel of four tax experts. Results: The panel found our proposed architectural viewpoint properly and sufficiently frames the concerns of taxation stakeholders. However, unclear jurisdictions of owners and potentially insufficient definitions of code ownership and software component introduce significant noise to the view that limits the usefulness and explanatory power of our software architecture description. Conclusion: While our software architecture description provides a solid foundation, we believe it only represents the tip of the iceberg. Future research is necessary to pave the way for advancements in tax compliance within software engineering. △ Less

Submitted 9 July, 2024; v1 submitted 1 December, 2023; originally announced December 2023.

Comments: submitted to EMSE

Status Quo and Problems of Requirements Engineering for Machine Learning: Results from an International Survey

Authors: Antonio Pedro Santos Alves, Marcos Kalinowski, Görkem Giray, Daniel Mendez, Niklas Lavesson, Kelly Azevedo, Hugo Villamizar, Tatiana Escovedo, Helio Lopes, Stefan Biffl, Jürgen Musil, Michael Felderer, Stefan Wagner, Teresa Baldassarre, Tony Gorschek

Abstract: Systems that use Machine Learning (ML) have become commonplace for companies that want to improve their products and processes. Literature suggests that Requirements Engineering (RE) can help address many problems when engineering ML-enabled systems. However, the state of empirical evidence on how RE is applied in practice in the context of ML-enabled systems is mainly dominated by isolated case s… ▽ More Systems that use Machine Learning (ML) have become commonplace for companies that want to improve their products and processes. Literature suggests that Requirements Engineering (RE) can help address many problems when engineering ML-enabled systems. However, the state of empirical evidence on how RE is applied in practice in the context of ML-enabled systems is mainly dominated by isolated case studies with limited generalizability. We conducted an international survey to gather practitioner insights into the status quo and problems of RE in ML-enabled systems. We gathered 188 complete responses from 25 countries. We conducted quantitative statistical analyses on contemporary practices using bootstrapping with confidence intervals and qualitative analyses on the reported problems involving open and axial coding procedures. We found significant differences in RE practices within ML projects. For instance, (i) RE-related activities are mostly conducted by project leaders and data scientists, (ii) the prevalent requirements documentation format concerns interactive Notebooks, (iii) the main focus of non-functional requirements includes data quality, model reliability, and model explainability, and (iv) main challenges include managing customer expectations and aligning requirements with data. The qualitative analyses revealed that practitioners face problems related to lack of business domain understanding, unclear goals and requirements, low customer engagement, and communication issues. These results help to provide a better understanding of the adopted practices and of which problems exist in practical environments. We put forward the need to adapt further and disseminate RE-related practices for engineering ML-enabled systems. △ Less

Submitted 10 October, 2023; originally announced October 2023.

Comments: Accepted for Publication at PROFES 2023

Requirements Quality Research: a harmonized Theory, Evaluation, and Roadmap

Authors: Julian Frattini, Lloyd Montgomery, Jannik Fischbach, Daniel Mendez, Davide Fucci, Michael Unterkalmsteiner

Abstract: High-quality requirements minimize the risk of propagating defects to later stages of the software development life cycle. Achieving a sufficient level of quality is a major goal of requirements engineering. This requires a clear definition and understanding of requirements quality. Though recent publications make an effort at disentangling the complex concept of quality, the requirements quality… ▽ More High-quality requirements minimize the risk of propagating defects to later stages of the software development life cycle. Achieving a sufficient level of quality is a major goal of requirements engineering. This requires a clear definition and understanding of requirements quality. Though recent publications make an effort at disentangling the complex concept of quality, the requirements quality research community lacks identity and clear structure which guides advances and puts new findings into an holistic perspective. In this research commentary we contribute (1) a harmonized requirements quality theory organizing its core concepts, (2) an evaluation of the current state of requirements quality research, and (3) a research roadmap to guide advancements in the field. We show that requirements quality research focuses on normative rules and mostly fails to connect requirements quality to its impact on subsequent software development activities, impeding the relevance of the research. Adherence to the proposed requirements quality theory and following the outlined roadmap will be a step towards amending this gap. △ Less

Submitted 19 September, 2023; originally announced September 2023.

Comments: Requirements Eng (2023)

Identifying Concerns When Specifying Machine Learning-Enabled Systems: A Perspective-Based Approach

Authors: Hugo Villamizar, Marcos Kalinowski, Helio Lopes, Daniel Mendez

Abstract: Engineering successful machine learning (ML)-enabled systems poses various challenges from both a theoretical and a practical side. Among those challenges are how to effectively address unrealistic expectations of ML capabilities from customers, managers and even other team members, and how to connect business value to engineering and data science activities composed by interdisciplinary teams. In… ▽ More Engineering successful machine learning (ML)-enabled systems poses various challenges from both a theoretical and a practical side. Among those challenges are how to effectively address unrealistic expectations of ML capabilities from customers, managers and even other team members, and how to connect business value to engineering and data science activities composed by interdisciplinary teams. In this paper, we present PerSpecML, a perspective-based approach for specifying ML-enabled systems that helps practitioners identify which attributes, including ML and non-ML components, are important to contribute to the overall system's quality. The approach involves analyzing 59 concerns related to typical tasks that practitioners face in ML projects, grouping them into five perspectives: system objectives, user experience, infrastructure, model, and data. Together, these perspectives serve to mediate the communication between business owners, domain experts, designers, software and ML engineers, and data scientists. The creation of PerSpecML involved a series of validations conducted in different contexts: (i) in academia, (ii) with industry representatives, and (iii) in two real industrial case studies. As a result of the diverse validations and continuous improvements, PerSpecML stands as a promising approach, poised to positively impact the specification of ML-enabled systems, particularly helping to reveal key components that would have been otherwise missed without using PerSpecML. △ Less

Submitted 14 September, 2023; originally announced September 2023.

The Upper Bound of Information Diffusion in Code Review

Authors: Michael Dorner, Daniel Mendez, Krzysztof Wnuk, Ehsan Zabardast, Jacek Czerwonka

Abstract: Background: Code review, the discussion around a code change among humans, forms a communication network that enables its participants to exchange and spread information. Although reported by qualitative studies, our understanding of the capability of code review as a communication network is still limited. Objective: In this article, we report on a first step towards evaluating the capability of… ▽ More Background: Code review, the discussion around a code change among humans, forms a communication network that enables its participants to exchange and spread information. Although reported by qualitative studies, our understanding of the capability of code review as a communication network is still limited. Objective: In this article, we report on a first step towards evaluating the capability of code review as a communication network by quantifying how fast and how far information can spread through code review: the upper bound of information diffusion in code review. Method: In an in-silico experiment, we simulate an artificial information diffusion within large (Microsoft), mid-sized (Spotify), and small code review systems (Trivago) modelled as communication networks. We then measure the minimal topological and temporal distances between the participants to quantify how far and how fast information can spread in code review. Results: An average code review participants in the small and mid-sized code review systems can spread information to between 72% and 85% of all code review participants within four weeks independently of network size and tooling; for the large code review systems, we found an absolute boundary of about 11000 reachable participants. On average (median), information can spread between two participants in code review in less than five hops and less than five days. Conclusion: We found evidence that the communication network emerging from code review scales well and spreads information fast and broadly, corroborating the findings of prior qualitative work. The study lays the foundation for understanding and improving code review as a communication network. △ Less

Submitted 11 July, 2024; v1 submitted 15 June, 2023; originally announced June 2023.

Comments: To appear in Empirical Software Engineering Journal

ACM Class: D.2.7; D.2.9

Preliminary Guideline for Creating Boundary Artefacts in Software Engineering

Authors: Raquel Ouriques, Fabian Fagerholm, Daniel Mendez, Tony Gorschek, Baldvin Gislason Bern

Abstract: Context: Software development benefits from having Boundary Artefacts (BAs), as a single artefact can supply stakeholders with different boundaries, facilitating collaboration among social worlds. When those artefacts display inconsistencies, such as incorrect information, the practitioners have decreased trust in the BA. As trust is an essential factor guiding the utilisation of BAs in software p… ▽ More Context: Software development benefits from having Boundary Artefacts (BAs), as a single artefact can supply stakeholders with different boundaries, facilitating collaboration among social worlds. When those artefacts display inconsistencies, such as incorrect information, the practitioners have decreased trust in the BA. As trust is an essential factor guiding the utilisation of BAs in software projects, it is necessary to understand which principles should be observed when creating them. Objective: This study aimed at develop and validate a preliminary guideline support the creation of trustworthy BAs. Method: We followed a multi-step approach. We developed our guideline through a literature review and previous results from our case study. Second, we submitted the guideline for an expert evaluation via two workshops and a survey. At last, we adjusted our guideline by incorporating the feedback obtained during the workshops. Results: We grouped the principles collected from a literature review into three categories. The first category (Scope) focuses on the scope, displaying principles referring to defining each boundary's target audience, needs, and terminology. The second category (Structure) relates to how the artefact's content is structured to meet stakeholders' needs. The third (Management) refers to principles that can guide the establishment of practices to manage the artefact throughout time. The expert validation revealed that the principles contribute to creating trustworthy BAs at different levels. Also, the relevance of the guideline and its usefulness. Conclusions: The guideline strengthen BA traits such as shared understanding, plasticity and ability to transfer. Practitioners can utilise the guideline to guide the creation or even evaluate current practices for existing BAs. △ Less

Submitted 9 June, 2023; originally announced June 2023.

Comments: 25 pages

Connecting the Dots of Knowledge in Agile Software Development

Authors: Raquel Ouriques, Tony Gorschek, Daniel Mendez, Fabian Fagerholm

Abstract: This article discusses the importance of managing knowledge as a resource due to its great potential to create economic value. We detail the types of knowledge resources, the challenges associated with their management, and potential solutions to maximise their utility. Our contribution is based on empirical studies performed in an industry context. This article discusses the importance of managing knowledge as a resource due to its great potential to create economic value. We detail the types of knowledge resources, the challenges associated with their management, and potential solutions to maximise their utility. Our contribution is based on empirical studies performed in an industry context. △ Less

Submitted 9 June, 2023; originally announced June 2023.

Comments: 8 pages

Taxing Collaborative Software Engineering

Authors: Michael Dorner, Maximilian Capraro, Oliver Treidler, Tom-Eric Kunz, Darja Šmite, Ehsan Zabardast, Daniel Mendez, Krzysztof Wnuk

Abstract: The engineering of complex software systems is often the result of a highly collaborative effort. However, collaboration within a multinational enterprise has an overlooked legal implication when developers collaborate across national borders: It is taxable. In this article, we discuss the unsolved problem of taxing collaborative software engineering across borders. We (1) introduce the reader to… ▽ More The engineering of complex software systems is often the result of a highly collaborative effort. However, collaboration within a multinational enterprise has an overlooked legal implication when developers collaborate across national borders: It is taxable. In this article, we discuss the unsolved problem of taxing collaborative software engineering across borders. We (1) introduce the reader to the basic principle of international taxation, (2) identify three main challenges for taxing collaborative software engineering making it a software engineering problem, and (3) estimate the industrial significance of cross-border collaboration in modern software engineering by measuring cross-border code reviews at a multinational software company. △ Less

Submitted 21 November, 2023; v1 submitted 13 April, 2023; originally announced April 2023.

Comments: 8 pages; 3 figures; submitted to IEEE Software

Let's Stop Building at the Feet of Giants: Recovering unavailable Requirements Quality Artifacts

Authors: Julian Frattini, Lloyd Montgomery, Davide Fucci, Jannik Fischbach, Michael Unterkalmsteiner, Daniel Mendez

Abstract: Requirements quality literature abounds with publications presenting artifacts, such as data sets and tools. However, recent systematic studies show that more than 80% of these artifacts have become unavailable or were never made public, limiting reproducibility and reusability. In this work, we report on an attempt to recover those artifacts. To that end, we requested corresponding authors of una… ▽ More Requirements quality literature abounds with publications presenting artifacts, such as data sets and tools. However, recent systematic studies show that more than 80% of these artifacts have become unavailable or were never made public, limiting reproducibility and reusability. In this work, we report on an attempt to recover those artifacts. To that end, we requested corresponding authors of unavailable artifacts to recover and disclose them according to open science principles. Our results, based on 19 answers from 35 authors (54% response rate), include an assessment of the availability of requirements quality artifacts and a breakdown of authors' reasons for their continued unavailability. Overall, we improved the availability of seven data sets and seven implementations. △ Less

Submitted 10 April, 2023; originally announced April 2023.

Automatic ESG Assessment of Companies by Mining and Evaluating Media Coverage Data: NLP Approach and Tool

Authors: Jannik Fischbach, Max Adam, Victor Dzhagatspanyan, Daniel Mendez, Julian Frattini, Oleksandr Kosenkov, Parisa Elahidoost

Abstract: Context: Sustainable corporate behavior is increasingly valued by society and impacts corporate reputation and customer trust. Hence, companies regularly publish sustainability reports to shed light on their impact on environmental, social, and governance (ESG) factors. Problem: Sustainability reports are written by companies themselves and are therefore considered a company-controlled source. Con… ▽ More Context: Sustainable corporate behavior is increasingly valued by society and impacts corporate reputation and customer trust. Hence, companies regularly publish sustainability reports to shed light on their impact on environmental, social, and governance (ESG) factors. Problem: Sustainability reports are written by companies themselves and are therefore considered a company-controlled source. Contrary, studies reveal that non-corporate channels (e.g., media coverage) represent the main driver for ESG transparency. However, analysing media coverage regarding ESG factors is challenging since (1) the amount of published news articles grows daily, (2) media coverage data does not necessarily deal with an ESG-relevant topic, meaning that it must be carefully filtered, and (3) the majority of media coverage data is unstructured. Research Goal: We aim to extract ESG-relevant information from textual media reactions automatically to calculate an ESG score for a given company. Our goal is to reduce the cost of ESG data collection and make ESG information available to the general public. Contribution: Our contributions are three-fold: First, we publish a corpus of 432,411 news headlines annotated as being environmental-, governance-, social-related, or ESG-irrelevant. Second, we present our tool-supported approach called ESG-Miner capable of analyzing and evaluating headlines on corporate ESG-performance automatically. Third, we demonstrate the feasibility of our approach in an experiment and apply the ESG-Miner on 3000 manually labeled headlines. Our approach processes 96.7 % of the headlines correctly and shows a great performance in detecting environmental-related headlines along with their correct sentiment. We encourage fellow researchers and practitioners to use the ESG-Miner at https://www.esg-miner.com. △ Less

Submitted 28 February, 2024; v1 submitted 13 December, 2022; originally announced December 2022.

An initial Theory to Understand and Manage Requirements Engineering Debt in Practice

Authors: Julian Frattini, Davide Fucci, Daniel Mendez, Rodrigo Spinola, Vladimir Mandic, Nebojsa Tausan, Muhammad Ovais Ahmad, Javier Gonzalez-Huerta

Abstract: Context: Advances in technical debt research demonstrate the benefits of applying the financial debt metaphor to support decision-making in software development activities. Although decision-making during requirements engineering has significant consequences, the debt metaphor in requirements engineering is inadequately explored. Objective: We aim to conceptualize how the debt metaphor applies to… ▽ More Context: Advances in technical debt research demonstrate the benefits of applying the financial debt metaphor to support decision-making in software development activities. Although decision-making during requirements engineering has significant consequences, the debt metaphor in requirements engineering is inadequately explored. Objective: We aim to conceptualize how the debt metaphor applies to requirements engineering by organizing concepts related to practitioners' understanding and managing of requirements engineering debt (RED). Method: We conducted two in-depth expert interviews to identify key requirements engineering debt concepts and construct a survey instrument. We surveyed 69 practitioners worldwide regarding their perception of the concepts and developed an initial analytical theory. Results: We propose a RED theory that aligns key concepts from technical debt research but emphasizes the specific nature of requirements engineering. In particular, the theory consists of 23 falsifiable propositions derived from the literature, the interviews, and survey results. Conclusions: The concepts of requirements engineering debt are perceived to be similar to their technical debt counterpart. Nevertheless, measuring and tracking requirements engineering debt are immature in practice. Our proposed theory serves as the first guide toward further research in this area. △ Less

Submitted 8 March, 2023; v1 submitted 11 November, 2022; originally announced November 2022.

Comments: Revised manuscript based on reviews

Model-based Analysis and Specification of Functional Requirements and Tests for Complex Automotive Systems

Authors: Carsten Wiecher, Constantin Mandel, Matthias Günther, Jannik Fischbach, Joel Greenyer, Matthias Greinert, Carsten Wolff, Roman Dumitrescu, Daniel Mendez, Albert Albers

Abstract: The specification of requirements and tests are crucial activities in automotive development projects. However, due to the increasing complexity of automotive systems, practitioners fail to specify requirements and tests for distributed and evolving systems with complex interactions when following traditional development processes. To address this research gap, we propose a technique that starts w… ▽ More The specification of requirements and tests are crucial activities in automotive development projects. However, due to the increasing complexity of automotive systems, practitioners fail to specify requirements and tests for distributed and evolving systems with complex interactions when following traditional development processes. To address this research gap, we propose a technique that starts with the early identification of validation concerns from a stakeholder perspective, which we use to systematically design tests that drive a scenario-based modeling and analysis of system requirements. To ensure complete and consistent requirements and test specifications in a form that is required in automotive development projects, we develop a Model-Based Systems Engineering (MBSE) methodology. This methodology supports system architects and test designers in the collaborative application of our technique and in maintaining a central system model, in order to automatically derive the required specifications. We evaluate our methodology by applying it at KOSTAL (Tier1 supplier) and within student projects as part of the masters program Embedded Systems Engineering. Our study corroborates that our methodology is applicable and improves existing requirements and test specification processes by supporting the integrated and stakeholder-focused modeling of product and validation systems, where the early definition of stakeholder and validation concerns fosters a problem-oriented, iterative and test-driven requirements modeling. △ Less

Submitted 15 November, 2023; v1 submitted 3 September, 2022; originally announced September 2022.

Understanding the Implementation of Technical Measures in the Process of Data Privacy Compliance: A Qualitative Study

Authors: Oleksandra Klymenko, Oleksandr Kosenkov, Stephen Meisenbacher, Parisa Elahidoost, Daniel Mendez, Florian Matthes

Abstract: Modern privacy regulations, such as the General Data Protection Regulation (GDPR), address privacy in software systems in a technologically agnostic way by mentioning general "technical measures" for data privacy compliance rather than dictating how these should be implemented. An understanding of the concept of technical measures and how exactly these can be handled in practice, however, is not t… ▽ More Modern privacy regulations, such as the General Data Protection Regulation (GDPR), address privacy in software systems in a technologically agnostic way by mentioning general "technical measures" for data privacy compliance rather than dictating how these should be implemented. An understanding of the concept of technical measures and how exactly these can be handled in practice, however, is not trivial due to its interdisciplinary nature and the necessary technical-legal interactions. We aim to investigate how the concept of technical measures for data privacy compliance is understood in practice as well as the technical-legal interaction intrinsic to the process of implementing those technical measures. We follow a research design that is 1) exploratory in nature, 2) qualitative, and 3) interview-based, with 16 selected privacy professionals in the technical and legal domains. Our results suggest that there is no clear mutual understanding and commonly accepted approach to handling technical measures. Both technical and legal roles are involved in the implementation of such measures. While they still often operate in separate spheres, a predominant opinion amongst the interviewees is to promote more interdisciplinary collaboration. Our empirical findings confirm the need for better interaction between legal and engineering teams when implementing technical measures for data privacy. We posit that interdisciplinary collaboration is paramount to a more complete understanding of technical measures, which currently lacks a mutually accepted notion. Yet, as strongly suggested by our results, there is still a lack of systematic approaches to such interaction. Therefore, the results strengthen our confidence in the need for further investigations into the technical-legal dynamic of data privacy compliance. △ Less

Submitted 18 August, 2022; originally announced August 2022.

Comments: The 16th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)

A Live Extensible Ontology of Quality Factors for Textual Requirements

Authors: Julian Frattini, Lloyd Montgomery, Jannik Fischbach, Michael Unterkalmsteiner, Daniel Mendez, Davide Fucci

Abstract: Quality factors like passive voice or sentence length are commonly used in research and practice to evaluate the quality of natural language requirements since they indicate defects in requirements artifacts that potentially propagate to later stages in the development life cycle. However, as a research community, we still lack a holistic perspective on quality factors. This inhibits not only a co… ▽ More Quality factors like passive voice or sentence length are commonly used in research and practice to evaluate the quality of natural language requirements since they indicate defects in requirements artifacts that potentially propagate to later stages in the development life cycle. However, as a research community, we still lack a holistic perspective on quality factors. This inhibits not only a comprehensive understanding of the existing body of knowledge but also the effective use and evolution of these factors. To this end, we propose an ontology of quality factors for textual requirements, which includes (1) a structure framing quality factors and related elements and (2) a central repository and web interface making these factors publicly accessible and usable. We contribute the first version of both by applying a rigorous ontology development method to 105 eligible primary studies and construct a first version of the repository and interface. We illustrate the usability of the ontology and invite fellow researchers to a joint community effort to complete and maintain this knowledge repository. We envision our ontology to reflect the community's harmonized perception of requirements quality factors, guide reporting of new quality factors, and provide central access to the current body of knowledge. △ Less

Submitted 13 June, 2022; originally announced June 2022.

Comments: 7 pages, 1 figure, requirements engineering conference

Accelerating X-Ray Tracing for Exascale Systems using Kokkos

Authors: Felix Wittwer, Nicholas K. Sauter, Derek Mendez, Billy K. Poon, Aaron S. Brewster, James M. Holton, Michael E. Wall, William E. Hart, Deborah J. Bard, Johannes P. Blaschke

Abstract: The upcoming exascale computing systems Frontier and Aurora will draw much of their computing power from GPU accelerators. The hardware for these systems will be provided by AMD and Intel, respectively, each supporting their own GPU programming model. The challenge for applications that harness one of these exascale systems will be to avoid lock-in and to preserve performance portability. We rep… ▽ More The upcoming exascale computing systems Frontier and Aurora will draw much of their computing power from GPU accelerators. The hardware for these systems will be provided by AMD and Intel, respectively, each supporting their own GPU programming model. The challenge for applications that harness one of these exascale systems will be to avoid lock-in and to preserve performance portability. We report here on our results of using Kokkos to accelerate a real-world application on NERSC's Perlmutter Phase 1 (using NVIDIA A100 accelerators) and the testbed system for OLCF's Frontier (using AMD MI250X). By porting to Kokkos, we were able to successfully run the same X-ray tracing code on both systems and achieved speed-ups between 13% and 66% compared to the original CUDA code. These results are a highly encouraging demonstration of using Kokkos to accelerate production science code. △ Less

Submitted 16 May, 2022; originally announced May 2022.

Work-From-Home is Here to Stay: Call for Flexibility in Post-Pandemic Work Policies

Authors: Darja Smite, Nils Brede Moe, Jarle Hildrum, Javier Gonzalez Huerta, Daniel Mendez

Abstract: In early 2020, the Covid-19 pandemic forced employees in tech companies worldwide to abruptly transition from working in offices to working from their homes. During two years of predominantly working from home, employees and managers alike formed expectations about what post-pandemic working life should look like. Many companies are currently experimenting with new work policies that balance both… ▽ More In early 2020, the Covid-19 pandemic forced employees in tech companies worldwide to abruptly transition from working in offices to working from their homes. During two years of predominantly working from home, employees and managers alike formed expectations about what post-pandemic working life should look like. Many companies are currently experimenting with new work policies that balance both employee- and manager expectations to where, when and how work should be done in the future. In this article, we gather experiences from 17 companies and their sites, covering 12 countries. We share the results of corporate surveys of employee preferences for working from home and analyse new work policies. Our results are threefold. First, through the new work policies all companies are formally giving more flexibility to the employees with regards to working time and work location. Second, there is a great variation in how much flexibility the companies are willing to yield to the employees. The variation is related both to industry type, size of the companies, and company culture. Third, we document a change in the psychological contract between employees and managers, where the option of working from home is converted from an exclusive perk that managers could choose to give to the few, to a core privilege that all employees feel they are entitled to. Finally, there are indications that as the companies learn and solicit feedback regarding the efficiency of the chosen strategies, we might see further developments and changes of the work policies with respect to how much flexibility to work whenever and from anywhere they grant. Through these findings, the paper contributes to a growing literature about the new trends emerging from the pandemic in tech companies and spells out practical implications onwards. △ Less

Submitted 21 March, 2022; originally announced March 2022.

Comments: Submitted to the Journal of Systems and Software, New Ideas and Trends track

Automatic Creation of Acceptance Tests by Extracting Conditionals from Requirements: NLP Approach and Case Study

Authors: Jannik Fischbach, Julian Frattini, Andreas Vogelsang, Daniel Mendez, Michael Unterkalmsteiner, Andreas Wehrle, Pablo Restrepo Henao, Parisa Yousefi, Tedi Juricic, Jeannette Radduenz, Carsten Wiecher

Abstract: Acceptance testing is crucial to determine whether a system fulfills end-user requirements. However, the creation of acceptance tests is a laborious task entailing two major challenges: (1) practitioners need to determine the right set of test cases that fully covers a requirement, and (2) they need to create test cases manually due to insufficient tool support. Existing approaches for automatical… ▽ More Acceptance testing is crucial to determine whether a system fulfills end-user requirements. However, the creation of acceptance tests is a laborious task entailing two major challenges: (1) practitioners need to determine the right set of test cases that fully covers a requirement, and (2) they need to create test cases manually due to insufficient tool support. Existing approaches for automatically deriving test cases require semi-formal or even formal notations of requirements, though unrestricted natural language is prevalent in practice. In this paper, we present our tool-supported approach CiRA (Conditionals in Requirements Artifacts) capable of creating the minimal set of required test cases from conditional statements in informal requirements. We demonstrate the feasibility of CiRA in a case study with three industry partners. In our study, out of 578 manually created test cases, 71.8 % can be generated automatically. Additionally, CiRA discovered 80 relevant test cases that were missed in manual test case design. CiRA is publicly available at www.cira.bth.se/demo/. △ Less

Submitted 13 October, 2022; v1 submitted 2 February, 2022; originally announced February 2022.

Causality in Requirements Artifacts: Prevalence, Detection, and Impact

Authors: Julian Frattini, Jannik Fischbach, Daniel Mendez, Michael Unterkalmsteiner, Andreas Vogelsang, Krzystof Wnuk

Abstract: Background: Causal relations in natural language (NL) requirements convey strong, semantic information. Automatically extracting such causal information enables multiple use cases, such as test case generation, but it also requires to reliably detect causal relations in the first place. Currently, this is still a cumbersome task as causality in NL requirements is still barely understood and, thus,… ▽ More Background: Causal relations in natural language (NL) requirements convey strong, semantic information. Automatically extracting such causal information enables multiple use cases, such as test case generation, but it also requires to reliably detect causal relations in the first place. Currently, this is still a cumbersome task as causality in NL requirements is still barely understood and, thus, barely detectable. Objective: In our empirically informed research, we aim at better understanding the notion of causality and supporting the automatic extraction of causal relations in NL requirements. Method: In a first case study, we investigate 14.983 sentences from 53 requirements documents to understand the extent and form in which causality occurs. Second, we present and evaluate a tool-supported approach, called CiRA, for causality detection. We conclude with a second case study where we demonstrate the applicability of our tool and investigate the impact of causality on NL requirements. Results: The first case study shows that causality constitutes around 28% of all NL requirements sentences. We then demonstrate that our detection tool achieves a macro-F1 score of 82% on real-world data and that it outperforms related approaches with an average gain of 11.06% in macro-Recall and 11.43% in macro-Precision. Finally, our second case study corroborates the positive correlations of causality with features of NL requirements. Conclusion: The results strengthen our confidence in the eligibility of causal relations for downstream reuse, while our tool and publicly available data constitute a first step in the ongoing endeavors of utilizing causality in RE and beyond. △ Less

Submitted 15 December, 2021; originally announced December 2021.

Comments: arXiv admin note: substantial text overlap with arXiv:2101.10766

Combining Design Thinking and Software Requirements Engineering to create Human-centered Software-intensive Systems

Authors: Jennifer Hehn, Daniel Mendez

Abstract: Effective Requirements Engineering is a crucial activity in softwareintensive development projects. The human-centric working mode of Design Thinking is considered a powerful way to complement such activities when designing innovative systems. Research has already made great strides to illustrate the benefits of using Design Thinking for Requirements Engineering. However, it has remained mostly un… ▽ More Effective Requirements Engineering is a crucial activity in softwareintensive development projects. The human-centric working mode of Design Thinking is considered a powerful way to complement such activities when designing innovative systems. Research has already made great strides to illustrate the benefits of using Design Thinking for Requirements Engineering. However, it has remained mostly unclear how to actually realize a combination of both. In this chapter, we contribute an artifact-based model that integrates Design Thinking and Requirements Engineering for innovative software-intensive systems. Drawing from our research and project experiences, we suggest three strategies for tailoring and integrating Design Thinking and Requirements Engineering with complementary synergies. △ Less

Submitted 10 December, 2021; originally announced December 2021.

Comments: Introductory chapter to the edited book on "Design Thinking for Software Engineering - Creating Human-oriented Software- intensive Products and Services". To appear in 2022

Only Time Will Tell: Modelling Information Diffusion in Code Review with Time-Varying Hypergraphs

Authors: Michael Dorner, Darja Šmite, Daniel Mendez, Krzysztof Wnuk, Jacek Czerwonka

Abstract: Background: Modern code review is expected to facilitate knowledge sharing: All relevant information, the collective expertise, and meta-information around the code change and its context become evident, transparent, and explicit in the corresponding code review discussion. The discussion participants can leverage this information in the following code reviews; the information diffuses through the… ▽ More Background: Modern code review is expected to facilitate knowledge sharing: All relevant information, the collective expertise, and meta-information around the code change and its context become evident, transparent, and explicit in the corresponding code review discussion. The discussion participants can leverage this information in the following code reviews; the information diffuses through the communication network that emerges from code review. Traditional time-aggregated graphs fall short in rendering information diffusion as those models ignore the temporal order of the information exchange: Information can only be passed on if it is available in the first place. Aim: This manuscript presents a novel model based on time-varying hypergraphs for rendering information diffusion that overcomes the inherent limitations of traditional, time-aggregated graph-based models. Method: In an in-silico experiment, we simulate an information diffusion within the internal code review at Microsoft and show the empirical impact of time on a key characteristic of information diffusion: the number of reachable participants. Results: Time-aggregation significantly overestimates the paths of information diffusion available in communication networks and, thus, is neither precise nor accurate for modelling and measuring the spread of information within communication networks that emerge from code review. Conclusion: Our model overcomes the inherent limitations of traditional, static or time-aggregated, graph-based communication models and sheds the first light on information diffusion through code review. We believe that our model can serve as a foundation for understanding, measuring, managing, and improving knowledge sharing in code review in particular and information diffusion in software engineering in general. △ Less

Submitted 1 September, 2022; v1 submitted 14 October, 2021; originally announced October 2021.

Comments: 10 pages, 6 figures

How Do Practitioners Interpret Conditionals in Requirements?

Authors: Jannik Fischbach, Julian Frattini, Daniel Mendez, Michael Unterkalmsteiner, Henning Femmer, Andreas Vogelsang

Abstract: Context: Conditional statements like "If A and B then C" are core elements for describing software requirements. However, there are many ways to express such conditionals in natural language and also many ways how they can be interpreted. We hypothesize that conditional statements in requirements are a source of ambiguity, potentially affecting downstream activities such as test case generation ne… ▽ More Context: Conditional statements like "If A and B then C" are core elements for describing software requirements. However, there are many ways to express such conditionals in natural language and also many ways how they can be interpreted. We hypothesize that conditional statements in requirements are a source of ambiguity, potentially affecting downstream activities such as test case generation negatively. Objective: Our goal is to understand how specific conditionals are interpreted by readers who work with requirements. Method: We conduct a descriptive survey with 104 RE practitioners and ask how they interpret 12 different conditional clauses. We map their interpretations to logical formulas written in Propositional (Temporal) Logic and discuss the implications. Results: The conditionals in our tested requirements were interpreted ambiguously. We found that practitioners disagree on whether an antecedent is only sufficient or also necessary for the consequent. Interestingly, the disagreement persists even when the system behavior is known to the practitioners. We also found that certain cue phrases are associated with specific interpretations. Conclusion: Conditionals in requirements are a source of ambiguity and there is not just one way to interpret them formally. This affects any analysis that builds upon formalized requirements (e.g., inconsistency checking, test-case generation). Our results may also influence guidelines for writing requirements. △ Less

Submitted 5 September, 2021; originally announced September 2021.

Vision for an Artefact-based Approach to Regulatory Requirements Engineering

Authors: Oleksandr Kosenkov, Michael Unterkalmsteiner, Daniel Mendez, Davide Fucci

Abstract: Background: Nowadays, regulatory requirements engineering (regulatory RE) faces challenges of interdisciplinary nature that cannot be tackled due to existing research gaps. Aims: We envision an approach to solve some of the challenges related to the nature and complexity of regulatory requirements, the necessity for domain knowledge, and the involvement of legal experts in regulatory RE. Method: W… ▽ More Background: Nowadays, regulatory requirements engineering (regulatory RE) faces challenges of interdisciplinary nature that cannot be tackled due to existing research gaps. Aims: We envision an approach to solve some of the challenges related to the nature and complexity of regulatory requirements, the necessity for domain knowledge, and the involvement of legal experts in regulatory RE. Method: We suggest the qualitative analysis of regulatory texts combined with the further case study to develop an empirical foundation for our research. Results: We outline our vision for the application of extended artefact-based modeling for regulatory RE. Conclusions: Empirical methodology is an essential instrument to address interdisciplinarity and complexity in regulatory RE. Artefact-based modeling supported by empirical results can solve a particular set of problems while not limiting the application of other methods and tools and facilitating the interaction between different fields of practice and research. △ Less

Submitted 30 August, 2021; originally announced August 2021.

Fine-Grained Causality Extraction From Natural Language Requirements Using Recursive Neural Tensor Networks

Authors: Jannik Fischbach, Tobias Springer, Julian Frattini, Henning Femmer, Andreas Vogelsang, Daniel Mendez

Abstract: [Context:] Causal relations (e.g., If A, then B) are prevalent in functional requirements. For various applications of AI4RE, e.g., the automatic derivation of suitable test cases from requirements, automatically extracting such causal statements are a basic necessity. [Problem:] We lack an approach that is able to extract causal relations from natural language requirements in fine-grained form. S… ▽ More [Context:] Causal relations (e.g., If A, then B) are prevalent in functional requirements. For various applications of AI4RE, e.g., the automatic derivation of suitable test cases from requirements, automatically extracting such causal statements are a basic necessity. [Problem:] We lack an approach that is able to extract causal relations from natural language requirements in fine-grained form. Specifically, existing approaches do not consider the combinatorics between causes and effects. They also do not allow to split causes and effects into more granular text fragments (e.g., variable and condition), making the extracted relations unsuitable for automatic test case derivation. [Objective & Contributions:] We address this research gap and make the following contributions: First, we present the Causality Treebank, which is the first corpus of fully labeled binary parse trees representing the composition of 1,571 causal requirements. Second, we propose a fine-grained causality extractor based on Recursive Neural Tensor Networks. Our approach is capable of recovering the composition of causal statements written in natural language and achieves a F1 score of 74 % in the evaluation on the Causality Treebank. Third, we disclose our open data sets as well as our code to foster the discourse on the automatic extraction of causality in the RE community. △ Less

Submitted 22 July, 2021; v1 submitted 21 July, 2021; originally announced July 2021.

Real-Time XFEL Data Analysis at SLAC and NERSC: a Trial Run of Nascent Exascale Experimental Data Analysis

Authors: Johannes P. Blaschke, Aaron S. Brewster, Daniel W. Paley, Derek Mendez, Asmit Bhowmick, Nicholas K. Sauter, Wilko Kröger, Murali Shankar, Bjoern Enders, Deborah Bard

Abstract: X-ray scattering experiments using Free Electron Lasers (XFELs) are a powerful tool to determine the molecular structure and function of unknown samples (such as COVID-19 viral proteins). XFEL experiments are a challenge to computing in two ways: i) due to the high cost of running XFELs, a fast turnaround time from data acquisition to data analysis is essential to make informed decisions on experi… ▽ More X-ray scattering experiments using Free Electron Lasers (XFELs) are a powerful tool to determine the molecular structure and function of unknown samples (such as COVID-19 viral proteins). XFEL experiments are a challenge to computing in two ways: i) due to the high cost of running XFELs, a fast turnaround time from data acquisition to data analysis is essential to make informed decisions on experimental protocols; ii) data collection rates are growing exponentially, requiring new scalable algorithms. Here we report our experiences analyzing data from two experiments at the Linac Coherent Light Source (LCLS) during September 2020. Raw data were analyzed on NERSC's Cori XC40 system, using the Superfacility paradigm: our workflow automatically moves raw data between LCLS and NERSC, where it is analyzed using the software package CCTBX. We achieved real time data analysis with a turnaround time from data acquisition to full molecular reconstruction in as little as 10 min -- sufficient time for the experiment's operators to make informed decisions. By hosting the data analysis on Cori, and by automating LCLS-NERSC interoperability, we achieved a data analysis rate which matches the data acquisition rate. Completing data analysis with 10 mins is a first for XFEL experiments and an important milestone if we are to keep up with data collection trends. △ Less

Submitted 31 December, 2023; v1 submitted 21 June, 2021; originally announced June 2021.

A Study about the Knowledge and Use of Requirements Engineering Standards in Industry

Authors: Xavier Franch, Martin Glinz, Daniel Mendez, Norbert Seyff

Abstract: Context: The use of standards is considered a vital part of any engineering discipline. So one could expect that standards play an important role in Requirements Engineering (RE) as well. However, little is known about the actual knowledge and use of RE-related standards in industry. Objective: In this article, we investigate to which extent standards and related artifacts such as templates or gui… ▽ More Context: The use of standards is considered a vital part of any engineering discipline. So one could expect that standards play an important role in Requirements Engineering (RE) as well. However, little is known about the actual knowledge and use of RE-related standards in industry. Objective: In this article, we investigate to which extent standards and related artifacts such as templates or guidelines are known and used by RE practitioners. Method: To this end, we have conducted a questionnaire-based online survey. We could analyze the replies from 90 RE practitioners using a combination of closed and open-text questions. Results: Our results indicate that the knowledge and use of standards and related artifacts in RE is less widespread than one might expect from an engineering perspective. For example, about 47% of the respondents working as requirements engineers or business analysts do not know the core standard in RE, ISO/IEC/IEEE 29148. Participants in our study mostly use standards by personal decision rather than being imposed by their respective company, customer, or regulator. Beyond insufficient knowledge, we also found cultural and organizational factors impeding the widespread adoption of standards in RE. Conclusions: Overall, our results provide empirically informed insights into the actual use of standards and related artifacts in RE practice and - indirectly - about the value that the current standards create for RE practitioners. △ Less

Submitted 6 September, 2021; v1 submitted 28 May, 2021; originally announced May 2021.

Comments: Preprint accepted for publication at IEEE Transactions on Software Engineering and presented as Journal First at the International Requirements Engineering conference 2021. Latest update: Several smaller corrections along the creation of the final version of the manuscript

Journal ref: Transactions on Software Engineering 2022

Using Process Models to understand Security Standards

Authors: Fabiola Moyón, Daniel Méndez, Kristian Beckers, Sebastian Klepper

Abstract: Many industrial software development processes today have to comply with security standards such as the IEC~62443-4-1. These standards, written in natural language, are ambiguous and complex to understand. This is especially true for non-security experts. Security practitioners thus invest much effort into comprehending standards and, later, into introducing them to development teams. However, our… ▽ More Many industrial software development processes today have to comply with security standards such as the IEC~62443-4-1. These standards, written in natural language, are ambiguous and complex to understand. This is especially true for non-security experts. Security practitioners thus invest much effort into comprehending standards and, later, into introducing them to development teams. However, our experience in the industry shows that development practitioners might very well also read such standards, but nevertheless end up inviting experts for interpretation (or confirmation). Such a scenario is not in tune with current trends and needs of increasing velocity in continuous software engineering. In this paper, we propose a tool-supported approach to make security standards more precise and easier to understand for both non-security as well as security experts by applying process models. This approach emerges from a large industrial company and encompasses so far the IEC62443-4-1 standard. We further present a case study with 16 industry practitioners showing how the approach improves communication between development and security compliance practitioners. △ Less

Submitted 27 May, 2021; originally announced May 2021.

Comments: Authors Copy

Journal ref: International Conference on Current Trends in Theory and Practice of Informatics SOFSEM 2021: Theory and Practice of Computer Science pp 458-471

Integration of Security Standards in DevOps Pipelines: An Industry Case Study

Authors: Fabiola Moyón Constante, Rafael Soares, Maria Pinto-Albuquerque, Daniel Méndez, Kristian Beckers

Abstract: In the last decade, companies adopted DevOps as a fast path to deliver software products according to customer expectations, with well aligned teams and in continuous cycles. As a basic practice, DevOps relies on pipelines that simulate factory swim-lanes. The more automation in the pipeline, the shorter a lead time is supposed to be. However, applying DevOps is challenging, particularly for indus… ▽ More In the last decade, companies adopted DevOps as a fast path to deliver software products according to customer expectations, with well aligned teams and in continuous cycles. As a basic practice, DevOps relies on pipelines that simulate factory swim-lanes. The more automation in the pipeline, the shorter a lead time is supposed to be. However, applying DevOps is challenging, particularly for industrial control systems (ICS) that support critical infrastructures and that must obey to rigorous requirements from security regulations and standards. Current research on security compliant DevOps presents open gaps for this particular domain and in general for systematic application of security standards. In this paper, we present a systematic approach to integrate standard-based security activities into DevOps pipelines and highlight their automation potential. Our intention is to share our experiences and help practitioners to overcome the trade-off between adding security activities into the development process and keeping a short lead time. We conducted an evaluation of our approach at a large industrial company considering the IEC 62443-4-1 security standard that regulates ICS. The results strengthen our confidence in the usefulness of our approach and artefacts, and in that they can support practitioners to achieve security compliance while preserving agility including short lead times. △ Less

Submitted 27 May, 2021; originally announced May 2021.

Comments: Author's Copy of the Manuscript

Journal ref: International Conference on Product-Focused Software Process Improvement PROFES 2020: 434-452

The human side of Software Engineering Teams: an investigation of contemporary challenges

Authors: Marco Hoffmann, Daniel Mendez, Fabian Fagerholm, Anton Luckhardt

Abstract: There have been recent calls for research on the human side of software engineering and its impact on various factors such as productivity, developer happiness and project success. An analysis of which challenges in software engineering teams are most frequent is still missing. We aim to provide a starting point for a theory about relevant human challenges and their causes in software engineerin… ▽ More There have been recent calls for research on the human side of software engineering and its impact on various factors such as productivity, developer happiness and project success. An analysis of which challenges in software engineering teams are most frequent is still missing. We aim to provide a starting point for a theory about relevant human challenges and their causes in software engineering. We establish a reusable set of challenges and start out by investigating the effect of team virtualization. Virtual teams often use digital communication and consist of members with different nationalities. We designed a survey instrument and asked respondents to assess the frequency and criticality of a set of challenges, separated in context "within teams" as well as "between teams and clients", compiled from previous empiric work, blog posts and pilot survey feedback. For the team challenges, we asked if mitigation measures were already in place. Respondents were also asked to provide information about their team setup. The survey also measured Schwartz human values. Finally, respondents were asked if there were additional challenges at their workplace. We report on the results obtained from 192 respondents. We present a set of challenges that takes the survey feedback into account and introduce two categories of challenges; "interpersonal" and "intrapersonal". We found no evidence for links between human values and challenges. We found some significant links between the number of distinct nationalities in a team and certain challenges, with less frequent and critical challenges occurring if 2-3 different nationalities were present compared to a team having members of just one nationality or more than three. A higher degree of virtualization seems to increase the frequency of some human challenges. △ Less

Submitted 31 January, 2022; v1 submitted 8 April, 2021; originally announced April 2021.

Comments: 18 pages, 6 Figures, Accepted by IEEE TSE

Towards Artefact-based Requirements Engineering for Data-Centric Systems

Authors: Tatiana Chuprina, Daniel Mendez, Krzysztof Wnuk

Abstract: Many modern software-intensive systems employ artificial intelligence / machine-learning (AI/ML) components and are, thus, inherently data-centric. The behaviour of such systems depends on typically large amounts of data processed at run-time rendering such non-deterministic systems as complex. This complexity growth affects our understanding on needs and practices in Requirements Engineering (RE)… ▽ More Many modern software-intensive systems employ artificial intelligence / machine-learning (AI/ML) components and are, thus, inherently data-centric. The behaviour of such systems depends on typically large amounts of data processed at run-time rendering such non-deterministic systems as complex. This complexity growth affects our understanding on needs and practices in Requirements Engineering (RE). There is, however, still little guidance on how to handle requirements for such systems effectively: What are, for example, typical quality requirements classes? What modelling concepts do we rely on or which levels of abstraction do we need to consider? In fact, how to integrate such concepts into approaches for a more traditional RE still needs profound investigations. In this research preview paper, we report on ongoing efforts to establish an artefact-based RE approach for the development of datacentric systems (DCSs). To this end, we sketch a DCS development process with the newly proposed requirements categories and data-centric artefacts and briefly report on an ongoing investigation of current RE challenges in industry developing data-centric systems. △ Less

Submitted 9 March, 2021; originally announced March 2021.

Comments: Authors' version accepted for publication at the 2nd International Workshop on Requirements Engineering for Artificial Intelligence, co-located with REFSQ 2021

On Understanding the Relation of Knowledge and Confidence to Requirements Quality

Authors: Razieh Dehghani, Krzysztof Wnuk, Daniel Mendez, Tony Gorschek, Raman Ramsin

Abstract: Context and Motivation: Software requirements are affected by the knowledge and confidence of software engineers. Analyzing the interrelated impact of these factors is difficult because of the challenges of assessing knowledge and confidence. Question/Problem: This research aims to draw attention to the need for considering the interrelated effects of confidence and knowledge on requirements qua… ▽ More Context and Motivation: Software requirements are affected by the knowledge and confidence of software engineers. Analyzing the interrelated impact of these factors is difficult because of the challenges of assessing knowledge and confidence. Question/Problem: This research aims to draw attention to the need for considering the interrelated effects of confidence and knowledge on requirements quality, which has not been addressed by previous publications. Principal ideas/results: For this purpose, the following steps have been taken: 1) requirements quality was defined based on the instructions provided by the ISO29148:2011 standard, 2) we selected the symptoms of low qualified requirements based on ISO29148:2011, 3) we analyzed five Software Requirements Specification (SRS) documents to find these symptoms, 3) people who have prepared the documents were categorized in four classes to specify the more/less knowledge and confidence they have regarding the symptoms, and 4) finally, the relation of lack of enough knowledge and confidence to symptoms of low quality was investigated. The results revealed that the simultaneous deficiency of confidence and knowledge has more negative effects in comparison with a deficiency of knowledge or confidence. Contribution: In brief, this study has achieved these results: 1) the realization that a combined lack of knowledge and confidence has a larger effect on requirements quality than only one of the two factors, 2) the relation between low qualified requirements and requirements engineers' needs for knowledge and confidence, and 3) variety of requirements engineers' needs for knowledge based on their abilities to make discriminative and consistent decisions. △ Less

Submitted 3 March, 2021; originally announced March 2021.

Comments: Preprint accepted for publication at the 27th International Working Conference on Requirement Engineering: Foundation for Software Quality

Journal ref: Proc. of the 27th International Working Conference on Requirement Engineering: Foundation for Software Quality 2021

Compliance Requirements in Large-Scale Software Development: An Industrial Case Study

Authors: Muhammad Usman, Michael Felderer, Michael Unterkalmsteiner, Eriks Klotins, Daniel Mendez, Emil Alegroth

Abstract: Regulatory compliance is a well-studied area, including research on how to model, check, analyse, enact, and verify compliance of software. However, while the theoretical body of knowledge is vast, empirical evidence on challenges with regulatory compliance, as faced by industrial practitioners particularly in the Software Engineering domain, is still lacking. In this paper, we report on an indust… ▽ More Regulatory compliance is a well-studied area, including research on how to model, check, analyse, enact, and verify compliance of software. However, while the theoretical body of knowledge is vast, empirical evidence on challenges with regulatory compliance, as faced by industrial practitioners particularly in the Software Engineering domain, is still lacking. In this paper, we report on an industrial case study which aims at providing insights into common practices and challenges with checking and analysing regulatory compliance, and we discuss our insights in direct relation to the state of reported evidence. Our study is performed at Ericsson AB, a large telecommunications company, which must comply to both locally and internationally governing regulatory entities and standards such as GDPR. The main contributions of this work are empirical evidence on challenges experienced by Ericsson that complement the existing body of knowledge on regulatory compliance. △ Less

Submitted 2 March, 2021; originally announced March 2021.

Comments: Full research paper accepted at International Conference on Product-Focused Software Process Improvement 2020

Journal ref: 21st International Conference on Product-Focused Software Process Improvement, PROFES 2020, Turin, Italy, 25 November 2020 through 27 November 2020

Enterprise-Driven Open Source Software: A Case Study on Security Automation

Authors: Florian Angermeir, Markus Voggenreiter, Fabiola Moyón, Daniel Mendez

Abstract: Agile and DevOps are widely adopted by the industry. Hence, integrating security activities with industrial practices, such as continuous integration (CI) pipelines, is necessary to detect security flaws and adhere to regulators' demands early. In this paper, we analyze automated security activities in CI pipelines of enterprise-driven open source software (OSS). This shall allow us, in the long-r… ▽ More Agile and DevOps are widely adopted by the industry. Hence, integrating security activities with industrial practices, such as continuous integration (CI) pipelines, is necessary to detect security flaws and adhere to regulators' demands early. In this paper, we analyze automated security activities in CI pipelines of enterprise-driven open source software (OSS). This shall allow us, in the long-run, to better understand the extent to which security activities are (or should be) part of automated pipelines. In particular, we mine publicly available OSS repositories and survey a sample of project maintainers to better understand the role that security activities and their related tools play in their CI pipelines. To increase transparency and allow other researchers to replicate our study (and to take different perspectives), we further disclose our research artefacts. Our results indicate that security activities in enterprise-driven OSS projects are scarce and protection coverage is rather low. Only 6.83% of the analyzed 8,243 projects apply security automation in their CI pipelines, even though maintainers consider security to be rather important. This alerts industry to keep the focus on vulnerabilities of 3rd Party software and it opens space for other improvements of practice which we outline in this manuscript. △ Less

Submitted 10 February, 2021; originally announced February 2021.

Comments: To be published in: Proceedings of the 43rd International Conference on Software Engineering: Software Engineering in Practice (SEIP)

ACM Class: D.2.0

Is Secure Coding Education in the Industry Needed? An Investigation Through a Large Scale Survey

Authors: Tiago Espinha Gasiba, Ulrike Lechner, Maria Pinto-Albuquerque, Daniel Mendez

Abstract: The Department of Homeland Security in the United States estimates that 90% of software vulnerabilities can be traced back to defects in design and software coding. The financial impact of these vulnerabilities has been shown to exceed 380 million USD in industrial control systems alone. Since software developers write software, they also introduce these vulnerabilities into the source code. Howev… ▽ More The Department of Homeland Security in the United States estimates that 90% of software vulnerabilities can be traced back to defects in design and software coding. The financial impact of these vulnerabilities has been shown to exceed 380 million USD in industrial control systems alone. Since software developers write software, they also introduce these vulnerabilities into the source code. However, secure coding guidelines exist to prevent software developers from writing vulnerable code. This study focuses on the human factor, the software developer, and secure coding, in particular secure coding guidelines. We want to understand the software developers' awareness and compliance to secure coding guidelines and why, if at all, they aren't compliant or aware. We base our results on a large-scale survey on secure coding guidelines, with more than 190 industrial software developers. Our work's main contribution motivates the need to educate industrial software developers on secure coding guidelines, and it gives a list of fifteen actionable items to be used by practitioners in the industry. We also make our raw data openly available for further research. △ Less

Submitted 10 February, 2021; originally announced February 2021.

Comments: Preprint accepted for publication at the 43rd International Conference on Software Engineering

Automatic Detection of Causality in Requirement Artifacts: the CiRA Approach

Authors: Jannik Fischbach, Julian Frattini, Arjen Spaans, Maximilian Kummeth, Andreas Vogelsang, Daniel Mendez, Michael Unterkalmsteiner

Abstract: System behavior is often expressed by causal relations in requirements (e.g., If event 1, then event 2). Automatically extracting this embedded causal knowledge supports not only reasoning about requirements dependencies, but also various automated engineering tasks such as seamless derivation of test cases. However, causality extraction from natural language is still an open research challenge as… ▽ More System behavior is often expressed by causal relations in requirements (e.g., If event 1, then event 2). Automatically extracting this embedded causal knowledge supports not only reasoning about requirements dependencies, but also various automated engineering tasks such as seamless derivation of test cases. However, causality extraction from natural language is still an open research challenge as existing approaches fail to extract causality with reasonable performance. We understand causality extraction from requirements as a two-step problem: First, we need to detect if requirements have causal properties or not. Second, we need to understand and extract their causal relations. At present, though, we lack knowledge about the form and complexity of causality in requirements, which is necessary to develop a suitable approach addressing these two problems. We conduct an exploratory case study with 14,983 sentences from 53 requirements documents originating from 18 different domains and shed light on the form and complexity of causality in requirements. Based on our findings, we develop a tool-supported approach for causality detection (CiRA). This constitutes a first step towards causality extraction from NL requirements. We report on a case study and the resulting tool-supported approach for causality detection in requirements. Our case study corroborates, among other things, that causality is, in fact, a widely used linguistic pattern to describe system behavior, as about a third of the analyzed sentences are causal. We further demonstrate that our tool CiRA achieves a macro-F1 score of 82 % on real word data and that it outperforms related approaches with an average gain of 11.06 % in macro-Recall and 11.43 % in macro-Precision. Finally, we disclose our open data sets as well as our tool to foster the discourse on the automatic detection of causality in the RE community. △ Less

Submitted 26 January, 2021; originally announced January 2021.