Model-based testing (MBT) has been previously used to validate embedded systems. However, (i) creation of a model conforming to the behavioural aspects of an embedded system, (ii) generation of executable test scripts and (iii) assessment of test verdict, re-quires a systematic process. In this paper, we have presented a three-phase tool-supported MBT workflow for the testing of an embedded system, that spans from requirements specification to test verdict assessment. The workflow starts with a simplistic, yet practical, application of a Domain-Specific Language (DSL) based on Gherkin-like style, which allows the requirements engineer to specify requirements and to extract information about model elements(i.e. states and transitions). This is done to assist the graphical modelling of the complete system under test (SUT). Later stages of the workflow generates an executable test script that runs on a domain-specific simulation platform. We have evaluated this tool-supported workflow by specifying the requirements, extracting information from the DSL and developing a model of a subsystem of the train control management system developed at Alstom Transport AB in Sweden. The C# test script generated from the SUT model is successfully executed at the Software-in-the-Loop (SIL) execution platform and test verdicts are visualized as a sequence of passed and failed test steps.
Testing is an important activity in engineering of industrial software. For such software, testing is usually performed manually by handcrafting test suites based on specific design techniques and domain-specific experience. To support developers in testing, different approaches for producing good test suites have been proposed. In the last couple of years combinatorial testing has been explored with the goal of automatically combining the input values of the software based on a certain strategy. Pairwise testing is a combinatorial technique used to generate test suites by varying the values of each pair of input parameters to a system until all possible combinations of those parameters are created. There is some evidence suggesting that these kinds of techniques are efficient and relatively good at detecting software faults. Unfortunately, there is little experimental evidence on the comparison of these combinatorial testing techniques with, what is perceived as, rigorous manually handcrafted testing. In this study we compare pairwise test suites with test suites created manually by engineers for 45 industrial programs. The test suites were evaluated in terms of fault detection, code coverage and number of tests. The results of this study show that pairwise testing, while useful for achieving high code coverage and fault detection for the majority of the programs, is almost as effective in terms of fault detection as manual testing. The results also suggest that pairwise testing is just as good as manual testing at fault detection for 64% of the programs.
In the context of building software systems, Service-oriented Systems (SOS) have become one of the major research topics in the past few years. In SOS, services are basic functional units that can be created, invoked, composed, and if needed deleted on-the-fly. Since these software systems are composed of different services there is no easy way to assure the Quality of Service (QoS), therefore, formal specification of both functional and extra-functional system behaviour, compatibility, and interoperability between different services have become important issues. As a way to address this issues, resource-aware timing behavioural language REMES was chosen to be extended towards service-oriented paradigm with service specific information, such as type, capacity, time-to-serve, etc., as well as Boolean predicate constraints on control flow guarantees. In this thesis we present a design framework that provides a graphical user interface for behaviour modelling of services based on REMES language. NetBeans Visual Library API is used to display editable service diagrams with support for graph-oriented models. A textual dynamic service composition language was implemented, together with means to automatically verify service composition correctness. We ensure also an automated traceability between service specification interfaces, where both modelling levels are combined in an efficient tool for designing SOS.
Numerous industrial sectors employ Programmable Logic Controllers (PLC) software to control safety-critical systems. These systems necessitate extensive testing and stringent coverage measurements, which can be facilitated by automated test-generation techniques. Existing such techniques have not been applied to PLC programs, and therefore do not directly support the latter regarding automated test-case generation. To address this deficit, in this work, we introduce PyLC, a tool designed to automate the conversion of PLC programs to Python code, assisted by an existing test generator called Pynguin. Our framework is capable of handling PLC programs written in the Function Block Diagram language. To demonstrate its capabilities, we employ PyLC to transform safety-critical programs from industry and illustrate how our approach can facilitate the manual and automatic creation of tests. Our study highlights the efficacy of leveraging Python as an intermediary language to bridge the gap between PLC development tools, Python-based unit testing, and automated test generation.
Model-based testing (MBT) is a test design technique that supports the automation of software testing processes and generates test artefacts based on a system model representing behavioural aspects of the system under test (SUT). Previous research has shown some positive aspects of MBT such as low-cost test case generation and fault detection effectiveness. However, it is still a challenge for both practitioners and researchers to evaluate MBT tools and techniques in real, industrial settings. Consequently, the empirical evidence regarding the mainstream use, including the modelling and test case generation using MBT tools, is limited. In this paper, we report the results of a case study on applying GraphWalker, an open-source tool for MBT, on an industrial cyber-physical system (i.e., a Train Control Management System developed by Bombardier Transportation in Sweden), from modelling of real-world requirements and test specifications to test case generation. We evaluate the models of the SUT for completeness and representativeness, compare MBT with manual test cases written by practitioners using multiple attributes as well as share our experiences of selecting and using GraphWalker for industrial application. The results show that a model of the SUT created using both requirements and test specifications provides better understanding of the SUT from testers' perspective, making it more complete and representative than the model created based only on the requirements specification alone. The generated model-based test cases are longer in terms of the number of test steps, achieve better edge coverage and can cover requirements more frequently in different orders while achieving the same level of requirements coverage as manually created test cases.
Since the early days of software testing, automatic test generation has been suggested as a way of allowing tests to be created at a lower cost. However, industrially useful and applicable tools fo ...
As systems continue to increase in complexity, some companies have turned to Model-Based Systems Engineering (MBSE) to address different challenges such as requirement complexity, consistency, traceability, and quality assurance during system development.Consequently, to foster the adoption of MBSE, practitioners need to understand what factors are impeding or promoting success in applying such a method in their existing processes and infrastructure.While many of the existing studies on the adoption of MBSE in specific contexts focus on its applicability, it is unclear what attributes foster a successful adoption of MBSE and what targets the companies are setting.Consequently, practitioners need to understand what adoption strategies are applicable.To shed more light on this topic, we conducted semi-structured interviews with 12 professionals working in the vehicular domain with roles in several MBSE adoption projects.The aim is to investigate their experiences, reasons, targets, and promoting and impeding factors.The obtained data was synthesized using thematic analysis.This study suggests that the reasons for MBSE adoption relate to two main themes: better management of complex engineering tasks and communication between different actors.Furthermore, engagement, activeness and access to expert knowledge are indicated as factors promoting MBSE adoption success, while the lack of MBSE knowledge is an impeding factor for successful adoption.
The Unified Modeling Language (UML) is a well known approach for specifying and designing software components. UML for hardware designs of embedded systems is also possible in the simulation process, when the hardware is in the software form. The large number of tools for UML, the generl adoption of this technology for heterogeneous system design and verification, makes UML a very powerful and robust design instrument. Based on UML, the SysML [1] language has been developed in order to support all the details of system designs. SysML extends UML towards the systems engineering domain. As a good example, a SysML model for hardware components that perform error detection and correction, based on polynomial registers mod p(x), will be presented. The approach is justified as efficient and flexible.
This position paper presents and illustrates the concept of security requirements as code – a novel approach to security requirements specification. The aspiration to minimize code duplication and maximize its reuse has always been driving the evolution of software development approaches. Object-Oriented programming (OOP) takes these approaches to the state in which the resulting code conceptually maps to the problem that the code is supposed to solve. People nowadays start learning to program in the primary school. On the other hand, requirements engineers still heavily rely on natural language based techniques to specify requirements. The key idea of this paper is: artifacts produced by the requirements process should be treated as input to the regular object-oriented analysis. Therefore, the contribution of this paper is the presentation of the major concepts for the security requirements as the code method that is illustrated with a real industry example from the VeriDevOps project.