Numerous industrial sectors employ Programmable Logic Controllers (PLC) software to control safety-critical systems. These systems necessitate extensive testing and stringent coverage measurements, which can be facilitated by automated test-generation techniques. Existing such techniques have not been applied to PLC programs, and therefore do not directly support the latter regarding automated test-case generation. To address this deficit, in this work, we introduce PyLC, a tool designed to automate the conversion of PLC programs to Python code, assisted by an existing test generator called Pynguin. Our framework is capable of handling PLC programs written in the Function Block Diagram language. To demonstrate its capabilities, we employ PyLC to transform safety-critical programs from industry and illustrate how our approach can facilitate the manual and automatic creation of tests. Our study highlights the efficacy of leveraging Python as an intermediary language to bridge the gap between PLC development tools, Python-based unit testing, and automated test generation.
Many industrial application domains utilize safety-critical systems to implement Programmable Logic Controllers (PLCs) software. These systems typically require a high degree of testing and stringent coverage measurements that can be supported by state-of-the-art automated test generation techniques. However, their limited application to PLCs and corresponding development environments can impact the use of automated test generation. Thus, it is necessary to tailor and validate automated test generation techniques against relevant PLC tools and industrial systems to efficiently understand how to use them in practice. In this paper, we present a framework called PyLC, which handles PLC programs written in the Function Block Diagram and Structured Text languages such that programs can be transformed into Python. To this end, we use PyLC to transform industrial safety-critical programs, showing how our approach can be applied to manually and automatically create tests in the CODESYS development environment. We use behaviour-based, translation rules-based, and coverage-generated tests to validate the PyLC process. Our work shows that the transformation into Python can help bridge the gap between the PLC development tools, Python-based unit testing, and test generation.
Programmable Logic Controllers are computer devices often used in industrial control systems as primary components that provide operational control and monitoring. The software running on these controllers is usually programmed in an Integrated Development Environment using a graphical or textual language defined in the IEC 61131-3 standard. Although traditionally, engineers have tested programmable logic controllers' software manually, test automation is being adopted during development in various compliant development environments. However, recent studies indicate that choosing a suitable test automation framework is not trivial and hinders industrial applicability. In this paper, we tackle the problem of choosing a test automation framework for testing programmable logic controllers, by focusing on the COntroller DEvelopment SYStem (CODESYS) development environment. CODESYS is deemed popular for device-independent programming according to IEC 61131-3. We explore the CODESYS-supported test automation frameworks through a grey literature review and identify the essential criteria for choosing such a test automation framework. We validate these criteria with an industry practitioner and compare the resulting test automation frameworks in an industrial case study. Next, we summarize the steps for selecting a test automation framework and the identification of 29 different criteria for test automation framework evaluation. This study shows that CODESYS Test Manager and CoUnit are mentioned the most in the grey literature review results. The industrial case study aims to increase the know-how in automated testing of programmable logic controllers and help other researchers and practitioners identify the right framework for test automation in an industrial context.
Regulatory standards for engineering safety-critical systems often demand both traceable requirements and specification-based testing, during development. Requirements are often written in natural language, yet for specification purposes, this may be supplemented by formal or semi-formal descriptions, to increase clarity. However, the choice of notation of the latter is often constrained by the training, skills, and preferences of the designers.The Easy Approach to Requirements Syntax (EARS) addresses the inherent imprecision of natural language requirements with respect to potential ambiguity and lack of accuracy. This paper investigates requirement formalization using EARS and specification-based testing of embedded software written in the IEC 61131-3 language, a programming standard used for developing Programmable Logic Controllers (PLC). Further, we investigate, by means of an experiment, how human participants translate natural language requirements into EARS and how they use the latter to test PLC software. We report our observations during the experiments, including the type of EARS patterns participants use to structure natural language requirements and challenges during the specification phase, as well as present the results of testing based on EARS-formalized requirements.
'%3e%3cpath fill='%23012169' d='M0 0v30h60V0z'/%3e%3cpath stroke='%23fff' stroke-width='6' d='m0 0 60 30m0-30L0 30'/%3e%3cpath stroke='%23C8102E' stroke-width='4' d='m0 0 60 30m0-30L0 30' clip-path='url(%23b)'/%3e%3cpath stroke='%23fff' stroke-width='10' d='M30 0v30M0 15h60'/%3e%3cpath stroke='%23C8102E' stroke-width='6' d='M30 0v30M0 15h60'/%3e%3c/g%3e%3c/svg%3e)
'/%3e%3cpath fill='%23fff' d='m8.751-17.959 10.11 11.373L3.997-9.844l13.94-6.1-7.692 13.129z'/%3e%3c/svg%3e)
