Evaluation of the Pattern-based method for Secure Development (PbSD): A controlled experiment

Context: Security in general, and database protection from unauthorized access in particular, are crucial for organizations. Although it has been long accepted that the important system requirements should be considered from the early stages of the development process, non-functional requirements such as security tend to get neglected or dealt with only at later stages of the development process. Objective: We present an empirical study conducted to evaluate a Pattern-based method for Secure Development - PbSD - that aims to help developers, in particular database designers, to design database schemata that comply with the organizational security policies regarding authorization, from the early stages of development. The method provides a complete framework to guide, enforce and verify the correct implementation of security policies within a system design, and eventually generate a database schema from that design. Method: The PbSD method was evaluated in comparison with a popular existing method that directly specifies the security requirements in SQL and Oracle's VPD. The two methods were compared with respect to the quality of the created access control specifications, the time it takes to complete the specification, and the perceived quality of the methods. Results: We found that the quality of the access control specifications using the PbSD method for secure development were better with respect to privileges granted in the table, column and row granularity levels. Moreover, subjects who used the PbSD method completed the specification task in less time compared to subjects who used SQL. Finally, the subjects perceived the PbSD method clearer and more easy to use. Conclusion: The pattern-based method for secure development can enhance the quality of security specification of databases, and decrease the software development time and cost. The results of the experiment may also indicate that the use of patterns in general has similar benefits; yet this requires further examinations.