Model Evasion Attacks Against Partially Encrypted Deep Neural Networks in Isolated Execution Environment

It is important to hide DNN models from adversaries not only for protecting intellectual property but also for preventing attacks. Isolated execution environments (IEEs) are necessary to protect the DNN model in an edge device. Conventional studies on DNN inference in IEEs have focused on model confidentiality and not the threat of model evasion attacks. If some of the model parameters or intermediate results are leaked to adversaries, model evasion attacks may occur even if the confidentiality of the model is secured. In this work, we performed attacks against partially encrypted DNN models that are executed on IEEs. In an existing proposal, a feature extractor of the model is executed in a normal world and a classifier is executed in a secure enclave, but there is still a threat that an adversary may perform a gradient-based model evasion attack against a feature extractor. We performed gradient-based model evasion attacks against the feature extractor more efficiently by preparing multiple guide images. Our results clarified that all parameters on the feature map should be kept secret by the parameter encryption. In addition, we consider another risk case where calculated values on the feature extractor are stored in the unencrypted memory and demonstrated the gradient estimation-based model evasion attack by exploiting the intermediate feature maps. Our results indicate that both DNN model parameters and intermediate feature maps should be concealed not only for protecting intellectual property but also for preventing model evasion attacks.