Groups for which it is easy to detect graphical regular representations

We say that a finite group G is DRR-detecting if, for every subset S of G, either the Cayley digraph Cay(G,S) is a digraphical regular representation (that is, its automorphism group acts regularly on its vertex set) or there is a nontrivial group automorphism φ of G such that φ(S) = S. We show that every nilpotent DRR-detecting group is a p-group, but that the wreath product Zp wr Zp is not DRR-detecting, for every odd prime p. We also show that if G1 and G2 are nontrivial groups that admit a digraphical regular representation and either gcd(|G1|, |G2|) = 1, or G2 is not DRR-detecting, then the direct product G1 x G2 is not DRR-detecting. Some of these results also have analogues for graphical regular representations.