Magnetically induced elastic deformations in model systems of magnetic gels and elastomers containing particles of mixed size