Vector and Matrix Differentiation

Frequently, we need to differentiate quantities like tr(AX) with respect to the elements of X, or quantities like Ax, z′Ax with respect to the elements of (the vectors) x and/or z.