Don't Judge a Letter by its Title: Linguistic Analysis of Letters of Recommendation by Author's Academic Rank.

OBJECTIVE This study analyzed the linguistic differences in letters of recommendation (LORs) for general surgery residency applicants written by authors of various academic ranks. Given that many general surgery residency programs require a LOR from the Chair of surgery, this study also examined whether LORs written by the Chair demonstrate linguistic differences to support this practice. DESIGN A single institution, retrospective review analyzed LORs from two application cycles of general surgery residency applicants who were selected for interview at a large academic institution. Word count (WC) and linguistic characteristics of LORs were analyzed with a previously developed institution-specific dictionary using the Linguistic Inquiry and Word Count software (LIWC2015; Pennebaker Conglomerates, Inc., Austin, Texas). WC and linguistic characteristics of LORs reported as frequencies of terms within twenty-four categories were examined based on the letter authors' academic rank. Further examination compared LORs written by a Chair of surgery with those written by non-Chairs. SETTING A single large, Midwestern academic general surgery residency program PARTICIPANTS: Four hundred and sixty-five letters of recommendation received during two interview cycles were included for analysis. RESULTS A total of 465 LORs written by assistant (n = 82), associate (n = 94), and full professors (n = 289) were included in the study. No statistically significant difference was noted in the WC of LORs based on the letter writers' academic ranks (p = 0.95). Assistant professors utilized grindstone, communal, and technical skill terms with higher frequencies compared to associate professors and full professors. LORs written by assistant professors demonstrated the highest authentic variable score followed by associate professors then full professors (4.94, 3.92, 3.28, p < 0.01). LORs written by Chairs (n = 128) had lower authentic variable scores compared to LORs written by non-Chairs (n = 337; 2.71 vs. 3.91, p = 0.001). Only 50 (39%) LORs written by Chairs indicated working directly with the applicant, and sub-group analysis demonstrated a higher authentic variable score in this group compared with LORs written by Chairs who did not indicate having worked directly with the applicant (3.51 vs. 2.5, p = 0.01). CONCLUSIONS Linguistic analysis of LORs for general surgery residency applicants demonstrated minor yet statistically significant differences based on the author's academic rank. If applicants can obtain linguistically similar LORs from surgeons of any academic rank, but less authentic LORs from writers with higher academic ranks, these LORs may be less valuable for the residency programs when evaluating applicants. Based on the subgroup analysis, less than 40% of Chair LORs indicated that the Chair worked directly with the applicant, calling into question the utility of the Chair LORs as meaningful evaluation of applicants. Further study to compare LORs of applicants selected and not selected for interview may add additional insight into linguistic differences in LORs written by authors of different academic ranks.