Abstract This research paper explores the role of non-cognitive and affective (NCA) factors in influencing student achievement and thriving. The research team has developed and deployed a survey with evidence of validity and reliability to measure 28 NCA factors from n=2339 undergraduates at 17 U.S. institutions nationally. The factors examined include personality, grit, meaning and purpose, engineering identity, mindset, motivation, test anxiety, test and study environment, perceptions of faculty caring, self-control, stress, gratitude, mindfulness, and sense of belonging. While there are a myriad of ways to characterize each student’s NCA profile, a recently completed cluster analysis using Gaussian Mixture Modeling has identified five distinct clusters of students using these NCA factors, which accounted for 50.8% of participants. In summary, the five clusters can be described as (i) the normative cluster, (ii) high positive NCA factors but experiencing stress, (iii) future-oriented but disconnected from engineering, (iv) disengaged from engineering, faculty and peers, and (v) low stress and supported. A preliminary analysis indicated that membership within any of the five clusters was only weakly, if at all, associated with academic performance, as measured by self-reported, overall grade-point-average (GPA). In this study we explore this association in more detailed and nuanced ways to assess whether (a) cluster membership is truly unassociated with academic performance; i.e., students can achieve academically while having various NCA cluster profiles, or (b) one or more clusters is associated with differential academic performance. If the finding is the latter, the results would naturally suggest the need for interventions to support those students whose profiles may predict poor academic outcomes. Finally, we acknowledge that achievement or thriving by undergraduate engineering students cannot be simply measured by the GPA when, obviously, many other factors are at play. This study is necessary, however, since academic performance is currently the predominant measure of progress and achievement in higher education.
In this work in progress research paper, we investigated whether non-cognitive and affective (NCA) factors that predict academic success through GPA may relate to other forms of success such as student retention in engineering programs. Studies show that most students leave engineering within the first two years, making national retention rates in engineering less than 50%. Furthermore, the students who leave engineering are often academically talented, indicating a need to examine other success measures beyond GPA such as non-cognitive and affective (NCA) factors. Using data from a single institution (n = 540), we explore the NCA differences between students who remained in engineering after their first year and those who are no longer enrolled in engineering, or even in college at all. Results show that only one demographic and five NCA measures are statistically significant predictors of continued enrollment. Overall, a better understanding of student success as measured by retention using NCA profiles might assist researchers and practitioners with developing interventions and supportive environments that promote students' academic success and thriving in engineering.
Abstract Each engineering and computing student admitted to a university has clear potential for academic and personal success in their undergraduate curriculum. While some thrive academically, others flounder. Why is it that highly credentialed and previously successful students do not see the same success in college? We posit that some collection of characteristics—apparently not visible on their admission applications and perhaps not related to their talent or intelligence—is an important piece of the student performance puzzle. We developed a survey to measure various non-cognitive and affective factors that we believe are important for student achievement, academically, personally, and professionally. This research examines the validity evidence for our piloted SUCCESS survey (Studying Underlying Characteristics for Computing and Engineering Student Success), which measures latent factors of personality, community, grit, thriving, identity, mindset, motivation, perceptions of faculty caring, stress, gratitude, self-control, mindfulness, and belongingness. These non-cognitive and affective factors are representative of multifaceted aspects of undergraduate student success in prior literature. Each of the constructs we chose had validity evidence from prior studies, some within an engineering population. We piloted the survey across two different universities, one West Coast and one Midwest (n = 490), in Summer 2017. We used Exploratory Factor Analysis (EFA) to evaluate instrument performance to decide which items to include in the national release of the survey in Fall 2017. Our results provide preliminary validity evidence for items that measure various non-cognitive and affective factors. The wide-ranging constructs within the SUCCESS survey provide multiple pathways to understand students’ likelihood for success in engineering and computing. Our future work includes distributing this survey to over a dozen universities across the U.S., yielding a broad dataset of non-cognitive profiles of engineering and computing students broadly. In parallel, we will link these results with students’ registrar information at three study sites to develop predictive models for student success.
Abstract This paper addresses the growing need for a clear definition of ‘thriving’ relevant to engineering students and institutions. This paper was inspired by a research project that examines the impact of non-cognitive factors on engineering student success (NSF #redacted). This project developed a survey to measure several non-cognitive factors using validated instruments reported in the literature. After collecting preliminary data from 490 undergraduate engineering students, exploratory factor analysis (EFA) did not produce a factor structure consistent with previous reports, suggesting a need to develop items with validity evidence for engineering students around these constructs. Given the survey questions on thriving showed strong evidence of internal consistency in a broader higher education population [1], and the unique experiences and curriculum demands of undergraduate engineering students [2], it is unlikely that flaws in the survey questions led to these poor EFA results. It is more likely that current models of thriving, which were not developed specifically for undergraduate engineering students, may not fully apply to this population. Thus, the primary focus on this paper is to engage in a theoretical and practical discussion of how thriving in the engineering context can be understood and conceptualized. The outcome of this paper is to propose a conceptual framework for thriving relevant to engineering students and institutions. This proposed conceptual framework results from a discussion of thriving by connecting topics (such as mindfulness, gratitude, creativity diversity and inclusion) from Engineering Education, Positive Psychology, and other fields. This framework addresses engineering thriving at both the individual and institutional levels. At the individual level, constructs such as mindfulness, creativity, and meaning are discussed to support engineering students and staff to develop a tolerance for ambiguity and find meaning from external events. At the institutional level, broader topics such as engineering culture are discussed to support the wellbeing of the engineering community based on a systems perspective. Overall, focusing on thriving in the engineering context in many ways represents a paradigm shift in engineering education that has great potential to inform new strategies that further improve the way engineering is learned, taught, and practiced. Findings from research on engineering thriving is meant to complement, rather than replace, the traditional engineering education in supporting engineering students’ and institutions’ success. This proposed conceptual framework may serve the engineering education community by providing a first step in understanding and measuring thriving in the engineering context to support more engineering students to thrive through graduation and beyond. References [1] R. Su, L. Tay, and E. Diener, “The Development and Validation of the Comprehensive Inventory of Thriving (CIT) and the Brief Inventory of Thriving (BIT),” Appl. Psychol. Heal. Well-Being, vol. 6, no. 3, pp. 251–279, 2014. [2] C. P. Veenstra, E. L. Dey, and G. D. Herrin, “Is Modeling of Freshman Engineering Success Different from Modeling of Non-Engineering Success?,” J. Eng. Educ., vol. 97, no. 4, pp. 467–479, 2008.
This research-to-practice full paper describes the deliberate and arduous process we recently went through to develop a national survey to study the non-cognitive traits of undergraduate engineering and computing students. The goal of this survey is to characterize student profiles in order to develop and examine particular interventions to guide students toward success in engineering and computing majors. This survey measures non-cognitive attributes including personality, sense of belonging, engineering or computing identity, study skills, well-being, and a variety of other constructs that are not routinely measured in engineering populations nor integrated into admission decisions, advising processes, or academic curricula. Prior research indicates that these non-cognitive attributes are important for students' academic success and retention. However, no studies have examined a comprehensive set of non-cognitive traits holistically to understand how they influence student success. This collaborative project, funded by three linked NSF grants, merges the interests of researchers at three campuses in understanding and supporting students with varied non-cognitive profiles. As part of this research, we negotiated the content of a national survey, suitable for use on our own campuses as well as with other national partners, to probe more than a dozen constructs collectively describing student non-cognitive attributes. The construction of the survey itself was non-trivial, and involved significant negotiations among the researchers including initial collection of instruments with validity evidence to serve as a basis for discussion; an in-person kick-off meeting; multiple follow-up teleconferences; multiple rounds of inclusion/exclusion decisions based upon mutually-agreed upon guiding principles; pilot survey testing; pilot data evaluations such as exploratory factor analysis; and final decisions about instruments/items to include in the final version, all while considering survey length, distribution channels, and key IRB concerns. This paper details the 10-month effort to construct a survey that meets the research needs and intellectual curiosity of partners at three diverse campuses. In this process, we had to balance the different institutional contexts of the funded partner sites while also maintaining flexibility for national distribution. The deliberate processes we used may serve as a template for future survey creation, starting from constructs of interest, to selection of specific instruments (or sub-scales thereof), and factor analysis to consider further down-selection of individual items to include in the final survey. The outcomes of this paper may serve the engineering education community by highlighting previously undocumented processes in collaborative survey construction that introduce intellectual complexity or time delays into the development timeline.
In this work-in-progress research category paper, we explore diversity and inclusion in undergraduate engineering programs by examining racial/ethnic and gender differences in undergraduate engineering students' growth and fixed mindsets. We use multiple regression analyses on survey data to examine the main and interaction effects of race/ethnicity and gender on growth and fixed mindset to answer our overarching research questions: "How do fixed and growth mindset differ by race/ethnicity and gender for undergraduate engineering students?" and "To what extent do undergraduate engineering students' race/ethnicity and gender interact for growth or fixed mindset?" Our results indicate that there are significant racial/ethnic group differences but no gender differences on growth and fixed mindset. Also, we found that gender did not moderate the relationship between race/ethnicity and mindset. Thus, while the men and women in our sample appear to share similar mindsets, most of the participants of different racial/ethnic backgrounds did not share similar mindsets.
Abstract This IUSE (Improving Undergraduate STEM Education) NSF (National Science Foundation) grantee poster describes our work deploying a national survey (the SUCCESS survey—Studying Underlying Characteristics of Computing and Engineering Student Success) to collect data on students’ non-cognitive and affective (NCA) factors. This survey, which is the first of its kind to be launched on a national scale, measures 28 NCA factors that may contribute to student success including personality, grit, identity, mindset, motivation, stress, gratitude, mindfulness, and belongingness. Many engineering and computing students have strong incoming academic records and standardized test scores that indicate potential for success in their programs; nonetheless, many struggle when they reach university. Cognitive measures like SAT/ACT are weak predictors of academic success, and NCA measures may form the constellation of characteristics that offer further predictive power. In this paper, we present construct validity evidence from a confirmatory factor analysis for the SUCCESS survey using a national sample of n = 2672 students, as well as findings from our think-aloud interviews to support face validity. Through confirmatory factor analysis, we removed several items from our survey that did not load onto factors as expected thus improving the measurements and reducing survey length. In addition, the think-aloud interviews allowed us to adjust the wording of questions and to add further demographic options to the survey. Our future work includes using cluster analysis to develop non-cognitive profiles of our participants. We will also use our national dataset to develop predictive models for student success, defined in both academic (e.g., GPA, etc.) and non-academic terms.
Abstract This work-in-progress investigates the impact of a novel course on engineering thriving on twelve undergraduate engineering students' own thriving attitudes. More specifically, we investigate their non-cognitive competencies (including mindfulness, gratitude, identity, and sense of meaning) relevant to student success and psychological well-being. Research findings from positive psychology and related fields suggest that improving students' abilities to thrive improves their academic performance, retention, engagement, and satisfaction. Despite the increasing interest in non-cognitive factors relevant to engineering student thriving, intervention studies of this kind on undergraduate engineering populations are currently lacking. As a first step towards addressing this, we developed a one-credit course for undergraduate engineering students based on research from positive psychology and related fields. This research-based class served the purpose of introducing the concept and language of thriving to undergraduate engineering students to allow them to articulate their own reflections of thriving in the engineering context. We evaluated the impact of this course as an intervention to support engineering students' non-cognitive competencies. Using a qualitative analysis of course documents and survey data, we seek to understand whether providing undergraduate engineering students knowledge and language about thriving affects their own non-cognitive profiles, and further, whether those changes continue to endure six months after completing the course. We examined changes in students' pretest (n = 12), posttest (n = 12), and six-month follow-up (n = 5) scores using a survey that examines the impact of various non-cognitive factors relevant to engineering student success (NSF #redacted). To better understand the observed changes, we also reviewed students' written course reflections, written feedback, and notes from class discussions. Together, our findings indicate that engineering students' non-cognitive competencies are malleable over time, can be taught and learned, and individual non-cognitive competencies should not be researched in isolation. Preliminary findings from survey data showed that the distinct non-cognitive variables we measured changed in similar patterns over time. Preliminary findings from ENGR 396 course documents suggest that distinct non-cognitive competencies are interconnected and function synergistically to impact students. Overall, this study serves as a first step in advancing our knowledge of thriving in the context of undergraduate engineering students. We conclude with a broader discussion of the importance and implications of focusing on positive strengths of undergraduate engineering students, presenting an opportunity to enhance the ways we attract, retain, educate, and graduate engineering students.
'/%3e%3c/defs%3e%3cpath fill='%23012169' d='M0 0h10080v5040H0z'/%3e%3cpath stroke='%23fff' stroke-width='.6' d='m0 0 6 3m0-3L0 3' clip-path='url(%23b)' transform='scale(840)'/%3e%3cpath stroke='%23e4002b' stroke-width='.4' d='m0 0 6 3m0-3L0 3' clip-path='url(%23c)' transform='scale(840)'/%3e%3cpath stroke='%23fff' stroke-width='840' d='M2520 0v2520M0 1260h5040'/%3e%3cpath stroke='%23e4002b' stroke-width='504' d='M2520 0v2520M0 1260h5040'/%3e%3cg fill='%23fff'%3e%3cuse xlink:href='%23d' x='2520' y='3780'/%3e%3cuse xlink:href='%23a' x='7560' y='4200'/%3e%3cuse xlink:href='%23a' x='6300' y='2205'/%3e%3cuse xlink:href='%23a' x='7560' y='840'/%3e%3cuse xlink:href='%23a' x='8680' y='1869'/%3e%3cuse xlink:href='%23e' x='8064' y='2730'/%3e%3c/g%3e%3c/svg%3e)