Characterization of Instructional Practices and Assessment Selection in Undergraduate STEM Education

Abstract

Accurate identification of instructional practices is important for improving undergraduate STEM education, yet commonly used classroom observation tools have not been systematically validated for distinguishing specific practices beyond broad categories. This dissertation investigates the capabilities and limitations of the Classroom Observation Protocol for Undergraduate STEM (COPUS) and explores how chemistry faculty select assessment instruments for teaching and research purposes. The first study (Chapter 2) examined whether COPUS reliably captures instructional practices in undergraduate STEM classrooms. A conceptual framework mapped documented practices to expected COPUS codes, highlighting which practices share observable behaviors and what COPUS observations should theoretically reveal. Validation analyses compared COPUS observations of 16 courses with instructor interviews, analyzed through the lenses of Pedagogical Content Knowledge and Content Knowledge frameworks. Results showed that COPUS accurately captures general instructional structures but performs poorly in identifying specific practices, with technology use and pedagogical reasoning largely invisible to behavioral observation. The second study (Chapter 3) used COPUS to characterize current teaching styles across North American undergraduate STEM courses. Three instructional styles were identified from the analyses: Didactic, Interactive Lecture, and Student-Centered styles. Comparisons with earlier research revealed differences in the distribution of instructional styles such as a decrease in Didactic instruction and an increase in Student-Centered approaches. However, these differences likely reflect characteristics of the sample used rather than genuine shifts in teaching practices over time. The third study (Chapter 4) explored how chemistry faculty select assessment instruments through semi-structured interviews. Analysis revealed ten influential factors, with alignment to desired content as the primary criterion. While validity and reliability were noted, many faculty lacked the assessment literacy to interpret psychometric evidence and relied on author reputation, publication venue, or expert recommendations. Professional identity influenced the prioritization of factors and standards applied in research versus teaching contexts. The three studies together clarify COPUS’s strengths and limitations for classroom observation, document contemporary teaching practices, and illuminate faculty decision-making in assessment selection. The findings inform the appropriate use of observation protocols, suggest improvements for assessment resources, and reveal the factors shaping faculty assessment decisions in both instructional and research settings.