Brown, Corina E.
Mosher, Michael D.
Hyslop, Richard M.
Paek, Sue Hyeon
College of Natural and Health Sciences; Department of Chemistry and Biochemistry, Chemical Education
University of Northern Colorado
Type of Resources
Place of Publication
University of Northern Colorado
This three-phase study was conducted to design, develop, and psychometrically analyze the Reaction Mechanisms Concept Proficiency Inventory (RMCPI). In the first phase, open-ended interviews were conducted with organic chemistry instructors (N=11) to obtain their opinion on concepts pertinent to developing proficiency in organic reaction mechanisms. The results yielded 11 pertinent concepts. Additionally, participants believed that organic reaction mechanisms are essential for mastering organic chemistry and students have difficulty with understanding the meaning and utility of the curved-arrows. The difficulties that students have with reaction mechanisms could be due to a lack of understanding of fundamental general chemistry and organic chemistry concepts. The second phase of this study consisted of a national survey of organic chemistry instructors (N=183) to generalize the results obtained from the first phase. Organic chemistry instructors were asked to rate the importance of the concepts identified in phase 1. The results indicated a general consensus at the national level regarding these concepts and a list of 10 concepts was obtained. Additionally, there was a consensus at the national level regarding the importance of organic reaction mechanisms and the difficulties students face with this important area of organic chemistry education. The third phase of this study consisted of the development of the inventory, test administration, and psychometric analysis. During the development of the inventory, open-ended questions under each of the pertinent concepts identified were administered to first-semester organic chemistry students (N=138) to obtain distractors from their alternate conceptions. Open-ended interviews were conducted with first-semester organic chemistry students (N=22) to obtain information on their thought process while answering the questions and identify additional alternate conceptions. These alternate conceptions were used as distractors for the two-tier items in the inventory. A pilot version, and a beta version of the inventory were administered to 109 and 359 first-semester organic chemistry students respectively. The 26-item alpha version of the RMCPI was administered to 753 first-semester organic chemistry students from 14 different universities across the U.S.A. At the item level, Classical Test Theory and Rasch analysis were used to assess item functioning. At the test level, face validity, content validity, and construct validity using Rasch analysis were utilized to establish the validity of the data obtained using the RMCPI. The reliability of the data obtained using the RMCPI was assessed by computing the Cronbach’s alpha value, and the item and person separation reliability. The results indicate that the items on the alpha version of the RMCPI are functioning well and the instrument is measuring a unidimensional construct which suggests that the RMCPI could be used by organic chemistry instructors as an effective assessment tool to detect students’ alternate conceptions on concepts pertinent to developing proficiency in organic reaction mechanisms. Additionally, the validity and reliability of the data meet the acceptable standards for a concept inventory.
Copyright is held by the author.