Evaluating student understanding of core concepts in computer architecture
Porter L.; Garcia S.; Tseng H.-W.; Zingaro D.
2013
Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE
23
10.1145/2462476.2462490
Many studies have demonstrated that students tend to learn less than instructors expect in CS1. In light of these studies, a natural question is: to what extent do these results hold for subsequent, upper-division computer science courses? In this paper we describe our work in creating high-level concept questions for an upper-division computer architecture course. The questions were designed and agreed upon by subject-matter and teaching experts to measure desired minimum proficiency of students post-course. These questions were administered to four separate computer architecture courses at two different institutions: a large public university and a small liberal arts college. Our results show that students in these courses were indeed not learning as much as the instructors expected, performing poorly overall: the per-question average was only 56%, with many questions showing no statistically significant improvement from pre-course to post-course. While these results follow the trend from CS1 courses, they are still somewhat surprising given that the courses studied were taught using research-based pedagogy that is known to be effective across the CS curriculum. We discuss implications of our findings and offer possible future directions of this work. Copyright 2013 ACM.
Assessment; Computer architecture; Curriculum
Asanovic K., Bodik R., Catanzaro B.C., Gebis J.J., Husbands P., Keutzer K., Patterson D.A., Plishker W.L., Shalf J., Williams S.W., Yelick K.A., The landscape of parallel computing research: A view from Berkeley, Technical Report Technical Report No. UCB/EECS-2006-183, (2006); Crouch C.H., Watkins J., Fagen A.P., Mazur E., Peer instruction: Engaging students one-on-one, all at once, Research-Based Reform of University Physics. American Association of Physics Teachers, (2007); Goldman K., Gross P., Heeren C., Herman G.L., Kaczmarczyk L., Loui M.C., Zilles C., Setting the scope of concept inventories for introductory computing subjects, Trans. Comput. Educ., 10, 2, pp. 1-29, (2010); Herman G.L., Loui M.C., Zilles C., Creating the digital logic concept inventory, Proc. of the 41st SIGCSE, (2010); Lister R., Adams E.S., Fitzgerald S., Fone W., Hamer J., Lindholm M., McCartney R., Mostrom J.E., Sanders K., Seppala O., Simon B., Thomas L., A multi-national study of reading and tracing skills in novice programmers, ITiCSE-WGR '04: Working Group Reports from ITiCSE, (2004); Patterson D.A., Hennessy J.L., Computer Organization and Design: The Hardware/software Interface, (2008); Porter L., Bailey-Lee C., Simon B., Zingaro D., Peer instruction: Do students really learn from peer discussion in computing?, Proc. of the 7th ICER, (2011); Robins A., Haden P., Garner S., Problem distributions in a CS1 course, Proc. of the 8th ACE, (2006); Schulte C., Bennedsen J., What do teachers teach in introductory programming?, Proc of the 2nd ICER, (2006); Sheard J., Carbone A., Lister R., Simon B., Thompson E., Whalley J.L., Going solo to assess novice programmers, Proc. of the 13th ITiCSE, (2008); Sutter H., The free lunch is over: A fundamental turn toward concurrency in software, Dr. Dobb's Journal, 30, (2005); Tew A.E., Guzdial M., Developing a validated assessment of fundamental CS1 concepts, Proc. of the 41st SIGCSE, (2010); Tew A.E., Guzdial M., The fcs1: A language independent assessment of cs1 knowledge, Proc. of the 42nd SIGCSE, (2011)
Conference paper
All Open Access; Green Open Access
Scopus