Formative Assessments: Paper and Pencil vs. Digital Medium 

Formative assessment is an integral part of instruction because it allows teachers to gather information on student understanding and use this information to adjust teaching and learning activities (Black & Wiliam, 1998). Formative assessment is critical in mathematics education, where the content is a blend of concepts and skills. With the advancement of technology, teachers have a choice of implementing formative assessment with paper and pencil or with a digital format. It is imperative to understand if the medium used for a formative assessment impacts student performance, as decisions about levels of student support and interventions often rely on formative assessment data (Hensley et al., 2017). When students perform at the best of their abilities on formative assessments, it allows teachers to dial in on remaining misconceptions and adjust their instruction accordingly to improve student learning.

During the Covid-19 pandemic, I found formative assessment more critical than ever as I taught in-person and remote students simultaneously. I gained a lot of experience with instructional technology, which made me curious about how the medium of a formative assessment impacts my students. In the spring of 2022, I implemented an action research project with three sections of Accelerated Precalculus. My research questions were as follows: 1. Will my students perform better on a formative assessment with a paper/pencil or digital medium? 2. Do my students prefer one medium over the other? 3. If so, why?

Methods

I conducted my research during an instructional unit on counting problems and probability. The instruments I developed were two eight-question formative assessments and a seven-question survey. I wrote each assessment in Microsoft Word and then duplicated each assessment in a digital format using formative.com. This platform allows for multiple choice and open response questions and can automatically grade the assessments. Formative Assessment #1 asked students to classify if a scenario represented a combination or a permutation and to solve problems using combinations and permutations. Formative Assessment #2 required students to calculate probabilities of compound events. I created a survey in Microsoft forms. I used a Likert scale to measure if the formative assessments helped students identify gaps in their understanding, learn the material, and if each medium accurately measured understanding. The survey also asked whether students preferred a particular medium and why.

I planned to administer each formative assessment a week apart and the survey two days after the second formative assessment. My participants were 74 students enrolled in Accelerated Precalculus: 29 in first period, 22 in third period, and 23 in seventh period. The first-period students took Formative Assessment #1 on paper and Formative Assessment #2 using the digital medium. The third and seventh-period students took Formative Assessment #1 using the digital medium and Formative Assessment #2 on paper.

Data

            After administering each formative assessment, I graded them and compiled the results in a spreadsheet. For each student, I recorded which of the eight questions were answered correctly and determined the class averages (Figure 1 and Figure 2). After the second formative assessment, I entered the scores for each student side by side so I could compare the results (Figure 3). Finally, I exported the survey results to a spreadsheet to gather both quantitative and qualitative data on my students’ opinions and preferences.

Figure 3: Data by Student

Data Analysis

            For both formative assessments, the group that used a paper medium performed better on average than the group that completed the formative assessment using a digital medium. When I analyzed the results question by question, the lowest scoring questions were consistent across all classes. Of the 59 students who took both formative assessments, 29 (49%) scored above the class average on both, 17 (29%) scored above average only on the paper assessment, 5 (8%) scored above average only on the digital assessment, and 8 (14%) scored below average on both assessments.

The survey results indicated that 89% agreed or strongly agreed that the formative assessments helped them identify gaps in their understanding, while the other 11% were neutral. 90% of my students agreed or strongly agreed that the formative assessment they took using a paper medium accurately measured their understanding at the moment, compared to 65% for the digital medium. 78% answered that paper and pencil formative assessments better measured their understanding than digital formative assessments, and 22% responded that both mediums measure their understanding with the same accuracy. No students answered that the digital medium better measured their understanding. 90% of my students responded that they prefer a paper medium for formative assessments, 3% prefer a digital medium, and 3% had no preference. Many of the students who answered that they prefer a paper medium mentioned that it gives them a more accessible opportunity to write out their work, focus more, and remember the material better when they work on paper. Students that preferred a digital medium mentioned that they appreciated that their answers were scored immediately and they did not have to wait for feedback.

Conclusion

Through my action research, I found that my students performed slightly better on a formative assessment with a paper/pencil medium, though there were exceptions. The vast majority of my students prefer a paper medium because of the ease of writing out their work. Some students also prefer paper because they find it easier to focus than using a screen. Some students felt that they could better remember what they learned from the formative assessment experience when they wrote it down. The students who preferred a digital medium appreciated the immediacy of the feedback, so timely feedback is something I will prioritize when administering formative assessments.

In the future, I would be interested in repeating this research in a different mathematical context, perhaps in a unit with graphs or multi-step equations, to discover if the results are more pronounced. A limitation of this research was student absences. Of my 74 Accelerated Precalculus students, only 59 took both assessments. Still, my findings have led me to see the importance of continuing to implement formative assessments in my classroom. With my group of students, I have found that the value of paper outweighs the convenience of a digital medium.

References

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5, 7–74.

Hensley, K., Rankin, A., & Hosp, J. (2017). Comparing student performance on paper- and computer-based math curriculum-based measures. Assistive Technology, 29(3), 140–145. https://doi.org/10.1080/10400435.2016.1212129

Find a printable PDF version of this article here.