Assessing Online Learning Experiences

Assessment comes in a variety of forms. Each of these forms has its own purpose.  In general, a good way to think about assessment is in the following three ways:

• Assessment for learning (diagnostic assessment)
• Assessment as learning (formative assessment)
• Assessment of learning (summative assessment)

Each of these basic types of assessment can be accomplished within the Marquette D2L Learning Management System (LMS) by utilizing a variety of flexible built-in tools.

Assessment for learning is often referred to as diagnostic assessment. Diagnostic assessment most often occurs before the bulk of the learning happens in the classroom and can be thought of the origin point of learning. Anytime we teach a class, we generally have expectations for the knowledge, skills and experiences that students bring with them. Often the department defines “prerequisite” skills or courses that students need to have in order to be successful. These prerequisites represent an “ideal” level of knowledge, however. In reality, students vary in their actual prerequisite skills. It is easy to image two students writing skills being quite different even though they may both have completed the same prerequisite writing course for example. The best way to ensure their success in our course is to quickly evaluate their current knowledge of a particular topic.

Assessment as learning, or formative assessment, is where the bulk of student learning happens in an educational experience. Formative assessments are designed around course learning objectives and provide students feedback related to the progress that they have made in achieving course outcomes as well as information on how they can improve their performance in the future. 

Summative assessments are often what students think of when they hear the word assessment. They are relatively “high stakes” evaluations of students’ skill.  Traditionally, summative assessment traditionally make up the bulk of a student’s grade for a course, but may not be indicative of the learning that has taken place.  They also provide the student little information on where they can improve. As summative assessment occurs at the end of the learning experience, there is likely opportunity to improve even when quality feedback is provided. Typically, summative assessments are used as credentialing tools to measure a students’ competence in an area at the conclusion of an instructional period.

Just as teaching online is different than teaching face-to-face, online learning is different than learning in a classroom. Because of the geographical and temporal separation of teacher and student, the monitoring of learning progress shifts from the instructor to the student. As a result, students in online classes often need to be more autonomous than those in face-to-face classrooms. While the instructor cannot control the autonomy of their students, they can create assessment scaffolds that help students moderate their own learning. The following is a list of practices that can help in this process:

1. Reevaluate whether assessments reinforce learning goals and objectives at both the course and modular level. If an assessment or an assessment item does not evaluate or reinforce a learning goal, neither eliminate it or consider modifying the goals to include this content. 
2. Consider using diagnostic assessment early in the course or a particular lesson to ensure that your assumptions of students’ prior knowledge is accurate. Be sure to communicate early with students’ that have knowledge gaps that would threaten their success in the course.
3. Diagnostic assessment can sometimes be effective at the end of a lesson to provide information about how well content was absorbed by students. Consider using short quizzes or surveys to assess student understanding. These types of assessment can be graded or ungraded (even anonymous). They can be used to assess if students got out of a lecture, video, or assignment what you expected them to. 
4. Consider using more frequent, shorter, and lower stakes assignments and assessments as your main evaluative strategy for students. This ensures that students have ample time to practice skills and learn from errors before the higher-stakes cumulative assessment. Students will appreciate this in that they will have a better idea of what you expect them to be able to do. You will appreciate this as it will increase the quality of that final paper or culminating assignment.
5. Consider varying your method of assessment. Every assessment type has a cost and a benefit – multiple choice questions are easy to grade but provide little insight into students’ conceptual understandings... Asynchronous discussion forums provide insight into student thinking, but can be difficult to facilitate and ensure all students are participating... – Combine different types of assignments that elicit different levels of student thinking in order to gather a complete picture of what your students know and are able to do.
6. Give serious consideration to alternative assessment methods during this transition given the complicated home environments of many of our students.

Whichever way you choose to evaluate, it is best to have your plan laid out ahead of the semester and all expectations clearly stated for students. All assessments should be built ahead of the start of the course and mechanisms for communicating feedback to students should be clearly established in the syllabus. Establishing criteria “on-the-fly” or changing criteria mid-course can be a recipe for disaster in an online course. You are best advised to choose your strategy ahead of time with clear expectations and then use formative and summative data to evaluate the effectiveness of your assessment plan after the semester ends.

Assessing student learning in a large class is a challenge, and there is no “magic bullet” technological solution. Here are a couple of strategies you can use to mitigate the grading burden of teaching large classes.

• Rubrics! Rubrics are a great way to provide specific feedback to students quickly and accurately. 
• Encourage self-assessment of skills. This does not have to mean students give themselves a grade.  Requiring short, reflective writing prompts for students is a good way to gauge their understanding.
• Utilize peer assessment. This can be particularly effective strategy when anticipating a paper, presentation, performance assessment, etc. Peer assessment allows student to get a sense of the quality of their work in relation to a peer and it improves the overall quality of the final project because it has been reviewed by someone other than the author.
• Provide automated feedback in the D2L quiz tool. You can set targeted feedback to be given to students that answer incorrectly in the D2L quiz tool. For example, if a student answers “b” on a multiple-choice question, you can set the system to give feedback explaining why this is not the correct answer or where they might be able to find the correct answer.
• Allow multiple takes. Used in conjunction with automated feedback, you can give students a chance to learn from their incorrect answers to D2L quiz questions.
• Spot-check with students. One of the disadvantages of multiple choice and true/false questions is that you can’t get a sense of student thinking. Tell students that you will randomly select a set number of students after a quiz to provide further information about their thinking (“Why did you select a particular answer?”). This can cut back on guessing as well as deter academic dishonesty.
• Rely on your experience with the content and students! Research in the Learning Sciences shows that students acquire content knowledge along certain learning trajectories. That is to say, content that students have found difficult in the past will be difficult for students in the future. Identify the most common misconceptions of students and use these “sticking points” to define learning objectives and guide the design of your assessments. 

D2L offers a variety of tools to help you orchestrate learning experiences and facilitate the dissemination of feedback to students. There is no “one-size-fits-all” tool that will be appropriate for every learning assessment you use so you should become familiar with all of them. Together these tools can offer rich assessment options that should suit many of your needs as an instructor. The following is a list of these tools and some suggestions for how to use them:

• The Survey Tool – This tool is perfect for a variety of informal assessments. The tool itself works very much like the quiz tool in D2L. You can utilize a variety of question types (multiple choice, true/false, short answer, essay, Liket, etc.) to collect valuable formative information as well as perform quick assessments of student understandings. Survey data can be disaggregated a number of ways and can provide a nice snapshot of student learning in your online course.
• The Dropbox Tool – The Dropbox tool is a place where students can submit different types of files or text (a Word document, PowerPoint, .jpeg, etc.) The dropbox is specifically useful with assessments that involve the creation of some sort of “product” (term paper, info graphic, data collection, etc.). You can disseminate the activity description, collect student work and assess the assignment all within an individual dropbox. The dropbox tool also has seamless integration with the rubrics tool as well as the gradebook. 
  

  If students work in groups, you may use a Dropbox where they can submit the assessment. Online Group Checklist

  Students with accommodations through the Office of Disability Services (ODS) may request additional time on quizzes. Instructors can provide this access through the Special Access feature.

• Discussion Tool – The discussion tool can be used in a number of different situations. The key to successful use of the discussion tool is when the intent of the activity is foster peer-to-peer interaction. The name of the tool is a little misleading as this tool facilitates lots of different types of interaction. Not just discussion interaction.
  • Video - Formative Assessments in D2L Discussion Forums
• The Quiz Tool – Like the Survey tool, the quiz tool is a good tool to conduct quick diagnostic or formative assessment. The quiz tool is also effective for assessments that have are “time critical”. The quiz tool can be set so that students have a finite amount of time to complete a task. This can be an attractive feature for faculty that are concerned with issues of academic dishonesty.  Remember, however that constraints to time are not the ONLY way to discourage cheating. When using this tool, it is advisable to use time constraints strategically, and only when time constraints make sense in relation to the learning outcomes.
  • D2L Quiz FAQs
  • Video - D2L Quizzes: The Many Joys and Uses
• The Rubrics Tool – The rubric tool is a powerful way to give relatively directed feedback to large numbers of students efficiently. Anyone that has experience teaching knows that students make similar errors. It can be tedious to correct the same mistake over and over again. The rubric tool allows you to leverage your experience with students’ misconceptions to provide targeted feedback to students. Rubrics can be built in a variety of ways and can range from general (holistic) feedback to quite specific (analytic) feedback. The effort put into thinking through a rubric ahead of an assignment will pay off in terms of grading after the fact. Research has also shown that the quality of student products (relative to instructor expectations) is better when a rubric accompanies the assignment.  This also makes the grading process quicker and more satisfying for instructors.
  • Video - Creating and Using Rubrics in D2L
• The News Tool and Group Emails – Group feedback can also be helpful. Like rubrics, the D2L News tool and the Email tool are good ways to summarize class misconceptions without having to write the same feedback over and over. Use these tools strategically to cut down on repeating yourself to multiple students.
  • Video - Using the D2L News Item Function for Consistent Communication with Students
• The Gradebook Tool – The gradebook tool is the heartbeat of any course. A fully integrated and completed gradebook can make grading tasks much easier for students and can help online students monitor their progress in the course.
• Additional D2L Video Tutorials

Note: Be wary of using outside tools to assess. While you may be “more familiar” with them, students are likely to have more experience with university sanctioned tools. Using outside tools (like Google Doc, etc.) can open you up to serious FERPA violations.

All students, whether taking courses online, or in a classroom, are expected to adhere to the Academic Integrity Standards set forth by Marquette University, as well as any additional procedures provided by the instructor in the course syllabus. The Fostering Academic Integrity in Online Learning webpage provides strategies and tools to preserve the integrity of online assessments.

For Professors who need a Student resource to link Students to for course projects, the Student Media Creation webpage will have media creation guides and directions on how students should upload their videos to MS Stream, change permissions, and share their media content for any media related assignment you may supply. Please feel free to use the site as a resource to link in your D2L course page.