Moodle 1.9 Testing and Assessment — Save 50%
Develop and evaluate quizzes and tests using Moodle modules
Using computers to test is becoming commonplace. Computers are everywhere and they are able to speed up and do so many things that they have become necessary in many fields, and testing is no exception. The Moodle team has put together a variety of activities that allow us to create and deliver a variety of tests with ease, many of them graded automatically and with instant feedback, making Moodle Quiz a very useful tool. In this article we will take a look at creating a full quiz in the Quiz module. We will learn how to create a complete quiz using the Quiz module, as well as use some new options.
In this article by Jason Myrick, author of Moodle 1.9 Testing and Assessment, we will:
- Create a complete test
- Use a variety of options we have not used before
|Read more about this book|
(For more resources on Moodle, see here.)
To start with, we need to select a topic or theme for our test. We are going to choose general science, since the subject matter will be easy to incorporate each of the item types we have seen previously.
Now that we have an idea of what our topic is going to be, we will get started in the creation of the test. We will be creating all new questions for this test, which will give us the added benefit of a bit more practice in item creation. So, let's get started and work on making our first real test!
Let's open our Moodle course, go to the Activity drop-down, and select Create a new Quiz. Once it has been selected, we will be taken to the Quiz creation page and we'll be looking at the General section.
The General section
Here need to give the test a name that describes what the test is going to cover. Let's call it 'General Science Final Exam' as it describes what we will be doing in the test.
The introduction is also important.this is a test students will take and an effective description of what they will be doing is an important point for them. It helps get their minds thinking about the topic at hand, which can help them prepare, and a person who is prepared can usually perform better. For our introduction, we will write the following, 'This test will see how much you learned in our science class this term. The test will cover all the topics we have studied, including, geology, chemistry, biology, and physics. In this test, there are a variety of question types (True/False, Matching, and others). Please look carefully at the sample questions before you move on. If you have any questions during the test, raise your hand. You will have 'x' attempts with the quiz.
We have now given the test an effective name and we have given the students a description of what the test will cover. This will be shown in the Info tab to all the students before they take the test, and if we want in the days running up to the test. That's all we need to do in this section.
In this section, we need to make some decisions about when we are going to give the test to the students. We will also need to make a decision about how long we will give the students to complete the test. These are important decisions, and we need to make sure we give our students enough time to complete the test. The default Timing section is shown in the next screenshot:
We probably know when our final exam will be. So, when we are creating the test, we can set the date that the test will be available to the students and the date it will stop being accessible to them. Because this is our final exam, we only want it to be available for one day, for a specified time period.
We will start by clicking on the Disable checkboxes next to Open the Quiz and Close the Quiz dates. This step will enable the date/time drop-down menus and allow us to set them for the test. For us, our test will start on March 20, 2010 at 16:55 p.m. and it will end the same day, one hour later. So we will change the appropriate menus to reflect our needs. If these dates are not set, a student in the course will be able to take the quiz any time after you finish creating it.
We will need to give the students time to get in class, settle down, and have their computers ready. However, we also need to make sure the students finish the test in our class, so we have decided to create a time limit of 45 minutes. This means that the test will be open for one hour, and in that one hour time frame, once they start the test, they will have 45 minutes to finish it. To do this, we need to click on the Enable checkbox next to the Time Limit (minutes) textbox. Clicking on this will enable the textbox, and in it we will enter 45. This value will limit the quiz time to 45 minutes, and will show a floating, count-down timer in the test, causing it to auto-submit 45 minutes after it is started. It is good to note that many students get annoyed by the floating timer and its placement on the screen. The other alternative is to have the test proctor have the students submit the quiz at a specified time.
Now, we have decided to give a 45 minute time limit on the test, but without any open-ended questions, the test is highly unlikely to take that long. There is also going to be a big difference in the speed at which different students work. The test proctor should explain to the students how much time they should spend on each question and reviewing their answers.
Under the Time Limit (minutes) we see the Time delay between first and second attempt and Time delay between later attempts menus. If we are going to offer the test more than once, we can set these, which would force the students to wait until they could try again. The time delays range from 30 minutes to 7 days, and the None setting will not require any waiting between attempts on the quiz. We are going to leave these set to None because this is a final exam and we are only giving it once.
Once all the information has been entered into the Timing section, this dialog box is what we have, as shown in the next screenshot:
Here, we will make some decisions about the way the quiz will look to the students. We will be dividing questions over several pages, which we will use to create divisions in the test. We will also be making decisions about the shuffle questions and shuffle within questions here.
Firstly, as the test creators, we should already have a rough idea of how many questions we are going to have on the test. Looking at the Questions Per Page drop-down menu, we have the option of 1 to 50 questions per page. We have decided that we will be displaying six questions per page on the test. Actually, we will only have five questions the students will answer, but we also want to include a description and a sample question for the students to see how the questions look and how to answer them' thus we will have six on each page.
We have the option to shuffle questions within pages and within questions. By default, Shuffle Questions is set to No and Shuffle within Questions is set to Yes. We have decided that we want to have our questions shuffled. But wait, we can't because we are using Description questions to give examples, and if we chose shuffle, these examples would not be where they need to be. So, we will leave the Shuffle Questions setting at the default No. However, we do want to shuffle the responses within the question, which will give each student a slightly different test using the same questions and answers.
When the display settings are finished, we can see the output shown in the next screenshot:
In this section, we will be setting the number of attempts possible and how further attempts are dealt with. We will also make a decision about the Adaptive Mode.
Looking at the Attempts allowed drop-down menu, we have the option to set the number from 1 to 10 or we can set it to Unlimited attempts. For our test, we have already decided to set the value to 1 attempt, so we will select 1 from the drop-down menu.
We have the option of setting the Each Attempt Builds on the Last drop-down menu to Yes or No. This feature does nothing now, because we have only set the test to have a single attempt. If we had decided to allow multiple attempts, a Yes setting would have shown the test taker all the previous answers, as if the student were taking the test again, as well as indicating whether he or she were correct or not. If we were giving our students multiple attempts on the test, but we did not want them to see their previous answers, we would set this to No.
We are also going to be setting Adaptive mode to No. We do not want our students to be able to immediately see or correct their responses during the test; we want the students to review their answers before submitting anything.
However, if we did want the students to check their answers and correct any mistakes during the test, we would set the Attempts Allowed to a number above 1 and the Adaptive Mode to Yes, which would give us the small Submit button where the students would check and correct any mistakes after each question. If multiple attempts are not allowed, the Submit button will be just that, a button to submit your answer.
Here is what the Attempts section looks like after we have set our choices:
In this section, we will set the way Moodle will score the student. We see three choices in this section, Grading method, Apply penalties, and Decimal digits in grades; however, because we have only selected a single attempt, two of these options will not be used.
Grading Method allows us to determine which of the scores we want to give our student after multiple tries. We have four options here: Highest Grade, Average Grade, First Attempt, and Last Attempt. Highest Grade uses the highest grade achieved from any attempt on any individual question. The Average Grade will take the total number of tries and grades and average them. The First Attempt will use the grade from the first attempt and the Last Attempt will use the grade from the final attempt. Since we are only giving one try on our test, this setting has no function and we will leave it set at its default, Highest Grade, because either option would give the same result.
Apply penalties is similar to Grading method, in that it does not function because we have turned off Adaptive Mode. If we had set Adaptive Mode to Yes, then this feature would give us the option of applying penalties, which are set in the individual question setup pages. If we were using Adaptive Mode and this option feature set to No, then there would be no penalties for mistakes as in previous attempts. If it were set to Yes, the penalty amount decided on in the question would be subtracted for each incorrect response from the total points available on the question. However, our test is not set to Adaptive Mode, so we will leave it at the default setting, Yes. It is important to note here that no matter how often a student is penalized for an incorrect response, their grade will never go below zero.
The Decimal digits in grades shows the final grade the student receives with the number of decimal places selected here. There are four choices available in this setting: 0, 1, 2, and 3. If, for example, the number is set to 1, the student will receive a score calculated to 1 decimal place, and the same follows for 2 and 3. If the number is set to 0, the final score will be rounded. We will set our Decimal digits in grades to 0.
After we have finished, the Grades section appears as shown in the next screenshot:
This sectopm is where we set when and what our students will see when they look back at the test. There are three categories: Immediately after the attempt; Later, while quiz is still open; and After the quiz is closed.
The first category, Immediately after the attempt, will allow students to see whatever feedback we have selected to display immediately after they click on the Submit all and finish button at the end of the test, or Submit, in the case of Adaptive mode. The second category, Later, while quiz is still open, allows students to view the selected review options any time after the test is finished, that is, when no more attempts are left, but before the test closes. Using the After the quiz is closed setting will allow the student to see the review options after the test closes, meaning that students are no longer able to access the test because a close date was set. The After the quiz is closed option is only useful if a time has been set for the test to close, otherwise the review never happens because the test doesn't ever close.
Each of these three categories contains the same review options: Responses, Answers, Feedback, General feedback, Scores, and Overall feedback.Here is what these options do:
- Responses are the student's response to the question and whether he or she were wrong or correct.
- Answers are the correct response to the question.
- Feedback is the feedback you enter based on the answer the student gives. This feedback is different from the General quiz feedback they may receive.
- General feedback are the comments all students receive, regardless of their answers.
- Scores are the scores the student received on the questions.
- Overall feedback are the comments based on the overall grade on the test.
We want to give our students all of this information, so they can look it over and find out where they made their mistakes, but we don't want someone who finishes early to have access to all the correct answers. So, we are going to eliminate all feedback on the test until after it closes. That way there is no possibility for the students to see the answers while other students might still be taking the test. To do remove such feedback, we simply unclick all the options available in the categories we don't want. Here is what we have when we are finished:
Regardless of the options and categories we select in the Review options, students will always be able to see their overall scores. Looking at our settings, the only thing a student will be able to view immediately after the test is complete is the score. Only after the test closes, will the student be able to see the full range of review material we will be providing.
If we had allowed multiple attempts, we would want to have different settings. So, instead of After the quiz is closed, we would want to set our Review options to Immediately after the attempt, because this setting would let the student know where he or she had problems and which areas of the quiz need to be focussed on.
One final point here is that even a single checkbox in any of the categories will allow the student to open and view the test, giving the selected review information to the student. This option may or may not be what you want. Be careful to ensure that you have only selected the options and categories you want to use.
This section is where we can increase quiz security, but it is important to note that these settings will not eliminate the ability of tech-savvy students to cheat. What this section does is provide a few options that make cheating a bit more difficult to do. We have three options in this section: Browser security, Require password, and Require network address.
The Require password does exactly what you think it would. It requires the students to enter a password before taking the test. To keep all your material secure, I recommend using a password for all quizzes that you create. This setting is especially important if you are offering different versions of the quiz to different classes or different tests in the same class and you want to make sure only those who should be accessing the quiz can. There is also an Unmask checkbox next to the password textbox. This option will show you the password, just in case you forget!
Finally, we have the Require network address option, which will only allow those at certain IP Addresses to access the test. These settings can be useful to ensure that only students in the lab or classroom are taking the test. This setting allows you to enter either complete IP Addresses (for example. 123.456.78.9), which require that specific address to begin the test; partial IP Addresses (for example 123.456), which will accept any address as long as it begins with the address prefixes; and what is known as Classless Inter-Domain Routing (CIDR) notation, (for example 123.456.78.9/10), which only allows specific subnets. You might want to consult with your network administrator if you want to use this security option.
By combining these settings, we can attempt to cut down on cheating and improper access to our test. In our case here, we are only going to use the fullscreen option. We will be giving the test in our classroom, using our computers, so there is no need to turn on the IP Address function or require a password. When we have finished, the Security section appears as shown in the next screenshot:
eBook Price: $26.99
Book Price: $44.99
|Read more about this book|
(For more resources on Moodle, see here.)
Common module settings
In this section, we have four options available to us: Group mode, Visible, ID number, and Grade category.
The Group mode doesn't affect the students, it is only for the teacher. When looking at the test results, the teacher will see them in smaller groups instead of as a whole. In the Group mode drop-down menu, there are three options. The first is the default setting, None. This setting means that the students taking the test are all part of the same group. The second option is Separate groups, which means that the students are part of their own group and take the test in that group. Visible groups means that students work in their own groups, but they are able to see the work of other groups. This setting, however, does not mean that they will see the other student's answers.
Using the Visible feature is identical to the Open/Closed eye in the course. This function will make the test visible or invisible to the students. It is a good idea to hide the quiz while you are developing it, just in case a student accidentally accesses it or takes it before you are ready. Note that invisible quizzes do not show up on the event calendar.
The ID number function allows you to set an ID for the quiz, which will help identify it for grading. You can develop an ID system for your course that will help you identify types of activities, quizzes, and others. For example, you could set up an ID system with the hundred's placeholder acting as an ID for quizzes, activities, and others. The ten's placeholder might act as a week identifier. The one's placeholder might be the numerical order of the assignments. So, using our idea, let's say we decide to use 300 as an identifier for quizzes. We are in our fourth week of class. This is the second quiz we have given that week. In our ID system this quiz would be 342, so that is what we would place in the ID Number textbox.
The Grade category function is used if you have set up grading categories in Gradebook. The default setting is Uncategorised, which will not place the results into any preset category. If you have created categories in Gradebook, you will see them here and you can select the appropriate category for the test results to be included in.
For our test, we will be leaving the Group set at None; changing Visible to Hide; setting ID number to 1; and the Grade category will be set to Uncategorised, because we have not yet created any. Here is what the Common module settings looks like when we finish. Some users may have a Show Advanced button in this field that will allow them to restrict the quiz to particular groups.
In this section, we can give our students feedback based on the results of their test. These fields are set up with what is called a Grade boundary. A Grade boundary will give the students the feedback based on the range of scores in the boundary.
When setting up the Grade boundary, we can enter either numbers or percentages with a preset maximum grade of 100% and a preset minimum grade of 0%. By default, there are seven Grade boundaries, but we can set as many as we'd like, by clicking on Add 3 more feedback fields. Any fields that we do not want to use, we leave blank. Here is what we will enter as Overall feedback for our test:
As you can see, Grade boundary 100% has been given the Feedback Excellent!. The next Grade boundary is 90%. This value means that a student who scores between 100% and 90% will receive 'Excellent!' as their Overall feedback. A student scoring 90% to 80% will receive Good job!; 80% to 70% will receive OK!; and 70% or lower will receive You need to review the material!.
Saving the test
Now that we have entered in all of our requirements for the test, we need to save it. We have the option to either Save and return to course, Save and display, or Cancel. We will click on Save and display and we will be directed to the Add questions to Quiz screen, where we will begin making the questions for our first complete test in Moodle Quiz.
Designing the test
Now that we have the test parameters worked out, we need to start organizing the actual questions for the test. When designing a test, it is a good idea to give students an introduction to the test, clear instructions, and examples of how the questions work. It is also a good idea to give some easier questions at the beginning, to build up the students confidence.
For our test, we will be using all the types of questions we have worked with before. We will create six questions for each page, with each page containing a single question type. We want to give a Description question first, to explain what the section is about and we also want to give a sample of the question type.
On the first page of the test, we will use a Description question to give the students an overview of the test. On this page of the test, we will tell the students what types of questions to expect, how many questions they will have, the time limit, when they can check their results and see our feedback, and what to do if they have any questions.
Once we have created the text in the Description question, we Save and Preview it to make sure it looks like we want it to. The first page of the test is as follows:
Save without Submitting
This caution cannot be repeated enough! You can include it in your quiz introduction or verbally, but make sure your students are clicking on the Save without Submitting buttons every 5 to 10 minutes. If anything happens during the test, such as power loss or Internet failure, all their work will be lost unless they save their work. You must stress the need to do this to your students!
Now the real fun begins! We will start to create our questions for the test. On the second page, we will be asking True/False questions about Space.
To begin with, we will create a Description question explaining the section and the types of questions to expect. In the description, we will include a sample image of how the True/False questions look and some information about the section. The example we will show them is a screenshot of a True/False question that I have already prepared and uploaded to the image folder. Once we have finished writing the description of the section, we will use the Add Image to Display drop-down and select the already prepared image. Here is what the section introduction for True/False looks like when we've finished:
Reading this section, the student quickly sees how many items they will have to answer, what the topic covered is, the type of items they will be working with, and a sample of exactly what they will be looking at. This is the kind of information we need to supply before each section in order to prepare them as well as we can for what they need to do.
Note that I have added some bold underlined text at the top called Instructions. I have also placed all the text in Bold. This emphasis helps the students realize that the text is not a question and is something that they should pay attention to.
Now all that remains is to create the five items associated with the Instructions. We will create the five True/False questions we need to complete the section.
For the remaining sections, we will do the same thing as we have done previously: create a Section introduction and the five items to be used in the section.
Adding to the Quiz
Once all the questions have been created and are part of the Question Bank, we will move them to the test. Once we have added all the items to the test, including the Introductions, we will go to the bottom of the Questions in this Quiz section of the Editing Quiz page and we see two options, Show page breaks and Show the reordering tool. We are going to click on the Show page breaks checkbox.
Showing page breaks will give us a visual representation of where each page begins and ends. It will also give us the Up or Down Arrow next to the page break lines, which enables us to move the page breaks to wherever we want. There is also a delete option, the X, in case we want to get rid of a page break. We also have the option to add more page breaks by using the Repaginate with 'X' questions per page, located at the bottom left-hand side of the page. If we decide that we want to add more pages, or we have deleted some page breaks and want them back, just select the number of items per page you want, click on the Go button, and the pages will be added to the quiz depending on the settings you select. This is not the easiest interface to work with, and you have no way to simply add a single page break. If you need to add another page break, you will need to repaginate and get rid of page breaks you don't need. What we have now is shown in the next screenshot:
As you can see, we have a clear visual representation of how the test will look and how it is organized. If we want to adjust any questions, we can use the Move arrows to place them where we want. Once we are happy with the order, we will click on the Save changes button.
The second option, Show the reordering tool, functions in a similar manner. It places a numerical textbox next to the questions starting with '10' and increasing by multiples of 10 for each item included on the test. The 10 represents the item's place and where it will be shown in the test. The items increase by 10s and you are able to insert items between them by changing the numbers and putting everything in the numerical order you want the questions displayed on the test, for example, 10, 11, 12, 20, 30, 31, and so on .Once you have the numbers in the order you want them to be displayed, click on the Save changes button and the new order will be displayed. The ordering numbers will change back to multiples of 10, allowing you to reorder the questions again if needed.
You do not have to use integers and you have the option to use decimal places should you desire. Page breaks are also included in the numbering system. If you have only checked the Repaginate button, you will not see the Page Breaks, but you will see missing numbers in the sequence. If you decide to use the Show the reordering tool, I would recommend selecting both options for visual clarity.
Previewing the test
Now we will look at the test to see what we have. The first thing we notice is that all the questions are shuffled. This is not what we want. We need to have the descriptions first, to explain to the students what they need to do. We have two options for what to do here.
The first option is simple. We can Update Quiz and turn off the Shuffle function. If we do this, the questions will be placed exactly as they appear in the Questions in this quiz area. This option is easy and is the fastest way to deal with this issue. The drawback is that every student will have the test given in the exact same order, which may increase cheating. On the upside, the responses inside the answers are shuffled, so some variation is still included in the test.
First, we will turn off the shuffle question order feature in the Quiz setup page. Then, we will go to the Edit tab above the Question Bank and click on the Categories link directly below the Edit tab. Once inside Edit Categories, we will see a list of all the categories related to this Quiz that we have so far.
We will scroll down to the bottom and we see the Add category area. In the Parent drop-down menu, we are going to select Default for General Science Test because we want the create a subcategory for this quiz. For Name, we will enter General Science ExamTrue/False Introduction. We will enter the same information for Category Info. Once these steps are complete, we will click on the Add category button, located below the Category Info textbox. The new category will appear as a subcategory in the Default for General Science Test category. We will do this for each of the other question types we have.
Now that we have created our categories, we will add our questions to their appropriate categories. Once the questions have been added to their categories, we need to go to the Category drop-down menu just below the Question Bank header and select the category we want to use. We will begin by selecting T/F Questions about Space. This will bring us to the questions we just added to the category. Now, we want to add these questions to the exam in a random order. We go to the bottom of the Quiz Bank and we can see Add 'X' random questions with an Add button on the right-hand side. We select the number of items we want to move, click on the Add button, and that number of randomly selected items from the category selected will be placed in the test. We will manually move them to where they need to be and repeat the process for each of the other categories we will be using. When we finish adding the random questions we see this screen in the Quiz:
What this quiz will do is select five random items from the related category. However, if there are only five items in the category, the items will always be in the same order. You should include at least one additional item to allow for true variation in the questions. While more options for variation are probably better, one thing to consider is the item quality. Are all the items in the category of equal quality or difficulty? In case you are reviewing items and you find some are significantly more or less difficult, you may want to consider removing them to improve the test's validity.
Once you have organized the test and it looks the way you want it to, click on the Save changes button. Then click on the Preview button and take the test, looking for any errors or issues with format or display, until you are happy with the results. If you made any adjustments, don't forget to save again.
As we have seen, Quiz provides an effective, fairly easy-to-use way of testing students. Tests created and delivered via Quiz have the added benefit of being easy to create and they can be used over and over again. Quiz also has the ability to do all kinds of things that most instructors need, including: setting test dates and times; altering page length; including a variety of item formats; shuffling items and responses; setting security levels; creating test groups; giving a variety of feedback and results; and even providing basic data analysis tools. Using Quiz, we are able to design and alter tests quickly and easily if required.
- What's New in Moodle 2.0 [Article]
- New Modules for Moodle 2 [Article]
- Securing a Moodle Instance [Article]
- Moodle: Authentication Methods [Article]
- Moodle 2.0: What's New in Add a Resource [Article]
eBook Price: $26.99
Book Price: $44.99
About the Author :
Jason is interested in testing and assessments and computer delivery methods for testing. He has spent many hours playing with Moodle and teaching colleagues how to deliver tests with it. He decided that instead of a piecemeal approach, he would write a book that covers the basics of how to use Moodle to deliver tests and assignments for assessment.
Aside from working, he likes scuba diving, cooking, and good beer! He is currently developing a research proposal for a PhD in testing focused on computerized delivery methods and validity.