Effective test with Studyx: quick start, maximum result

Author - Maxim Prikhodko, founder of Studyx, Ph.D., research and development director of Studyx, an expert in data analysis and adaptive learning with more than 10 years of experience in the field of e-Learning and the introduction of the most advanced educational technologies.

In order to achieve the maximum results from a knowledge control using a test, you must correctly prepare the test questions.

Even if you plan to ask your students only 2 questions in the test, it is better to prepare a set of 6-10 questions. In this case, you will be able to more broadly and more thoroughly check the knowledge of students, protect yourself from the students' mechanical remembering of the correct answers in the test during its retake, and the students themselves will have an opportunity to learn something new and strengthen their knowledge during testing.

Below you will find not only technical recommendations for completing the test template itself, but also methodical recommendations for compiling a questions bank which will help you to make full use of testing as a tool for knowledge control.

The current version of the test template can be downloaded from the following link:

Studyx test template

The size of the questions bank

The questions bank is a set of all questions developed for the test, some of which will then be included in the test task, formed individually for each testing participant.

The recommended size of the questions bank is determined by the following formula:

The size of The questions bank = (number of questions in a topic) * (number of topics / thematic blocks of the test).

For example, if the test questions are divided into 3 topics with 15 questions each, then the recommended size of the questions bank is 45 questions.

The minimum number of questions in each topic is determined by the general parameters of testing, namely - the number of questions from the topic that are selected during testing, and the level of variability:

Number of questions in the topic = (number of questions from the topic in the test) * (level of variability),

where the minimum value of the level of variability is 3, the recommended value is 5.

For example, if 5 questions are selected from each topic during the testing, and the level of variability is chosen equal to 3, then the total number of questions in the topic will be 3 * 5 = 15.

Questions difficulty levels

Questions in The questions bank can be ranked by difficulty levels - one for all topics. This is not an obligatory procedure, however, the definition of the difficulty levels makes it possible to further use this bank of questions for carrying out not only classical, but also adaptive testing.

The recommended number of difficulty levels is at least 3 (better - from 5 to 10). Within each topic (thematic block) it is recommended to have questions of different difficulty levels, as this directly affects the variability of the test and the ability to adapt the test to the personal characteristics of the test participants.

Thus, when compiling The questions bank it is necessary to take into account that the questions within each topic should represent at least 3-5 different levels of complexity (by default, the difficulty levels increase from "simple to complex", for example from 1 to 10, where 1 represents the simplest questions, 10 - the most complex).

Test duration

Test duration is determined by the total number of questions a testing participant is being asked. The optimum test duration is calculated using the following formula:

Total number of questions in the test = (number of questions asked from a topic) * (number of topics / thematic blocks of the test).

In the final test it is recommended to ask a participant at least 5 questions for each topic. Then, if the test questions are divided into 3 topics, the recommended test duration will be 15 questions.

The requirement is not mandatory, however, if the duration of the test is shortened, the system may not have enough data to sufficiently reliably determine the level of knowledge of individual topics.

Question types

The following four main types of questions are possible:

with the choice of one correct answer;

Fig. 1. An example of a question with the choice of one variant of the correct answer

with the choice of several correct answers (multiple choice) (multiple choice);

Fig. 2. An example of a question with a choice of several variants of the correct answer

with text response input;

Fig. 3. An example of a question with text response input

for comparison / ordering.

Fig. 4. An example of a question for comparison / ordering

Important!

When you import a test template in Excel format, the question type is determined automatically based on the analysis of acceptable response options. Be careful with the indication of the correctness of the answer.

General recommendations for the test questions preparation

  • There can be any number of correct answers in a question from 1 to N, where N is the total number of options for answering this question.
  • All answers can also be correct at the same time. I.e. the question may not contain any wrong answer at all.
  • If a question contains exactly one answer and this answer is correct, then this question is treated as a question with test response input.
  • If a question contains more than one answer and exactly one correct answer, then this question is treated as a question with the choice of one variant of the correct answer.
  • If a question contains more than one correct answer, then this question is treated as a question with a choice of several variants of the correct answer.
  • The questions with filled "Correspondence" column are treated as the questions for comparison / ordering.

Important!

Due to the automated nature of checking the answers to questions, it is necessary to keep the following condition: each question must have at least one option of the correct answer.

Filling the test template in Excel format

An example of a correctly filled template can be downloaded from the following link (for convenience, the blocks of the lines of each question are highlighted in their own color):

an example of a completed Studyx test template

Below are the main recommendations for filling out the template fields:

Fig. 5. Appearance and structure of the test template

  • Topic title ("TOPIC_TITLE" column)
    Here you must specify the title of the topic (thematic block) to which the question belongs.
  • Lesson title ("LESSON_TITLE" column)
    Here you must specify the title of the lesson within the topic to which the question belongs.
  • The text of the question ("QUESTION_TEXT" column)
    Here you must specify the text of the question. It is also recommended that, together with the correct answers, the text of the question be an affirmative sentence.
  • Question with a picture ("PIC_QUESTION" colummn)
    Fill this column in the case if the question contains an image or is completely presented not in a text format, but in the form of an image. Specify here only the name of the graphic file, without additional directories, etc. All graphic images for uploading should be presented in one common directory together with the filled template.
  • CORRECT ANSWER ("CORRECT" column)
    The mark 1 is placed only opposite the correct answer options (the answer options text is the next "ANSWER_TEXT" column). Opposite the wrong options you should place 0.
  • The text of the answer ("ANSWER_TEXT" column)
    Here you must specify the text of the answer option. Each answer option is indicated on a separate line.
  • Correspondence ("CORRESPONDENCE" column)
    This column is only filled for matching / ordering questions. In the column "CORRESPONDENCE" the second - "right" - part of the answer (the correct pair) is indicated. The first - "left" - part of the answer (the correct pair) is indicated in the column "ANSWER_TEXT".
  • Explanation of the correct answer ("CORRECT_EXPLANATION" column)
    Here you must specify a short explanation of the correct answer, which can be automatically shown to the student in the test results, if an incorrect or incomplete answer was given. The explanation of the correct answer is indicated once in the same line, where the text of the question is indicated.
  • Source ("SOURCE" column)
    Here you must specify a reference to additional information / literature, to which the student can apply for a more in-depth study of the issue. The link is indicated once in the same line, where the text of the question is indicated.
  • Difficulty ("DIFFICULTY" column)
    Here you must specify the level of difficulty of the question in the chosen scale (from 1 to 10, for example, where 1 is for the simplest questions and 10 is for the most complex - see "Distribution of questions by difficulty levels" for details). The question's difficulty level is indicated once in the same line, where the text of the question is indicated.

Important!

  • The topic title in the column «TOPIC_TITLE» is indicated 1 time when the questions of a new topic begin and it is not repeated for new lessons, questions or answers in this topic.
  • The lesson title in the column «LESSON_TITLE» is indicated 1 time when the questions of the new lesson begin and it is not repeated for new questions and answers in this lesson.
  • The first lesson is indicated in the same line as the title of the topic.
  • The text of the question in the column «QUESTION_TEXT» is indicated 1 time when a new question begins and it is not repeated for the answers in this question.
  • The first question of the lesson is indicated in the same line where the name of the lesson is indicated.
  • The first answer option to the question is indicated in the same line where the text of the question is indicated.
  • Important! Due to the automated nature of answer options verifying it is necessary to keep the following condition: not only the pair as a whole, but individually, each value in the column "ANSWER_TEXT" and "CORRESPONDENCE" should be unique. It is not allowed to use the same value of the answer or corresponding option in two or more pairs.

Distribution of questions by difficulty levels

There are several options for the distribution of questions by difficulty levels:

  • Uniform distribution
    Questions are distributed uniformly along difficulty levels, while it is recommended to have at least 10 questions of each difficulty level.
  • "Pyramid" distribution
    It is possible to correct the number of questions in the form of a "pyramid", when the number of more complex questions issomewhat less than the number of simple ones (because complex questions are much less frequent in the test). In this case, the following ratio is recommended: at least 6 questions of the highest difficulty levels (9-10), at least 8 questions of average difficulty levels (7-8) and at least 10 questions of lower difficulty levels (1-6).
  • Automatic distribution
    It is possible to automatically distribute questions along difficulty levels. For this, initial testing (on the sample group of participants) is conducted, and questions difficulty levels are determined in accordance with the responses of the participants. Questions that get most of the right answers are considered to be the simplest, and questions with the fewest correct answers are the most difficult.

Working with errors

As the completed template is imported by automated tools, no semantic errors or typos will be found.

The errors that occur when the template is imported are reported to the customer, they are recommended to be fixed at the level of the template, and not in the database, in order to avoid logical errors.

Typos and other minor inaccuracies found after successful import are fixed at the database level.

Comments (0)