ChoiceTextResponse: a submission that omits a required numtolerance_input key is graded correct

Đang mở
#306 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
3/5
Thời gian dự kiến
1-2 ngày
Mức phù hợp với người mới
65/100
Loại issue
Lỗi
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
python

Hướng nghiên cứu

The bug is in xblocks_contrib/problem/capa/responsetypes.py, specifically in ChoiceTextResponse._check_student_inputs. Start by reading the existing tests in the same file (TEST_INPUTS, TEST_SCENARIOS, TEST_PROBLEM_ARGS) to understand the grading logic. Then, modify the method to iterate over self.correct_inputs first, ensuring missing keys set inputs_correct to False. Run the existing test suite to verify no regressions, and add a new test case for omitted keys as described in the issue.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

xblocks_contrib/problem/capa/responsetypes.py in openedx/xblocks-core (formerly openedx/xblocks-contrib; f8b51bf, current main), ChoiceTextResponse.get_score (3672-3696) and _check_student_inputs (3789-3843). openedx-platform's xmodule/capa_block.py imports this module (from xblocks_contrib.problem.capa import responsetypes, line 46).

# get_score, 3686-3693
answer_dict = student_answers.get(self.answer_id, "")
binary_choices, numtolerance_inputs = self._split_answers_dict(answer_dict)
choices_correct = self._check_student_choices(binary_choices)
inputs_correct = self._check_student_inputs(numtolerance_inputs)
correct = choices_correct and inputs_correct

# _check_student_inputs, 3806-3807 and 3840-3843
inputs_correct = True
for answer_name, answer_value in six.iteritems(numtolerance_inputs):
    ...
    if answer_name in self.correct_inputs and not partial_correct:
        inputs_correct = False
return inputs_correct

_check_student_inputs starts from True and loops over the keys the learner sent (numtolerance_inputs), not the keys the problem expects (self.correct_inputs). A required numtolerance_input that is missing from the submitted dict is never compared, so it can never set inputs_correct to false. When the same "no answer" arrives from the UI as a blank text box (""), it is graded incorrect (ChoiceTextResponseTest scenario 1_choice_2_inputs_inputs_blank). The verdict therefore depends on whether "no answer" arrives as "" or as a missing key.

Measured

The method text of setup_response, assign_choice_names, get_score, _split_answers_dict, _check_student_choices and _check_student_inputs, plus util.compare_with_tolerance, was copied unmodified from the files above and run in a plain Python process. Problems were built as lxml XML and passed through the copied setup_response. Stand-ins replaced calc.evaluator (blank -> NaN, as calc does, plus + - * / arithmetic), CorrectMap, contextualize_text and i18n.

Problem A: choice 0 is correct and has one numtolerance_input (answer 123, tolerance 1); choice 1 is wrong. Problem B: choice 0 is correct and has two inputs (123 tol 1, 456 tol 10). C0 = 1_2_1_choiceinput_0bc, N0 = 1_2_1_choiceinput_0_numtolerance_input_0, N1 = ..._input_1. Each row gives the same result under radiotextgroup and checkboxtextgroup.

problem submitted dict result
A {C0: "choiceinput_0", N0: "123"} (control) correct
A {C0: "choiceinput_0", N0: "122.5"} (control, within tolerance) correct
A {C0: "choiceinput_0", N0: "5"} (control) incorrect
A {C0: "choiceinput_0", N0: ""} (control, blank box) incorrect
A {C1: "choiceinput_1", N0: "123"} (control, wrong choice) incorrect
A {} (control) incorrect
A {C0: "choiceinput_0"} (N0 omitted) correct
B {C0: "choiceinput_0", N0: "0", N1: "456"} (control) incorrect
B {C0: "choiceinput_0", N0: "123", N1: "456"} (control) correct
B {C0: "choiceinput_0"} (N0, N1 omitted) correct
B {C0: "choiceinput_0", N0: "123"} (N1 omitted) correct

Run through the same copied text, all 28 ChoiceTextResponseTest scenarios (TEST_INPUTS / TEST_SCENARIOS / TEST_PROBLEM_ARGS, read with ast.literal_eval) match under both group types. The 5 test_checkbox_grades scenarios also match. None of these scenarios selects a choice that has inputs while leaving the input keys out. When only the correct choice keys are submitted, the repo's own problems 1_choice_1_input, 1_choice_2_inputs, 1_input_script, 2_choices_1_input_1 and 2_choices_2_inputs are all graded correct.

Consequence

The stock UI does not trigger this: choicetextinput.js serialises every input.ctinput (blank ones as "") into the hidden input_<id>{} field. However, make_dict_of_responses accepts arbitrary JSON for a {}-suffixed key, and submit_problem passes the result to grade_answers without checking which keys are present. A hand-built problem_check request that carries only the choice key is therefore graded correct without any numeric answer. For that learner the problem reduces to picking the right radio button or checkbox. The resulting correct feeds the grade and the tracking event like any other correct submission.

This makes it a grading-integrity hardening issue rather than a grading error that ordinary learners run into.

Suggested fix

The same file already fails closed on a missing answer: NumericalResponse.get_score (1603-1604), StringResponse.get_score (1912-1913) and ChoiceResponse.get_score (830-833). The class docstring says ChoiceTextResponse's "Desired semantics match those of NumericalResponse and ChoiceResponse". The fix is to require every expected input to be present:

inputs_correct = True
for expected_name in self.correct_inputs:
    if expected_name not in numtolerance_inputs:
        inputs_correct = False
for answer_name, answer_value in six.iteritems(numtolerance_inputs):
    ...

self.correct_inputs only holds inputs of correct="true" choices, and if a correct choice is not selected, choices_correct is already false. The change therefore affects only submissions that selected every correct choice. Applied to a copy of the method text, the existing scenarios give the same results (28 of 28 under both group types, 5 of 5 checkbox scenarios, test_unchecked_input_not_validated still incorrect, and test_interpret_error still raises StudentInputError). Omitted-key submissions on 1_choice_1_input, 1_choice_2_inputs, 1_input_script, 2_choices_1_input_1 and 2_choices_2_inputs become incorrect.

A regression test could add a new TEST_INPUTS row such as "1_choice_1_input_key_omitted": [(True, [])] expecting incorrect on 1_choice_1_input, plus a case on 1_choice_2_inputs that sends one of the two inputs and omits the other.

Happy to open the PR.

Ngôn ngữ chính
JavaScript
Star
0
Fork
19
Merge trung bình
8 giờ
Pull request đã merge (30 ngày)
7

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của openedx/xblocks-core

Tất cả issue của openedx/xblocks-core

Issue tương tự

Thêm issue về JavaScript

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.