Feedback wanted: Intuitive scoring for just-in-time sets

Aperta
#1,791 4 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 4 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
15/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Ferma
Stack tecnologico
perl
Ambito
backend

Direzione di ricerca

The proposal concerns the jitar_problem_adjusted_status routine in lib/WeBWorK/Utils.pm, where child-problem weights are summed into $childScore. Read that routine and the just-in-time set grading logic first, then decide how a new scoring mode would sit beside the current one. Done would mean a maintainer-agreed design, since the thread has no decision and the issue has been open about four years.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Problem

The current grading algorithm for just-in-time problem sets is rather unintuitive in my opinion, since for child problems it abuses the meaning of the value field to store some factor which - when considering all child problems which count towards the parent - results in a weight. Also, the points for a child problem shown to the student do not correspond to the actually achievable points if I'm not mistaken.

Solution

It would be better in my opinion if the value field retained its meaning (maximum points to reach for the child problem). The relevant jitar_problem_adjusted_status routine in /lib/WeBWorK/Utils.pm could simply proceed as follows when computing the status of a parent problem:

  1. Gather the weighted sum of all subproblems (as is currently done and saved in $childScore)
  2. If this weighted sum is greater than the value of the parent, return one as the status (i.e. when the values of all child problems counting towards the parent problem add up to something at least as large as the value of the parent problem, the score is capped)
  3. Otherwise divide the weighted sum by the parent problem's value and return the maximum of the result and the parent problem's current status.

Instead of the capping in step two, one could simply consider the case where child problems can yield more points than the parent when combined as illegal and throw an error when the user tries to import or save such a set, but this would unfortunately break backwards compatibility (although I'm sure that everybody will switch to the new algorithm...).

Advantages

In addition to being much more intuitive for the end user, this would also enable to apply penalties when a student reaches the child problems after not being able to solve the parent. With the current algorithm, I see no alternative to the insertion of a dummy child problem (always counted as false).

Instead of the capping in step two, it would be better to consider the case where child problems can yield more points than the parent when combined as illegal and throw an error when the user tries to import or save such a set. However, this would break backwards compatibility.

Implementation

I intend to add another routine for the adjusted status to Utils.pm and then offer either a checkbox labelled “use alternative jit scoring” or a dropdown “scoring algorithm for jit sets” with the current algorithm as default to keep backwards compatibility.

Before implementing this, I'd be thankful for some feedback, in particular whether this has a chance to be included in the main tree as I don't want to maintain a local fork.

Lingua principale
Perl
Stelle
164
Fork
167
Merge medio
6g 7h
PR unite (30g)
27

Preparare l'ambiente

  • Include un Dockerfile o un file Docker Compose
  • Nessun modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di openwebwork/webwork2

Tutte le issue di openwebwork/webwork2

Issue simili

Altre issue su Perl

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.