Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[FAQ] Homework 4 (2026) Q2/Q6: Why does my model's AUC (0.732) lose to `lead_score` alone, and why does C=0.001 beat C=1?

Aperta Adatta ai principianti
  • Qualcosa non va?
#489 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

@github-actions ci sta già lavorando.

Dal 21/10/2026.

  • #490 di @github-actions — aperta

Valutazione

Difficoltà
1/5
Tempo stimato
Meno di un'ora
Idoneità per principianti
87/100
Tipo di issue
Documentazione
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python, scikit-learn
Ambito
documentation, machine-learning

Direzione di ricerca

Questa è una risposta completa alle FAQ con un esempio svolto collegato nell’issue; non viene indicato alcun file del repository né alcun test. Leggi le FAQ vicine per conoscere il formato del repository, poi verifica come aggiungere questa risposta. Il lavoro è completato quando la FAQ è reperibile nel repository e conserva la spiegazione, l’esempio di codice e le risposte agli esercizi.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

faq-proposal
Course

machine-learning-zoomcamp

Question

Homework 4 (2026): In Q1 lead_score alone has a training AUC of 0.788, but in Q2 the full logistic regression model only gets 0.732 on validation, even though lead_score is one of its features. In Q6 the heavily regularized C=0.001 (0.749) also beats C=1 (0.732). Is my model broken?

Answer

Your model isn't broken, and 0.732 and C=0.001 are still the correct homework answers. The low score comes from feature scale, not from the evaluation code.

annual_income is around 50,000, while every other feature is between 0 and 10 (one-hot columns are 0/1). Logistic regression with an L2 penalty (the sklearn default) is not scale invariant. With C=1.0 the fit leans on income, and its weight comes out around −1e-5. On validation the model predicts converted for 82% of leads when the real rate is 59%. The useful features get drowned out, so the model's ranking is worse than ranking by lead_score alone (validation AUC 0.783).

That also explains Q6. A smaller C shrinks the weights, which limits how much the unscaled income column can distort the fit. That's why 0.001 wins. With C=1e-6 every weight is close to zero and the model underfits.

Scale the numeric columns (fit on the training data only) and the effect goes away:

from sklearn.preprocessing import StandardScaler

numerical = ["number_of_courses_viewed", "annual_income", "interaction_count", "lead_score"]
scaler = StandardScaler().fit(df_train[numerical])
df_train[numerical] = scaler.transform(df_train[numerical])
df_val[numerical] = scaler.transform(df_val[numerical])
# then DictVectorizer + LogisticRegression(solver='liblinear', C=1.0, max_iter=1000) as usual
setup validation AUC 5-fold mean ± std, C=1e-6 / 0.001 / 1
homework (unscaled) 0.732 0.617±0.019 / 0.749±0.010 / 0.732±0.007
lead_score alone 0.783 —
numeric features scaled 0.788 0.792±0.013 for all three C

After scaling, C has no effect on AUC. Even C=1e-6 ranks leads just as well, because AUC only depends on the order of the scores, not their size. Q6's ranking of C values is caused by the unscaled income column.

Takeaways:

  • Always compare your model against the best single feature (Q1 hands you that baseline). If the model can't beat it, check preprocessing.
  • Scale numeric features before regularized linear models.
  • For the homework, submit the unscaled values the questions ask for (0.732 and C=0.001).

Full worked example with plots: https://github.com/devotuoma/machine-learning-zoomcamp/tree/hw04-evaluation/cohorts/2026/homework/04-evaluation/solution

Checklist
  • I have searched existing FAQs and this question is not already answered
  • The answer provides accurate, helpful information
  • I have included any relevant code examples or links
Lingua principale
Python
Stelle
7
Fork
19
Merge medio
9g 22h
PR unite (30g)
10

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di DataTalksClub/faq

Tutte le issue di DataTalksClub/faq

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.