Add native Hugging Face Hub integration (push_to_hub / from_pretrained)

未关闭
  • 哪里不对？
#109 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
32/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
huggingface, python
领域
machine-learning

调研方向

Read the wrapper class and any existing v2 save/load methods, then check how TextEmbedder, CategoricalVariableNet, and ClassificationHead store their state. Define a self-contained Hub save/load flow for weights, configuration, tokenizer, labels, and categorical encodings, with safetensors and a generated model card. Done means users can push a trained classifier and reload it with from_pretrained using the installed library.

由索引模型根据 Issue 内容生成。

描述

Motivation

We want to publish trained torchTextClassifiers models on the Hugging Face Hub under the InseeFr org so external users can reuse them. At the moment, there is no path to do this with the required automation/quality...

Proposal

Make the torchTextClassifiers wrapper class (and possibly TextClassificationModel) inherit from huggingface_hub.ModelHubMixin, so users can do:

from torchTextClassifiers import torchTextClassifiers

clf = torchTextClassifiers.from_pretrained("InseeFrLab/<model>", revision="v1.0.0")
clf.predict(...)

clf.push_to_hub("InseeFrLab/<model>")
Serialized artifact (self-contained, no pickle)
File Content
model.safetensors weights of all components
config.json architecture + hyperparameters (TextEmbedder, CategoricalVariableNet, ClassificationHead), task type (multiclass / multilabel), library version
tokenizer/ HF tokenizer files, or n-gram tokenizer params/vocab
labels.json index ↔ label code ↔ human-readable label
categorical.json vocabularies / encodings of categorical variables
README.md auto-generated model card

Reloading must only require pip install torchTextClassifiers==<version>.

Implementation notes/proposals
  • Implement _save_pretrained / _from_pretrained on the wrapper. PyTorchModelHubMixin alone only covers the nn.Module weights, not the tokenizer, labels or categorical encodings.
  • If v2 already has internal save / load methods, the mixin should wrap them rather than duplicate them.
  • Set class-level mixin metadata: library_name="torchtextclassifiers", repo_url, docs_url, tags=["text-classification", ...].
  • Override generate_model_card() so the YAML header includes library_name, pipeline_tag: text-classification, license, language, and optional datasets / model-index (metrics) passed in by the user.
  • Allow extra card fields to be passed through push_to_hub(...), e.g. dataset ID + revision, training code commit, metrics.
  • Save with safetensors only; never torch.save of full objects.
  • Keep huggingface_hub / safetensors as an optional extra (pip install torchTextClassifiers[hub]) if we want a lean core.

Open questions

  • Should from_pretrained warn or raise when the saved library version differs (major version) from the installed one?
  • Do we also want ONNX export for serving, or keep that in a separate issue?
  • Should we register torchtextclassifiers as a library on the Hub (huggingface.js) to get a "Use this model" snippet?

References

主要语言
Python
星标
23
派生
7
30 天内没有已合并 PR

环境准备

这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

InseeFrLab/torchTextClassifiers 的其他 Issue

查看 InseeFrLab/torchTextClassifiers 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。