Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Gemma 4 Text-Only Support

Abierto
  • ¿Algo no está bien?
#33 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
15/100
Tipo de issue
Nueva funcionalidad
Claridad
Bien especificado
Estado de actividad
Estancado
Stack tecnológico
kotlin
Área
backend, machine-learning

Línea de trabajo

Start by inspecting the existing Gemma3n ingestion and runtime paths, then read the Gemma 4 config, tokenizer, and generation metadata requirements in this issue. The work spans model loading, attention, KV-cache, CLI integration, and CI; completion means the listed unit, integration, and smoke tests pass for Gemma 4 E2B, with Gemma3n unaffected.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Summary

Implement complete text-only inference for Gemma 4 models (E2B, E4B, optionally 31B) in SKaiNET-transformers. This covers config parsing, weight loading, tokenizer with special tokens, prompt formatting with Gemma 4 control tokens, hybrid attention with p-RoPE, KV-cache sharing, and kgemma CLI integration.

Motivation

Gemma 4 text-only is the foundation for all further Gemma 4 work. Correct config parsing, weight loading, and attention semantics must be in place before multimodal or MoE extensions.

Functional Requirements

1. Config Parser & Model Metadata
  • New Gemma4ConfigParser recognizing model_type=gemma4 and architectures=["Gemma4ForConditionalGeneration"]
  • Parse text_config fields: layer_types, max_position_embeddings, sliding_window, num_kv_shared_layers, rope_parameters (full vs sliding, rope_type="proportional", partial_rotary_factor), global_head_dim
  • New Gemma4ModelMetadata class with all parsed fields
  • Validate bos_token_id, eos_token_id, pad_token_id consistency with generation_config.json
2. Weight Loading (SafeTensors & GGUF)
  • Gemma4SafeTensorsWeightLoader with Gemma-4-specific HF tensor name -> runtime name mapping
  • Support single-file and sharded-index (model.safetensors.index.json)
  • DType handling (BF16/F16/F32) and necessary transposes
  • PLE (Per-Layer Embeddings) table loading
  • GGUF streaming load path with QuantPolicy handling (DEQUANTIZE_TO_FP32, allowQuantized=true)
3. Tokenizer & Special Tokens
  • Load tokenizer.json + tokenizer_config.json correctly
  • Support add_bos_token=true, padding_side=left
  • Register all Gemma 4 special tokens: <|turn>, <turn|>, <|think|>, <|tool>, <tool|>, <|tool_call>, <tool_call|>, <|tool_response>, <tool_response|>, <|image|>, <|audio|>
  • Ensure token ID consistency (E2B: bos=2, eos=1, pad=0)
  • Backward compatibility with existing GGUFTokenizer.fromTokenizerJson(...) path
4. Prompt Formatter
  • Gemma4PromptFormatter implementing the official turn schema: 
    <|turn>system
{system_message}<turn|>
<|turn>user
{user_message}<turn|>
<|turn>model
  • Thinking mode: <|think|> injection in system prompt
  • Tool-calling: <|tool>...<tool|>, <|tool_call>...<tool_call|>, <|tool_response>...<tool_response|> sequences
  • <|tool_response> as stop sequence during inference
  • Output escaping via <|"|> delimiter for tool arguments
5. Runtime: Hybrid Attention, KV-Cache, Long-Context
  • Configuration-driven hybrid attention from layer_types (sliding_attention / full_attention interleaving; last layer always global)
  • Proportional RoPE (p-RoPE) for global attention layers with separate rope_theta for sliding vs full
  • global_head_dim support for global attention layers
  • KV-sharing: num_kv_shared_layers — last N layers share KV cache
  • Respect max_position_embeddings (128K for E2B/E4B, 256K for 31B)
  • KV-cache compression compatibility with TurboQuant policies
6. kgemma CLI Integration
  • Update kgemma selection logic to differentiate Gemma3n vs Gemma4 based on config.json model_type
  • Unified ingestion abstraction (analogous to Gemma3nIngestion) for Gemma4
  • tok/s measurement (already present) serves as regression metric
  • Extend smoke-models.json with Gemma4 E2B entry
7. Tests & CI
  • Unit tests:
    • testGemma4ConfigE2B_parsesLayerTypesAndRope()
    • testGemma4Tokenizer_specialTokensPresent_turn_tool_think()
    • testGemma4PromptFormatter_matchesDocsExample()
  • Integration tests:
    • testGemma4LoadAndGenerate_oneStep_temperatureZero() (SafeTensors load + 1-step decode, CPU FP32, deterministic)
  • Smoke tests:
    • Gemma4 E2B added to smoke-models.json, runner via kgemma
    • tok/s performance regression baseline
  • Golden tests: tokenizer parity checks against HF reference, prompt template string equality against official examples

Acceptance Criteria

  • Gemma4ConfigParser correctly parses config.json for E2B, E4B, and 31B model variants
  • Gemma4ModelMetadata contains all required fields: layer_types, rope_parameters, num_kv_shared_layers, max_position_embeddings, global_head_dim
  • Gemma4SafeTensorsWeightLoader loads single-file and sharded SafeTensors without errors for E2B
  • PLE tables are loaded and applied correctly
  • Tokenizer roundtrip (encode -> decode) is stable for all Gemma 4 special tokens
  • bos/eos/pad token IDs match generation_config.json values
  • Gemma4PromptFormatter output matches official documentation examples exactly (string equality)
  • Thinking mode and tool-calling token sequences are correctly formatted
  • <|tool_response> functions as a stop sequence during generation
  • Hybrid attention produces correct outputs: sliding-window layers use local RoPE, global layers use p-RoPE with global_head_dim
  • KV-sharing reduces memory usage for shared layers (measurable)
  • Generation runs without crash on 128K context inputs (E2B/E4B)
  • kgemma CLI auto-detects Gemma4 models and routes to correct runtime
  • Smoke test passes for Gemma4 E2B with measurable tok/s
  • All unit and integration tests pass in CI (allTests)
  • Gemma3n code path and tests remain unaffected

Risks & Mitigations

Risk Mitigation
Tokenizer mismatch (special tokens, left padding) Golden token tests from tokenizer_config.json + reference fixtures; fail-fast validation of bos/eos/pad
Attention semantics (p-RoPE / global_head_dim / KV-sharing) Stepwise activation: sliding-only on short contexts first, then global+RoPE; golden logits for small inputs
Long-context memory Strict KV-sharing + KV compression (TurboQuant policies) as performance phase

Estimated Effort

Component Effort
Config Parser & Metadata High
Weight Loader High
Tokenizer & Special Tokens Medium-High
Prompt Formatter Medium
Hybrid Attention + KV-Cache High
kgemma Integration Medium
Tests & CI Medium
Lenguaje dominante
Kotlin
Estrellas
8
Forks
3
Merge medio
1 d 16 h
PR fusionados (30 d)
48

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de SKaiNET-developers/SKaiNET-transformers

Todos los issues de SKaiNET-developers/SKaiNET-transformers

Issues similares

Más issues de Kotlin

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.