[Scala] Common Scala types fail with requireClassRegistration(true) because ForyScala does not pre-register them

Aperta
  • Qualcosa non va?
#4,101 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
52/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
scala
Ambito
backend

Direzione di ricerca

Start in ScalaSerializers.registerSerializers and inspect CollectionKindsTest, along with the supported types listed in docs/object-serialization/scala/native.md. Determine how native registrations and xlang Option handling should cover the listed Scala types without destabilizing existing type IDs. Done means the affected values round trip with requireClassRegistration(true), and the relevant kinds can move from the unregistered tests in #4100.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug scala
Search before asking
  • I had searched in the issues and found no similar issues.
Version

main (1.8.0-SNAPSHOT), Scala 2.13.18 and 3.3.8

Component(s)

Scala

Minimal reproduce step
import org.apache.fory.scala.ForyScala

val fory = ForyScala.builder()
  .withXlang(false)
  .withRefTracking(true)
  .requireClassRegistration(true) // the default
  .build()

fory.deserialize(fory.serialize(Some(1)))
// InsecureException: class scala.Some is not registered ...

requireClassRegistration(true) is the builder default. The Scala security guide also recommends it. So the Option and Either examples in docs/object-serialization/scala/native.md fail as written.

What did you expect to see?

The Scala types that docs/object-serialization/scala/native.md lists as supported should round trip with class registration enabled, without the application registering Scala library classes. Collection kinds should be registered too, including their internal node and factory classes.

What did you see instead?

In native mode, compatible and schema-consistent, InsecureException: class ... is not registered is thrown for the types below. The class named is the first unregistered class reached:

Value Unregistered class
Some(1) / None scala.Some / scala.None$
Right(1) / Left("a") scala.util.Right / scala.util.Left
(1, 2) scala.Tuple2$mcII$sp (specialised tuples)
Duration(1, "s") scala.concurrent.duration.FiniteDuration
Duration.Inf scala.concurrent.duration.Duration$$anon$2
Vector.empty scala.collection.immutable.Vector0$
immutable.ArraySeq(1, 2) / ArraySeq("a") ArraySeq$ofInt / ArraySeq$ofRef
mutable.ArraySeq("a") scala.collection.mutable.ArraySeq$ofRef
immutable.ListSet(1) scala.collection.immutable.ListSet$Node
immutable.ListMap(1 -> 1) scala.collection.immutable.ListMap$Node
immutable.IntMap(1 -> "a") / LongMap(1L -> "a") IntMap$Tip / LongMap$Tip
TreeSet, mutable.TreeSet, mutable.BitSet scala.collection.EvidenceIterableFactory$ToFactory
TreeMap, mutable.TreeMap scala.collection.SortedMapFactory$ToFactory
mutable.LongMap(1L -> 1) scala.collection.mutable.Map$
new mutable.StringBuilder("x") scala.collection.mutable.IndexedSeq$

Sorted collections also write their Ordering by reference, so the ordering classes, for example scala.math.Ordering$Int$, probably need registering as well.

In xlang mode, a reflective (non-macro) case class with an Option[T] field fails the same way: ClassUnregisteredException: Class scala.Some is not registered, thrown from the generated codec's writeClassAndObject. Turning off requireClassRegistration does not help in xlang mode.

These types all round trip with requireClassRegistration(false). Separately, some of them come back as a different class; that will be raised as its own issue.

Anything Else?

The fix most likely belongs in ScalaSerializers.registerSerializers, which already registers many Scala collection classes. Points to settle:

  • Type IDs: adding registrations assigns new internal type IDs in native mode. Where should new entries go so that IDs of existing registrations, and of classes the user registers afterwards, stay stable? Or is an ID change acceptable for native mode?
  • Scala versions: some internal classes (Vector0–Vector6, ListSet$Node, IntMap$Tip, …) are Scala library implementation details. They should be registered by name, as some already are, so that differences between 2.13 and 3 library versions don't break installation.
  • Xlang Option: this may need a nullable-wrapper field path rather than a type registration, because xlang Option[T] is meant to carry T as nullable.
  • Tests: once fixed, the kinds in CollectionKindsTest that are currently tested without registration (#4100) can move to the registration-required group.
Are you willing to submit a PR?
  • I'm willing to submit a PR!
Lingua principale
Java
Stelle
4.6k
Fork
470
Merge medio
9h 28m
PR unite (30g)
33

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di apache/fory

Tutte le issue di apache/fory

Issue simili

Altre issue su Java

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.