CodeGen fails when case class fields are reserved java keywords
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Domain
- data-engineering, devtools
Research direction
Start by reproducing the Scala case class example and inspect the CodeGen path that produces generated.java. Focus on how accessors are emitted for fields named with reserved Java keywords. Done means the example compiles and TypedDataset.create succeeds without the generated.java selector error.
Written by the indexing model from the issue text.
Description
It is possible to define a case class with reserve field names using back-ticks.
case class Foo(a: String, `if`: Int)
val t = TypedDataset.create(Seq(Foo("a",2), Foo("b",2)))
Fails with the following error:
17/06/01 00:45:54 ERROR CodeGenerator: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 49, Column 44: Unexpected selector 'if' after "."
/* 001 */ public java.lang.Object generate(Object[] references) {
/* 002 */ return new SpecificUnsafeProjection(references);
/* 003 */ }
/* 004 */
/* 005 */ class SpecificUnsafeProjection extends org.apache.spark.sql.catalyst.expressions.UnsafeProjection {
/* 006 */
/* 007 */ private Object[] references;
/* 008 */ private UnsafeRow result;
/* 009 */ private org.apache.spark.sql.catalyst.expressions.codegen.BufferHolder holder;
/* 010 */ private org.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter rowWriter;
/* 011 */
/* 012 */
/* 013 */ public SpecificUnsafeProjection(Object[] references) {
/* 014 */ this.references = references;
/* 015 */ result = new UnsafeRow(2);
/* 016 */ this.holder = new org.apache.spark.sql.catalyst.expressions.codegen.BufferHolder(result, 32);
/* 017 */ this.rowWriter = new org.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter(holder, 2);
/* 018 */ }
/* 019 */
/* 020 */ // Scala.Function1 need this
/* 021 */ public java.lang.Object apply(java.lang.Object row) {
/* 022 */ return apply((InternalRow) row);
/* 023 */ }
/* 024 */
/* 025 */ public UnsafeRow apply(InternalRow i) {
/* 026 */ holder.reset();
/* 027 */
/* 028 */ rowWriter.zeroOutNullBytes();
/* 029 */
/* 030 */
/* 031 */ $line48.$read$$iw$$iw$$iw$$iw$$iw$$iw$Foo value2 = ($line48.$read$$iw$$iw$$iw$$iw$$iw$$iw$Foo)i.get(0, null);
/* 032 */
/* 033 */ boolean isNull1 = false;
/* 034 */ final java.lang.String value1 = isNull1 ? null : (java.lang.String) value2.a();
/* 035 */ isNull1 = value1 == null;
/* 036 */ boolean isNull = isNull1;
/* 037 */ final UTF8String value = isNull ? null : org.apache.spark.unsafe.types.UTF8String.fromString(value1);
/* 038 */ isNull = value == null;
/* 039 */ if (isNull) {
/* 040 */ rowWriter.setNullAt(0);
/* 041 */ } else {
/* 042 */ rowWriter.write(0, value);
/* 043 */ }
/* 044 */
/* 045 */
/* 046 */ $line48.$read$$iw$$iw$$iw$$iw$$iw$$iw$Foo value4 = ($line48.$read$$iw$$iw$$iw$$iw$$iw$$iw$Foo)i.get(0, null);
/* 047 */
/* 048 */ boolean isNull3 = false;
/* 049 */ final int value3 = isNull3 ? -1 : value4.if();
/* 050 */ if (isNull3) {
/* 051 */ rowWriter.setNullAt(1);
/* 052 */ } else {
/* 053 */ rowWriter.write(1, value3);
/* 054 */ }
/* 055 */ result.setTotalSize(holder.totalSize());
/* 056 */ return result;
/* 057 */ }
/* 058 */ }
- Dominant language
- Scala
- Stars
- 895
- Forks
- 135
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 3
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from typelevel/frameless
-
Scala 3 builds Openenhancement feature help wanted
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
-
enhancement feature
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
Difficulty 5/5 Over a week Newbie friendliness 30/100
All issues in typelevel/frameless
Similar issues
-
Area: Excel support
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
orbeon/orbeon-forms#7893 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
lichess-org/lila#21788 · 2 comments ·
-
area:Iceberg area:writer documentation requires-triage
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
apache/datafusion-comet#6147 ·
-
feature
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
GiganticMinecraft/SeichiAssist#2978 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
chipsalliance/rocket-chip#3833 ·