RecordBuilder should confirm non-nullable fields do not contain null rows when building Record
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 35/100
Research direction
Start at array.NewRecordBuilder and RecordBuilder.NewRecord, using the reported schema and null-containing field as the reproduction case. Confirm the intended behavior against the Rust example and ensure that a non-nullable field containing null rows is rejected when building the record.
Written by the indexing model from the issue text.
Description
Describe the bug, including details regarding any error messages, version, and platform.
I am able to construct an arrow record, where a field should be non-nullable, but it contains null rows. I'm wondering if maybe RecordBuilder.NewRecord() should confirm that the fields's arrays do not contain nulls if the field is specified as Nullable: false in the schema?
version = v18.2.0
platform = arm mac
package main
import (
"log"
"github.com/apache/arrow-go/v18/arrow"
"github.com/apache/arrow-go/v18/arrow/array"
"github.com/apache/arrow-go/v18/arrow/memory"
)
func main() {
allocator := memory.NewGoAllocator()
schema := arrow.NewSchema([]arrow.Field{
// should be not null:
{Name: "a", Type: arrow.PrimitiveTypes.Int32, Nullable: false},
}, nil)
recordBuilder := array.NewRecordBuilder(allocator, schema)
recordBuilder.Field(0).(*array.Int32Builder).Append(1)
recordBuilder.Field(0).(*array.Int32Builder).Append(2)
recordBuilder.Field(0).AppendNull()
recordBuilder.Field(0).(*array.Int32Builder).Append(4)
record := recordBuilder.NewRecord() // this works
defer record.Release()
log.Printf("record: %v", record)
/* prints
2025/05/08 20:32:47 record: record:
schema:
fields: 1
- a: type=int32
rows: 4
col[0][a]: [1 2 (null) 4]
*/
}
This is not allowed in the Rust implementation:
use std::sync::Arc;
use arrow_array::{Int32Array, RecordBatch};
use arrow_schema::{DataType, Field, Schema};
fn main() {
let schema = Arc::new(Schema::new(vec![Field::new("a", DataType::Int32, false)]));
let mut ids = Int32Array:: builder(4);
ids.append_value(1);
ids.append_value(2);
ids.append_null();
ids.append_value(4);
let ids = ids.finish();
// this panics:
// called `Result::unwrap()` on an `Err` value: InvalidArgumentError("Column 'a' is declared as non-nullable but contains null values")
let batch = RecordBatch::try_new(schema.clone(), vec![Arc::new(ids)]).unwrap();
println!("batch: {:?}", batch);
}
Component(s)
Other
- Dominant language
- Assembly
- Stars
- 406
- Forks
- 146
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 93
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/arrow-go
-
Type: enhancement
Difficulty 5/5 Over a week Newbie friendliness 30/100
-
Type: usage
Difficulty 4/5 3-5 days Newbie friendliness 38/100
-
Difficulty 3/5 1-2 days Newbie friendliness 90/100
-
good-first-issue Type: enhancement Type: usage
Difficulty 4/5 3-5 days Newbie friendliness 52/100
-
Difficulty 4/5 3-5 days Newbie friendliness 52/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
cisagov/cyhy-reports#149 · 3 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
open-compass/VLMEvalKit#1698 ·