[Go] enhancement request to expose AnyValue() on Scalar

Open
#67 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
38/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
go
Domain
data

Research direction

Start at the Scalar interface and trace scalar.GetScalar and the existing scalar implementations to understand how values are represented. Check whether each scalar type can expose its underlying typed value without conversion, then verify that callers can build []any values for serialization without a type switch.

Written by the indexing model from the issue text.

Description

Type: enhancement
Describe the enhancement requested

I wanted to gauge interest in a method on the Scalar interface to expose the value via any/interface{} like:

type Scalar interface {
	…
	AnyValue() any
	…
}

Usage: This would greatly help scenarios where the user is intentionally serializing to another format (e.g. JSON) and would like to incur memory overhead.
For example, creating an array of anys for serialization. I know that String() can also be used for this, but it doesn't preserve the type.

I'm thinking that this also allows the user to recover the type via casting subsequently. However, it allows for handling the scalar value without casting of the Scalar value first.

Example usage:

func RecordsToAnyArr(recs []arrow.Record) (rec [][]any, err error) {
	rows := make([][]any, 0)
	for _, rec := range recs {
		for r := 0; r < int(rec.NumRows()); r++ {
			row := make([]any, rec.NumCols())
			for c := 0; c < int(rec.NumCols()); c++ {
				col := rec.Column(c)
				s, err := scalar.GetScalar(col, r)
				if err != nil {
					return err
				}
				row[c] = s.AnyValue()
			rows = append(rows, row)
		}
	}
}

Today, this would large switch case statement to achieve this.
Let me know if I'm missing something.

Component(s)

Go

Related to apache/arrow#10972

Dominant language
Assembly
Stars
406
Forks
146
Avg merge
2d 8h
Merged PRs (30d)
93

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/arrow-go

All issues in apache/arrow-go

Similar issues

More Data Engineering issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.