New Datastore Interfaces
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
Research direction
No files or tests are named. Start by reviewing the existing go-datastore interfaces, then compare the proposed StreamingDatastore, ReadClosurer, and OutParamer designs and the earlier discussion in #28; done requires a decided interface design and agreed scope.
Written by the indexing model from the issue text.
Description
The interfaces in go-datastore have been needing updating for a while. This came up in #28 so i decided to start the conversation up. Lets collect all these ideas together here so we can move forward with this.
Interface additions i've been looking for involve streaming interfaces:
type StreamingDatastore interface {
// GetR returns a reader to read the value directly from the source (sans buffering)
GetR(k ds.Key) (io.Reader, error)
// PutW returns a new put writer that can be used to stream a value
// into the datastore
PutW() (WritePutter, error)
}
// WritePutter is used to stream values into the datastore. Since the caller may not
// know the key they want to store this value as until the write is complete (content
// addressing) the key is set after the write is complete
type WritePutter interface{
io.Writer
FinishWithKey(k ds.Key) error
}
Also, for queries and maybe 'gets' in general, it would be nice to have a readonly view interface. Many storage backends (bolt in particular) allow you to read and operate on the data within the datastore without making a copy (i.e. using the datastores memory) within a closure. This would be really useful for things like 'load this protobuf object from disk' where we don't need our own copy of the bytes.
// A datastore implementing ReadClosurer allows passing a closure to operate
// on a readonly view of the data in question
type ReadClosurer interface {
GetFunc(k ds.Key, func(i interface{}) error) error
}
Alternatively, this could be done with an output parameter and interfaces:
type OutParamer interface{
GetOut(k ds.Key, out DSUnmarshaler) error
}
type DSUnmarshaler interface {
DSUnmarshal(val []byte) error
}
This would behave similarly to the json.Unmarshaler interfaces.
- Dominant language
- Go
- Stars
- 248
- Forks
- 68
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ipfs/go-datastore
-
exp/novice help wanted kind/enhancement status/ready
Difficulty 5/5 Over a week Newbie friendliness 25/100
ipfs/go-datastore#160 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
ipfs/go-datastore#139 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
ipfs/go-datastore#116 · 31 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 30/100
ipfs/go-datastore#102 · 5 comments ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
ipfs/go-datastore#101 · 4 comments · 1 reaction ·
All issues in ipfs/go-datastore
Similar issues
-
affects-8.5 component/session severity/moderate sig/transaction type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
enhancement good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
harvester/harvester#12311 · 1 comment ·
Maintainers usually reply within 1 day
-
verifyrun --bracket-digest --credit: relative --witness is could-not-check, absolute path creditsOpenbug effort:s impact:low raised-by:desk risk:low
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
medici-finance/assay#6580 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 80/100
blinklabs-io/gouroboros#3026 ·
Maintainers usually reply within 1 day