akka/akka-core

give users options to control the starting up behavior of the distributed data local replicator

Open

#29,948 opened on Jan 13, 2021

View on GitHub
 (2 comments) (0 reactions) (0 assignees)Scala (3,547 forks)batch import
1 - triagedhelp wantedt:distributed-data

Repository metrics

Stars
 (13,277 stars)
PR merge metrics
 (Avg merge 8d 19h) (10 merged PRs in 30d)

Description

Currently, the distributed data local replicator behaves like the following.

  1. the replicator first tries to load everything stored in the local durable storage.
  2. if there is any deserialization failure, the replicator crashes, or keep retrying. the concequence is that the replicator does not participate in the ddata cluster and does not handle any commands.

It would be great if users can specify in the config file which durable keys must be fully loaded during (re)starting. For other keys that are not among these, the replicator should just log an error and ignore the deserialization failure.

one example that can benefit from this is to use ddata as a schema registry. at starting time, only the schema files stored with a schema durable key need to be 100% loaded, the other serializers will rely on the loaded schemas to deserialize data in other durable keys.

Contributor guide

Research direction
Examine the current distributed data replicator code to understand the loading sequence. Then propose a configuration mechanism to specify durable keys that must be loaded fully, while ignoring deserialization failures for others.
Tech stack
java
Domain
backenddatabase
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
3-5 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Git
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60