Scripts for exporting partitions from a table and reinitialising a table from those exported partitions

Open
#8,625 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
54/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
aws, java
Domain
cli, databases

Research direction

Find the existing ExportPartitions and ReinitialiseTableFromExportedPartitions classes and the scripts that invoke similar table operations. Start by checking how those scripts accept configuration and arguments, then compare them with the classes' inputs and outputs. Done means users can run scripts to export a table's partitions to JSON and reinitialise another table from that export.

Written by the indexing model from the issue text.

Description

enhancement scripts
User Story

As a user I want a script to let me output the partitions from a table, and a script to let me reinitialise another table with the exported partitions from the first table.

Description / Background

The ExportPartitions class is used to export the partitions information from a table to a JSON file. This file can then be used by the class ReinitialiseTableFromExportedPartitions to reinitialise a table with the same partitions as the first table. This is useful when creating a new table with the same partition structure as another table (e.g. when a table contains data from one day and we want to create a table for tomorrow with the same partition structure).

We need scripts to allow these classes to be used easily.

Dominant language
Java
Stars
107
Forks
29
Avg merge
1d 33m
Merged PRs (30d)
115

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from gchq/sleeper

All issues in gchq/sleeper

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.