confluentinc/ksql

Topic <--> SQL Conversion Tool

Open

#8444 aperta il 3 dic 2021

Vedi su GitHub
 (0 commenti) (0 reazioni) (0 assegnatari)Java (1048 fork)batch import
enhancementgood first issue

Metriche repository

Star
 (5739 star)
Metriche merge PR
 (Merge medio 4g 20h) (6 PR mergiate in 30 g)

Descrizione

Is your feature request related to a problem? Please describe.

Often we run into situations when we try to replicate a user's environment and data, and we end up using mock data that we create which is similar-ish to what the user is seeing. This isn't always sufficient to help us get to the root cause of an issue - it would be great if we could have a way to reliably "load" a topic into ksqlDB from a topic that isn't in the same kafka cluster (and can be sent over public internet, such as a file)

Describe the solution you'd like

It'd be cool if we could take a topic and dump it in standard SQL format (e.g. using INSERT INTO ... VALUES). That way, we could take the file and just run it as a script into ksqlDB and set up the data with exactly the correct timestamps, keys and values (though perhaps not offsets) agnostic to any format (e.g. AVRO/JSON/PROTOBUF).

Describe alternatives you've considered

If such a tool exists, I'd love to hear about it!

Guida contributor