Problem: can't discover stream identifiers
Nadie ha tomado este issue todavía.
Evaluación
Este issue todavía no se ha evaluado.
Descripción
Separate comment to hide and add to specs if appropriate
I also have a request that I don't know how to put in the specs properly, and I need a lot of input from the data team (@joequant, should I tag anyone else?).
The issue is that we explicitly need a way to discover and share stream identifiers across our teams.
I see that
truflation-ingestorsdefine a property callednamefor eachExportDetails. Is that the identifier I'm expecting?If so, it's hard to follow the output from the code; there's no centralized list (that I know of) of all the pipelines available, then it's hard to agree on an identifier for each.
While previously ingesting from the DB, we've used the table names as identifiers for primitive streams (which seems reasonable). For transformed data, in our case, composed streams (e.g., Transport), we've been using a hash of category names, which are very arbitrary, and no one says it won't change a space or a comma.
As we're targeting building a new destination and eventually migrating to use this destination, having mappable identifiers from the current system is a must for a smooth migration process. It would help us detect drifts much more easily.
How do you (the data team) think this could be handled? Could we commit a list to any of these repos mapping all output exports and maybe the pipeline name or localization that originated them?
Could this problem be incorporated into this goal to start the identifiers correctly already?
Originally posted by @outerlook in #493
- Lenguaje dominante
- Go
- Estrellas
- 7
- Forks
- 3
- Merge medio
- 3 h 2 min
- PR fusionados (30 d)
- 13
Guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de trufnetwork/node
-
type: goal
Dificultad 5/5 Más de una semana Aptitud para principiantes 35/100
trufnetwork/node#1436 ·
-
trufnetwork/node#1435 · 1 asignado ·
-
trufnetwork/node#1429 · 2 comentarios · 1 asignado ·
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 25/100
trufnetwork/node#1313 · 2 comentarios · 1 reacción ·
-
Dificultad 4/5 3-5 días Aptitud para principiantes 35/100
trufnetwork/node#1200 · 1 comentario ·
Todos los issues de trufnetwork/node
Issues similares
-
Dificultad 1/5 Menos de una hora Aptitud para principiantes 60/100
github/gh-aw-mcpg#13748 ·
-
agentic-workflows
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
-
needs-triage
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
googleapis/librarian#7670 · 2 comentarios ·