Pipeline representation and relationship betwen KFP/Vertex AI Pipelines and TFX
@jeelania10 ci sta già lavorando.
Dal 12/8/2026.
Valutazione
Questa issue non è ancora stata valutata.
Descrizione
If the feature is related to a specific library below, please raise an issue in
the respective repo directly:
TensorFlow Data Validation Repo
TensorFlow Model Analysis Repo
System information
- TFX Version (you are using):
- Environment in which you plan to use the feature (e.g., Local
(Linux/MacOS/Windows), Interactive Notebook, Google Cloud, etc..): - Are you willing to contribute it (Yes/No):
Describe the feature and the current behavior/state.
Today, TFX pipelines have to be compiled to kubeflow pipelines to run on vertex. We should also support the other direction, taking kubeflow pipelines (e.g. pipeline spec/yaml representation) and converting them to a representation runnable by the tfx runner.
Once we have this working, we should enable vertex ai pipelines to be a runner for TFX pipelines natively. The reason for this is it is quite complex to build these pipelines without kubernetes knowhow. If we can build that bridge, and support jax, we can make it a lot easier to build large scale ML pipelines using re-usable components in a way that doesn't require knowledge/expertise of the underlying infrastructure.
Will this change the current API? How?
TFX would get a transpiler for kubeflow pipelines that use TFX components.
For the second step (which cannot be implemented as part of this project, but part of vertex ai pipelines), vertex ai pipelines would become a runner for TFX pipelines directly without having to go through the KFP route.
Who will benefit with this feature?
ML practitioners who do now want to deal with the infrastructure complexity. It would be a similar model to dataflow, write one pipeline, run it anywhere. Implementing this would improve the bridge between TFX and KFP, which today is quite brittle.
Do you have a workaround or are completely blocked by this? :
I have kubernetes infrastructure and knowhow, so I'm not blocked by this.
Name of your Organization (Optional)
Intuitive.ai
Any Other info.
- Lingua principale
- Python
- Stelle
- 2.2k
- Fork
- 727
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Nessun modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di tensorflow/tfx
-
Penguin simple tutorial fails with Keras 3 at model.save(..., save_format='tf')Forse già presa @jeelania10 l’ha presa 5 giorni fa. Apertastat:awaiting response type:docs
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
tensorflow/tfx#7919 · 1 commento · 1 assegnatario ·
-
Different artifact type names across execution environmentsForse già presa @jeelania10 l’ha presa 90 giorni fa. Apertastat:awaiting tensorflower type:bug
tensorflow/tfx#7849 · 2 commenti · 1 assegnatario ·
-
Roadmap for TensorFlow ExtendedForse di nuovo libera @jeelania10 l’ha presa 90 giorni fa e non c’è nessuna pull request aperta. Apertastat:awaiting tensorflower type:feature
tensorflow/tfx#7788 · 24 commenti · 6 reazioni · 1 assegnatario ·
-
Support Python 3.11Forse di nuovo libera @bharatjetti l’ha presa 365 giorni fa e non c’è nessuna pull request aperta. Apertastat:contributions welcome type:feature
tensorflow/tfx#7743 · 5 commenti · 1 assegnatario ·
-
Documentation for custom artifact in Vertex AIForse di nuovo libera @lego0901 l’ha presa 1202 giorni fa e non c’è nessuna pull request aperta. Apertastat:awaiting tensorflower type:docs
tensorflow/tfx#6102 · 2 commenti · 1 assegnatario ·
Tutte le issue di tensorflow/tfx
Issue simili
-
area: providers/aws priority: p2 size: S type: bug type: testing
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
finos/open-resource-broker#851 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 1/5 1-3 ore Idoneità per principianti 88/100
sandialabs/atlas-ui-3#1196 ·
I maintainer di solito rispondono entro 1 giorno
-
enhancement triage
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
ShaftHQ/SHAFT_ENGINE#7902 ·
I maintainer di solito rispondono entro 1 giorno
-
[Bug]: GraphML export writes invalid XML when a value contains a control character (e.g. form feed)Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
topoteretes/cognee#6646 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 75/100
DOI-USGS/pywatershed#428 ·