globus/globus-compute

Stream status information from endpoint to client

オープン

#509 opened on 2021/06/10

 (0 件のコメント) (0 件のリアクション) (0 人の担当者)Python (52 件のフォーク)auto 404
enhancementgood first issuehelp wanted

Repository metrics

Stars
 (162 個のスター)
PR merge metrics
 (PR metrics pending)

説明

Is your feature request related to a problem? Please describe.

Currently when a function is launched via the older API you get a function_id, and via the executor you get a future to track the progress of your function. However, in either case, there is no mechanism to get you the richer tasks status info that's available to the web-service as of #444. Users have expressed frustration about the lack of updates regarding the current state of the function within the function lifecycle.

Describe the solution you'd like

In the older system:

func_id = fxc.run(...)
try:
     fxc.get_result(res)
except TaskPending as e:
     print("Task is pending due to : ", e) # <- This will only say "waiting-for-ep"

In the newer executor:

fx_future = fx_executor.submit(...)

fx_future.done() -> # Returns a bool
# proposed:
fx_future.status() -> ['submitted', 'waiting-for-ep', 'dispatched-to-ep', 'waiting-for-resources', 'running', 'completed']

This task also requires mapping out the complete task lifecycle, as well as the states that we will be able to report.

To make this work, we'll need the following changes:

To support in the query API

  1. Task status query to the web-service to return richer status info
  2. client to report the status via the TaskPending exception? or via fxc.get_status(func_id)

To support via executor

  1. Web-socket service to monitor task status changes via REDIS, we currently do not support streaming events via rabbit from web-service, although this could change.
  2. Web-socket service to stream task status changes to client
  3. Extend the executor to update status info via future.status field.

コントリビューターガイド