globus/globus-compute

Stream status information from endpoint to client

開放

#509 建立於 2021年6月10日

 (0 則留言) (0 個反應) (0 位負責人)Python (52 個分叉)auto 404
enhancementgood first issuehelp wanted

倉庫指標

星標
 (162 顆星)
PR 合併指標
 (平均合併 9天 19小時) (30 天內合併 16 個 PR)

描述

Is your feature request related to a problem? Please describe.

Currently when a function is launched via the older API you get a function_id, and via the executor you get a future to track the progress of your function. However, in either case, there is no mechanism to get you the richer tasks status info that's available to the web-service as of #444. Users have expressed frustration about the lack of updates regarding the current state of the function within the function lifecycle.

Describe the solution you'd like

In the older system:

func_id = fxc.run(...)
try:
     fxc.get_result(res)
except TaskPending as e:
     print("Task is pending due to : ", e) # <- This will only say "waiting-for-ep"

In the newer executor:

fx_future = fx_executor.submit(...)

fx_future.done() -> # Returns a bool
# proposed:
fx_future.status() -> ['submitted', 'waiting-for-ep', 'dispatched-to-ep', 'waiting-for-resources', 'running', 'completed']

This task also requires mapping out the complete task lifecycle, as well as the states that we will be able to report.

To make this work, we'll need the following changes:

To support in the query API

  1. Task status query to the web-service to return richer status info
  2. client to report the status via the TaskPending exception? or via fxc.get_status(func_id)

To support via executor

  1. Web-socket service to monitor task status changes via REDIS, we currently do not support streaming events via rabbit from web-service, although this could change.
  2. Web-socket service to stream task status changes to client
  3. Extend the executor to update status info via future.status field.

貢獻者指南