globus/globus-compute

Stream status information from endpoint to client

开放

#509 创建于 2021年6月10日

 (0 条评论) (0 个反应) (0 位负责人)Python (52 个派生)auto 404
enhancementgood first issuehelp wanted

仓库指标

星标
 (162 个星标)
PR 合并指标
 (PR 指标待抓取)

描述

Is your feature request related to a problem? Please describe.

Currently when a function is launched via the older API you get a function_id, and via the executor you get a future to track the progress of your function. However, in either case, there is no mechanism to get you the richer tasks status info that's available to the web-service as of #444. Users have expressed frustration about the lack of updates regarding the current state of the function within the function lifecycle.

Describe the solution you'd like

In the older system:

func_id = fxc.run(...)
try:
     fxc.get_result(res)
except TaskPending as e:
     print("Task is pending due to : ", e) # <- This will only say "waiting-for-ep"

In the newer executor:

fx_future = fx_executor.submit(...)

fx_future.done() -> # Returns a bool
# proposed:
fx_future.status() -> ['submitted', 'waiting-for-ep', 'dispatched-to-ep', 'waiting-for-resources', 'running', 'completed']

This task also requires mapping out the complete task lifecycle, as well as the states that we will be able to report.

To make this work, we'll need the following changes:

To support in the query API

  1. Task status query to the web-service to return richer status info
  2. client to report the status via the TaskPending exception? or via fxc.get_status(func_id)

To support via executor

  1. Web-socket service to monitor task status changes via REDIS, we currently do not support streaming events via rabbit from web-service, although this could change.
  2. Web-socket service to stream task status changes to client
  3. Extend the executor to update status info via future.status field.

贡献者指南