O-14: Execution contexts for user control and performance feedback (optional)

Open
#194 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Stale
Tech stack
python
Domain
distributed-systems

Research direction

No files, tests, or entry points are named. Start by mapping the distributed array computation APIs, then define the decorator and context-manager scope for debugging, performance feedback, computation strategy, and implicit communication. Done means users can control these behaviors as described.

Written by the indexing model from the issue text.

Description

SBIR-year-2

Objective: To provide features that allow an ODIN user to control how certain operations are carried out, and to provide performance feedback.

Relevance: By providing features that allow users to control how operations are performed, ODIN users can adapt the framework to their specific usecase, and choose where they fall on the performance-usability spectrum.

Description: This task will implement, via Python decorators and context managers, features that allow the end user to control how distributed array computations are performed. These modifications will allow capabilities such as:

• Debugging and performance feedback – this will turn on full feedback for easy debugging, allowing a user to easily determine if an exception was raised on a worker node and get the context and contents of that exception. The performance feedback will optionally provide information on what and how much data was communicated between nodes, and how much communication overhead was involved in a computation, for example.

• Control over the computation strategy for distributed array expressions: whether source computes, destination computes, or whether the operation should minimize communication.

• Control over whether communication should be allowed implicitly. For performance reasons, a user may desire the ability to disallow any implicit communication for an array computation, and this context manager would provide this performance-oriented feature, requiring the end user to explicitly allocate all arrays and ensure that they are distributed in an identical fashion before performing computations with them.

Dominant language
Python
Stars
5
Forks
1
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from enthought/distarray

All issues in enthought/distarray

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.