Angel-ML/angel

[CORE] RDMA Support for Angel

Open

#225 opened on Oct 11, 2017

View on GitHub
 (0 comments) (5 reactions) (0 assignees)Java (6,686 stars) (1,630 forks)batch import
help wanted

Description

In Angel, there's lots of model transformation, which means it will bring large amount of network transformation and costs lots of CPU time.

To improve this, we would like to introduce RDMA, which can bring better network performance and save CPU time. And with these improvement, Angel can achieve better performance and speed up model training time.

Contributor guide

Tech stack
java
Domain
machine learningperformance
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
5
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
needs investigation
Prerequisites
RDMA knowledgeJava networkingAngel architecture
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
10
Research direction
Start by examining the network communication layer in Angel, likely in the `angel ps core` module. Research RDMA libraries for Java, such as JVerbs or the RDMA CM bindings. Propose a high level design for integrating RDMA with the existing parameter server architecture, focusing on model transformation data flows. Look for any existing discussions or stalled PRs in the repository.
[CORE] RDMA Support for Angel · Angel-ML/angel#225 | Good First Issue