[API] `flashdreams.core` Refactor
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Refactor
- Clarity
- Needs clarification
- Activity status
- Quiet
- Tech stack
- python
- Domain
- backend, distributed-systems, performance
Research direction
Start by reviewing dependent issues #473 and #482, then inspect flashdreams/flashdreams/core/attention, flashdreams/core/distributed, flashdreams/infra/acceleration, and the proposed flashdreams.accelerated work. Map the current modules, acceleration, distributed features, and StreamInferencePipeline before changing layout. Done means the layered core-to-pipeline-to-runtime structure is in place and the obsolete flashdreams/infra code and pipeline are removed.
Written by the indexing model from the issue text.
Description
Issue Description
Current flashdreams spreads the "core level" feature in multiple places under flashdreams/flashdreams. We need to refactor/reorganize the code layout to have a layered structure (flashdreams.core -> flashdreams.pipeline -> flashdreams.runtime)
Proposed Solution
The following features should go into the new flashdreams.core:
flashdreams.core.modules: accelerated flashdreams modules with their Triton kernel. Currently, they are in https://github.com/NVIDIA/flashdreams/tree/main/flashdreams/flashdreams/core/attention (special rope module & rope triton kernel) and my newflashdreams.acceleratedPR (WIP)flashdreams.core.acceleration: CUDA graph, prewarm, and future flashdreams auto tune system. These feature are currently inflashdreams/infra/accelerationflashdreams.core.distributed: Distributed related features. Including context parallel (currently inflashdreams/core/attention/cp.pyandflashdreams/core/distributed), rank orchestration (currently inflashdreams/core/distributed)
We also need to remove all the old StreamInferencePipeline in favor of the composable inference pipeline once implemented. This should mean that we no longer need flashdreams/infra after the refactor.
Dependency
This issue is gated by #473 and #482
- Dominant language
- Python
- Stars
- 510
- Forks
- 61
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 49
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from NVIDIA/flashdreams
-
Priority-P2 Type-clean up
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
NVIDIA/flashdreams#590 ·
Maintainers usually reply within 1 day
-
Priority-P2
Difficulty 5/5 Over a week Newbie friendliness 35/100
NVIDIA/flashdreams#673 ·
Maintainers usually reply within 1 day
-
[Crazy Robotaxi] Add README detailing game settings.Possibly taken @aidanfnv claimed this 3 days ago. Open
NVIDIA/flashdreams#662 · 1 assignee ·
Maintainers usually reply within 1 day
-
Finish the v2 migration: rename `integrations_v2` and `flashdreams-run-v2`Possibly taken @wlewNV claimed this 1 day ago. OpenPriority-P1
Difficulty 5/5 Over a week Newbie friendliness 28/100
NVIDIA/flashdreams#661 · 1 assignee ·
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
NVIDIA/flashdreams#657 ·
Maintainers usually reply within 1 day
All issues in NVIDIA/flashdreams
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
BasedHardware/omi#20271 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 92/100
openai/openai-cookbook#3153 ·
Maintainers usually reply within 1 day
-
cvss-severity:high devguard l3montree-cybersecurity/devguard/devguard pkg:golang/github.com/l3montree-dev/devguard risk:low state:open
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
l3montree-dev/devguard#3146 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
bug confirmed issue
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
open-webui/open-webui#31849 · 2 comments ·
Maintainers usually reply within 1 day