Ability to define number of CPU/GPUs to be used when creating the TF Session
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- scala, spark, tensorflow
- Domain
- distributed-systems, machine-learning
Research direction
Start by locating where tensorframes creates the TensorFlow Session inside Spark executor nodes, then read the surrounding resource and configuration handling. Define how callers would specify CPU and GPU counts and how completion should be verified without claiming resources from unrelated workloads; the issue names no files or tests, so the relevant entry point and validation approach must first be established.
Written by the indexing model from the issue text.
Description
Tensorflow provides means to specify the number of CPU/GPUs to be used for each Session. Otherwise, it will try to use all CPUs available in the machine. Currently tensorframes does provide any means to specify this information and it ends up tensorflow using all the CPU cores in the executor nodes. This is more problematic when an executor has more than one cores assigned which results in more contention. An alternative would be to set only one core for each executor but this is not optimal in mix workloads where CNN inference is only a part of the workload.
- Dominant language
- Scala
- Stars
- 743
- Forks
- 158
- PR merge metrics
- No merged PRs in 30d
Getting set up
- Ships a Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from databricks/tensorframes
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
databricks/tensorframes#185 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
databricks/tensorframes#181 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 30/100
databricks/tensorframes#179 ·
-
Difficulty 5/5 Over a week Newbie friendliness 15/100
databricks/tensorframes#176 · 2 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 30/100
databricks/tensorframes#165 ·
All issues in databricks/tensorframes
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
scala-native/scala-native#5238 ·
Maintainers usually reply within 1 day
-
CI/CD ⚒️ Flaky-tests 🐦 Rust ⚙️
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
valkey-io/valkey-glide#7658 ·
Maintainers usually reply within 1 day
-
[kunminghu-v3] Debug SBA 32/64-bit accesses hang: missing TLWidthWidget(1) on error_xbar pathPossibly taken A pull request linked to this issue is open or already merged. Openmodule: backend type: bug/reported
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
OpenXiangShan/XiangShan#6938 ·
Maintainers usually reply within 3 days
-
documentation good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Babali42/DrumBeatRepo#639 ·
Maintainers usually reply within 1 day
-
module: core presentation conversion
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
bigbluebutton/bigbluebutton#26508 ·
Maintainers usually reply within 1 day