Very high memory usage when running under sudo in Linux
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 32/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- linux, python
- Domain
- backend, operating-systems
Research direction
Start with the minimal Python reproducer and compare the memory readings before and after softioc.iocInit when run normally and under sudo. Investigate the initialization path reached by softioc.iocInit; done means the sudo run no longer shows the roughly 950 MB increase while preserving normal IOC startup.
Written by the indexing model from the issue text.
Description
Hi all,
It seems that creating and initializing an IOC takes up a crazy high amount of memory when ran under sudo. This is not the case when ran as user. Here is a minimal example to demonstrate:
import psutil
from softioc import softioc, builder, asyncio_dispatcher
def print_memory_usage(loc: str):
print(f"{loc:32s}: {psutil.Process().memory_info().rss:,}")
print_memory_usage("Before PV")
pv = builder.longOut("myVariable")
print_memory_usage("Before loadDatabase")
builder.LoadDatabase()
print_memory_usage("Before dispatcher")
dispatcher = asyncio_dispatcher.AsyncioDispatcher()
print_memory_usage("Before iocInit")
softioc.iocInit(dispatcher)
print_memory_usage("Before start")
softioc.interactive_ioc()
Output when ran as user:
INFO: PVXS QSRV2 is loaded, permitted, and ENABLED.
2025-01-16T13:55:56.327190606 WARN pvxs.tcp.setup Server unable to bind port 5075, falling back to [::]:37275
Before PV : 85,139,456
Before loadDatabase : 85,139,456
Before dispatcher : 85,139,456
Before iocInit : 87,236,608
Starting iocInit
############################################################################
## EPICS 7.0.7.1-DEV
## Rev. 7.0.7.99.1.1
## Rev. Date 7.0.7.99.1.1
############################################################################
cas WARNING: Configured TCP port was unavailable.
cas WARNING: Using dynamically assigned TCP port 43649,
cas WARNING: but now two or more servers share the same UDP port.
cas WARNING: Depending on your IP kernel this server may not be
cas WARNING: reachable with UDP unicast (a host's IP in EPICS_CA_ADDR_LIST)
iocRun: All initialization complete
Before start : 89,600,000
Python 3.12.3 (main, Nov 6 2024, 18:32:19) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>>
Output when ran as sudo:
INFO: PVXS QSRV2 is loaded, permitted, and ENABLED.
2025-01-16T13:56:32.146488207 WARN pvxs.tcp.setup Server unable to bind port 5075, falling back to [::]:39775
Before PV : 85,114,880
Before loadDatabase : 85,114,880
Before dispatcher : 85,114,880
Before iocInit : 87,212,032
Starting iocInit
############################################################################
## EPICS 7.0.7.1-DEV
## Rev. 7.0.7.99.1.1
## Rev. Date 7.0.7.99.1.1
############################################################################
cas WARNING: Configured TCP port was unavailable.
cas WARNING: Using dynamically assigned TCP port 37929,
cas WARNING: but now two or more servers share the same UDP port.
cas WARNING: Depending on your IP kernel this server may not be
cas WARNING: reachable with UDP unicast (a host's IP in EPICS_CA_ADDR_LIST)
iocRun: All initialization complete
Before start : 950,546,432
Python 3.12.3 (main, Nov 6 2024, 18:32:19) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>>
As you can see, the issue seems to happen in softioc.iocInit. Packages used to test (on Ubuntu):
softioc 4.5.0
psutil 6.1.1
This issue occurs both on Ubuntu and NixOS.
- Dominant language
- Python
- Stars
- 47
- Forks
- 15
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from DiamondLightSource/pythonSoftIOC
-
bug
Difficulty 4/5 3-5 days Newbie friendliness 30/100
-
enhancement
DiamondLightSource/pythonSoftIOC#121 · 2 comments · 1 assignee ·
-
enhancement
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
enhancement
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Replace devIocStats dependency with python device support implementing the same record interface Openenhancement
Difficulty 5/5 Over a week Newbie friendliness 20/100
All issues in DiamondLightSource/pythonSoftIOC
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
learningequality/ricecooker#747 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
BSData/horus-heresy-3rd-edition#3171 ·
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
run-llama/llama_index#23199 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
KhronosGroup/glTF-Blender-IO#2769 ·