Parameter management is a performance sinkhole for large documents

Is there an existing issue for this?

• I have searched the existing issues

Problem description

In the process of importing or opening large documents (sample has 40000 objects), a considerable fraction of the time is spent looking up "Parameters". These represent the chosen options from the Edit->Options dialog. The existing code does linear searches using string equality within an XML document on each lookup. These parameters are consulted several times during each object creation, for instance to determine if Labels should be unique in the document. FortyThousandLines.zip

operation	total time	percent spent in Parameters code
Import FortyThousandLines.dxf	1:27	27%
Save imported file	0:14	5%
Re-open saved file	1:34	22%

Full version info

Timings are based on the commit in PR #17944 which removes other gross inefficiencies masking this particular one. That PR is based on commit 45134cef.

OS: Windows 10 build 19045
Architecture: x86_64
Version: 1.1.0dev.39288 (Git)
Build type: Release
Branch: (no branch, rebasing Issue16849)
Hash: 45134cef165ecc11aca69f81673e7b69421cebf1
Python 3.12.4, Qt 6.7.2, Coin 4.0.2, Vtk 9.3.0, OCC 7.8.0
Locale: English/Canada (en_CA)
Stylesheet/Theme/QtStyle: FreeCAD Light.qss/FreeCAD Classic/

Subproject(s) affected?

None

Anything else?

The existing code seems to be trying to keep the XML up to date as parameters are changed even though it is keeping a private copy of the XML. It would seem that better performance would be obtained if the XML were converted to a tree structure of objects and discarded, to be re-created if the Settings have to be saved again.

I'm not sure if there is any need here to preserve foreign content (e.g. comments, unrecognized elements) in the XML.

The code does create (on demand) a tree of objects for the nested groups, but still fondles the XML each time these are the subject of a lookup.

There is also some code so that these nodes representing the groups are resurrected rather than newly-created if they are deleted and created again, and other code holds a handle to them. The comments allude to observers but don't really describe why this behaviour, which makes the code more complex, is necessary. If anything the comments should elaborate a bit more on what is being accomplished by doing this.

Also, there are methods to obtain several setting values as a sequence based on a name pattern, but no methods to create such a sequence in the XML file. The actual order of the resulting vector is undefined, although it turns out to return these in the order they exist in the XML file and so in the order the original individual values were set, provided the code starts with a blank slate. This is not necessarily alphabetical order if, for instance, the names of the individual items are numbered with variable-width numbers. It would probably be a good idea for formalize the methods for saving such a list so these can have a well-defined ordering. This is used for most-recently-used lists and search path lists, both of which should be well-ordered.

Code of Conduct

• I agree to follow this project's Code of Conduct