Separate base configuration from configuration updates

The issue names no files, tests, or entry points, and says the suggested implementation is flawed. Start by reviewing the project's configuration handling and unresolved discussion, then define the accepted base/customisation model and validation behavior; done should cover compatible updates plus add, modify, and remove cases.

Is your feature request related to a problem? Please describe.

Consider a device with a standard configuration. Then consider a user with a slightly modified config file.

If the standard config file is changed in a update. The user has no way to distil their changed settings, they just have to remember (or manually compare to an old un-updated config file).

It would be nice to be able to update the base config file without loosing customisations (unless they are incompatible).

EDITED TO ADD: The suggested implementation is a bad idea. See comment below

Describe the solution you'd like

The option to have a base config file and a customisation file:

Base File:

{
    "things": {
        "/camera/": {
            "class": "openflexure_microscope_server.things.camera.picamera:StreamingPiCamera2",
            "kwargs": {
                "camera_board": "picamera_v2"
            }
        },
        "/stage/": "openflexure_microscope_server.things.stage.sangaboard:SangaboardThing",
        "/autofocus/": "openflexure_microscope_server.things.autofocus:AutofocusThing",
        "/camera_stage_mapping/": "openflexure_microscope_server.things.camera_stage_mapping:CameraStageMapper",
        "/system/": "openflexure_microscope_server.things.system:OpenFlexureSystem",
        "/smart_scan/": {
            "class": "openflexure_microscope_server.things.smart_scan:SmartScanThing",
            "kwargs": {
                "scans_folder": "/var/openflexure/scans/"
            }
        },
        "/stage_measure/":"openflexure_microscope_server.things.stage_measure:RangeofMotionThing"
    },
    "settings_folder": "/var/openflexure/settings/",
    "log_folder": "/var/openflexure/logs/"
}

Modifications - File example 1: Modifying Thing kwargs.

{
    "modifyThings": {
        "/camera/": {
            "kwargs": {
                "camera_board": "picamera_hq"
            }
        }
    }
}

Modifications - File example 2: Modifying Thing class.

{
    "modifyThings": {
        "/stage/": {
            "class": "path_to.custom_stage:CustomStageThing"
    }
}

Modifications - File example 3: Add Thing.

{
    "addThings": {
        "/horn/": "path_to.a_loud:BeepBeepThing"
    }
}

Modifications - File example 4: Remove Thing.

{
    "removeThings": ["/stage_measure/"]
}

Further discussion

I have shown them individually, but you could have modifyThings, addThings, removeThings in the same file.

It seems like this would play nicely with the mythical Congifuration Server?

It is possible that a new config could still break this as it could:

• Want to modify something that does not exist (or supplies an bad kwarg)
• Tries to add something with the same name as something that now exists
• Tries to remove a things that is no longer there

This however would give a clear error state. It seems that most times we change the base file it is to add/remove a kwarg or to add a new Thing or remove a now redundant one. This seems like it is unlikley to regularly crash.