[import/order] non-deterministic behavior with "alphabetize" and "pathGroups"

Long story short, I'm getting some extremely weird and seemingly non-deterministic behavior when using import/order with alphabetize and pathGroups.

The import/order section of my config is the following:

{
    rules: {
        // ...
        "import/order": [
            "error",
            {
                groups: [
                    ["builtin", "external"],
                    "internal",
                    ["sibling", "parent"],
                    "object",
                    "type",
                    "index",
                    "unknown",
                ],
                alphabetize: {
                    order: "asc",
                    orderImportKind: "asc",
                    caseInsensitive: true,
                },
                pathGroups: [
                    {
                        pattern: "~/**",
                        group: "internal",
                    },
                ],
            },
        ],
        // ...
    },
    settings: {
        // ...
        "import/resolver": {
            typescript: {
                project: ["./tsconfig.json", "packages/*/tsconfig.json"],
            },
            node: true,
        }
        // ...
    }
}

And now, in a few files, I'm trying to import from that ~/** path group.

import { /* ... */ } from "~/models.js";
import { /* ... */ } from "~/util.js";

This, as expected, conforms to the rule, both within VSC and in the terminal. Except, that's not always true...

The first time I run the linter in the terminal, things are fine. However, if I run it again (while doing absolutely nothing else other than "up arrow + enter"), it now fails:

error  `~/util.js` import should occur before import of `~/models.js`  import/order

That's not alphabetized correctly now...

If I run it a few more times, it'll pass about 1-in-5 to 1-in-6 ish times, failing with that error the other times. If I then swap the imports, the opposite becomes true and it fails in VSC until I Restart ESLint Server where it also switches behavior like the terminal.

Environment:

"eslint": "8.39.0"
"eslint-plugin-import": "2.27.5"
"node": "16.16.0"

Not sure how to best create a minimal reproducible because it's extremely inconsistent on where and how the error appears.

To me it feels like some sort of bad cache/state is being kept between runs somehow, but I'm unfamiliar with the inner workings of eslint.