@databricks/sdk-experimental (pinned by AppKit): SCIM v2 list() iterators never advance startIndex and never terminate

Open
  • Is something wrong?
#656 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 3 days

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
Half a day
Newbie friendliness
42/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
javascript, typescript
Domain
api

Research direction

The bug is in the shipped dist/apis/iam/api.js of @databricks/sdk-experimental, in the async *list generators of the six SCIM V2 services (for example ServicePrincipalsV2Service). Read that loop first: request never changes, so startIndex never advances. Done means each generator adds the page length to startIndex and stops on an empty page, verified with a mocked SCIM response. The source likely lives in the upstream SDK repo, not this one, so confirm where to make the change first.

Written by the indexing model from the issue text.

Description

Summary

The six SCIM v2 list() async iterators in @databricks/sdk-experimental request the same page on every pass. When the result is not empty they yield the first page again and again and never finish.

AppKit pins @databricks/sdk-experimental 0.17.0 and gives apps its WorkspaceClient, so an AppKit app that iterates users, groups or service principals this way hangs and keeps calling the SCIM API.

I am filing this here because the SDK's own tracker (the bugs.url in its package.json, github.com/databricks/databricks-sdk-js/issues) is not publicly reachable. Please move it if there is a better place.

Affected
  • @databricks/sdk-experimental 0.17.0 and 0.18.0 (the latest), in dist/apis/iam/api.js.
  • Workspace client: usersV2.list, groupsV2.list, servicePrincipalsV2.list.
  • Account client: accountUsersV2.list, accountGroupsV2.list, accountServicePrincipalsV2.list.

I checked every async iterator in 0.18.0 (127): these six are the only ones that loop on an unchanged request.

Code as shipped (0.18.0, ServicePrincipalsV2Service)
async *list(request, context) {
    while (true) {
        const response = await this._list(request, context);
        if (context?.cancellationToken &&
            context?.cancellationToken.isCancellationRequested) {
            break;
        }
        if (!response.Resources || response.Resources.length === 0) {
            break;
        }
        for (const v of response.Resources) {
            yield v;
        }
    }
}

request is never changed, so startIndex stays where the caller set it. The loop ends only on an empty page, and a first page that has items never becomes empty.

Reproduce

On any workspace that has at least one service principal:

import {WorkspaceClient} from "@databricks/sdk-experimental";

const w = new WorkspaceClient({});
let n = 0;
for await (const sp of w.servicePrincipalsV2.list({})) {
  n++;
  if (n > 10_000) throw new Error("still iterating"); // always reached
}

The same items come back in the same order on every pass.

Expected

After each page, add the number of items returned to startIndex and stop on an empty page. The modular SDK already does this in databricks/sdk-js, packages/scim/src/v1/client.ts:

pageReq.startIndex = (resp.startIndex ?? 0n) + BigInt(items.length);
Workaround

Page explicitly: call the SCIM list endpoint with startIndex and count, and add each page's length to startIndex until a page comes back empty.

Dominant language
TypeScript
Stars
119
Forks
36
Avg merge
4d 16h
Merged PRs (30d)
37

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from databricks/appkit

All issues in databricks/appkit

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.