environments: changing deployment_protection_rules deletes and re-creates all rules

Open Beginner friendly
#1,134 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
76/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
github, javascript
Domain
backend

Research direction

Read lib/plugins/environments.js, especially the change detection and update logic around the referenced lines 134 and 233–249. Start by tracing how configured app_id values are compared with the existing rules returned by the deployment protection rules API. Add or run the supplied mocked unit tests; done means only added rules are POSTed, only removed rules are DELETEd, and unchanged rules produce no writes.

Written by the indexing model from the issue text.

Description

Problem Description

What is actually happening

Changing an environment's deployment_protection_rules deletes every existing custom protection rule, then POSTs every configured rule, including ones that already exist.

Change detection compares app_ids correctly. But in update(), configured rules are matched against existing ones with _rule.id === rule.id. Config entries only have app_id, so nothing matches and every configured rule is POSTed. After that, the plugin deletes every entry in existingRules without filtering.

For example, with an existing rule for app 1 and config [1, 2], the plugin sends:

  1. POST .../deployment_protection_rules {integration_id: 1}
  2. POST .../deployment_protection_rules {integration_id: 2}
  3. DELETE .../deployment_protection_rules/{id of the app 1 rule}

Removing a rule has the same problem. With config [1] and existing rules for apps [1, 3], the plugin sends POST 1, DELETE (app 1 rule) and DELETE (app 3 rule).

These request sequences come from a mocked unit test. I haven't checked what GitHub does with the duplicate POST for app 1. If it is rejected, the rest of the update is likely aborted. If it succeeds, the app 1 rule is deleted right after.

What is the expected behavior

Only the difference is applied:

  • config [1, 2], existing [1]: only POST {integration_id: 2}
  • config [1], existing [1, 3]: only DELETE (app 3 rule)

When the app_id sets are equal, no requests are sent. That already works.

Error output, if available

There's no runtime error. Output of the failing unit tests below:

✕ BUG: adding a rule keeps the existing one (POST only the new app, no DELETE)

expect(received).toEqual(expected) // deep equality

- Expected  -  0
+ Received  + 18

  Array [
    Object {
      "opts": Object {
        "environment_name": "prod",
+       "integration_id": 1,
+       "org": "my-org",
+       "repo": "my-repo",
+     },
+     "route": "POST /repos/:org/:repo/environments/:environment_name/deployment_protection_rules",
+   },
+   Object {
+     "opts": Object {
+       "environment_name": "prod",
        "integration_id": 2,
        "org": "my-org",
        "repo": "my-repo",
      },
      "route": "POST /repos/:org/:repo/environments/:environment_name/deployment_protection_rules",
+   },
+   Object {
+     "opts": Object {
+       "environment_name": "prod",
+       "org": "my-org",
+       "repo": "my-repo",
+       "rule_id": 101,
+     },
+     "route": "DELETE /repos/:org/:repo/environments/:environment_name/deployment_protection_rules/:rule_id",
    },
  ]

✕ BUG: removing a rule deletes only that rule and re-creates nothing

expect(received).toEqual(expected) // deep equality

- Expected  -  0
+ Received  + 18

@@ -1,9 +1,27 @@
  Array [
    Object {
      "opts": Object {
        "environment_name": "prod",
+       "integration_id": 1,
+       "org": "my-org",
+       "repo": "my-repo",
+     },
+     "route": "POST /repos/:org/:repo/environments/:environment_name/deployment_protection_rules",
+   },
+   Object {
+     "opts": Object {
+       "environment_name": "prod",
+       "org": "my-org",
+       "repo": "my-repo",
+       "rule_id": 101,
+     },
+     "route": "DELETE /repos/:org/:repo/environments/:environment_name/deployment_protection_rules/:rule_id",
+   },
+   Object {
+     "opts": Object {
+       "environment_name": "prod",
        "org": "my-org",
        "repo": "my-repo",
        "rule_id": 102,
      },
      "route": "DELETE /repos/:org/:repo/environments/:environment_name/deployment_protection_rules/:rule_id",
Steps to reproduce

Environment prod on my-org/my-repo already has a custom protection rule for app 1. Change the config to:

environments:
  - name: prod
    deployment_protection_rules:
      - app_id: 1
      - app_id: 2
Code references
Failing test

Fails on 3.0.0 and on main-enterprise. The control test (equal app_id sets produce no writes) passes.

it('BUG: adding a rule keeps the existing one (POST only the new app, no DELETE)', async () => {
  const github = mockGithub({
    prod: { api: apiEnv('prod'), rules: [{ id: 101, app: { id: 1 } }] }
  })
  const plugin = new Environments(false, github, { owner: org, repo }, [
    { name: 'prod', deployment_protection_rules: [{ app_id: 1 }, { app_id: 2 }] }
  ], log, [])
  await plugin.sync()
  expect(writeCalls(github)).toEqual([
    { route: POST_RULE, opts: { org, repo, environment_name: 'prod', integration_id: 2 } }
  ])
})

it('BUG: removing a rule deletes only that rule and re-creates nothing', async () => {
  const github = mockGithub({
    prod: { api: apiEnv('prod'), rules: [{ id: 101, app: { id: 1 } }, { id: 102, app: { id: 3 } }] }
  })
  const plugin = new Environments(false, github, { owner: org, repo }, [
    { name: 'prod', deployment_protection_rules: [{ app_id: 1 }] }
  ], log, [])
  await plugin.sync()
  expect(writeCalls(github)).toEqual([
    { route: DELETE_RULE, opts: { org, repo, environment_name: 'prod', rule_id: 102 } }
  ])
})
Suggested fix

Match on app_id. POST configured rules whose app_id isn't among the existing ones, and DELETE only existing rules whose app_id isn't in the config.

Context

Are you using the hosted instance of probot/settings or running your own?

Running our own. I reproduced this with unit tests against the code, using a mocked GitHub API, so it doesn't depend on how the app is deployed.

If running your own instance, are you using it with github.com or GitHub Enterprise?

Not relevant here: the reproduction uses a mocked API, and the code path is the same for both.

Version of probot/settings

safe-settings 3.0.0 (a9a1f366ad6a738b55682d346d94f8d6c4893b09), and main-enterprise at e19444e0d69de050ba55d0262b71fcd8a98edfef.

Version of GitHub Enterprise

N/A

Test helpers (mocks)
/* eslint-disable no-undef */
// Reproductions for reported Environments plugin bugs. Tests assert the
// CORRECT behaviour; the ones marked "BUG" fail against the current code.
const { when } = require('jest-when')
const Environments = require('../../../../lib/plugins/environments')
const Settings = require('../../../../lib/settings')
const NopCommand = require('../../../../lib/nopcommand')

const org = 'my-org'
const repo = 'my-repo'
const DELETE_ENV = 'DELETE /repos/:org/:repo/environments/:environment_name'
const PUT_ENV = 'PUT /repos/:org/:repo/environments/:environment_name'
const POST_RULE = 'POST /repos/:org/:repo/environments/:environment_name/deployment_protection_rules'
const DELETE_RULE = 'DELETE /repos/:org/:repo/environments/:environment_name/deployment_protection_rules/:rule_id'
const DELETE_VAR = 'DELETE /repos/:org/:repo/environments/:environment_name/variables/:variable_name'

// GitHub API shape of GET /repos/{owner}/{repo}/environments entries.
function apiEnv (name, { waitTimer, reviewers, preventSelfReview, branchPolicy = null } = {}) {
  const protectionRules = []
  if (waitTimer) protectionRules.push({ id: 1, type: 'wait_timer', wait_timer: waitTimer })
  if (reviewers) {
    protectionRules.push({
      id: 2,
      type: 'required_reviewers',
      prevent_self_review: !!preventSelfReview,
      reviewers: reviewers.map(r => ({ type: r.type, reviewer: { id: r.id } }))
    })
  }
  return { name, protection_rules: protectionRules, deployment_branch_policy: branchPolicy }
}

// existing: { [envName]: { api, variables: [{name,value}], rules: [{id, app: {id}}] } }
function mockGithub (existing) {
  const github = {
    request: jest.fn(() => Promise.resolve({})),
    paginate: jest.fn().mockResolvedValue([])
  }
  when(github.request)
    .calledWith('GET /repos/:org/:repo/environments', { org, repo })
    .mockResolvedValue({ data: { environments: Object.values(existing).map(e => e.api) } })
  for (const [name, e] of Object.entries(existing)) {
    when(github.request)
      .calledWith('GET /repos/:org/:repo/environments/:environment_name/deployment_protection_rules', { org, repo, environment_name: name })
      .mockResolvedValue({ data: { total_count: (e.rules || []).length, custom_deployment_protection_rules: e.rules || [] } })
    when(github.paginate)
      .calledWith('GET /repos/{owner}/{repo}/environments/{environment_name}/variables', expect.objectContaining({ environment_name: name }), expect.any(Function))
      .mockResolvedValue(e.variables || [])
  }
  return github
}

function writeCalls (github) {
  return github.request.mock.calls
    .filter(([route]) => !route.startsWith('GET '))
    .map(([route, opts]) => ({ route, opts }))
}

const log = { debug: jest.fn(), info: jest.fn(), warn: jest.fn(), error: jest.fn() }
Dominant language
JavaScript
Stars
942
Forks
238
PR merge metrics
No merged PRs in 30d

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from github-community-projects/safe-settings

All issues in github-community-projects/safe-settings

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.