environments: nop mode reports no planned changes

Open Beginner friendly
#1,135 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
github, javascript
Domain
backend, devtools

Research direction

Start in lib/plugins/environments.js at add(), update(), remove(), and the sync() return path referenced in the issue. Read the failing test described in the issue and trace how its NopCommand results reach Settings.appendToResults() in lib/settings.js. Done means the test sees a NopCommand whose action message is "Delete environment" in nop mode.

Written by the indexing model from the issue text.

Description

Problem Description

What is actually happening

In nop (dry-run) mode, the environments plugin reports none of its planned changes, so dry-run PR checks never show environment changes. sync() resolves to [undefined].

add(), update() and remove() await this.nopifyRequest(...) but don't return the result, so resArray collects undefined. sync() also returns resArray without waiting for the pending changes. Settings.appendToResults() then drops the falsy entries, so nothing about environments reaches the dry-run output.

What is the expected behavior

The dry-run results include the planned environment changes, for example "Delete environment" for an environment that is no longer in the config.

Error output, if available

There's no runtime error. Output of the failing unit test below:

✕ BUG: sync() in nop mode returns the planned DELETE environment as a NopCommand
  Expected value: "Delete environment"
  Received array: []
Steps to reproduce

Repo my-org/my-repo has environments test and prod. Open a PR on the admin repo that changes settings.yml to:

environments:
  - name: prod

The dry-run check shows nothing about deleting test.

Code references
Failing test

Fails on 3.0.0 and on main-enterprise.

it('BUG: sync() in nop mode returns the planned DELETE environment as a NopCommand', async () => {
  const github = mockGithub({ prod: { api: apiEnv('prod') }, old: { api: apiEnv('old') } })
  const plugin = new Environments(true, github, { owner: org, repo }, [{ name: 'prod' }], log, [])
  const res = await plugin.sync()
  const commands = [res].flat(3).filter(x => x instanceof NopCommand)
  expect(commands.map(c => c.action.msg)).toContain('Delete environment')
})
Suggested fix

Return the nopifyRequest results from add/update/remove, and await Promise.all(changes) before returning resArray in nop mode.

This is separate from #1133: fixing the additive check there doesn't change what nop mode reports.

Context

Are you using the hosted instance of probot/settings or running your own?

Running our own. I reproduced this with unit tests against the code, using a mocked GitHub API, so it doesn't depend on how the app is deployed.

If running your own instance, are you using it with github.com or GitHub Enterprise?

Not relevant here: the reproduction uses a mocked API, and the code path is the same for both.

Version of probot/settings

safe-settings 3.0.0 (a9a1f366ad6a738b55682d346d94f8d6c4893b09), and main-enterprise at e19444e0d69de050ba55d0262b71fcd8a98edfef.

Version of GitHub Enterprise

N/A

Test helpers (mocks)
/* eslint-disable no-undef */
// Reproductions for reported Environments plugin bugs. Tests assert the
// CORRECT behaviour; the ones marked "BUG" fail against the current code.
const { when } = require('jest-when')
const Environments = require('../../../../lib/plugins/environments')
const Settings = require('../../../../lib/settings')
const NopCommand = require('../../../../lib/nopcommand')

const org = 'my-org'
const repo = 'my-repo'
const DELETE_ENV = 'DELETE /repos/:org/:repo/environments/:environment_name'
const PUT_ENV = 'PUT /repos/:org/:repo/environments/:environment_name'
const POST_RULE = 'POST /repos/:org/:repo/environments/:environment_name/deployment_protection_rules'
const DELETE_RULE = 'DELETE /repos/:org/:repo/environments/:environment_name/deployment_protection_rules/:rule_id'
const DELETE_VAR = 'DELETE /repos/:org/:repo/environments/:environment_name/variables/:variable_name'

// GitHub API shape of GET /repos/{owner}/{repo}/environments entries.
function apiEnv (name, { waitTimer, reviewers, preventSelfReview, branchPolicy = null } = {}) {
  const protectionRules = []
  if (waitTimer) protectionRules.push({ id: 1, type: 'wait_timer', wait_timer: waitTimer })
  if (reviewers) {
    protectionRules.push({
      id: 2,
      type: 'required_reviewers',
      prevent_self_review: !!preventSelfReview,
      reviewers: reviewers.map(r => ({ type: r.type, reviewer: { id: r.id } }))
    })
  }
  return { name, protection_rules: protectionRules, deployment_branch_policy: branchPolicy }
}

// existing: { [envName]: { api, variables: [{name,value}], rules: [{id, app: {id}}] } }
function mockGithub (existing) {
  const github = {
    request: jest.fn(() => Promise.resolve({})),
    paginate: jest.fn().mockResolvedValue([])
  }
  when(github.request)
    .calledWith('GET /repos/:org/:repo/environments', { org, repo })
    .mockResolvedValue({ data: { environments: Object.values(existing).map(e => e.api) } })
  for (const [name, e] of Object.entries(existing)) {
    when(github.request)
      .calledWith('GET /repos/:org/:repo/environments/:environment_name/deployment_protection_rules', { org, repo, environment_name: name })
      .mockResolvedValue({ data: { total_count: (e.rules || []).length, custom_deployment_protection_rules: e.rules || [] } })
    when(github.paginate)
      .calledWith('GET /repos/{owner}/{repo}/environments/{environment_name}/variables', expect.objectContaining({ environment_name: name }), expect.any(Function))
      .mockResolvedValue(e.variables || [])
  }
  return github
}

function writeCalls (github) {
  return github.request.mock.calls
    .filter(([route]) => !route.startsWith('GET '))
    .map(([route, opts]) => ({ route, opts }))
}

const log = { debug: jest.fn(), info: jest.fn(), warn: jest.fn(), error: jest.fn() }
Dominant language
JavaScript
Stars
942
Forks
238
PR merge metrics
No merged PRs in 30d

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from github-community-projects/safe-settings

All issues in github-community-projects/safe-settings

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.