environments: additive_plugins is ignored, undeclared environments are deleted

Open Beginner friendly
#1,133 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

@yaunj is already working on this.

Since Nov 9, 2026.

  • #1138 by @yaunj — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
github, javascript, node.js
Domain
api, backend

Research direction

Start with lib/plugins/environments.js and read Environments.sync() around lines 324–351, then compare its removal logic with the additive handling in lib/plugins/diffable.js. The issue includes focused reproductions for Settings.updateRepos() and nop mode; run the relevant plugin tests or add these cases. Done means additive mode leaves undeclared environments untouched and reports suppressed deletions in nop mode.

Written by the indexing model from the issue text.

Description

Problem Description

What is actually happening

With additive_plugins: [environments], environments that exist on the repo but are not in the YAML are still deleted (DELETE /repos/{owner}/{repo}/environments/{name}). In nop mode the deletion is planned, and no "deletion(s) suppressed" message is shown.

The README lists environments as supported ("Extra environments not in YAML are kept"), and Settings sets .additive = true on the Environments instance. But Environments overrides sync() with its own loop. That loop calls this.remove() for every undeclared environment and never checks this.additive. The additive check only exists in Diffable.sync().

What is the expected behavior

Undeclared environments are left alone. In nop mode, the "Additive mode active: N deletion(s) suppressed" message is shown, as it is for other additive plugins.

Error output, if available

There's no runtime error. Output of the failing unit test below:

✕ BUG: via Settings.updateRepos with additive_plugins: [environments], nop=false: DELETE environment is not sent
    Expected: not "DELETE /repos/:org/:repo/environments/:environment_name", Anything
    Received
           4:     "DELETE /repos/:org/:repo/environments/:environment_name", {"environment_name": "team-owned", "org": "my-org", "repo": "my-repo"}

✕ BUG: additive=true, nop=true: no deletion is planned
    Expected: not "DELETE /repos/:org/:repo/environments/:environment_name", Anything, Anything
    Received:     0, ["DELETE /repos/:org/:repo/environments/:environment_name", {"environment_name": "team-owned", "org": "my-org", "repo": "my-repo"}, "Delete environment"]
Steps to reproduce

Repo my-org/my-repo has environments test and prod. .github/settings.yml:

additive_plugins:
  - environments

environments:
  - name: prod

On sync, DELETE /repos/my-org/my-repo/environments/test is sent.

Code references
Failing test

The first test uses the real Settings.updateRepos() wiring and checks that additive === true reaches the plugin before asserting on the request. Both tests fail on 3.0.0 and on main-enterprise.

const existing = () => ({
  prod: { api: apiEnv('prod') },
  'team-owned': { api: apiEnv('team-owned') }
})

it('BUG: via Settings.updateRepos with additive_plugins: [environments], nop=false: DELETE environment is not sent', async () => {
  const github = mockGithub(existing())
  const context = { payload: { installation: { id: 1 } }, octokit: github, log }
  const settings = new Settings(false, context, { owner: org, repo }, {
    additive_plugins: ['environments'],
    environments: [{ name: 'prod' }]
  }, 'main')
  settings.subOrgConfigs = {}
  settings.repoConfigs = {}

  // Record the additive flag the real wiring sets on the real plugin instance.
  const seenAdditive = []
  const realSync = Environments.prototype.sync
  jest.spyOn(Environments.prototype, 'sync').mockImplementation(function () {
    seenAdditive.push(this.additive)
    return realSync.call(this)
  })

  await settings.updateRepos({ owner: org, repo }, false)

  expect(seenAdditive).toEqual([true]) // wiring works: .additive reaches Environments
  expect(github.request).not.toHaveBeenCalledWith(DELETE_ENV, expect.anything())
})

it('BUG: additive=true, nop=true: no deletion is planned', async () => {
  const github = mockGithub(existing())
  const plugin = new Environments(true, github, { owner: org, repo }, [{ name: 'prod' }], log, [])
  plugin.additive = true
  const nopify = jest.spyOn(plugin, 'nopifyRequest')
  await plugin.sync()
  expect(nopify).not.toHaveBeenCalledWith(DELETE_ENV, expect.anything(), expect.anything())
})
Suggested fix

In Environments.sync(), skip the existingRecords.forEach(... this.remove(x) ...) loop when this.additive is set. In nop mode, emit the same suppression NopCommand that Diffable.sync() emits.

Context

Are you using the hosted instance of probot/settings or running your own?

Running our own. I reproduced this with unit tests against the code, using a mocked GitHub API, so it doesn't depend on how the app is deployed.

If running your own instance, are you using it with github.com or GitHub Enterprise?

Not relevant here: the reproduction uses a mocked API, and the code path is the same for both.

Version of probot/settings

safe-settings 3.0.0 (a9a1f366ad6a738b55682d346d94f8d6c4893b09), and main-enterprise at e19444e0d69de050ba55d0262b71fcd8a98edfef.

Version of GitHub Enterprise

N/A

Test helpers (mocks)
/* eslint-disable no-undef */
// Reproductions for reported Environments plugin bugs. Tests assert the
// CORRECT behaviour; the ones marked "BUG" fail against the current code.
const { when } = require('jest-when')
const Environments = require('../../../../lib/plugins/environments')
const Settings = require('../../../../lib/settings')
const NopCommand = require('../../../../lib/nopcommand')

const org = 'my-org'
const repo = 'my-repo'
const DELETE_ENV = 'DELETE /repos/:org/:repo/environments/:environment_name'
const PUT_ENV = 'PUT /repos/:org/:repo/environments/:environment_name'
const POST_RULE = 'POST /repos/:org/:repo/environments/:environment_name/deployment_protection_rules'
const DELETE_RULE = 'DELETE /repos/:org/:repo/environments/:environment_name/deployment_protection_rules/:rule_id'
const DELETE_VAR = 'DELETE /repos/:org/:repo/environments/:environment_name/variables/:variable_name'

// GitHub API shape of GET /repos/{owner}/{repo}/environments entries.
function apiEnv (name, { waitTimer, reviewers, preventSelfReview, branchPolicy = null } = {}) {
  const protectionRules = []
  if (waitTimer) protectionRules.push({ id: 1, type: 'wait_timer', wait_timer: waitTimer })
  if (reviewers) {
    protectionRules.push({
      id: 2,
      type: 'required_reviewers',
      prevent_self_review: !!preventSelfReview,
      reviewers: reviewers.map(r => ({ type: r.type, reviewer: { id: r.id } }))
    })
  }
  return { name, protection_rules: protectionRules, deployment_branch_policy: branchPolicy }
}

// existing: { [envName]: { api, variables: [{name,value}], rules: [{id, app: {id}}] } }
function mockGithub (existing) {
  const github = {
    request: jest.fn(() => Promise.resolve({})),
    paginate: jest.fn().mockResolvedValue([])
  }
  when(github.request)
    .calledWith('GET /repos/:org/:repo/environments', { org, repo })
    .mockResolvedValue({ data: { environments: Object.values(existing).map(e => e.api) } })
  for (const [name, e] of Object.entries(existing)) {
    when(github.request)
      .calledWith('GET /repos/:org/:repo/environments/:environment_name/deployment_protection_rules', { org, repo, environment_name: name })
      .mockResolvedValue({ data: { total_count: (e.rules || []).length, custom_deployment_protection_rules: e.rules || [] } })
    when(github.paginate)
      .calledWith('GET /repos/{owner}/{repo}/environments/{environment_name}/variables', expect.objectContaining({ environment_name: name }), expect.any(Function))
      .mockResolvedValue(e.variables || [])
  }
  return github
}

function writeCalls (github) {
  return github.request.mock.calls
    .filter(([route]) => !route.startsWith('GET '))
    .map(([route, opts]) => ({ route, opts }))
}

const log = { debug: jest.fn(), info: jest.fn(), warn: jest.fn(), error: jest.fn() }
Dominant language
JavaScript
Stars
942
Forks
238
PR merge metrics
No merged PRs in 30d

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from github-community-projects/safe-settings

All issues in github-community-projects/safe-settings

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.