Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

testIntegration > Autocomplete.testAutocomplete fails at Autocomplete.kt:42 (latent bug masked by FFmpeg 7.1 404)

Open
  • Is something wrong?
#13,173 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 3 days

Nobody has claimed this yet.

  • #13169 by @santhiprakash — closed without merging
  • #13177 by @santhiprakash — closed without merging

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
github-actions, kotlin
Domain
ci-cd, testing-qa

Research direction

Start with extensions/intellij/src/testIntegration/kotlin/com/github/continuedev/continueintellijextension/Autocomplete.kt and its testAutocomplete() assertion, then run testIntegration with the Setup FFmpeg step passing. Inspect the TestLLM fixture at extensions/intellij/src/testIntegration/kotlin/com/github/continuedev/continueintellijextension/test-continue/config.json, build.gradle.kts, and the jb-failure-report artifact to determine why the expected response is absent. Done means the test reliably passes in CI or the issue's chosen temporary resolution is documented.

Written by the indexing model from the issue text.

Description

Summary

:testIntegration > Autocomplete.testAutocomplete() fails at Autocomplete.kt:42 on every PR that successfully passes the Setup FFmpeg step. The test has been latent for ~11 months — it was added on 2025-09-19 via PR #7690 ("test: use intellij-ide-starter for e2e tests", CON-3723) but has effectively never run to completion in CI because the AnimMouse/setup-ffmpeg@v1 version: 7.1 pin in .github/actions/run-jetbrains-tests/action.yml returns 404 first (issue #13164).

Failure detail

  • Test class: com.github.continuedev.continueintellijextension.Autocomplete
  • Test method: testAutocomplete()
  • Failing assertion (Autocomplete.kt:42): assertTrue(text.contains("TEST_LLM_RESPONSE_0"))
  • Error: expected: <true> but was: <false> (org.opentest4j.AssertionFailedError)
  • Duration to failure: 1m04s (the test runs to completion; the assertion fails at the end)
  • IDE under test: IC-243.21565.193 (IntelliJ Community 2024.3)
  • Test framework: intellij-ide-starter (TestFrameworkType.Starter), pinned in extensions/intellij/build.gradle.kts
  • JUnit: Jupiter 5.7.1
  • Artifact (video + test reports): jb-failure-report from run 32468648181 — video at video/testAutocomplete_recording_2026_21_08_09_43_36.avi

Why this surfaces now

Issue #13164 reported that the FFmpeg 7.1 pin returns 404. PR #13169 fixes it by pinning to 8.1 + adding a preflight check. Once the FFmpeg step passes on the PR head (2e6fadcbd), testIntegration runs and this assertion fires.

Why this is pre-existing (not caused by the FFmpeg fix)

  • The PR diff is only .github/actions/run-jetbrains-tests/action.yml (parameterized version + preflight check). No JetBrains plugin source is touched.
  • The test framework, IntelliJ version, and TestLLM fixture have not changed since 2025-09-19.
  • The test was effectively dead code in CI for the entire window between #7690 and #13164: main-branch version: 7.1 always 404s the FFmpeg step first, so testIntegration is never reached.
  • The closest prior PR (#13165 by ScrewTSW) was self-closed on 2026-08-19 before CI ran.

Likely root causes (not exhaustive)

  1. Autocomplete never fires in the new IDE under test. The tab() key after enterText("TEST_USER_MESSAGE_0") + space() may not trigger Continue's inline completion — could be a default-settings issue (autocomplete disabled) or a keystroke routing issue.
  2. TestLLM not loaded — the test fixture config in extensions/intellij/src/testIntegration/kotlin/com/github/continuedev/continueintellijextension/test-continue/config.json (inferred from CONTINUE_GLOBAL_DIR in build.gradle.kts) may not be picked up by the 2024.3 IDE under test.
  3. Prompt-to-response mismatch — TestLLM.findResponse() requires the regex [^0-9]*TEST_USER_MESSAGE_(\d+)[^0-9]*. If the IDE injects surrounding text that breaks the digit boundaries, findResponse returns undefined and the stream yields PROMPT: <full prompt> instead of TEST_LLM_RESPONSE_0.
  4. Timing — the test waits 2.seconds after typing, which may not be enough for the autocomplete provider to compute a response under the new test framework.

Suggested next step

Pick one:

  • (A) Investigate and fix the test (preferred — the test should be a working CI gate).
  • (B) Mark the test @Disabled with a TODO referencing this issue until a fix lands.
  • (C) Accept that this test is flaky in the new framework and remove it.

Related

  • Fix PR (unblocks this test from being reached): #13169 "fix(ci): parameterize pinned FFmpeg version with explicit preflight check (BtbN dropped 7.1)"
  • Root-cause bug: #13164
  • Original test introduction: #7690 (CON-3723), commit 507cdfc61a on 2025-09-19
  • Failed CI run with artifacts: https://github.com/continuedev/continue/actions/runs/32468648181
Dominant language
TypeScript
Stars
36.1k
Forks
5.4k
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from continuedev/continue

All issues in continuedev/continue

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.