[jvm] Snapshots are not being written to disk when there are tests with numbered names
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 45/100
Research direction
Start by running the supplied JUnit 5 ReproduceSelfieBugTest example with Selfie 2.5.3 in IntelliJ, then inspect how snapshot files are written and ordered for numbered test names. Done means foo10, test10, and testFoo10 are consistently present in the .ss file, including after tests are added or removed and the file is rerun.
Written by the indexing model from the issue text.
Description
There is an issue when creating test methods that are numbered (e.g. test1(), test2(), ...). At some point snapshots of tests are no longer written to disk. I discovered this while writing integration tests for a Spring Boot app that I generically name test1(), test2() etc. because I use @DisplayName anyway.
I cannot disclose the actual code but here is an example test file using JUnit 5, Java 21 and Selfie 2.5.3 that I could reproduce this with:
import com.diffplug.selfie.Selfie;
import org.junit.jupiter.api.Test;
class ReproduceSelfieBugTest {
//SELFIEWRITE
@Test
void test() {
Selfie.expectSelfie("0").toMatchDisk();
}
@Test
void test1() {
Selfie.expectSelfie("1").toMatchDisk();
}
@Test
void test2() {
Selfie.expectSelfie("2").toMatchDisk();
}
@Test
void test3() {
Selfie.expectSelfie("3").toMatchDisk();
}
@Test
void test4() {
Selfie.expectSelfie("4").toMatchDisk();
}
@Test
void test5() {
Selfie.expectSelfie("5").toMatchDisk();
}
@Test
void test6() {
Selfie.expectSelfie("6").toMatchDisk();
}
@Test
void test7() {
Selfie.expectSelfie("7").toMatchDisk();
}
@Test
void test8() {
Selfie.expectSelfie("8").toMatchDisk();
}
@Test
void test9() {
Selfie.expectSelfie("9").toMatchDisk();
}
@Test
void test10() {
Selfie.expectSelfie("10").toMatchDisk();
}
@Test
void testFoo() {
Selfie.expectSelfie("testFoo").toMatchDisk();
}
@Test
void testFoo1() {
Selfie.expectSelfie("testFoo1").toMatchDisk();
}
@Test
void testFoo2() {
Selfie.expectSelfie("testFoo2").toMatchDisk();
}
@Test
void testFoo3() {
Selfie.expectSelfie("testFoo3").toMatchDisk();
}
@Test
void testFoo4() {
Selfie.expectSelfie("testFoo4").toMatchDisk();
}
@Test
void testFoo5() {
Selfie.expectSelfie("testFoo5").toMatchDisk();
}
@Test
void testFoo6() {
Selfie.expectSelfie("testFoo6").toMatchDisk();
}
@Test
void testFoo7() {
Selfie.expectSelfie("testFoo7").toMatchDisk();
}
@Test
void testFoo8() {
Selfie.expectSelfie("testFoo8").toMatchDisk();
}
@Test
void testFoo9() {
Selfie.expectSelfie("testFoo9").toMatchDisk();
}
@Test
void testFoo10() {
Selfie.expectSelfie("testFoo10").toMatchDisk();
}
@Test
void foo() {
Selfie.expectSelfie("foo").toMatchDisk();
}
@Test
void foo1() {
Selfie.expectSelfie("foo1").toMatchDisk();
}
@Test
void foo2() {
Selfie.expectSelfie("foo2").toMatchDisk();
}
@Test
void foo3() {
Selfie.expectSelfie("foo3").toMatchDisk();
}
@Test
void foo4() {
Selfie.expectSelfie("foo4").toMatchDisk();
}
@Test
void foo5() {
Selfie.expectSelfie("foo5").toMatchDisk();
}
@Test
void foo6() {
Selfie.expectSelfie("foo6").toMatchDisk();
}
@Test
void foo7() {
Selfie.expectSelfie("foo7").toMatchDisk();
}
@Test
void foo8() {
Selfie.expectSelfie("foo8").toMatchDisk();
}
@Test
void foo9() {
Selfie.expectSelfie("foo9").toMatchDisk();
}
@Test
void foo10() {
Selfie.expectSelfie("foo10").toMatchDisk();
}
}
Running this test file in IntelliJ 2024.3 produces this ReproduceSelfieBugTest.ss file:
╔═ foo ═╗
foo
╔═ foo1 ═╗
foo1
╔═ foo2 ═╗
foo2
╔═ foo3 ═╗
foo3
╔═ foo4 ═╗
foo4
╔═ foo5 ═╗
foo5
╔═ foo6 ═╗
foo6
╔═ foo7 ═╗
foo7
╔═ foo8 ═╗
foo8
╔═ foo9 ═╗
foo9
╔═ test ═╗
0
╔═ test1 ═╗
1
╔═ test2 ═╗
2
╔═ test3 ═╗
3
╔═ test4 ═╗
4
╔═ test5 ═╗
5
╔═ test6 ═╗
6
╔═ test7 ═╗
7
╔═ test8 ═╗
8
╔═ test9 ═╗
9
╔═ testFoo ═╗
testFoo
╔═ testFoo1 ═╗
testFoo1
╔═ testFoo2 ═╗
testFoo2
╔═ testFoo3 ═╗
testFoo3
╔═ testFoo4 ═╗
testFoo4
╔═ testFoo5 ═╗
testFoo5
╔═ testFoo6 ═╗
testFoo6
╔═ testFoo7 ═╗
testFoo7
╔═ testFoo8 ═╗
testFoo8
╔═ testFoo9 ═╗
testFoo9
╔═ [end of file] ═╗
You will notice the snapshots for these tests are missing: foo10(), test10(), testFoo10, i.e. all tests ending with 10.
Additional information:
If you remove ALL tests except test(), test1() and test10() then the snapshot for test10() is correctly saved to disk.
After that, if test2() is added back and after re-running the test file, the snapshot for test10() is missing again. Then, after removing test2() once more and re-running the test file, the snapshot for test10() is STILL missing until the test file is run one more time.
- Dominant language
- Kotlin
- Stars
- 101
- Forks
- 18
- Avg merge
- 6d 11h
- Merged PRs (30d)
- 5
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from diffplug/selfie
-
Selfie JVM Snapshot Garbage Collection is overly zealous when using multiple test suites in gradle Openbug jvm
Difficulty 4/5 3-5 days Newbie friendliness 48/100
-
enhancement jvm
Difficulty 5/5 Over a week Newbie friendliness 25/100
-
bug jvm
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
bug docs jvm
Difficulty 2/5 1-3 hours Newbie friendliness 50/100
-
bug jvm py
Difficulty 2/5 1-3 hours Newbie friendliness 48/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
2 rename bugs Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
ankidroid/Anki-Android#21999 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
openwallet-foundation/multipaz#2028 ·
-
Improvement Java Platform: Java
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
getsentry/sentry-java#6145 · 1 comment ·