Combination of component and e2e test seems random. vite-plugin-istanbul

Bắt đầu với package.json, vite.ts, cypress.config.ts và Dockerfile, sau đó chạy các lệnh unit và integration theo thứ tự được ghi trong tài liệu. So sánh coverage/coverage-summary.json giữa các môi trường Windows, Docker và pipeline; được coi là hoàn tất khi các lần chạy component và e2e tạo ra branch coverage nhất quán hoặc đã xác định được nguyên nhân do môi trường.

Logs and screenshots
I won't be able to add logs as it's over GitHub limits

Versions

• What is this plugin's version? ^3.11.0
• What is Cypress version? ^12.17.2
• What is your operating system? Windows 10 x64 / Linux Ubuntu
• What is the shell?
• What is the Node version? 20.3.1
• What is the NPM version? 9.7.2
• How do you instrument your application? Using vite-plugin-istanbul verionn ^5.0.0.
• When running tests, if you open the web application in regular browser, and open DevTools, do you see window.__coverage__ object? Yes
• Is there .nyc_output folder? Is there .nyc_output/out.json file. Is it empty? It's not empty and has +31K lines content.
• Do you have any custom NYC settings in package.json (nyc object) or in other [NYC config files] Yes

   istanbul({
        include: 'src/*',
        exclude: ['node_modules'],
        extension: [ '.js', '.ts', '.jsx', '.tsx',  ],
        requireEnv: !process.env.CYPRESS,
        checkProd: true,
      }),
  

• Do you run Cypress tests in a Docker container? Yes

Describe the bug
I am trying to combine the unit test coverage and e2e. But the coverage is different in different environments. I have tested this in 3 different environments.

1. My Windows (Win 10 x64) machine, which normally shows 5 uncovered branches. FYI, I delete .nyc and coverage folder before running the commands.
2. A docker file which I use to test with. This one show 3 uncovered branches.
3. The pipeline (GitHub Actions) which build the same docker file. This one show 8 uncovered branches.

I have RUN npm run components:ci && npm run e2e:ci && npm run build in my docker file and run the same commands in the same order in windows too. However, if I run the e2e before components I will end up with 32 uncovered branches.

I am testing a page using component test which results in 100% code coverage. When I run the e2e (to test the routing to this page), it will be reduced to just over 80%.

Link to the repo
Sadly, I cannot make the project public, and if I remove the codes/tests the issue with go away. But the setup can be find in this template. But, I will be able to give access to a contributor.

package.json

{
  "name": "my-vite-app",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "concurrently --kill-others \"npx json-server -w json-server/data.cjs -r json-server/routing.json -m json-server/middleware.cjs --host 127.0.0.1 -p 4000\" \"vite\"",
    "build": "tsc && vite build",
    "lint": "eslint src --ext ts,tsx --report-unused-disable-directives --max-warnings 0",
    "preview": "vite preview",
    "integration": "start-server-and-test dev \"http://localhost:5173|http://127.0.0.1:4000\" \"cypress open --e2e\"",
    "integration:ci": "start-server-and-test dev \"http://localhost:5173|http://127.0.0.1:4000\" \"cypress run --e2e\"",
    "unit": "cypress open --component",
    "unit:ci": "cypress run --component"
  },
  "dependencies": {
    "jalaali-js": "^1.2.6",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "react-easy-crop": "^5.0.0",
    "react-router-dom": "^6.14.2"
  },
  "devDependencies": {
    "@cypress/code-coverage": "^3.11.0",
    "@faker-js/faker": "^8.0.2",
    "@types/jalaali-js": "^1.1.1",
    "@types/react": "^18.2.16",
    "@types/react-dom": "^18.2.7",
    "@typescript-eslint/eslint-plugin": "^6.2.0",
    "@typescript-eslint/parser": "^6.2.0",
    "@vitejs/plugin-react": "^4.0.3",
    "autoprefixer": "^10.4.14",
    "concurrently": "^8.2.0",
    "cypress": "^12.17.2",
    "cypress-file-upload": "^5.0.8",
    "eslint": "^8.45.0",
    "eslint-plugin-react-hooks": "^4.6.0",
    "eslint-plugin-react-refresh": "^0.4.1",
    "json-server": "^0.17.3",
    "postcss": "^8.4.27",
    "start-server-and-test": "^2.0.0",
    "tailwindcss": "^3.3.3",
    "typescript": "^5.0.2",
    "vite": "^4.4.7",
    "vite-plugin-istanbul": "^5.0.0",
    "vite-plugin-pwa": "^0.16.4",
    "vite-tsconfig-paths": "^4.2.0"
  }
}

vite.ts

import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react';
import tsconfigPaths from 'vite-tsconfig-paths';
import istanbul from 'vite-plugin-istanbul';
import { VitePWA } from 'vite-plugin-pwa'

// https://vitejs.dev/config/
export default defineConfig({
  build: {
    sourcemap: true,
  },
  plugins: [
    react(),
    tsconfigPaths(),
    istanbul({
      include: 'src/*',
      exclude: ['node_modules'],
      extension: [ '.js', '.ts', '.jsx', '.tsx',  ],
      requireEnv: !process.env.CYPRESS,
      checkProd: true,
    }),
    VitePWA({ 
      registerType: 'autoUpdate',
      injectRegister: 'auto',
      includeAssets: ['favicon.ico', 'apple-touch-icon.png', 'mask-icon.svg'],
      manifestFilename: "manifest.json",
      manifest: {
        name: '*****',
        short_name: '****',
        description: '*****',
        theme_color: '#ffffff',
        icons: [
          {
            src: 'logo192.png',
            sizes: '192x192',
            type: 'image/png'
          },
          {
            src: 'logo512.png',
            sizes: '512x512',
            type: 'image/png'
          }
        ]
      }
     }),
  ],
})

cypress.config.ts

import { defineConfig } from "cypress";
import registerCodeCoverageTasks from "@cypress/code-coverage/task";

export default defineConfig({
  e2e: {
    screenshotOnRunFailure: false,
    video: false,
    retries: {
      runMode: 2,
    },
    baseUrl: "http://localhost:5173",
    setupNodeEvents(on, config) {
      registerCodeCoverageTasks(on, config);
      // implement node event listeners here

      return config;
    },
  },

  component: {
    screenshotOnRunFailure: false,
    retries: {
      runMode: 2,
    },
    video: false,
    devServer: {
      framework: "react",
      bundler: "vite",
    },
    setupNodeEvents(on, config) {
      registerCodeCoverageTasks(on, config);
      // implement node event listeners here

      return config;
    },
  },
});

Dockerfile

FROM node:20-slim as build

RUN apt-get update && apt-get install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2 libxtst6 xauth xvfb jq -y

WORKDIR /app

COPY package*.json ./
RUN npm ci

COPY . .
RUN npm run unit:ci && npm run integration:ci && npm run build

RUN echo "Current branch coverage is $(cat ./coverage/coverage-summary.json | jq ".total.branches.total - .total.branches.covered")."

RUN [ $(cat ./coverage/coverage-summary.json | jq ".total.branches.total - .total.branches.covered") -le 4 ] \
   || touch ./coverage/failed.flag

FROM scratch as test-result
COPY --from=build /app/coverage /

FROM scratch as deployable
COPY --from=build /app/dist /

FROM nginx:1.25-alpine-slim as deployment
COPY ./nginx.conf /etc/nginx/conf.d/default.conf
CMD ["/bin/sh", "-c", "nginx -g 'daemon off;'"]
COPY --from=build /app/dist /usr/share/nginx/html