bytedeco/javacv

extract frame from video in android device use JavaCV, but every time the model's inference results are not same

Aperta

#1938 aperta il 15 nov 2022

 (0 commenti) (0 reazioni) (0 assegnatari)Java (1583 fork)batch import
help wantedquestion

Metriche repository

Star
 (6985 stelle)
Metriche merge PR
 (Metriche PR in attesa)

Descrizione

Hello, I am using the JavaCV to extract frame from video in android device and as input to my machine learning model, and I used the FFmpegFrameFilter to set the fps of video to get same number of frame every time. but every time I run the model, the model's inference results are not same, do you know the reason? these my code below :

       FFmpegLogCallback.set();
      grabber.start();
      FFmpegFrameFilter frameFilter = new FFmpegFrameFilter("fps=fps=5", "anull",
              grabber.getImageWidth(), grabber.getImageHeight(), grabber.getAudioChannels());
      frameFilter.setSampleFormat(grabber.getSampleFormat());
      frameFilter.setSampleRate(grabber.getSampleRate());
      frameFilter.setPixelFormat(grabber.getPixelFormat());
      frameFilter.setFrameRate(grabber.getFrameRate());
      frameFilter.start();
      Frame capturedFrame;
      Frame pullFrame;
      while (true) {
        try {
          capturedFrame = grabber.grab();
          if (capturedFrame == null) {
            Log.w("!!! end cvQueryFrame..video:{}", frontcamVideoSourcePath);
            break;
          }
          if (capturedFrame.image != null) {
            frameFilter.push(capturedFrame);
          }
          if ((pullFrame = frameFilter.pull()) != null) {
            if (pullFrame.image != null) {
              Bitmap bitmap = convertToBitmap.convert(pullFrame);
              setImage(bitmap, frontcamRunner.getSurface(), true);
              ResultProto objectResult = frontcamRunner.run();
              resultsObject.add(objectResult));

When I do the snapshot test, it shows every time the inference results are not same like:

Changed content at line 922:
expecting:
  ["scores: 0.9639015",
   "scores: 0.4762144",
   "scores: 0.29251003",
   "boxes: 359.04742",
   "boxes: 108.379776",
   "boxes: 366.3406",
   "boxes: 112.13722",
   "boxes: 359.66046",
   "boxes: 108.81177",
   "boxes: 366.08936",
   "boxes: 116.47825",
   "boxes: 358.7235",
   "boxes: 110.984955",
   "boxes: 366.55127",
   "boxes: 114.66323"]
but was:
  ["scores: 0.9536576",
   "scores: 0.47547817",
   "scores: 0.25218058",
   "boxes: 358.9765",
   "boxes: 108.3856",
   "boxes: 366.24115",
   "boxes: 112.15494",
   "boxes: 359.57233",
   "boxes: 108.810745",
   "boxes: 366.00732",
   "boxes: 116.48662",
   "boxes: 358.6304",
   "boxes: 110.98422",
   "boxes: 366.46204",
   "boxes: 114.66691"]

Changed content at line 941:
expecting:
  ["boxes: 358.75476",
   "boxes: 108.227554",
   "boxes: 365.989",
   "boxes: 111.96691"]
but was:
  ["boxes: 358.73907",
   "boxes: 108.22945",
   "boxes: 365.9668",
   "boxes: 111.97165"]

Changed content at line 949:
expecting:
  ["boxes: 358.5876",
   "boxes: 108.080986",
   "boxes: 365.78546",
   "boxes: 111.79181"]
but was:
  ["boxes: 358.58386",
   "boxes: 108.081505",
   "boxes: 365.78018",
   "boxes: 111.79303"]

Changed content at line 957:
expecting:
  ["boxes: 358.52716",
   "boxes: 107.90219",
   "boxes: 365.63968",
   "boxes: 111.619354"]
but was:
  ["boxes: 358.5263",
   "boxes: 107.902336",
   "boxes: 365.63843",
   "boxes: 111.619675"]

Changed content at line 965:
expecting:
  ["boxes: 358.5914",
   "boxes: 107.647606",
   "boxes: 365.6294",
   "boxes: 111.37422"]
but was:
  ["boxes: 358.5912",
   "boxes: 107.64763",
   "boxes: 365.62915",
   "boxes: 111.37428"]

Changed content at line 977:
expecting:
  ["boxes: 358.92163"]
but was:
  ["boxes: 358.92157"]

Changed content at line 979:
expecting:
  ["boxes: 365.94983",
   "boxes: 111.76167"]
but was:
  ["boxes: 365.94977",
   "boxes: 111.761696"]

Changed content at line 985:
expecting:
  ["boxes: 359.38873"]
but was:
  ["boxes: 359.3887"]

Changed content at line 987:
expecting:
  ["boxes: 365.98245",
   "boxes: 115.30535"]
but was:
  ["boxes: 365.98242",
   "boxes: 115.305374"]

Changed content at line 995:
expecting:
  ["boxes: 366.0174"]
but was:
  ["boxes: 366.01736"]

Guida contributor