GalaxyCore GC2145 image corruption on Nicla Vision over serial

Aperta
#1,077 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Bug
Chiarezza
Da chiarire
Stato di attività
Ferma
Stack tecnologico
cpp, python

Direzione di ricerca

Inizia con le chiamate a Serial.write mostrate nell’issue e con i controlli del ricevitore Python per START_BYTE, END_BYTE, size e CRC. Poi esamina la gestione dei formati della fotocamera di Nicla Vision intorno a CAMERA_RGB565 e CAMERA_BAYER, inclusi i registri citati per il cambio di riga e il formato di output. Il lavoro è completato quando è stato isolato se la corruzione deriva dall’acquisizione, dall’incapsulamento seriale o dalla ricezione, ed è stata documentata una correzione riproducibile.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Hi,

We have implemented a library to trasmit the sensors data of the Nicla: https://github.com/ADVRHumanoids/nicla_vision_drivers through WiFi (UDP or TCP)

Recently I am trying to transmit such data over Serial, using the USB cable, which if I am not wrong utilizes UART connection.
By doing so, some images are received (partially) corrupted. This is noticeable since it happens frequently, but most of the time images are fine:

RGB565, 320x240

Image Image

With WiFi connection, the problem happens but very infrequently and hence tolerable.

Hardware

Hardware-wise, I am trying with different USB cables and different nicla boards and the behaviour is the same. If the problem is not already there, just shaking the nicla (to try to see if cable/port connection is bad) does not make the corruption to occur.

Software

Software-wise, I am sending data like this:

Serial.write((uint8_t*)header_buffer, headerLength);
Serial.write((uint8_t*)img_ptr, camPtr->frameSize());
Serial.write(END_BYTE);

In receiving the package (on PC, python) where I check for the START_BYTE, END_BYTE. Furthermore I check the packet size, and checksum/CRC-32 (tried both), from bytes inside the header. They are always correct, even when the corruption happen.

Digging out more, I found out that the corruption is indeed just a byte's shift. On the receiver side, when shifting the data of one byte with data_shifted = data[1:] + data[:1], the corruption is "dual":

Image

I am not super sure, but this corruption seems to happen as soon strong light/white areas are captured by the camera:
https://github.com/user-attachments/assets/4fbb36ea-12e4-4101-9246-63add3e82088

Bayer

For now, I am practically solving trasmitting CAMERA_BAYER pixel format instead of CAMERA_RGB565 (tried CAMERA_R320x240 as well CAMERA_R320x320 this time) . But I am not sure if the problem is solved or only less noticeable because of the Bayer representation. Indeed it seems that images turns from more "blue-ish" to "red-ish" and viceversa:

BAYER, 320x320

Image Image <\div>

https://github.com/user-attachments/assets/de73398b-6bbd-4399-b224-d3ea21504f6f

I can also see some RGB spurious pixel in the contours but this may related to the Bayer demosacing.
More, arduino code put a warning for BAYER mode but I am unsure about its meaning and if I should handle the data differently:

        case CAMERA_BAYER:
            // There's no BAYER support so it will just look off.
            // Make sure odd/even row are switched to work with our bayer conversion.
            ret |= regWrite(GC2145_I2C_ADDR,
                    REG_SYNC_MODE, REG_SYNC_MODE_DEF | REG_SYNC_MODE_ROW_SWITCH);
            ret |= regWrite(GC2145_I2C_ADDR,
                    REG_OUTPUT_FMT, REG_OUTPUT_SET_FMT(reg, REG_OUTPUT_FMT_BAYER));
            break;
Bonus

As a bonus issue, here is what happens with bayer and 640 resolution: (CAMERA_R640x480)
BAYER, 640x480

Image

Colors are always wrong, no matter all the OpenCV available conversions for Bayer (with cv2.COLOR_BayerGBRG2BGR good for 320 resolution and that should be the right one according to GC2145 documentation).
I can also see the mosaic RGB squared pattern on the image.


Anyone has any suggestions?
Thanks

Lingua principale
C
Stelle
411
Fork
229
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di arduino/ArduinoCore-mbed

Tutte le issue di arduino/ArduinoCore-mbed

Issue simili

Altre issue su C

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.