DREDge differences (parameters, online mode, math)
I maintainer di solito rispondono entro 2 giorni
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 3/5
- Tempo stimato
- Mezza giornata
- Idoneità per principianti
- 65/100
- Tipo di issue
- Bug
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Attiva
- Stack tecnologico
- python
- Ambito
- machine-learning
Direzione di ricerca
Compare the online DREDge implementation with the cwindolf/dredge repository to identify parameter mismatches (win_scale_um, win_step_um) and missing raster transforms (np.log1p). Fix the time_horizon bug in cross-chunk matrix extraction and add missing weights to the AP version's cross-correlation computation. Run the online algorithm tests to verify changes.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Hi all,
After @chrishalcrow and @alejoe91 released the DREDge online mode in SpikeInterface, I was quite excited to integrate it into dartsort, so I started looking into it more. I realized that there were a couple of details that I had not thought of when we discussed the implementation. Separately, I have been meaning to look into possible parameter mismatches between https://github.com/cwindolf/dredge and here for a while. I thought I'd gather up the findings here and break them into pieces so that we can discuss which ones you'd be happy to have changed, then I can open PRs to do so. Apologies for not thinking of these before / having bugs!
Online algorithm issues
There are two main differences between the implementation here and what I just wrote up in the dredge repo:
- Weights. The AP version of dredge does some weighting of each bin when computing cross correlations, but this isn't done in the LFP version. I think it's important to include this, and I'm happy to add it here.
- Cross-chunk time_horizon bug. I wasn't using time_horizon_s much in the LFP code, so I never noticed that it is applied incorrectly in the online algorithm. Right now, it exctracts a "main diagonal band" from the cross-chunk matrices, rather than grabbing elements on the bottom of the lower triangle / top of upper triangle as it should. The main diagonals are used within the same chunk, but the time horizon works differently in the cross-chunk terms!
There was also a math issue in my dredge code, a tiny thing involving one term missed in the prior/regularizer across the online algorithm's chunks. If I'm adding the above stuff, I think it would be cool to fix this smaller thing as well.
Parameters and raster processing
Compared to the dredge repo (D), here (SI) we have:
win_scale_um: D 300 (dredge parametrizes it as 600/2), SI does seem to have the same default, but across several files default values of 150, 400, and 300 all appear.win_step_um: D has 400, in SI values of both 400 and 200 appear. I think 200 is the actually used value, which is a lot of windows!- Raster transform: in D, the raster has a
np.log1papplied, which is not done in SI. I remember this being important in my initial development, but I don't have evidence at the moment.
There are also some other differences which seem minor.
So, what would be the best moves that I could make for you guys? I think that the online algorithm stuff is important to fix before this gets much usage. For the AP stuff, I'd like to at least go to step 400 if that's acceptable. I feel a bit afraid to add the log1p, but I do think it is a good thing to have.
- Lingua principale
- Python
- Stelle
- 858
- Fork
- 281
- Merge medio
- 3g 18h
- PR unite (30g)
- 45
Preparare l'ambiente
Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di SpikeInterface/spikeinterface
-
testing
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
SpikeInterface/spikeinterface#4756 ·
I maintainer di solito rispondono entro 2 giorni
-
enhancement
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
SpikeInterface/spikeinterface#4510 · 2 commenti ·
I maintainer di solito rispondono entro 2 giorni
-
Implement filtering by banks of channels to reduce RAMForse già presa @h-mayorquin l’ha presa oggi. Apertapreprocessing
SpikeInterface/spikeinterface#4837 · 1 assegnatario ·
I maintainer di solito rispondono entro 2 giorni
-
Extend `TimeSeriesExecutor` to `num_chunks_per_job`Forse già presa @samuelgarcia l’ha presa 1 giorno fa. Apertaconcurrency
SpikeInterface/spikeinterface#4831 · 1 commento · 1 assegnatario ·
I maintainer di solito rispondono entro 2 giorni
-
bug in sorter
Difficoltà 4/5 3-5 giorni Idoneità per principianti 45/100
SpikeInterface/spikeinterface#4826 · 16 commenti ·
I maintainer di solito rispondono entro 2 giorni
Tutte le issue di SpikeInterface/spikeinterface
Issue simili
-
changelog investigate
Difficoltà 2/5 1-3 ore Idoneità per principianti 62/100
ramnes/notion-sdk-py#409 ·
-
good first issue help wanted
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
lindicaphxag-tech/kaggle#28 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 62/100
BSData/horus-heresy-3rd-edition#3211 ·
I maintainer di solito rispondono entro 1 giorno
-
bug needs-triage
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
I maintainer di solito rispondono entro 1 giorno
-
Unreachable-proxy mount test depends on fixed port 9999Forse già presa Una pull request collegata a questa issue è aperta o già unita. Apertabug tests
Difficoltà 2/5 1-3 ore Idoneità per principianti 76/100
I maintainer di solito rispondono entro 1 giorno