Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Implications of compression algorithms

Aperta
  • Qualcosa non va?
#17 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
20/100
Tipo di issue
Funzionalità
Chiarezza
Da chiarire
Stato di attività
Ferma
Stack tecnologico
python
Ambito
data, performance

Direzione di ricerca

Inizia esaminando la discussione esistente nell’issue 8 e le questioni relative a compressione e linearizzazione sollevate qui, inclusi i riferimenti collegati a DIPY e alla segmentazione. Il lavoro sarà considerato completato quando il progetto avrà preso una decisione sugli effetti downstream, sulle aspettative di decompressione o campionamento e su come rendere chiare queste implicazioni agli utenti.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Hi all,

My apologies for getting to this conversation late, but I had some questions about the implications of the compression/linearization capabilities afforded by this new implementation. There is an existing discussion here, but to my reading it largely deals with the proximal details as they relate to implementation and immediate consequences. My considerations relate to implications that are a bit further downstream.

This may be an arbitrary delineation, but to me, it seems like the discussed streamline-focused, lossy compression methods can be divided into two tiers of “lossyness”:

  • Insignificantly to mildly lossy
    • Conversion to less precise data formats (e.g. float 16)
    • Implementation of dictionary-based storage schemas (e.g. sphere dictionary;QFib)
  • Moderately to appreciably lossy
    • “Downsampling”
    • Linearization

To me, these two categories are distinct in that the first corresponds to a slight loss in precision (and thus the introduction of a slight bit of noise), while the latter actually modifies the data representation itself through the removal of nodes/vertices.

The implications of these latter strategies are partially determined by how “decompression” is implemented (if at all). For example:

  • Are we enforcing or pushing regular sampling rates?
    • If so, it seems that we would then be “regenerating” these dropped nodes. Is there a standard procedure for this?
    • If not, this seems to preclude the use of dictionary-based compression algorithms (which, to my understanding, are predicated upon standard step sizes).
  • Are we preventing down-sampling which alters the path or trajectory of streamlines?

I understand that there are significant efficiencies to be gained through some of these methods, and that for some of us (e.g. those doing connectomics) the quantifications are unaltered. Moreover, I understand that some of us have gone so far as to quantify the impact of such approaches to varying degrees (e.g here and here). That being said, I have some concerns about the implications of such compression methods for contemporary tractogram segmentation methods themselves (explored to some extent in the later of the aforementioned papers), as opposed to tractometry or derived measurements.

Specifically, there seem to be a plentiful number of cases wherein these methods would impact the outcome of a segmentation non-trivially (as discussed in the caveats at the bottom of this page). Namely:

  • Uneven sampling rates could allow for planar ROIs (highly common and widely utilized) to “slip through” linearized areas.
    • Given that the architecture of the the composite streamlines of some particularly linear tracts are highly similar, (e.g. tracts like the IFOF or ILF), it seems that:
      • Some structures may be impacted more than others by this issue (e.g. non-random impact across structures)
      • Those structures that are impacted will likely have particular sub-components of their structure/morphology that are deferentially impacted (e.g. non-random impact within structures)
  • This is even more true of down-sampling algorithms which (1) introduce these same spacing issues and (2) impact the trajectory / volumetric occupancy of streamlines.
  • Assuming that streamlines aren’t “up-sampled” back to some higher spatial resolution, quantifications like density masks and atlases/masks (and group variants thereof) are likely impacted as well, with all of the complications noted in the first bullet point in this list (i.e. differential within and across structure impacts).
    • This paper describes some methods which partially address this, but it should be noted that:
      • Not all (or even most?) packages implement this alteration
      • It’s not algorithmically (in terms of computational implementation) or intuitively (in terms of how one would intuitively interact with a streamline) simple (subjective assessments, admittedly).
      • It achieves performance increases at the cost of increasing algorithmic opacity and shifts developmental burden off on to those who develop downstream uses (e.g. segmentations or segmentation tools; a potential source of personal bias, I admit).

To some extent, I recognize that these concerns aren’t actually arising from the compression methods being proposed here, but rather downstream consequences of their application. As such, it may be beyond the scope of concerns we are intending to address with this format implementation. That being said, if the goal is to implement a tractography standard that can be used across the community, then it stands to reason that we are designing for a range of potential users, some non-trivial proportion of which won’t be familiar with the nuanced implications discussed above. As such it is quite possible that such users could unintentionally run afoul of these issues and/or encounter difficulty replicating results using “the same data set”.

Are we at all concerned about these possibilities, or are we adopting a more laissez-faire approach to use of this format and its features? If we are concerned about these possibilities, how do we make them salient, or how do we attempt to shape users’ default dispositions?

Lingua principale
Python
Stelle
27
Fork
20
Merge medio
2g 25m
PR unite (30g)
1

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di tee-ar-ex/trx-python

Tutte le issue di tee-ar-ex/trx-python

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.