Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

Implications of compression algorithms

Đang mở
  • Có gì không ổn?
#17 3 bình luận 0 reaction 0 người được giao Xem trên GitHub

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
20/100
Loại issue
Tính năng
Độ rõ ràng
Cần làm rõ
Mức độ hoạt động
Đình trệ
Công nghệ
python
Lĩnh vực
data, performance

Hướng nghiên cứu

Bắt đầu bằng việc xem lại cuộc thảo luận hiện có trong issue 8 và các câu hỏi về nén và tuyến tính hóa được nêu ở đây, bao gồm các tham chiếu DIPY và phân đoạn được liên kết. Công việc được xem là hoàn tất khi dự án đưa ra quyết định về các tác động downstream, kỳ vọng về giải nén hoặc lấy mẫu, cũng như cách làm rõ những hệ quả đó cho người dùng.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Hi all,

My apologies for getting to this conversation late, but I had some questions about the implications of the compression/linearization capabilities afforded by this new implementation. There is an existing discussion here, but to my reading it largely deals with the proximal details as they relate to implementation and immediate consequences. My considerations relate to implications that are a bit further downstream.

This may be an arbitrary delineation, but to me, it seems like the discussed streamline-focused, lossy compression methods can be divided into two tiers of “lossyness”:

  • Insignificantly to mildly lossy
    • Conversion to less precise data formats (e.g. float 16)
    • Implementation of dictionary-based storage schemas (e.g. sphere dictionary;QFib)
  • Moderately to appreciably lossy
    • “Downsampling”
    • Linearization

To me, these two categories are distinct in that the first corresponds to a slight loss in precision (and thus the introduction of a slight bit of noise), while the latter actually modifies the data representation itself through the removal of nodes/vertices.

The implications of these latter strategies are partially determined by how “decompression” is implemented (if at all). For example:

  • Are we enforcing or pushing regular sampling rates?
    • If so, it seems that we would then be “regenerating” these dropped nodes. Is there a standard procedure for this?
    • If not, this seems to preclude the use of dictionary-based compression algorithms (which, to my understanding, are predicated upon standard step sizes).
  • Are we preventing down-sampling which alters the path or trajectory of streamlines?

I understand that there are significant efficiencies to be gained through some of these methods, and that for some of us (e.g. those doing connectomics) the quantifications are unaltered. Moreover, I understand that some of us have gone so far as to quantify the impact of such approaches to varying degrees (e.g here and here). That being said, I have some concerns about the implications of such compression methods for contemporary tractogram segmentation methods themselves (explored to some extent in the later of the aforementioned papers), as opposed to tractometry or derived measurements.

Specifically, there seem to be a plentiful number of cases wherein these methods would impact the outcome of a segmentation non-trivially (as discussed in the caveats at the bottom of this page). Namely:

  • Uneven sampling rates could allow for planar ROIs (highly common and widely utilized) to “slip through” linearized areas.
    • Given that the architecture of the the composite streamlines of some particularly linear tracts are highly similar, (e.g. tracts like the IFOF or ILF), it seems that:
      • Some structures may be impacted more than others by this issue (e.g. non-random impact across structures)
      • Those structures that are impacted will likely have particular sub-components of their structure/morphology that are deferentially impacted (e.g. non-random impact within structures)
  • This is even more true of down-sampling algorithms which (1) introduce these same spacing issues and (2) impact the trajectory / volumetric occupancy of streamlines.
  • Assuming that streamlines aren’t “up-sampled” back to some higher spatial resolution, quantifications like density masks and atlases/masks (and group variants thereof) are likely impacted as well, with all of the complications noted in the first bullet point in this list (i.e. differential within and across structure impacts).
    • This paper describes some methods which partially address this, but it should be noted that:
      • Not all (or even most?) packages implement this alteration
      • It’s not algorithmically (in terms of computational implementation) or intuitively (in terms of how one would intuitively interact with a streamline) simple (subjective assessments, admittedly).
      • It achieves performance increases at the cost of increasing algorithmic opacity and shifts developmental burden off on to those who develop downstream uses (e.g. segmentations or segmentation tools; a potential source of personal bias, I admit).

To some extent, I recognize that these concerns aren’t actually arising from the compression methods being proposed here, but rather downstream consequences of their application. As such, it may be beyond the scope of concerns we are intending to address with this format implementation. That being said, if the goal is to implement a tractography standard that can be used across the community, then it stands to reason that we are designing for a range of potential users, some non-trivial proportion of which won’t be familiar with the nuanced implications discussed above. As such it is quite possible that such users could unintentionally run afoul of these issues and/or encounter difficulty replicating results using “the same data set”.

Are we at all concerned about these possibilities, or are we adopting a more laissez-faire approach to use of this format and its features? If we are concerned about these possibilities, how do we make them salient, or how do we attempt to shape users’ default dispositions?

Ngôn ngữ chính
Python
Star
27
Fork
20
Merge trung bình
2 ngày 25 phút
Pull request đã merge (30 ngày)
1

Chuẩn bị môi trường

Dự án này không cung cấp dev container, Dockerfile hay hướng dẫn đóng góp, nên bạn cần tự thiết lập môi trường: hãy bắt đầu từ README và xem hướng dẫn đóng góp lần đầu của chúng tôi để biết các bước chung.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của tee-ar-ex/trx-python

Tất cả issue của tee-ar-ex/trx-python

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.