hound-search/hound

Index is apparently limited to 4 GB

開放

#351 建立於 2020年10月1日

 (7 則留言) (0 個反應) (0 位負責人)JavaScript (606 個分叉)batch import
enhancementhelp wanted

倉庫指標

星標
 (5,470 顆星)
PR 合併指標
 (30 天內沒有已合併 PR)

描述

:wave: Hound developers!

I am trying to index a pretty large repo (144GB - all current sources of openSUSE), and unsurprisingly the index turns out to be larger than 4GB, thus I hit this fatal message:

https://github.com/hound-search/hound/blob/e3b1b43eb872e47af1de1b0e15d0ec3ac5c51dc4/codesearch/index/write.go#L561

Would it be possible / how hard would it be to support larger indexes?

I only had a brief look at read.go and it seems to me that 32 bit offsets are part of the index file format, so changing that would require re-indexing/converting/supporting two file formats, is that correct?

Thanks for all your efforts on Hound!

貢獻者指南