mattn/go-sqlite3

Next and Scan appear to be relatively slow?

开放

#379 创建于 2017年2月7日

 (10 条评论) (3 个反应) (0 位负责人)C (1,105 个派生)batch import
help wantedresearch

仓库指标

星标
 (7,000 个星标)
PR 合并指标
 (平均合并 1天 5小时) (30 天内合并 9 个 PR)

描述

Howdy. I'm not sure this really merits a Issues or if it would be better as a conversation some place. In using sqlite3 for a project and I noticed that it's quite a bit slower to select a large number of rows as compared to doing the same via their CLI client. ~4-5x slower. A little profiling shows that the time is mostly spent in Next and Scan. For small row sets it's not a big deal but I'm testing up to ~500K rows.

I suppose my starting question is how much overhead is expected going back and forth between Go and the C API? If it's expected to be fairly fast/low-overhead, any suspicions on where I might start looking?

Here's a sketch of approximately what the code is doing:

https://gist.github.com/mhat/ded97fe58922b4af9d03f375b465c339

There are two variations, one uses the new ScanTypes while the other is cheating with a list of string pointers. Neither appreciably moved the needly forScan.

贡献者指南