questdb/questdb

OOM logging is badly formatted

Aberta

#4.806 aberto em 23 de jul. de 2024

 (15 comentários) (0 reação) (0 responsável)Java (972 forks)batch import
Bug 0EnhancementGood first issue

Métricas do repositório

Stars
 (13.403 estrelas)
Métricas de merge de PR
 (Mesclagem média 5d 13h) (108 fundiu PRs em 30d)

Description

To reproduce

2024-07-22T17:51:34.453589Z I i.q.c.TableWriter o3 commit [table=testWalWriteFullRandom_nonwal, o3RowCount=1769558]
sun.misc.Unsafe.allocateMemory() OutOfMemoryError [RSS_MEM_USED=4368849184, size=1055916032, memoryTag=45], original message: 2024-07-22T17:51:36.654908Z C i.q.c.TableWriter could not sort varsize o3 column [table=testWalWriteFullRandom_nonwal, column=var_top, type=VARCHAR, long0=1717520080960, long1=-1, long2=-1, long3=-1, errno=-1, ex=sun.misc.Unsafe.allocateMemory() OutOfMemoryError [RSS_MEM_USED=4368849184, size=1055916032, memoryTag=45], original message: ]

2024-07-22T17:51:37.098299Z I i.q.c.p.WriterPool << [table=`testWalWriteFullRandom_nonwal~`, thread=811]
2024-07-22T17:51:37.124256Z E i.q.t.AbstractCairoTest Error in test: 
io.questdb.cairo.CairoException: [-1] commit failed, see logs for details [table=testWalWriteFullRandom_nonwal, tableDir=testWalWriteFullRandom_nonwal~]
	at io.questdb.cairo.CairoException.instance(CairoException.java:323)
	at io.questdb.cairo.CairoException.critical(CairoException.java:69)
	at io.questdb.cairo.TableWriter.checkO3Errors(TableWriter.java:3399)
	at io.questdb.cairo.TableWriter.consumeColumnTasks(TableWriter.java:3706)
	at io.questdb.cairo.TableWriter.dispatchColumnTasks(TableWriter.java:4631)
	at io.questdb.cairo.TableWriter.o3Commit(TableWriter.java:5478)
	at io.questdb.cairo.TableWriter.commit(TableWriter.java:3525)
	at io.questdb.cairo.TableWriter.commit(TableWriter.java:994)
	at io.questdb.test.griffin.wal.FuzzRunner.applyNonWal(FuzzRunner.java:233)
	at io.questdb.test.griffin.wal.FuzzRunner.runFuzz(FuzzRunner.java:800)
	at io.questdb.test.griffin.wal.AbstractFuzzTest.lambda$fullRandomFuzz$0(AbstractFuzzTest.java:135)
	at io.questdb.test.AbstractCairoTest.lambda$assertMemoryLeak$8(AbstractCairoTest.java:975)
	at io.questdb.test.tools.TestUtils.assertMemoryLeak(TestUtils.java:582)
	at io.questdb.test.AbstractCairoTest.assertMemoryLeak(AbstractCairoTest.java:972)
	at io.questdb.test.AbstractCairoTest.assertMemoryLeak(AbstractCairoTest.java:955)
	at io.questdb.test.griffin.wal.AbstractFuzzTest.fullRandomFuzz(AbstractFuzzTest.java:135)
	at io.questdb.test.griffin.wal.WalWriterFuzzTest.testWalWriteFullRandom(WalWriterFuzzTest.java:209)
  • OOM exception is logged using println, not using loggers
  • The logging is 2 times nested, OOM inside OOM
  • RSS limit is not logged, only used memory and (allocation?) size.
  • Human readable memory sizes would be preferable

QuestDB version:

8.0.3+

OS, in case of Docker specify Docker and the Host OS:

Windows

File System, in case of Docker specify Host File System:

ntfs

Full Name:

Alex Pelagenko

Affiliation:

QuestDB

Have you followed Linux, MacOs kernel configuration steps to increase Maximum open files and Maximum virtual memory areas limit?

  • Yes, I have

Additional context

No response

Guia do colaborador