tag:blogger.com,1999:blog-9149523927864751087.post5990473545648128051..comments2024-03-26T09:43:01.052-07:00Comments on Small Datum: MyRocks, malloc and fragmentation -- a strong case for jemallocMark Callaghanhttp://www.blogger.com/profile/09590445221922043181noreply@blogger.comBlogger2125tag:blogger.com,1999:blog-9149523927864751087.post-83226232767261673972018-05-08T10:46:37.036-07:002018-05-08T10:46:37.036-07:00I am interested in reading results from such tunin...I am interested in reading results from such tuning, but I won't run those tests.<br /><br />I am wary of depending on huge pages. I have smart friends who prefer we don't use them on production servers. I am also wary of tuning malloc, we already have too much tuning in RocksDB so I don't want to extend that cost to more things.<br /><br />I have yet to repeat tests to determine the impact of arena_block_size. It is 8mb on MyRocks in Percona Server 5.7.21. That seems large enough to avoid fragmentation, but I don't have time to determine the impact of changing it. Eventually we run out of time and HW for running perf tests and need systems that don't require so much tuning.Mark Callaghanhttps://www.blogger.com/profile/09590445221922043181noreply@blogger.comtag:blogger.com,1999:blog-9149523927864751087.post-47723881413466917862018-05-07T19:07:42.740-07:002018-05-07T19:07:42.740-07:00Have you used hugepages via https://github.com/fac...Have you used hugepages via https://github.com/facebook/rocksdb/wiki/Allocating-Some-Indexes-and-Bloom-Filters-using-Huge-Page-TLB and fiddled with arena_block_size as well? Moving indexes to huge pages should reduce your fragmentation.... Also arena_block_size also helps force use of the rocksdb private allocator as opposed to malloc/jemalloc at all.... <br />Anonymousnoreply@blogger.com