The idea: one malloc API, three designs
malloc and free are library calls, not system calls. The library gets memory from the kernel in pages (brk, mmap) and cuts it into the small blocks a program asks for. Stack vs Heap in C shows how glibc does that for one thread: chunks with headers, the tcache, top. This page puts three allocators next to each other: glibc malloc (ptmalloc2, the default on Linux), jemalloc (FreeBSD, Redis, Rust programs for years) and tcmalloc (Google's, in its current per-CPU form). The same calls from 4 threads run on all three at once.
All three have the same shape. A front cache answers most calls without a lock. A miss goes to a shared structure behind a lock. Below that, memory comes in pages from the kernel, and some of it goes back.
Reading the canvas
- One column per allocator. At the top: 4 threads on 2 CPUs (T1, T2 on CPU0; T3, T4 on CPU1), one colour per thread.
- Front caches: how many free blocks of each size each cache holds, out of its capacity.
- Shared: one chip per lock, with the number of times it was taken. Orange = taken in this step, red = a thread waits for it.
- Memory: one row per heap or arena (glibc, jemalloc) or the page heap (tcmalloc). Grey cells are pages in the process's RSS; blocks are drawn to scale inside them, in the thread's colour when live, yellow when they sit in a front cache, light green when free inside the allocator.
- Below: RSS, live bytes, locks taken and lock waits so far, and what happened in this step.
Time runs in ticks. In one tick each CPU runs one call. If a call needs a lock the other CPU took in the same tick, it waits a tick: that is lock contention. The caches and batches are smaller than the real ones so the drawing fits (see the last section).
The fast path: a cache in front
| glibc | jemalloc | tcmalloc | |
|---|---|---|---|
| cache | tcache, one per thread | tcache, one per thread | one per CPU, shared by the threads that run there |
| sizes | requests up to 1032 B | up to tcache_max (32 KiB) | all small sizes (up to 256 KiB) |
| capacity | 7 chunks per size | 20–200 per size class | a byte budget per CPU |
| refilled by | free only (and leftovers of a bin) | a batch from the arena's bin | a batch from the central free list |
A cache hit is a few instructions on memory only this thread (or CPU) touches. tcmalloc's per-CPU cache works without locks or atomic instructions through restartable sequences (rseq): if the thread is preempted or moved to another CPU in the middle of the push or pop, the kernel restarts it. In Demo: one thread: malloc, free, malloc, the second malloc is a hit in all three columns and takes no lock.
Behind the cache: arenas, bins and central lists
- glibc keeps several arenas, each with its own heap and one mutex for every size. A thread is attached to an arena; glibc creates new ones up to 8 × cores (on 64-bit) as threads start allocating. A miss takes the arena lock, looks in the bins (best fit) and else carves a new chunk from top. A 4 KiB request is over the tcache limit, so every 4 KiB malloc and free in glibc takes that lock.
- jemalloc also has arenas (4 × CPUs by default), but each arena has a separate bin with its own lock per size class. A miss locks the bin and fills half the tcache at once, so the next calls are hits.
- tcmalloc has no arenas. A miss locks the central free list of the size class (in front of it a transfer cache, also per size class) and moves a batch to the CPU's cache. New memory comes from the page heap as a span: a run of pages that holds objects of one size.
The layout differs too. glibc puts chunks of every size one after another in the same heap, each with an 8-byte size header, and merges free neighbours. jemalloc slabs and tcmalloc spans hold objects of one size only, page aligned, with the bookkeeping kept outside the objects.
Many threads: contention and memory
Demo: 4 threads on 2 CPUs, twice: each thread mallocs 1 KiB, then again. glibc gives each thread its own arena, so nobody waits, but its tcache is empty both times: 8 calls, 8 lock acquisitions. jemalloc locks once per thread and the batch serves the second round: 4. In tcmalloc, T1 and T3 both miss on the same central list in the first tick, so T3 waits one tick; then T2 is served from the cache T1 just filled, because they share CPU0: 2 central and 2 page heap locks, 1 wait.
Demo: producer and consumer threads: T1 allocates 4 KiB buffers and T3 frees them. In glibc the free must lock the arena the chunk came from (T1's main arena), while T1 needs the same lock to allocate: T3 waits twice. jemalloc and tcmalloc put the freed block in T3's own cache, no lock at all. The price is that the memory now sits in a cache on the other side; jemalloc returns it to its owning arena when that cache is flushed.
Per-thread structures also cost memory. With hundreds of threads, glibc's arenas and every thread's cache each keep their own free memory, and a block freed in one arena cannot serve a malloc in another. Per-CPU caches grow with the number of CPUs, not threads.
Fragmentation and giving memory back
A freed block only lowers RSS when a whole page is free and the allocator tells the kernel (madvise or munmap). glibc does this only at the top of a heap, and only when top is larger than M_TRIM_THRESHOLD (128 KiB). Free chunks below a chunk still in use stay resident until malloc_trim() is called. jemalloc returns an empty slab to its arena as a dirty page and purges dirty pages over dirty_decay_ms (10 s by default) wherever they are; tcmalloc returns empty spans to the page heap, and its background thread releases them.
Demo: giving memory back to the OS: T1 allocates six 4 KiB blocks and one 1 KiB block G, frees the six and waits 10 s. glibc merges A–F into one free chunk under G and keeps all 28 KiB. jemalloc's tcache flushed two blocks, their slab pages became empty and were purged: RSS drops to 20 KiB. tcmalloc released the one span that became empty: 24 KiB. The blocks still in the caches keep their pages in both.
Choosing and tuning
| glibc | jemalloc | tcmalloc | |
|---|---|---|---|
| use it | the default | preload it with LD_PRELOAD or link it | link tcmalloc (needs rseq for per-CPU mode) |
| common knobs | MALLOC_ARENA_MAX, the tcache_count tunable, M_TRIM_THRESHOLD | MALLOC_CONF with narenas, dirty_decay_ms, background_thread | release rate, per-CPU cache size |
| stats | malloc_stats(), malloc_info() | malloc_stats_print() and mallctl | GetStats() in MallocExtension |
A common fix for a multithreaded service whose RSS keeps growing on glibc is MALLOC_ARENA_MAX=2 (fewer arenas, less memory held per thread, more contention) or switching to jemalloc or tcmalloc. Measure both RSS and latency: the page shows why they move in opposite directions.
What the page leaves out
Real caches and batches are bigger (glibc keeps 7 chunks per tcache bin, jemalloc 20–200 slots, tcmalloc moves 16–64 objects and sizes its per-CPU caches in bytes), and there are many size classes, not two. glibc's fastbins, unsorted bin and large bins, its mmap path for blocks over 128 KiB and its main-arena brk are left out; jemalloc's extents for large sizes, muzzy pages, tcache garbage collection and arena choice by load; tcmalloc's transfer caches, huge pages (Temeraire) and its byte-based cache resizing. Threads never migrate between CPUs here, and a waiting thread simply loses one tick.