How to reduce memory usage in Ruby

Question

Sidekiq workers consume a lot of memory: how can I fix that?

I have a Rails application that can usually run with about 1 GB of RAM. However there are some Sidekiq workers that constantly grow over time – for days – and consume a lot of memory (e.g. 2.5GB).

Are there any solutions?

Answer

The problem is related to Ruby threads and how memory allocation works at low-level. In many cases it’s not a memory leak, but just a memory bloat.

Here’s two effective solution that we have used in production to reduce the memory usage:

  • Reduce the number of Sidekiq threads (e.g. use 10 threads instead of 25)
  • Set the environment variable MALLOC_ARENA_MAX=2 for tuning glibc memory behavior.

The environment variable is a no-brainer. Here’s the results that we got by adding it (max RAM in 16h):

  • web: 1191Mi -> 966Mi (~20% less memory)
  • worker type 1: 911Mi -> 348Mi (~60% less memory)
  • worker type 2: 549Mi -> 304Mi (~45% less memory)

Note that: