Large RSS memory growth with :celluloid and (less so) :hiredis drivers

[richardkmichael]

I'm working with a Celluloid / Redis system which is experiencing very fast RSS growth. I'm not sure this is "an issue" with celluloid-redis per se, but I'm uncertain where/how to discuss it.

Here is a minimal test-case, 1 Actor publishing into a Redis socket. It's Redis 2.8.7, Rubinius 2.2.6, on Linux 3.13.x.

require 'redis'
require 'hiredis'
require 'celluloid'
require 'celluloid/redis'
require 'celluloid/autostart'

class Leaky
  include Celluloid

  attr_reader :driver, :redis

  def initialize
    @driver = :celluloid
    # @driver = :hiredis

    # @redis = ::Redis.new url: 'redis://localhost:6379', driver: @driver
    @redis = ::Redis.new path: '/tmp/redis.sock',       driver: @driver

    @publish_count = 0
  end

  def publish
    @redis.publish "test", @publish_count += 1
  end
end

leaky = Leaky.new
1000000.times { leaky.publish }

Using the :celluloid driver, VmRSS starts at ~66MB and after 140s grows to ~927MB.

If I change to the :hiredis driver, VmRSS starts at ~62MB and after 190s grows to ~116MB. Still quite an increase, but obviously much much less than with :celluloid.

I've heap dumped (-Xagent.start, rbx console ...) every 10s or so while the process is running and RBX doesn't see this amount of memory churn; object counts and consumed bytes are nowhere near this amount of memory and are GC'd as expected. (Increased/decreased object counts using heap compare.) This makes me wonder if there's a problem outside the RubyVM.

Crunched data from /proc/<pid>/status, rbx compared heap dumps, the same code is in this gist.

I've likely misunderstood, or am doing something wrong, but I don't know what. Any comments or suggestions are very much appreciated.

(I will test with MRI and on MacOS shortly.)