Thoughts and tutorials on programming

Thursday, June 18, 2009

How to save MUCH RAM when running rails (linode/slicehost) and mod_rails passenger

If you're using mod_rails on a VPS with little RAM [ex: linode, slicehost], then there are a few things you'll want to do to save RAM when running rails. Here are a few things I did to allow me to run multiple rails processes on one linode slice.

1: Install a 32-bit OS

Ruby uses twice as much RAM if you're in 64-bit than in 32-bit [and most other things do, too]. Use 32-bit! [linode has options to do this easily].

Savings: 50%

2: Don't use as many processes per rails app.
If you have low volume sites (and most of us do), then don't create too many processes per app.

PassengerMaxInstancesPerApp 1

3: If you are only using one process per rails app, then turn off the spawner process--it does you no help.

By default mod_rails spins up one "spawner process" and then "x" "actual working processes" The spawner thread just preloads rails so that it can be shared amongst instances of that app.

In my case, from top (~1 app process):
total mem, RSS, ... name

117m 49m 3156 S 0 13.8 0:00.93 ruby # spawner 49M
171m 89m 2240 S 0 24.7 0:00.61 ruby # instance 89m

fix:
use
RailsSpawnMethod conservative
in your apache2.conf

this results in [slower startup times and]
RAM, RSS...process name
143m 72m 3480 S 0 20.1 0:01.69 ruby
Savings: (assuming you only want one process per rails app): 49m (on 64-bit).

If you're worried about slow startup speed, you can set your rails processes to never expire[4].

4: If you use multiple processes per rails app, then set your spawner process to die quickly.

Though I haven't used it, theoretically the spawner process will be killed eventually--so set it to die quickly, to free up that expensive RAM[5].

You could do something like
ab -n10 -c10 http://yourhost
to ensure it fires up all the processes from the spawner, then the spawner is free to die quickly.

If you don't use REE, then don't use the spawner at all--little savings there RAM wise.

I have a suspicion that even with REE, you don't see much RAM savings, but having not used it, I can't say for certain.

4: use the MBARI patches to MRI.

Originally, one of my rails apps started at 89MB RSS then grew in total RAM usage by ~ 8MB per request, linearly. Odd? Yes. Fix: I recompiled ruby using the MBARI patches to 1.8.6/1.8.7. It now starts at 60MB RSS and stays there solid [1]. That's right--it stays solid at LESS than an unpatched ruby starts at. And with higher speeds [6].

Also avoid the ruby that comes bundled with ubuntu--though it uses the same amount of RAM as normal, it is compiled with pthreads enabled so it is slower.

Using 1.9 might also yield RAM savings like this. Haven't tried it though.

Savings: unknown since it appeared to growing forever (a lot though).

Overall Result:

With these suggestions in place, I can now run 3 or 4 rails apps on my "cheap grade" 360MB linode. Much better than the 1 I was able to originally.

Enjoy!

Other potential tricks:

use nginx instead of apache--faster, much better RAM usage. Potential savings:

Tweak mysql to use less memory [3]

Possibly tweak GC settings (37signals', evan weaver [2], etc.) though this appears to not increase speed too much over the MBARI patches [6].

refs:

http://groups.google.com/group/phusion-passenger/browse_thread/thread/df1fc1073dbef38
[1] http://www.ruby-forum.com/topic/170608#new
[2] http://blog.evanweaver.com/articles/2009/04/09/ruby-gc-tuning/
[3] http://articles.slicehost.com/2007/9/11/ubuntu-feisty-mysql-and-ror
[4] http://www.modrails.com/documentation/Users%20guide%20Apache.html#PassengerPoolIdleTime
[5] http://www.modrails.com/documentation/Users%20guide%20Apache.html#_railsframeworkspawneridletime_lt_integer_gt
[6] http://www.nabble.com/-ruby-core:19846---Bug--744--memory-leak-in-callcc--to20447794.html#a21140287

7 comments:

Roger Pack said...

make sure to not use the 1.8 that ships by default with ubuntu/debian--it isn't RAM friendly and it runs slowly!

Roger Pack said...

wait a second malloc_limit should grow tons if you're using 168MB RAM, shouldn't it? With more RAM use you want less frequent GC, I think.

Roger Pack said...

2K GC blocks, too

Roger Pack said...

if you use apache, this link
http://blog.spikesource.com/apache_keepalive.htm
might be useful

Anonymous said...

Okay, but how does this affect performance, especially under load?

Roger Pack said...

Depends what you mean.
For me, I'm running low traffic sites, and it runs just fine.

In general the MBARI patches don't slow things down [1] (they speed things up). It is true that you'll only have "the equivalent of one mongrel process per site" but if you need more, you can add more now, since you have more RAM available. In general on linode [for me at least] it's RAM that's in short supply, not CPU time.


[1] http://www.nabble.com/-ruby-core:19846---Bug--744--memory-leak-in-callcc--to20447794.html#a21140287

Hongli Lai said...

Please, do not use 'top' to measure memory usage! 'top' includes shared memory, which shouldn't be counted. You should use passenger-memory-stats instead, which reports memory usage correctly. This is the main reason why passenger-memory-stats was created in the first place.

With REE you'll save approximately 33% RAM if you have more than 1 process per application.

Blog Archive

Followers