Ruby on Rails Love-In

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ruby on Rails Love-In

«‹›178 »

Hammertime: May 21, 2003; stop

Operation Atlas posted:

If you're using an old version of RMagick, there's a memory leak in it. Once the mongrel process gets too big the OS will kill it. That's my #1 guess as to what's going on.

First, upgrade to the latest RMagick/ImageMagick combo. Second (if possible) upgrade to Nginx/Passenger. Way better than mongrel.

Another vote for Nginx+Passenger. Having to care about mongrels was horrible, let alone having to combine RMagick (i.e. memory leak central) with persistent mongrels. Talk about an admin nightmare.

If you're just doing basic stuff, try MiniMagick, it's a very thin wrapper around calling ImageMagick with shell commands with the benefit that your Rails processes won't leak memory/bloat. It's not exactly speedy, but it's clean.

# ? Dec 29, 2009 08:09

Adbot: ADBOT LOVES YOU

# ? May 14, 2024 03:40

GroceryBagHead: Nov 28, 2000

You don't know how much you hate Mongrel until you need to do a rolling restart on a cluster of 4 app servers with 4 processes on each while fighting with Monit so it doesn't kill newly created mongrel processes. Passenger is just a must now.

# ? Dec 29, 2009 14:27

Flobbster: Feb 17, 2005; "Cadet Kirk, after the way you cheated on the Kobayashi Maru test I oughta punch you in tha face!"

Thanks for all the advice. I didn't realize RMagick was so awful memory-wise, so I'll check out replacing it with MiniMagick since all I'm using it for is thumbnail generation.

Unfortunately switching over to something like Nginx+Passenger is out of my hands for now but I'll keep it in mind if I ever move the site off its current hosting.

# ? Dec 29, 2009 16:02

asveepay: Jul 7, 2005; internobody

Randuin posted:

I just have a question since the new thing seems to be running passenger/nginx. Is there any reason to use that combo during development at all?

I use apache/passenger, on my ubuntu machine, and it is awesome. once you set everything up (which takes like 10 minutes) it's total cake, it's always on and you never have to stop/start webrick. Just set a bookmark, and bam.

it is so choice. if you have the means I highly recommend it.

# ? Dec 29, 2009 20:48

Evil Trout: Nov 16, 2004; The evilest trout of them all

GroceryBagHead posted:

You don't know how much you hate Mongrel until you need to do a rolling restart on a cluster of 4 app servers with 4 processes on each while fighting with Monit so it doesn't kill newly created mongrel processes. Passenger is just a must now.

Unicorn is worth a look. You can do a rolling restart pretty much out of the box by sending your master process a USR2.

You also only need monit to monitor one process (the master) because the children are restarted before you can even notice they're down.

I've run basically every Rails deployment system at one point or another and it's my current favorite.

# ? Dec 29, 2009 21:50

Randuin: Dec 26, 2003; O-Overdrive~

asveepay posted:

I use apache/passenger, on my ubuntu machine, and it is awesome. once you set everything up (which takes like 10 minutes) it's total cake, it's always on and you never have to stop/start webrick. Just set a bookmark, and bam.

it is so choice. if you have the means I highly recommend it.

Hrm, what about development logging? Where does it actually show up

# ? Dec 29, 2009 22:10

bitprophet: Jul 22, 2004; Taco Defender

Evil Trout posted:

Unicorn is worth a look. You can do a rolling restart pretty much out of the box by sending your master process a USR2.

You also only need monit to monitor one process (the master) because the children are restarted before you can even notice they're down.

I've run basically every Rails deployment system at one point or another and it's my current favorite.

Unicorn looks like has some excellent ideas behind it, but practically speaking I don't see any obvious benefits it has over Passenger for the average deployment situation. E.g. a soft restart is as easy as touch-ing a file or doing /etc/init.d/apache2 reload. If you migrated to Unicorn from Passenger I'd be interested in hearing why

# ? Dec 29, 2009 22:48

Hop Pocket: Sep 23, 2003

Randuin posted:

Hrm, what about development logging? Where does it actually show up

Shows up in your log/development.log. Actually, make sure to use

code:

RailsEnv development

In your VirtualHost config if you'll be using it for development. Else it will default to production and log to log/production.log.

# ? Dec 30, 2009 15:23

Evil Trout: Nov 16, 2004; The evilest trout of them all

bitprophet posted:

Unicorn looks like has some excellent ideas behind it, but practically speaking I don't see any obvious benefits it has over Passenger for the average deployment situation. E.g. a soft restart is as easy as touch-ing a file or doing /etc/init.d/apache2 reload. If you migrated to Unicorn from Passenger I'd be interested in hearing why

I only ran Passenger for about a month. My site handles millions of Rails requests a day, and I very quickly discovered something about Passenger that I didn't like.

One dark secret about Ruby is that long running processes gain memory. Every major Rails deployment I know of uses some combination of Monit/God to watch each process and restart them when they go above a certain threshold. You can and should minimize them by doing profiling on your memory usage, but even a finely tuned app will still require process restarts from time to time.

In Passenger there is no built in way to monitor the memory of its processes. It does have a system to restart a process after a certain amount of requests though, PassengerMaxRequests. I've also seen people set up cron jobs to just check their memory on the command line and kill them when they get too high and that works too.

The bad part is that when you restart a process, either by sending it a terminating signal on the command line, touching tmp/restart.txt or by using something like PassengerMaxRequests is that Passenger seems to stall all future requests while the new process spins up. In my apps case, this could take 5-10 seconds, and that was noticeable and annoying.

So after a month of running Passenger I switched back to Mongrels and Monit. With Nginx in front, if one mongrel was restarting it would just proxy the request to another one and users wouldn't notice any interruption.

One flaw of Monit, as has been pointed out, is that hot restarts are quite difficult. In Unicorn, they're trivial. If you send a USR2 signal to a master process, it actually spins up a whole new application in the background, and only kills the original one once it's fully loaded. Unlike rolling restarts, there's no risk of people getting a mix of the old and new version of your app while other mongrels restart.

I've also found it performs a lot better, and the nginx confirguration is a lot simpler with a unix domain socket. If I add or remove processes, the nginx configuration can stay the same. My monit configuration is also a lot easier, since I only monitor the master process. My memory bloat is managed by a simple script that runs every 30 minutes.

Anyway, if your site is not receiving many requests per second, you might not experience these problems. For a medium-level traffic site like my own, Passenger was a headache

# ? Dec 31, 2009 05:58

NotShadowStar: Sep 20, 2000

From what you posted, it looks like Rails can get out of control because AR is inefficient (not a surprise). Rails 3 should be much much better, it looks much more intelligent like DataMapper.

http://gist.github.com/264500

Also, Ruby 1.9 and I think Ruby Enterprise Edition (from the Passenger dudes) are much better with the memory management. I've been running 1.9 for a good while now on production stuff without too much difficulty.

# ? Jan 1, 2010 21:16

bitprophet: Jul 22, 2004; Taco Defender

Two almost entirely unrelated questions:

1. Any interest in a plain-old-Ruby thread? Based on the general traffic of this one, I'm guessing there's no need, but figured I'd ask. I could see some (a few...maybe one or two...OK, maybe just me) people interested in a thread about Ruby that doesn't assume Rails use.

2. How do people develop Ruby code for distribution? Meaning that if I'm working on a project that's intended to be used via a script or imported as a library into other code, what's the best way to get it into the system load path and executable path and so forth? (As opposed to using e.g. ./bin/myscript every time, or having client code hard-code the full filesystem path to the library, etc.)

I know there's stuff like Jeweler and Echoe that involve gem building and provide rake tasks, but it looks like all such tools include tons of extra junk I don't want (e.g. forces Github use, becomes a hard dependency of your code, barfs crap all over your tree, etc.) plus it's still unclear whether I'd have to run a Rake task every time I want to "reinstall" my code.

Starting to wonder if it comes down to a choice between manually symlinking everything, or making a shorthand alias for gem build && gem install myproject.gem and mashing it all the time.

(Python has a great solution to this called python setup.py develop which "installs" your code via symlinks -- so you only have to do it once.)

# ? Jan 7, 2010 02:47

Hop Pocket: Sep 23, 2003

bitprophet posted:

Two almost entirely unrelated questions:

1. Any interest in a plain-old-Ruby thread? Based on the general traffic of this one, I'm guessing there's no need, but figured I'd ask. I could see some (a few...maybe one or two...OK, maybe just me) people interested in a thread about Ruby that doesn't assume Rails use.

I wouldn't mind having such a thread around.

# ? Jan 7, 2010 14:43

Flobbster: Feb 17, 2005; "Cadet Kirk, after the way you cheated on the Kobayashi Maru test I oughta punch you in tha face!"

Hop Pocket posted:

I wouldn't mind having such a thread around.

Seconded. The Rails project I have going on is something that I touch maybe once a year, but I've started using Ruby more and more on its own just to do little scripting/data processing jobs. A separate thread would be great for picking up new tricks.

# ? Jan 7, 2010 15:00

bitprophet: Jul 22, 2004; Taco Defender

Good to know. I might try my hand at an OP, but I'm still on the road between neophyte and intermediate user so unsure if I'm the best person for the job. (Plus I'd have a hard time resisting lots of [serious, possibly useful, not snarky] comparisons to Python, as that's my "main" language. I kinda doubt there are a lot of others with that particular orientation.)

OTOH now I'm having lots of possibly good ideas, so maybe I'll write up a test one and post it in here or something. We'll see

# ? Jan 7, 2010 15:12

dizzywhip: Dec 23, 2005

dizzywhip fucked around with this message at 21:51 on Nov 9, 2020

# ? Jan 11, 2010 07:07

Pardot: Jul 25, 2001

I'm not sure what's causing your problem, however you should switch over to gem bundler. You can use the disable_system_gems feature to, well, do what it says and only use the gems you've vendored which is loving awesome.

Read these:
http://yehudakatz.com/2009/11/03/using-the-new-gem-bundler-today/
http://litanyagainstfear.com/blog/2009/10/14/gem-bundler-is-the-future/
and the readme for the project http://github.com/wycats/bundler

It's going to be default in rails 3, but you can use it now. And you should.

# ? Jan 11, 2010 08:16

dizzywhip: Dec 23, 2005

dizzywhip fucked around with this message at 21:51 on Nov 9, 2020

# ? Jan 11, 2010 09:24

NotShadowStar: Sep 20, 2000

I made it a habit since Rails < 1.0 to always kill the development server when I'm not interacting with the app. It's much better than it used to be, but it's still sometimes questionable when you need a server restart or not.

# ? Jan 12, 2010 04:48

Hammertime: May 21, 2003; stop

NotShadowStar posted:

I made it a habit since Rails < 1.0 to always kill the development server when I'm not interacting with the app. It's much better than it used to be, but it's still sometimes questionable when you need a server restart or not.

Anything in /app gets reloaded cleanly, anything in /lib /config or gem related needs a restart. I restart on route changes too, though not sure if that's required these days.

# ? Jan 12, 2010 09:10

dizzywhip: Dec 23, 2005

dizzywhip fucked around with this message at 21:51 on Nov 9, 2020

# ? Jan 12, 2010 09:38

Evil Trout: Nov 16, 2004; The evilest trout of them all

Hammertime posted:

Anything in /app gets reloaded cleanly, anything in /lib /config or gem related needs a restart. I restart on route changes too, though not sure if that's required these days.

Actually /lib hasn't needed a restart in at least a year's worth of Rails releases.

# ? Jan 12, 2010 16:21

Hammertime: May 21, 2003; stop

Evil Trout posted:

Actually /lib hasn't needed a restart in at least a year's worth of Rails releases.

Interesting, thanks! Fair enough, I hook some /lib stuff through my initializers, so I have to anyway. I've been on a single rails project for over a year, so my misconception makes sense.

# ? Jan 13, 2010 12:00

NotShadowStar: Sep 20, 2000

script/console needs restarting if you look at it funny.

# ? Jan 13, 2010 20:48

Lamont Cranston: Sep 1, 2006; how do i shot foam

I've got a bit of a problem getting rails to use the right version of Ruby. My hosting provider has 1.8.6 living in /usr/bin, while I have 1.8.7 in /usr/local/bin. When I run the ruby command I get 1.8.7 but Rails has been using 1.8.6 for some reason. Any idea where this gets set or if there's an environment variable I can set up to get it looking in the right place? I'm running Thin behind Apache.

ed: I get this error when I try to run script/console, if it helps:

code:

/usr/local/lib/ruby/site_ruby/1.8/rbconfig.rb:6: ruby lib version (1.8.6) doesn't match executable version (1.8.7) (RuntimeError)
	from /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:62:in `require'
	from /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:62
	from ./script/../config/../vendor/rails/railties/lib/rails/vendor_gem_source_index.rb:1:in `require'
	from ./script/../config/../vendor/rails/railties/lib/rails/vendor_gem_source_index.rb:1
	from ./script/../config/../vendor/rails/railties/lib/rails/gem_dependency.rb:1:in `require'
	from ./script/../config/../vendor/rails/railties/lib/rails/gem_dependency.rb:1
	from ./script/../config/../vendor/rails/railties/lib/initializer.rb:10:in `require'
	from ./script/../config/../vendor/rails/railties/lib/initializer.rb:10
	from ./script/../config/boot.rb:45:in `require'
	from ./script/../config/boot.rb:45:in `load_initializer'
	from ./script/../config/boot.rb:38:in `run'
	from ./script/../config/boot.rb:11:in `boot!'
	from ./script/../config/boot.rb:109
	from script/console:2:in `require'
	from script/console:2

Lamont Cranston fucked around with this message at 21:52 on Jan 13, 2010

# ? Jan 13, 2010 21:50

Pardot: Jul 25, 2001

All the script/* scripts have #!/usr/bin/env ruby, so make sure whatever PATH is in env, has the ruby you want first. There's also GEM_HOME and GEM_PATH that may cause problems, but I'm not sure.

I just use the rvm gem to manage different versions of ruby for me. You might want to take a look at that, it does a fantastic job, and keeps everything local to your user.

# ? Jan 13, 2010 22:04

NotShadowStar: Sep 20, 2000

Pardot posted:

I just use the rvm gem

Holy poo poo. This owns beyond everything that's owned. No more loving around with environments.

# ? Jan 14, 2010 00:13

jonnii: Dec 29, 2002; god dances in the face of the jews

NotShadowStar posted:

Holy poo poo. This owns beyond everything that's owned. No more loving around with environments.

Yeah, it's pretty fab. You can even do stuff like set it to run your rake tasks sequentially across various versions of ruby, so you can test on REE/1.9/1.8 etc...

# ? Jan 14, 2010 00:19

Splinter: Jul 4, 2003; Cowabunga!

Edit: found answer on first page of thread. Who woulda thought?

Splinter fucked around with this message at 03:11 on Jan 19, 2010

# ? Jan 14, 2010 02:58

Lamont Cranston: Sep 1, 2006; how do i shot foam

Looks like the problem came from installing 1.8.7 overtop of 1.8.6. I removed my /usr/local/lib/ruby and reinstalled and everything seems to be going well.

# ? Jan 14, 2010 06:22

Evil Trout: Nov 16, 2004; The evilest trout of them all

NotShadowStar posted:

script/console needs restarting if you look at it funny.

True. Fortunately there's an easy way to do it, by typing reload!.

# ? Jan 14, 2010 16:38

Nigger Goku: Dec 11, 2004; あっちに行け

Evil Trout posted:

True. Fortunately there's an easy way to do it, by typing reload!.

Though that only reloads the same stuff that script/server would between requests, so config/etc. changes still need a full restart. It'll also keep all of your in-scope objects using the old versions of their classes, so if you get weird errors like 'A copy of Foo has been removed from the module tree' you should reassign/recreate those objects.

# ? Jan 17, 2010 08:45

NotShadowStar: Sep 20, 2000

Yikes, Rails 3 is going to depreciate a lot of very common ways of finding records in AR. Be warned. Rails 3 is going to be a huge transition for existing projects if you don't want to rely on some sort of 'classic queries' plugin.

http://m.onkey.org/2010/1/22/active-record-query-interface

# ? Jan 22, 2010 21:55

Sewer Adventure: Aug 25, 2004

NotShadowStar posted:

Yikes, Rails 3 is going to depreciate a lot of very common ways of finding records in AR. Be warned. Rails 3 is going to be a huge transition for existing projects if you don't want to rely on some sort of 'classic queries' plugin.

http://m.onkey.org/2010/1/22/active-record-query-interface

At first I thought it was stupid and then I saw the lazy loading and the where stuff and now I think it's a good move.

# ? Jan 22, 2010 22:03

Ghotli: Dec 31, 2005; what am art?
art am what?
what.

Yeah it also says in that article that deprecated code will be moved into a plugin so it won't mean 100% code breakage. It's also not going to be deprecated until rails 3.1 so you can get your feet wet with rails 3 and have some time to install a plugin or rewrite some of your models.

# ? Jan 22, 2010 22:13

Splinter: Jul 4, 2003; Cowabunga!

Anyone else having trouble downloading from RubyForge? All last night and today I haven't been able to get more than around 8KB of a file before the download died.

# ? Jan 22, 2010 23:54

Operation Atlas: Dec 17, 2003; Bliss Can Be Bought

Splinter posted:

Anyone else having trouble downloading from RubyForge? All last night and today I haven't been able to get more than around 8KB of a file before the download died.

Rubyforge is dead. The official gem source is now Gemcutter. Do a gem update --system and that may take care of it. If not add gemcutter to your sources and remove rubyforge.

# ? Jan 23, 2010 04:42

Pardot: Jul 25, 2001

Operation Atlas posted:

Rubyforge is dead. The official gem source is now Gemcutter. Do a gem update --system and that may take care of it. If not add gemcutter to your sources and remove rubyforge.

The gemcutter transition was really fast, like a month tops. Yet as a community we still haven't really embraced 1.9 :smith:

# ? Jan 23, 2010 09:34

GroceryBagHead: Nov 28, 2000

Pardot posted:

The gemcutter transition was really fast, like a month tops. Yet as a community we still haven't really embraced 1.9

Few things helped: Github gem hosting went tits-up and Github recommended Gemcutter as an alternative. Rubyforge was good for nothing other than being 'official' gem repository. Eventualy guy behind Rubyforge agreed that Gemcutter will server better for hosting gems. .. and then rubygems started to use Gemcutter as default.

# ? Jan 23, 2010 20:53

Pardot: Jul 25, 2001

I'm also curious as to how many accidental hits gemcutter.com is getting now. I type that in wrong every other day :argh:

# ? Jan 23, 2010 21:11

Adbot: ADBOT LOVES YOU

# ? May 14, 2024 03:40

NotShadowStar: Sep 20, 2000

Pardot posted:

The gemcutter transition was really fast, like a month tops. Yet as a community we still haven't really embraced 1.9

It's easy to redirect to a different location, it's really hard to do things differently when significant changes in your language can cause breakage. The 1.9 encoding bit me the other week on an i18n Rails app, getting TemplateErrors all over the place and took me forever to figure out and fix (the 'official' sqlite3 driver always returns ASCII-8. Always. Rails defaults to all data in templates being UTF-8 and if not you get a template error)

# ? Jan 23, 2010 21:31

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ruby on Rails Love-In

«‹›178 »