Ruby on Rails Love-In

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ruby on Rails Love-In

«‹›178 »

luminalflux: May 27, 2005

wunderbread384 posted:

The Rails folks don't believe in foreign keys so you probably won't ever see support for them in core. It's stupid, but one of the many downsides to Rails being "opinionated software" is when their opinion is wrong.

Is it just me or is there a huge "don't trust the database" vibe coming from the Rails camp?

# ? Aug 5, 2008 09:39

Adbot: ADBOT LOVES YOU

# ? May 14, 2024 05:11

skidooer: Aug 6, 2001

luminalflux posted:

Is it just me or is there a huge "don't trust the database" vibe coming from the Rails camp?

It's not a matter of trust. It basically boils down to the fact that SQL is the wrong tool for the job. The right tools, unfortunately, are still too immature for real-world use.

Things will get better. As those projects mature you'll see Rails move away from SQL. Those who wrote their applications properly will be able to migrate with ease. Those who decided to put all their faith in the database will be in for a world of hurt or stuck with SQL forever.

Using SQL to it's full potential has it's place; just not when using ActiveRecord.

# ? Aug 5, 2008 15:42

wunderbread384: Jun 24, 2004; America's favorite bread

So here's the thing about ActiveRecord and foreign keys. I completely buy the argument that ActiveRecord should be handling your referential integrity. That's something that should be handled at the model layer, not in your lower level data layer.

But here's what I don't get. ActiveRecord wraps database transactions and all kinds of queries, treating the SQL (and the database) not as the final goal but as a means to an end. They need to store an object, so they use a SQL INSERT and throw it in the database. They need an atomic operation, so they wrap it in database transactions. For whatever reason though they draw the line at foreign keys. Why not wrap foreign keys to perform referential integrity just like you wrap database transactions to perform atomic operations?

I think what happened is they get caught up in treating foreign keys as an end in themselves and forget that they're really just a tool for getting the job done. Probably this is because a lot of database people TREAT foreign keys as an end rather than a means, but that doesn't make it true. So rather than using foreign keys as a tool they explicitly re-invent the wheel and re-implement referential integrity checks in their code because they (probably correctly) assume that RDMSs are going out of favor for storing objects.

And that's all fine, it certainly works OK most of the time, except the problem is that we DO use RDBMSs underneath, at least for now. And by rewriting referential integrity checks in ActiveRecord rather than just wrapping the existing database functionality they miss out on all the very real benefits that foreign keys give you in an RDBMS environment. Plus you'd think they of all people should understand DRY and the advantages of using a heavily tested codebase rather than rewriting it.

Of course the real tragedy of all this is that they're forcing people who either need or want foreign keys to either give up on ActiveRecord or explicitly set foreign keys through plugins or vanilla SQL. And that's the OPPOSITE of what they intended, which was for model layers to be agnostic to whether it's using an RDBMS. Instead of ActiveRecord implicitly detecting whether to use foreign keys and making the code future proof they're forcing people to explicitly declare their keys and locking them in to an RDBMS.

To me it just seems like they're trying to make a statement (which is probably correct) about which way storing objects is headed, but doing it in such a misguided way that they're actually going to move many codebases backwards.

I do love opinionated software most of the time...I think it's what kept Rails from swallowing the XML/WS-* pill.

Carabus posted:

You mean :include and such? I started off just using select and haven't had any use for anything else until joins were called for when displaying informative results like names efficiently. Running calculations on the hash only requires the ids (foreign keys) which the favorites table provides kindly.

So maybe a raw SQL statement would be more appropriate? Although I'm not sure how that would improve my weird amateurish-looking each method.

I don't know if you saw what skidooer posted above you but that's essentially what I was getting at (except he said it in a way more understandable way).

wunderbread384 fucked around with this message at 02:35 on Aug 6, 2008

# ? Aug 6, 2008 02:32

HIERARCHY OF WEEDZ: Aug 1, 2005

So what's the buzz on Sequel? It looks really awesome but I don't hear any news about people using it in any projects.

# ? Aug 6, 2008 02:57

Pardot: Jul 25, 2001

Panic! at the Fist Jab posted:

So what's the buzz on Sequel? It looks really awesome but I don't hear any news about people using it in any projects.

I think datamapper is drawing most of the non-active record users. Of course, non-ar isn't exclusive, but sequel really seems like the 3rd party candidate.

# ? Aug 6, 2008 04:40

luminalflux: May 27, 2005

wunderbread384 posted:

There's a bunch of plugins you can get that will do it, though, try Foreign Key Migrations.

1500 rows of code that can't automatically figure out that my primary key columns aren't named "id", even though the primary keys are specified in the schema, migrations and models. Quality code right there.

Yes, I should probably just let my database be a stupid object store, but a lot of habits (foreign keys, naming primary keys to make joins easier) are hard to shake.

# ? Aug 6, 2008 08:01

HIERARCHY OF WEEDZ: Aug 1, 2005

Pardot posted:

I think datamapper is drawing most of the non-active record users. Of course, non-ar isn't exclusive, but sequel really seems like the 3rd party candidate.

That's bizarre; I've never found DataMapper to be that great, and the new syntax they've introduced for most of their instantisation stuff bugs the hell out of me. In addition, while the community is helpful, their documentation is really sparse. Ah well. Maybe I'll write about using both in a project soon.

# ? Aug 6, 2008 14:43

Nolgthorn: Jan 30, 2001; The pendulum of the mind alternates between sense and nonsense

luminalflux posted:

Yes, I should probably just let my database be a stupid object store, but a lot of habits (foreign keys, naming primary keys to make joins easier) are hard to shake.

Really I found that easy, as soon as I didn't have to touch the database anymore I simply didn't. The added benefit of never having to bother doing anything other than change the data store any time I wish was a great benefit as well.

Edit: Also the ease of using sqlite3, which is my favorite data store.

# ? Aug 6, 2008 23:12

derdewey: Dec 3, 2004

Panic! at the Fist Jab posted:

That's bizarre; I've never found DataMapper to be that great, and the new syntax they've introduced for most of their instantisation stuff bugs the hell out of me. In addition, while the community is helpful, their documentation is really sparse. Ah well. Maybe I'll write about using both in a project soon.

I think sequel is all kinds of awesome. The documentation is actually pretty good. It was better before Jeremy took down the wiki with the quick tips, but still, you can get the job done. And if you look one page back you can see I'm using it for a project (it's barely functional... but still).

Do you have any questions about it? The code is pretty mature and it's got 100% test coverage (line based, I think).

# ? Aug 9, 2008 22:41

functional: Feb 12, 2008

Serialization/Packing

I'm using straight Ruby.

I have a large array of arrays of floats. So fa=[[1.1111, 2.23424],[1.0,1.0,33.442424,9.9999],...]

I can write this to a file using Marshal.dump, and read using Marshal.load, but this gives me a very large file since the representation is in ASCII instead of binary. We would save a lot if 1.33333333333 was stored as a few float bytes instead of the many-byte string "1.3333333333"

This is how I'm doing it with strings. This takes up too much space.

php:

<?
def retrieveFA
  name='myfile.dat'
  if File.exist?(name)
    myf=File.new(name,'r')
    str=myf.read
    fa=Marshal.load(str)
  else
    fa=SOMESTUFF()
    myf=File.new(name,'w')
    str=Marshal.dump(fa)
    myf.write(str)
  end
  fa
end
?>

I've tried unsuccessfully to do this using Array.pack. If anyone could show me a minimal, working example of reading and writing in this way to a file I would be very appreciative.

functional fucked around with this message at 00:50 on Aug 16, 2008

# ? Aug 16, 2008 00:19

Pardot: Jul 25, 2001

functional posted:

Serialization/Packing

Array#pack is indeed what you want. I didn't know about until just now, but check it out:

code:

a = (1..1000).map{ |n| n.to_f/11.0 }
packed = a.pack("G*")

Marshal.dump(a).size      # => 21144
packed.size               # => 8000
packed.unpack("G*") == a  # => true

G does double precision binary, and * tells it to do the whole array.

# ? Aug 16, 2008 05:37

functional: Feb 12, 2008

Pardot posted:

Array#pack is indeed what you want. I didn't know about until just now, but check it out:
code:
a = (1..1000).map{ |n| n.to_f/11.0 }
packed = a.pack("G*")

Marshal.dump(a).size      # => 21144
packed.size               # => 8000
packed.unpack("G*") == a  # => true
G does double precision binary, and * tells it to do the whole array.

I'm not home this weekend so I can't check: does the exact same method work on an array of arrays of floats? When I tried something similar (no star) it was giving me problems.

# ? Aug 17, 2008 05:40

functional: Feb 12, 2008

functional posted:

I'm not home this weekend so I can't check: does the exact same method work on an array of arrays of floats? When I tried something similar (no star) it was giving me problems.

I spent some time with it and I think it's an area in which Ruby is grossly deficient. You shouldn't have to do anything on a multi-dimensional array in terms of specifying dimensionality or type (much less running pack() on each of the members and sorting out that mess whenever you want to write or read). Either Array or Marshal should have a built in function that autodetects all of this and writes it out in a very compact form. With optional settings for endian encodings and such. Ruby serialization needs work.

# ? Aug 18, 2008 20:21

skidooer: Aug 6, 2001

functional posted:

Either Array or Marshal should have a built in function that autodetects all of this and writes it out in a very compact form.

If size is your main concern, wouldn't you be better off writing your own file format? The flexibility of Marshal will require that it always contains more information than necessary for what you are trying to do.

However:

code:

class Float
  def self._load(value)
    value.unpack("G").first
  end
  
  def _dump(*args)
    [self].pack("G")
  end
end

# ? Aug 19, 2008 01:58

functional: Feb 12, 2008

skidooer posted:

If size is your main concern, wouldn't you be better off writing your own file format? The flexibility of Marshal will require that it always contains more information than necessary for what you are trying to do.

However:
code:
class Float
  def self._load(value)
    value.unpack("G").first
  end
  
  def _dump(*args)
    [self].pack("G")
  end
end

Ah, so this is how you would do it. You're a true friend.

functional fucked around with this message at 17:31 on Aug 19, 2008

# ? Aug 19, 2008 17:25

functional: Feb 12, 2008

functional posted:

You're a true friend.

You would be a true friend, but your fix causes problems with mod_ruby. I knew Marshal should have had this built in.

# ? Aug 19, 2008 22:01

skidooer: Aug 6, 2001

functional posted:

Ah, so this is how you would do it.

No, I would still go with a custom format.

code:

module FloatMarshal
  def self.dump(array)    
    header = array.map { |element| element.size }
    header.unshift(header.size)
        
    header.pack("NC*") + array.flatten.pack("G*")
  end
end

I'll leave the load method as an exercise for you.

# ? Aug 20, 2008 15:37

Pardot: Jul 25, 2001

I heard that MonetDB is the new hotness. Has anyone used it at all? There is an active record adapter for it.

# ? Aug 24, 2008 05:05

Nolgthorn: Jan 30, 2001; The pendulum of the mind alternates between sense and nonsense

http://weblog.rubyonrails.org/2008/8/23/dos-vulnerabilities-in-rexml

quote:

The ruby-security team have published an advisory about a DoS bug affecting REXML users. Almost all rails applications will be affected by this vulnerability and you�re strongly advised to take the mitigating steps recommended in the advisory. If you�re not sure whether your application could be affected, you should upgrade.

The announcement contains details describing the monkeypatch solution, but to summarise:
Versions 1.2.6 and earlier

1. Copy the fix file into RAILS_ROOT/lib
2. Require the file from environment.rb require �rexml-expansion-fix�

Versions 2.0.0 and later

Copy the fix file into RAILS_ROOT/config/initializers, it will be required automatically.

This fix is also available as a gem, to install it run:

gem install rexml-expansion-fix

Then add require �rexml-expansion-fix� to your environment.rb file. The manual fix and the gem are identical, if you have applied one you do not need to apply the other.

I also got a message from engineyard urging me to fix my installation.

quote:

Security researchers have uncovered a denial of service vulnerability in the standard Ruby REXML library[1] used to parse XML. Ruby on Rails will attempt to process incoming XML requests by default, regardless of whether your application utilizes XML explicitly. All customers utilizing Ruby on Rails are encouraged to apply the fix immediately. The patch is available from the official Ruby on Rails Weblog[2] along with instructions appropriate to the different versions of Rails.

[1] - http://www.ruby-lang.org/en/news/2008/08/23/dos-vulnerability-in-rexml/
[2] - http://weblog.rubyonrails.org/2008/8/23/dos-vulnerabilities-in-rexml

But as usual I cannot figure out what the big deal is, what happens if someone utilizes this security hole in one of my many Ror applications?

# ? Aug 28, 2008 04:24

sorghum: Jul 9, 2001

It just means that anyone can send a short request to your server that is very resource-intensive for it to process. For example, this sample you linked to would expand to over 30MB in memory (and it would be easy to make something much worse):

code:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE member [
  <!ENTITY a "&b;&b;&b;&b;&b;&b;&b;&b;&b;&b;">
  <!ENTITY b "&c;&c;&c;&c;&c;&c;&c;&c;&c;&c;">
  <!ENTITY c "&d;&d;&d;&d;&d;&d;&d;&d;&d;&d;">
  <!ENTITY d "&e;&e;&e;&e;&e;&e;&e;&e;&e;&e;">
  <!ENTITY e "&f;&f;&f;&f;&f;&f;&f;&f;&f;&f;">
  <!ENTITY f "&g;&g;&g;&g;&g;&g;&g;&g;&g;&g;">
  <!ENTITY g "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx">
]>
<member>
&a;
</member>

By sending a request like this several times someone could easily overload your server(s) and essentially prevent everyone else from getting to your site.

# ? Aug 28, 2008 06:07

manero: Jan 30, 2006

sorghum posted:

It just means that anyone can send a short request to your server that is very resource-intensive for it to process. For example, this sample you linked to would expand to over 30MB in memory (and it would be easy to make something much worse):

I thought they fixed this poo poo in REXML once already. Sheesh.

# ? Aug 28, 2008 17:47

Carabus: Sep 30, 2002; Booya.

I'm using the will_paginate plugin to list some items;

code:

has_many :versions
Package.paginate(:all, :page => page, :conditions => ['name like ?', "%#{search}%"])

However, if I want to order by some field in the versions table, I can't just add :order => 'versions.latest.updated_at'. That's just passed to SQL. Any ideas on how best to use associations on pagination order? I could use paginate_by_sql but seems like a bit much.

# ? Aug 28, 2008 22:27

manero: Jan 30, 2006

Carabus posted:

I'm using the will_paginate plugin to list some items;
code:
has_many :versions
Package.paginate(:all, :page => page, :conditions => ['name like ?', "%#{search}%"])
However, if I want to order by some field in the versions table, I can't just add :order => 'versions.latest.updated_at'. That's just passed to SQL. Any ideas on how best to use associations on pagination order? I could use paginate_by_sql but seems like a bit much.

how about adding an :include => :versions in the paginate() call?

# ? Aug 28, 2008 22:28

Carabus: Sep 30, 2002; Booya.

manero posted:

how about adding an :include => :versions in the paginate() call?

Doesn't seem to make a difference; the exceptions are the same (no such column). It would be a shame if will paginate did not support this, since they have such a nice view helper.

# ? Aug 29, 2008 00:48

Pardot: Jul 25, 2001

Carabus posted:

Doesn't seem to make a difference; the exceptions are the same (no such column). It would be a shame if will paginate did not support this, since they have such a nice view helper.

What sql does it generate for
Package.paginate(:all, :include => :versions :page => page, :conditions => ['name like ?', "%#{search}%"], :order => "versions.created_at DESC")
?

# ? Aug 29, 2008 00:55

Carabus: Sep 30, 2002; Booya.

Pardot posted:

What sql does it generate

code:

SELECT DISTINCT "package".id FROM "package" 
WHERE (name like '%%') 
ORDER BY versions.created_at DESC LIMIT 10 OFFSET 0

# ? Aug 29, 2008 01:15

Pardot: Jul 25, 2001

Carabus posted:

code:

SELECT DISTINCT "package".id FROM "package" 
WHERE (name like '%%') 
ORDER BY versions.created_at DESC LIMIT 10 OFFSET 0

Try :joins => :versions ? That's my last idea.

edit: It's not joining in the versions table with :include, so maybe :joins will work. I haven't used will_paginate in a long time, so I'm not familiar with what it's doing behind the scenes.

Pardot fucked around with this message at 01:23 on Aug 29, 2008

# ? Aug 29, 2008 01:19

Carabus: Sep 30, 2002; Booya.

Aha! It works now:

code:

Package.paginate(:all, :joins => :versions, 
:page => page, :conditions => ['package.name like ?', "%#{search}%"], 
:order => "version.created_at DESC")

Thanks for the help.

# ? Aug 29, 2008 01:30

Jackdonkey: May 31, 2007

I use rails for my company's database or to edit and view my company's database for those who wish to remind me that rails isn't a database. Anyway to print out job travelers and such I use WIN32OLE to print out forms from a copy of Access on the server. Is anyone else in a position where they have to print forms out and if so what do you use?

# ? Sep 16, 2008 21:47

Pardot: Jul 25, 2001

Huge bump, but I thought this was important enough.

I just realized that tin tin is a cartoon version of dhh:

Also has anyone set up nginx to proxy to different apps, but not using subdomains? Like me.com/app1 me.com/app2? I can't quite get it to work.

Pardot fucked around with this message at 07:03 on Oct 10, 2008

# ? Oct 10, 2008 07:01

Nolgthorn: Jan 30, 2001; The pendulum of the mind alternates between sense and nonsense

Pardot posted:

Also has anyone set up nginx to proxy to different apps, but not using subdomains? Like me.com/app1 me.com/app2? I can't quite get it to work.

The only way to do this is to not have public_html be a symlink, as *nix does not support running two daisy-chained symlinks. Or at least that's what I remember the issue being, there is definitely an issue there whatever the issue is.

# ? Oct 11, 2008 15:01

Freakus: Oct 21, 2000

I've been looking at rails to build a website for an online game a few of my friends and I run. However, all the rails tutorials I've ran through seem to assume the project is a new website, but we already have a database schema we need to work off. It also seems like a lot of the 'magic' of rails assumes no pre-existing schema. Please correct me if I'm wrong.

Initially it will be read only, but the eventual goal is to have a web interface for many of the admin tools we currently use in-game. Is rails going to be fighting me a lot on this?

# ? Oct 17, 2008 21:39

Pardot: Jul 25, 2001

Freakus posted:

we already have a database schema

Check out http://wiki.rubyonrails.org/rails/pages/howtouselegacyschemas

It can be done, you just have to be a lot more verbose in all of the relationship decorations. You have to say what the other table's name is, what the primary key is and all of that.

Alternatively, you could check out merb with the datamapper orm which is more flexible. It should go 1.0 by the end of this month. They've had an RC or two.

I should also note that I've never had the, uh pleasure, to use a legacy schema myself.

# ? Oct 17, 2008 21:49

skidooer: Aug 6, 2001

Freakus posted:

Is rails going to be fighting me a lot on this?

Rails, no. ActiveRecord, maybe.

What does your schema look like? If it's close to what ActiveRecord expects, you just have to apply additional configuration options to specify what is different. If your database is doing something completely different you might consider evaluating other ORM options (DataMapper, ruby-sequel, etc.) or writing your own database layer.

skidooer fucked around with this message at 21:53 on Oct 17, 2008

# ? Oct 17, 2008 21:50

Nolgthorn: Jan 30, 2001; The pendulum of the mind alternates between sense and nonsense

I am trying to create a site that is kin to okcupid and need some location based plugin to sort out where cities are and their proximity to one another.

All of the geography based plugins seem to place a lot of emphasis on street names and zip codes, following which it performs and off site lookup using google and whatnot. All I really want is some database tables filled with countries, provinces and cities with the latitude and longitude filled in, then a method which can do lookups based on location and search radius.

Where can I find something more along the lines of and specific to what I need?

For my purposes there is not any reason to be able to type in the city name, a drop down list is ok as that would be close enough.

Help!

# ? Oct 21, 2008 03:10

dustgun: Jun 20, 2004; And then the doorbell would ring and the next santa would come

http://www.geonames.org/ is a good source of data for such things

# ? Oct 21, 2008 03:12

Nolgthorn: Jan 30, 2001; The pendulum of the mind alternates between sense and nonsense

What the heck I'll just GeoKit it, this seems to be the new way of doing things.

# ? Oct 21, 2008 04:27

bitprophet: Jul 22, 2004; Taco Defender

Nolgthorn posted:

What the heck I'll just GeoKit it, this seems to be the new way of doing things.

FWIW my workplace seems to use GeoKit for the newer sites and they seem to like it fine. (I haven't gotten to the geocoding part of the site I'm updating yet, so can't speak about it myself)

# ? Oct 21, 2008 04:40

Hop Pocket: Sep 23, 2003

Confirming that GeoKit is a pretty good solution. Have used it on a couple of projects.

# ? Oct 21, 2008 13:09

Adbot: ADBOT LOVES YOU

# ? May 14, 2024 05:11

skidooer: Aug 6, 2001

Hop Pocket posted:

Confirming that GeoKit is a pretty good solution. Have used it on a couple of projects.

The API is pretty ugly, especially with the changes to AR in 2.1, but it seems to be the best pre-made solution going.

# ? Oct 21, 2008 16:12

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ruby on Rails Love-In

«‹›178 »