Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
shrughes
Oct 11, 2008

(call/cc call/cc)

Markov Chain Chomp posted:

The rule of thumb is to prefer more abstraction, not less, unless you have a very particular reason for doing so.

The rule of thumb of a fresh college graduate.

Adbot
ADBOT LOVES YOU

Catalyst-proof
May 11, 2011

better waste some time with you

Ithaqua posted:

It saves time and effort. How many times have you written code that:

1) Sets up a SQL command
2) Executes the command and gets back a dataset (with proper exception handling, etc)
3) Parses the dataset into an object model of some kind

Using Python's SqlSoup or Ruby's Sequel? Never.

blorpy
Jan 5, 2005

shrughes posted:

The rule of thumb of a fresh college graduate.

Are you officially stalking me now

Johnny Cache Hit
Oct 17, 2011

Markov Chain Chomp posted:

There's literally no reason to be operating with your DB directly

Markov Chain Chomp posted:

Also, nothing forbids you from using both.

But... but the goalposts :mad:

npe posted:

I wasn't seriously advocating that it be used everywhere

Nah, I didn't think you were. I'm sure it has its places -- I have seen it before, and it was on a moderately complex web app, and it was terrible, but everything in that codebase was some degree degrees of :gonk: so I've got lots of bad tastes in my mouth.

Hammerite
Mar 9, 2007

And you don't remember what I said here, either, but it was pompous and stupid.
Jade Ear Joe

wwb posted:

I'll also add it is possible to nest transactions.

Not in MySQL.

A A 2 3 5 8 K
Nov 24, 2003
Illiteracy... what does that word even mean?

Markov Chain Chomp posted:

The rule of thumb is to prefer more abstraction, not less, unless you have a very particular reason for doing so.

I hope you remember saying this ten years from now. You'll get a kick out of it.

csammis
Aug 26, 2003

Mental Institution

Hammerite posted:

Not in MySQL.

Haha seriously?

MySQL Reference posted:

Transactions cannot be nested. This is a consequence of the implicit commit performed for any current transaction when you issue a START TRANSACTION statement or one of its synonyms.

Transactions cannot be nested as a consequence of their inability to be nested :thumbsup:

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe

Markov Chain Chomp posted:

The rule of thumb is to prefer more abstraction, not less, unless you have a very particular reason for doing so.

I apparently have very good particular reasons all the time, then.

Furthermore, SQL is already an amazing abstraction. It completely takes out of the picture of how to store data and just lets you write "please insert this data, thanks". All an ORM does is let you pretend that tables are classes and rows are objects and columns are members.

Not only are there cases when pretending things are objects is not OK, there are cases where ORMs do not provide the expressive power of raw SQL. Just like compiling from a high-level language to assembly restricts you to the subset of features that the high-level language has, compiling from an ORM's language to SQL restricts you to the subset of features that the ORM has.

And because SQL is a unique language in that it lets you express what you want to do, not being bogged down by implementation details, you're cutting out a lot of features when you restrict yourself to an ORM's feature set. For simple CRUD operations, this is perfectly fine, but when you start wanting asking complex queries like "what's the average of all the star ratings that customer A, B and C for products they ordered in the last thirty days that shipped in under a week?", you start pushing the limits of the ORM. Yes, this was a real report that I implemented two weeks ago.

McGlockenshire
Dec 16, 2005

GOLLOCKS!

Hammerite posted:

Not in MySQL.

They do have transaction savepoints, which can be used for similar, though not identical purposes as nested transactions.

NotShadowStar
Sep 20, 2000

Suspicious Dish posted:

I apparently have very good particular reasons all the time, then.

Furthermore, SQL is already an amazing abstraction. It completely takes out of the picture of how to store data and just lets you write "please insert this data, thanks". All an ORM does is let you pretend that tables are classes and rows are objects and columns are members.

Not only are there cases when pretending things are objects is not OK, there are cases where ORMs do not provide the expressive power of raw SQL. Just like compiling from a high-level language to assembly restricts you to the subset of features that the high-level language has, compiling from an ORM's language to SQL restricts you to the subset of features that the ORM has.

And because SQL is a unique language in that it lets you express what you want to do, not being bogged down by implementation details, you're cutting out a lot of features when you restrict yourself to an ORM's feature set. For simple CRUD operations, this is perfectly fine, but when you start wanting asking complex queries like "what's the average of all the star ratings that customer A, B and C for products they ordered in the last thirty days that shipped in under a week?", you start pushing the limits of the ORM. Yes, this was a real report that I implemented two weeks ago.

code:
records = Order.where(order_date: Date.today..1.month.ago, customer: [a, b, c]).all
records.reject! { |record| (record.shipped_date - record.arrival_date) > 1.week }
records.inject { |sum, rec| sum += rec.rating}.to_f / records.size
Welp

NotShadowStar fucked around with this message at 08:02 on Dec 10, 2011

fryzoy
Sep 21, 2005
What.

Suspicious Dish posted:

And because SQL is a unique language in that it lets you express what you want to do, not being bogged down by implementation details.

Ho boy, wait until you actually DO use a different database software than MySQL! Yeah sure, as long as you stick to simple select/update queries you will be fine, but if you ever get as far as set operations or the likes and switch between DB2/pgsql/mysql/whatever, you'll be poo poo out of luck.

gonadic io
Feb 16, 2011

>>=

wwb posted:

Please use the 2nd answer. Works great :)

For a second you got me all excited that Java had anonymous functions. Nope, C# :(

tef
May 30, 2004

-> some l-system crap ->

NotShadowStar posted:

code:
records = Order.where(order_date: Date.today..1.month.ago, customer: [a, b, c]).all
records.reject! { |record| (record.shipped_date - record.arrival_date) > 1.week }
records.inject { |sum, rec| sum += rec.rating}.to_f / records.size
Welp

doesn't that do a simpler query and run the rest in the language? for aggregates I would imagine doing it in sql to be cheaper

Zombywuf
Mar 29, 2008

I've yet to meet an ORM I liked, I've found them difficult to use to do aggregates, non-trivial joins, set operations, anything that looks like a CTE, self joins, atomic updates to multiple rows or do get the kind of fine grained control over transactions that I often need.

Also this is way easier for me to read than NotShadowStar's ORM example. It's smaller too.
code:
select
    avg(star_rating)
from
    Order
where
    datediff(m, order_date, getdate()) >= 1
    and customer in (a, b, c)
    and datediff(w, shipped_date, arrival_date) < 1;
ORMs also make it hard to work with views, stored procedures, functions, triggers, merges (DUI), basically everything I've ever done with a DB would be harder with an ORM.

Zamujasa
Oct 27, 2010



Bread Liar
To get away from the ORM/SQL talk for a moment, let me share a gem from a file at work:

php:
<?
(some code here...)


$mysql = "Select [some stuff here] where username = '" . $user . "' AND password = '".$pass."'";
$result = mysql_query("$mysql");
$num_rows = mysql_num_rows($result);
if ($num_rows > 0){
$row = mysql_fetch_assoc($result);
$mylocarray = explode(",",$row['locations']);
foreach($mylocarray as $k) {
    if ($k = $locationid) {
    $goodtogo = 1;
}
}
mysql_free_result($result);
if (isset($locationid)) {
if ($goodtogo=1) {

$mysql = "select [more stuff here] where locationid = " . $locationid . " and idnum = " . $numid;
 
$result = mysql_query("$mysql");
$num_rows = mysql_num_rows($result);
if ($num_rows > 0){




/*
 about 280 lines of similar code later...
*/



mysql_free_result($result3);
}
}

$xml_output .= "</results>\r\n";
echo $xml_output;




// END CODE FROM <some file>.php
}}}}}}
}// goodtogo = 1, locationid passed is available to the passed username/password
 }// end of check to see if location id is passed
} // end of valid user/pass test and location return
} // end of is api command '<xxx>'
mysql_close($con);
} // End of is apifuncset

?>

?>
The lack of any sort of meaningful indentation or much of any other signs of intelligent life are not a copy-paste artifact, it really is like that.

I like to imagine that the line of }s at the end was just "let's see how many it takes to make this work", save, reload, add a }, repeat until page works.

They also use $row=mysql_fetch_assoc($res); do { ... } while ($row=mysql_fetch_assoc($res)) which is all sorts of wrong.



:smithicide:

NotShadowStar
Sep 20, 2000

tef posted:

doesn't that do a simpler query and run the rest in the language? for aggregates I would imagine doing it in sql to be cheaper

I can do it entirely in SQL methods with AR but this would actually be cheaper since Ruby's Enumerable is written in C plus it was one in the morning and I'm posting on an internet comedy forum so gently caress haters.

SavageMessiah
Jan 28, 2009

Emotionally drained and spookified

Toilet Rascal
I use EODSQL at work instead of a normal ORM. You write your own SQL but it deals with all the boilerplate crap like marshalling to and from SQL types, binding parameters, etc. Best of both worlds if you prefer to write your own SQL.

Cocoa Crispies
Jul 20, 2001

Vehicular Manslaughter!

Pillbug

NotShadowStar posted:

this would actually be cheaper since Ruby's Enumerable is written in C

Might be more expensive since any rows not returned by the DB don't have to be sent over the network.

2nd Rate Poster
Mar 25, 2004

i started a joke
Demanding SQL only though an ORM is the coding horror.

Even basic CRUD apps break down with ORMs when you have a sizable data set. Temp tables can do a lot to help performance on simple queries on large amounts of data, even in cases where you're only using a couple joins.

Let me know when you find an ORM that does that optimization for you.

trex eaterofcadrs
Jun 17, 2005
My lack of understanding is only exceeded by my lack of concern.

NotShadowStar posted:

code:
records = Order.where(order_date: Date.today..1.month.ago, customer: [a, b, c]).all
records.reject! { |record| (record.shipped_date - record.arrival_date) > 1.week }
records.inject { |sum, rec| sum += rec.rating}.to_f / records.size
Welp

In order to avoid sql you'd write all this? This is probably an order of magnitude slower than just writing the SQL.

Hammerite
Mar 9, 2007

And you don't remember what I said here, either, but it was pompous and stupid.
Jade Ear Joe

McGlockenshire posted:

They do have transaction savepoints, which can be used for similar, though not identical purposes as nested transactions.

I had no idea about this, thanks for pointing it out to me.

NotShadowStar
Sep 20, 2000

TRex EaterofCars posted:

In order to avoid sql you'd write all this? This is probably an order of magnitude slower than just writing the SQL.

Point out where anybody said speed or efficiency, that was a response to 'you just can't do it'.

Good lord for being sperg kingdom you guys sure do miss details.

Zombywuf
Mar 29, 2008

NotShadowStar posted:

Point out where anybody said speed or efficiency, that was a response to 'you just can't do it'.

Good lord for being sperg kingdom you guys sure do miss details.

*writes code that could take days to complete when displaying a web page*

*complains that "You never said it had to be fast"*

NotShadowStar
Sep 20, 2000

Zombywuf posted:

*writes code that could take days to complete when displaying a web page*

*complains that "You never said it had to be fast"*

*posts piles of hyperbole

tef
May 30, 2004

-> some l-system crap ->
a quick recap for those who haven't been reading the thread

quote:

when you start wanting asking complex queries [...] , you start pushing the limits of the ORM.

Pushing the limits. No-one said it had to be fast explicitly, but the implication was that orms have limitations and sometimes you must use sql. speed is one of those mitigating factors in choosing to use sql over an orm.

Your 'it can be done in an orm :v:' was a facile statement beside the point. orms can model a large chunk of things. we know this. however it turns out there are tradeoffs, like performance,e that are not made in your favour.

If you're going to contribute with your deadly wit and canny ripostes, it would help if we didn't have to explain all the words in the post you're replying to. :3:



also, 'ruby is written in c so it's fast' :allears:

NotShadowStar
Sep 20, 2000
.

Internet Janitor
May 17, 2008

"That isn't the appropriate trash receptacle."

tef posted:

also, 'ruby is written in c so it's fast' :allears:

Boy, just imagine if it was written in hand-rolled assembly.

Factor Mystic
Mar 20, 2006

Baby's First Post-Apocalyptic Fiction

tef posted:

also, 'ruby is written in c so it's fast' :allears:

ruby + mysql via orm, the fastest way to write your web three point blah apps for

Blotto Skorzany
Nov 7, 2008

He's a PSoC, loose and runnin'
came the whisper from each lip
And he's here to do some business with
the bad ADC on his chip
bad ADC on his chiiiiip

tef posted:

also, 'ruby is written in c so it's fast' :allears:

I know what you're poking fun at here, but taking a slightly different (and I think complementary) tack:

The database is written in C, and it's fast.

Even lovely databases like MySQL have had a ton of work done to make them fast, especially for common cases, often by very smart people with lots of domain experience who were being paid to do nothing but make the database faster. Beating the database's query optimizer or whatever by manually specifying a bunch of poo poo is generally not something to try your hand at if you don't have a good reason (although it's not as quixotic as trying to beat the compiler's register allocator*), and you're almost certainly not going to beat it by handing off a bunch of operations to your ORM layer (which will abuse the database by pretending it's a flat file or some other poo poo frighteningly often).

This isn't to say that ORMs aren't useful, but rather that it's misleading to say that moving work to that layer instead of the DB is 'fast'. Performance statements are always in comparison to something else; letting the ORM do most of the work may (or may not) be 'fast' in comparison to the minimum acceptable level of performance for your application at the problem size you expect to encounter, but it probably isn't 'fast' in comparison to letting the DB do what it was designed to do.


*there are specific cases where even this usually thankless task pays off, usually in the realm of vectorized code

Atimo
Feb 21, 2007
Lurking since '03
Fun Shoe
Found this poking around one of our business objects, trying to figure out why putting a single white space in a text field was crashing an lob web app

code:
public class SomeBusinessObject {/*a zillion properties*/}

public class SomeUIObject : UserControl
{
    //a bunch of garbage here ...

    public void getParam(ref SomeBusinessObject instance)
    {
        //mutate the class, and hell, 
        //while we are here, mutate some global page state to!
    }

    //more garbage to follow ...
}
REF DOES NOT WORK THIS WAY

For even more fun, the calling site of getParam is on the page hosting the user control.

Entire goddamn 100+ user site is written like this, and worse.

The contractor who wrote most of this was offered a position but turned it down - they didn't offer enough money.


edit - Almost forgot, the getParam has to pull some vital peice of information about the current work. What's the best way to pull it, considering we already queried the database in a seperate page?

I know!

var theValue = ((Label)Page.Master.Master.Master.FindControl("xxx").FindControl("xxx").FindControl("xxx").FindControl("xxx").FindControl(someLabel)).Text;

Atimo fucked around with this message at 04:17 on Dec 11, 2011

tef
May 30, 2004

-> some l-system crap ->

Otto Skorzeny posted:

I know what you're poking fun at here, but taking a slightly different (and I think complementary) tack:

The database is written in C, and it's fast.

Even lovely databases like MySQL have had a ton of work done to make them fast, especially for common cases, often by very smart people with lots of domain experience who were being paid to do nothing but make the database faster. Beating the database's query optimizer or whatever by manually specifying a bunch of poo poo is generally not something to try your hand at if you don't have a good reason (although it's not as quixotic as trying to beat the compiler's register allocator*), and you're almost certainly not going to beat it by handing off a bunch of operations to your ORM layer (which will abuse the database by pretending it's a flat file or some other poo poo frighteningly often).

This isn't to say that ORMs aren't useful, but rather that it's misleading to say that moving work to that layer instead of the DB is 'fast'. Performance statements are always in comparison to something else; letting the ORM do most of the work may (or may not) be 'fast' in comparison to the minimum acceptable level of performance for your application at the problem size you expect to encounter, but it probably isn't 'fast' in comparison to letting the DB do what it was designed to do.


*there are specific cases where even this usually thankless task pays off, usually in the realm of vectorized code

I am fully expecting not shadow star to reply to this post with a thoughtful ruby snippet.

NotShadowStar
Sep 20, 2000

tef posted:

I am fully expecting not shadow star to reply to this post with a thoughtful ruby snippet.

What the gently caress is your problem?

Brecht
Nov 7, 2009

NotShadowStar posted:

What the gently caress is your problem?
You:

NotShadowStar posted:

this would actually be cheaper since Ruby's Enumerable is written in C
You are the coding horror.

blorpy
Jan 5, 2005

tef posted:

a quick recap for those who haven't been reading the thread


Pushing the limits. No-one said it had to be fast explicitly, but the implication was that orms have limitations and sometimes you must use sql. speed is one of those mitigating factors in choosing to use sql over an orm.

Your 'it can be done in an orm :v:' was a facile statement beside the point. orms can model a large chunk of things. we know this. however it turns out there are tradeoffs, like performance,e that are not made in your favour.

If you're going to contribute with your deadly wit and canny ripostes, it would help if we didn't have to explain all the words in the post you're replying to. :3:



also, 'ruby is written in c so it's fast' :allears:

Are you daft, tef? Any decent ORM would turn that given example into exactly the same SQL that a human would write (although not if you tried to write it the way notshadowstar did). And with a good JIT the differences would disappear outright. Maybe you ought to step out of the world of turtles and into the modern era. :)

blorpy
Jan 5, 2005

For reference, here's a code snippet from one ORM[1] which makes quick work out of that example
code:
class Avg(Aggregate):
    is_computed = True
    sql_function = 'AVG'

class Count(Aggregate):
    is_ordinal = True
    sql_function = 'COUNT'
    sql_template = '%(function)s(%(distinct)s%(field)s)'

    def __init__(self, col, distinct=False, **extra):
        super(Count, self).__init__(col, distinct=distinct and 'DISTINCT ' or '', **extra)

class Max(Aggregate):
    sql_function = 'MAX'

class Min(Aggregate):
    sql_function = 'MIN'

class StdDev(Aggregate):
    is_computed = True

    def __init__(self, col, sample=False, **extra):
        super(StdDev, self).__init__(col, **extra)
        self.sql_function = sample and 'STDDEV_SAMP' or 'STDDEV_POP'

class Sum(Aggregate):
    sql_function = 'SUM'

class Variance(Aggregate):
    is_computed = True

    def __init__(self, col, sample=False, **extra):
        super(Variance, self).__init__(col, **extra)
        self.sql_function = sample and 'VAR_SAMP' or 'VAR_POP'
______________________________________________
[1] Django Project, The. https://github.com/django/django/blob/master/django/db/models/sql/aggregates.py .

Tad Naff
Jul 8, 2004

I told you you'd be sorry buying an emoticon, but no, you were hung over. Well look at you now. It's not catching on at all!
:backtowork:
The Something Awful Forums > Discussion > Serious Hardware / Software Crap > The Cavern of COBOL > Coding horrors: post the code that makes you laugh (or cry) > Spergin' on the ORM

OTOH, I horrify myself daily.

code:
    var left,top,findptr,i,j,pos,x,y;
    if(v.hits.records && v.hits.records[0] && v.hits.records[0].pointer){
      findptr=v.hits.records[0].pointer;
      for(i in v.pages){
       	if(v.pages[i].ptr===findptr){
          v.setNewPage(i);
          for(j=0;j<v.hits.records.length;j++){
            if(v.hits.records[j].pointer==findptr){
              pos=v.hits.records[j].positions;
              if(pos){
               	pos=pos[0].position;
               	x=parseInt(pos.x,10)+parseInt(pos.w,10)/2;
               	y=parseInt(pos.y,10)+parseInt(pos.h,10)/2;
               	left=Math.min(0,-x+v.viewport.width/2);
              	top=Math.min(0,-y+v.viewport.height/2);
               	left=Math.max(left,-v.image.width+v.viewport.width);
               	top=Math.max(top,-v.image.height+v.viewport.height);
               	setTimeout(
                  function(){
               	    v.setNewZoom(0,[left,top],true);
               	  },500);
               	break;
              }
            }
          }
       	}
      }
Someday soon I will rewrite this. Just have to figure out what the hell I was saying in the first place.

Brecht
Nov 7, 2009

Markov Chain Chomp posted:

Are you daft, tef? Any decent ORM would turn that given example into exactly the same SQL that a human would write (although not if you tried to write it the way notshadowstar did). And with a good JIT the differences would disappear outright. Maybe you ought to step out of the world of turtles and into the modern era. :)
SQL is already a DSL for (deep breath here) structured queries, if at the end of the day you're operating on structured data you're necessarily writing structured queries and you might as well do it with the tool that's been expressly designed for that purpose. You don't need another layer of abstraction for what is already a purpose-built layer of abstraction, the only possible consequence of that is you take a net loss. Not having to write SELECT FROM WHERE is a false economy when you just hide it behind the context-destroying façade of an ORM.

tl;dr SQL isn't hard, use it

Zombywuf
Mar 29, 2008

Markov Chain Chomp posted:

For reference, here's a code snippet from one ORM[1] which makes quick work out of that example

How would you do this with an ORM?

code:
with exceptions as (
  select
    item_id
  from
    banned
  where
    ban_date > get_date() - 1
) select
  item,
  category,
  sum(rating) over (partition by category order by item)
from
  things t
where
  not exists (select * from exceptions e where e.item_id = t.item_id);

tef
May 30, 2004

-> some l-system crap ->

Markov Chain Chomp posted:

Are you daft, tef?

Yup.

Which is why I asked the first time if it was the case - I wasn't sure if the ORM could map it. I didn't think the original example was the best to show the limitations of orm.

tef fucked around with this message at 13:56 on Dec 11, 2011

Adbot
ADBOT LOVES YOU

tef
May 30, 2004

-> some l-system crap ->

NotShadowStar posted:

What the gently caress is your problem?

To quote someone else ”I’m posting on an internet comedy forum so gently caress haters.”. I just feel that as a resident of sperg kingdom I shouldn't miss the details of making GBS threads all over your posts.

Brecht posted:

You are the coding horror.

Hell is other peoples code. We are all the horror.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply