Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
Sapozhnik
Jan 2, 2005

Nap Ghost
programmer shitposting

Adbot
ADBOT LOVES YOU

jesus WEP
Oct 17, 2004


cinci zoo sniper posted:

c tp mysql s:

SQL code:
SELECT
  foo.a,
  foo.b,
  foo.c,
  GROUP_CONCAT(bar.b) AS X
FROM table foo
  LEFT JOIN (SELECT
               baz.a,
               baz.b,
               baz.c
             FROM table baz) bar ON foo.a = bar.a AND foo.c > bar.c
yields, figuratively speaking X1 X2 X1 X2 X1 X2 X1 X2 X1 X2 as the output of single X. substituting for inner join gives me X1 X1 X1 X1 X1 X2 X2 X2 X2 X2

:eng99:
what do you want the result to be

graph
Nov 22, 2006

aaag peanuts

gonadic io posted:

lol nice. would be better with "terrible programmers:" at the front though

no room

jesus WEP
Oct 17, 2004


you don't truly become a terrible programmer until you know just enough to really gently caress yourself and your colleagues

gonadic io
Feb 16, 2011

>>=

St Evan Echoes posted:

you don't truly become a terrible programmer until you know just enough to really gently caress yourself and your colleagues

this is the thing, newbie programmers aren't terrible programmers. they're bad, but they're bad and ineffectual. it's the subtle bugs that kill you not the glaring syntax errors

cinci zoo sniper
Mar 15, 2013




St Evan Echoes posted:

what do you want the result to be

let me rewrite the query better first
SQL code:
SELECT
  foo.a,
  foo.b,
  foo.c,
  GROUP_CONCAT(bar.a) AS bar_a
FROM table foo
  LEFT JOIN (SELECT
               baz.a,
               baz.b,
               baz.c
             FROM table baz) bar ON foo.b = bar.b AND foo.c > bar.c
WHERE foo.c BETWEEN '2016-01-01' AND '2016-12-13'
the data itself has a couple more years prior to 2016

what i want to do is to retrieve foo.* for every 2016 record, and every historical foo.a (which is incremental) that matches the group id number foo.b, as a string.

the problem im running in at the moment is oddball scenarios where i get loads of duplicates (for example, if i replace WHERE clause with "WHERE foo.a = 21969420" that matches some late 2016).

i can fix this by slamming DISTINCT into GROUP_CONCAT, or by doing some seemingly painful (this query humongous irl) refactoring so that the innermost loop contains all foos, which seems to invoke a lot of problems related to filtering and ranking. im reluctant to do the latter, and i wonder if there is something faster than solution with DISTINCT, which appears to be significantly slowing down the already terrible query (100k target foo rows with joins to a couple dozen tables, lots of GROUP_CONCATS and general querying without existing indices)

also i would like to understand what is going off in the first place, since the DISTINCT option sounds like a bandaid, and it really shouldnt be spitting out 10 copies of bar.a, each with all 5 baz.a in it (for that WHERE substition example). my best guess was that something related to joining a value on an array, but COALESCE(bar.b) in ON clause did not change anything

the understand will help to fix other dozen of sub queries with the exact same problem

cinci zoo sniper fucked around with this message at 17:28 on Nov 15, 2017

gonadic io
Feb 16, 2011

>>=

gonadic io posted:

this is the thing, newbie programmers aren't terrible programmers. they're bad, but they're bad and ineffectual. it's the subtle bugs that kill you not the glaring syntax errors

It's like president trump vs president pence

cinci zoo sniper
Mar 15, 2013




gonadic io posted:

It's like president trump vs president pence

there's nothing subtle about shocking the gay away

Symbolic Butt
Mar 22, 2009

(_!_)
Buglord
javascript is losing some steam in the language popularity contest, it may become the new ruby I think

jesus WEP
Oct 17, 2004


cinci zoo sniper posted:

let me rewrite the query better first
SQL code:
SELECT
  foo.a,
  foo.b,
  foo.c,
  GROUP_CONCAT(bar.a) AS bar_a
FROM table foo
  LEFT JOIN (SELECT
               baz.a,
               baz.b,
               baz.c
             FROM table baz) bar ON foo.b = bar.b AND foo.c > bar.c
WHERE foo.c BETWEEN '2016-01-01' AND '2016-12-13'
the data itself has a couple more years prior to 2016

what i want to do is to retrieve foo.* for every 2016 record, and every historical foo.a (which is incremental) that matches the group id number foo.b, as a string.

the problem im running in at the moment is oddball scenarios where i get loads of duplicates (for example, if i replace WHERE clause with "WHERE foo.a = 21969420" that matches some late 2016).

i can fix this by slamming DISTINCT into GROUP_CONCAT, or by doing some seemingly painful (this query humongous irl) refactoring so that the innermost loop contains all foos, which seems to invoke a lot of problems related to filtering and ranking. im reluctant to do the latter, and i wonder if there is something faster than solution with DISTINCT, which appears to be significantly slowing down the already terrible query (100k target foo rows with joins to a couple dozen tables, lots of GROUP_CONCATS and general querying without existing indices)

also i would like to understand what is going off in the first place, since the DISTINCT option sounds like a bandaid, and it really shouldnt be spitting out 10 copies of bar.a, each with all 5 baz.a in it (for that WHERE substition example). my best guess was that something related to joining a value on an array, but COALESCE(bar.b) in ON clause did not change anything

the understand will help to fix other dozen of sub queries with the exact same problem

hmm i got lost like a sentence and half into your post, just throw away the whole query and start again imo

akadajet
Sep 14, 2003

Symbolic Butt posted:

javascript is losing some steam in the language popularity contest, it may become the new ruby I think

not as long as it's the default language for web browsers.

akadajet
Sep 14, 2003

this thread title sucks, I thought maybe the terrible programmers thread got deleted or something.

cinci zoo sniper
Mar 15, 2013




St Evan Echoes posted:

hmm i got lost like a sentence and half into your post, just throw away the whole query and start again imo

:rip: will see what i can do i guess. tl;dr on my post was that i want to look up same table entries preceding the scope i'm querying, and i keep getting duplicates without hacky solutions

hobbesmaster
Jan 28, 2008

developer holding a screw driver and jtag cable: “this is a lot easier than fixing the JavaScript”

jesus WEP
Oct 17, 2004


cinci zoo sniper posted:

:rip: will see what i can do i guess. tl;dr on my post was that i want to look up same table entries preceding the scope i'm querying, and i keep getting duplicates without hacky solutions

what happens if you put
code:
GROUP BY foo.a, foo.b, foo.c
at the end of that query

Sapozhnik
Jan 2, 2005

Nap Ghost

hobbesmaster posted:

developer holding a screw driver and jtag cable: “this is a lot easier than fixing the JavaScript”

this but unironically

cinci zoo sniper
Mar 15, 2013




St Evan Echoes posted:

what happens if you put
code:
GROUP BY foo.a, foo.b, foo.c
at the end of that query
should it do anything if i get one foo.a per line. the duplicates are from the subquery that goes into concat

jesus WEP
Oct 17, 2004


idk, but then idk why you would be able to do a group function with no group by in the first place, loving mysql

JawnV6
Jul 4, 2004

So hot ...

hobbesmaster posted:

developer holding a screw driver and jtag cable: “this is a lot easier than fixing the JavaScript”
yah p. much

Arcsech
Aug 5, 2008

akadajet posted:

not as long as it's the default language for web browsers.

getting there: https://caniuse.com/#feat=wasm

Powerful Two-Hander
Mar 10, 2004

Mods please change my name to "Tooter Skeleton" TIA.


cinci zoo sniper posted:

:rip: will see what i can do i guess. tl;dr on my post was that i want to look up same table entries preceding the scope i'm querying, and i keep getting duplicates without hacky solutions

why not split it into two sub queries? like


code:

Select foo. *
From
Foo 
Where foo.date Between [whatever it was] 

Union

Select
*
From 
Baz
Where baz.id in (select id from foo) 

and then concat that?

Powerful Two-Hander
Mar 10, 2004

Mods please change my name to "Tooter Skeleton" TIA.


no wait can't you do this instead?


code:

Select
Foo. *, 
(
Select
Baz. A
From baz
Where
Baz.b =foo. B
And Baz.c <foo. C
) as bar
From 
Foo 
Where
Foo.c between (...) 

i guess you can concat bar? If this was mssql I'd stick a for xml path('') on there to get the concatenation because there's no built in function for it yet but I assume group_concat will do that for you...

cinci zoo sniper
Mar 15, 2013




Powerful Two-Hander posted:

no wait can't you do this instead?


code:

Select
Foo. *, 
(
Select
Baz. A
From baz
Where
Baz.b =foo. B
And Baz.c <foo. C
) as bar
From 
Foo 
Where
Foo.c between (...) 

i guess you can concat bar? If this was mssql I'd stick a for xml path('') on there to get the concatenation because there's no built in function for it yet but I assume group_concat will do that for you...

i can't use external variables inside subquery for mysql reasons. at least in join part i can't - i think i did successfully do this in the select part, but with the number of variables im getting through this query it might cause repeated full scans (not sure how this things work out under the hood if i would pull 10 variables from the same table that way)

what do you mean by concating bar itself, you mean inside the subquery? ill try that at work tomorrow

cinci zoo sniper fucked around with this message at 19:26 on Nov 15, 2017

redleader
Aug 18, 2005

Engage according to operational parameters

gonadic io posted:

this is the thing, newbie programmers aren't terrible programmers. they're bad, but they're bad and ineffectual. it's the subtle bugs that kill you not the glaring syntax errors

worse: broken, unfixable architecture

a witch
Jan 12, 2017

has anyone here used D? do you have opinions on D?

i wanna write a network daemon thingy and I’m looking at D idk

jesus WEP
Oct 17, 2004


a witch posted:

has anyone here used D? do you have opinions on D?

i wanna write a network daemon thingy and I’m looking at D idk
giggled like a child at this entire post

cinci zoo sniper
Mar 15, 2013




St Evan Echoes posted:

giggled like a child at this entire post

this but unsuccesfully tried to make an adequate joke about it

Captain Foo
May 11, 2004

we vibin'
we slidin'
we breathin'
we dyin'

cinci zoo sniper posted:

this but unsuccesfully tried to make an adequate joke about it

we know your D is an inadequate joke

cinci zoo sniper
Mar 15, 2013




Captain Foo posted:

we know your D is an inadequate joke

:argh: gdi, i walked right into this, didnt i

Zemyla
Aug 6, 2008

I'll take her off your hands. Pleasure doing business with you!
From what I heard, Rust sucked away pretty much all consumers of D.

Ellie Crabcakes
Feb 1, 2008

Stop emailing my boyfriend Gay Crungus

cinci zoo sniper posted:

:argh: gdi, i walked right into this, didnt i
Short answer: yes.

a witch
Jan 12, 2017

Zemyla posted:

From what I heard, Rust sucked away pretty much all consumers of D.

what if I love rust but hate tokio?

LordSaturn
Aug 12, 2007

sadly unfunny


the prequel to b-trees, a-hives

HoboMan
Nov 4, 2010


is there a reason you are using a subquery instead of a normal join?

also you can just use CONCAT() i think
e: like
SQL code:
SELECT
  foo.a,
  foo.b,
  foo.c,
  CONCAT(baz.a) AS bar_a
FROM foo
  LEFT JOIN baz ON foo.b = baz.b AND foo.c > baz.c
WHERE foo.c BETWEEN '2016-01-01' AND '2016-12-31'
not actually helping you on your question because i still don't understand it, but simplifying syntax can be a good first step to wrapping your head around it

HoboMan fucked around with this message at 20:40 on Nov 15, 2017

Workaday Wizard
Oct 23, 2009

by Pragmatica

a witch posted:

what if I love rust but hate tokio?

if i understand correctly trait objects impl Trait is supposed to simplify the return types a whole drat lot (e.g. instead of having AndThen<Join<Flatten<WhateverElseImplementsFuture<.... you will just have impl Future)

cinci zoo sniper
Mar 15, 2013




HoboMan posted:

is there a reason you are using a subquery instead of a normal join?

also you can just use CONCAT() i think
e: like
SQL code:
SELECT
  foo.a,
  foo.b,
  foo.c,
  CONCAT(baz.a) AS bar_a
FROM foo
  LEFT JOIN baz ON foo.b = baz.b AND foo.c > baz.c
WHERE foo.c BETWEEN '2016-01-01' AND '2016-12-31'
not actually helping you on your question because i still don't understand it, but simplifying syntax can be a good first step to wrapping your head around it

i'm joining the table onto itself, but i havent tried doing it without a subquery. im not sure i can do that, but i can try it in the morning

to fram the question the other way:


i can "solve" this via
code:
GROUP_CONCAT(DISTINCT
but that feels like a hack to me, i have a naive assumption that a competently written query would just return what i want once. testin one specific request/category pair, that hack also bumps execution time by nearly a half, so im worried what are the implication for a full-sized query

cinci zoo sniper fucked around with this message at 20:57 on Nov 15, 2017

HoboMan
Nov 4, 2010

cinci zoo sniper posted:

what i want to do is to retrieve foo.* for every 2016 record, and every historical foo.a (which is incremental) that matches the group id number foo.b, as a string.

the problem im running in at the moment is oddball scenarios where i get loads of duplicates (for example, if i replace WHERE clause with "WHERE foo.a = 21969420" that matches some late 2016).

i can fix this by slamming DISTINCT into GROUP_CONCAT, or by doing some seemingly painful (this query humongous irl) refactoring so that the innermost loop contains all foos, which seems to invoke a lot of problems related to filtering and ranking. im reluctant to do the latter, and i wonder if there is something faster than solution with DISTINCT, which appears to be significantly slowing down the already terrible query (100k target foo rows with joins to a couple dozen tables, lots of GROUP_CONCATS and general querying without existing indices)

also i would like to understand what is going off in the first place, since the DISTINCT option sounds like a bandaid, and it really shouldnt be spitting out 10 copies of bar.a, each with all 5 baz.a in it (for that WHERE substition example). my best guess was that something related to joining a value on an array, but COALESCE(bar.b) in ON clause did not change anything

the understand will help to fix other dozen of sub queries with the exact same problem

if more than one foo can have the same group id then you are going to have duplicates. (period)

the solution is to use GROUP_CONCAT(DISTINCT foo.whatever), it's not hacky

the reason COALESCE does nothing is because comparing null to anything (including null) is always false


it is slow because you are concatenating 100k rows into a goddamn string wtf are you doing!?!?!

Maluco Marinero
Jan 18, 2001

Damn that's a
fine elephant.

redleader posted:

yeah, you're right. i misspoke; i said 'overengineered' when i actually meant 'verbose and boilerplatey'

I get this argument, however you can always build less components if you don’t need as many for ten plating. either way, it’s a super strong way of assembling components into a functional UI that doesn’t tend to bite you in the rear end, so much so that we use React for pretty much every client job no matter what. server side rendering, client side rendering, doesn’t matter, use React all the same and that way we can change seamlessly, because in the end of the day it’s just JavaScript objects in, DOM/HTML out, and React is a solid way to express that.

definitely feel the verbose and boilerplatey though, so I made a project specific snippet/templating tool that asks a few questions about each component and where it needs to go, so you don’t have to worry about the boilerplate so much and don’t feel the friction of not wanting to right new components to avoid the boilerplate. works for me anyway.

bob dobbs is dead
Oct 8, 2017

I love peeps
Nap Ghost

gonadic io posted:

YOSPOS > i actually have reasons to post here unlike half of the people with impostor syndrome

lol nice. would be better with "terrible programmers:" at the front though

e:


now i need to work out how to draw a border. i want the line to be inside the hex, so it can't be specified using the same coords as the hex verts i think

e2: also i'm pretty sure this framework doesn't support selection mode so i have to work out which hex gets clicked on directly from the mouse pos hurray. luckily the hex resource i posted about earlier tells you the math to do this

is the lowercase a's a nethack reference

Adbot
ADBOT LOVES YOU

cinci zoo sniper
Mar 15, 2013




HoboMan posted:

if more than one foo can have the same group id then you are going to have duplicates. (period)

the solution is to use GROUP_CONCAT(DISTINCT foo.whatever), it's not hacky

the reason COALESCE does nothing is because comparing null to anything (including null) is always false


it is slow because you are concatenating 100k rows into a goddamn string wtf are you doing!?!?!

check the pic above. category can have a few dozen requests, and i have around 100k categories, where i "forward concatenate" every request into the lastest one. no idea if that makes it any faster or slower than concatenating 100k rows into a string.

a bit confused about coalesce, since docs say it just selects the first non-null value.

quote:

if more than one foo can have the same group id then you are going to have duplicates. (period)
so it is related to the fact that each record has the same identifier it is joined on? it just feels weird that join of N rows results in N*2 copies of all N rows

going to face the inevitable bloodshed and roll with GROUP_CONCAT(DISTINCT i guess, and cry when i need to refactor it because it runs for eternity or something. and our new etl dev is not here for another 3.5 weeks :qq:

cinci zoo sniper fucked around with this message at 21:35 on Nov 15, 2017

  • Locked thread