Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
simble
May 11, 2004

ratbert90 posted:

H3Y GuY5; Ju57 P0571NG 0N 7h3 B357 P4G3.

1f j00 k4N r34d 7h12 J00 h4V3 4 sp3c14L K1nD 0F M1nD

Adbot
ADBOT LOVES YOU

akadajet
Sep 14, 2003

meatpotato posted:

hi, am I the terrible programmer? It's convention where I work to prefix class data members with 'm_' and I didn't think it's that weird. Similarly, static variables typically get 's_' prefixes. This is all C and C++ stuff.

those prefixes are garbage, but mostly harmless garbage.

simble
May 11, 2004

meatpotato posted:

hi, am I the terrible programmer? It's convention where I work to prefix class data members with 'm_' and I didn't think it's that weird. Similarly, static variables typically get 's_' prefixes. This is all C and C++ stuff.

on the plus side, now you can use reflection (if your runtime supports it) and filter your fields by name alone!

edit: c/c++ well in that case you can write a parser to do it

NihilCredo
Jun 6, 2011

iram omni possibili modo preme:
plus una illa te diffamabit, quam multæ virtutes commendabunt

simble posted:

on the plus side, now you can use reflection (if your runtime supports it) and filter your fields by name alone!

edit: c/c++ well in that case you can write a parser to do it

look at this scrub who doesn't even use uncontacted amazonian tribe tree bark scratches glyphs to implement ad-hoc polymorphism

FlapYoJacks
Feb 12, 2009

NihilCredo posted:

look at this scrub who doesn't even use uncontacted amazonian tribe tree bark scratches glyphs to implement ad-hoc polymorphism

I once had polymorphism go 3 levels deep and I felt REALLY bad.

Condiv
May 7, 2008

Sorry to undo the effort of paying a domestic abuser $10 to own this poster, but I am going to lose my dang mind if I keep seeing multiple posters who appear to be Baloogan.

With love,
a mod


hey terrible programmers. can you guys make a case for why I should use push notifications with rabbitmq or kafka instead of polling a Postgres db. I know that polling sucks, but I have a coworker who's very against changing anything ever, including polling a remote db when we don't need to (he's even against me putting a rest API in front of the DB so that the website is the only one directly accessing the db cause he's worried about latency)

The MUMPSorceress
Jan 6, 2012


^SHTPSTS

Gary’s Answer

Condiv posted:

hey terrible programmers. can you guys make a case for why I should use push notifications with rabbitmq or kafka instead of polling a Postgres db. I know that polling sucks, but I have a coworker who's very against changing anything ever, including polling a remote db when we don't need to (he's even against me putting a rest API in front of the DB so that the website is the only one directly accessing the db cause he's worried about latency)

what is your use case. polling is a really lovely thing to do if your software is running on someone else's device and will eat their bandwidth to do it, especially mobile. if its just a regular internet webzone that you dont expect people to idle on for a long time then do whatever.

Condiv
May 7, 2008

Sorry to undo the effort of paying a domestic abuser $10 to own this poster, but I am going to lose my dang mind if I keep seeing multiple posters who appear to be Baloogan.

With love,
a mod


cis autodrag posted:

what is your use case. polling is a really lovely thing to do if your software is running on someone else's device and will eat their bandwidth to do it, especially mobile. if its just a regular internet webzone that you dont expect people to idle on for a long time then do whatever.

use case is a hpc system needs to receive job requests and the current method of doing that is polling a job table in a DB

my homie dhall
Dec 9, 2010

honey, oh please, it's just a machine

Condiv posted:

hey terrible programmers. can you guys make a case for why I should use push notifications with rabbitmq or kafka instead of polling a Postgres db. I know that polling sucks, but I have a coworker who's very against changing anything ever, including polling a remote db when we don't need to (he's even against me putting a rest API in front of the DB so that the website is the only one directly accessing the db cause he's worried about latency)
you can use a database table as a queue

you can also build an entire house by hammering the nails with rocks

The MUMPSorceress
Jan 6, 2012


^SHTPSTS

Gary’s Answer

Condiv posted:

use case is a hpc system needs to receive job requests and the current method of doing that is polling a job table in a DB

well, i guess my question is "what prompted you to want to change it?" because if the polling method is working fine and it's not manifesting any serious drawbacks for you or causing you to do heinous things elsewhere in the codebase, "leave well enough alone" is generally a good policy.

LordSaturn
Aug 12, 2007

sadly unfunny

DuckConference posted:

Hungarian notation was in the coding standard here when I started and I never really questioned it because it seemed fine, but now I'm kinda questioning if it's worth it. I mean I guess it's a bit redundant since the compiler is checking your types

the explanation I read is that Hungarian Notation was explicitly meant for same-typed data with different functional meaning, like integer position vs. integer size. obviously that's not what it's come to be

JewKiller 3000
Nov 28, 2006

by Lowtax

Condiv posted:

hey terrible programmers. can you guys make a case for why I should use push notifications with rabbitmq or kafka instead of polling a Postgres db. I know that polling sucks, but I have a coworker who's very against changing anything ever, including polling a remote db when we don't need to (he's even against me putting a rest API in front of the DB so that the website is the only one directly accessing the db cause he's worried about latency)

if he really loves postgres and doesn't want to use a separate queue, did you know that postgres has built in listen/notify? https://www.postgresql.org/docs/current/static/sql-notify.html :getin:

Chalks
Sep 30, 2009

DuckConference posted:

Hungarian notation was in the coding standard here when I started and I never really questioned it because it seemed fine, but now I'm kinda questioning if it's worth it. I mean I guess it's a bit redundant since the compiler is checking your types

The only time Hungarian notation has had any affect on any codebase I've seen is when it's causing developers to chase their tails by obscuring simple bugs due to variables claiming to be something they're not.

The MUMPSorceress
Jan 6, 2012


^SHTPSTS

Gary’s Answer

LordSaturn posted:

the explanation I read is that Hungarian Notation was explicitly meant for same-typed data with different functional meaning, like integer position vs. integer size. obviously that's not what it's come to be

hungarian notation is a relic from a time when we had code editors with no intellisense and no easy way to alias types to differentiate usage. there's like 0 reason in a modern language not to just alias, say, long to two different other identifiers if you have two specific types of long that should only be used in certain ways. or even better wrap them in a class that enforces the semantics of how you want them to be used.

and besides, anything that depends on other developers to name variables appropriately is a suicide mission from day one.

necrotic
Aug 2, 2005
I owe my brother big time for this!
fuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuck i have to setup a mail server. an external client is having issues sending email to an address and they have been terrible at providing any useful information (like how they are even sending the email). google apps email logging is not helping, which could also mean its not even getting as far as the mail server on our end.

in any case, ive been tasked with attempting to spin up a mail server so we have more debugging tools on oru end. are there any good docker images or "hold my hand" setups? we would like to deploy it to our kubernetes cluster, too.

just the backend parts, no dumb web ui.

Sapozhnik
Jan 2, 2005

Nap Ghost

ThePeavstenator posted:

it's me, the guy who has his IDE automatically generate getters and setters for every member

well, use AutoValue then and make your poo poo immutable where possible, sheesh

ComradeCosmobot
Dec 4, 2004

USPOL July

Sapozhnik posted:

if you add extension methods to string then i hope you loving die.

yes i too like having to keep utility classes around because the java string api only has case-sensitive endsWith and contains methods

i mean it's not like a modern ide can keep track of where extension methods are defined, no sir!

Condiv
May 7, 2008

Sorry to undo the effort of paying a domestic abuser $10 to own this poster, but I am going to lose my dang mind if I keep seeing multiple posters who appear to be Baloogan.

With love,
a mod


cis autodrag posted:

well, i guess my question is "what prompted you to want to change it?" because if the polling method is working fine and it's not manifesting any serious drawbacks for you or causing you to do heinous things elsewhere in the codebase, "leave well enough alone" is generally a good policy.

it's not really working fine, and we have multiple services polling every minute. plus, the db only has one username and password and I'm trying to abstract it away so I can clean it up

since I'm rewriting the web service to get rid of the unholy hacked together php abomination, I have an opportunity to have the web service deliver task requests to a message queue if it would be a good idea. I'm almost certainly gonna hide the db behind a rest API and block outside access to it asap if I can though

Condiv fucked around with this message at 21:31 on May 31, 2017

ulmont
Sep 15, 2010

IF I EVER MISS VOTING IN AN ELECTION (EVEN AMERICAN IDOL) ,OR HAVE UNPAID PARKING TICKETS, PLEASE TAKE AWAY MY FRANCHISE

JewKiller 3000 posted:

if he really loves postgres and doesn't want to use a separate queue, did you know that postgres has built in listen/notify? https://www.postgresql.org/docs/current/static/sql-notify.html :getin:

Warning: to actually use postgres' listen/notify without polling in Java you'll need to use a different set of JDBC drivers, as the default ones don't support it.
http://impossibl.github.io/pgjdbc-ng/

darthbob88
Oct 13, 2011

YOSPOS

LordSaturn posted:

the explanation I read is that Hungarian Notation was explicitly meant for same-typed data with different functional meaning, like integer position vs. integer size. obviously that's not what it's come to be
Correct. Apps Hungarian is a pretty good idea, even if it's obsolete like cis autodrag says, Systems Hungarian is a foolish notion that poisoned the term.

jre
Sep 2, 2011

To the cloud ?



Condiv posted:

it's not really working fine, and we have multiple services polling every minute. plus, the db only has one username and password and I'm trying to abstract it away so I can clean it up

since I'm rewriting the web service to get rid of the unholy hacked together php abomination, I have an opportunity to have the web service deliver task requests to a message queue if it would be a good idea. I'm almost certainly gonna hide the db behind a rest API and block outside access to it asap if I can though

Things to note about kafka:

It's sure as gently caress not lightweight, hope you enjoy managing a shitload of jvm things (e.g. at least 3 zookeeper nodes as well as the brokers )
It guarantees at least once delivery for things in a consumer group
It only guarantees ordering if your topic has 1 partition.
It's really really really good at pushing enormous message volumes on meh hardware, and it does scale horizontally if you can shard the topic

Re latency, How many calls / s are made to the db ?

jre fucked around with this message at 23:13 on May 31, 2017

fritz
Jul 26, 2003

Share Bear posted:



does that help? I'm trying to figure out what the curly brackets mean or imply

i think symbolic butt is correct, you've got observed input data x1, x2, x3, ...., xn and observed output data y1, y2, y3, ..., yn

the xs may be vector-valued, and that's where the p comes from

so if you had n=4 p=3, you might have data like

x1 = 0,0,0 y1 = 1
x2 = 1,0,1 y2 = 0
x3= -1,1,0 y3 = 1
x4 = 1,0,2 y4 = -1

eschaton
Mar 7, 2007

Don't you just hate when you wind up in a store with people who are in a socioeconomic class that is pretty obviously about two levels lower than your own?

Chalks posted:

Yeah, extension methods are awesome, finding out what god drat using statement you need to add in order for them to work is not.

extension methods are better in Objective-C, they're called categories there and there's no difference in syntax between a method in a category and a method on the class itself

there's especially no bullshit about passing self to the method or anything like that

and of course Swift got categories from Objective-C so they don't suck there either

cool av
Mar 2, 2013

cis autodrag posted:

ya, in vb6 there's this horror show where the first 200 lines of your class end up being all these private global variables that control way too much of your class's behavior. you can of course do that in any language, but vb6 practically requires it to get any kind of complex behavior out of the thing.

are member variables in vb6 really called "global" cause that sounds crazy (just crazy enough to be vb6?)

eschaton
Mar 7, 2007

Don't you just hate when you wind up in a store with people who are in a socioeconomic class that is pretty obviously about two levels lower than your own?

cool av posted:

are member variables in vb6 really called "global" cause that sounds crazy (just crazy enough to be vb6?)

vb6 doesn't have inheritance

does it even have a class/instance distinction, or does it have developers define "objects"

the latter would make "m_ indicates a global" sensible

Condiv
May 7, 2008

Sorry to undo the effort of paying a domestic abuser $10 to own this poster, but I am going to lose my dang mind if I keep seeing multiple posters who appear to be Baloogan.

With love,
a mod


jre posted:

Things to note about kafka:

It's sure as gently caress not lightweight, hope you enjoy managing a shitload of jvm things (e.g. at least 3 zookeeper nodes as well as the brokers )
It guarantees at least once delivery for things in a consumer group
It only guarantees ordering if your topic has 1 partition.
It's really really really good at pushing enormous message volumes on meh hardware, and it does scale horizontally if you can shard the topic

Re latency, How many calls / s are made to the db ?

4-8 with things being quiet

The MUMPSorceress
Jan 6, 2012


^SHTPSTS

Gary’s Answer

eschaton posted:

vb6 doesn't have inheritance

does it even have a class/instance distinction, or does it have developers define "objects"

the latter would make "m_ indicates a global" sensible

there's not even a concept of static methods. you can only access a class's stuff through an instance of it, which results in a whole gently caress ton of passing around framework objects in nutso daisy chains of code.

The MUMPSorceress
Jan 6, 2012


^SHTPSTS

Gary’s Answer
also eschaton just once id like you to know less about the things i do every day than i do :p

tef
May 30, 2004

-> some l-system crap ->

Condiv posted:

use case is a hpc system needs to receive job requests and the current method of doing that is polling a job table in a DB

i am literally writing something about this question but my advice would be don't use a broker, the end

tef
May 30, 2004

-> some l-system crap ->

Ploft-shell crab posted:

you can use a database table as a queue

you can also build an entire house by hammering the nails with rocks

at the point where you are assigning work to a queue and not directly to a machine, you're not really worried about latency

i mean, most of what you used a queue for was avoiding service discovery/configuration

at the point where your queue outlives the clients requesting work, your queue needs to handle recovery too

which is y'know, restarting things, retrying things

and queues that handle retying of jobs, or jobs with a more complex lifecycle than 'enqueued -> delivered' for messages do start to look more like a database than a buffer.

(this is one of those things where someone starts with a started_at of NULL, and a ended_at of NULL and then a cacophony of booleans to indicate each new state that doesn't fit in exactly, just use a text column called `state`)

but in the end you end up building a lot of a database atop, although you're in for a world of pain if you try to wrap every job inside a transaction

this all makes sense for long lived jobs, high thoughput but not low latency or short stuff

on the other hand, i guess if a database is overkill for short lived jobs, so is a queue

at this point you'd be doing recovery in the client, so you don't need to do any real replication in the queue, or ha

youll end up with some system for handing off work to active workers, maybe workers polling the queue

or, forward any requests that come in to the most recent idle worker (hello join-idle-queue)/random

so, once you've done that well you can get rid of the queue because you don't need it


so i guess what i am saying is that queues are pretty much the worst way to distribute work

tef
May 30, 2004

-> some l-system crap ->
i am sure someone will tell you to use a broker but that is advice from people who don't support them in production

tef
May 30, 2004

-> some l-system crap ->

jre posted:

Things to note about kafka:

It's sure as gently caress not lightweight, hope you enjoy managing a shitload of jvm things (e.g. at least 3 zookeeper nodes as well as the brokers )
It guarantees at least once delivery for things in a consumer group
It only guarantees ordering if your topic has 1 partition.
It's really really really good at pushing enormous message volumes on meh hardware, and it does scale horizontally if you can shard the topic

Re latency, How many calls / s are made to the db ?

hi i saw a trash meme




and well i do not want to make more effort than this to talk about kafka

tef
May 30, 2004

-> some l-system crap ->
i loving hate queues

JewKiller 3000
Nov 28, 2006

by Lowtax
imo if you just need a basic work queue, and if you can get away with using a postgres 9.5+ table and SELECT FOR UPDATE SKIP LOCKED, you should probably just do that, because it's hella ez

raminasi
Jan 25, 2005

a last drink with no ice
what is a good way to distribute work

Shaggar
Apr 26, 2006
if your messages are important some part of them is gonna be living in your database so just do it all there.

Shaggar
Apr 26, 2006

raminasi posted:

what is a good way to distribute work

a bunch of thread pools gang banging the database

tef
May 30, 2004

-> some l-system crap ->

Shaggar posted:

if your messages are important some part of them is gonna be living in your database so just do it all there.

shaggar was right, again

tef
May 30, 2004

-> some l-system crap ->

JewKiller 3000 posted:

imo if you just need a basic work queue, and if you can get away with using a postgres 9.5+ table and SELECT FOR UPDATE SKIP LOCKED, you should probably just do that, because it's hella ez

also if your next job query gets expensive just hack up a temporary table and amortize filling it up from diff workers / stick it in a buffer somewhere


fwiw: this is one of the workloads where mysql can be a better fit than postgres specifically because of handling in-place updates, on high volume write/update paths

Adbot
ADBOT LOVES YOU

Shaggar
Apr 26, 2006
I treat my messaging as idempotent cause I don't care if the client whines that they got duplicate faxes.

  • Locked thread