Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Cybernetic Vermin
Apr 18, 2005

Shaggar posted:

are you still using manual jar management and builds ittyoogl 2013 instead of maven, the best build tool in the universe?

eh, i am fine with solving that part of the problem by specifying precise jar versions in maven xml in the repository, as long as all information to automatically recreate a version of the project precisely exists. what is not ok is having source under version control and then finding that it depends on a jar that you cannot find a bit-by-bit perfect copy of, risking that some random bug has been introduced/fixed/changed from the time when the commit happened. even leaving jars aside you really should be able to put everything that you have any reason to want to track in, whether a 200 megabyte stress-testing database dump, a bunch of training material, a special version of a binary tool that a vendor shipped to you to fix a bug exhibited in revision 21238

super-clean repositories is a bad sperg tendency, the tighter you can tie the whole environment to a version number the better, just ignore everything but /src until you need it

Adbot
ADBOT LOVES YOU

double sulk
Jul 2, 2010

Jerry SanDisky posted:

jsut use rake

Notorious b.s.d.
Jan 25, 2003

by Reene

Tiny Bug Child posted:

i would like a sql that you could just bolt more servers onto to make it faster

oracle does this, they call it RAC for "real application clustering"

it is not log-shipping or an in-memory database hack

the way it works inside is a distributed block cache in RAM, shared across all servers. so no matter where you receive a request, the writes will be issued to the node that has those blocks in RAM

i have never built one bigger than 2 nodes, but i have heard of people having dozens of hosts in a cluster

Notorious b.s.d.
Jan 25, 2003

by Reene

Cybernetic Vermin posted:

eh, i am fine with solving that part of the problem by specifying precise jar versions in maven xml in the repository, as long as all information to automatically recreate a version of the project precisely exists. what is not ok is having source under version control and then finding that it depends on a jar that you cannot find a bit-by-bit perfect copy of, risking that some random bug has been introduced/fixed/changed from the time when the commit happened. even leaving jars aside you really should be able to put everything that you have any reason to want to track in, whether a 200 megabyte stress-testing database dump, a bunch of training material, a special version of a binary tool that a vendor shipped to you to fix a bug exhibited in revision 21238

this stuff doesn't go in version control, it goes in your artifactory / artifact store. with maven, the no-brainer artifact store is nexus.


you tag your release in source control, build it, and automatically upload the jar to nexus w/ the tag attached

nexus will never ever ever allow that jar to be overwritten. it is write-once, read-many. anyone who requests that version of the jar will get that exact jar.

Sapozhnik
Jan 2, 2005

Nap Ghost
yeah if it's an open source thing you can use the public maven.org repository but you can also run your own corporate repository behind your firewall if you need a server under your control to host those JARs.

Shaggar
Apr 26, 2006
one of the goals of maven is being able to have it recreate a project from the project descriptor without anything other than maven. this way you don't have to worry that some idiot built all his stuff against a specific version of tomcat that he had installed locally with a pile of jars he always assumed would be installed along side tomcat.

For the major 3rd party libs they're all in there and its gonna be fine for you to just use central or a local proxy of central to get them. If you have highly specific 3rd party libs that aren't in central or if you have your own libs/snapshot libs you can install them in your own local repo server (running sonatype nexus).

The way I have it setup is I have nexus running on the same server as subversion with a proxy of central and then internal snapshot(working buildings) and release(final builds) repos for internal code. and then my own 3rd party repo for hosting stuff not in central or stuff that is highly specific. it works real well.

Cybernetic Vermin
Apr 18, 2005

Notorious b.s.d. posted:

this stuff doesn't go in version control, it goes in your artifactory / artifact store. with maven, the no-brainer artifact store is nexus.


you tag your release in source control, build it, and automatically upload the jar to nexus w/ the tag attached

nexus will never ever ever allow that jar to be overwritten. it is write-once, read-many. anyone who requests that version of the jar will get that exact jar.

fine, so now we are up to three different tools to do what svn already did, but why are we using three tools?

Notorious b.s.d.
Jan 25, 2003

by Reene
the reason that is important to have the jars in a separate artifact store is that I have no way to tie a jar in vcs back to the revision that actually created the jar

like i have rev N checked out, and in my checked out copy of N, there is a jar with version (N - X), where X is unknown and unknowable. this gets worse for third party jars that got downloaded by some moron

with nexus or similar, you can explicitly state the dependencies of an application in your maven/ivy manifest and nothing is implicit or unknown

prefect
Sep 11, 2001

No one, Woodhouse.
No one.




Dead Man’s Band

Cybernetic Vermin posted:

fine, so now we are up to three different tools to do what svn already did, but why are we using three tools?

you can write web applications in assembly language, but it's generally a better idea to use a tool that's designed for that task :shrug:

Shaggar
Apr 26, 2006
I was a maven skeptic forever and now I couldn't live w/out it. my biggest problem was the group that wanted me to use it was using it way wrong and now that I understand it better its the best thing.

Cybernetic Vermin
Apr 18, 2005

ah, well, that is a good point, never actually worked in a situation where we had jars arriving from other projects in an integrated enough way that we could tie it back, and it would no doubt have been an excellent idea. i indeed learned something from asking this question

still, i am very comfortable maintaining that the loss of scalability in git is a huge pain, there are tons of projects that have substantial non-source non-jar assets that should be versioned, and there is little need for anything more complex than having the version control not be a retard about a couple of hundred megs of related assets.

if nothing else training the documentation and testing teams to use tortoisesvn was relatively painless, so alternatives need to be reasonably simple.

Cybernetic Vermin
Apr 18, 2005

prefect posted:

you can write web applications in assembly language, but it's generally a better idea to use a tool that's designed for that task :shrug:

cvs/svn are fine with large binaries, i want to version large binaries in tandem with source, there is no real design problem there. jars was a bad example, i clearly don't have the right experience there, i'm fine with secondary version-linked repositories if they have some pay-off in themselves

Shaggar
Apr 26, 2006
maven does more than just dependency management, tho. its a complete build tool and will handle common tasks like resource filtering and test running during builds. this way you can have a build server constantly checkout build and test ur code. since all the dependency management info is in the pom.xml the build server doesn't need anything other than maven to figure out the build. its kewl.

Notorious b.s.d.
Jan 25, 2003

by Reene

Cybernetic Vermin posted:

still, i am very comfortable maintaining that the loss of scalability in git is a huge pain, there are tons of projects that have substantial non-source non-jar assets that should be versioned, and there is little need for anything more complex than having the version control not be a retard about a couple of hundred megs of related assets.

if it isn't text, it does not belong in source control. source control is for source.

yes, art assets and binary documents and sw artifacts need to be version controlled, but you should be using something suited to the purpose. nexus is very well suited to most binaries, because it exists only to do that. it doesn't try to diff or delta them because they are not loving text

there is also poo poo like alfresco and documentum for word/pdf docs but corporate library poo poo aint my ballgame so i can't comment on that

Notorious b.s.d.
Jan 25, 2003

by Reene

Cybernetic Vermin posted:

cvs/svn are fine with large binaries, i want to version large binaries in tandem with source, there is no real design problem there. jars was a bad example, i clearly don't have the right experience there, i'm fine with secondary version-linked repositories if they have some pay-off in themselves

the problem exists in every language platform. managing artifacts is hard to do right. some platforms do better than others. jars were a great example because the java ecosystem loving owns bones

doing the exact same stuff for e.g. a C application would make me cry blood. autotools, makefiles, ftp dumps, RPM/pkgadd for artifacts, oh god

and the less we talk about ruby the better. rake is an abomination, bundler a necessary evil

Zombywuf
Mar 29, 2008

Cybernetic Vermin posted:

ah, well, that is a good point, never actually worked in a situation where we had jars arriving from other projects in an integrated enough way that we could tie it back, and it would no doubt have been an excellent idea. i indeed learned something from asking this question

still, i am very comfortable maintaining that the loss of scalability in git is a huge pain, there are tons of projects that have substantial non-source non-jar assets that should be versioned, and there is little need for anything more complex than having the version control not be a retard about a couple of hundred megs of related assets.

if nothing else training the documentation and testing teams to use tortoisesvn was relatively painless, so alternatives need to be reasonably simple.

This really sounds like the most horrific XY problem. What conceivable circumstances do you have for having to forensically recreate some old version of the software beyond your deployment cycle being a pile of poo poo? If you're trying to fix something it must be deployed somewhere so get the files from there. If your problem is that a deployment overwrote something and killed it then a) you didn't test b) if your solution is to roll back then this is why people still use IE6.

Also, you seem to think SVN has no problem with 100s of MBs of blobs in it. It does. Try branching and merging with that.

Cybernetic Vermin
Apr 18, 2005

Notorious b.s.d. posted:

if it isn't text, it does not belong in git.

yes, art assets and binary documents and sw artifacts need to be version controlled, but you should be using something suited to the purpose. nexus is very well suited to most binaries, because it exists only to do that. it doesn't try to diff or delta them because they are not loving text

there is also poo poo like alfresco and documentum for word/pdf docs but corporate library poo poo aint my ballgame so i can't comment on that

my whole complaint is that it is horrifyingly stupid that git can't deal with some binaries ityool 2013, it was something that worked in most of the things that came before it. if i do a release and do 'svn tag/branch' i really ideally want all assets involved in that release to be tagged/branched.

i am sure nexus is a fantastic way to fix all this and all will be rainbows and ice-cream, but it is deeply frustrating to move away from having docs/test/management people just send their stuff into the same repository and having it work satisfactorily for everyone involved

Zombywuf posted:

This really sounds like the most horrific XY problem. What conceivable circumstances do you have for having to forensically recreate some old version of the software beyond your deployment cycle being a pile of poo poo? If you're trying to fix something it must be deployed somewhere so get the files from there. If your problem is that a deployment overwrote something and killed it then a) you didn't test b) if your solution is to roll back then this is why people still use IE6.

Also, you seem to think SVN has no problem with 100s of MBs of blobs in it. It does. Try branching and merging with that.

program trading desk is still running version 2 of the client whereas sales trading has moved to version 3, we need to backport a feature from version 3 to version 2 for them, in the process the documentation and testing procedures should be updated accordingly.

also lol at the idea of getting back to a customer liaison saying "oh yeah we'll fix that bug, just ask the customer to zip up the system and mail us a copy!"

besides, you make it sound like this is something that has to happen every day to be worth it. saving old data is free, it is a single keypress and you are insured forever that you can recreate it whenever you want. is ~~repository purity~~ so important to you that you would give up a chance of saving yourself a future headache?

FlapYoJacks
Feb 12, 2009

gucci void main posted:

shaggar do you even have a repo in real life? do you even commit?

I don't always commit, but when I do I break the build. :smug:

Zombywuf
Mar 29, 2008

Cybernetic Vermin posted:

program trading desk is still running version 2 of the client whereas sales trading has moved to version 3, we need to backport a feature from version 3 to version 2 for them, in the process the documentation and testing procedures should be updated accordingly.

also lol at the idea of getting back to a customer liaison saying "oh yeah we'll fix that bug, just ask the customer to zip up the system and mail us a copy!"

besides, you make it sound like this is something that has to happen every day to be worth it. saving old data is free, it is a single keypress and you are insured forever that you can recreate it whenever you want. is ~~repository purity~~ so important to you that you would give up a chance of saving yourself a future headache?

Program trading desk have upgraded to version 2.1 of the client which luckily was compatible with your version 2 interface, however your backported changes run against the version 2 client in your repo don't work with those in version 2.1. You deploy and it all falls over.

Meanwhile, your company, presumably to cheap to set up a test environment that mimics its production environments, get's audited for licensing violations and finds an unlicensed copy of version 2 of the client on everyone's machine.

Catalyst-proof
May 11, 2011

better waste some time with you

ratbert90 posted:

I don't always commit, but when I do I break the build. :smug:

cool reddit poo poo thanks you can leave now

Cybernetic Vermin
Apr 18, 2005

Zombywuf posted:

Program trading desk have upgraded to version 2.1 of the client which luckily was compatible with your version 2 interface, however your backported changes run against the version 2 client in your repo don't work with those in version 2.1. You deploy and it all falls over.

Meanwhile, your company, presumably to cheap to set up a test environment that mimics its production environments, get's audited for licensing violations and finds an unlicensed copy of version 2 of the client on everyone's machine.

uh, in this scenario the "client" is the thing in the repository, and i can't actually decipher what the components of the scenario you are imagining are



either which way, this is getting pretty spergy and dull; make sure everything exists at some well-defined point in a global version history, you're not my dad and can't tell me what not to put in svn, linus torvalds is retarded, it is friday and i am going drinking, peace

tef
May 30, 2004

-> some l-system crap ->
w
e
l
p

Zombywuf
Mar 29, 2008

Cybernetic Vermin posted:

either which way, this is getting pretty spergy and dull; make sure everything exists at some well-defined point in a global version history, you're not my dad and can't tell me what not to put in svn, linus torvalds is retarded, it is friday and i am going drinking, peace

If you ever put 100MB blobs in my repo I will hunt you down and kill you while I wait for the checkout to complete. HAND

Notorious b.s.d.
Jan 25, 2003

by Reene

Cybernetic Vermin posted:

my whole complaint is that it is horrifyingly stupid that git can't deal with some binaries ityool 2013, it was something that worked in most of the things that came before it.

[...]

it is deeply frustrating to move away from having docs/test/management people just send their stuff into the same repository and having it work satisfactorily for everyone involved

so you liked having people stuff random garbage into your vcs with a gui
and now you are mad that newer source control tools broke your bad, stupid use case



help my new 2013 macbook doesn't drive nails as well as my 1983 compaq portable. ugh why can't portable computers be used as a hammer anymore

Notorious b.s.d. fucked around with this message at 16:45 on Mar 15, 2013

libcxx
Mar 15, 2013
thread_local post<shit> shit_post("lol if u");
Great thread everyone, good discussion.

Shaggar
Apr 26, 2006
this reminds me of that dude who, in addition to his binaries, kept the entire .net toolchain in his vcs.

Jonny 290
May 5, 2005



[ASK] me about OS/2 Warp

Shaggar posted:

this reminds me of that dude who, in addition to his binaries, kept the entire .net toolchain in his vcs.

tagging this so i can put my eclipse directory in a git repo tonight








:haw:

Squinty Applebottom
Jan 1, 2013

cd / && git add . && git commit -m "3/15 backup"

Janitor Prime
Jan 22, 2004

PC LOAD LETTER

What da fuck does that mean

Fun Shoe
Sad I missed out on maven chat, but loled at

Cybernetic Vermin posted:

you're not my dad and can't tell me what not to put in svn, linus torvalds is retarded, it is friday and i am going drinking, peace

Max Facetime
Apr 18, 2009

storing your bitmaps, audio clips and other binaries in git would be perfectly fine if only linus had become the benevolent dictator of Linux UI development as well

trex eaterofcadrs
Jun 17, 2005
My lack of understanding is only exceeded by my lack of concern.

Shaggar posted:

I was a maven skeptic forever and now I couldn't live w/out it. my biggest problem was the group that wanted me to use it was using it way wrong and now that I understand it better its the best thing.

same here except mvn 1 & 2 were also dogshit but 3 really got me bonered up

PleasingFungus
Oct 10, 2012
idiot asshole bitch who should fuck off
I legit learned something from this conversation tho

something has gone wrong with this thread, that's not supposed to happen

(in the same vein: shaggar was right)

Cocoa Crispies
Jul 20, 2001

Vehicular Manslaughter!

Pillbug

Internaut! posted:

does ruby feature some exotic data structures

or were these talks more along the lines of "this is a list and this is why you should use it"

special counters and sets that you can update in different places (i.e in london and tokyo and texass) and still have them converge on the same value after replication

http://hal.inria.fr/docs/00/60/93/99/PDF/RR-7687.pdf

Cocoa Crispies
Jul 20, 2001

Vehicular Manslaughter!

Pillbug

Notorious b.s.d. posted:

oracle does this, they call it RAC for "real application clustering"

it is not log-shipping or an in-memory database hack

the way it works inside is a distributed block cache in RAM, shared across all servers. so no matter where you receive a request, the writes will be issued to the node that has those blocks in RAM

i have never built one bigger than 2 nodes, but i have heard of people having dozens of hosts in a cluster

lol, granny scalin'

isn't it really sensitive to machine configuration and stability?

FlapYoJacks
Feb 12, 2009

horse mans posted:

cool reddit poo poo thanks you can leave now

Hey man, what crawled up your butt and died. You keep acting that way and someone might actually piss in your cheerios. :mad:

Gazpacho
Jun 18, 2004

by Fluffdaddy
Slippery Tilde

Shaggar posted:

this reminds me of that dude who, in addition to his binaries, kept the entire .net toolchain in his vcs.
so what, microsoft does this routinely

e: i mean the toolchain part, not the built binaries DUH

Gazpacho fucked around with this message at 20:42 on Mar 15, 2013

Cocoa Crispies
Jul 20, 2001

Vehicular Manslaughter!

Pillbug

Shaggar posted:

this reminds me of that dude who, in addition to his binaries, kept the entire .net toolchain in his vcs.

isn't that entirely reasonable because microsoft routinely breaks it without updating version numbers or telling anybody?

Gazpacho
Jun 18, 2004

by Fluffdaddy
Slippery Tilde
in fact every company i've worked at since 05 does that in one form or another

Zombywuf
Mar 29, 2008

This thread is making me realise there are far worse places I could have worked at. How you guys stay sane I do not know.

Adbot
ADBOT LOVES YOU

Shaggar
Apr 26, 2006

Cocoa Crispies posted:

isn't that entirely reasonable because microsoft routinely breaks it without updating version numbers or telling anybody?

No you're thinking of apple.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply