Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
geeves
Sep 16, 2004

quote:

At work we have a couple of junior devs that my boss has asked me to work with to improve their code (ugh).

I thought I would hate this, too. But I've really enjoyed mentoring junior devs over the last year. It's actually helped with my own code because I have them review mine as well as each others. Several of the junior devs regularly pair program or at least are very communicative with each other to make sure knowledge is shared.

Clean Code is a great book and should be read by everyone. Effective Java and Java Concurrency in Practice are two must reads as well. It's great to discuss this with them, but to make it stick, you need to work it into projects and code reviews as well.

Good Will Hrunting posted:

You could try thinking about what you would have wanted when you were at their level and asking them if they feel like it fits their learning style. Everyone is different after all.

This and ask them what they want to get better at and in what technology they are interested. If it lines up see if it's possible to work it into a project. Sometimes they may not know what they want to work on until you broach the subject with them. For instance, I recently mavenized a lot of our applications and when talking about it with one of the junior devs, she told me she wanted to know more about the entire build process because it was a blackbox.

If they are too timid to tell you in person or even in email, set up an anonymous Google Doc for them to bullet point out things and add some things that you want to learn to help.

geeves fucked around with this message at 19:22 on Apr 2, 2017

Adbot
ADBOT LOVES YOU

Good Will Hrunting
Oct 8, 2012

I changed my mind.
I'm not sorry.
I was the same and owe a lot of my success to myself, especially on the Java side. Motivation isn't something you can change and I love doing this which is why (you too probably) progress came easy.

That said, transitioning to Scala (insert new tech that's hard really) has been easy on the "can I get stuff done?" front and I taught myself a decent amount but on the "am I doing this idiomatically and efficiently" side I really needed review of almost all the code I was writing to get comfortable and have things pointed out. I'm still getting there.

Carbon dioxide
Oct 9, 2012

I went to a new team recently.

The first story I did for them was something using an internal API I was very familiar with from the stuff I did last year.

Then I moved on, working on something different that made heavy use of RxJava. I hadn't done anything with ReactiveX before so that was quite a challenge for me. And I got a bit annoyed with my more senior colleagues because they were barely coaching me.

We sat down and talked about it last week - turned out that because I had done so well on that first story, they thought I was much more knowledgable about all frameworks they use than I really am. We agreed to spend more time on them explaining unfamiliar stuff to me.

What I'm saying is, everyone has gaps in their knowledge - and often they might not even realize it themselves, or have trouble expressing it. Don't fall for the trap thinking that because someone is good at some things, they know everything. Bring it up as something to discuss.

Tactical Shitpost
Jun 13, 2016

venutolo posted:

Does anyone have experience with how to get junior guys up to speed or how to approach this?

Others have already mentioned some good methods, so let me throw in this: On the code review itself i'd try asking them why they went with a certain solution - what worked in my neck of woods is guiding them to let them come up with solutions themselves (don't push on them too hard though, or they'll "shut down"). Besides that I'd try to show them what i would consider good code and why it's good - this can be part of your application, chapter from a book or something in an article - whatever. IM'ing them links to interesting programming articles from hackernews/reddit could have some positive effect too.

Truth to the matter is, if they don't want to improve themselves nothing will make them.

venutolo
Jun 4, 2003

Dinosaur Gum
First off, much thanks to all for the input.


geeves posted:

I thought I would hate this, too. But I've really enjoyed mentoring junior devs over the last year. It's actually helped with my own code because I have them review mine as well as each others. Several of the junior devs regularly pair program or at least are very communicative with each other to make sure knowledge is shared.

I've done a good amount of work mentoring junior devs and interns, but that has been mostly helping people from the start, and not trying to salvage guys who have been working for 18 months and few years, respectively. And also, simply put, I'm also tired of doing all that. We are a small group and I feel l get squeezed into this role of teaching all the junior guys because I'm the only one who can do it. And if I don't do, nobody does. So I end up spending a couple hours a day trying to get people to use good method names. Some other peer gets to spend that time working with AWS, or whatever, and advancing their career technical skills. I'd rather be doing that, and I'm frustrated. I am planning to look for a new job in the coming months, for what that is worth.

geeves posted:

Clean Code is a great book and should be read by everyone. Effective Java and Java Concurrency in Practice are two must reads as well. It's great to discuss this with them, but to make it stick, you need to work it into projects and code reviews as well.

My idea with getting them to read Clean Code would be to expose them to all those ideas, and then when I help them implement stuff or review their code, I can point back to the book.

I think both Effective Java and Concurrency in Practice are great, but I think they're both over the heads of the devs I'll be working with (CiP definitely is). I'll certainly point them to those two books, but for now I honestly think they're too advanced for both guys.

geeves posted:

This and ask them what they want to get better at and in what technology they are interested. If it lines up see if it's possible to work it into a project. Sometimes they may not know what they want to work on until you broach the subject with them. For instance, I recently mavenized a lot of our applications and when talking about it with one of the junior devs, she told me she wanted to know more about the entire build process because it was a blackbox.

I imagine there's a lot of stuff that is blackboxes to these guys. When I was in their spot and things like Maven, Spring stuf, servlet stuff, etc were blackboxes to me and so I did the work and research to make them not blackboxes to me. I imagine they do a lot of copy/pasting of existing stuff without much knowledge of how/why it works, which would drive me crazy. I need to know why and how things work. In the past I've tried to tell junior devs to never put in anything in the project that they don't know what it is and why it works. I don't know that there is a way to instill that without them being naturally that way, or disciplined enough to be able to do such a thing that does not come naturally.



Tactical Shitpost posted:

Others have already mentioned some good methods, so let me throw in this: On the code review itself i'd try asking them why they went with a certain solution - what worked in my neck of woods is guiding them to let them come up with solutions themselves (don't push on them too hard though, or they'll "shut down"). Besides that I'd try to show them what i would consider good code and why it's good - this can be part of your application, chapter from a book or something in an article - whatever. IM'ing them links to interesting programming articles from hackernews/reddit could have some positive effect too.

Truth to the matter is, if they don't want to improve themselves nothing will make them.

I have a couple small projects I wrote in the last year, which I think could be good to go over with them to demonstrate good design and code. I will try to do a more interactive and interrogative review of their work before they even push their work for pull request review.


e: Removed some non-productive, and overly dramatic stuff that was mostly born out of building work frustrations and isn't really indicative of how I feel.

venutolo fucked around with this message at 04:34 on Apr 3, 2017

Jo
Jan 24, 2005

:allears:
Soiled Meat
Is there a way for a javax.crypto.Cipher to act as both an encoder and a decoder without having to call .init() over and over again? Right now I've got this:

code:
		// Send the file we would like.
		val sharedCipher = Cipher.getInstance(SHARED_KEY_ALGORITHM)
		sharedCipher.init(Cipher.ENCRYPT_MODE, sharedKey)
		val message = java.util.Base64.getEncoder().encode(sharedCipher.doFinal("GET OUT!".toByteArray())).toString()
		fout.write(message)

		// Listed for the request.
		sharedCipher.init(Cipher.DECRYPT_MODE, sharedKey)
		return sharedCipher.doFinal(java.util.Base64.getDecoder().decode(fin.readText()))

I'd like to be able to avoid that whole additional sharedCipher.init or making two sharedCipher objects, but if it's low enough overhead I guess I can live with it.

nmx
May 16, 2004

venutolo posted:

I think what I struggle with most is relating to these guys who are not wired the same way I am. In sports, the great players generally do not make good coaches because they are unable to relate to and get frustrated by the non-great talent they have to coach (and who may not be driven to the same extent).

Yeah, I don't think your mentees are the problem here.

venutolo
Jun 4, 2003

Dinosaur Gum

nmx posted:

Yeah, I don't think your mentees are the problem here.

Haha, yeah my analogy is not great, and my personal frustrations are building up. Certainly being a good dev does not mean one cannot teach.

I definitely am coming across as much more of an rear end than I actually am. Again, I'm not saying I'm incredible, but I am motivated and driven enough to have learned a lot on my own and to not be complacent.

I'm just really, really frustrated.

E: Thanks for forcing me to have a bit of perspective, breathe, and reconsider things. I removed some of the previous post as it wasn't productive or truly indicative of how I'm feeling. This weekend has been uber frustrating.

venutolo fucked around with this message at 04:37 on Apr 3, 2017

denzelcurrypower
Jan 28, 2011
I'm working on a Java EE project using Glassfish. I have an EJB module which contains session and entity beans. I have an entity bean library which is a separate project also containing the entities, which I import to a remote class library which has remote interfaces for the session beans. In my web application, I've added the remote class library and the entity library. I can deploy both the EJB and war project, and am able to get a reference to the remote session bean in my web project. I can also instantiate classes from the entity library in my web project. However, when I try to use my generic session bean by passing a classname e.g. sessionBean.find(Customer.class, id) I get an error saying:

org.apache.jasper.JasperException: javax.ejb.EJBException: java.rmi.MarshalException: CORBA MARSHAL 1330446349 Maybe; nested exception is:
org.omg.CORBA.MARSHAL: ----------BEGIN server-side stack trace----------
org.omg.CORBA.MARSHAL: WARNING: 00810013: Could not find class entity.Customer in CDRInputStream.readClass vmcid: OMG minor code: 13 completed: Maybe

Any idea what could be causing this? The exact same project works fine on other peoples' computer, just not on mine. If any more info is needed please let me know.

For further reference, this person seems to have the same issue: http://stackoverflow.com/questions/40180092/ejb-remote-sending-class-class-throws-classnotfoundexception as well as this: https://community.oracle.com/thread/3844486

denzelcurrypower fucked around with this message at 01:34 on Apr 4, 2017

PierreTheMime
Dec 9, 2004

Hero of hormagaunts everywhere!
Buglord
So I managed to wrestle Eclipse and Maven into working order and get a working SOAP test created which returns my expected result from my API. What is the best (or at least a semi-standard) way of using SOAP for websites? I have absolutely zero experience with HTML/Javascript (outside of HTML 1.0 from the late 90s), but I have a feeling I'll need to get a few things in place. I tried searching for good basic examples and didn't turn up much--does anyone have a good idea of where I should start?

poemdexter
Feb 18, 2005

Hooray Indie Games!

College Slice

PierreTheMime posted:

So I managed to wrestle Eclipse and Maven into working order and get a working SOAP test created which returns my expected result from my API. What is the best (or at least a semi-standard) way of using SOAP for websites? I have absolutely zero experience with HTML/Javascript (outside of HTML 1.0 from the late 90s), but I have a feeling I'll need to get a few things in place. I tried searching for good basic examples and didn't turn up much--does anyone have a good idea of where I should start?

SOAP is really for service to service communication. Why not use REST? Is your API only allowing SOAP requests?

PierreTheMime
Dec 9, 2004

Hero of hormagaunts everywhere!
Buglord

poemdexter posted:

SOAP is really for service to service communication. Why not use REST? Is your API only allowing SOAP requests?

I may in the future, the current build is just a proof of concept for a requirement by our infosec to use SOAP to access the software's API. In all likelihood they could also use REST but I wanted to provide an example that met their requirement.

At this point I'm completely new to it all so I'm still assessing best options/practices.

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb

PierreTheMime posted:

I may in the future, the current build is just a proof of concept for a requirement by our infosec to use SOAP to access the software's API. In all likelihood they could also use REST but I wanted to provide an example that met their requirement.

At this point I'm completely new to it all so I'm still assessing best options/practices.

Salesforce has a SOAP API. Maybe you can get a free trial sandbox and look at what they do?

Carbon dioxide
Oct 9, 2012

PierreTheMime posted:

So I managed to wrestle Eclipse and Maven into working order and get a working SOAP test created which returns my expected result from my API. What is the best (or at least a semi-standard) way of using SOAP for websites? I have absolutely zero experience with HTML/Javascript (outside of HTML 1.0 from the late 90s), but I have a feeling I'll need to get a few things in place. I tried searching for good basic examples and didn't turn up much--does anyone have a good idea of where I should start?

For basic testing purposes I regularly use SoapUI. The GUI is absolutely horribly designed, and the free edition is quite limited, but you can just put in a SOAP endpoint URL, put in a SOAP request and it'll show the response. You could also use the paid version for making stub services for your testing environment.

For making a SOAP service you could use Spring: https://spring.io/guides/gs/producing-web-service/

For using one, well, just tell maven to generate classes from the service's WSDL file and then in code make an instance of that using the correct endpoint address. Once you do that you could just unit test on those as well, by mocking those classes. Saves you from setting up all kind of nonsense in SoapUI.

Ariong
Jun 25, 2012

Get bashed, platonist!

I am reinstalling Java, and have hit a snafu which is both baffling and infuriating. Here is the situation. I have a folder on my desktop that looks like this:


The 32 bit version of Java is installed to the "Java 32 Bit" folder. The 64 bit version of Java is installed to the "Java 64 Bit" folder. I close the folder. I install the 64 bit version of Java. I begin the installation of the Java 64 bit JDK. I reopen the folder to get the filetree. The folder now looks like this:



How did this happen? The 64 bit version of Java has been obliterated from my computer. This is reproducible, it happens whenever I try to install the JDK.

Carbon dioxide
Oct 9, 2012

Did it uninstall the 'old' version of 64 bit JRE, and install the 'new' version in its Program Files location?

One of the install settings is the folder to install it to.

Once you have installed java just add the folder to your windows PATH and you can run it from anywhere. I've seen people recommend also adding it as a JAVA_HOME windows environment variable but I'm not completely sure what tools require that.

TheresaJayne
Jul 1, 2011

Ariong posted:

I am reinstalling Java, and have hit a snafu which is both baffling and infuriating. Here is the situation. I have a folder on my desktop that looks like this:


The 32 bit version of Java is installed to the "Java 32 Bit" folder. The 64 bit version of Java is installed to the "Java 64 Bit" folder. I close the folder. I install the 64 bit version of Java. I begin the installation of the Java 64 bit JDK. I reopen the folder to get the filetree. The folder now looks like this:



How did this happen? The 64 bit version of Java has been obliterated from my computer. This is reproducible, it happens whenever I try to install the JDK.

Normally when you install it will go into Program Files and Program Files (x86) have you looked under "C:\Program Files\Java"

Also the first thing the installer does is ask which old versions do you want to uninstall, I just deselect Java 1.5 1.6 1.7 as i need to keep a version of each for work purposes (yes we have some projects still using 1.5)

TheresaJayne fucked around with this message at 08:31 on Apr 6, 2017

Boz0r
Sep 7, 2006
The Rocketship in action.
What's the simplest way to export a group of objects, with no circular references, to XML?

Tactical Shitpost
Jun 13, 2016

Boz0r posted:

What's the simplest way to export a group of objects, with no circular references, to XML?

jaxb or jackson with appropriate adapter

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE
One of my co-workers spent three weeks debugging a problem that was essentially traced back to a difference in floating-point addition between 1.4 and 1.8. Apparently you get numbers that are sliiiiiiightly different - at about 5 sigfigs. He thought he updated the jar he was comparing against 1.8 but I guess he forgot.

I also have not been allowed to touch the terrible junior-design-project (literally) level code in that library for weeks now until he could figure it out. Today I razzed him through the debugging process - so are the inputs the same at function Y? But they're different at the returns? OK, well, that's just a static function, so we should be able to write tests for the results. You're really really sure the inputs are the same and the outputs are different? OK well then it literally must be a platform problem, I can't imagine any other difference. I was about halfway through writing the test when the sheepish email came through.

So what exactly is different in FP math between 1.4 and 1.8, if anyone has a reference? I guess a difference at 5 sigfigs isn't too alarming after a couple cumulative operations (let's call it 6?), given that floating-point only has about 7 decimal points of significance - but still, that's a little weird in my book.

Paul MaudDib fucked around with this message at 20:48 on Apr 6, 2017

a witch
Jan 12, 2017

Which jvm, which platform? Implementations don't have to strictly follow IEEE 754. Any use of strictfp anywhere?

Ariong
Jun 25, 2012

Get bashed, platonist!

Carbon dioxide posted:

Did it uninstall the 'old' version of 64 bit JRE, and install the 'new' version in its Program Files location?

No, I completely deleted the "Java" folder from Program Files and Program Files(x86).

quote:


One of the install settings is the folder to install it to.

When I start up the installer, I check the "change destination folder" box, then put in the correct file location when prompted. The installation completes successfully, then is destroyed when I start up the jdk installer.

quote:

Once you have installed java just add the folder to your windows PATH and you can run it from anywhere. I've seen people recommend also adding it as a JAVA_HOME windows environment variable but I'm not completely sure what tools require that.
You're going to have to break this one down for me.

TheresaJayne posted:

Also the first thing the installer does is ask which old versions do you want to uninstall, I just deselect Java 1.5 1.6 1.7 as i need to keep a version of each for work purposes (yes we have some projects still using 1.5)

It always says there are three old versions to uninstall, but fails to do so. This has been the case for years, but was never a problem because I only recently started programming. I assumed that there was just some little remnant somewhere that made the installer give a false positive, since I cannot find these old versions anywhere I look. This persists even now that I've deleted the "Java" files from both Program Files and Program Files(x86):



a witch posted:

which platform?
Windows 7.


a witch posted:

Which jvm, (...) Implementations don't have to strictly follow IEEE 754. Any use of strictfp anywhere?
O_____O

Carbon dioxide
Oct 9, 2012

Ariong posted:

It always says there are three old versions to uninstall, but fails to do so. This has been the case for years, but was never a problem because I only recently started programming. I assumed that there was just some little remnant somewhere that made the installer give a false positive, since I cannot find these old versions anywhere I look. This persists even now that I've deleted the "Java" files from both Program Files and Program Files(x86):


That'd do it! You'll have Java stuff left somewhere in AppData, or in the Windows registry. The easiest way to get rid of it all might just be to reinstall all those versions in their default locations, then run their proper uninstallers, instead of just deleting folders. Randomly deleting program folders is almost always a bad idea in Windows, because Windows programs store configuration poo poo everywhere.

quote:

You're going to have to break this one down for me.

Okay. Go to your JDK folder. Inside, you see a /bin folder, right?
Take the full path of the bin folder, by clicking on Windows Explorer's address bar. In my case that's C:\Program Files (x86)\Java\jdk1.8.0_60\bin
Next, right click on This PC and choose Properties. In there, click on Advanced System Settings. In there, go to Environment Variables:



You'll see two blocks of settings: user variables at the top and system variables at the bottom. In the system variables you will find the Path variable. Choose to edit that, then add to the FRONT of the entire path string the full path such as C:\Program Files (x86)\Java\jdk1.8.0_60\bin , followed by a semicolon, and then the rest of the Path variable as it was.

Save your changes and now all of your dev tools will be able to find Java from anywhere on the system. It's not necessary, but it makes things easier when setting up new projects.

Additionally, in the same system variables screen, you can add a completely new variable called JAVA_HOME, which should have the same folder path but without the /bin, so something like: C:\Program Files (x86)\Java\jdk1.8.0_60 . As I said, some tools like having that around, but I'm not sure which ones, and if everything works fine without it, don't bother adding it.

Carbon dioxide fucked around with this message at 21:53 on Apr 6, 2017

Ariong
Jun 25, 2012

Get bashed, platonist!

Carbon dioxide posted:

That'd do it! You'll have Java stuff left somewhere in AppData, or in the Windows registry. The easiest way to get rid of it all might just be to reinstall all those versions in their default locations, then run their proper uninstallers, instead of just deleting folders. Randomly deleting program folders is almost always a bad idea in Windows, because Windows programs store configuration poo poo everywhere.
Alright! You know, this has been bothering me for a long time and it makes me really happy to ha

>:(((((

quote:

Okay. Go to your JDK folder.
I have not installed the JDK yet. Is it okay to do that before I have the corresponding version of Java installed?

Carbon dioxide
Oct 9, 2012

Ariong posted:

Alright! You know, this has been bothering me for a long time and it makes me really happy to ha

>:(((((

I have not installed the JDK yet. Is it okay to do that before I have the corresponding version of Java installed?

The JDK has the JRE included. If you plan on using the JDK, you won't need to install a JRE at all unless you need an older version of the JRE to run ancient Java programs that aren't compatible with the latest version anymore.

Edit: okay, because that reinstall idea didn't work - here's a guide by Oracle itself on how to fix this situation:
https://java.com/en/download/help/manual_regedit.xml

Carbon dioxide fucked around with this message at 22:08 on Apr 6, 2017

a witch
Jan 12, 2017

e: Oops, my original post was replying to someone elses question about FP.


Welcome to hell.

JLS Section 4.2.3 posted:

The float, float-extended-exponent, double, and double-extended-exponent value sets are not types. It is always correct for an implementation of the Java programming language to use an element of the float value set to represent a value of type float; however, it may be permissible in certain regions of code for an implementation to use an element of the float-extended-exponent value set instead. Similarly, it is always correct for an implementation to use an element of the double value set to represent a value of type double; however, it may be permissible in certain regions of code for an implementation to use an element of the double-extended-exponent value set instead.

Basically, Java is allowed to use extended precision, especially as an intermediate value during calculations involving floating point numbers.

I'm sorry I don't know exactly when a change may have occurred, but if you weren't using FP-strict expressions, then results aren't portable between different Java versions or platforms.

a witch fucked around with this message at 22:16 on Apr 6, 2017

ulmont
Sep 15, 2010

IF I EVER MISS VOTING IN AN ELECTION (EVEN AMERICAN IDOL) ,OR HAVE UNPAID PARKING TICKETS, PLEASE TAKE AWAY MY FRANCHISE

Paul MaudDib posted:

So what exactly is different in FP math between 1.4 and 1.8, if anyone has a reference?

...I think there were bugs fixed every so often, particularly for display purposes, so God only knows.

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE

a witch posted:

e: Oops, my original post was replying to someone elses question about FP.


Welcome to hell.


Basically, Java is allowed to use extended precision, especially as an intermediate value during calculations involving floating point numbers.

I'm sorry I don't know exactly when a change may have occurred, but if you weren't using FP-strict expressions, then results aren't portable between different Java versions or platforms.

This is me. Yes, that sounds like a pretty good explanation of what happened. Nailed it. :hfive:

I know that Java isn't as platform independent as it pretends. That said, that's still pretty ugly. No, we certainly didn't have strictfp enabled.

Super fun thought: we are also running this on UltraSPARC in production. If it varies like this, I can't imagine that it's the same across x86 and SPARC. Super fun to be working out classifier bugs with this bullshit.

Thank gently caress for our serverdad who buys older hardware for a decent test environment. I owe that guy a beer, production heart-attack averted :sigh:

Paul MaudDib fucked around with this message at 22:37 on Apr 6, 2017

PierreTheMime
Dec 9, 2004

Hero of hormagaunts everywhere!
Buglord
So I have a report application that pulls data from an API and adding results to a List. At this point everything works fine but I'm looking to increase performance by having the pulls subdivided into multiple threads to save on time. Is using a Collections.SynchronizedList() and a synchronized method for adding items okay or is there a better thread-safe option? This would be running on a server with 1.7.

a witch
Jan 12, 2017

A SynchronizeList has an internal mutex to synchronize all methods. You don't need to wrap them in another synchronized method.

A common pattern for what you're doing is to use a ConcurrentLinkedQueue, which can be shared by producers and consumers, and will likely have better performance.

TheresaJayne
Jul 1, 2011

Ariong posted:

Alright! You know, this has been bothering me for a long time and it makes me really happy to ha

>:(((((

I have not installed the JDK yet. Is it okay to do that before I have the corresponding version of Java installed?

The JDK is the corresponding version

The JDK comes with its own version of the JRE

C:\Program Files\Java\jdk1.8.0_101\jre

you don't need to install the JRE seperately at all, just point the JAVA_HOME to the JDK and you should be fine.

CPColin
Sep 9, 2003

Big ol' smile.

PierreTheMime posted:

So I have a report application that pulls data from an API and adding results to a List. At this point everything works fine but I'm looking to increase performance by having the pulls subdivided into multiple threads to save on time. Is using a Collections.SynchronizedList() and a synchronized method for adding items okay or is there a better thread-safe option? This would be running on a server with 1.7.

If you don't want to roll your own thread pool, you could use an ExecutorService and submit your tasks to that, instead of adding them to a queue.

PierreTheMime
Dec 9, 2004

Hero of hormagaunts everywhere!
Buglord

CPColin posted:

If you don't want to roll your own thread pool, you could use an ExecutorService and submit your tasks to that, instead of adding them to a queue.

That's what I'm using now. I have a pool for threads for each report type and number of servers (216 so far) so if two of the reports have to use a sub-pool it's going to be quite a bit more.

The killer here is the API limits returns on data to 5000 objects, which caps out really quickly since it's a report for process executions on busy servers. So I am breaking it up by appdev team and further by the hour. We're just spinning up this new tool so we only have a fraction of teams moved over and even still that would mean thousands of additional threads. It's not all that much data overall and I doubt there will be memory issues but it's a lot to keep track of.

The Laplace Demon
Jul 23, 2009

"Oh dear! Oh dear! Heisenberg is a douche!"

a witch posted:

A SynchronizeList has an internal mutex to synchronize all methods. You don't need to wrap them in another synchronized method.

A common pattern for what you're doing is to use a ConcurrentLinkedQueue, which can be shared by producers and consumers, and will likely have better performance.

Just don't expect a ConcurrentLinkedQueue to behave exactly like a synchronized one (it's lock-free, synchronized is blocking). You might also encounter some timing weirdness of when thread A publishes and when thread B consumes. An ArrayBlockingQueue with capacity equal to some multiple of the number of consumers is a direct upgrade to the synchronized list approach with built-in backpressure.

If the performance improvement is to parallelize the fetches and you don't use the results until you're all done, I'd say keep it simple: have each thread append to its own ArrayList, and combine them when they're all done, i.e. the ExecutorService approach.

The Laplace Demon fucked around with this message at 19:43 on Apr 7, 2017

FateFree
Nov 14, 2003

Ok time for some friendly architecture discussion. I'm interested in knowing how you gents would model a list of objects that should all be displayed on a UI in a row, but each one having different properties.

For instance HTML form fields would be a perfect analogy - some things are input fields with a name and value, other inputs are dropdowns that have their own collection of items to display, a text field may have a max length etc.

So from an inheritance perspective it probably makes sense to have a parent class with common attributes like name and value and child classes for the rest, but how would you model your DB tables? Different tables for each subclass and an ugly query to pull your list of objects? (I'm using Hibernate/JPA with Spring Boot btw.) Or what about a single 'Inputs' table with a JSON column for all the variable data (like the dropdown options). That would make the db very simple and I'll never need to query any of the JSON stuff anyway. But then you might have to do some manual casting to your child classes after you retrieve everything. Any model a similar structure before?

Tactical Shitpost
Jun 13, 2016

FateFree posted:

Ok time for some friendly architecture discussion. I'm interested in knowing how you gents would model a list of objects that should all be displayed on a UI in a row, but each one having different properties.

For instance HTML form fields would be a perfect analogy - some things are input fields with a name and value, other inputs are dropdowns that have their own collection of items to display, a text field may have a max length etc.

So from an inheritance perspective it probably makes sense to have a parent class with common attributes like name and value and child classes for the rest, but how would you model your DB tables? Different tables for each subclass and an ugly query to pull your list of objects? (I'm using Hibernate/JPA with Spring Boot btw.) Or what about a single 'Inputs' table with a JSON column for all the variable data (like the dropdown options). That would make the db very simple and I'll never need to query any of the JSON stuff anyway. But then you might have to do some manual casting to your child classes after you retrieve everything. Any model a similar structure before?

More data with what the child classes can be would be useful, but assuming your database can't do JSON natively (recent Postgres can do it) i'd probably model the db like this: Have columns (itemId [number, pk], propertyName [varchar2, pk], propertyValue [varchar2]) with the itemId having the value taken from a sequence. PropertyName and PropertyValue would be just a key-value store for whatever you need stored in these classes.

The options available in the dropdowns would probably go into a different table with columns like (groupId [number, pk], value [varchar2, pk], description [varchar2])

The trick is that you'd probably need to have a separate db model classes to do stores/loads in addition to the domain model.

FateFree
Nov 14, 2003

It always felt so unnecessary to me to have all these tables with property names and property values so I'm leaning towards the json. I don't mean native via the db, I mean registering a converter in JPA to convert an object into json and store it in a string field. I'm always torn because it feels hacky but then again if the structure doesnt fit well into a db schema then it might just be the next best thing..

TheresaJayne
Jul 1, 2011

FateFree posted:

It always felt so unnecessary to me to have all these tables with property names and property values so I'm leaning towards the json. I don't mean native via the db, I mean registering a converter in JPA to convert an object into json and store it in a string field. I'm always torn because it feels hacky but then again if the structure doesnt fit well into a db schema then it might just be the next best thing..

i once worked on a system with one database containing 3 tables all identical HistoryTable, CurrentTable, DeletedTable

all tables had the following structure

Primary - Varchar(255)
Secondary - VarChar(255)
Description - VarChar(255)
Type - Varchar(255)
name - Varchar(255)
body - CLOB

in the body they had xml for data value pairs

<entry>
<name></name>
<value></value>
</entry>


For dev work it was amazing, no recreating DBs or anything just add a new entry for the new column
for Production we tried to send 1000000 SMS text messages consisting of 3 total messages = 3,000,000 total SMS messages.

Preparing the data took over 72 hours, we stopped the process rewrote it quickly to use queues and standard SQL and the new system sent all 3,000000 messages in 20 minutes.

Volguus
Mar 3, 2009

TheresaJayne posted:

i once worked on a system with one database containing 3 tables all identical HistoryTable, CurrentTable, DeletedTable

all tables had the following structure

Primary - Varchar(255)
Secondary - VarChar(255)
Description - VarChar(255)
Type - Varchar(255)
name - Varchar(255)
body - CLOB

in the body they had xml for data value pairs

<entry>
<name></name>
<value></value>
</entry>


For dev work it was amazing, no recreating DBs or anything just add a new entry for the new column
for Production we tried to send 1000000 SMS text messages consisting of 3 total messages = 3,000,000 total SMS messages.

Preparing the data took over 72 hours, we stopped the process rewrote it quickly to use queues and standard SQL and the new system sent all 3,000000 messages in 20 minutes.

That's a "Coding Horrors" story.

Adbot
ADBOT LOVES YOU

Data Graham
Dec 28, 2009

📈📊🍪😋



Database, that's like a spreadsheet right ??

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply