|
I was illustrating a point.
|
# ? Jan 18, 2015 18:08 |
|
|
# ? Jun 12, 2024 19:57 |
|
carry on then posted:I was illustrating a point. Which you did. Just not the one you meant to.
|
# ? Jan 18, 2015 18:15 |
|
Sorry then. Checked exceptions are good and any language without them is utter garbage, I was wrong.
|
# ? Jan 18, 2015 18:21 |
|
carry on then posted:Sorry then. Checked exceptions are good and any language without them is utter garbage, I was wrong. God bless Java
|
# ? Jan 18, 2015 18:23 |
|
New to Java. Trying to get the Interactive Brokers API working. How do I install packages? I'm using code like this to import: import com.ib.client.ComboLeg; I ran an installer from Interactive Brokers, which created a directory on my hard drive that includes a directory tree. There are also files called . classpath, .project and build.xml. Based on searching, it looks like these are Ant files, but I've only found instructions that involve installing IDEs. Is there a console command or something? In Python, you'd run 'pip install packagename'. , or 'python setup.py install' . It looks like dropping the package folder in the same directory as my code works, but I'd like to install it as an official module. Dominoes fucked around with this message at 19:22 on Jan 22, 2015 |
# ? Jan 22, 2015 19:09 |
|
Dominoes posted:New to Java. Trying to get the Interactive Brokers API working. How do I install/import packages? If you're not using an IDE, you can specify the classpath at execution time. Basically, compile all the Interactive Brokers classes into one directory, then when you run your program, point the -cp flag to that directory. For more information, checkout this documentation from Oracle. Really though, if you're doing Java development for any extended period of time, you should use an IDE and a build system (Eclipse and Maven are free, and reasonable choices).
|
# ? Jan 22, 2015 19:26 |
|
Just run 'ant' in that directory and it should pop you out a jar
|
# ? Jan 22, 2015 19:28 |
|
Brain Candy posted:I love this example, because even if you check before you do anything there's a race if something else to deletes the file or changes permissions between your check and the read. This can happen more often then people expect especially if there is a network mount. I've also seen it occur in a log monitor which went to open the log right as the log was rolling. As to check exception vs unchecked exceptions, put me in the camp that it would be nice to not always have to deal with the boilerplate. However, the problem I have personally dealt with both in C++ and C# is that it can be hard to figure out all possible exceptions that might be thrown. Of course, a lot of bad lazy Java devs will simply make a try catch(Exception) block and then throw a RuntimeException out of that if I'm lucky. The worst part is, they often don't save the exception, or print out a message that will be System.err.println("Exception occured" + e). Maybe the solution is to have an IDE tell me all possible exceptions and create warnings if I don't deal with them? Actually, that is bad too since most programmers tend to treat a warning as it is all good.
|
# ? Jan 22, 2015 19:53 |
|
Dominoes posted:New to Java. Trying to get the Interactive Brokers API working. How do I install packages? .classpath and .project are eclipse metadata project files. build.xml is almost certainly for Apache Ant. Use ant to build that raw source into a JAR, then use the jar as your module.
|
# ? Jan 22, 2015 22:37 |
|
I'm looking for the absolute fastest way to process a large volume of data and turn it into a file based hash table. I have about 25,000 files, each around 256 megs worth of data thats divided into 52 byte chunks. What I need to do is take each chunk, find its hash value, and write it to a file with the name of the hash value. The hash table has 2048 files total (named /0.hash, /1.hash ... /2047.hash) As you can imagine there is a lot of IO, and performance here is in terms of days saved so I'm really looking for the fastest approach. My first approach was to create a map of hash value > bytearray, load a file one a time, break it apart into chunks and place each chunk into the appropriate bytearray from the map. When the file is finished, I iterate through the map and flush each bytearray into its own file. This works but I think there is a big slowdown writing to 2048 files after each file is read. Maybe an improvement is to batch several files before flushing them? Or using another data structure instead of the bytearray (technically a ByteArrayOutputStream). Also this is using standard java IO, I'm not very familiar with NIO so I don't know if it could apply here. I'd appreciate any ideas for speed that I can look into and start profiling. Thanks
|
# ? Jan 27, 2015 02:53 |
|
What's the background of this app? Consider on Linux mounting the source files in XFS/ext4 read-only then using an overlayfs with a tmpfs based read/write volume.
|
# ? Jan 27, 2015 03:14 |
|
MrMoo posted:What's the background of this app? Consider on Linux mounting the source files in XFS/ext4 read-only then using an overlayfs with a tmpfs based read/write volume. Its runnin on a 6TB NTFS harddrive so I don't think thats an option. Another thing is the hd is filled with about 4.5 TB of data, so the unprocessed files will have to be deleted in order to make space for the hash files.
|
# ? Jan 27, 2015 03:24 |
|
Even reading at 100MB/s that is 18 hours, you are going to need some form of accelerated storage, either SSD or a multi-spindle RAID array. Where is the data sourced from, can you acquire it in parallel to feed a farm of servers? MrMoo fucked around with this message at 03:46 on Jan 27, 2015 |
# ? Jan 27, 2015 03:43 |
|
Honestly 18 hours is great, my previous projection was 11 days. The data was in fact sourced in parallel (over several days), but it was dumped onto this drive so thats where I'm at now. Its just a matter of hashing it as fast as possible.
|
# ? Jan 27, 2015 03:56 |
|
With a single disk you could get ~100MB/s read, so two seconds to slurp an entire file into memory, then process, then dump out the files. However a single disk will have an IOPS of ~100 so that can be 40+ seconds (1 IO for file, 1 IO for metadata) to write out which is your 11 days for the entire data set. You need a compromise somewhere, either better hardware or a break in the output requirements. MrMoo fucked around with this message at 04:12 on Jan 27, 2015 |
# ? Jan 27, 2015 04:10 |
|
Thats a good point.. 2048 was the size chosen for the hash so the hash files can later be sorted in memory (at ~2 gig per file). A potential compromise is to cut the number of files down to 1024 or 512 even, and then I'd need to find a sorting solution that can sort a file externally in pieces rather than in memory all at once..
|
# ? Jan 27, 2015 04:22 |
|
Curious if anyone here has experience with Jetty hot deployment. It is a feature which supposedly reloads your .war packaged web application when you replace the existing .war file. I'm encountering a bug where Jetty will have difficulty finding a class in my application (even though the class is obviously there) and will throw a "class not found" exception until Jetty has been fully stopped and restarted. So currently I just do a full restart each time I deploy a new version of the application. This would normally not be a big deal but even worse, sometimes Jetty takes a while (1, 2 minutes tops) to restart since I have it configured to acquire a static chunk of memory on startup. Anyway, if there is any advice people have for getting bulletproof performance out of Jetty, I would love to hear it.
|
# ? Jan 27, 2015 04:49 |
|
FateFree posted:I have about 25,000 files, each around 256 megs worth of data thats divided into 52 byte chunks. FateFree posted:As you can imagine there is a lot of IO, and performance here is in terms of days saved so I'm really looking for the fastest approach. If you're using a 128 bit hash, you're only writing 4 kB to each of the 2048 output files after processing one 256 MB input file, so yes, you're being destroyed in seeks. Furthermore, since you're only writing 4 kB to each file at a time, the file system itself is probably not a great job on block allocation, such that the contents of each one of those files is widely spread, making read-back be a slow operation later. Honestly I'd probably write the hashes to one giant file and figure out the right way of sorting it later. Merge sort using temporary external files is a common approach, and I think is what GNU sort uses (although it operates on line data).
|
# ? Jan 27, 2015 17:35 |
|
I'm looking for a little help using Xstream Currently I have my output looking like this: code:
code:
|
# ? Feb 1, 2015 01:34 |
|
XStream just does a Bean to XML mapping right? So presumably right now you have a Venue class with street, city, state, etc. fields right? Instead you need an Address class with those fields and make that a field of Venue instead. Am I way off?
|
# ? Feb 1, 2015 02:25 |
|
I'm sure you're right, but I'm having trouble getting it implemented. I made an Address class, added the alias, and had the Venue class extend Address. Now I just get all the Address information first, and it still lacks the field blocks of <address>
|
# ? Feb 2, 2015 19:08 |
|
elite_garbage_man posted:had the Venue class extend Address Pretty sure this is the problem (never worked with xstream, but i did work with jaxb and friends). drat curriculum and their "inheritance" focus. Why would Venue extend Address? Polymorphism is ... sometimes needed. Mostly it is not. Do not extend objects for the sake of it. Professors should be getting rid of this "polymorphism everywhere" bug. This is not 1995 anymore. Think about it like this: "A is a special case of B. Having either A or B most of the time won't make a difference." If that's true, then sure, A extends B. If that's false, then hell no, do not ever ever extend anything. And in your case: Venue has Address. What does that tell you in term of their relationship?
|
# ? Feb 2, 2015 20:04 |
|
Regarding Apache HttpClient 4.3: Does anyone know how to change the user-agent after I've constructed an HttpClient? During building I can change it using HttpClientBuilder's setUserAgent(String) but I can't figure out how to change it afterwards. It used to be fairly easy to change (via HttpProtocolParams.setUserAgent) but they've now deprecated that class. It says to use "use configuration classes provided 'org.apache.http.config' and 'org.apache.http.client.config'" in the docs but I can't find anything in those packages that allows changing the user-agent. I can change it manually on every request by doing setHeader but I'd rather use a safer method where I won't accidentally forget to setup a request like that.
|
# ? Feb 3, 2015 02:59 |
|
I'm not finding anything that'd allow you to change the default UserAgent within an HttpClient. I expect that this has to do with thread-safety concerns, and trying to make the HttpClient immutable/stateless. You can mess with configuration when it's still a Builder, because there's no risk of some other thread changing state in the middle of a request. You can specify the header during a request, because there's no threat of that impacting another concurrent HttpRequest. If you are able to change the Client's default UserAgent, however, you can get into situations where you've started creating a request, get half of a config change, submit the request, and then get the other half of the config-change. Java code:
|
# ? Feb 3, 2015 04:53 |
|
Gravity Pike posted:I'm not finding anything that'd allow you to change the default UserAgent within an HttpClient. I expect that this has to do with thread-safety concerns, and trying to make the HttpClient immutable/stateless. You can mess with configuration when it's still a Builder, because there's no risk of some other thread changing state in the middle of a request. You can specify the header during a request, because there's no threat of that impacting another concurrent HttpRequest. If you are able to change the Client's default UserAgent, however, An option: HTTPRequestInterceptor hooked into the client that tweaked each request as it went out?
|
# ? Feb 4, 2015 21:55 |
|
Maven seems to waste a stunning amount of time doing nothing in particular. Kicking off each step in the chain incurs massive wait time as the plugin launches - for example it takes like 1:30 when it it launches maven-surefire-plugin with skipped tests. All in all I'm looking at a 9-minute build for a 7MB WAR file, almost all of which is precompiled libraries, on an i7-4770k. This happens under both Netbeans and Eclipse. Is there anything I can do to fix my build times here?
|
# ? Feb 4, 2015 23:40 |
|
Maven is slow but not that slow. How long does it take for these?pre:mvn test -DskipTests mvn test -Dmaven.test.skip=true
|
# ? Feb 4, 2015 23:48 |
Paul MaudDib posted:Maven seems to waste a stunning amount of time doing nothing in particular. Kicking off each step in the chain incurs massive wait time as the plugin launches - for example it takes like 1:30 when it it launches maven-surefire-plugin with skipped tests. All in all I'm looking at a 9-minute build for a 7MB WAR file, almost all of which is precompiled libraries, on an i7-4770k. This happens under both Netbeans and Eclipse. Is there anything I can do to fix my build times here? Are you running anti-virus? I've never had a problem with it but some coworkers were complaining of slow build times and we fixed it by whitelisting the directories that have source code and the maven repo. Build times substantially improved after that.
|
|
# ? Feb 4, 2015 23:55 |
|
fletcher posted:Are you running anti-virus? I've never had a problem with it but some coworkers were complaining of slow build times and we fixed it by whitelisting the directories that have source code and the maven repo. Build times substantially improved after that. Yes, it's antivirus. I spun up a Lubuntu VM on the same machine, it did a "clean package" in 7 seconds flat. My Tomcat deploy times went from 61 seconds to 6 seconds. Come to think of it, Selenium performed a lot better on a server VM that didn't include the standard AV. At the time I blamed that one on not being able to get it to load a clean browser profile properly, with Tomcat I just didn't have a frame of reference. But in hindsight I think all three of those apps are probably tripping the read/write scanning agent. I never put the pieces together, but Sophos just doesn't play nice with Java. We run Sophos+Bit9 and some other anti-malware. This combo gave me hell installing Netbeans, it would just silently kill the installer. Today I tried Gradle, and I realized it was doing the same silent kill thing to the gradle daemon, and the pieces fell into place for me. Paul MaudDib fucked around with this message at 01:10 on Feb 6, 2015 |
# ? Feb 6, 2015 01:05 |
|
A clarifying question; static methods in a class increase the memory footprint of instances of that class, right? Isn't this is one of the arguments for independent factory classes? edit: A second question on memory footprints; if methods, static and non-static, increase the memory footprint of an object, what would be the argument (aside from looser scope) against creating a 'reader' class to get and set the fields of an object? Woodsy Owl fucked around with this message at 11:56 on Feb 8, 2015 |
# ? Feb 8, 2015 11:50 |
|
Woodsy Owl posted:A clarifying question; static methods in a class increase the memory footprint of instances of that class, right? Isn't this is one of the arguments for independent factory classes? http://stackoverflow.com/a/1826807 I don't know why you would assume that methods, static or otherwise, would be a part of the instance. The instance contains a reference to the Class, and any member variables. It would make sense for instances to have references to functions in languages like JavaScript, although iirc in JavaScript you just overload methods with the default falling through to your prototype so even there it's not abysmal.
|
# ? Feb 8, 2015 15:36 |
|
I've got a Java question which I'm pretty sure is pretty easy, but I'm quite new to this whole Java programming thing. (I'm taking a class in it right now.) I've got a homework question that requires me to process a string through a loop and reverses the order of the string. So, I'm trying to use the charAt() method in the string class to concatenate that letter to the end of a string. So... code:
Thanks for your help! Frank Zappa fucked around with this message at 18:58 on Feb 8, 2015 |
# ? Feb 8, 2015 18:52 |
|
Frank Zappa posted:I've got a Java question which I'm pretty sure is pretty easy, but I'm quite new to this whole Java programming thing. (I'm taking a class in it right now.) Let's look at the JavaDoc: http://docs.oracle.com/javase/7/docs/api/java/lang/String.html#concat(java.lang.String) quote:public String concat(String str) It takes a String, not a character, so that's why yours doesn't work. There are a several ways to take a character and put it on the end of the string, but the common one is to use the + operator, which is overridden to concatenate a value and a String if one of the arguments is a String already. code:
Which has an append() method, which is overloaded to have variants that take all manner of primitive types as well as any Object. So whenever the conditions are met to make + a concatenation operation, the line I gave you would be replaced at compile time with something like code:
carry on then fucked around with this message at 19:09 on Feb 8, 2015 |
# ? Feb 8, 2015 18:59 |
|
carry on then posted:very helpful stuff Cool... thanks a bunch!!
|
# ? Feb 8, 2015 20:01 |
|
And to follow up, if you're allowed to for homework, you should just use StringBuilder. A typical novice mistake for java is to do what you're doing, which is to build a string by appending to an accumulator string over and over, since that involves creating a new builder and a new immutable string in every single round of your loop.
|
# ? Feb 8, 2015 23:40 |
|
Volmarias posted:http://stackoverflow.com/a/1826807 Thanks for the link, that cleared it up. Woodsy Owl fucked around with this message at 15:51 on Feb 10, 2015 |
# ? Feb 9, 2015 03:54 |
|
Trying to wrap my head around the concept of a Map. I understand they can be compared in some respects to an associative array in PHP. What I am wondering is, is it possible to create a Map constructor which assigns the index names. But have the ability to assign the values later by index.
|
# ? Feb 11, 2015 15:30 |
|
DholmbladRU posted:Trying to wrap my head around the concept of a Map. I understand they can be compared in some respects to an associative array in PHP. There is no concept of index in a map. You have key-value. The keys and values can be anything (objects), with some requirements on the keys behaviour depending on the map implementation that you're using (a common one is HashMap).
|
# ? Feb 11, 2015 15:38 |
|
Volguus posted:There is no concept of index in a map. You have key-value. The keys and values can be anything (objects), with some requirements on the keys behaviour depending on the map implementation that you're using (a common one is HashMap). So when creation of the map is performed the values have to be assigned? I guess instead of a map I could use a object to accomplish something similar.
|
# ? Feb 11, 2015 15:51 |
|
|
# ? Jun 12, 2024 19:57 |
|
DholmbladRU posted:Trying to wrap my head around the concept of a Map. I understand they can be compared in some respects to an associative array in PHP. To add onto Volguus, the best you could do is something like: code:
|
# ? Feb 11, 2015 15:56 |