|
Subjunctive posted:Wouldn't you want to verify independently based on the computation as described in the paper, rather than assume that the software was a faithful implementation of it? Yes. However, if there's a problem, then part of the process of verification is consulting with other people and asking about their implementations, because either you or they missed something. This is where comprehensible code becomes a godsend SurgicalOntologist posted:Is verification really always necessary? What if you have a 20-line script that generates your figures, puts the labels in the right places, etc? I can at least understand what verification would entail for (e.g.) a script that reads data and spits out the results of a statistical test, but even then, if scientists can't trust any packages/libraries they're using, nothing would get done. The 20-line script that you're describing isn't what people mean when they say "scientific code". There's no need to verify that the legends are in the exact same place and that the title font on all of the fonts is exactly the same, because who gives a poo poo. Setting up figures to look a specific way isn't science, it's graphic design or something At some point someone generated data that went onto those plots. Verifying the results and the procedure is what needs to happen. And those common packages/libraries that you mentioned are extensively tested before hitting prime time. Some people even write papers about that, just to get the information out in the open and reviewed
|
# ? May 17, 2014 23:09 |
|
|
# ? May 17, 2024 16:19 |
|
Hughlander posted:Yes. If a third party can't verify it, it's not science. QuarkJets posted:The 20-line script that you're describing isn't what people mean when they say "scientific code". There's no need to verify that the legends are in the exact same place and that the title font on all of the fonts is exactly the same, because who gives a poo poo. Setting up figures to look a specific way isn't science, it's graphic design or something. Yes, that's exactly the point I was trying to make (regarding the libraries). Most "scientific code", in my field, is a consumer of scientific libraries but isn't doing anything all that interesting on its own. And it could be more elaborate than just producing a figure, there might be numerical ODE solutions, goodness-of-fits, whatever. But all that stuff is verified by the library, we're just doing a specific instantiation of it. If that doesn't count as scientific code, okay (maybe those of us who do the same old analyses over and over, on datasets from different experiments, aren't really scientists). Point is, it seemed like posters were insinuating that not only should libraries be tested, but also code that uses those libraries to do something completely boring should be independently tested. Which is extreme, even from the (correct) POV that scientists suck at coding. As far as verification, in many fields the results and procedure refer to real-world phenomena and not some piece of code as the object of study. I'm not trying to say that scientific results don't need to be verified. Just that not every piece of code ever written needs to be verified. I don't think we disagree. Of course scientists are terrible coders and have no idea about best practices, even the ones that are common sense and second nature to posters here. If "never write one-off scripts" was just "never write one-off scripts for novel algorithms", well then sure. But of all the problems with people's code in my field at least, publishing novel algorithms with one-off scripts is not one of them. Anyone coming up with a novel algorithm is going to do more than make a one-off script, because the hope is that the method will be taken up by people without much programming experience. It'll be packaged as easily as possible, or someone trying to make a career publishing novel algorithms will not last long. Will it be properly tested? Maybe, maybe not. This is a stupid argument because I'm pretty sure everyone here is in agreement. It's just that some people were talking about novel computational techniques and others had in mind code that uses tried-and-tested algorithms to study real-world phenomena.
|
# ? May 18, 2014 01:48 |
|
QuarkJets posted:Yes. However, if there's a problem, then part of the process of verification is consulting with other people and asking about their implementations, because either you or they missed something. This is where comprehensible code becomes a godsend Also, your code implicitly documents all the implicit assumptions, judgements, and tweaks that are probably not going to be clearly described in your paper's methodology section. If I'm comparing two techniques, I want to evaluate your technique, not my interpretation of how you wrote up your technique. If the code is at all part of the subject of your research, it needs to be out there.
|
# ? May 18, 2014 01:57 |
|
Gazpacho posted:MUMPS on Morphine This is the most amazing thing I've ever read.
|
# ? May 18, 2014 17:13 |
|
Nippashish posted:That is how I interpret "A paper written based on the output of a one-off script is pretty much worthless.". While bad code is certainly common in one-off scripts, one does not imply the other. A one-off script can be coded well (although it's usually a waste of effort), and existing in a vaguely reusable form is certainly not enough to make code not bad.
|
# ? May 18, 2014 19:02 |
|
SurgicalOntologist posted:Yes, that's exactly the point I was trying to make (regarding the libraries). Most "scientific code", in my field, is a consumer of scientific libraries but isn't doing anything all that interesting on its own. And it could be more elaborate than just producing a figure, there might be numerical ODE solutions, goodness-of-fits, whatever. But all that stuff is verified by the library, we're just doing a specific instantiation of it. If that doesn't count as scientific code, okay (maybe those of us who do the same old analyses over and over, on datasets from different experiments, aren't really scientists). Point is, it seemed like posters were insinuating that not only should libraries be tested, but also code that uses those libraries to do something completely boring should be independently tested. Which is extreme, even from the (correct) POV that scientists suck at coding. But even if you're using libraries correctly, how does the reader of your paper know that you didn't divide by 3 when you meant to divide by 13, or accidentally omit some data? A good example is a case where some economists just flat-out omitted five rows in their Excel spreadsheet.
|
# ? May 18, 2014 19:06 |
|
And no one ever found out about it!
|
# ? May 18, 2014 19:12 |
|
Plorkyeran posted:While bad code is certainly common in one-off scripts, one does not imply the other. A one-off script can be coded well (although it's usually a waste of effort), and existing in a vaguely reusable form is certainly not enough to make code not bad. In that case I'm not really sure what you're saying. If you're saying it would be good if more scientists used better software engineering practices then I agree with you. If you are saying that no good science can come from bad engineering then you are wrong. If you're saying something else then I don't understand.
|
# ? May 18, 2014 19:22 |
|
ohgodwhat posted:And no one ever found out about it! I'm not sure what your point is. Are you asking me to post an example of undiscovered errors in scientific studies?
|
# ? May 18, 2014 19:59 |
|
Nippashish posted:In that case I'm not really sure what you're saying. If you're saying it would be good if more scientists used better software engineering practices then I agree with you. If you are saying that no good science can come from bad engineering then you are wrong. If you're saying something else then I don't understand. The code not being a giant pile of garbage would certainly be helpful, but that is far less important than whether it even exists.
|
# ? May 18, 2014 20:08 |
|
Data analysis code not having reproducible, environment-dependent results is unacceptable for scientific applications. Code being badly written, hard to compile or read, etc. but still providing reproducible results when under peer review is fine.
|
# ? May 18, 2014 22:04 |
|
KaneTW posted:Code being badly written, hard to compile or read, etc. but still providing reproducible results when under peer review is fine. But also far from ideal, since part of the reproduction process may involve using or reviewing that code
|
# ? May 18, 2014 22:50 |
|
So, what I'm seeing here is that good code is good but bad code is bad? Holy gently caress, y'all.
|
# ? May 18, 2014 23:59 |
|
.
Nickopops fucked around with this message at 09:37 on Nov 1, 2019 |
# ? May 19, 2014 01:37 |
|
Nickopops posted:I'm only aware of two journal families (PLoS and I think the Royal Society) that have this as a requirement, and it's only recent (like, introduced this year). There may be other journals, but certainly no major scientific journals require code (or data) be made available, until quite recently. Yes, and that's terrible.
|
# ? May 19, 2014 02:48 |
|
etcetera08 posted:So, what I'm seeing here is that good code is good but bad code is bad? Holy gently caress, y'all. Maybe if you're only capable of reading at a 3rd grade level, then sure
|
# ? May 19, 2014 04:45 |
|
I saw a thing the other week where a prof decided to do an experiment where he wrote to authors asking for their software and had a grad student try to build it in under 30 minutes and there were lots and lots of things that wouldn't even compile.
|
# ? May 19, 2014 05:13 |
|
fritz posted:I saw a thing the other week where a prof decided to do an experiment where he wrote to authors asking for their software and had a grad student try to build it in under 30 minutes and there were lots and lots of things that wouldn't even compile. Was it this thing from two days ago from this thread JawnV6 posted:Seems a fine time to link this: http://reproducibility.cs.arizona.edu/
|
# ? May 19, 2014 05:36 |
|
Researchers should hire a full devops team, essentially.
|
# ? May 19, 2014 05:53 |
|
vOv posted:But even if you're using libraries correctly, how does the reader of your paper know that you didn't divide by 3 when you meant to divide by 13, or accidentally omit some data? A good example is a case where some economists just flat-out omitted five rows in their Excel spreadsheet. ohgodwhat posted:And no one ever found out about it! vOv posted:I'm not sure what your point is. Are you asking me to post an example of undiscovered errors in scientific studies? I think the point is that the code (or spreadsheets or whatever) should be published alongside the article. That kind of error would get caught much sooner. Releasing analysis code is a far better solution than writing test suites for it. Not that they're mutually exclusive, but the former is achievable, the latter probably not.
|
# ? May 19, 2014 06:07 |
|
fritz posted:I saw a thing the other week where a prof decided to do an experiment where he wrote to authors asking for their software and had a grad student try to build it in under 30 minutes and there were lots and lots of things that wouldn't even compile. Yes, one of my friends did his masters thesis on embedded GPU programming languages in Haskell. According to him, it was a nightmare even getting the stuff to build and run, and Haskell isn't even that of a tricky language to compile. I think the issue is that research code often depends on obscure libraries or crazy setups.
|
# ? May 19, 2014 11:23 |
|
Sorry to break up the argument about scientific code and whatnot, but I was going through some of my own work and I found this line:code:
VVVV TopicNode is a member of a TreeView control, and Text is a property so setting it can have side-effects. This particular line comes right after I set the font of the node to bold. This causes the text to take up more horizontal room. However, the control doesn't recalculate the size of the label when I do this, and the text gets cut off as a result. Setting Text equal to itself is the easiest way to force recalculation. Alien Arcana fucked around with this message at 18:05 on May 19, 2014 |
# ? May 19, 2014 16:45 |
Can you tell us exactly *why* that does something?
|
|
# ? May 19, 2014 16:53 |
|
If it's C#, Text could be a Property and reading or assigning to it could have arbitary side-effects.
|
# ? May 19, 2014 16:55 |
Yeah probably thiscode:
|
|
# ? May 19, 2014 17:04 |
|
yay, my first quote =/= edit
|
# ? May 19, 2014 18:04 |
|
Ages ago I posted about one of our developers implementing an XML webservice using what appeared to be his own XML parser written using regex's, today I figured out why when I got to look at what it was being used for - the application is sending data to the service by embedding XML as a string into an HTML form then posting the form back to the webservice! Wonderful! And it gets better! For searches, there's a field in the form named "where" another named "text" and one named "values", what do these contain? code:
Bonus: All user authentication is done by passing in a numeric user id which a) is used improperly in the first place b) goes against internal standards because it's not tied to our SSO c) does not actually authenticate anything (and there's no back end checking either) and d) you can override by changing the ID in the POST request to anything you like which then gets committed with no validation. Tomorrow I'll check if this also allows me to access anybody's account details by just changing the user id in the request which would be a great feature in a system that contains personal data .
|
# ? May 19, 2014 23:16 |
|
So this new JS framework Famo.us was linked on HN.. Check it out: http://famo.us/?new Just make sure you have a scroll wheel on your mouse because otherwise you can't scroll or navigate up or down. Why? Well, you see (emphasis mine) quote:First famo.us employee here, want to give people some food for thought to consider as you gripe about things like zoom and and the scrollbar (rightfully so). Some of the more controversial points, especially those regarding accessibility and progressive enhancement, made below are my own.
|
# ? May 20, 2014 02:21 |
|
Steve French posted:Was it this thing from two days ago from this thread Oh whoops yeah. I was looking at some code today that works on structs w/ two POD elements and there's a memcpy in there "for speed" that copies structs one at a time instead of, say, using a copy constructor or setting the two elements individually.
|
# ? May 20, 2014 04:15 |
|
Why would I use that JS framework, famo.us? What does it give me?
|
# ? May 20, 2014 04:58 |
|
Broken scrollbars, it sounds like.
|
# ? May 20, 2014 05:22 |
|
Famous is suppose to offer near-native widgets built on top of html/javascript/css using 3d transforms AKA mark zuckerberg's dream. I'm guessing that's why there's no scrollbar support.
|
# ? May 20, 2014 05:52 |
|
tl;dr: accessibility is really important. That's why we've released a framework that doesn't support it.
|
# ? May 20, 2014 07:29 |
|
Damiya posted:So this new JS framework Famo.us was linked on HN.. Check it out: quote:These examples just begin to scratch the surface of what already exists
|
# ? May 20, 2014 12:15 |
|
I've been following famo.us for some time and I've come to believe that it's a bit of a turd. It seems like what it offers is very gimmicky and could be accomplished using other means. The API is such a low level that it doesn't seem to actually make anything easier, and the ridiculous staged-rollout they are doing trying to build hype is only hurting their cause. Also their documentation is garbage. Furthermore, all of the demo apps I've seen so far have been less than impressive. Sure 60fps animations are awesome, but the framework has so many issues and limitations all you can build is another loving Twitter app with (useless but flashy) animations. I have a direct, specific need for something just like famo.us to do realtime data visualization, but I'll end up having to use WebGL or something to do what I need. It's unfortunate. I don't want a library to make page transitions fly off the screen, I need a grown-up library that can get real things done.
|
# ? May 20, 2014 13:23 |
|
They don't use their own API. They haven't made their own app using their own framework. Your feedback is the only way they know to know how bad their API is. If they built their own app, and this is the framework they built in the process, that would be cool. But no, this is a startup that's focusing on this garbage. Run away.
|
# ? May 20, 2014 13:51 |
|
From the employee:quote:What's the lesser of two evils? Dude. You're launching a web framework. Nobody needs you. You are 100% replaceable. If you disappoint me, there's 20 around the corner that I can go to to actually develop my app. In order to convince people to use you, you have to go a mile and beyond. And if you can't build a product that people actually want to use... maybe do something else?
|
# ? May 20, 2014 13:54 |
|
Clearly you don't understand what it is to be a founder.
|
# ? May 20, 2014 14:15 |
|
Suspicious Dish posted:From the employee: I don't think he knows how much a qa and an sdet would cost. Also made for mobile: FamDav fucked around with this message at 14:50 on May 20, 2014 |
# ? May 20, 2014 14:21 |
|
|
# ? May 17, 2024 16:19 |
|
pseudorandom name posted:Clearly you don't understand what it is to be a founder. Founder of a thing that provides zero value. Does that mean, "I couldn't get a job at AAPL because they don't understand great ideas and people that is why I've made this revolutionary groundbreaking JavaScript framework for cloud fog usability for the modern web on tablets and mobile devices."
|
# ? May 20, 2014 14:57 |