Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
QuarkJets
Sep 8, 2008

Subjunctive posted:

Wouldn't you want to verify independently based on the computation as described in the paper, rather than assume that the software was a faithful implementation of it?

Yes. However, if there's a problem, then part of the process of verification is consulting with other people and asking about their implementations, because either you or they missed something. This is where comprehensible code becomes a godsend

SurgicalOntologist posted:

Is verification really always necessary? What if you have a 20-line script that generates your figures, puts the labels in the right places, etc? I can at least understand what verification would entail for (e.g.) a script that reads data and spits out the results of a statistical test, but even then, if scientists can't trust any packages/libraries they're using, nothing would get done.

The 20-line script that you're describing isn't what people mean when they say "scientific code". There's no need to verify that the legends are in the exact same place and that the title font on all of the fonts is exactly the same, because who gives a poo poo. Setting up figures to look a specific way isn't science, it's graphic design or something

At some point someone generated data that went onto those plots. Verifying the results and the procedure is what needs to happen.

And those common packages/libraries that you mentioned are extensively tested before hitting prime time. Some people even write papers about that, just to get the information out in the open and reviewed

Adbot
ADBOT LOVES YOU

SurgicalOntologist
Jun 17, 2004

Hughlander posted:

Yes. If a third party can't verify it, it's not science.

QuarkJets posted:

The 20-line script that you're describing isn't what people mean when they say "scientific code". There's no need to verify that the legends are in the exact same place and that the title font on all of the fonts is exactly the same, because who gives a poo poo. Setting up figures to look a specific way isn't science, it's graphic design or something.

At some point someone generated data that went onto those plots. Verifying the results and the procedure is what needs to happen.
And those common packages/libraries that you mentioned are extensively tested before hitting prime time. Some people even write papers about that, just to get the information out in the open and reviewed

Yes, that's exactly the point I was trying to make (regarding the libraries). Most "scientific code", in my field, is a consumer of scientific libraries but isn't doing anything all that interesting on its own. And it could be more elaborate than just producing a figure, there might be numerical ODE solutions, goodness-of-fits, whatever. But all that stuff is verified by the library, we're just doing a specific instantiation of it. If that doesn't count as scientific code, okay (maybe those of us who do the same old analyses over and over, on datasets from different experiments, aren't really scientists). Point is, it seemed like posters were insinuating that not only should libraries be tested, but also code that uses those libraries to do something completely boring should be independently tested. Which is extreme, even from the (correct) POV that scientists suck at coding.

As far as verification, in many fields the results and procedure refer to real-world phenomena and not some piece of code as the object of study. I'm not trying to say that scientific results don't need to be verified. Just that not every piece of code ever written needs to be verified.

I don't think we disagree. Of course scientists are terrible coders and have no idea about best practices, even the ones that are common sense and second nature to posters here. If "never write one-off scripts" was just "never write one-off scripts for novel algorithms", well then sure. But of all the problems with people's code in my field at least, publishing novel algorithms with one-off scripts is not one of them. Anyone coming up with a novel algorithm is going to do more than make a one-off script, because the hope is that the method will be taken up by people without much programming experience. It'll be packaged as easily as possible, or someone trying to make a career publishing novel algorithms will not last long. Will it be properly tested? Maybe, maybe not.

This is a stupid argument because I'm pretty sure everyone here is in agreement. It's just that some people were talking about novel computational techniques and others had in mind code that uses tried-and-tested algorithms to study real-world phenomena.

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe

QuarkJets posted:

Yes. However, if there's a problem, then part of the process of verification is consulting with other people and asking about their implementations, because either you or they missed something. This is where comprehensible code becomes a godsend

Also, your code implicitly documents all the implicit assumptions, judgements, and tweaks that are probably not going to be clearly described in your paper's methodology section. If I'm comparing two techniques, I want to evaluate your technique, not my interpretation of how you wrote up your technique. If the code is at all part of the subject of your research, it needs to be out there.

Pollyanna
Mar 5, 2005

Milk's on them.


Gazpacho posted:

MUMPS on Morphine

This is the most amazing thing I've ever read.

Plorkyeran
Mar 22, 2007

To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

Nippashish posted:

That is how I interpret "A paper written based on the output of a one-off script is pretty much worthless.".

While bad code is certainly common in one-off scripts, one does not imply the other. A one-off script can be coded well (although it's usually a waste of effort), and existing in a vaguely reusable form is certainly not enough to make code not bad.

vOv
Feb 8, 2014

SurgicalOntologist posted:

Yes, that's exactly the point I was trying to make (regarding the libraries). Most "scientific code", in my field, is a consumer of scientific libraries but isn't doing anything all that interesting on its own. And it could be more elaborate than just producing a figure, there might be numerical ODE solutions, goodness-of-fits, whatever. But all that stuff is verified by the library, we're just doing a specific instantiation of it. If that doesn't count as scientific code, okay (maybe those of us who do the same old analyses over and over, on datasets from different experiments, aren't really scientists). Point is, it seemed like posters were insinuating that not only should libraries be tested, but also code that uses those libraries to do something completely boring should be independently tested. Which is extreme, even from the (correct) POV that scientists suck at coding.

But even if you're using libraries correctly, how does the reader of your paper know that you didn't divide by 3 when you meant to divide by 13, or accidentally omit some data? A good example is a case where some economists just flat-out omitted five rows in their Excel spreadsheet.

ohgodwhat
Aug 6, 2005

And no one ever found out about it!

Nippashish
Nov 2, 2005

Let me see you dance!

Plorkyeran posted:

While bad code is certainly common in one-off scripts, one does not imply the other. A one-off script can be coded well (although it's usually a waste of effort), and existing in a vaguely reusable form is certainly not enough to make code not bad.

In that case I'm not really sure what you're saying. If you're saying it would be good if more scientists used better software engineering practices then I agree with you. If you are saying that no good science can come from bad engineering then you are wrong. If you're saying something else then I don't understand.

vOv
Feb 8, 2014

ohgodwhat posted:

And no one ever found out about it!

I'm not sure what your point is. Are you asking me to post an example of undiscovered errors in scientific studies?

Plorkyeran
Mar 22, 2007

To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

Nippashish posted:

In that case I'm not really sure what you're saying. If you're saying it would be good if more scientists used better software engineering practices then I agree with you. If you are saying that no good science can come from bad engineering then you are wrong. If you're saying something else then I don't understand.
If you write some code, it tells you X, and then you publish a paper about X, the code needs to be published along with the paper in a somewhat usable form, just as a paper about the result of an experiment needs to describe what was actually done in the experiment in enough detail that a reader has a chance of judging whether the methodology was valid. Thus, the code needs to be something that makes sense to run again at a later date, which a one-off script by definition is not.

The code not being a giant pile of garbage would certainly be helpful, but that is far less important than whether it even exists.

KaneTW
Dec 2, 2011

Data analysis code not having reproducible, environment-dependent results is unacceptable for scientific applications.

Code being badly written, hard to compile or read, etc. but still providing reproducible results when under peer review is fine.

QuarkJets
Sep 8, 2008

KaneTW posted:

Code being badly written, hard to compile or read, etc. but still providing reproducible results when under peer review is fine.

But also far from ideal, since part of the reproduction process may involve using or reviewing that code

etcetera08
Sep 11, 2008

So, what I'm seeing here is that good code is good but bad code is bad? Holy gently caress, y'all.

Nickopops
Jan 8, 2006
You must be this funky to ride.
.

Nickopops fucked around with this message at 09:37 on Nov 1, 2019

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Nickopops posted:

I'm only aware of two journal families (PLoS and I think the Royal Society) that have this as a requirement, and it's only recent (like, introduced this year). There may be other journals, but certainly no major scientific journals require code (or data) be made available, until quite recently.

Yes, and that's terrible.

QuarkJets
Sep 8, 2008

etcetera08 posted:

So, what I'm seeing here is that good code is good but bad code is bad? Holy gently caress, y'all.

Maybe if you're only capable of reading at a 3rd grade level, then sure

fritz
Jul 26, 2003

I saw a thing the other week where a prof decided to do an experiment where he wrote to authors asking for their software and had a grad student try to build it in under 30 minutes and there were lots and lots of things that wouldn't even compile.

Steve French
Sep 8, 2003



fritz posted:

I saw a thing the other week where a prof decided to do an experiment where he wrote to authors asking for their software and had a grad student try to build it in under 30 minutes and there were lots and lots of things that wouldn't even compile.

Was it this thing from two days ago from this thread

Coffee Mugshot
Jun 26, 2010

by Lowtax
Researchers should hire a full devops team, essentially.

SurgicalOntologist
Jun 17, 2004

vOv posted:

But even if you're using libraries correctly, how does the reader of your paper know that you didn't divide by 3 when you meant to divide by 13, or accidentally omit some data? A good example is a case where some economists just flat-out omitted five rows in their Excel spreadsheet.

ohgodwhat posted:

And no one ever found out about it!

vOv posted:

I'm not sure what your point is. Are you asking me to post an example of undiscovered errors in scientific studies?

I think the point is that the code (or spreadsheets or whatever) should be published alongside the article. That kind of error would get caught much sooner. Releasing analysis code is a far better solution than writing test suites for it. Not that they're mutually exclusive, but the former is achievable, the latter probably not.

Athas
Aug 6, 2007

fuck that joker

fritz posted:

I saw a thing the other week where a prof decided to do an experiment where he wrote to authors asking for their software and had a grad student try to build it in under 30 minutes and there were lots and lots of things that wouldn't even compile.

Yes, one of my friends did his masters thesis on embedded GPU programming languages in Haskell. According to him, it was a nightmare even getting the stuff to build and run, and Haskell isn't even that of a tricky language to compile. I think the issue is that research code often depends on obscure libraries or crazy setups.

Alien Arcana
Feb 14, 2012

You're related to soup, Admiral.
Sorry to break up the argument about scientific code and whatnot, but I was going through some of my own work and I found this line:

code:
TopicNode.Text = TopicNode.Text; // MAGIC
The best part is that this actually does something, and it was the simplest way I could find to do that something. Magic indeed.

VVVV
TopicNode is a member of a TreeView control, and Text is a property so setting it can have side-effects. This particular line comes right after I set the font of the node to bold. This causes the text to take up more horizontal room. However, the control doesn't recalculate the size of the label when I do this, and the text gets cut off as a result. Setting Text equal to itself is the easiest way to force recalculation.

Alien Arcana fucked around with this message at 18:05 on May 19, 2014

silvergoose
Mar 18, 2006

IT IS SAID THE TEARS OF THE BWEENIX CAN HEAL ALL WOUNDS




Can you tell us exactly *why* that does something?

Internet Janitor
May 17, 2008

"That isn't the appropriate trash receptacle."
If it's C#, Text could be a Property and reading or assigning to it could have arbitary side-effects.

Polio Vax Scene
Apr 5, 2009



Yeah probably this
code:
        string _Text;
        public string Text
        {
            get { return _Text; }
            set
            {
                _Text = value;
                DoMagic();
            }
        }

Alien Arcana
Feb 14, 2012

You're related to soup, Admiral.
yay, my first quote =/= edit

Powerful Two-Hander
Mar 10, 2004

Mods please change my name to "Tooter Skeleton" TIA.


Ages ago I posted about one of our developers implementing an XML webservice using what appeared to be his own XML parser written using regex's, today I figured out why when I got to look at what it was being used for - the application is sending data to the service by embedding XML as a string into an HTML form then posting the form back to the webservice! Wonderful!

And it gets better! For searches, there's a field in the form named "where" another named "text" and one named "values", what do these contain?

code:
values: column1,column2,column3
where: lower(foo.name+bar.name) like'%##TEXT###%'
text: butts
A genuine "post arbitrary unsanitised user input to the application, append to an SQL query and execute on the DB" in 2014, god drat.

Bonus: All user authentication is done by passing in a numeric user id which a) is used improperly in the first place b) goes against internal standards because it's not tied to our SSO c) does not actually authenticate anything (and there's no back end checking either) and d) you can override by changing the ID in the POST request to anything you like which then gets committed with no validation. Tomorrow I'll check if this also allows me to access anybody's account details by just changing the user id in the request which would be a great feature in a system that contains personal data :ughh: .

Damiya
Jul 3, 2012
So this new JS framework Famo.us was linked on HN.. Check it out:

http://famo.us/?new

Just make sure you have a scroll wheel on your mouse because otherwise you can't scroll or navigate up or down.

Why? Well, you see (emphasis mine)

quote:

First famo.us employee here, want to give people some food for thought to consider as you gripe about things like zoom and and the scrollbar (rightfully so). Some of the more controversial points, especially those regarding accessibility and progressive enhancement, made below are my own.

I think I'm the only person at the company that browses the web with JavaScript disabled, so I know how broken the original world wide web from when I first went online in 1994 is. I also remember using text-only browsers like lynx, and demonstrate it to co-workers on occasion. Lastly, I've been doing the web app development schtick for several years now and seen the good and the bad.

With all that in mind, the push towards progressive enhancement starting in 2003 when AJAX was just starting to become popular has proven to be a false prophet. IMHO, no idea has done more to make the web less accessible while also holding it back in terms of interactivity. You simply cannot use the same abstraction for both documents and applications and expect decent results for either.

What we should have done instead is allowed the two to diverge long ago and especially after 2007 when mobile computing took off and that change in screen size afforded us the opportunity to return to screens not all that dissimilar from those I remember using in '94. The first iPhone had a effective resolution of 320×480. You can effectively show as much on an iPhone as a 1994 computer monitor and have it be readable on both. In fact that's basis of the "Mobile First" design paradigm that frameworks like Zurb Foundation have pushed for.

Designing content for a single-wide column that scrolls up and down, that you can jump backward and forward on via anchor hypertext references was a really smart abstraction for lots of content. The site http://www.motherfuckingwebsite.com/ illustrates this poignantly.

The technology direction we should have taken should have made sure that search engines, the blind (or otherwise disabled or dexterally challenged) actually remained the first class citizens on the web instead of taking the back seat to print designers and interaction designers. It's easy to take the content designed for those constitutents and make it available via APIs that can then be consumed by people making rich web apps.

We even had something like that, it's called RSS and it is glorious. RSS may have actually survived if it had not been for progressive enhancement which was crap but just good enough that people could get away with killing RSS off. Every technologist should make themselves familiar with the concept of path dependency (this Slate articles on the persistence of rockets as our primary way of putting objects in space is illustrative: http://www.slate.com/articles/technology/future_tense/2011/0... ). Had progressive enhancement not become the road taken, no one would have ever argued to kill off RSS in favor of a hack jobbed DOM implemented where maybe 1 in 100 developers actually understood how to actually implement accessibility. RSS and other means of programmatic content consumption would have become the prime way of being indexable by search engines. Everyone would have maintained content that way because it would have been in their self interest to do so. Not only that, progressive enhancement is just usable enough to not get sued under the ADA. Without progressive enhancement, businesses (read: at least every fortune 500 company and then some) would have poured gobs money to into making sure they have the digital equivalent of wheelchair ramps. Progressive enhancement gave everyone the opportunity to do the least amount of effort and still maintain a defensible position if challenged legally.

The screen reading experience has been poo poo for like for ever and it's only gotten worse as designs have become more complex. Skipping over the level of abstraction that made content usable programmatically not only impoverished those that need it, but those who don't as well because we're deprived of richer, more flexible APIs.

Having worked with webapps for a while, I think one of the most amazing improvements in the last 5 years has been the move towards displaying content on many more screens than just the desktop. This has forced us to go back to API solutions (usually REST, but sometimes RPC over websockets) that often ape what we originally tried to accomplish with RSS. This is the only sane way to provide content that needs to be displayed in different ways, from a Mobile First single long scroll page rendered by the server to a Famous/Ember/Angular/Backbone/React app that takes that server-rendered page and replaces it entirely with a rich interactive experience in the browser.

We (at famo.us) don't yet have a solution for this future that treats everyone as a first class citizen, but we eventually plan to address this if the community doesn't tackle it first. We've been inspired by a few attempts at figuring out how to achieve this. Spike Brehm and crew over at AirBnB have been exploring this with Rendr, that works with Backbone.js, https://github.com/rendrjs/rendr , and August Lilleaas has worked on a solution for React, http://augustl.com/blog/2014/jdk8_react_rendering_on_server/ . We feel like there is potential for a generalized solution that doesn't need to be necessarily tied to how one framework works.

If you look deep at Famo.us, we don't throw out the DOM wholesale, instead we use the DOM where it matters. In Famo.us, there is a concept of a Surface (which is similar to Layers in CoreAnimation). The Surface is basically a container for HTML Document Fragments, usually fragments that are very strongly structured semantically. The entire scene graph above each Surface node basically helps you programmatically abstract away all the bullshit that many today handle awkwardly with levels and levels of DIV hell controlled by JavaScript. This is not only the abstraction that makes building things in Famo.us easy and fast, but also provides a boundary where content gets lumped into semantic chunks that oftentimes should be consumable for use in a long-scroll, semantically rich, Mobile First design where navigation is hyperlinked anchor based instead of spatial navigation.

When browsers were first designed, they made a decision to have a 1 to 1 relationship between the window object and the document object. While reasonable at the time (remember: small screens, low bandwidth, high latency), this proved unscalable to larger screens, greater bandwidth and lower latency. Take Twitter for example. Twitter.com is basically an application for displaying a tiny document called a Tweet. There hundreds of tweet documents shown at a time. The code for tweets is semantically rich. The code for the twitter app? That's DIV hell. This is what Famo.us helps with. Like Flash and Silverlight before it, it helps apps be apps, but unlike its predecessors, it allows documents to be documents. Keeping these two separate keeps abstractions from leaking into each other in the code you write.

So while some things may be broken, and the way its broken may inspire ire, it's important to understand that it's not where the puck is that matters, but where it is going. The scrollview we've made is still inferior to the native scroller in a few ways, such as missing keyboard bindings, but are temporary shortcomings that we and the community are going to address in time. Eventually the feature and quality gap will narrow and the Famo.us scrollview(s) will reach parity with native scrollers.

Furthermore, some of the abstractions in the Famo.us scroller are remarkable for their programmatic flexibility. When most people think of a scroller, they only think of three scenarios: vertical scrolling, horizontal scrolling and a pannable tiled area (like Google Maps or Open Street Maps). This is actually a pretty limited view of what's possible to explore if you abstract a scroller to its essence. A scroller at is essential is a mathematical set with a curser and that all scrolling does (whether by mouse or keyboard) is move the cursor position in that set. When you think about it that way, many more things you've seen are scrollers. For example, the Apple Time machine view is a scroller in the Z-direction. CoverFlow is another scroller that is horizontal, but where the affine matrix transform applied to a surface is based on the distance of a surface from the cursor and inverts on each side of the cursor. One team of beta testers used the scroll view to display a DNA double helix. It was basically two position linked vertical-scrollers, where the positioner function didn't just set the position on the Y-axis, but also manipulated the affine matrix to create the helix shape.

While probably not the best approach, even Google's new interactive Rubik's cube doodle could probably be implemented as a series of Famo.us scrollers. Each scroller would contain 12 surfaces (doubly linked in a ring), with pagination occurring at every three surfaces. The pagination function instead of translating the surfaces along only one axis instead rotates all the surfaces 90 degrees around an axis. All surfaces would belong to two scrollers "perpendicular" to one another at any given time, being lifted from one to the other and back depending on which one was in motion.

And since all this is based on the same scroller, which will eventually reach parity with native scrollers, it means that they get all the same features. Home goes to the top of a vertical scroller, the left of a western horizontal scroller, the beginning of the double helix or the begging of time in z-axis time machine like scroller. End does the opposite.

These examples just begin to scratch the surface of what's possible when you expose primitives that map to mathematical concepts like a set and current index to developers.

Last, but not least, we're going to look at all the feedback on the scrollview and try to address all the issues you all have encountered.

fritz
Jul 26, 2003

Steve French posted:

Was it this thing from two days ago from this thread

Oh whoops yeah.

I was looking at some code today that works on structs w/ two POD elements and there's a memcpy in there "for speed" that copies structs one at a time instead of, say, using a copy constructor or setting the two elements individually.

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
Why would I use that JS framework, famo.us? What does it give me?

Smugdog Millionaire
Sep 14, 2002

8) Blame Icefrog
Broken scrollbars, it sounds like.

Strong Sauce
Jul 2, 2003

You know I am not really your father.





Famous is suppose to offer near-native widgets built on top of html/javascript/css using 3d transforms AKA mark zuckerberg's dream. I'm guessing that's why there's no scrollbar support.

Soricidus
Oct 21, 2010
freedom-hating statist shill
tl;dr: accessibility is really important. That's why we've released a framework that doesn't support it.

qntm
Jun 17, 2009

Damiya posted:

So this new JS framework Famo.us was linked on HN.. Check it out:

http://famo.us/?new

Just make sure you have a scroll wheel on your mouse because otherwise you can't scroll or navigate up or down.

Why? Well, you see (emphasis mine)

quote:

These examples just begin to scratch the surface of what already exists

revmoo
May 25, 2006

#basta
I've been following famo.us for some time and I've come to believe that it's a bit of a turd. It seems like what it offers is very gimmicky and could be accomplished using other means. The API is such a low level that it doesn't seem to actually make anything easier, and the ridiculous staged-rollout they are doing trying to build hype is only hurting their cause. Also their documentation is garbage. Furthermore, all of the demo apps I've seen so far have been less than impressive. Sure 60fps animations are awesome, but the framework has so many issues and limitations all you can build is another loving Twitter app with (useless but flashy) animations.

I have a direct, specific need for something just like famo.us to do realtime data visualization, but I'll end up having to use WebGL or something to do what I need. It's unfortunate. I don't want a library to make page transitions fly off the screen, I need a grown-up library that can get real things done.

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
They don't use their own API. They haven't made their own app using their own framework. Your feedback is the only way they know to know how bad their API is.

If they built their own app, and this is the framework they built in the process, that would be cool. But no, this is a startup that's focusing on this garbage.

Run away.

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
From the employee:

quote:

What's the lesser of two evils?

1) Launch an imperfect product that works well for some use cases; or 2) Perfect everything and be ridiculed as vaporware

I wish we were Apple and had $159 billion in cash reserves. We're not though and have to operate within different constraints that don't afford us the same luxuries. You can't really compare Apples to Famo.uses

Dude. You're launching a web framework. Nobody needs you. You are 100% replaceable. If you disappoint me, there's 20 around the corner that I can go to to actually develop my app. In order to convince people to use you, you have to go a mile and beyond. And if you can't build a product that people actually want to use... maybe do something else?

pseudorandom name
May 6, 2007

Clearly you don't understand what it is to be a founder.

FamDav
Mar 29, 2008

Suspicious Dish posted:

From the employee:


Dude. You're launching a web framework. Nobody needs you. You are 100% replaceable. If you disappoint me, there's 20 around the corner that I can go to to actually develop my app. In order to convince people to use you, you have to go a mile and beyond. And if you can't build a product that people actually want to use... maybe do something else?

I don't think he knows how much a qa and an sdet would cost.

Also made for mobile:

FamDav fucked around with this message at 14:50 on May 20, 2014

Adbot
ADBOT LOVES YOU

leper khan
Dec 28, 2010
Honest to god thinks Half Life 2 is a bad game. But at least he likes Monster Hunter.

pseudorandom name posted:

Clearly you don't understand what it is to be a founder.

Founder of a thing that provides zero value. Does that mean, "I couldn't get a job at AAPL because they don't understand great ideas and people that is why I've made this revolutionary groundbreaking JavaScript framework for cloud fog usability for the modern web on tablets and mobile devices."

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply