|
By the way, you shouldn't store currency in floats.
|
|
#
?
Jun 11, 2017 20:57
|
|
|
|
| # ? May 15, 2024 04:22 |
|
|
Thermopyle posted:This thread has always been open to beginners. The poster who has been asking questions is obviously a beginner. Floating point stuff obviously is weird to beginners. As it should be (open to beginners). My issues was not with him, it was with the guy who thinks an acceptable answer is there's something wrong with you if you think you have to worry about the 64th decimal place, and the guy who thinks theres something wrong with needing to know arcane implementation details. Some of us work on projects where we do care about the 64th decimal place and we do care about messy internals. Eela6 posted:As a note, the fact that approximate equality is often useful when comparing floats comes up often enough that isclose was added to the math and cmath modules in Python 3.5. It's also in Numpy if you're not on Python 3
|
|
#
?
Jun 11, 2017 20:58
|
|
|
Nippashish posted:Floating point weirdness is also one of those topics that programmers love to expand on at length when it comes up (I'm slightly surprised no one has warned you about storing currency in floats yet) and my suggestion to "not worry about it" should be taken in the context of your post triggering a page of responses. The fact that you went away from the discussion with the impression that you should "watch out" when using floats is exactly what I was trying to mitigate. But you should worry about it though? To the point that you're aware that there are some basic pitfalls, and then keep them in mind when you're using floats. The poster you're replying to was already not worrying about it, that's why they hit a problem and had no idea what was going wrong I mean yeah, the other extreme is the 'you can't trust floats' one, where people pretend they're full of gremlins adding random inaccuracy when your back is turned, but that's not the same as being wary and knowing what does and doesn't work
|
|
#
?
Jun 11, 2017 21:03
|
|
|
Eela6 posted:By the way, you shouldn't store currency in floats. I'm assuming that you should use doubles but please expand on this
|
|
#
?
Jun 11, 2017 21:07
|
|
|
KernelSlanders posted:As it should be (open to beginners). My issues was not with him, it was with the guy who thinks an acceptable answer is there's something wrong with you if you think you have to worry about the 64th decimal place, and the guy who thinks theres something wrong with needing to know arcane implementation details. Nippashish is not saying either of those things. He gave an extremely reasonable rule of thumb for how to think of floats and suggested that most people don't have to read the Floating Point Arithmetic chapter of a CS textbook just to use floats effectively
|
|
#
?
Jun 11, 2017 21:07
|
|
baka kaba posted:I mean yeah, the other extreme is the 'you can't trust floats' one, where people pretend they're full of gremlins adding random inaccuracy when your back is turned, but that's not the same as being wary and knowing what does and doesn't work You can't trust them, though. They seem to stay still when your back is turned, but I can feel the heat of their hideous eyes. Watching. Waiting. Ready to pounce. They may not be Real, but their claws are sharp enough. funny Star Wars parody posted:I'm assuming that you should use doubles but please expand on this Stack Overflow has a reasonable answer. You should not use doubles - you should use Decimals. To quote the relevant part: Zneak from Stack Overflow posted:The problem with floats and doubles is that the vast majority of money-like numbers don't have an exact representation as a integer times a power of two. In fact, the only fractions of a hundred between 0/100 and 100/100 (which are significant when dealing with money because they're integer cents) that can be represented exactly as an IEEE-754 binary floating-point number are 0, 0.25, 0.5, 0.75 and 1. All the others are off by a small amount. I mostly posted the point about currency to tweak Nippanish's nose, but I'm actually glad I did. More people should know. Eela6 fucked around with this message at 21:19 on Jun 11, 2017 |
|
|
#
?
Jun 11, 2017 21:08
|
|
|
funny Star Wars parody posted:I'm assuming that you should use doubles but please expand on this Back when bitcoin was still newer a bunch of people learned first-hand that floating-point arithmetic has precision issues. Say I have $2 in my account and I deposit an additional $0.10 cents. Because my financial software is bad and written by someone with almost no coding experience, now my account balance reads $2.09 (in memory it reads as 2.099999....). But even if your website is smarter than that and chooses to round up, these precision errors will slowly accumulate over time and the balance will be different than if someone had gone through with a calculator and tried to calculate the balance by hand. Similar example, compound interest calculations are performed repeatedly and can result in numerous floating point errors compounding into a significant deviation between what's reported in the account and what should actually be in the account.
|
|
#
?
Jun 11, 2017 21:14
|
|
|
baka kaba posted:But you should worry about it though? To the point that you're aware that there are some basic pitfalls, and then keep them in mind when you're using floats. The poster you're replying to was already not worrying about it, that's why they hit a problem and had no idea what was going wrong My stance is that you should be aware floats are not reals, and but you should also not worry about it. The poster I was replying to was not aware of this (and they got lots of explanations in this thread, so that part worked out well).
|
|
#
?
Jun 11, 2017 21:16
|
|
|
Eela6 posted:I mostly posted the point about currency to tweak Nippanish's nose, but I'm actually glad I did. More people should know. I used to write modelling software for an insurance company and we did 100% of our currency calculations with floats. I hope this annoys you  .
|
|
#
?
Jun 11, 2017 21:23
|
|
|
KernelSlanders posted:the guy who thinks theres something wrong with needing to know arcane implementation details. i was making a joke about the guy psyducking at a beginner who asked a question about how numbers work. i am positively thrilled about your non-web-dev coding challenges and wish you well
|
|
#
?
Jun 11, 2017 21:36
|
|
|
Cingulate posted:Python is a language that's being used by people with very little CS knowledge (e.g., me). That's, from what I can tell, by design: Python is easy to learn, easy to read, welcoming and forgiving. I think the thread has a good track record of being exactly like that, too. Nippashish posted:That's why I'm suggesting a very simple mental model that is intuitive and also sufficient for even non-beginners. Teaching people that floating point numbers are dark and spooky and complicated isn't very productive, because very few people need to care about that level of detail. Actually this is the worst possible takeaway because Numerical analysis is a) hard and b) for even most practical purposes requires an understanding of when it'll fail on you. Catastrophic cancellation and loss of precision can lead to cases where your component will fail and fail hard. Unless you don't want to engineer robustly, sure you can ignore it. I have run into many cases where poor or no understanding of float approximations have lead to pernicious bugs in production systems costing lots of money While I don't expect people to understand IEEE-754 the standard in its entirety, it is immensely unhelpful to present it as a real number/exact fraction abstraction since it's leakier than a sieve (but designed in such a way to be useful for those in the know) and frequently beginners will smash their heads against the wall for days until they are told how the semantics of floats work when they run into "random" "non-deterministic" errors that are 100% deterministic. For example, addition is commutative but not associative. This isn't trivial nor particularly expected. funny Star Wars parody posted:I'm assuming that you should use doubles but please expand on this True A way of salami slicing Perfectly fine if your algorithms are insensitive to things like that (i use floats in quant finance optimization models) but try simulating a chaotic system and watch as floats are completely useless It gets better when you have no idea how chaotic your system is!
|
|
#
?
Jun 11, 2017 21:50
|
|
|
LochNessMonster posted:As the root cause of this discussion as well as a beginner in terms of programming I can tell you that you're over simplifying things. Floats are 100% deterministic. They will do 100% of what you tell them to do, but that's a pretty complex operation involving rescales and possibly higher precision intermediaries. If you are calculating mathematical functions, use a library or look up a numerically stable algorithm.
|
|
#
?
Jun 11, 2017 21:52
|
|
|
Malcolm XML posted:Actually this is the worst possible takeaway because Numerical analysis is a) hard and b) for even most practical purposes requires an understanding of when it'll fail on you. Catastrophic cancellation and loss of precision can lead to cases where your component will fail and fail hard. Unless you don't want to engineer robustly, sure you can ignore it. I have run into many cases where poor or no understanding of float approximations have lead to pernicious bugs in production systems costing lots of money Malcolm XML posted:Floats are 100% deterministic. They will do 100% of what you tell them to do, but that's a pretty complex operation involving rescales and possibly higher precision intermediaries. This is the level of worrying that Nippashish said a newbie shouldn't be at, and he was 100% right. "Why is this single line of floating-point arithmetic giving me a funny-looking answer" requires a very basic understanding of the issue, not "read this essay on the difficult subject of numerical analysis" QuarkJets fucked around with this message at 21:58 on Jun 11, 2017 |
|
#
?
Jun 11, 2017 21:56
|
|
|
QuarkJets posted:This is the level of worrying that Nippashish said a newbie shouldn't be at, and he was 100% right. "Why is this single line of floating-point arithmetic giving me a funny-looking answer" requires a very basic understanding of the issue, not "read this essay on the difficult subject of numerical analysis" I'd actually go further and say that it's not just beginners, but most programmers. If you need to worry about the details of floating point numbers then you are doing something quite unusual.
|
|
#
?
Jun 11, 2017 22:00
|
|
|
On the other hand, if those tiny floating point inaccuracies are leading to your results being 10% out, you definitely need to know what's up
|
|
#
?
Jun 11, 2017 22:08
|
|
|
Malcolm XML posted:Perfectly fine if your algorithms are insensitive to things like that (i use floats in quant finance optimization models) but try simulating a chaotic system and watch as floats are completely useless Umm I simulate chaotic systems... what should I be using if not floats?
|
|
#
?
Jun 11, 2017 22:23
|
|
|
Nippashish posted:
Well, I was not worrying about them until I found some behaviour I couldn't figure out  I'll read up op on the subject to see what kind of pitfalls I have to look out for. Nippashish posted:I'd actually go further and say that it's not just beginners, but most programmers. If you need to worry about the details of floating point numbers then you are doing something quite unusual. I thought I had a pretty easy use case: figuring out how long and on which intervals my script should run. Is that an unusual way to calculate it, and if so what would a regular way of solving that be?
|
|
#
?
Jun 11, 2017 22:24
|
|
|
Nippashish posted:I'd actually go further and say that it's not just beginners, but most programmers. If you need to worry about the details of floating point numbers then you are doing something quite unusual. Nah, floating point correctness and approximations come up all the time in scientific computing, especially if you have unit tests. Scientific computing isn't "quite unusual"; it's the primary driver for the biggest floating-point computers in the world: https://www.top500.org/lists/2016/11/
|
|
#
?
Jun 11, 2017 22:28
|
|
|
SurgicalOntologist posted:Umm I simulate chaotic systems... what should I be using if not floats? a complicated algorithm that compensates for the fact that the system is very sensitive to accumulated approximation error and a lot of whiskey when b/c even then long term behavior is sketchy Nippashish posted:I'd actually go further and say that it's not just beginners, but most programmers. If you need to worry about the details of floating point numbers then you are doing something quite unusual. any programmer that's using floats for numerical work needs to know when and how they work so when they gently caress up they aren't surprised. math.fsum is in the standard library for a reason Better to learn this now than with actual repercussions on the line QuarkJets posted:This is the level of worrying that Nippashish said a newbie shouldn't be at, and he was 100% right. "Why is this single line of floating-point arithmetic giving me a funny-looking answer" requires a very basic understanding of the issue, not "read this essay on the difficult subject of numerical analysis" Newbies shouldn't worry about how to use lathes, just have them lose a finger or break a few thousand parts instead of having them learn how to use their tools
|
|
#
?
Jun 11, 2017 22:34
|
|
|
pmchem posted:Nah, floating point correctness and approximations come up all the time in scientific computing, especially if you have unit tests. Scientific computing isn't "quite unusual"; it's the primary driver for the biggest floating-point computers in the world: https://www.top500.org/lists/2016/11/ I'm well aware of this, pushing floats around is my day job. Most of the careful thinking about stability is done in at the level of BLAS and friends and very few people actually write that kind of code (the few that do do need to care about the details of floats). When you're writing scientific software your reasoning about stability typically happens at a higher level of abstraction than the properties of floats. Instead you think about the properties of matrix factorizations and DE solvers and other high level operations (which inherit their properties from floats, but not in ways you control). Reasoning at the boundary is dominated by "things can get weird with really big or really small numbers", which I why I suggested that as a mental model earlier.
|
|
#
?
Jun 11, 2017 22:47
|
|
|
Malcolm XML posted:Newbies shouldn't worry about how to use lathes, just have them lose a finger or break a few thousand parts instead of having them learn how to use their tools Is writing bad analogies required for numerical analysis, now? You're seriously fighting for the position that says "a person new to programming has to have a solid understanding of the difficult topic of numerical analysis before writing a program that uses floats". Is that really the hill that you want to die on?
|
|
#
?
Jun 11, 2017 23:06
|
|
|
I am on that side. That said, I learned how to program in a numerical analysis class, so I recognize I am a bit of an outlier here.
|
|
|
#
?
Jun 11, 2017 23:20
|
|
|
LochNessMonster posted:I thought I had a pretty easy use case: figuring out how long and on which intervals my script should run. There's nothing wrong with what you were doing (and the results you were printing were still correct to a very high accuracy) - you just need to be aware that you might get some awkward numbers. Really what you needed to do was format your print output to a fixed number of decimal places, that's all, same as you would if you were doing some awkward division where you knew the answer would be like 7.4042780206425627001183 For other stuff, it really depends. If you're doing a test and you want to compare a result to an expected answer, and that answer is the result of a particular calculation (like a ratio), you can perform that calculation and compare to that instead of the literal result you expect. That way your expected answer will have the same error as the actual answer (because they represent the same floating point calculation) and they'll be equal If your expected answer is a little harder to express, and you want to just say what the value you expect is, accept that the actual answer might not be exactly what you're looking for, but within a certain tolerance - like a real-world measurement. Usually in that case you'd specify an acceptable error amount and make sure the difference between the expected and actual answers doesn't exceed that There's a python section on this stuff and it gives you some handy ways to deal with all these things
|
|
#
?
Jun 11, 2017 23:35
|
|
|
Nippashish posted:I'm well aware of this, pushing floats around is my day job. Cool, mine too. You ignored the point about unit tests in my post. If you write scientific software that has numerical tests of floating point results, then as I'm sure you're aware, when you change (compiler / compiler flags / arch / libraries), the float results may slightly change. That's a pretty common occurrence, and is a reason why anyone involved with scientific software should be aware of floats. Not unusual at all.
|
|
#
?
Jun 11, 2017 23:40
|
|
|
Malcolm XML posted:any programmer that's using floats for numerical work needs to know when and how they work so when they gently caress up they aren't surprised. math.fsum is in the standard library for a reason pmchem posted:anyone involved with scientific software should be aware of floats I would think the entire neural network community largely ignores floating point issues, what with them running near-exclusively on single precision. Malcolm XML posted:Actually this is the worst possible takeaway because Numerical analysis is a) hard and b) for even most practical purposes requires an understanding of when it'll fail on you. Catastrophic cancellation and loss of precision can lead to cases where your component will fail and fail hard. Unless you don't want to engineer robustly, sure you can ignore it. I have run into many cases where poor or no understanding of float approximations have lead to pernicious bugs in production systems costing lots of money
|
|
#
?
Jun 11, 2017 23:48
|
|
|
pmchem posted:Cool, mine too. You ignored the point about unit tests in my post. If you write scientific software that has numerical tests of floating point results, then as I'm sure you're aware, when you change (compiler / compiler flags / arch / libraries), the float results may slightly change. That's a pretty common occurrence, and is a reason why anyone involved with scientific software should be aware of floats. Not unusual at all. Yes sorry I forgot to respond to the unit test part. The the version of this that I see most often is that the results of parallel reductions change because floats aren't associative so results can change even between multiple runs of the same binary on the same hardware. I deal with this in exactly the same way I deal with any other float comparison. Anyway the point I'm making isn't that people should ignore that floats are not a perfect representation of real numbers. My point is that although the precise behavior of floats is quite complicated, there are very simple ways of thinking about them that explain their behavior at a level that is appropriate for the vast majority of people, including most of the people writing scientific software.
|
|
#
?
Jun 12, 2017 00:01
|
|
|
QuarkJets posted:Is writing bad analogies required for numerical analysis, now? well it keeps me in a job, so pragmatically no but yes you should understand your tools before you use them, maybe read the fine manual as well Cingulate posted:I would think the entire neural network community largely ignores floating point issues, what with them running near-exclusively on single precision. They have run the analysis and it doesn't matter to them. OP poster was running into self admitted errors and had no idea what was going on, because they had no idea what they were doing. quote:
it's not as fun
|
|
#
?
Jun 12, 2017 00:53
|
|
|
Nippashish posted:
You're wrong and should feel wrong. If you're advocating writing scientific software or numerical software without understanding what's going on, please let me know what you make so I can never use it Not everyone programs fart apps exclusively so its worth knowing when you can and when you can't ignore abstractions OP go read http://floating-point-gui.de/basic/
|
|
#
?
Jun 12, 2017 01:02
|
|
|
Are there any descent python graphing packages that do not require pip or c++ packages? I'm working with a manufacturing software packages that uses python 2.75. While it has some graphing options, I would like more robust options. In order for me to be able to use the package, the only thing I can do dump the library into the python libraries folder
|
|
#
?
Jun 12, 2017 01:52
|
|
|
Malcolm XML posted:well it keeps me in a job, so pragmatically no Malcolm XML posted:You're wrong and should feel wrong. If you're advocating writing scientific software or numerical software without understanding what's going on, please let me know what you make so I can never use it We're not talking about scientific software or numerical software. You keep saying that a careful analysis of numerical precision is necessary before writing any code at all if there's a chance that it will have a double variable somewhere in it. You're basically preaching premature optimization of anything more complex than a Hello World program
|
|
#
?
Jun 12, 2017 02:05
|
|
|
QuarkJets posted:We're not talking about scientific software or numerical software. You keep saying that a careful analysis of numerical precision is necessary before writing any code at all if there's a chance that it will have a double variable somewhere in it. You're basically preaching premature optimization of anything more complex than a Hello World program
|
|
#
?
Jun 12, 2017 02:26
|
|
|
Malcolm XML posted:a complicated algorithm that compensates for the fact that the system is very sensitive to accumulated approximation error and a lot of whiskey when b/c even then long term behavior is sketchy But yeah, dealing with chaotic systems, good luck if you need to make specific predictions. Investigating the qualitative behavior of the system is more useful anyway. In any case, you started by saying "floats are completely useless" for simulating complex systems, which I still don't understand--should I not be using floats? Or are just making the point that chaos magnifies errors? In which case, I have errors of much higher orders of magnitude I'm already concerned about.
|
|
#
?
Jun 12, 2017 02:49
|
|
|
jusion posted:I think you have to be intellectually dishonest to think something like 0.1 + 0.2 == 0.3 would be rare in any sort of program. If you want to program you have to understand boolean logic. If you want to program you have to at least understand the basics of floating point math and its pitfalls. This isn't really an extreme position. Maybe Malcolm was a little mean, but come on... No no no, I'm arguing for an understanding of the basics. Malcolm is arguing for a far more extreme position than that
|
|
#
?
Jun 12, 2017 02:54
|
|
|
QuarkJets posted:No no no, I'm arguing for an understanding of the basics. Malcolm is arguing for a far more extreme position than that
|
|
#
?
Jun 12, 2017 03:00
|
|
|
jusion posted:Ok - I think you may be talking past each other then, because his link (http://floating-point-gui.de/basic/) is not that. QuarkJets posted:Is writing bad analogies required for numerical analysis, now? Malcolm XML posted:well it keeps me in a job, so pragmatically no That page is "here are the basics", Malcolm's posts are saying that you actually need much more than a basic understanding. I think his intention was that a beginner should read all of the pages on that web site, not just the page that was linked.
|
|
#
?
Jun 12, 2017 03:08
|
|
|
Cingulate posted:I would think the entire neural network community largely ignores floating point issues, what with them running near-exclusively on single precision. I dunno. I mean, a casual google search reveals several papers and posts on the topic; their fp issues are just different from most others'. They prefer to use as little precision as possible to get their useful result in order to do things like reduce file size issues for trained models: https://www.tensorflow.org/performance/quantization
|
|
#
?
Jun 12, 2017 03:09
|
|
|
yup, google's TPUs use quantization to improve inference performance so you can't say that ML practitioners can ignore FP issues
|
|
#
?
Jun 12, 2017 03:41
|
|
|
baka kaba posted:There's nothing wrong with what you were doing (and the results you were printing were still correct to a very high accuracy) - you just need to be aware that you might get some awkward numbers. Really what you needed to do was format your print output to a fixed number of decimal places, that's all, same as you would if you were doing some awkward division where you knew the answer would be like 7.4042780206425627001183 Just read some stuff on the Floating-Point Guide that was linked earlier as well as the official docs and it makes some more sense now. At leas now I know to round my floats when using them for basic calculations! (and to use decimals if I want to do something with currency)
|
|
#
?
Jun 12, 2017 08:18
|
|
|
Malcolm XML posted:it's not as fun I don't mean this to disparage you or as a counterargument to your position - which it is not - but as advice from someone who should've been given that lesson much earlier themselves, too. shrike82 posted:yup, google's TPUs use quantization to improve inference performance so you can't say that ML practitioners can ignore FP issues (I'm obviously not saying these people are ignorant of the issues, just that the issues are largely irrelevant, and being ignored.)
|
|
#
?
Jun 12, 2017 10:54
|
|
|
|
| # ? May 15, 2024 04:22 |
|
|
An ML practitioner or academic could decide to ignore precision based on their needs, but I'd find it hard to argue that one can make an informed decision without understanding the basics of floating point arithmetic. No argument about Malcolm acting like a douchebag but hey, being a coder in the finance industry sucks so it's not surprising that people like him behave that way.
|
|
#
?
Jun 12, 2017 11:06
|
|