|
We changed that because people started writing tests in Google that explicitly relied on default map ordering and then complained when it was modified by the runtime in following releases for their production binaries. Also, since the Go spec does not define the ordering, the two implementations of the compiler, gc and gccgo, had two different default orderings. GCC picks a default the last time I worked on it, at least. So, at the time, within Google, we had people building ppc and arm binaries with gccgo where their ordering assumptions were now wrong. Overall, we never found the real intention behind relying on a consistent default ordering. If you wanted a particular ordering, it seems like you would necessarily need to sort the data in some way. Anyways, it wasn't some random decision and I think Russ agreed with your reasoning around go1.3, before we noticed how misused this "feature" was.
|
# ? Jul 6, 2017 20:44 |
|
|
# ? Jun 5, 2024 09:09 |
|
I don't know Go, but maybe if iterating the elements of a Hashtable doesn't makes sense unless they are sorted first, then it should only be possible if you call a .sort () method to return a (k*v) list? Not that I can think of a programming language that does this though.
|
# ? Jul 6, 2017 21:02 |
|
TheBlackVegetable posted:I don't know Go, but maybe if iterating the elements of a Hashtable doesn't makes sense unless they are sorted first, then it should only be possible if you call a .sort () method to return a (k*v) list? Would this mean that maps, in general, would not be able to be iterable unless they were sorted? It's not that iterating elements of a hashtable doesn't make sense, but relying on a particular order without specifying it directly in code (with a comparator, for example) is probably a bug or a bad assumption.
|
# ? Jul 6, 2017 21:24 |
|
Coffee Mugshot posted:Would this mean that maps, in general, would not be able to be iterable unless they were sorted? It's not that iterating elements of a hashtable doesn't make sense, but relying on a particular order without specifying it directly in code (with a comparator, for example) is probably a bug or a bad assumption. for k, v := range mahMap using sort.Ints seems pretty readable. orderby instead of using, if you feel like it. IDK what all the reserved words are in Go
|
# ? Jul 6, 2017 21:36 |
Coffee Mugshot posted:Would this mean that maps, in general, would not be able to be iterable unless they were sorted? It's not that iterating elements of a hashtable doesn't make sense, but relying on a particular order without specifying it directly in code (with a comparator, for example) is probably a bug or a bad assumption. You can iterate through a map no problem by simply going through all the keys. Since keys are unique, there's no danger of missing or repeating a key unless you modify the map mid-way. There are map implementations that preserve insert order; such as the one that Python 3.6+ uses.. This is both predictable and occasionally useful. If there's no runtime cost, why not have a default that makes sense? Note: in Python 3.6, this is currently an implementation detail and your code should use collections.OrderedDict to specify your intent. Eela6 fucked around with this message at 21:46 on Jul 6, 2017 |
|
# ? Jul 6, 2017 21:43 |
|
ullerrm posted:It's actually even worse than that -- it's "programmers are uniformly stupid and will blown their own arm off given the chance, so let's make a language even more stupid as the programmers that makes it hard as possible to blow off their arms." For example: If you iterate over the keys in a map, what order do they come in? In most languages, it's either "ordered ascending" (because the map is backed by a red-black tree) or "implementation-defined" which means unknown but consistent (because the map is backed by a hash table). That's fascinating, are there more of these that you could summarize? I tried looking around for articles complaining about other weird Go design features but could only find things there were like "Go has pointers, I have to google for Golang instead of Go, a bloo bloo bloo "
|
# ? Jul 6, 2017 21:46 |
|
Coffee Mugshot posted:Would this mean that maps, in general, would not be able to be iterable unless they were sorted? It's not that iterating elements of a hashtable doesn't make sense, but relying on a particular order without specifying it directly in code (with a comparator, for example) is probably a bug or a bad assumption. It would. Maybe couple it with an .unsortedkeys method I dunno. Basically, make invalid code unrepresentable. Edit: or at least explicit. TheBlackVegetable fucked around with this message at 21:50 on Jul 6, 2017 |
# ? Jul 6, 2017 21:47 |
|
TheBlackVegetable posted:I don't know Go, but maybe if iterating the elements of a Hashtable doesn't makes sense unless they are sorted first, then it should only be possible if you call a .sort () method to return a (k*v) list? Iterating a hash does make sense without ordering semantics as long as whatever operation you're doing isn't interested in the order. Ruby does exactly what you suggested when ordering is requested: code:
|
# ? Jul 6, 2017 21:58 |
QuarkJets posted:That's fascinating, are there more of these that you could summarize? I tried looking around for articles complaining about other weird Go design features but could only find things there were like "Go has pointers, I have to google for Golang instead of Go, a bloo bloo bloo " Well, suppose I wanted to iterate through a map of an arbitrary type, sorted by the map value. Suppose it's a map called "teams": [Player] -> [Team]. We want to iterate through sorted by team color, then name as a fallback. Here it is in python: Python code:
code:
Eela6 fucked around with this message at 22:17 on Jul 6, 2017 |
|
# ? Jul 6, 2017 22:13 |
|
Eela6 posted:If I want to change how I sort, I have to create an entire new type and do the whole song and dance. There's sort.Slice now, which takes a comparison function. If you're really lucky, it'll actually be available on your target system! (It is not available on our target system, which means we have sortableInt32, sortableInt64, etc. types in our codebase just so we can sort a goddamn list of numbers)
|
# ? Jul 6, 2017 22:19 |
TooMuchAbstraction posted:(It is not available on our target system, which means we have sortableInt32, sortableInt64, etc. types in our codebase just so we can sort a goddamn list of numbers) there you have it, folks
|
|
# ? Jul 6, 2017 22:21 |
|
Coffee Mugshot posted:We changed that because people started writing tests in Google that explicitly relied on default map ordering and then complained when it was modified by the runtime in following releases for their production binaries. Also, since the Go spec does not define the ordering, the two implementations of the compiler, gc and gccgo, had two different default orderings. GCC picks a default the last time I worked on it, at least. So, at the time, within Google, we had people building ppc and arm binaries with gccgo where their ordering assumptions were now wrong. Overall, we never found the real intention behind relying on a consistent default ordering. If you wanted a particular ordering, it seems like you would necessarily need to sort the data in some way. Anyways, it wasn't some random decision and I think Russ agreed with your reasoning around go1.3, before we noticed how misused this "feature" was. If you rely on the iteration order of an explicitly unordered data structure (like a java HashMap) you should probably be shot into space. I'm not a huge fan of go, but randomizing the order of maps is awesome.
|
# ? Jul 7, 2017 00:25 |
|
That seems OK to me as well. If iteration order is not defined or guaranteed, then randomizing the order is perfectly acceptable. I suppose randomization does have a performance cost, but would it matter on a practical level?
|
# ? Jul 7, 2017 03:40 |
|
FWIW, in Java you get to choose between a HashMap (which has no defined iteration order), and a LinkedHashMap which has some additional bookkeeping overhead but iterates elements in the order they were inserted. Of course, even if it wasn't part of the library, you could implement that sort of utility yourself if you really needed it. If only Go had the mechanisms that would allow people to implement their own collections libraries for niche things that aren't built-in to the language.
|
# ? Jul 7, 2017 03:40 |
|
Coffee Mugshot posted:We changed that because people started writing tests in Google that explicitly relied on default map ordering Coffee Mugshot posted:Overall, we never found the real intention behind relying on a consistent default ordering. Coffee Mugshot posted:If you wanted a particular ordering, it seems like you would necessarily need to sort the data in some way.
|
# ? Jul 7, 2017 04:47 |
|
boo_radley posted:This seems like a bonkers thing to do. It's always been my assumption that in hashes, the ordering and storage of elements in part of the black box and subject to the whims of designers, compilers and etc. etc. etc. all the time. It's very easy to make a test that accidentally relies on ordering; just have a function that outputs a list of items, in the order in which it iterates over the contents of a hashmap, and then compare that list to your test value. Surprise, your test now relies on the ordering of keys in a hashmap, even though you don't really care about that ordering, only that the two lists have the same items in them. Literally every time Go has broken my tests due to randomized hash iteration order, it's been due to that type of thing. The actual behavior was just fine, it just wasn't consistently verifiable, and it was easier to say "gently caress it, we sort the drat keys" than it was to write a set-comparison function that takes two lists of <whatevers>. I mean, the tests are broken either way, you shouldn't rely on ordering of hash iteration.
|
# ? Jul 7, 2017 05:02 |
|
Yeah, I get the motivation behind the library enforcing, "No really, the iteration order is effectively random!" but I'm wary of a language where the designers come up with that as a solution to the problem of programmers relying on undefined behavior. (Now that the behavior is defined, it's documented, right?)
|
# ? Jul 7, 2017 05:08 |
|
CPColin posted:(Now that the behavior is defined, it's documented, right?) ...the behavior is still not defined - the ordering can be anything.
|
# ? Jul 7, 2017 06:06 |
|
Here's the question that leads to further horror: is the ordering's randomness of cryptographic quality?
|
# ? Jul 7, 2017 06:07 |
|
Avoiding delivering any behaviors that people might rely upon is sensible practice, despite the howling of some noobs ITC. Even with an ordered map holding the data I've had occasion to return elements in reverse order, just to keep API consumers on edge.
|
# ? Jul 7, 2017 06:27 |
|
CPColin posted:Yeah, I get the motivation behind the library enforcing, "No really, the iteration order is effectively random!" but I'm wary of a language where the designers come up with that as a solution to the problem of programmers relying on undefined behavior. (Now that the behavior is defined, it's documented, right?) I mean I would almost prefer this sort of thing in debug/test builds because it would expose unintentional reliance on unspecified behavior, but being unable to turn it off in release builds seems like a big red flag.
|
# ? Jul 7, 2017 06:28 |
|
TooMuchAbstraction posted:It's very easy to make a test that accidentally relies on ordering; just have a function that outputs a list of items, in the order in which it iterates over the contents of a hashmap, and then compare that list to your test value. Surprise, your test now relies on the ordering of keys in a hashmap, even though you don't really care about that ordering, only that the two lists have the same items in them. Literally every time Go has broken my tests due to randomized hash iteration order, it's been due to that type of thing. The actual behavior was just fine, it just wasn't consistently verifiable, and it was easier to say "gently caress it, we sort the drat keys" than it was to write a set-comparison function that takes two lists of <whatevers>. Why would I use a language without a set type where I can make that mistake in the first place? Why should I have to write code to sort a hash map when I want to see if two sets are equal? How is that simple?
|
# ? Jul 7, 2017 06:35 |
|
TooMuchAbstraction posted:It's very easy to make a test that accidentally relies on ordering; just have a function that outputs a list of items, in the order in which it iterates over the contents of a hashmap, and then compare that list to your test value. Surprise, your test now relies on the ordering of keys in a hashmap, even though you don't really care about that ordering, only that the two lists have the same items in them. Literally every time Go has broken my tests due to randomized hash iteration order, it's been due to that type of thing. The actual behavior was just fine, it just wasn't consistently verifiable, and it was easier to say "gently caress it, we sort the drat keys" than it was to write a set-comparison function that takes two lists of <whatevers>. But it's a hash map; I'm assuming it's just as easy to iterate through your list of test values and check whether each one is also a key in the hashmap as it is to check whether the list of keys returned by the hashmap is the same as your test list e: Are the hashmap keys not also a set? QuarkJets fucked around with this message at 06:40 on Jul 7, 2017 |
# ? Jul 7, 2017 06:36 |
|
QuarkJets posted:But it's a hash map; I'm assuming it's just as easy to iterate through your list of test values and check whether each one is also in the hashmap as it is to check whether the list of items returned by the hashmap is the same as your test list Yeah, surely that would be faster than sorting and then comparing... edit: hmm, not necessarily. I am dumb.
|
# ? Jul 7, 2017 06:38 |
|
ulmont posted:...the behavior is still not defined Sure it is: ulmont posted:the ordering can be anything. (What I'm suggesting is that if an implementation is purposefully avoiding having a consistent iteration order, it might help to state that in the documentation.) CPColin fucked around with this message at 06:49 on Jul 7, 2017 |
# ? Jul 7, 2017 06:46 |
|
Bongo Bill posted:Here's the question that leads to further horror: is the ordering's randomness of cryptographic quality? You can't fix
|
# ? Jul 7, 2017 07:24 |
|
What's the overhead of having to generate a random number without data races for iteration? What's the overhead of increasing memory cache misses on iteration? I think it's an okay philosophy for the standard library to incur performance penalties to make it easier to do the right thing, but it becomes a dumb philosophy when you can't roll your own without the performance penalties.
|
# ? Jul 7, 2017 09:38 |
|
The .NET System.String type explicitly states that a string's hash code is not guaranteed to be stable and should not be relied on or persisted anywhere, etc. The docs are filled with warnings about this. Relying on a stable string hash code got to be such a problem within Microsoft that they took a drastic approach to preventing MS devs from trusting it: code:
I won't award points to anyone who can guess why I know this.
|
# ? Jul 7, 2017 12:32 |
|
Enforcing a random order - at least in debug - so that people are likely to notice when they've written broken code seems like such an obviously good idea that I'm surprised it's proving controversial in this thread. I'm not sure what the alternatives are supposed to be, besides "don't have implementation-defined behaviour in the language" (unnecessarily constricting) or "programmers should just be good enough at their jobs that they don't need that" (good luck with that).
|
# ? Jul 7, 2017 13:49 |
|
Alternatives to the current design problem include:
I think that this is a good example of Go boxing itself into a language-design corner in pursuit of Rob Pike's weird definition of "simplicity".
|
# ? Jul 7, 2017 14:08 |
|
Internet Janitor posted:Alternatives to the current design problem include: otoh forcing everyone to use an ordered map unnecessarily hurts performance for most use cases
|
# ? Jul 7, 2017 15:51 |
|
That's why you have more than one type of map, my man.
|
# ? Jul 7, 2017 16:01 |
|
I just realized that this reminds me of picking a random initial partition location in quicksort, which can be good practice.TooMuchAbstraction posted:It's very easy to make a test that accidentally relies on ordering; just have a function that outputs a list of items, in the order in which it iterates over the contents of a hashmap, and then compare that list to your test value. Surprise, your test now relies on the ordering of keys in a hashmap, even though you don't really care about that ordering, only that the two lists have the same items in them. Literally every time Go has broken my tests due to randomized hash iteration order, it's been due to that type of thing. The actual behavior was just fine, it just wasn't consistently verifiable, and it was easier to say "gently caress it, we sort the drat keys" than it was to write a set-comparison function that takes two lists of <whatevers>. But isn't this only a problem because Go doesn't have an easy way to check for list membership? comedyblissoption posted:What's the overhead of having to generate a random number without data races for iteration? Sounds like it only picks a random initial bucket when the map is created, which wouldn't be a big deal. https://github.com/golang/go/issues/5362#issuecomment-66078620
|
# ? Jul 7, 2017 16:16 |
|
Hammerite posted:"programmers should just be good enough at their jobs that they don't need that" (good luck with that). I kinda have a hard time understanding how Googlers are at the absolute pinnacle of the profession, top notch in every way, incredibly high bar for any applicant, except their in-house language keeps having to file down anything sharper than a marble. FWIW I like all of IJ's solutions, especially sets if the root cause is testers don't have an easy way to check lists.
|
# ? Jul 7, 2017 16:36 |
|
QuarkJets posted:But it's a hash map; I'm assuming it's just as easy to iterate through your list of test values and check whether each one is also a key in the hashmap as it is to check whether the list of keys returned by the hashmap is the same as your test list You can fake a set in Go by using map[whatever]bool, so long as you only care about membership (and not intersection/unity/set equality/etc.). But note that I said the code was outputting a list of elements, that happened to be ordered based on the order in which the code iterated over a map. So basically our options were either a) sort the map keys, to enforce a consistent order in the output list, or b) write a comparison function that takes two input lists, sorts them (or converts them to maps, etc.), then iterates over both of them to ensure that every element in each is also in the other. It happens that this pattern shows up many times in our codebase, and that the keys are always int32s (they're database primary keys, and yes, 4 billion of them should be enough for many, many years), so we just implemented SortableInt32. We still have to remember to sort the keys every time we iterate over a map that's feeding into a result list, lest our tests break, though.
|
# ? Jul 7, 2017 17:05 |
|
TooMuchAbstraction posted:You can fake a set in Go by using map[whatever]bool, so long as you only care about membership (and not intersection/unity/set equality/etc.). But note that I said the code was outputting a list of elements, that happened to be ordered based on the order in which the code iterated over a map. So basically our options were either a) sort the map keys, to enforce a consistent order in the output list, or b) write a comparison function that takes two input lists, sorts them (or converts them to maps, etc.), then iterates over both of them to ensure that every element in each is also in the other. Why are you using this language if it's painful to do anything in?
|
# ? Jul 7, 2017 17:17 |
Go is a terrible language but a decent back-end environment. Easy deployment and compilation to small native code binaries is often worth a lot of hassle. It's in many ways the opposite of Python, which is an excellent language but often hellish to deploy. That's not quite as easy to point and laugh at in this thread, however. Despite all my complaints about Go, I'd rather work in it than, say, C, C++, or Javascript. The problem with Go is that it's so close to being a truly excellent language, and then you run right into For example, the entire math library. Why the gently caress is doing basic math such a nightmare in go. It's just easy to complain about because so much of this stuff would be trivial to fix, but it would ruin rob pike's dumb vision. If go were just bad it would be easy to ignore. It's the mixture of good and bad that makes it infuriating. Eela6 fucked around with this message at 17:37 on Jul 7, 2017 |
|
# ? Jul 7, 2017 17:27 |
|
I want to, ever so slowly, strangle to death the bastard that decided to not only implement a Repository pattern for this jank-rear end website but hid it under *3* interfaces. Shift-F12, Shift-F12, Shift-F12, gently caress YOU!
|
# ? Jul 7, 2017 17:42 |
|
Wait. Go doesn't have any built-in Sets?
|
# ? Jul 7, 2017 17:58 |
|
|
# ? Jun 5, 2024 09:09 |
|
HardDiskD posted:Wait. Go doesn't have any built-in Sets? Its built-in data structures are array, slice, map, and struct.
|
# ? Jul 7, 2017 18:05 |