Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
NickPlowswell
Sep 14, 2014

The exoplanet has failed :qq:

Azazel posted:

When you start importing a handful of 3rd party libraries and share those libraries among multiple developers/projects "go get" alone becomes very unsatisfying.

See, I always thought people were more worried about not having proper versioning though this does make sense.

Adbot
ADBOT LOVES YOU

Bozart
Oct 28, 2006

Give me the finger.
Does Docker alleviate some of that problem? I've never used it in a professional environment.

SavageMessiah
Jan 28, 2009

Emotionally drained and spookified

Toilet Rascal
Docker is a containerization tool for linux, how on earth would it help with this problem? The only thing that really alleviates it is tools like goven that help you seamlessly manage local copies of code from remote sources.

I understand what they were going for but I think the urls in imports things is definitely one of the weaker parts of go.

Bozart
Oct 28, 2006

Give me the finger.

SavageMessiah posted:

Docker is a containerization tool for linux, how on earth would it help with this problem?

It works in more environments than just linux. You could add 3rd party packages to the docker image, and then use that image for dev work. Then all of the devs are using the same version of each of the packages, which may alleviate the problem he described of multiple devs coordinating multiple packages. The downside may be as Look Around You mentioned being "frowned upon" but I don't know one way or the other, because as I said, I haven't used it professionally.

Also, they aren't urls in the imports, they are file paths relative to $GOROOT. They are only interpreted as urls by the go tool.

Pham Nuwen
Oct 30, 2010



Bozart posted:

It works in more environments than just linux. You could add 3rd party packages to the docker image, and then use that image for dev work. Then all of the devs are using the same version of each of the packages, which may alleviate the problem he described of multiple devs coordinating multiple packages. The downside may be as Look Around You mentioned being "frowned upon" but I don't know one way or the other, because as I said, I haven't used it professionally.

This sounds like the equivalent of distributing a full Python installation with your program because of Python's version hell. (This is a thing people actually do)

Bozart
Oct 28, 2006

Give me the finger.

Pham Nuwen posted:

This sounds like the equivalent of distributing a full Python installation with your program because of Python's version hell. (This is a thing people actually do)

To distribute it you would just distribute the compiled binary?

Look Around You
Jan 19, 2009

Bozart posted:

Also, they aren't urls in the imports, they are file paths relative to $GOROOT. They are only interpreted as urls by the go tool.

Yes, which devs then rely on the tool to use to fetch the code from the url and then compile it to use as a library that is placed in an identicaly named sub-directory in $GOPATH. I mean, to the compiler it's treated as just another plain import path, but that's not why it's bad. It's bad because it's treated as an actual content source by the built in dependency manager, and it's way too fragile and dumb of an idea.

Plorkyeran
Mar 22, 2007

To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

NickPlowswell posted:

See, I always thought people were more worried about not having proper versioning though this does make sense.

Seems like the obvious solution to both the versioning problem and the unreliable third-party repo problem is to just fork all of the repos you depend on. Doesn't really let you have different versions of deps on different branches (without a fork per branch...), but if you control the repo you're pulling code from then you can just always have master pointing at the appropriate release.

Ferg
May 6, 2007

Lipstick Apathy

Plorkyeran posted:

Seems like the obvious solution to both the versioning problem and the unreliable third-party repo problem is to just fork all of the repos you depend on. Doesn't really let you have different versions of deps on different branches (without a fork per branch...), but if you control the repo you're pulling code from then you can just always have master pointing at the appropriate release.

This is what we do at my shop. It's never posed a problem, and you always have the original repo if you need to pull any changes/fixes that made their way into the upstream.

Brecht
Nov 7, 2009
Forking the repos you depend on is called vendoring, and it works fine. Great, even.

http://github.com/tools/godep

Azazel
Jun 6, 2001
I bitch slap for a living - you want some?

Brecht posted:

Forking the repos you depend on is called vendoring, and it works fine. Great, even.

http://github.com/tools/godep

Vendoring is what the core Go devs at Google recommend (and is what they do internally). I do it myself, definitely the best solution given what we have.

For the record here is a list of various solutions: https://code.google.com/p/go-wiki/wiki/PackageManagementTools

pram
Jun 10, 2001
I'd agree 'get' feels pretty fragile considering anyone can just delete something you depend on. However i'm not sure something like pip/cpan/npm would be better, those tools turn into something else you have to constantly janitor.

I actually enjoy how simple it is. You know it's sitting there in src, you don't need to go hunt it down in some obscure directory 10 levels deep in lib.

Linear Zoetrope
Nov 28, 2011

A hero must cook
Oh... oh. A slice's capacity upon reallocation by append is not uniform between compiler targets. I kind of knew this, which is why I never rely on underlying buffers being the same or different when it comes to append, but actually seeing it in action is kind of surreal.

code:
a := []int{}
a = append(a, 1)
b := append(a, 1)
Do a and b share the same backing array? Nobody knows! It depends how the compiler decides to grow slices at capacity. (It comes down to the question: at the line where we declare b, is the capacity of a 1 or larger than 1?)

This isn't theoretical, the answer actually changes between the playground and my machine.

Linear Zoetrope fucked around with this message at 12:23 on Dec 27, 2014

mdxi
Mar 13, 2006

to JERK OFF is to be close to GOD... only with SPURTING

Here's a thing I wrote: http://firepear.net/adminsock/

It's a mostly-automatic control/admin plugin for server software.

Bozart
Oct 28, 2006

Give me the finger.
Uh, aren't you supposed to say it is simple, lightweight, or minimalist?

edit:

More seriously, I don't know enough about the domain to give advice in that regard, but you could run golint, which would pick up:
code:
	for n := 1; true; n++ { ... }
can just be
code:
	for n := 1;; n++ { ... }
It is also a bit odd to me how the sockAccept function has to handle some of the wait group and then some of it is handled in connHandler, and some of it in sockWatchDog, and some in New. I would be quite worried that it could have edge cases where after calling quit, it either hangs or can try to send to a closed channel. There's got to be a way to consolidate.

Bozart fucked around with this message at 05:27 on Dec 30, 2014

mdxi
Mar 13, 2006

to JERK OFF is to be close to GOD... only with SPURTING

Bozart posted:

It is also a bit odd to me how the sockAccept function has to handle some of the wait group and then some of it is handled in connHandler, and some of it in sockWatchDog, and some in New. I would be quite worried that it could have edge cases where after calling quit, it either hangs or can try to send to a closed channel. There's got to be a way to consolidate.

I'm definitely still finding my sea-legs where writing concurrent code is concerned. Instead of passing around another synchronization channel to track completion of goroutines (so that some single point in the code can decrement the waitgroup's counter), I chose to pass around pointers to the waitgroup and have everything issue a deferred call to sync.WaitGroup.Done() as soon as it starts executing. This way, as each goroutine returns, the waitgroup's counter is decremented.

I feel like it's six of one, half-a-dozen of the other. Or is there some other option, which I'm not considering?

I'm still extremely wary of deadlocks and synchronization issues myself. But the adminsock test suite contains every "bad" thing I was able to come up with (so far), and makes hundreds of concurrent requests across up to 5 threads of execution. If you have any suggestions on things to try to shake out faulty behavior, I'd love to hear them :science:

Bozart
Oct 28, 2006

Give me the finger.
One thing you might want to do is get rid of lines sending on the q channel. Receives on a closed channel always succeed (with the zero value, but you're not using it...) so you could make q a chan struct{}, and then

code:
func (a *Adminsock) Quit() {
	close(a.q)
	a.w.Wait()
	close(a.Msgr)
}
Which has the advantage of quitting everything "at once" and taking up one less byte!.

The way I would change the code structure is to take the common parameters out sockAccept and connHandler, and put them into the Adminsock type. Then sockAccept would just be (*a Adminsock) sockAccept(), and connHandler would just be (*a Adminsock) connHandler(c net.Conn). It should also be possible to split out the wait group handling into a function like
code:
   (*a Adminsock) execute(c connHandler) { 
      a.Add(1)
      go func() {
          defer a.Done()
          a.connHandler(c)
      }()
   }
Also, instead of a wait group, it is possible to use just channels, although I am not sure if it would be faster. What you would do is have each iteration in sockAccept create a "done" channel, which is passed into the handler. When the handler finishes the done channel is closed. These done channels are sent into a channel of channels to an equivalent sockWatchdog, which waits to be told that the sockAccept function is no longer accepting, checks that all the done channels are closed, and then sends an all clear to quit.

As I said, I'm not sure that would be faster (although it might be under heavy load, because I believe there is less synchronization done when closing channels and blocking on them), but it is another approach, and it feels a bit more "go-like" to me. Do whatever works for you though.

mdxi
Mar 13, 2006

to JERK OFF is to be close to GOD... only with SPURTING

I'll mull over the channels-vs-WaitGroup stuff, but you're completely right about sockAccept and connHandler being part of the Adminsock methodset. It is basically an accident of rapid development that they aren't already. Thank you for pointing that out.

blorpy
Jan 5, 2005

Here's a q for the Go experts -- why is Go's type system brain dead?

Bozart
Oct 28, 2006

Give me the finger.
thats a good question to be honest

Linear Zoetrope
Nov 28, 2011

A hero must cook
It is because you are small

But really, I think it's because it's a type system designed by people largely comfortable in C as a primary language, with a focus on compiling extremely fast.

Karate Bastard
Jul 31, 2007

Soiled Meat
Huh. That's a really bad blog post.

e: I should probably be less obtuse. I don't like it because it reads like it's written by someone who cannot get beyond syntax and is sore about go not being java, without addressing the flaws of java that go actually sidesteps. The final sentence is pretty telling: "Someone might port Java to that runtime and reap all of the benefits", which I really really doubt.

Karate Bastard fucked around with this message at 08:35 on Feb 25, 2015

Linear Zoetrope
Nov 28, 2011

A hero must cook
It is, but it does kind of manage to capture some of the culture on golang-nuts.

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
I much prefer Brand X tbh

blorpy
Jan 5, 2005

Karate Bastard posted:

Huh. That's a really bad blog post.

e: I should probably be less obtuse. I don't like it because it reads like it's written by someone who cannot get beyond syntax and is sore about go not being java, without addressing the flaws of java that go actually sidesteps. The final sentence is pretty telling: "Someone might port Java to that runtime and reap all of the benefits", which I really really doubt.

It's a really good one b/c it adequately captures the feeling of writing go (and searching for go, &c)

sarehu
Apr 20, 2007

(call/cc call/cc)
It's a bad blog post because it should have had an editing pass by rjmccall to spice it up a bit.

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe
Ha ha ha, like I would ever publicly discuss other programming languages on the internet.

:ninja: :tinfoil: :sigh:

Karate Bastard
Jul 31, 2007

Soiled Meat

Markov Chain Chomp posted:

It's a really good one b/c it adequately captures the feeling of writing go (and searching for go, &c)

You think? Please go ahead and rip its head off then (in a constructive manner if you'd be so kind!) I've been looking for coherent, nuanced and competent criticism of go, and so far I've seen very little of that. People generally seem very happy with it, and whatever malcontents I've come across generally offer the kind of lame (misguided / misinformed / malicious?) whining like in that blog post. Like this one seriously argues verbosity being a bigger problem in go than in java, and rags on a few examples where a oneliner becomes five individually shorter lines, without discussing any of the rationale behind the design choices that put us in this situation. (Yes, ternary expressions are useful ~if used right~, and of course ~I~ use them right, even though ~everyone else~ that use them wrong ~use them wrong~, but they are ~no true Scotsmen~). When an author does things like that I have a big problem taking anything else they write seriously.

I might have gotten this from here, but I think Will Yager did a pretty good job of this in his blog post on Why Go Is Not Good.

sarehu
Apr 20, 2007

(call/cc call/cc)

Markov Chain Chomp posted:

Here's a q for the Go experts -- why is Go's type system brain dead?

My real theory is that the compiler is a complete disaster (I looked) so it would be hard to do. I feel like a hypocrite because I'm 1 for 1 when it comes to having your compiler be a disaster, but I can't imagine having to add generics to their existing implementation. I just remember seeing some struct with pages of fields in it, like out of some 80's horror movie. Now they've got it implemented in Go but apparently that was with a C to Go translation so it's probably still the same.

sarehu
Apr 20, 2007

(call/cc call/cc)
Also I enjoyed reading this a day or so ago: http://dtrace.org/blogs/wesolows/2014/12/29/golang-is-trash/

Linear Zoetrope
Nov 28, 2011

A hero must cook

sarehu posted:

My real theory is that the compiler is a complete disaster (I looked) so it would be hard to do. I feel like a hypocrite because I'm 1 for 1 when it comes to having your compiler be a disaster, but I can't imagine having to add generics to their existing implementation. I just remember seeing some struct with pages of fields in it, like out of some 80's horror movie. Now they've got it implemented in Go but apparently that was with a C to Go translation so it's probably still the same.

Nah, they've said frequently that they can do it. In fact, forks of the compiler with various generic implementations exist. The issue they always give is that they need an implementation that has:

1. Basically no impact on compile times (and requires no symbol table to parse)
2. A very small (at worse) impact at runtime
3. Is syntactically pleasing in Go

3, of course, gives then carte blanche to deny anything. I think it's 1 that's the sticking point. Their core design absolutely requires that they can compile a Google-sized codebase from scratch in about a minute (assuming no Cgo), but also runs quickly. It's maybe not an impossible situation, but it's extremely unlikely they'll find a good solution with those constraints. The best thing I came up with was a silly proposal I made for fun that tried to be as anally Go as possible (it's essentially "the make keyword for interfaces and functions"), which I think meets most constraints, and should be implementable as a simple extension of make.

Whatever though, I only need it for 3D Linear Algebra which I already resorted to using code generation for.

E: The Plan9 assembly language they use is, indeed, garbage though. Please do not ask me about trying to write SIMD in that thing. It has a lot of dumb quirks. For instance, did you know that you can write an assembly function that takes a value receiver, but NOT a pointer receiver?

It's true. You declare an assembly function like this:

code:
TEXT ·det3(SB),4,$0-12
This is an assembly function, with some flags that evaluate to 4 (not important), with 12 bytes of input/output stack. If you write the stub

code:
func det3(mat *Mat3) float32
The compiler will auto-bind the definition of det3 as that assembly routine. You can also do this on methods.

code:
TEXT ·Mat3·det(SB),4,$0-40
Which corresponds to

code:
func (mat Mat3) det() float32 
Note that the stack is now size 40, because a Mat3 is a 3x3 array of 32-bit float values so we have 36-input, 4-output = 40. But that's giant, if you benchmark it's legitimately slower by a factor of almost 5 to do the copy. So we want to implement it on a pointer.

code:
TEXT ·*Mat3·det(SB),4,$0-12
This won't work. So maybe it's some weird quirk, right? Like how Go uses the cdot · instead of the period in the assembly due to symbol conflicts. Nope. It is literally impossible. It's not even hard to fix, someone found it and fixed it in like a day! But they refused to accept the pull request because they don't want people doing that for... reasons that aren't clear.

So make a passthrough function? Like, maybe

code:
func (mat *Mat3) Det3() float32 {
    return det3(mat)
}
Nope, at the speeds you're working with doing SIMD optimization, the passthrough is almost as slow as the by-value copy. I don't recall my benchmarks off the top of my head, but SIMD-with-pointer is like 2.1 ns/op, SIMD-with-copy is like 10.8 ns/op, SIMD-with-passthrough-by-pointer is like 9.5 ns/op.

I hate everything about working with Go's intermediate assembler.

Linear Zoetrope fucked around with this message at 05:24 on Feb 26, 2015

Coffee Mugshot
Jun 26, 2010

by Lowtax

sarehu posted:

My real theory is that the compiler is a complete disaster (I looked) so it would be hard to do. I feel like a hypocrite because I'm 1 for 1 when it comes to having your compiler be a disaster, but I can't imagine having to add generics to their existing implementation. I just remember seeing some struct with pages of fields in it, like out of some 80's horror movie. Now they've got it implemented in Go but apparently that was with a C to Go translation so it's probably still the same.

For the record the original compiler was most written by Ken Thompson which is why it is a bit of disaster to read and modify. Russ converted it to go mechanically and we plan to improve it and flat out remove certain parts, replacing them with the go/* packages where useful.

blorpy
Jan 5, 2005

"we"

blorpy
Jan 5, 2005

Compile times are important to us. That's why we have offloaded compilation onto our programmers.

sarehu
Apr 20, 2007

(call/cc call/cc)

Sorry there's no Rap Genius thread.

blorpy
Jan 5, 2005

sarehu posted:

Sorry there's no Rap Genius thread.

There's probably one in yospos or smth

Coffee Mugshot
Jun 26, 2010

by Lowtax

idgi

mdxi
Mar 13, 2006

to JERK OFF is to be close to GOD... only with SPURTING

It may be helpful to remember that Go is a language designed at Google, for Google, to solve Google's problems, in a way which is efficacious for Google. It may also be illuminating to remember that it was designed and written by people who are already titans in their field and have absolutely nothing to prove to anyone.

It is basically a pleasant side effect that Go is useful to people outside of Google.

As an aside, if you don't like the design or implementation of a language, you'll feel much better (and get a lot more respect) if you (a) write code to solve a cool problem, in a language which you do enjoy, instead of (b) writing prose to tell the internet that you have decreed a language to be bad.

Linear Zoetrope
Nov 28, 2011

A hero must cook

mdxi posted:

It may be helpful to remember that Go is a language designed at Google, for Google, to solve Google's problems, in a way which is efficacious for Google. It may also be illuminating to remember that it was designed and written by people who are already titans in their field and have absolutely nothing to prove to anyone.

It is basically a pleasant side effect that Go is useful to people outside of Google.

As an aside, if you don't like the design or implementation of a language, you'll feel much better (and get a lot more respect) if you (a) write code to solve a cool problem, in a language which you do enjoy, instead of (b) writing prose to tell the internet that you have decreed a language to be bad.

Eh, I like Go but it has problems. I can go on a rant about any language I use enough to understand. Go's assembly and type system are somewhat problematic, but it has a lot of good aspects. For instance, I can pretty much read anybody's Go code without many problems (unless it's deliberately obfuscated or it's a problem domain I have no experience in), which is actually kind of amazing.

Adbot
ADBOT LOVES YOU

Plorkyeran
Mar 22, 2007

To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed
Being able to read code written by other teams you've never had any contact with is the sort of problem that Google would like to solve.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply