Go: Google's Plan9 refugees are staging a breakout

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Go: Google's Plan9 refugees are staging a breakout

«‹›14 »

DONT THREAD ON ME: Oct 1, 2002; by Nyc_Tattoo; Floss Finder

Helicity posted:

I personally disagree with the "web services" part unless you're only responsible for a small API footprint. Serialization/deserialization feels tedious to me for a few reasons, and web services are one of the best places to lean heavily on functional programming ideas and patterns since the whole request/response pipeline is essentially a mapreduce.

edit: Golang would be a lot more fun for me on certain things if there was an equivalent of JS's lodash library, or even better, Kotlin+arrow. Is there such a thing?

any such library would need to lean heavily on interface{}. in a typed language you need generics if you want a library like lodash.

i agree though, go can really only support certain kinds of web services, ones that don�t stray much into business domains.

# ? Aug 12, 2018 20:58

Adbot: ADBOT LOVES YOU

# ? May 14, 2024 16:24

Infinotize: Sep 5, 2003

Khorne posted:

Go is good tool for web services and similar applications. I've used it off and on since 2014 and as long as you don't venture too far from that realm it's truly well designed. It's also not bad for command-line/one-off programs that most people would probably reach for Python/bash/perl/whatever to make. Its portability, single day learning curve, ease of developing in, and ease of setting up its environment are all great. It has similar level performance to pretty much anything for web stuff. It's not slow and carrying five tons of baggage like lots of the common web backend languages, and it doesn't require significant developer investment like C++ or functional languages. It also doesn't require consuming a bloated corpse like the JVM languages.

Its package management left a lot to be desired last time I used it, but it was supposed to be fixed at some ambiguous time in the future which may have already happened.

Due to the positives, people often use it for things it's real bad at. It is missing language features that are very useful for certain types of applications, but it's mostly intentional to make the language better at what it's good at. It's a current-gen tool for a specific type of program and a drat good one.

Yes Yes Yes

I would echo that Go is great in a ~devops~ type tooling space. I also use Go for some more heavyweight backend services (replacing a JVM application) and it's done very well. I'm pretty pleased with the community and momentum of the project.

# ? Aug 14, 2018 20:04

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

It's great in a devops space because compiling to a statically-linked binary makes deployment dead easy. No need to faff about with containers to bundle up (say) Python and all its deps.

# ? Aug 15, 2018 02:36

spiritual bypass: Feb 19, 2008; Grimey Drawer

Basically every natively compiled language except C compiles all the same-language libraries into the binary

# ? Aug 15, 2018 02:52

luchadornado: Oct 7, 2004; A boombox is not a toy!

minato posted:

It's great in a devops space because compiling to a statically-linked binary makes deployment dead easy. No need to faff about with containers to bundle up (say) Python and all its deps.

Everytime I have to explain pip, virtualenv, how OSX and brew toss around Python files, etc. I die a little on the inside. My old team's devops guys just went all-in on Go after getting frustrated with a mix of Bash/Go/Python/Ruby/Groovy. They're loving it so far.

# ? Aug 15, 2018 03:42

rjmccall: Sep 7, 2007; no worries friend; Fun Shoe

rt4 posted:

Basically every natively compiled language except C compiles all the same-language libraries into the binary

Swift takes dynamic linking seriously and it's been a huge pain in the rear end because we have to work around all the lovely ways that PE, ELF, and Mach-O are different.

Like if you flip a switch to just build everything for static linking, the Windows port pretty much just works, but if you use dynamic linking it's just an unending horror show of "gently caress I guess we can't emit code that way on Windows".

# ? Aug 15, 2018 05:30

Hadlock: Nov 9, 2004

minato posted:

It's great in a devops space because compiling to a statically-linked binary makes deployment dead easy. No need to faff about with containers to bundle up (say) Python and all its deps.

I rolled out my first container to prod with awless yesterday. We needed access to one aws api, but didn't want to install aws cli, as aws cli is a python script/app which necessitates installing python + at least one additional third party library (boto) + hundreds of files + all of Python's dependencies....

Awless is basically (for my purposes anyways) a clone of aws cli written in go, which means to drop in aws support into a container, I went from a huge dependency liability and hundreds of files due to python (and it's dependencies), to just drop a single semanticallly versioned 10mb binary in the container and I'm done.

Really like the vertical integration that 1 versioned service = 1 versioned go binary = 1 versioned container. In Jira ticket XRY-123 is for X-RAY release 3.1.45, release 3.1.45 is X-RAY service version 3.1.45 , which is tagged to container version X-RAY 3.1.45. So when the customer calls in that the new feature in 3.1.45 is conflicting with another feature, I can go look at all the jira tickets in that release, drill down to the tickets on that release and get the developer that broke it to go fix it and roll it 3.1.46 by the end of day. The product, engineering and customer success teams all care about different things but can communicate clearly because they all use the same versions.

# ? Aug 15, 2018 17:30

Bruegels Fuckbooks: Sep 14, 2004; Now, listen - I know the two of you are very different from each other in a lot of ways, but you have to understand that as far as Grandpa's concerned, you're both pieces of shit! Yeah. I can prove it mathematically.

rjmccall posted:

Swift takes dynamic linking seriously and it's been a huge pain in the rear end because we have to work around all the lovely ways that PE, ELF, and Mach-O are different.

Like if you flip a switch to just build everything for static linking, the Windows port pretty much just works, but if you use dynamic linking it's just an unending horror show of "gently caress I guess we can't emit code that way on Windows".

yeah, just don't use dynamic linking on windows for goddamn anything if you can get away with it. they used to call it dll hell for a reason.

# ? Aug 16, 2018 13:16

an skeleton: Apr 23, 2012; scowls @ u

Why do I have to use CombinedOutput() instead of just Output() to get the output of a script I'm executing from go?

# ? Aug 22, 2018 10:57

ErIog: Jul 11, 2001

an skeleton posted:

Why do I have to use CombinedOutput() instead of just Output() to get the output of a script I'm executing from go?

Because the thing you're running outputs to stderr instead of stdout, and so getting the combined stderr/stdout gets the output you expect?

os/exec is pretty solid. I've built a number of research projects around it. I did wrap it to be more convenient for my uses, but I've never found a bug in it despite some code being just numerous wrappers around things called by os/exec.

ErIog fucked around with this message at 13:24 on Aug 22, 2018

# ? Aug 22, 2018 13:22

Small White Dragon: Nov 23, 2007; No relation.

I'm new to GO, and working through the tutorial. I noticed that you can do:

x, y := <- ch1, <- ch2

As well as

x, ok := <- ch

Isn't this sort of thing ambiguous? Can you not have multiple right hand arguments that return multiple values?

# ? Aug 24, 2018 20:06

Eela6: May 25, 2007; Shredded Hen

Small White Dragon posted:

I'm new to GO, and working through the tutorial. I noticed that you can do:

x, y := <- ch1, <- ch2

As well as

x, ok := <- ch

Isn't this sort of thing ambiguous? Can you not have multiple right hand arguments that return multiple values?

You can't have multiple right hand arguments that return multiple values. This removes the ambiguity at the expense of expressiveness, which is very go, fwiw.

Try it yourself at the go playground

# ? Aug 24, 2018 20:39

Linear Zoetrope: Nov 28, 2011; A hero must cook

Note that these assignments are evaluated left to right, though. This is important if your function calls are stateful or there's deadlock potential.

code:

package main

import (
	"fmt"
)

func syncrx(quit chan<- struct{}, sync <-chan struct{}) {
	<-sync
	
	quit <- struct{}{}
}

func synctx(quit chan<- struct{}, sync chan<- struct{}) {
	quit <- struct{}{}
	
	sync <- struct{}{}
}

func main() {
	quit1,quit2,sync := make(chan struct{}), make(chan struct{}), make(chan struct{})
	
	go syncrx(quit1,sync)
	go synctx(quit2,sync)
	
	x,y := <-quit1, <-quit2
	
	fmt.Printf("%v, %v\n", x, y)
}

Playground

Will deadlock because syncrx is waiting on synctx, synctx is waiting on main, and main is waiting on syncrx. Swapping quit1 and quit2 in the assignment will make it run properly.

Linear Zoetrope fucked around with this message at 21:42 on Aug 24, 2018

# ? Aug 24, 2018 21:40

Coffee Mugshot: Jun 26, 2010; by Lowtax

The comma-ok syntax is special to certain operations and can't be mixed with multiple assignment operations otherwise. I.e. you'll never be able to write I, ok1, j, ok2 <- ch1, ch2

# ? Aug 24, 2018 22:31

DONT THREAD ON ME: Oct 1, 2002; by Nyc_Tattoo; Floss Finder

here we go:
https://blog.golang.org/go2draft
https://go.googlesource.com/proposal/+/master/design/go2draft-contracts.md
https://go.googlesource.com/proposal/+/master/design/go2draft-generics-overview.md
https://go.googlesource.com/proposal/+/master/design/go2draft-error-handling-overview.md

DONT THREAD ON ME fucked around with this message at 17:41 on Aug 28, 2018

# ? Aug 28, 2018 17:35

Small White Dragon: Nov 23, 2007; No relation.

Any goons at GopherCon?

# ? Aug 29, 2018 01:05

Literally Elvis: Oct 21, 2013

Small White Dragon posted:

Any goons at GopherCon?

I am, it�s rad.

# ? Aug 29, 2018 03:08

ErIog: Jul 11, 2001

I tried to Google this question, but nothing useful comes up.

I'm working on a web server that has a REST-style API. You give it a newline-delimited list of things in a POST body and it vomits back JSON objects separated by newlines.

I thought I could do something like below because hey, it's an io.ReadCloser so why not.

code:

scanner := bufio.NewScanner(request.Body)
for scanner.Scan() {
  line := strings.TrimSpace(scanner.Text())
  .. the rest of my function
}

This led to not all JSON objects requested being returned. When I check the ContentLength of the request it is as expected. The full Body does exist inside the request.

So to see what's happening I moved to bufio.Reader and reader.ReadString('\n'). It turns out the response.Body is being closed somewhere along the way. I'm not closing it anywhere. I grepped all the code and there's not a single Close() anywhere.

My workaround for this was to ioutil.ReadAll the request body to pull the data before the request gets closed by whatever is closing it. The amount of data in the request Body is only like 5 KB. So it's not like there's a memory issue happening and the process is getting killed. However I would like to develop this into more of a streaming style because there's potential in the future for a lot of data to be running through this server.

I'm using Gorilla Mux for URL routing, and then just http.ListenAndServe for the server. Is there some timeout for reading the request.Body or for processing the request that I'm not aware of?

ErIog fucked around with this message at 09:04 on Aug 30, 2018

# ? Aug 30, 2018 09:02

Vanadium: Jan 8, 2005

Do you really care about the newlines or can you just put the body into a json.Decoder directly and keep asking it to decode until .More() or w/e is false?

# ? Aug 31, 2018 03:40

ErIog: Jul 11, 2001

After spending way too much time investigating it turns out that net/http will close the Request body if you write to the Response writer, but not immediately.

Vanadium posted:

Do you really care about the newlines or can you just put the body into a json.Decoder directly and keep asking it to decode until .More() or w/e is false?

I'm somewhat wary of doing that because this is going to be a public API used by academic researchers, not professional programmers. I'm worried that people who using random-off-the-shelf-slow JSON parser in their language of choice might not be able to parse the JSON as a stream like that. I don't want to get e-mails saying my API is slow when the problem is they're using a parser that requires there to be a complete root object in memory before it can parse anything.

The number of objects being serialized could be anywhere between 1 and a million. So taking input/output as a stream of newline-separated objects seemed like the better way to go. I'm also supporting XML output, and so I have the same concerns about those parsers too. I'm trying to support the lowest common denominator of parser/encoder.

If someone tells me I'm solving a problem that doesn't exist then I'll knock it off. I just chose this approach based on my limited experience with serialization libraries in different languages. I'll do some testing and see which approach works best.

ErIog fucked around with this message at 02:13 on Sep 1, 2018

# ? Aug 31, 2018 04:31

spiritual bypass: Feb 19, 2008; Grimey Drawer

Are there any tools available to automatically generate a module file for an existing project? I'd really like to ditch this GOPATH stuff, but I also don't feel like enumerating my dependencies by hand.

# ? Sep 7, 2018 14:22

Eela6: May 25, 2007; Shredded Hen

rt4 posted:

Are there any tools available to automatically generate a module file for an existing project? I'd really like to ditch this GOPATH stuff, but I also don't feel like enumerating my dependencies by hand.

If you're using modules, `go build` will do what you want automatically

https://github.com/golang/go/wiki/Modules posted:

Version Selection
If you add a new import to your source code that is not yet covered by a require in go.mod, any go command run (e.g., 'go build') will automatically look up the proper module and add the highest version of that new direct dependency to your module's go.mod as a require directive. For example, if your new import corresponds to dependency M whose latest tagged release version is v1.2.3, your module's go.mod will end up with require M v1.2.3, which indicates module M is a dependency with allowed version >= v1.2.3 (and < v2, given v2 is considered incompatible with v1).

# ? Sep 7, 2018 15:15

Pollyanna: Mar 5, 2005; Milk's on them.

This may or may not be the best place to ask, but:

We have a protobuf defined that represents a particular set of data, most of it nested. We want to expose filters based on that data, such that given a particular protobuf instance you know you can say �I want to filter on this message�s value between X and Y� or �I want to filter on this enum being of either value Foo or value Bar, and allow any others�.

We currently have to define these filters manually by being aware of these data �leaves� and what type they are, and manually changing what our filters do and allow based on that. What I�m wondering is, is there a way to list all data leaves and their type so we can programmatically filter on them? I�m hoping there�s a way to know when a protobuf message has a new age value and thereby programmatically generate a filter on that new value, without having to manually do so.

Is this something a protobuf can do? If it helps, we are using the Ruby implementation of gRPC and protobufs.

# ? Sep 7, 2018 15:48

Pham Nuwen: Oct 30, 2010

Pollyanna posted:

This may or may not be the best place to ask, but:

We have a protobuf defined that represents a particular set of data, most of it nested. We want to expose filters based on that data, such that given a particular protobuf instance you know you can say �I want to filter on this message�s value between X and Y� or �I want to filter on this enum being of either value Foo or value Bar, and allow any others�.

We currently have to define these filters manually by being aware of these data �leaves� and what type they are, and manually changing what our filters do and allow based on that. What I�m wondering is, is there a way to list all data leaves and their type so we can programmatically filter on them? I�m hoping there�s a way to know when a protobuf message has a new age value and thereby programmatically generate a filter on that new value, without having to manually do so.

Is this something a protobuf can do? If it helps, we are using the Ruby implementation of gRPC and protobufs.

I was going to say it sounds like you want the reflect package, but apparently you're using Ruby, not Go? Check if Ruby has any reflection capabilities, I guess?

# ? Sep 7, 2018 15:51

Pollyanna: Mar 5, 2005; Milk's on them.

I was hoping the Ruby gem or something intrinsic about protobufs would allow them to know about their own data leaves, but I don�t see any helper methods or anything for that. As it is, it will be impossible to programmatically list the specific stuff I want since it will also be accompanied by a bunch of irrelevant constants and members that aren�t what I want to filter on.

That said, I just dug into the Descriptor API, and I think I got what I need. Thanks!

# ? Sep 7, 2018 16:57

spiritual bypass: Feb 19, 2008; Grimey Drawer

Eela6 posted:

If you're using modules, `go build` will do what you want automatically

Oh, I just need a go.mod and the compiler handles the rest? That's lovely

# ? Sep 7, 2018 18:18

Coffee Mugshot: Jun 26, 2010; by Lowtax

Pollyanna I don't know what data leaves refers to when talking about protobufs, maybe you're looking for proto extensions?

# ? Sep 7, 2018 22:14

2nd Rate Poster: Mar 25, 2004; i started a joke

Pollyanna posted:

This may or may not be the best place to ask, but:

We have a protobuf defined that represents a particular set of data, most of it nested. We want to expose filters based on that data, such that given a particular protobuf instance you know you can say �I want to filter on this message�s value between X and Y� or �I want to filter on this enum being of either value Foo or value Bar, and allow any others�.

We currently have to define these filters manually by being aware of these data �leaves� and what type they are, and manually changing what our filters do and allow based on that. What I�m wondering is, is there a way to list all data leaves and their type so we can programmatically filter on them? I�m hoping there�s a way to know when a protobuf message has a new age value and thereby programmatically generate a filter on that new value, without having to manually do so.

Is this something a protobuf can do? If it helps, we are using the Ruby implementation of gRPC and protobufs.

This is certainly something you can do with python's implementation through walking the descriptors for field names/types and then accessing the appropriate attribute on hte message -- it looks like ruby's implementation works about the same way. You want to start poking at message.descriptor which is a Google::Protobuf:

escriptor object https://github.com/ruby-protobuf/protobuf-core/blob/master/lib/protobuf/descriptors/google/protobuf/descriptor.pb.rb.

The times I've been asked to do this though -- it's always ended up less cumbersome and less work to just specifically call out the filter fields in the rpc definition. Yeah its a little more lines of code, but it forces you to keep your rpcs tight, allows you to rely on the proto type system for guard rails, and the small purpose rpc limits scope creep from other apps, allowing you to go wild on implementation of your specific rpc to meet performance goals.

I realize that above advice isn't applicable to every case (and likely not applicable to yours!)-- but I'd be interested in anyone else's opinion on the tradeoffs between a generic vs specific filtering in rpc definitions.

# ? Sep 7, 2018 23:44

Eela6: May 25, 2007; Shredded Hen

rt4 posted:

Oh, I just need a go.mod and the compiler handles the rest? That's lovely

Let me know your experience. At {CORPORATION} we're still using `dep` and go 1.10, so I haven't played around with it myself.

# ? Sep 7, 2018 23:47

Pham Nuwen: Oct 30, 2010

Eela6 posted:

Let me know your experience. At {CORPORATION} we're still using `dep` and go 1.10, so I haven't played around with it myself.

Yeah switching to go modules from dep is one of my issues for the next major release of {COMMERCIAL SOFTWARE} so I'd be interested in hearing how it goes.

# ? Sep 7, 2018 23:59

spiritual bypass: Feb 19, 2008; Grimey Drawer

I encountered a little issue, but it was easy, really. Took me 20 minutes to switch my project to modules. My project is located in GOPATH, so I had to set an env var to make it work. I took way more steps than this but then filtered out the ones that weren't really necessary:

GO111MODULE=on go build
Oh no! A dependency's latest release version is old and contains a bug. Run GO111MODULE=on go get github.com/alexedwards/scs@master to install it at the latest revision
GO111MODULE=on go build

Now you're cooking with modules. Skip the GO111MODULE var if you're outside the gopath. Too bad the docs for this are all focused on theoretical bullshit instead of "here's how to use it"

# ? Sep 8, 2018 19:30

InAndOutBrennan: Dec 11, 2008

rt4 posted:

Too bad the docs for this are all focused on theoretical bullshit instead of "here's how to use it"

docs.txt

# ? Sep 8, 2018 22:35

Pham Nuwen: Oct 30, 2010

Russ is also a legit Computer Science guy from way back, so he's going to spend a lot of time talking about algorithms and such. Plus, this is superseding dep, which made some people buttmad, so I don't blame him for exhaustively documenting how it handles corner cases etc. to keep them from bitching

# ? Sep 8, 2018 22:50

spiritual bypass: Feb 19, 2008; Grimey Drawer

iirc the dep guy is a reject from the Drupal world which is funny because Drupal ignored the one thing PHP got right, dependency management with Composer. Now Go has a Composer-like solution instead of whatever the hell dep was trying to do.

# ? Sep 8, 2018 22:56

consensual poster: Sep 1, 2009

Pham Nuwen posted:

Plus, this is superseding dep, which made some people buttmad, so I don't blame him for exhaustively documenting how it handles corner cases etc. to keep them from bitching

I'll admit that I'm mad about the change away from dep. Not because I'll miss the mediocrity of dep, but because I have no faith that the Go team can pick a dependency management solution and actually stick with it. They should have had a proper dependency management system in place before releasing Go 1.0, but it took them something like 4 or 5 years after 1.0 to even decide that they needed something like dep.

I generally enjoy programming in Go, but the fact that I have to spend mental cycles on something that should have been solved long ago is making me really salty.

# ? Sep 9, 2018 08:30

luchadornado: Oct 7, 2004; A boombox is not a toy!

consensual poster posted:

I generally enjoy programming in Go, but the fact that I have to spend mental cycles on something that should have been solved long ago is making me really salty.

Dependency management and generics are the two big ones for me. Having rudimentary higher-order functions like map and fold, and some sort of monadic way of dealing with errors like Either/Result would be amazing. "Better than C++" is kind of a low bar in 2018, and sometimes it feels like even Java is advancing faster than Go.

The dep guy whining about his solution not being chosen has been a source of amusement for me at least.

# ? Sep 9, 2018 17:13

Riven: Apr 22, 2002

Its less �the dep guy� and more �the entire dependency management community up to that point.� Dep was developed over years with input and oversight from the folks who made Glide, godep, Peter Bourgon and like, all the others. Peter has a great post mortem of it where he realizes that both sides had basically been talking past each other, but the Go core team managed to alienate/jade a sizable contingent of contributors and leading community members, not just �the dep guy.�

# ? Sep 9, 2018 19:54

luchadornado: Oct 7, 2004; A boombox is not a toy!

I think I was referring to this Sam guy who was involved, but I kind of understand what it would feel like. I'll have to read Peter's post-mortem - it's fascinating getting insight into the parts of software development that you can't learn from a book. I'm assuming this is what you were mentioning?

https://peter.bourgon.org/blog/2018/07/27/a-response-about-dep-and-vgo.html

# ? Sep 9, 2018 21:04

Riven: Apr 22, 2002

Yeah that�s the one. And I think reading that will capture more of the betrayal that a portion of the community felt than anything I could write here being fairly removed from it. Many, many people poured their hearts and time over several years to fix the biggest part of the Go ecosystem with i put from the community taking into account as many needs as they could of users and open source developers, with the knowledge of the core team, to basically have Russ Meyer go, �huh, interesting prototype, I went and thought about this problem for awhile and It sure is an interesting CS problem that I have solved the best way for CS considerations, what is a user?�

Riven fucked around with this message at 22:17 on Sep 9, 2018

# ? Sep 9, 2018 22:14

Adbot: ADBOT LOVES YOU

# ? May 14, 2024 16:24

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

Riven posted:

to basically have Russ Meyer go,

I think you have the names mixed up. Russ Meyer was all about tits. Russ Cox is an rear end.

# ? Sep 9, 2018 22:54

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Go: Google's Plan9 refugees are staging a breakout

«‹›14 »