|
I'm considering picking up Go as an extra tool. Most of the work I do involves image manipulation, numerics, and machine learning. I saw some discussion on the Go user group saying academics were migrating away from Go, but I'm slightly scrupulous. Is there any reason not to get involved if that's the kind of work I'm doing? What are the de-facto linear algebra and image packs?
|
#
?
Jun 20, 2014 14:49
|
|
|
|
| # ? Apr 20, 2024 05:59 |
|
|
Go has some rudimentary image support built in: http://golang.org/pkg/image/ No idea about the other stuff.
|
|
#
?
Jun 20, 2014 14:56
|
|
|
Jo posted:I'm considering picking up Go as an extra tool. Most of the work I do involves image manipulation, numerics, and machine learning. I saw some discussion on the Go user group saying academics were migrating away from Go, but I'm slightly scrupulous. Is there any reason not to get involved if that's the kind of work I'm doing? What are the de-facto linear algebra and image packs? Two of the main issues with doing heavy science and math stuff in go is the lack of generics and operator overloading which makes it kind of a pita to do stuff like matrix math and the like. Regarding packages for image manipulation the one in the core library is pretty decent, but I'm not sure what exactly you need to do. Right now golang is great for server side apps that you'd normally code in python and java (for the most part), science libs and the like are kinda lacking libraries at the moment (someone feel free to prove me wrong though, I haven't specifically looked for said libraries that's just the gist I've gotten from reading mailing lists and hanging around go people on IRC)
|
|
#
?
Jun 20, 2014 15:15
|
|
|
Jo posted:I'm considering picking up Go as an extra tool. Most of the work I do involves image manipulation, numerics, and machine learning. I saw some discussion on the Go user group saying academics were migrating away from Go, but I'm slightly scrupulous. Is there any reason not to get involved if that's the kind of work I'm doing? What are the de-facto linear algebra and image packs? AFAIK, there's one particular person who's been claiming this, but it's not clear that there's any kind of consensus on the matter. As Baby Nanny said, the lack of generics and operator overloading make it unappealing to do work the areas you mentioned. That being said, there are people in the community writing tools in those areas (from the community wiki): https://code.google.com/p/go-wiki/wiki/Projects#Machine_Learning. I won't make any claims that these projects are idiomatic at all, but it might interesting to look through them for reasons to use or not use Go.
|
|
#
?
Jun 20, 2014 16:19
|
|
|
Thanks for the feedback. I'm not too bothered by a lack of operator overloading. A lack of generics, however, is a little bit cumbersome. Maybe I'll wait for it to be a little farther along.
|
|
#
?
Jun 20, 2014 19:36
|
|
|
Jo posted:Thanks for the feedback. I'm not too bothered by a lack of operator overloading. A lack of generics, however, is a little bit cumbersome. Maybe I'll wait for it to be a little farther along. You'll be waiting for a really long time. Generics aren't coming for years, if ever.
|
|
#
?
Jun 21, 2014 18:22
|
|
|
Does anyone feel up to giving me some feedback on an ORM I am putting together? The core of it is done, I just want to know what people think of its interface & docs. However, I don't want it public yet so I'll have to give you access to my github repo. If you're interested then either pm me or post here.
|
|
#
?
Jun 29, 2014 22:00
|
|
|
Is it actually relational? Theres already like 10 'ORM's and the only one I've seen with actual relational stuff is Beego
|
|
#
?
Jun 29, 2014 22:20
|
|
|
pram posted:Is it actually relational? Theres already like 10 'ORM's and the only one I've seen with actual relational stuff is Beego You bet. It is actually, really, really relational, in the sense that there are no nulls, all relations are distinct, and I don't implement an ordering. Here's an example from the readme. This works. Edit: either pm me or post your github account if you feel up to giving me some advice. pre:// Here's an example of sparse matrix multiplication using rel (relational meets linear):
type matrixElem struct {
R int
C int
V float64
}
type multElemA struct {
R int
M int
VA float64
}
type multElemB struct {
M int
C int
VB float64
}
type multElemC struct {
R int
C int
M int
VA float64
VB float64
}
type groupTup struct {
VA float64
VB float64
}
type valTup struct {
V float64
}
groupAdd := func(val <-chan groupTup) valTup {
res := valTup{}
for vi := range val {
res.V += vi.VA * vi.VB
}
return res
}
// representation of a matrix:
// 1 2
// 3 4
A := New([]matrixElem{
{1, 1, 1.0},
{1, 2, 2.0},
{2, 1, 3.0},
{2, 2, 4.0},
}, [][]string{[]string{"R", "C"}})
// representation of a matrix:
// 4 17
// 9 17
B := New([]matrixElem{
{1, 1, 4.0},
{1, 2, 17.0},
{2, 1, 9.0},
{2, 2, 17.0},
}, [][]string{[]string{"R", "C"}})
C := A.Rename(multElemA{}).Join(B.Rename(multElemB{}), multElemC{}).
GroupBy(matrixElem{}, groupAdd)
fmt.Println("\n%#v",C)
// Output: (might be in any order)
// rel.New([]struct {
// R int
// C int
// V float64
// }{
// {1, 1, 22, },
// {1, 2, 51, },
// {2, 1, 48, },
// {2, 2, 119, },
// })
|
|
#
?
Jun 29, 2014 23:00
|
|
|
I'll have to play with this! I look forward to it. Too bad I'm half asleep.
|
|
#
?
Jun 30, 2014 03:28
|
|
|
Dommer416 posted:I'll have to play with this! I look forward to it. Too bad I'm half asleep. Alrighty, it's public at https://github.com/jonlawlor/rel
|
|
#
?
Jul 2, 2014 05:37
|
|
|
I've been using Go at work for some time, mainly coding server-side web stuff. It's not perfect (far from it) but it's so much better than the alternatives, it's a no brainer. We start every new project in Go now and we have long term plans to migrate everything there (from Python, PHP, and Java) Go runs as fast as Java, with a startup time as small as a C executable or Bash script; it's as easy to write as Python / Ruby, if not easier; its standard library is already as good as Python's; and the compiler and standard toolchain is blazing fast and useful. etcetera08 posted:If it had not horrible package management […] I'd recommend it unreservedly. What. If by "package management" you mean the fact that you have to pull all the sources for the libraries you use inside your project's source tree, or use your version control "linked repo" feature, and then it compiles everything down to a static binary that you can deploy anywhere, then I believe Go has the best package management I've ever seen. Otherwise I'm not sure what you mean. Bozart posted:
That looks interesting! I'll make sure to give it a look. After all, relational algebra is what makes databases tolerable. I just wish they had a nicer syntax than SQL. (My idea of a nice syntax is APL, so I know I'm in the minority there.) One note though: I've seen a lot of libraries using empty structs (or their pointers: &Foo{}) as a Go type literal, like Foo.class in Java. The problem with it is that it allocates and zeroes an entire struct of that type for nothing. That could be large, if it contains fixed arrays or other embedded structs. Unfortunately Go does not have a clean type literal that can be passed around, but there is a cheap alternative in the form of a typed nil pointer: pre:(*Foo)(nil) It only takes two words of memory (as interface values do, IIRC) and it's not too verbose. Users of your library may even define their own nil pointers as global vars, to be used as type literals, and that's even easier to write: var FooType *Foo See: http://play.golang.org/p/nom0uJBeDs The fact that it needs two sets of parentheses is another of the many unfortunate choices the Go designers made. See here for an explanation. I wish they had used Pascal syntax. etatoby fucked around with this message at 21:47 on Jul 15, 2014 |
|
#
?
Jul 15, 2014 21:44
|
|
|
etatoby posted:After all, relational algebra is what makes databases tolerable. I just wish they had a nicer syntax than SQL. (My idea of a nice syntax is APL, so I know I'm in the minority there.) You might like ISBL, then: http://posted-stuff.blogspot.com/2009/01/databases-information-system-base.html (sorry, best link I can find—it was never big)
|
|
#
?
Jul 15, 2014 22:11
|
|
|
Thanks for actually looking at the package! The single value allocation thing is a little ugly and I'll try incorporating your suggestion. I was planning on using sync.Pool to help with the amount of literal allocation that takes place, but the community's opinion of it seems mixed. I really haven't gotten much feedback on the rel package because people take a look at its size, and the fact that it uses a ton of reflection, and run away as fast as they can. For now I'm putting that project on the back burner while I implement the relational operations in idiomatic go. There is something really interesting about connecting relational algebra with Go's pipeline mechanisms, so I'm going to  start a blog to walk people through them as gently as I can, which means no reflection, and no query rewrite. By now I've rewritten them all a few times, so it isn't too hard, and I'm still figuring out what's good and bad in different situations. For example, ordered relational operations are at a huge advantage because you can make assumptions about what tuples will be sent to each pipeline stage, while the rel package doesn't assume any ordering exists (it doesn't even define a sort function!) so I basically use hash-joins and hash-groups for everything.
|
|
#
?
Jul 16, 2014 00:19
|
|
|
etatoby posted:One note though: I've seen a lot of libraries using empty structs (or their pointers: &Foo{}) as a Go type literal, like Foo.class in Java. The problem with it is that it allocates and zeroes an entire struct of that type for nothing. That could be large, if it contains fixed arrays or other embedded structs. There is a reason people use empty structs, they are zero sized.
|
|
#
?
Jul 16, 2014 00:56
|
|
|
Azazel posted:There is a reason people use empty structs, they are zero sized. That isn't what he's talking about though. The rel.Relation interface has methods which take an example tuple into an interface{} so that it can reflect on it to determine how to construct result tuples. He's saying I should have callers construct a nil pointer of the appropriate type instead to avoid the overhead.
|
|
#
?
Jul 16, 2014 04:18
|
|
|
Bozart posted:That isn't what he's talking about though. The rel.Relation interface has methods which take an example tuple into an interface{} so that it can reflect on it to determine how to construct result tuples. He's saying I should have callers construct a nil pointer of the appropriate type instead to avoid the overhead. Not trying to nitpick too hard here, it's clear he was talking about your library. However, "I've seen a lot of libraries using empty structs" is a very general statement, which is the part I was addressing. Feel free to ignore my comment if it does not apply.
|
|
#
?
Jul 16, 2014 13:45
|
|
|
etatoby posted:If by "package management" you mean the fact that you have to pull all the sources for the libraries you use inside your project's source tree, or use your version control "linked repo" feature, and then it compiles everything down to a static binary that you can deploy anywhere, then I believe Go has the best package management I've ever seen. Are you using any of the 3rd party dependency management tools? Or checking in your entire GOPATH? It's a little frustrating that One True Way hasn't emerged yet, and I'm just not doing enough go development to be able to judge the competitors.
|
|
#
?
Jul 16, 2014 18:39
|
|
|
What's the current frontrunner for web frameworks? I've heard good things about Revel.
|
|
#
?
Jul 16, 2014 18:48
|
|
|
Azazel posted:Not trying to nitpick too hard here, it's clear he was talking about your library. However, "I've seen a lot of libraries using empty structs" is a very general statement, which is the part I was addressing. Feel free to ignore my comment if it does not apply. That quote finished with "…as a Go type literal, like Foo.class in Java." I know empty struct types are useful, for example as method receivers, channel element types, singletons, and so on. What I was ranting about was the amount of libraries that use empty (rather, "zero") struct values as a way to convey type information, like you would use a class literal in Java (MyClass.class) or the class name in Python, and so on. The standard library fortunately avoids it. For example, when the various Unmarshal() methods accept an interface{} value, it's not just used to convey information about its type: it's the actual pointer to a variable that the Unmarshal() method is supposed to fill. good jovi posted:Are you using any of the 3rd party dependency management tools? Or checking in your entire GOPATH? Frankly I haven't even looked into dependency management tools, because to me, the only way to make sure whatever I commit is reproducible is to check every dependency in. If and when I want to update a library, it will be a deliberate action. I will use go get, then I'll make sure that everything still works, update my code as needed, and commit everything into my tree in the correct branch. A consequence of this is that every project (repository) of mine is a GOPATH in its own right, with a src/ dir containing every dependency needed to build the project, alongside my code; a build.sh script in the root, that sets GOPATH (plus other env vars or go/cgo flags) and calls go install; a tpl/ dir for HTML templates; various other dirs for static resources (js, css, img…); more dirs for sources in other languages (sass, coffeescript…); their build or watch scripts in the root; and finally the pkg/ and bin/ dirs created by go install, that I configure my VCS to ignore. Of course, different commits or branches in the history of your project will depend on different versions of various libraries, that's just natural. If you check everything in, then your update/checkout commands will bring your repository into the exact state it was when you made that commit, even if it was years ago. No need to access remote repositories that may no longer exist, have changed names or hosts, their history been rewritten and hashes changed, and so on. Hah! Where is your "dependency management tool" now?  You can even have the go releases in different places (/usr/local/go1.2, go1.3…) and reference the right one in your build script, to be committed alongside the other source changes. In fact, I think I will do just that, starting tomorrow. Perfectly reproducible commits, across all dependencies and language versions, on as many parallel projects and branches as you need, using your favorite VCS and nothing else. I've always considered this to be the Go way, so I'm quite puzzled that people look for "dependency management tools" in the first place. /rant
|
|
#
?
Jul 16, 2014 19:45
|
|
|
Well tools like godeps and goven just automate something like what you're talking about anyway. One of them imports the dependencies into a sub-package of your project and then rewrites all the import paths - this way you don't need the separate script to manage GOPATH and stuff.
|
|
#
?
Jul 16, 2014 22:28
|
|
|
QPZIL posted:What's the current frontrunner for web frameworks? I've heard good things about Revel. Martini owns, but I guess it's not really a 'framework.' I've used it in most of my stuff. I didn't like Revel when I looked at it. Beego looks really fantastic but most of the documentation/discussions are in chinese. https://github.com/astaxie/beego Beego has a lot of neat things baked in like monitoring statistics for health checks etc
|
|
#
?
Jul 17, 2014 04:34
|
|
|
QPZIL posted:What's the current frontrunner for web frameworks? I've heard good things about Revel. I use a combination of the standard net/http and gorilla/mux libraries myself, but hear good things about https://github.com/zenazn/goji if you are looking for something minimal. Also, this has been floating around twitter today: http://arslan.io/ten-useful-techniques-in-go - Good advice in there.
|
|
#
?
Jul 17, 2014 05:22
|
|
|
Azazel posted:I use a combination of the standard net/http and gorilla/mux libraries myself, This. More often than not it's all I ever need, even for larger applications.
|
|
#
?
Jul 17, 2014 14:48
|
|
|
So I have the first post ready in a new blog: http://jonlawlor.github.io/2014/07/18/distinct/  Do you guys have any feedback? Otherwise I guess I'll be posting it on reddit tomorrow.
|
|
#
?
Jul 18, 2014 05:23
|
|
|
etatoby posted:If by "package management" you mean the fact that you have to pull all the sources for the libraries you use inside your project's source tree, or use your version control "linked repo" feature, and then it compiles everything down to a static binary that you can deploy anywhere, then I believe Go has the best package management I've ever seen. Everyone wants a "go get"-able repo, but that's not the best way to manage a large project. IIRC this is at least part of what killed that Haunts game: they had external dependencies, and then all of a sudden github.com/dickhole/poo is gone because dickhole decided to delete his repo, or maybe he just completely changed the API and now your code doesn't build anymore. We've started managing our Go code similarly to a well-run C project: You have a src/ directory under your top-level which includes subdirectories for your code AND for the external libraries you need. There's a script at the top level which sets GOPATH to the current directory and builds your code, compiling the dependencies in the process. This keeps you safe from dickhole deleting his repository, and it lets you adapt to an updated API on your own schedule, not at 4 p.m. on a Friday when dickhole decides to push his changes. When you're ready to update your library dependencies, you just check out the latest version of their repo, copy it in, make sure it builds, and commit.
|
|
#
?
Jul 28, 2014 23:07
|
|
|
Pham Nuwen posted:Everyone wants a "go get"-able repo, but that's not the best way to manage a large project. IIRC this is at least part of what killed that Haunts game: they had external dependencies, and then all of a sudden github.com/dickhole/poo is gone because dickhole decided to delete his repo, or maybe he just completely changed the API and now your code doesn't build anymore.
|
|
#
?
Jul 28, 2014 23:38
|
|
|
Brecht posted:Vendoring your deps and keeping your repo go-gettable aren't mutually exclusive. Your repo shouldn't represent its own GOPATH. I mean, what's wrong with having a script to setup a repo you pulled down from your VCS as its own GOPATH?
|
|
#
?
Jul 29, 2014 00:26
|
|
|
Gustavo Niemeyer, creator of the esteemed mgo mongodb library, and doing a lot of awesome work with a QML library, has created this service for making sane go-gettable packages: http://labix.org/gopkg.in It doesn't fix the "some rear end in a top hat deletes his repository" problem, but it does make for a fairly solid way of getting consistent packages versions as long as the library maintainer does proper tagging.
|
|
#
?
Jul 29, 2014 03:43
|
|
|
Jo posted:I'm considering picking up Go as an extra tool. Most of the work I do involves image manipulation, numerics, and machine learning. I saw some discussion on the Go user group saying academics were migrating away from Go, but I'm slightly scrupulous. Is there any reason not to get involved if that's the kind of work I'm doing? What are the de-facto linear algebra and image packs? I'm one of the owners of Gonum, which isn't super mature right now, but it does have blas bindings and a higher-level matrix package. We also just started a statistics package. It benchmarks pretty well, actually, but you have to remember there's no operator overloading so code looks verbose. It's not quite as bad as you might think. Even in Numpy matrix multiplication is a.dot(b), so compared to packages like that you're not missing much, only really addition/subtraction and scalar multiplication see much use with operator overloading in most other languages. Image processing itself is a beast because you probably want something with OpenCV-ish functionality, which just doesn't exist in Go. I did some Kinect stuff in Go just to see how it works and it was a mess. Wrapping libfreenect was fairly straightforward, but using the better functionality of OpenCV requires a bunch of Cgo nonsense with wrapping heap allocated matrices in (void*)'s and oh by the way make sure your headers are in pure C and only include pure C, but you can still implement the function in .cxx files which can have C++ and...  it's all pretty terrible. (Also: SWIG seems to not play super nice with Open CV for some drat reason)The thing is Go could be a very good language for this stuff. If you're willing to help write it the potential is there. I knocked out a pretty decent and fast Canny edge detection filter in an hour or so, but if you're just looking for nice frameworks that let you do your work, you need to find a more established language. spongeh posted:How is Go for game dev? I've found a couple of bindings, but nobody seems to support SDL2 yet. I seemed to have the impression that even though it's now 1.0, there's not a whole lot of use, but it may be more server-oriented which I'm not really looking at. It really depends. There's nothing stopping you if you already really like Go. I'm writing an engine in it and it's enjoyable. However, if you're looking to get into go FOR game programming I'd say to stay away from it. I won't enumerate all the minutae (especially if you want an asynchronous engine), but the libraries really aren't there and once you get into graphics and audio drivers you run into... speedbumps with the way the runtime works re: goroutines and threading. The best bindings right now are for pure Open GL, the SDL ones and other "nice" libraries are fairly incomplete or abandoned. Even audio you'll probably have to resort to wrapping Open AL (most of the currently available wrappers are poo poo) or Port Audio (obnoxious because you have to attenuate and mix audio manually) or something. Linear Zoetrope fucked around with this message at 11:24 on Aug 10, 2014 |
|
#
?
Aug 10, 2014 11:12
|
|
|
Also, for game dev you might want to wait for Go 1.5 which will have GC improvements.
|
|
#
?
Aug 10, 2014 16:14
|
|
|
Jsor, as a followup, how are you supposed to get gonum running? Going off of the readme at https://github.com/gonum/lapack didn't help. I'm running osx mav, installed gfortran: code:No error messages, ran code:test.go:18: undefined: clapack.Lapack after about a zillion warnings in clapack: ../../gonum/lapack/clapack/lapack.go:21582:3: warning: plain '_Complex' requires a type specifier; assuming '_Complex double' /usr/include/complex.h:37:17: note: expanded from macro 'complex' So at this point I don't know what to do. Does gonum have some kind of installation guide that I'm missing?
|
|
#
?
Aug 10, 2014 21:03
|
|
|
Jsor posted:The thing is Go could be a very good language for this stuff. If you're willing to help write it the potential is there. I knocked out a pretty decent and fast Canny edge detection filter in an hour or so, but if you're just looking for nice frameworks that let you do your work, you need to find a more established language. http://www.evanmiller.org/why-im-betting-on-julia.html
|
|
#
?
Aug 11, 2014 00:36
|
|
|
I don't personally maintain the LAPACK bindings, so I'm not sure. I could figure it out, but it may be faster to just ask on the Google Group. I'll try to approve you as soon as I can. I do know that I'm the only one in the group that semi-regularly uses anything other than Linux (I have a Mac and a PC that I do dev on pretty frequently), so it's entirely plausible the OS X bindings for LAPACK are just screwed up. LAPACK is a really new package. Scaevolus posted:As much as I like Go, Julia is looking more promising for scientific workloads. Go's developers want a language to write networked services, so things like maximal performance (including FFI), good matrix support, and operator overloading aren't particularly high on their list of improvements. Julia probably has a better feature set, and I'll probably use it at some point (I've already done a couple small projects in it). I encourage anyone to use the tools for the job, and Julia fits the bill. Probably better than Go too. Go is still pretty good, and is especially good at make concurrent, cache-oblivious algorithms painlessly. The main problem I have with Julia is that reading it is a pain, for whatever reason I have trouble parsing Julia code quickly and efficiently. I find it easier to knock out something in Go, and that was true even did when I didn't know the language. There's something about Go's design that agrees with my particular brain and makes it almost effortless to think in the language, in a way I've never had with C++ or Matlab or even Python. It's really a personal preference thing, to me.  E: Oh, and I haven't found GC latency to be a big deal for games. If you do some research, Go's GC has fairly predictable criteria for firing, most of it relating to allocating <x> times more memory than the last time the GC ran. So if you front-load all your allocations with some sort of pool it's pretty negligible. Linear Zoetrope fucked around with this message at 02:16 on Aug 11, 2014 |
|
#
?
Aug 11, 2014 02:10
|
|
|
Is there a good way to generate bindings for a large C dynamic library? Something with tens of header files and a hundred or so functions, more than I could create or maintain by hand.
|
|
#
?
Aug 11, 2014 02:35
|
|
|
Ninja Rope posted:Is there a good way to generate bindings for a large C dynamic library? Something with tens of header files and a hundred or so functions, more than I could create or maintain by hand. That sounds like SWIG's job.
|
|
#
?
Aug 11, 2014 02:39
|
|
|
Scaevolus posted:As much as I like Go, Julia is looking more promising for scientific workloads. Go's developers want a language to write networked services, so things like maximal performance (including FFI), good matrix support, and operator overloading aren't particularly high on their list of improvements. I think it really depends on the particular application. I don't think most matlab code (or probably julia code) deals with matrix algebra - most of it is the plain old programming that everyone does. I'm particularly excited about time series in matlab. I was going to take gonum for a test drive by implementing recursive lease squares estimation and/or a kalman filter, but first I'll have to figure out how to get it running. I will have to read up a little on julia to see what a comparable version there would look like.
|
|
#
?
Aug 11, 2014 04:45
|
|
|
good jovi posted:Are you using any of the 3rd party dependency management tools? Or checking in your entire GOPATH? I don't really see how using go get doesn't satisfy people. If I recall correctly one of Go's goals is to make everything backwards compatible and it is adapted by pretty much all major packages. I'm actually really glad that there aren't such ridiculous things like the depreciation mechanics Java has that are so blatantly ignored by most of it's users.
|
|
#
?
Sep 27, 2014 08:42
|
|
|
NickPlowswell posted:I don't really see how using go get doesn't satisfy people. If I recall correctly one of Go's goals is to make everything backwards compatible and it is adapted by pretty much all major packages. When you start importing a handful of 3rd party libraries and share those libraries among multiple developers/projects "go get" alone becomes very unsatisfying.
|
|
#
?
Sep 27, 2014 11:58
|
|
|
|
| # ? Apr 20, 2024 05:59 |
|
|
NickPlowswell posted:I don't really see how using go get doesn't satisfy people. If I recall correctly one of Go's goals is to make everything backwards compatible and it is adapted by pretty much all major packages. Having to import "github.com/randomfuckehead/shittylibrary/awfulsublibrary" is both ugly and extremely fragile, even with go get. If randomfuckhead takes down shittylibrary or even leaves github altogether, you have to figure out what to do for future deployments. It's saved there locally with you having used go get during development, but it won't work anymore on new machines after his project/whole github is down. e: and yes I know you can locally clone it and store it yourself but that seems to be frowned upon since for whatever reason you include fragile source URIs directly in your source code.
|
|
#
?
Sep 30, 2014 21:16
|
|
