Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

More people should use NPM as their build management thingy.

I'm on mobile and won't be around a PC for a few days, but we discussed this a little bit in one of these threads in the past week or so. Check my post history to find it. Someone posted a link to an article about the subject...

Adbot
ADBOT LOVES YOU

Maluco Marinero
Jan 18, 2001

Damn that's a
fine elephant.
Yeah, I've seen the npm as build script articles. It's good, but it's really not the whole story imo because it spreads out config, or over does it all into a bunch of key vals in uncommentable json. It also can make doing distribution as opposed to development builds kind of awkward or not dry.

Either way, I'm not really fishing for particular tools, npm scripts are certainly a recommended option, but only if what you're building is simple enough to not be a cluster gently caress of unwieldy shell commands. I think there's still a place for more complex builds if they facilitate development effectively, but then we start to get into the many flaws of grunt and gulp and their respectively forever 0.x plugin ecosystems.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Right.

By "more people should use NPM" I meant that too many people over complicate their build stuff because they feel like they should be using Gulp when NPM with a couple of commands is more than adequate to run babel and run a file watcher.

There's a couple of general lessons and advice you can draw from that observation.

Lumpy
Apr 26, 2002

La! La! La! Laaaa!



College Slice

Maluco Marinero posted:

Hey all, I'm working on a conference proposal talk and would like to get a feel for what kind of things people would like to hear in it. The title is want to use if I was allowed would be "Unfuck your poo poo show of a toolchain" but I'll have to settle with "Fix yo toolchain".

I want to focus on a bunch of little/large things + code snippets people can use to make their build tools less annoying to use in day to day. Better watching, stable incremental builds, good error handling, that kind of thing. Some of the focus is also going to be on fixing build systems so they're not constantly tripping up junior devs as well.

I was more or less wondering just what particular things folks hate about build systems, task runners, etc these days.

I hate that there are 9242147 of them. Tell everyone to use NPM and webpack.

Maluco Marinero
Jan 18, 2001

Damn that's a
fine elephant.

Thermopyle posted:

There's a couple of general lessons and advice you can draw from that observation.

Oh absolutely, as simple as possible but no simpler.

Lumpy posted:

I hate that there are 9242147 of them. Tell everyone to use NPM and webpack.

My format is more to cover dot points which address particular tools as a 'this is really easy with this tool, or, this is how you do it with this tool', rather than making the entire talk about a couple of them.

This is mainly so people will actually get a takeaway that isn't 'burn your build system to the ground and replicate with npm and webpack'. Not always the most practical advice, especially if with a few base points in mind and some tweaks they could end up with something pretty serviceable.


Anyway, a rough list of dot points I've got at the moment is:

Basics....
Separation of source and output.
Disallow manual changes to the output.
Isolate build departments as far as practicable.
Inspectable development output.


Strategy...
Simple as possible, but no simpler.
As few tools as possible, but no fewer.
Make a pipeline, not a web.
Document each pipeline well.
If you don't need it, don't put it in.
One shot/Development, Packed/Development.


Ergonomics...
Don't crash the watcher.
Report errors effectively.
Incrementalism.
Facilitate Isolated Workflows.

Bhaal
Jul 13, 2001
I ain't going down alone
Dr. Infant, MD
This might be out of scope for what you're doing but one possibly related topic would be standardizing environments w/ vagrant, docker, etc. Some of the pain of build systems to me can be trying to get it running for many different workstations and running into versioning problems of all the build tools. Always fun to finally discover the reason Julie's build is bombing out is because way upstream she somehow had an incompatible gem version installed for compass. (This could be rolled into a "Checklists for troubleshooting unhelpful error messages" section too I suppose)

Bhaal fucked around with this message at 23:44 on Aug 3, 2015

Maluco Marinero
Jan 18, 2001

Damn that's a
fine elephant.
I'm more focussing on more the front end/node/JavaScript side of the project but it indeed makes sense to give Vagrant and the like an honourable mention. At the very least I should probably mention that you never build with global system packages, like a sass or compass library installed for the global command line. Great way to waste hours when you come back to the project weeks/months later.

M31
Jun 12, 2012
I hate that...

... NPM executes random code it downloads from the internet

... NPM uses symantic versioning to resolve dependencies by default and will randomly break your application if you don't use shrinkwrap or commit everything into a VCS

... the build system needs to execute random code to infer which tasks exist

... every tool uses it's own configuration file that does not support profiles and since it's JSON also does not allow any comments

... all non-package include paths are relative

... Gulp has a terrible syntax. Also the inability to create different profiles (eg. for a debug build) in a sane manner. Double that when you want hot reloading and basically have to write your configuration twice.

... sourcemaps are implemented in one thousand different ways and the tool you use doesn't support it

... intergration of different tools in general is terrible

... people revert to loving shell scripts which won't work on Windows without jumping through hoops that are on fire and are just plain terrible

... creating a proxy for you REST api / legacy resources is not trivial

... most tools only have an api description and zero getting started documentation

... there is no dependency check when building by default, so if you for example run 'npm update' (which seems like a reasonable thing to do) or try a different version and forget to revert, nothing will warn you that you are using completely different versions from those in package.json

Maluco Marinero
Jan 18, 2001

Damn that's a
fine elephant.
Nice! I will definitely be addressing profiles, although I just called it config it amounts to that, my main two switches are watching/single build and unpacked/packed (minify, concat, etc) but it would be cool to have more of a profile bent on it.

luchadornado
Oct 7, 2004

A boombox is not a toy!

M31 posted:

... people revert to loving shell scripts which won't work on Windows without jumping through hoops that are on fire and are just plain terrible

Between UAC, things being compiled against ancient versions of Visual Studio, etc., the node experience in Windows is just lovely. All the node guys are using Macbooks to make these tools, and your life is easier if you do the same, unfortunately.

IAmKale
Jun 7, 2007

やらないか

Fun Shoe

Helicity posted:

Between UAC, things being compiled against ancient versions of Visual Studio, etc., the node experience in Windows is just lovely. All the node guys are using Macbooks to make these tools, and your life is easier if you do the same, unfortunately.
My favorite part about using Node on Windows is when you try to delete packages manually but you can't because the file tree is so deep that you hit the filepath character limit. Cue diving a dozen layers deep only to cut-paste some subfolders a dozen levels higher, then cross your fingers and hit delete. And that's just for the first dependency of some other module that you still need to delete.

The Node people know about this issue but I don't know that anything significant has been done to address it since this statement from one of the NPM higher-ups back in June 2014: https://github.com/joyent/node/issues/6960#issuecomment-46704998

Skandranon
Sep 6, 2008
fucking stupid, dont listen to me

Karthe posted:

My favorite part about using Node on Windows is when you try to delete packages manually but you can't because the file tree is so deep that you hit the filepath character limit. Cue diving a dozen layers deep only to cut-paste some subfolders a dozen levels higher, then cross your fingers and hit delete. And that's just for the first dependency of some other module that you still need to delete.

The Node people know about this issue but I don't know that anything significant has been done to address it since this statement from one of the NPM higher-ups back in June 2014: https://github.com/joyent/node/issues/6960#issuecomment-46704998

It's really annoying how both Microsoft and Node continue to point at each other as at fault and refuse to do anything about it. I have had luck using npm uninstall getting rid of impossible to delete folders, as it will wipe out the entire chain.

piratepilates
Mar 28, 2004

So I will learn to live with it. Because I can live with it. I can live with it.



The deep nesting issue is being resolved in NPM 3, which has a beta version out for it now.

luchadornado
Oct 7, 2004

A boombox is not a toy!

Karthe posted:

My favorite part about using Node on Windows is when you try to delete packages manually but you can't because the file tree is so deep that you hit the filepath character limit. Cue diving a dozen layers deep only to cut-paste some subfolders a dozen levels higher, then cross your fingers and hit delete. And that's just for the first dependency of some other module that you still need to delete.

The Node people know about this issue but I don't know that anything significant has been done to address it since this statement from one of the NPM higher-ups back in June 2014: https://github.com/joyent/node/issues/6960#issuecomment-46704998

Cygwin rm -rf is the best answer in my mind, but yeah, stupid poo poo.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

M31 posted:

I hate that...

stuff

Three are good points.

To be honest I gave up development in Windows several years ago because it's such a pile of crap. Now I develop in a VMware VM running Ubuntu on a Windows host OS and it's like a million times better experience.

Maluco Marinero
Jan 18, 2001

Damn that's a
fine elephant.
How's performance with file watching when running node on say a vagrant box? I'll usually just provision back end stuff, but then the front end build system with watchers and reload just relies on my package.json. Maybe I should be trying to move to full vagrant, more control over environment.

Heskie
Aug 10, 2002

Thermopyle posted:

To be honest I gave up development in Windows several years ago because it's such a pile of crap. Now I develop in a VMware VM running Ubuntu on a Windows host OS and it's like a million times better experience.

Do you run the full GUI inside the VM and work within it (Editor/IDE etc) or just use the VM as a terminal/server and use Editors/IDEs on the Windows host?

I usually work on a Mac, but my new job has a Windows work station so I've been trying to use Windows more to get a better idea of a workflow.

Mr Shiny Pants
Nov 12, 2012

Heskie posted:

Do you run the full GUI inside the VM and work within it (Editor/IDE etc) or just use the VM as a terminal/server and use Editors/IDEs on the Windows host?

I usually work on a Mac, but my new job has a Windows work station so I've been trying to use Windows more to get a better idea of a workflow.

I bought VMware fusion and run a Windows 7 install in OSX on which I use Visual Studio. A maximized window or Unity is wonderful on a 27" screen.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Heskie posted:

Do you run the full GUI inside the VM and work within it (Editor/IDE etc) or just use the VM as a terminal/server and use Editors/IDEs on the Windows host?

I usually work on a Mac, but my new job has a Windows work station so I've been trying to use Windows more to get a better idea of a workflow.

I run a full Ubuntu desktop in the VM and run my IDE in it. I just use the free VMware player.

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb

Heskie posted:

Do you run the full GUI inside the VM and work within it (Editor/IDE etc) or just use the VM as a terminal/server and use Editors/IDEs on the Windows host?

I usually work on a Mac, but my new job has a Windows work station so I've been trying to use Windows more to get a better idea of a workflow.

I'm in the same boat as Thermopyle, Windows as my host OS and 99% of the time I'm in a CentOS VM. I have the full GUI inside the VM, IDE and all. We have a Chef recipe that will take a base install to a full dev environment with just `vagrant up` - installing git, postgresql, tomcat, python, java, etc etc exactly like production. Any changes to the chef recipes can be applied to the VM just with a `git pull && vagrant provision` - and the changes are applied exactly like they get applied in dev/qa/prod. It's pretty sweet!

Thermopyle - how come you went with vmware? I've just been using virtualbox, wasn't sure if I'm missing out on anything.

oversteer
Jun 6, 2005

I'm glad I found this thread, my day job has me hacking about with Ext JS but I'm keen to pick up something else for a little side project I want to set up. You've all convinced me to try React, at first glance it looks pretty neat.

Any views on how popular the different libraries and frameworks are in terms of job postings? I see a lot of Angular used in house, not sure how that maps to the wider world. Ext JS is certainly a funny one now, aimed squarely at the big corporates.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

fletcher posted:


Thermopyle - how come you went with vmware? I've just been using virtualbox, wasn't sure if I'm missing out on anything.

When I moved to this setup a couple years ago there was some issue with Virtualbox that I can't recall the details of.

It's people fixed by now, but I don't have any problems with VMware either...

Heskie
Aug 10, 2002

Thermopyle posted:

I run a full Ubuntu desktop in the VM and run my IDE in it. I just use the free VMware player.

Would you mind sharing your specs and any tips to get the best performance from the guest? Trying this out on my home PC at the moment.

I'm running Windows 10, 8gb, SSD, i5-3750k @ 4.4ghz. Ubuntu 15.05 (In VBox 5) runs well enough, but not quite enough that I'd be happy spending all day in it. For example, I have an issue where sometimes the keyboard input lags, so if I use caps lock I end up typing WORds LIke THIs.

Plorkyeran
Mar 22, 2007

To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed
VMWare generally performs significantly better than VirtualBox.

The Merkinman
Apr 22, 2007

I sell only quality merkins. What is a merkin you ask? Why, it's a wig for your genitals!
http://i.imgur.com/v9prDJt.gif That's pretty much the discussions in this thread right?

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb

Heskie posted:

Would you mind sharing your specs and any tips to get the best performance from the guest? Trying this out on my home PC at the moment.

I'm running Windows 10, 8gb, SSD, i5-3750k @ 4.4ghz. Ubuntu 15.05 (In VBox 5) runs well enough, but not quite enough that I'd be happy spending all day in it. For example, I have an issue where sometimes the keyboard input lags, so if I use caps lock I end up typing WORds LIke THIs.

My laptop is an i7-4600M w/ 16GB ram and an SSD.

I give my VM 2 CPUs, 4GB ram, and enable VT-x/AMD-V as well as "Host I/O Cache". I can't really tell the difference in performance between inside the VM and outside the VM, it is very snappy.

Lumpy
Apr 26, 2002

La! La! La! Laaaa!



College Slice

The Merkinman posted:

http://i.imgur.com/v9prDJt.gif That's pretty much the discussions in this thread right?

No, no, dig up, stupid!

v1nce
Sep 19, 2004

Plant your brassicas in may and cover them in mulch.

Plorkyeran posted:

VMWare generally performs significantly better than VirtualBox.
I'm interested in this too. Any resources you can throw out there to encourage a switch to VMWare over VirtualBox when using Vagrant? Any drawbacks anyone's noticed?

I've been trying out headless VirtualBox on a Surface (yay mobility), and it's adequate, but occasionally grinds to an underwhelming halt which seems to be entirely VirtualBox's fault. One of the biggest issues is sharing disk between the host and the guest, where read/write between them is slow at best, and de-synched at worst. The NFS integration on windows is a real poo poo show.

We're not far away from butchering the server architecture, either. Anyone got any experience using VMWare and Docker for dev?

Maluco Marinero
Jan 18, 2001

Damn that's a
fine elephant.
Yeah, that disk performance is probably my main concern with encapsulating node in vagrant. I notice even with Django dev that the auto reloader within vagrant is considerably slower that if I ran locally.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

All that I can say is that I can tell no difference between working in my VMWare Ubuntu instance and working on the host Windows OS.

Using:
  • i5-4670K (2 cores for VMWare)
  • 32GB RAM (8 GB for VMWare)
  • 256GB Samsung 850 Pro SSD

I can't really think of anything I did to Ubuntu other than just install it.

I should also say that I find this type of setup to be easier with multiple montiors, but then again I don't understand how anyone can do any dev work on a single monitor (or a gently caress-you-huge monitor).

Grim Up North
Dec 12, 2011

Yeah, I have a very similar setup, and the only thing I had to do to get near-native performance was get an Intel NIC because the Realtek on my motherboard turned out to be real poo poo when VMing. (Dropping packets when under load - I didn't try to fix it because 90ies LAN parties had already permanently linked Realtek NIC with being poo poo, and the fact that I bought a motherboard with one was really an easily corrected oversight.)

I guess 3D isn't anywhere natively fast but perfectly usable.

M31
Jun 12, 2012
Just note that you need a license to use VMware Player commercially (obviously you need one for the other products as well if you use them). When I entered my company everyone was using VMware Player 'because it's free' (now they just use Linux).

Heskie
Aug 10, 2002

Thermopyle posted:

All that I can say is that I can tell no difference between working in my VMWare Ubuntu instance and working on the host Windows OS.

Thanks for the info. I'll stop the de-rail now but just wanted to mention I tried VMware instead of Virtualbox tonight and had a big performance boost with the defaults. As you say, it feels like working on my host machine.

v1nce
Sep 19, 2004

Plant your brassicas in may and cover them in mulch.

M31 posted:

Just note that you need a license to use VMware Player commercially
poo poo. I'll have to try it and see if it's worth it over what I'm getting from VirtualBox, but it'll be difficult to persuade the people above me to spend $160 per developer just so we can standardise our dev environments.

Thermopyle posted:

All that I can say is that I can tell no difference between working in my VMWare Ubuntu instance and working on the host Windows OS.

But I'm guessing you've got your code on the VM's storage, and so you're not editing it in the host OS and then having the guest read the files off a "shared drive" kind of deal? That's where Vagrant with VirtualBox really shits the bed on Windows. Everything else I can kind of deal with.

piratepilates
Mar 28, 2004

So I will learn to live with it. Because I can live with it. I can live with it.



v1nce posted:

poo poo. I'll have to try it and see if it's worth it over what I'm getting from VirtualBox, but it'll be difficult to persuade the people above me to spend $160 per developer just so we can standardise our dev environments.

Spend at least $100,000 a year on just having an employee and refuses to spend $160 to increase productivity, that's a good move.

Maluco Marinero
Jan 18, 2001

Damn that's a
fine elephant.
Yah, it's a no brainer if it's stable. 2-3 hours billable cover it and a stable working environment will easily make that back.

v1nce
Sep 19, 2004

Plant your brassicas in may and cover them in mulch.

piratepilates posted:

Spend at least $100,000 a year on just having an employee and refuses to spend $160 to increase productivity, that's a good move.
Fair point. I forget I'm working for a real company now, not an agency.

BlackMK4
Aug 23, 2006

wat.
Megamarm
e: :psypop:

BlackMK4 fucked around with this message at 05:45 on Aug 15, 2015

Kekekela
Oct 28, 2004


Am I correct in interpreting this as meaning that the io/node split is no more?

Adbot
ADBOT LOVES YOU

Urit
Oct 22, 2010

Kekekela posted:



Am I correct in interpreting this as meaning that the io/node split is no more?

You are correct. Apparently io.js is the "canonical" one now.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply