Javascript questions which don't deserve their own thread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Javascript questions which don't deserve their own thread.

«‹›277 »

Knifegrab: Jul 30, 2014; Gadzooks! I'm terrified of this little child who is going to stab me with a knife. I must wrest the knife away from his control and therefore gain the upperhand.

Skandranon posted:

If using something like pgPromise, then yes, it should.

I am. I also just started using clustering via the cluster module and this is helping (though obviously that is hardly a scalable solution more of a temp fix);

# ? Jan 13, 2017 00:21

Adbot: ADBOT LOVES YOU

# ? May 16, 2024 17:24

Nolgthorn: Jan 30, 2001; The pendulum of the mind alternates between sense and nonsense

If all you want is "come back after other tasks are done" then just dump whatever you were working on to the bottom of the stack. Node will do everything else and come back around to it. This is not an ideal solution for your case since you are describing intense processing, which should definitely be offloaded out of your main thread.

But in any case, methods for doing so include: `setTimeout(() => { // your code }, 0);`

# ? Jan 13, 2017 01:29

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

I don't do any node stuff, but a pretty common thing in web server land is using a task queue or task manager. For example, when doing stuff with Django, you'd likely use Celery.

The basic idea is that when you need to do a long-running thing in a request, you put a message on a queue like Redis. Then you have workers that fetch that message and do the appropriate work. The user's web request returns quickly because it's basically instantaneous to put the message on the Redis queue.

Brief Googlin' leads me to believe that there are node-celery bindings.

# ? Jan 13, 2017 01:51

Skandranon: Sep 6, 2008; fucking stupid, dont listen to me

Nolgthorn posted:

If all you want is "come back after other tasks are done" then just dump whatever you were working on to the bottom of the stack. Node will do everything else and come back around to it. This is not an ideal solution for your case since you are describing intense processing, which should definitely be offloaded out of your main thread.

But in any case, methods for doing so include: `setTimeout(() => { // your code }, 0);`

This will only clear anything on the queue before you call setTimeout. Once his processing code starts, it will then proceed to block all other queued items until it is done, so this doesn't really change much. He needs to either have a completely separate thread do the work, or slice his processing on the main thread.

# ? Jan 13, 2017 03:17

Knifegrab: Jul 30, 2014; Gadzooks! I'm terrified of this little child who is going to stab me with a knife. I must wrest the knife away from his control and therefore gain the upperhand.

I solved it by rewriting the processing in postgres, so postgres does all the heavy lifting and frees up node to be a server.

# ? Jan 13, 2017 10:30

Knifegrab: Jul 30, 2014; Gadzooks! I'm terrified of this little child who is going to stab me with a knife. I must wrest the knife away from his control and therefore gain the upperhand.

I have some NPM packages that I forgot to init and do proper saving etc on. After I had a bunch of node_modules I did an npm init which threw everythign into the dependency category. Unfortunately a lot of these are devDependencies. Is there a command to change a dependency to be a devDependency, google has failed me.

# ? Jan 16, 2017 19:15

Kekekela: Oct 28, 2004

Knifegrab posted:

I have some NPM packages that I forgot to init and do proper saving etc on. After I had a bunch of node_modules I did an npm init which threw everythign into the dependency category. Unfortunately a lot of these are devDependencies. Is there a command to change a dependency to be a devDependency, google has failed me.

I'd edit package.json to get your dependencies correct then delete node_modules and do a fresh npm install.

# ? Jan 16, 2017 19:36

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Knifegrab posted:

I have some NPM packages that I forgot to init and do proper saving etc on. After I had a bunch of node_modules I did an npm init which threw everythign into the dependency category. Unfortunately a lot of these are devDependencies. Is there a command to change a dependency to be a devDependency, google has failed me.

Just move the lines around in the package.json file.

# ? Jan 16, 2017 19:37

Dominoes: Sep 20, 2007

Why doesn't javascript include more abstractions and convenience functions? Ie it seems like for common operations like checking if an array contains a value, rounding numbers, sorting etc, you need to code most things by hand.

Would there be interest in a basic abstractions/convenience functions library? You could use languages like Python, Haskell, Julia etc for examples. What am I missing? Is this about minimizing memory / storage footprint?

Dominoes fucked around with this message at 02:38 on Jan 17, 2017

# ? Jan 17, 2017 02:22

YO MAMA HEAD: Sep 11, 2007

Sure, do you mind adding it to IE9 when it's done?

# ? Jan 17, 2017 02:24

n4: Jul 26, 2001; Poor Chu-Chu : (

Dominoes posted:

Why doesn't javascript include more abstractions and convenience functions? Ie it seems like for common operations like even for common operations like checking if an array contains a value, rounding numbers, sorting etc, you need to code most things by hand.

Would there be interest in a basic abstractions/convenience functions library? You could use languages like Python, Haskell, Julia etc for examples. What am I missing? Is this about minimizing memory / storage footprint?

Isnt that basically what lodash / underscore do?

# ? Jan 17, 2017 02:25

Maluco Marinero: Jan 18, 2001; Damn that's a
fine elephant.

Yeah, the standard library is poo poo because it needs to be implemented by 4-5 different companies after those 4-5 companies and a bunch of other interested parties go through a slow and laborious standardisation process, and even then one can't assume full penetration for like half a decade or even more.

# ? Jan 17, 2017 02:26

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Dominoes posted:

Would there be interest in a basic abstractions/convenience functions library?

Yes, that's why there's a bunch of them.

# ? Jan 17, 2017 02:28

Dominoes: Sep 20, 2007

Thanks dudes. MM's explanation makes sense, and Lodash/Underscore look exactly like what I felt was missing.

Dominoes fucked around with this message at 02:47 on Jan 17, 2017

# ? Jan 17, 2017 02:41

Strong Sauce: Jul 2, 2003; You know I am not really your father.

Dominoes posted:

Why doesn't javascript include more abstractions and convenience functions? Ie it seems like for common operations like checking if an array contains a value, rounding numbers, sorting etc, you need to code most things by hand.

Would there be interest in a basic abstractions/convenience functions library? You could use languages like Python, Haskell, Julia etc for examples. What am I missing? Is this about minimizing memory / storage footprint?

!!~haystack.indexOf(needle), ~~number, and array_you_want_to_sort.sort(fn) ?

# ? Jan 17, 2017 03:22

Maluco Marinero: Jan 18, 2001; Damn that's a
fine elephant.

The tilde trick is loving disgusting and anyone who uses it in everyday intended to be read code should be ashamed... that is all.

# ? Jan 17, 2017 03:33

Sedro: Dec 31, 2008

Who needs a standard library? We've got npm packages!

# ? Jan 17, 2017 07:04

Wheany: Mar 17, 2006; Spinya^{ha^{ha^haha}ha}ha_{ha_{ha_haha}ha}ha!; Doctor Rope

Dominoes posted:

Thanks dudes. MM's explanation makes sense, and Lodash/Underscore look exactly like what I felt was missing.

I'm pretty sure underscore was my first JavaScript love :swoon:

e: These days I use lodash.

# ? Jan 17, 2017 07:26

Blinkz0rz: May 27, 2001; MY CONTEMPT FOR MY OWN EMPLOYEES IS ONLY MATCHED BY MY LOVE FOR TOM BRADY'S SWEATY MAGA BALLS

Maluco Marinero posted:

The tilde trick is loving disgusting and anyone who uses it in everyday intended to be read code should be ashamed... that is all.

qft

# ? Jan 17, 2017 13:36

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Strong Sauce posted:

!!~haystack.indexOf(needle), ~~number, and array_you_want_to_sort.sort(fn) ?

lol if you don't have to constantly remind yourself that nice things like indexOf are widely supported on arrays now

just... lol :smith:

# ? Jan 17, 2017 14:18

Wheany: Mar 17, 2006; Spinya^{ha^{ha^haha}ha}ha_{ha_{ha_haha}ha}ha!; Doctor Rope

Munkeymon posted:

lol if you don't have to constantly remind yourself that nice things like indexOf are widely supported on arrays now

just... lol

Uhh.. IE8 or earlier?

:whoptc:

# ? Jan 17, 2017 14:30

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Wheany posted:

Uhh.. IE8 or earlier?

I just spent too long avoiding transpilers and having to support lovely old versions of IE, so I constantly have to remind myself that the language is bigger than I'm used to now.

# ? Jan 17, 2017 15:13

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Dominoes posted:

Thanks dudes. MM's explanation makes sense, and Lodash/Underscore look exactly like what I felt was missing.

You may also want to transpile your code with something like babel or whatever. That lets you use ES(201)6 javascript wherever, and it's pretty great. On the other hand you have to add a build step to your javascripting, so sometimes its not worth it.

# ? Jan 17, 2017 17:53

Skandranon: Sep 6, 2008; fucking stupid, dont listen to me

Thermopyle posted:

You may also want to transpile your code with something like babel or whatever. That lets you use ES(201)6 javascript wherever, and it's pretty great. On the other hand you have to add a build step to your javascripting, so sometimes its not worth it.

For web app development, I've always found it worthwhile to put in the time to use TypeScript, but for trivial things not so much. It also helps that NodeJS now supports ES6 natively.

# ? Jan 17, 2017 17:58

Blinkz0rz: May 27, 2001; MY CONTEMPT FOR MY OWN EMPLOYEES IS ONLY MATCHED BY MY LOVE FOR TOM BRADY'S SWEATY MAGA BALLS

Skandranon posted:

For web app development, I've always found it worthwhile to put in the time to use TypeScript, but for trivial things not so much. It also helps that NodeJS now supports ES6 natively.

Only some of it: http://node.green

# ? Jan 18, 2017 00:30

ROFLburger: Jan 12, 2006

Blinkz0rz posted:

Only some of it: http://node.green

99% of es6, over the last 7 releases is "only some"?

# ? Jan 18, 2017 20:30

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

ROFLburger posted:

99% of es6, over the last 7 releases is "only some"?

The last four versions of Node are all less than a year and a few months old and 7 is only three months, so this is all still pretty new for the project.

# ? Jan 18, 2017 21:29

ROFLburger: Jan 12, 2006

I misunderstood

# ? Jan 19, 2017 00:12

mekkanare: Sep 12, 2008; We have detected you are using ad blocking software.

Please add us to your whitelist to view this content.

I'm working on porting a markov chat bot I wrote a while back from C++ to JS, and I'm in a bit of a pickle.
Instead of keeping the entire DB in memory and writing to a text file, which is what I did originally, I am trying to
keep everything into a mongoDB and query to and from it using mongoose when I need to. As it stands,
I have the writing to the DB fine and dandy. However now what I want to do is the following:

Query the word db with a prefix.
Append the prefix + a word chosen from the list of suffixes to a variable.
Use the last word from the prefix and the suffix to query the word db again.
Repeat until reaching a length limit or there's no suffixes found with the current prefix query.
Store the whole variable from step 2 into a sentence db.

As I've found out, the queries are asynchronous, and I'm completely new to JS. I've managed to avoid nesting anonymous functions
when using asynchronous calls at the very least!

My question: Is there a way to build a variable using a loop/recursion built from mongoose queries that are promises?

# ? Jan 19, 2017 21:55

Skandranon: Sep 6, 2008; fucking stupid, dont listen to me

mekkanare posted:

I'm working on porting a markov chat bot I wrote a while back from C++ to JS, and I'm in a bit of a pickle.
Instead of keeping the entire DB in memory and writing to a text file, which is what I did originally, I am trying to
keep everything into a mongoDB and query to and from it using mongoose when I need to. As it stands,
I have the writing to the DB fine and dandy. However now what I want to do is the following:

Query the word db with a prefix.

Append the prefix + a word chosen from the list of suffixes to a variable.

Use the last word from the prefix and the suffix to query the word db again.

Repeat until reaching a length limit or there's no suffixes found with the current prefix query.

Store the whole variable from step 2 into a sentence db.

As I've found out, the queries are asynchronous, and I'm completely new to JS. I've managed to avoid nesting anonymous functions
when using asynchronous calls at the very least!

My question: Is there a way to build a variable using a loop/recursion built from mongoose queries that are promises?

Yes, there are ways to do such a thing, though I think you have to use recursion, can't think of a way to do it purely with a loop, as there's no way to block your loop.

Well, there's a way to do this with promises. I don't know if mongoose is giving you promises. However, even if forced into success/fail callbacks, you should be able to wrap that up yourself into promises if sufficiently motivated.

# ? Jan 20, 2017 00:31

mekkanare: Sep 12, 2008; We have detected you are using ad blocking software.

Please add us to your whitelist to view this content.

Skandranon posted:

Yes, there are ways to do such a thing, though I think you have to use recursion, can't think of a way to do it purely with a loop, as there's no way to block your loop.

Well, there's a way to do this with promises. I don't know if mongoose is giving you promises. However, even if forced into success/fail callbacks, you should be able to wrap that up yourself into promises if sufficiently motivated.

Alright, I'm mainly struggling with how to make it waiting until the query returns. I'll have to look into Promises some more first.

# ? Jan 20, 2017 00:50

Skandranon: Sep 6, 2008; fucking stupid, dont listen to me

mekkanare posted:

Alright, I'm mainly struggling with how to make it waiting until the query returns. I'll have to look into Promises some more first.

Should look something like this.

code:

var words = [ "word1", "word2"];
var prefix = "p";

var doThing = (index) => {
	var word = prefix + words[index];
	makeRequest(word).then( (response) => {
		if (doneCondition) {
			// do the last thing
		}
		else
			doThing(index+1);
	});
}

// this starts it off at beginning
doThing(0);

# ? Jan 20, 2017 07:23

Roadie: Jun 30, 2013

mekkanare posted:

Alright, I'm mainly struggling with how to make it waiting until the query returns. I'll have to look into Promises some more first.

The key thing with Promises is that anything you return inside the Promise's callback gets supplied to the next thing in the chain (e.g. the callback you supply to whateverPromiseFunction.then()), unless what you return is a Promise itself, in which case it gets resolved first and the resolved value gets treated like it was the return value.

This lets you do stuff in an asynchronous way while still putting a specific flow on the overall process, so you can do stuff like...

code:

function somePromise () {
  return new Promise((resolve, reject) => {
    apiFunctionWithCallback(someApiValue => {
      resolve(someApiValue)
    })
  }).catch(err => {
    console.error('OH GOD EVERYTHING IS ON FIRE:', err)
  })
}

function returnsThing1 () {
  return somePromise.then(someApiValue => {
    return 'I love cats'
  }).catch(err => {
    console.error('OH GOD EVERYTHING IS ON FIRE:', err)
  })
}

function returnsThing2 () {
  return returnsThing1.then(iLoveCats => {
    return someOtherPromise('I love dogs')
  }).catch(err => {
    console.error('OH GOD EVERYTHING IS ON FIRE:', err)
  })
}

function returnsThing3 () {
  return returnsThing2.then(iLoveDogs => {
    return 'I love fish'
  }).catch(err => {
    console.error('OH GOD EVERYTHING IS ON FIRE:', err)
  })
}

...or...

code:

new Promise((resolve, reject) => {
  apiFunctionWithCallback(someApiValue => {
    resolve(someApiValue)
  })
}).then(someApiValue => {
  // do an operation here
  return 'I love cats'
}).then(iLoveCats => {
  // do an operation here
  return someOtherPromise('I love dogs')
}).then(iLoveDogs => {
  // do an operation here
  return 'I love fish'
}).catch(err => {
  console.error('OH GOD EVERYTHING IS ON FIRE:', err)
})

...and that way you can break out a bunch of stuff into individual Promises that each operate independently but that you can still chain together easily, without things getting nested ten levels deep like the classic callback hell style.

Roadie fucked around with this message at 09:32 on Jan 20, 2017

# ? Jan 20, 2017 09:29

mekkanare: Sep 12, 2008; We have detected you are using ad blocking software.

Please add us to your whitelist to view this content.

I got it to successfully query from the db and form a sentence. Turns out I didn't have it return the complete sentence in the promise.
Thanks for the simple examples! I started with them and built my way up to what code I had before and compared.
Now I just want to make sure to comment it properly because I will forget by Monday how the heck this works.

# ? Jan 20, 2017 23:23

fankey: Aug 31, 2001

I'm just starting to play around with node and I'm not clear exactly how the gc determines lifetimes of object. The first thing I've implemented is a web socket -> TCP bridge using the following code

code:

const net = require('net');

module.exports = {

  // called from WebSocketServer.on('connection', function(ws)
  // passing in newly created web socket
  add: function (ws) 
  {
    client = net.connect({ host: 'localhost', port: 1710 }, () => { console.log("connected to runtime!") });

    client.on('data', (chunk) => { ws.send(data); });

    client.on('error', () => { console.log('client.error'); });

    client.on('end', () => { console.log('client.end'); });

    client.on('close', () => {
      console.log('client.close');
      ws.terminate();
    });

    ws.on('message', function incoming(message) {
      console.log('<= %s', message);
      client.write(message + '\0')
    });

    ws.on('error', (er) => {
      console.log("ws.error %s", er);
    })
    ws.on('close', () => {
      console.log("ws.close");
      client.end();
    })
  }
};

Pretty simple - every time a new web socket comes in ( via require('ws').Server ) I create a TCP socket and set up handlers on both the socket and ws to shovel data between them. If an error occurs on either one I close down the other side. It all pretty much works but I'm not clear what I need to do with regards to clearing stuff out so the gc can delete the objects.

In my code I'm not storing any references to either the ws or socket - they appear to go out of scope when add() returns. At this point are they eligible for gc and could be yanked out from under me? If so, should I just store them in a collection manually? If they somehow aren't eligible for gc then I have the opposite quesiton - what do I need to do once I'm done with both connections to ensure the gc will clean up memory for me? Or do things just magically work and I don't need to worry about such things - which is hard for someone with years of C++ and C# development to come to terms with.

# ? Jan 26, 2017 22:27

HappyHippo: Nov 19, 2003; Do you have an Air Miles Card?

The inner functions create closures which will hold references to the local values in the context they were created in. The GC can't clean them up until the references to those functions are themselves cleaned up.

In general, when it comes to JavaScript objects, everything just "works," the GC will never clean up something that's still reachable, and will clean stuff up when it's no longer reachable.

# ? Jan 26, 2017 22:59

HappyHippo: Nov 19, 2003; Do you have an Air Miles Card?

Definitely look up closures if you aren't familiar with the concept. Understanding them is essential to programming in JavaScript.

# ? Jan 26, 2017 23:02

fankey: Aug 31, 2001

HappyHippo posted:

The inner functions create closures which will hold references to the local values in the context they were created in. The GC can't clean them up until the references to those functions are themselves cleaned up.

I understand that in

code:

client.on('data', (chunk) => { ws.send(data); });

closures are used so that ws is not destroyed as long as that function is attached to the socket event handler. In my code I think i have

ws event handlers which use closures containing the socket
socket event handlers which use closures containing the ws

so it makes sense that the socket will not be removed until the ws is and the ws won't be removed until the socket is. What's not clear to me what is making the ws still reachable after the add() method is returned. The WebSocketServer has an option to retain a list of clients but if I don't do that I don't think there is anything that tethers the ws to the global scope.

# ? Jan 26, 2017 23:20

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

It's likely that modern JS engines could GC that, I think. That one named function might defeat GC on that object, though, and it'd also depend on the internal behavior of a WebSocket.

See http://stackoverflow.com/a/35813427/301807

# ? Jan 26, 2017 23:23

Adbot: ADBOT LOVES YOU

# ? May 16, 2024 17:24

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

fankey posted:

What's not clear to me what is making the ws still reachable after the add() method is returned.

The handler functions that use it keep a reference* as long as they're around - that's the closure. They'll exist as long as something references them. When nothing references them, they should be GCed, but this is all engine-dependant and some are smarter than others.

# ? Jan 26, 2017 23:30

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Javascript questions which don't deserve their own thread.

«‹›277 »