Coding Horrors: You can gather all your technical debt into one easy framework!

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Coding Horrors: You can gather all your technical debt into one easy framework!

«‹›1503 »

redleader: Aug 18, 2005; Engage according to operational parameters

RandomBlue posted:

I almost was for minor crap until today. Thankfully I'm only using it on a personal project that I use with 3 other friends. Unfortunately considering it's a Google Wave clone that uses websockets with a 30 second timeout it's leaking memory at 2MB/minute or 2.8GB per day. A pittance.

Bounce it once or twice a day. Problem solved.

# ? Apr 14, 2017 07:29

Adbot: ADBOT LOVES YOU

# ? Jun 9, 2024 00:06

RandomBlue: Dec 30, 2012; hay guys!; Biscuit Hider

redleader posted:

Bounce it once or twice a day. Problem solved.

Yup. I love it when a plan comes together!

# ? Apr 14, 2017 07:46

McGlockenshire: Dec 16, 2005; GOLLOCKS!

redleader posted:

Bounce it once or twice a day. Problem solved.

Hey, if it's good enough for DHH, it's good enough for anybody.

# ? Apr 14, 2017 08:20

john donne: Apr 10, 2016; All suitors of all sorts themselves enthral;

So on his back lies this whale wantoning,

And in his gulf-like throat, sucks everything

That passeth near.

Every few months I try to make something in Core just to see if it's working yet - it never is, and additionally the entire framework has usually been replaced with a new version. Maybe in 2018 it'll be stable...

# ? Apr 14, 2017 13:39

feedmegin: Jul 30, 2008

RandomBlue posted:

Having so much fun with .Net Core.

Today I found that EF Core doesn't handle GroupBy at all OR Count() right. I mean, who uses group by or count when you can just pull back millions of rows and do all that in your app, right? That's the way almost every PHP app I've ever had the misfortune of checking out their code does it, so it has to be good enough for .Net Core and EF Core, right?

On top of that I noticed a huge slow down after a while where a request was taking around 2 seconds to generate a page. I thought that had to be some unindexed query or something, but no, the queries ran in about 10ms total. Restarted the service with a more detailed logging level and everything was fine. Well gently caress me. That usually means it's a memory leak or something worse.

So I throw together this API call that just returns a very basic JSON OK response and should consume zero additional memory:
code:
        [HttpGet]
        [Route("/api/wtf")]
        public IActionResult Wtf()
        {
            return new OkResult();
        }
Every call allocates 256KB without freeing it. On a web server.

This is .Net Core 1.1.1, the latest "release".

I thought .Net has GC?

# ? Apr 14, 2017 15:49

SirViver: Oct 22, 2008

feedmegin posted:

I thought .Net has GC?

GC doesn't mean an app can't leak memory. Simplified speaking, all a GC does is free memory/variables that it detects as not being referenced anymore (via mechanisms like mark & sweep, reference counting, etc.). If you, however, write a program that inadvertently keeps references around, e.g. via static lists or events that don't get unsubscribed from, then those will cause increasing memory usage over time.

Of course the vexing thing in this case is that the memory leak seems to be part of the .NET Core Web API framework, so you probably can't even do anything about it until MS fixes it.

# ? Apr 14, 2017 16:44

RandomBlue: Dec 30, 2012; hay guys!; Biscuit Hider

Restarted my dotnet core service ~12 hours ago and checked it this morning:

24GB of memory and all it's basically been doing is handling ~8 connections per minute during that time, give or take. Started at about 30MB. How in the gently caress did this big of a memory leak make it to release?

# ? Apr 14, 2017 17:07

Volguus: Mar 3, 2009

RandomBlue posted:

Restarted my dotnet core service ~12 hours ago and checked it this morning:

24GB of memory and all it's basically been doing is handling ~8 connections per minute during that time, give or take. Started at about 30MB. How in the gently caress did this big of a memory leak make it to release?

I don't think they have any QA anymore. You are the tester, enjoy.

# ? Apr 14, 2017 18:48

Kazinsal: Dec 13, 2011

Volguus posted:

I don't think they have any QA anymore. You are the tester, enjoy.

Microsoft truly embracing the open source way

# ? Apr 14, 2017 21:05

feedmegin: Jul 30, 2008

SirViver posted:

GC doesn't mean an app can't leak memory. Simplified speaking, all a GC does is free memory/variables that it detects as not being referenced anymore (via mechanisms like mark & sweep, reference counting, etc.). If you, however, write a program that inadvertently keeps references around, e.g. via static lists or events that don't get unsubscribed from, then those will cause increasing memory usage over time.

I'm aware but the quoted code doesn't do that as far as I can see...

Edit n/m misread the original. Well good job Microsoft :sun:

# ? Apr 14, 2017 22:11

RandomBlue: Dec 30, 2012; hay guys!; Biscuit Hider

feedmegin posted:

I'm aware but the quoted code doesn't do that as far as I can see...

Edit n/m misread the original. Well good job Microsoft

Tried the most basic code I could, just running the basic project generated by "dotnet new web" then using apache bench to generate 1000 hits with 20 concurrent connections and it leaks at ~1MB per 100 hits. Heap size stays about the same after the first batch but private memory keeps going up which tells me it's a leak in the unmanaged native code in the framework itself. Yay.

# ? Apr 14, 2017 22:36

Carthag Tuek: Oct 15, 2005; Tider skal komme,
tider skal henrulle,
sl�gt skal f�lge sl�gters gang

RandomBlue posted:

Tried the most basic code I could, just running the basic project generated by "dotnet new web" then using apache bench to generate 1000 hits with 20 concurrent connections and it leaks at ~1MB per 100 hits. Heap size stays about the same after the first batch but private memory keeps going up which tells me it's a leak in the unmanaged native code in the framework itself. Yay.

What if you make a static OkResult and return the same one each time lol

# ? Apr 14, 2017 22:46

RandomBlue: Dec 30, 2012; hay guys!; Biscuit Hider

Powaqoatse posted:

What if you make a static OkResult and return the same one each time lol

I know you're joking but this is the code from the basic web project scaffold that's being run in my last test:

code:

app.Run(async (context) =>
{
    await context.Response.WriteAsync("Hello World!");
});

It just responds to every request with Hello World, no routing, no logging, nothing being instantiated, etc..

# ? Apr 14, 2017 23:17

Carthag Tuek: Oct 15, 2005; Tider skal komme,
tider skal henrulle,
sl�gt skal f�lge sl�gters gang

Wow, nice. So on top of whatever references it's keeping to your stuff, it's probably also keeping a bunch of stuff inside itself that you can't even reason about. :bravo:

# ? Apr 15, 2017 00:05

xtal: Jan 9, 2011; by Fluffdaddy

RandomBlue posted:

Restarted my dotnet core service ~12 hours ago and checked it this morning:

24GB of memory and all it's basically been doing is handling ~8 connections per minute during that time, give or take. Started at about 30MB. How in the gently caress did this big of a memory leak make it to release?

24 GB ought to be enough for anyone

xtal fucked around with this message at 03:57 on Apr 15, 2017

# ? Apr 15, 2017 03:55

kitten emergency: Jan 13, 2008; get meow this wack-ass crystal prison

they call it .net core because you'll have a coronary when you see the resource usage!!

# ? Apr 15, 2017 04:19

Carthag Tuek: Oct 15, 2005; Tider skal komme,
tider skal henrulle,
sl�gt skal f�lge sl�gters gang

java -Xms24g web.jar

# ? Apr 15, 2017 06:21

Drastic Actions: Apr 7, 2009; FUCK YOU!
GET PUMPED!; Nap Ghost

Mastodon was posted a bit in our Slack, so I wanted to check it out and see if anyone was working on a UWP for it. This app is the only one listed on their GitHub app page.

The code for it is pretty nutty. There are no view models and little, if any, binding. Everything is done in code behind, and done poorly at that.

# ? Apr 15, 2017 23:35

RandomBlue: Dec 30, 2012; hay guys!; Biscuit Hider

Drastic Actions posted:

Mastodon was posted a bit in our Slack, so I wanted to check it out and see if anyone was working on a UWP for it. This app is the only one listed on their GitHub app page.

The code for it is pretty nutty. There are no view models and little, if any, binding. Everything is done in code behind, and done poorly at that.

# ? Apr 16, 2017 02:31

boo_radley: Dec 30, 2005; Politeness costs nothing

Drastic Actions posted:

Mastodon was posted a bit in our Slack, so I wanted to check it out and see if anyone was working on a UWP for it. This app is the only one listed on their GitHub app page.

The code for it is pretty nutty. There are no view models and little, if any, binding. Everything is done in code behind, and done poorly at that.

That can't be how any app implements themes, right?

# ? Apr 16, 2017 05:25

xtal: Jan 9, 2011; by Fluffdaddy

The link following code has a hard coded dark theme check to set the blank loading screen to black instead of white

# ? Apr 16, 2017 15:44

ChickenWing: Jul 22, 2010

Dug into some functionality that I've never really touched while fixing a defect this weekend and found some fun when I went to test it

code:

ApplicationParameter ap = new ApplicationParameter();
ap.setParameterValue("24");
when(applicationParameterService.getParameter(ApplicationParameterConstants.CLIENT_ACCESS_CODE_REQUEST_INTERVAL)).thenReturn(ap);        
ap.setParameterValue("3");
when(applicationParameterService.getParameter(ApplicationParameterConstants.MAX_ACCESS_CODE_RETRY_COUNT)).thenReturn(ap);        
ap.setParameterValue("3");
when(applicationParameterService.getParameter(ApplicationParameterConstants.MAX_CLIENT_ACCESS_CODE_REQUESTS)).thenReturn(ap);
ap.setParameterValue("60");
when(applicationParameterService.getParameter(ApplicationParameterConstants.ACCESS_CODE_EXPIRATION)).thenReturn(ap);

This, in the *single* unit test for a moderately complicated method :eng99:

# ? Apr 17, 2017 15:10

Pixelboy: Sep 13, 2005; Now, I know what you're thinking...

RandomBlue posted:

Having so much fun with .Net Core.

Today I found that EF Core doesn't handle GroupBy at all OR Count() right. I mean, who uses group by or count when you can just pull back millions of rows and do all that in your app, right? That's the way almost every PHP app I've ever had the misfortune of checking out their code does it, so it has to be good enough for .Net Core and EF Core, right?

On top of that I noticed a huge slow down after a while where a request was taking around 2 seconds to generate a page. I thought that had to be some unindexed query or something, but no, the queries ran in about 10ms total. Restarted the service with a more detailed logging level and everything was fine. Well gently caress me. That usually means it's a memory leak or something worse.

So I throw together this API call that just returns a very basic JSON OK response and should consume zero additional memory:
code:
        [HttpGet]
        [Route("/api/wtf")]
        public IActionResult Wtf()
        {
            return new OkResult();
        }
Every call allocates 256KB without freeing it. On a web server.

This is .Net Core 1.1.1, the latest "release".

Seems like a an easy repro case -- have you opened an issue on it? Their GitHub is very, very active.

# ? Apr 17, 2017 21:22

ATM Machine: Aug 20, 2007; I paid $5 for this

Pixelboy posted:

Seems like a an easy repro case -- have you opened an issue on it? Their GitHub is very, very active.

Definitely do lodge stuff on their GitHub, maybe you'll get in before they decide to flipflop back to ~~project.json~~ ~~.csproj~~ ~~.xproj~~ .csproj.

Honestly it sounds like the route has been cached more than anything else, and that ~250kb of memory is the stuff it caches for it, it's stupid that it takes that much memory, but welcome to "release" .Net core, we're still very much of the same mind that it's no where production ready.

The other thing is that .Net core doesn't do the app pool recycling natively like IIS did, which did free memory periodically (unless you disable that in the application pool), by effectively restarting the process for that site.

The EF stuff sounds like you may be doing a context.table.ToArray() or similar and loading the whole table into memory first, then doing a .Where(w => w.what == ever).Count(), do ignore me if I'm wrong. Though it might not be, EF can be weird sometimes, but usually that sort of issue comes from shoving it into memory with a method that makes a collection (ToList, ToArray, ToEtc), before generating the query itself with GroupBy/Where/Select etc.

This totally might not be the case and the query is structured right, but EF is generally rock solid when it creates the query before sending it to the database. That stuff goes out the window when you load a whole table into memory and its relationships, or generates stupid 4000 line queries when you're trying to check if the value of a column is inside an array you give it in a .Where().

But yeah, welcome to .Net core, the never ending cycle of questionable design choices.

# ? Apr 18, 2017 10:35

RandomBlue: Dec 30, 2012; hay guys!; Biscuit Hider

ATM Machine posted:

Definitely do lodge stuff on their GitHub, maybe you'll get in before they decide to flipflop back to ~~project.json~~ ~~.csproj~~ ~~.xproj~~ .csproj.

Honestly it sounds like the route has been cached more than anything else, and that ~250kb of memory is the stuff it caches for it, it's stupid that it takes that much memory, but welcome to "release" .Net core, we're still very much of the same mind that it's no where production ready.

The other thing is that .Net core doesn't do the app pool recycling natively like IIS did, which did free memory periodically (unless you disable that in the application pool), by effectively restarting the process for that site.

The EF stuff sounds like you may be doing a context.table.ToArray() or similar and loading the whole table into memory first, then doing a .Where(w => w.what == ever).Count(), do ignore me if I'm wrong. Though it might not be, EF can be weird sometimes, but usually that sort of issue comes from shoving it into memory with a method that makes a collection (ToList, ToArray, ToEtc), before generating the query itself with GroupBy/Where/Select etc.

This totally might not be the case and the query is structured right, but EF is generally rock solid when it creates the query before sending it to the database. That stuff goes out the window when you load a whole table into memory and its relationships, or generates stupid 4000 line queries when you're trying to check if the value of a column is inside an array you give it in a .Where().

But yeah, welcome to .Net core, the never ending cycle of questionable design choices.

ToListAsync() or the equivalent is the last thing I call with any EF query and even then I only do that because I'm using PostgreSQL and lazy loading isn't supported. It's really just that there's a lot of stuff not yet implemented for EF Core and it falls back to local processing on several things, most notably group by.

Here's a comparison of EF6 to EF Core features: https://docs.microsoft.com/en-us/ef/efcore-and-ef6/features . Moderately complex queries are marked as "Stabilizing", which effectively means broken because you can't count on it.

I do plan on reporting the "memory leaks" though I've found the supposed leak with the barebones kestrel project to cap out at 399MB so it may be less of a memory leak and more of some really really lovely design decisions. There's no reason for a tiny program that doesn't instantiate anything beyond Kestrel and returns a static string for every call and that only uses ~1MB of heap to allocate anywhere near that much memory.

# ? Apr 18, 2017 18:15

Meat Beat Agent: Aug 5, 2007; felonious assault with a sproinging boner

https://twitter.com/RichFelker/status/854421890135461890

# ? Apr 18, 2017 21:55

SupSuper: Apr 8, 2009; At the Heart of the city is an Alien horror, so vile and so powerful that not even death can claim it.

Represent everything as a string, problem solved!

# ? Apr 18, 2017 22:14

Hughlander: May 11, 2005

SupSuper posted:

Represent everything as a string, problem solved!

For an inode that's not a bad idea honestly.

# ? Apr 18, 2017 23:48

Eela6: May 25, 2007; Shredded Hen

The fact that Javascript still doesn't have integers is loving mind-boggling.

# ? Apr 19, 2017 00:14

Volguus: Mar 3, 2009

Eela6 posted:

The fact that Javascript still doesn't have integers is loving mind-boggling.

The fact that Javascript still exists is loving mind-boggling.

# ? Apr 19, 2017 00:29

RandomBlue: Dec 30, 2012; hay guys!; Biscuit Hider

Volguus posted:

The fact that Javascript still exists is loving mind-boggling.

It's totally sweet now though, you can run your whole server on it too!

# ? Apr 19, 2017 00:35

Dex: May 26, 2006; Quintuple x!!!

Would not escrow again.

VERY MISLEADING!

Eela6 posted:

The fact that Javascript still doesn't have integers is loving mind-boggling.

from a comment on that issue:

quote:

There really is no good solution for this until JS gets 64-bit type

i like the optimism behind that 'until'~

# ? Apr 19, 2017 00:37

Evil_Greven: Feb 20, 2007; Whadda I got to,
whadda I got to do
to wake ya up?

To shake ya up,
to break the structure up!?

RandomBlue posted:

It's totally sweet now though, you can run your whole server on it too!

You can even monitor your servers with it - and I can't lie, this setup is very pretty.

# ? Apr 19, 2017 01:36

LOOK I AM A TURTLE: May 22, 2003; "I'm actually a tortoise."; Grimey Drawer

A funny comment in the WPF source code (http://referencesource.microsoft.com/#PresentationFramework/src/Framework/System/Windows/Documents/TextRange.cs,204):

C# code:

void ITextRange.Save(Stream stream, string dataFormat, bool preserveTextElements)
{
    // ATTENTION: This implementation *must* be pure redirect to TextRangeBase. Otherwise TextSelection extensibility is broken
    // DO NOT ANY CODE IN THIS METHOD!
    TextRangeBase.Save(this, stream, dataFormat, preserveTextElements);
}

For the love of God, DO NOT ANY CODE IN THIS METHOD!

# ? Apr 19, 2017 10:44

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Do not ever, under any circumstances, any code.

# ? Apr 19, 2017 12:48

Nude: Nov 16, 2014; I have no idea what I'm doing.

LOOK I AM A TURTLE posted:

A funny comment in the WPF source code (http://referencesource.microsoft.com/#PresentationFramework/src/Framework/System/Windows/Documents/TextRange.cs,204):
C# code:
void ITextRange.Save(Stream stream, string dataFormat, bool preserveTextElements)
{
    // ATTENTION: This implementation *must* be pure redirect to TextRangeBase. Otherwise TextSelection extensibility is broken
    // DO NOT ANY CODE IN THIS METHOD!
    TextRangeBase.Save(this, stream, dataFormat, preserveTextElements);
}
For the love of God, DO NOT ANY CODE IN THIS METHOD!

I know this has been posted before but this is my favorite panic comment, from good old NASA:

code:


TC	BANKCALL	# TEMPORARY, I HOPE HOPE HOPE
CADR	STOPRATE	# TEMPORARY, I HOPE HOPE HOPE

# ? Apr 19, 2017 13:04

feedmegin: Jul 30, 2008

Dex posted:

from a comment on that issue:

i like the optimism behind that 'until'~

JS does have a 64 bit type. IEEE 64 bit floating point! :sun:

# ? Apr 19, 2017 15:42

Doom Mathematic: Sep 2, 2008

feedmegin posted:

JS does have a 64 bit type. IEEE 64 bit floating point!

Just use that 64-bit inode number directly as a float then, problem solved! :sax:

# ? Apr 19, 2017 21:09

Absurd Alhazred: Mar 27, 2010; by Athanatos

We're all floats down here. :cb:

# ? Apr 20, 2017 02:54

Adbot: ADBOT LOVES YOU

# ? Jun 9, 2024 00:06

vOv: Feb 8, 2014

Doom Mathematic posted:

Just use that 64-bit inode number directly as a float then, problem solved!

I know you're just joking but I feel compelled to point out that wouldn't work because there are different ways to encode NaNs depending on what's in the significand and sign bits and JS doesn't let you distinguish between them.

# ? Apr 20, 2017 03:48

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Coding Horrors: You can gather all your technical debt into one easy framework!

«‹›1503 »