.NET Megathread 3.5: await GetGoodPosts()

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > .NET Megathread 3.5: await GetGoodPosts()

«‹›3 »

redleader: Aug 18, 2005; Engage according to operational parameters

Malcolm XML posted:

the difference between Ienumerator and ienumerable is subtle and almost always you actually want the former

Any chance you could elaborate on this?

# ¿ Jun 21, 2014 23:18

Adbot: ADBOT LOVES YOU

# ¿ May 2, 2024 11:16

redleader: Aug 18, 2005; Engage according to operational parameters

Scaramouche posted:

Hey guys, nice shiny new thread. Let's get it dirty with some idiot posting.

The problem:
I'm using StringBuilder to concatenate a whole biggo bunch of strings, converting that StringBuilder.tostring, and then writing the resultant string to an XML file. I'm sure many of you know what's coming next since I said "biggo" and "StringBuilder"; that's right, the dreaded 'System.OutOfMemoryException' caused by StringBuilder over-allocating contiguous memory.

So the first question is obviously, any sneaky ways around this? The file is about 94mb raw text, but it can also fail randomly on smaller files (around 50mb), so 94mb is not a 'magic number' to stay under by any means. I've tried assigning double that to the StringBuilder when instantiating, but all that does is give me the memory error right up front at variable declaration instead of at the end when trying to convert to string. Based on what I can figure out it's a Bad Idea to even use StringBuilder this way anyway, and most of the workarounds are pretty dodgy.

So assuming the answer to the first question is "no", my second question is advice on chunking up this monster. Right now things are structured like so (this is not real code):
code:
Protected Sub Page_Load() Handles Me.Load
 dsThinger = Big database call to grab about 80,000 rows

 MakeXML(dsThinger.Tables(0))
End Sub

Protected Sub MakeXML(ByVal dtThinger As DataTable)
 Dim sbThinger as New StringBuilder

 sbThinger.AppendLine("<?xml version=""1.0"" ?>") 'etc start tags

 for each drThinger as datarow in dtThinger
   'About 2000 lines of if/then, case statements, value lookups, and translations, most of which look like:
  If drThinger("value")="other value" then
   sbThinger.AppendLine("<Thinger>whatever value=othervalue means</Thinger>")
  End If
   'Repeat 80,000 times
 Next
 
 sbThinger.AppendLine("stuff") 'etc end tags
 PostXMLDocument(sbThinger.ToString) '<-- Failure Point 1
End Sub

Protected Sub PostXMLDocument(ByVal strThinger as String)
 Dim xmlDoc As New XmlDocument
 Dim filePath As String = "c:\Feeds\Outbox\" & Left(System.Guid.NewGuid.ToString, 8) & ".xml"
 xmlDoc.LoadXml(strThinger) '<-- Failure Point 2, sometimes
 xmlDoc.Save(filePath)
 xmlDoc = Nothing
 xml = Nothing

 'API calls, etc. to publish the XML data
End Sub
One of the things I tried is having PostXMLDocument accept StringBuilder as a variable directly, but all that does is move the point of failure to Failure Point 2, because LoadXml can't overload to use a StringBuilder as input, this being kind of a dumb thing to try in the first place.

The obvious chunking candidate is the big SQL query, but that's kind of a pain because A)it's not my query and B)it's reused to generate OTHER files. However, on the other hand, I can't chunk the XML itself too much, because the publisher only accepts (x) amount of connections at once, dumps the ones over that, and this is relatively timely information that has to be generated twice a day.

The last thing I thought of is to chunk the stringbuilder itself before converting to string (and being able to go xml = string1 & string2 & string3), but by the time the stringbuilder is 'done' it's too big to modify since anything I've found to modify it either implicitly converts it to string, or explicitly requires it done.

I'm not looking to you guys to 'solve' the stringbuilder memory problem; I'm fully aware it's the result of bad design on my part. More trying to pick brains for possible workarounds.

EDIT-I also looked into splitting the datatable itself and then passing the table chunks in MakeXML, but the ways I've seen to do that seem pretty inefficient: http://stackoverflow.com/questions/8132208/c-how-to-partitioning-or-splitting-datatable

Sounds like an ideal case for System.Xml.XmlWriter! Plus, using an XmlWriter means you aren't building up XML by artisanal hand-crafted string concatenation, which is a bit gross

# ¿ Jun 27, 2014 10:41

redleader: Aug 18, 2005; Engage according to operational parameters

Does anyone use the VS ASP.NET template projects (the ones that generate a whole ton of controllers/views/etc for you) when starting a new project? Do you create a project using the MVC/Web API/SPA templates and modify it as required, or do you start from a blank project and add things according to your own requirements and ideas?

# ¿ Jul 12, 2014 09:00

redleader: Aug 18, 2005; Engage according to operational parameters

kingcrimbud posted:

Why would a sql query generated from Entity Framework take minutes to execute, yet when the same query is run within SSMS, it takes less than a second?

Entity Framework 6.1.3
SQL Server 2005 Enterprise

I'm grabbing the sql generated using the Infrastructure Interception DatabaseLogger registered in web.config. I'm unable to trace or profile in SSMS due to a current lack of permissions so I don't have any other details yet.

This might be because the execution plan used for the EF query is different to the one used in the SSMS query - SQL Server generates and caches execution plans based on a whole bunch of factors, and it's possible that the SSMS query has different SET options that renders SQL Server unable to reuse the plan from the EF query. Greybeard Erland Sommarskog goes into a hell of a lot of detail on this subject.

# ¿ Dec 5, 2015 02:54

redleader: Aug 18, 2005; Engage according to operational parameters

Kekekela posted:

I was under the impression the distinctions he is making regarding query plans haven't been true since early versions of Sql Server, and everything gets query plans generated now. (I know that's beside the point of why you linked it, its just something that I found curious)

Yeah, I thought the same thing. A close reading of that link reveals "If your application does not use stored procedures, but submits SQL statements directly, most of what I say this chapter is still applicable", which I interpret as meaning 'this applies equally to autogenerated SQL and suchlike'. It's been a while since I've run into a slow-in-app-fast-in-SSMS query though.

# ¿ Dec 6, 2015 04:23

redleader: Aug 18, 2005; Engage according to operational parameters

sarehu posted:

I have a question. Is writing C# that doesn't obey Microsoft's C# Coding Conventions actually a thing that real people do?

We mostly follow the MS guidelines, but use K&R-style braces rather than Allman. I kinda like it.

# ¿ Feb 13, 2016 23:08

redleader: Aug 18, 2005; Engage according to operational parameters

ljw1004 posted:

Here instead is an implementation of Whatever that might throw in both places. It achieves this by a wrapper method that lacks the async modifier.
code:
Task Whatever(string s) {
   if (s == null) throw new ArgumentNullException(named(s));
   return WhateverInner(s);
}

async Task WhateverInner(string s) {
   if (s.length < 1) throw new xyz;
   await Task.Delay(10);
   Throw new abc;
}
Some folks believe this is good practice. They are wrong. If you find anyone, report them, and they can be sent to the salt puts for education.

Stupid newbie question, but why is this considered bad practice? It looks very similar to the sort of thing you'd do when writing an iterator method:

C# code:


public static IEnumerable<string> Repeat(string s) {
    if (s == null) { throw new ArgumentNullException();}
    return RepeatInner(s);
}

private static IEnumerable<string> RepeatInner(string s) {
    while (true) { yield return s; }
}

I've never really had much chance to play around with async/await.

# ¿ Oct 20, 2016 08:07

redleader: Aug 18, 2005; Engage according to operational parameters

ljw1004 posted:

My other biggest takeaway has been unit testing. Honestly, I hate unit testing. I've never got into it, and almost every bug in my own code has been one that would require an integration-test or end-to-end-test to catch (i.e. unit tests wouldn't have helped). I guess compilers are notoriously poor candidates for unit tests too. But I've dutifully watched every video posted here about it, read every link, seen what people are doing, and learnt the principles.

I'm a little surprised to hear this, as I'd have thought a compiler is nearly a perfect candidate for all sorts of tests, including unit tests. You know, easily defined input/output, essentially zero external dependencies...

Are you going to be working with Eric Lippert at FB?

# ¿ Nov 15, 2016 00:52

redleader: Aug 18, 2005; Engage according to operational parameters

See if you can whip up a simple reproduction and create an issue for it on the Roslyn GitHub. There isn't anything obviously wrong with your code, so from what I know of local functions it should just work. I'd be interested to hear what the issue ends up being.

# ¿ Nov 28, 2016 03:43

redleader: Aug 18, 2005; Engage according to operational parameters

Note that the SQL Server query planner/optimizer can only do so much to optimize a poor query. If it takes too long to generate a good plan, it will hit a timeout and just use the best plan it was able to generate in that time. A crappy query might just be too hard to optimize well in the time available.

I've often been able to improve the performance of queries by taking a big SQL statement and breaking it into smaller, easier-to-optimize chunks. You can often do a lot with a strategic temporary table and some knowledge of the schema and how much relevant data is in each table.

# ¿ Dec 7, 2016 00:29

redleader: Aug 18, 2005; Engage according to operational parameters

Sab669 posted:

The issue is that we paid some Indian firm to rewrite our entire application last year. It's pretty large. And it's basically just me that maintains it.

So I have no idea what they did. Not intimately familiar with ASP MVC, although I understand MVC principals. Third party controls I've never used before.A handful of custom controls that I have minimal understanding of how it was cobbled together. The entire application is all stitched together so that page virtually never fully reloads, it just makes Ajax calls and updates the HTML of one div. So since the URL in the address bar never changes I don't really get how / why anything should ever be sent as a Get request in our case.

Pick up Fiddler (or a similar web debugging proxy), and watch/gently caress around with HTTP requests in real time.

If you know what "page" makes a request but can't pinpoint the JS, use Chrome's dev tools to set a breakpoint on the AJAX XHR. Once you hit the breakpoint, you can easily look through the stack trace and find what function made the AJAX request. I'm hoping that your poo poo isn't totally hosed and that you'd be able to locate that function in an actual .js file somewhere.

# ¿ Jan 10, 2017 09:02

redleader: Aug 18, 2005; Engage according to operational parameters

Could you add a last modified column or similar?

# ¿ Feb 21, 2017 07:42

redleader: Aug 18, 2005; Engage according to operational parameters

Just create local duplicates of the local databases! Add date modified columns, run a process regularly to compare the original local db with your duplicate, and update the timestamp of any modified rows.

# ¿ Feb 22, 2017 00:27

redleader: Aug 18, 2005; Engage according to operational parameters

NihilCredo posted:

What benefits would this (vastly more complex) solution offer over a SHA256 hash?

Well, by wasting (a hopefully significant amount of) disk space on the local drive, he could encourage the team in charge of the local databases to fix their design! And it's more secure if SHA256 gets cracked!!!

Nah, it doesn't have any advantages over the smarter solution. I've just had a horrible experience with a similar design (concat and hash a bunch of fields straight from the db).

# ¿ Feb 22, 2017 09:17

redleader: Aug 18, 2005; Engage according to operational parameters

And within 5 years, MS will introduce another UI framework and abandon their efforts on UWP. Such is life.

# ¿ Mar 31, 2017 23:04

redleader: Aug 18, 2005; Engage according to operational parameters

wilderthanmild posted:

So I am not sure if this is the right thread for this, but it's a C# Web Api project, so I am not sure where would be better.

I am trying to add a rewrite rule to my API that forces HTTPS, except when I am running locally for debugging. It doesn't seem to be working.

Here is what I have:
code:
                <rule name="HTTP to HTTPS redirect" stopProcessing="true">
                    <match url="(.*)" />
                    <conditions  logicalGrouping="MatchAll">
                        <add input="{HTTP_HOST}" pattern="localhost" negate="true" />
                        <add input="{HTTPS}" pattern="off" ignoreCase="true" />
                    </conditions>
                    <action type="Redirect" url="https://{HTTP_HOST}/{R:1}"
                        redirectType="Permanent" />
                </rule>        
Now it does work for rewriting all requests to https, but it's still rewriting them when locally debugging on localhost. I am almost wondering if IIS Express is just using the first version of the rule I put out and ignoring my changes, because even if I completely remove the rewrite section in the web.config, it still redirects me to https.

Is there something wrong with the rule above?

I've had hellish experiences with IIS Express where it just refuses to stop caching something. As much as it pains me to say this, but have you tried rebooting?

# ¿ Apr 13, 2017 23:23

redleader: Aug 18, 2005; Engage according to operational parameters

hirvox posted:

LINQ to Sharepoint provider

# ¿ Apr 14, 2017 21:52

redleader: Aug 18, 2005; Engage according to operational parameters

LOOK I AM A TURTLE posted:

A reason to use return await is that it helps with return type covariance, for instance when you want to return IList<T> instead of List<T>. I encounter this a lot when working with EF queries.
C# code:
public Task<IList<Entity>> GetEntitiesAsync()
{
	return Context.Entities.ToListAsync(); // compile error (cannot implicitly convert Task<List<Entity>> to Task<IList<Entity>>)
}

public async Task<IList<Entity>> GetEntitiesAsync()
{
	return await Context.Entities.ToListAsync(); // works
}

I'm a little surprised that Task<T> isn't covariant, but I'm sure there's a good reason for it.

# ¿ May 6, 2017 00:51

redleader: Aug 18, 2005; Engage according to operational parameters

It's a total mess. You'd need to be involved in dealing with new .NET stuff on a daily basis to keep up with the constant stream of changes, new platforms, renamed terminology, deprecations, ...

You'd need to be hella motivated to keep up with .NET in your day job, let alone in your own time.

# ¿ May 10, 2017 09:36

redleader: Aug 18, 2005; Engage according to operational parameters

Iverron posted:

"is this shop still running 4.x.x?" years from now when job hunting.

"Yes. We have no plans to change."

# ¿ May 10, 2017 20:17

redleader: Aug 18, 2005; Engage according to operational parameters

You could also cross-post to the DB thread, since they've probably seen similar stuff.

Do the timeouts always occur at the same time of day? Do you run any periodic processes (in or out of the DB)?

# ¿ May 12, 2017 23:06

redleader: Aug 18, 2005; Engage according to operational parameters

I've run into an weird/annoying problem trying to get a Winforms app working how (I think) it should.

I'm trying to write a Winforms program to run as a TortoiseSVN client-side post-commit hook. I've created a new Winforms project using the default project template. This runs just fine from VS, and shows the (empty, completely untouched Form1) as you'd expect.

If I set this same executable as a TortoiseSVN post-commit hook, the form simply doesn't show up. The program runs - I can see it in Task Manager (and have to kill it manually) - but the main form just doesn't display.

However, if I then add a call to MesageBox.Show() before Application.Run() and run the program as a post-commit hook, the message box pops up, then the main form displays.

Does anyone have any idea why showing the message box causes the form to show properly, or what I need to change to get the form to display when called from TortoiseSVN?

# ¿ Jun 5, 2017 07:25

redleader: Aug 18, 2005; Engage according to operational parameters

SirViver posted:

This may be a stupid question, but did you make sure to have the "Hide the script while running" hook script option turned off? That's usually there to prevent the command window from popping up when executing a batch file or console application, but could also affect your WinForm. Showing the MessageBox might simply interfere with TortoiseSVN's attempt to hide the main process window, or reset some flag that caused it to be hidden before.

TortoiseSVN hangs with "hide script while running" turned on, presumably because it's waiting for the (invisible?) form to close. It works fine with the message box.

e: haha, gently caress. Figured it out - I needed to uncheck "hide this script". Annoyingly, I have another hook that uses Winforms that needs this option selected. God, how dumb.

redleader fucked around with this message at 22:33 on Jun 5, 2017

# ¿ Jun 5, 2017 22:26

redleader: Aug 18, 2005; Engage according to operational parameters

So I asked another dev how their Winforms-based commit hook works. Turns out they hack it by having a timer call Form.Show() basically as soon as the program starts. Amazing. Beautiful. Elegant.

I suspect that it might have something to do with the way TSVN redirects standard input/output/error combined with some wonderful legacy behavior, but can't prove it - and I don't the time, inclination, knowledge, or energy to investigate further.

# ¿ Jun 6, 2017 13:24

redleader: Aug 18, 2005; Engage according to operational parameters

Pablo Bluth posted:

./stuff → find stuff only as a direct child

You can also omit the ./ to get direct children (e.g. node.SelectNodes("stuff"))

# ¿ Aug 13, 2017 00:10

redleader: Aug 18, 2005; Engage according to operational parameters

Slow in the Application, Fast in SSMS might be useful.

# ¿ Sep 13, 2017 20:55

redleader: Aug 18, 2005; Engage according to operational parameters

Pilsner posted:

The game is EverQuest, in case you're wondering.

Whoa, that's a name I haven't heard in a long while. Are you using some sort of third-party server?

# ¿ Sep 24, 2017 00:40

redleader: Aug 18, 2005; Engage according to operational parameters

Protocol7 posted:

Not really sure if this belongs here or in the SQL thread, but I have a hypothetical question about potential vectors for a SQL injection.

I called out a coworker for using string concatenation to build a SQL query as opposed to using a parameterized query. They think it's not an issue because it's a GUID parameter, but I think even if it isn't technically possible, we should still parameterize it as it's a good habit to get into.

Has anyone ever seen string + GUID concatenation create an opportunity for a SQL injection, or am I overreacting to the potential of such an attack? It's all C#/MSSQL, if that changes anything.

Assuming it's a System.Guid and that the GUID is the only source of untrusted input, then the string concatenation is safe.

I'd recommend writing a parameterized query anyway, since (1) you're right, it's a good habit (2) it makes it immediately clear that this query ISN'T vulnerable to an injection, and (3) it will still be safe if someone changes the data type later on.

I guess that using an ORM or statement mapper would be even better, but it sounds like your place does things the old fashioned way (like mine - I'm very familiar with manually fiddling about with SqlCommands).

# ¿ Oct 2, 2017 21:37

redleader: Aug 18, 2005; Engage according to operational parameters

NihilCredo posted:

The JVM to CLR transpiler was called IKVM.NET, it was reasonably popular and was only officially abandoned like a few weeks ago, so hey if you want a side project to pick up...

CLR to JVM, I don't think any project went past the proof of concept stage. Unless you count Xamarin Android, that is.

IKVM is still amazing to me. You throw this thing in with some JARs, and they just work side by side with your .NET assemblies with no friction. IIRC you could even navigate toy reference in VS.

After giving it almost no thought, I suspect that JVM -> CLR would be easier than the other way round, since the CLR has much better support for generics.

# ¿ Oct 14, 2017 01:14

redleader: Aug 18, 2005; Engage according to operational parameters

Munkeymon posted:

For one, you can parallelize the selects, if you want.

How does this work? Does EF spin up a few threads and query IDs 1-1000 on one thread, 1001-2000 on the next, etc?

# ¿ Nov 1, 2017 20:00

redleader: Aug 18, 2005; Engage according to operational parameters

How sure are you that data is the same between the two systems? Could there be some invisible characters (e.g. non-breaking spaces) in one of them? You could try comparing a hex dump of the two strings.

Find a third machine and see what that one does.

# ¿ Feb 12, 2018 22:29

redleader: Aug 18, 2005; Engage according to operational parameters

I'm sad that you need .NET 4.7 to use the new tuple stuff freely. I know there's a Nuget package for ValueTuple, but for Reasons^TM I can't use that, and our servers are still on .NET 4.6.x.

# ¿ Feb 15, 2018 02:34

redleader: Aug 18, 2005; Engage according to operational parameters

Have you considered memory-mapped files?

# ¿ May 11, 2018 05:46

redleader: Aug 18, 2005; Engage according to operational parameters

LOOK I AM A TURTLE posted:

Let's not forget about the fact that, as a lot of people were recently made aware of, you can have relative paths inside a zip, meaning that a zip file can contain a file at ../../../../usr/bin/fart.

Oh for Christ's sake.

# ¿ Jul 19, 2018 11:49

redleader: Aug 18, 2005; Engage according to operational parameters

MS were prototyping a standard, out-of-the-box command line parser in the corelabs repo a while back, but it looks like they've killed it off. More's the pity; I work at a place where all libraries need to be approved so using anything outside the standard .NET framework is a huge PITA that's generally not worth bothering about. Just roll your own! How hard could it be!

# ¿ Sep 7, 2018 03:54

redleader: Aug 18, 2005; Engage according to operational parameters

B-Nasty posted:

VB.NET is heaven on earth compared to old VB (a.k.a. VB6) and its web cousin, ASP Classic. VB.NET is just the same .NET as C#, but with an ugly, retarded syntax.

In fairness though, even VB6/Classic ASP wasn't a horrible language, and it was actually very powerful. Its real issue was that it was easy for non-programmers to use. That sounds good on paper, but once you get an application beyond trivial complexity/functionality, organization of code, DRY, standard patterns/practices, etc. starts to become really important. God help whoever has to maintain a 100K+LOC VB application cobbled together by someone who never programmed before.

In other words, classic ASP suffers from PHP Syndrome.

# ¿ Nov 4, 2018 00:40

redleader: Aug 18, 2005; Engage according to operational parameters

Che Delilas posted:

BUT BE CAREFUL ITS GOING TO DEADoh it's a console app carry on.

I hear this all the time, but some devs here have written ASP.NET code that uses Wait() and Result, and I've never seen or heard of a deadlock occurring.

Of course, that's probably due to our lacklustre monitoring and/or luck. Do IIS or ASP.NET kill deadlocked threads after they hit the request timeout? What symptoms would you expect if you were encountering deadlocks in that sort of environment?

# ¿ Dec 11, 2018 19:50

redleader: Aug 18, 2005; Engage according to operational parameters

Thanks all. And yeah, ASP.NET legacy - should have clarified.

Sweet, so it sounds like the reason we haven't run into any visible problems is (1) luck and (2) terrible monitoring.

# ¿ Dec 12, 2018 00:18

redleader: Aug 18, 2005; Engage according to operational parameters

Weird German B and weird Turkish I/i are essential parts of any text processing toolkit.

# ¿ Feb 2, 2019 09:58

Adbot: ADBOT LOVES YOU

# ¿ May 2, 2024 11:16

redleader: Aug 18, 2005; Engage according to operational parameters

New Yorp New Yorp posted:

On second thought. that list of requirements read like something provided by a recruiter or non-technical manager. I think it might be a technology graveyard "maintain our awful legacy poo poo written in 2003" gig, in which case either convince them to let you start modernizing it, or run as fast as possible.

The third option is do it, but charge bigly to cover the booze and therapy you'll need to cope.

# ¿ Mar 24, 2019 20:46

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > .NET Megathread 3.5: await GetGoodPosts()

«‹›3 »