Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
nielsm
Jun 1, 2009



I want to opportunistically update a file located on a network share, that may not always be available.
Using a real network service for the data isn't an option at this time.
Application is WPF, running as a regular user.

Is there a way to attempt opening a file and specify a timeout, or get an operation object that can be cancelled?

Adbot
ADBOT LOVES YOU

nielsm
Jun 1, 2009



hackbunny posted:

Even at the Win32 level, it's doable but not trivial. You'll notice that CreateFile, unlike pretty much all other low-level I/O functions, has no asynchronous or interruptible variant (CloseHandle, although not strictly file-specific, is another, that you'll have to consider - it's what Dispose/Close will call to release the native file handle). The only way to cancel an always-synchronous I/O routine like that is CancelSynchronousIo, which you'll have to call from another thread (naturally). If a little low-level Win32 code and some reasonable expectations on undocumented internals (e.g. the assumption that File.Open always ends up calling CreateFile) is acceptable, I can try to whip you up something

I hadn't read the CreateFile topic properly and I do see that it can't return an OVERLAPPED itself.
For now it's probably fine to just perform the entire operation on another thread that may take a long time to timeout.

The basic idea with CancelSynchronousIo would be to get the OS handle for the thread I perform the file open on, and attempt to cancel the IO, right? And perhaps try to wrap it in some synchronization to avoid accidentally cancelling a different IO than intended.

nielsm
Jun 1, 2009



Speaking of HTTP clients, as far as I can tell HttpWebRequest is the only way to use NTLM authentication with the current credentials? Is that right?

nielsm
Jun 1, 2009



EssOEss posted:

What I would prefer to do is to zip the files on the fly to a temporary Azure blob. That is, do not write to filesystem, do not write to a MemoryStream in RAM, just append the data directly to the blob. This avoids using significant server resources besides the CPU and limits the range of things that can go wrong. Once the write is finished, give the user a SAS token enabled link to the blob and get rid of the blob after N hours.

I do not know if this processing model works well with the Zip and Azure Storage APIs, however. I am also unsure if the ZIP file format is friendly to append-only construction.

The ZIP format keeps the directory at the end of the archive, and I believe it's legal to have junk between individual files' data streams, meaning you could easily make "multi-session" ZIP files where you add a bunch of files, write a new directory at the end pointing to all old files and new files contained, and then you just have an old directory index floating in the middle somewhere, taking up a bit of extra space. You can even "delete" files that way, just writing a new directory that no longer points to the files to be deleted.
Of course this accumulates garbage over time, so if you update some archives a lot you'd need to do a complete rewrite occasionally.

I don't know if any libraries exist that allow this usage pattern, but the file format itself can support it.

nielsm
Jun 1, 2009



And this is also why, if you have a service where you let untrusted users upload files of only a limited set of formats, you need to validate those files and ideally rewrite them, especially if other users are able to download the files. If you don't rewrite the file in its purported format, by parsing and re-creating the file structure, users would probably be able to stuff other data inside them e.g. via ZIP files.

nielsm
Jun 1, 2009



New Yorp New Yorp posted:

Microsoft provides a hosted agent. If you're a paid subscriber, it's free and unlimited. If you're a free tier user, you get a limited number of build minutes per month, but it's a pretty generous number (1500 I think).

Hey thanks for pointing this out. After taking a short look at it, I set up an experiment, and then managed to "sell" it to the other devs on OpenTTD, which has been wanting a new CI for a while now. They actually have unlimited build minutes for open projects, on the MS-hosted agents, even the macOS ones.

nielsm
Jun 1, 2009



I have a .NET Framework 4.7 program which runs in the background with no forms (it has some invisible windows for Win32 messages) which appears to be crashing silently at random. It may run a full day without problems, or it may crash after a minutes. I haven't noticed a pattern yet and have no idea where to begin debugging. Any suggestions for how to capture some backtraces, minidumps, or anything else useful?

nielsm
Jun 1, 2009



The event log was obviously the first thing I should have checked, and I did find some useful backtraces. I'll augment those locations and add some more logging first.

nielsm
Jun 1, 2009



Followup: Adding some log4net and logging unhandled exceptions in the App object was in fact a really good idea. (Thank you for making it easy to log exceptions for once.) It turns out a server was delivering truncated JSON data which caused the parser to throw, and I was not handling that.

nielsm
Jun 1, 2009



DI question, I'm using SimpleInjector to set up my application. Now I need to have a dynamic configuration, depending on a configuration setting I'd like to supply a different object for a specific interface. I think.
I'd like to have an IUserIdentitySupplicant, usually fulfilled by an AutoDetectIdentitySupplicant, but the installer should be able to set a setting that would select a FixedIdentitySupplicant that doesn't dynamically detect and supply identity data, just having a static result.

Does this sound like the right approach, and what would be the pattern to implement this?

nielsm
Jun 1, 2009



I've used wkhtmltopdf for generating PDFs, it's a standalone binary with all dependencies compiled in statically. But it's not terribly good to work with/debug and can be rather slow.

nielsm
Jun 1, 2009



WiX is the way to go.

If you didn't need to allow headless install, customization of destination and change configuration, then I'd have suggested ClickOnce, but while it's easy it's not very flexible.

nielsm
Jun 1, 2009



I have two WPF (.NET Framework 4.5) applications. I'd like to make a sort of soft integration between them: If both are installed, one can detect the other is installed, and offer to send a message to the other. If the other is already running, the message is sent to the existing instance, otherwise start a new instance.
This sounds very similar to what an out-of-process COM server can do, but it seems that pattern is not well supported by .NET Framework.
Any suggestions for other technologies to look at, or should I just do something "stupid" with Win32 messages, named pipes, or similar?

nielsm
Jun 1, 2009



Yes both are in-house applications. It would be a UI nightmare to properly combine them, but reasonably simple to add some basic "send this item to the other app" UI.

I looked at the class in that blog post, and it does look like an overly specialized solution for people who don't want to understand what IPC is, but the IpcChannel class it uses seems useful as a way to send structured messages.

nielsm
Jun 1, 2009



Win32/NT kernel mutex objects can be created as either system global or session local, by adding a Global\ or Local\ prefix to the mutex name. I don't know how well that translates to .NET Framework. A session local mutex will only be visible in the logon session that created it.
I'm surprised .NET does not have a good (and easy) local RPC mechanism.

nielsm
Jun 1, 2009



Adventures in making a local out-of-process COM server in C#/Framework 4.6

Continuing my previous question about local RPC, I did implement a working call mechanism over named pipes, only missing a way to start the server if it isn't running yet.
However I also kept looking into COM interop in .NET Framework, and came upon this blog post. It turns out creating a local COM server isn't that hard after all?

So I go about to declare an interface, give it an IID, make it ComVisible and dual. I can build my assembly, getting a typelib from it is more difficult. After seeing various warnings about regasm.exe and then trying tlbexp.exe, I figure out that the proper way is after all regasm.exe /tlb, to build the typelib TLB file and get it registered.
Next is implementing a class implementing the interface, give it a CLSID, and calling RegisterTypeForComClients on startup, and it fails? Okay the class needs to be marked public. Now it registers without errors.

Can I then instantiate it? Try with PowerShell: [System.Activator]::CreateInstance([System.Type]::GetTypeFromCLSID([guid]'C4086A64-17AB-48DC-AA1D-0650329210B5'))
It doesn't error!

Can I then call the returned object?
Nope, no useful methods on it. Trying to call Invoke on the Type object returned by GetTypeFromCLSID, nope the best I get is an error about wrong number of arguments to the call. I try calling both by name and by DispID. I can't query anything about the object's actual type.

Maybe the class registration is incomplete, and COM isn't associating the typelib with the object?
I try manually adding the CLSID to registry, pointing LocalServer32 to my EXE file, and referencing the TypeLib by GUID.
Nope, even restarting PowerShell doesn't get me any type information for the dispatch interface.

Well, can I at least have it start the server process when it's not running with class factory registered?
Uh, it hangs. It does start the process indicated in LocalServer32, but never detects the class factory registration. After lots of searching and experimentation, it turns out this only works if the assembly is compiled as x86 or x64, AnyCPU does not work. So far so good.

Okay then let's try calling it straight. I'll load up another C# project and add my typelib as a reference. Oh, that fails because it's produced from a .NET asssembly, it tells me to add the assembly directly. But I don't want to do that, I don't get out-of-process activation then.
Try to do it the classic way, get a Type object by CLSID, then instantiate it with Activator. Copy the interface declaration from the server project, and cast the instantiated object to the interface type.
I can create the object, the server starts, and the interface cast succeeds. Great!
Call the method? Fails. Error indicates something about wrong thread context. But it's an out-of-process object, thread-affinity should be a non issue?!? Just marshal the call and send it over.

This is where I give up and decide to just go ahead with the named pipe and do my own, custom server activation.
Classic answer: "How hard can it be?" Very hard.

nielsm
Jun 1, 2009



You can do some things but definitely not everything with WSL. I wouldn't try to depend on it for system configuration. The point where PowerShell really scores on Windows is the built-in capabilities for scripting WMI objects and loading and use pretty much any .NET assembly, and those are basically required for any non-trivial configuration on Windows. A much better cross-platform bet (but would still require special-casing for Windows) would be Python or Perl, except those obviously require installation.

nielsm
Jun 1, 2009



Yeah, when I log into my account I can download at least VS 2010, 2013, 2015, 2017, and 2019, and also a few even older versions.

nielsm
Jun 1, 2009




Nice, they added constreadonly correctness. Thank you C++.

What's the difference between an interface with default members and an abstract class with partial implementation?

nielsm
Jun 1, 2009



mortarr posted:

I'm after some advice on copying files (typically images) from usb-attached android / iphones to local disk.

...

I'd prefer to work with the ShellObjects directly, but I can't seem to find any relevant CopyTo, GetBytes or GetStream methods I can use. Google gives back a lot of c++ stuff or low level windows api docs back, but I don't have any experience there at all.

You probably need to use IFileOperation for that.

nielsm
Jun 1, 2009



Anything that could itself fail should not be in a catch block. (Assume logging can't fail, if logging fails it's fine to explode violently.)

nielsm
Jun 1, 2009



Why is my list not sorted... I must be missing something obvious here.
WPF on Framework 4.6.1

C# code:
        private void InsertLogFileInList(LogFileData data, ObservableCollection<LogFileData> collection)
        {
            bool inserted = false;
            for (int pos = 0; pos < collection.Count; pos++)
            {
                if (collection[pos].MachineName == "(sim)") continue;
                if (collection[pos].DateSortable < data.DateSortable)
                {
                    Dispatcher.Invoke(() => collection.Insert(pos, data));
                    inserted = true;
                    break;
                }
            }
            if (!inserted) Dispatcher.Invoke(() => collection.Insert(0, data));
        }
All items in the ObservableCollection are added via this method, except a single one which is the "(sim)" one, which is appended. All items except for the "(sim)" are discovered and added in the same task.
The actual result is that I get several individually sorted sequences of items in the list.

I'd consider putting a SortDescription on the ListBox.Items, but can't figure out from the docs how to handle the (sim) item, which needs to be last.


Edit: Never mind, I figured out what I'm doing wrong. It's the if (!inserted) at the end, that should be appending the item rather than prepending it.

nielsm fucked around with this message at 10:22 on Mar 16, 2021

nielsm
Jun 1, 2009



NihilCredo posted:

You could put the (sim) item/s in one ObservableCollection, everything else in another, and wrap a CompositeCollection over both, putting the (sim) collection last.

CompositeCollection definitely sounds useful, I'll have to remember that for future uses.

nielsm
Jun 1, 2009



Working in .NET 5, is there a way to customize the deployment of a desktop (WPF) app?
I want to run the EDITBIN program on the generated DLL and EXE files to set the /swaprun:net flag so it can be run from a network share without locking the files.

I'm not sure what to search for regarding customizing the build/publish steps, but I'm open to writing MSBuild XML if that's needed.

nielsm fucked around with this message at 14:18 on Oct 4, 2021

nielsm
Jun 1, 2009



Yep you want to use a grid there for the vertical stack. Set the rows with the labels to height="auto" and the rows with the text boxes to height="1*".

nielsm
Jun 1, 2009



Also look at whether some of those things that need doing can be automated, you mention renewing certificates as one.

Since you do have a service management system, consider if those things that need doing shouldn't go into it, as tasks with a deadline, as soon as it becomes known it needs doing. That should make a clearer audit trail.

nielsm
Jun 1, 2009



Is this just to learn the tech? Because it seems an odd choice to use Blazor for the UI otherwise, when the app is meant to run entirely on localhost. A more traditional GUI app seems like it'd be the path of less resistance right now.

nielsm
Jun 1, 2009



It might be possible to put a huge effort into developing viewmodel classes for WPF and then let the "users" supply a WPF XML file and just hook up the viewmodel to that at runtime.
It would still be very fragile and require a lot of knowledge from the UI designers ("users") and yeah, continuous developer support for the viewmodels.

And unless the devices are very similar on a technical level, you'd have to develop a viewmodel and underlying support for each anyway. After that, making a WPF GUI on top would be the lesser part.

nielsm
Jun 1, 2009



Case folding is a particular algorithm/transformation specified by the Unicode standard, yes, and it isn't quite the same as an "uppercase" or "lowercase" function, which is by definition human language sensitive.

Searching I find this: https://github.com/dotnet/corefxlab/pull/2637
But I can't figure out where this is supposed to be implemented or available. The PR says it's merged nearly 3 years ago, but I don't see any trace of it in the .NET 5 or 6 library. Maybe System.Text.CaseFolding is available as a NuGet package?

nielsm
Jun 1, 2009



Rocko Bonaparte posted:

I'm implementing Python % operator in a personal interpreter project so I'm having to accept printf-like formatting operators.

It's a mini language inside the language, just write a parser for it. You can try making it compile/translate to the .NET format string language if you think that's easier, just consider the impact for being able to accurately report errors.

nielsm
Jun 1, 2009



beuges posted:

Is there a visual studio extension or something which easily lets you manage secrets? I often need to switch connection strings and tokens and stuff between my local dev environment and a remote test environment and it gets to be a pain to go into 7 different projects secret files to update the config.
Ideally if I could create different environment configs somewhere, and then just tell VS to swap out the secrets for all the projects with the proper ones for the environment that would be great, but I haven’t been able to find anything that seems to do that

In a .NET 6 WPF project I can open "Debug Properties" from the Debug menu and configure various launch profiles. It's at least possible to set command line parameters, working directory, and environment variables, as well as some flags for remote debugging and more.

nielsm
Jun 1, 2009



Is it for a one-off thing or a program you expect to be using and extending a lot in the future?
What will be consuming the data, your own program or something else that you don't control? Will you be reading the data or only writing?

If it's something you expect to be developing further and using/extending, it's probably worth it to design a data model in C# classes that describes the data, and then set up serialization with some JSON library.
If it's just a one-off, maybe you're fine just doing something with lists of dictionaries or similar awkward stuff.

nielsm
Jun 1, 2009



If it's only for your own use, and not making data for someone else's game, then definitely start out defining an object model for your data in C# that makes sense to use for your game code.

When you have the object model, you can look at ways to serialize it using standard libraries.

nielsm
Jun 1, 2009



I have a .NET 5.0 (yes it needs to be upgraded) webservice running via Kestrel.
I have installed a new HTTPS certificate for the host in the Windows certificate store, and deleted the old certificate (with the same common name), but Kestrel keep serving with the old certificate, which has now expired.

Where does Kestrel find an old certificate that has been deleted from the Windows certificate store, and how do I make it pick up the new cert?

This is in production, dev certificates should not be involved in any way.


Never mind all this, it turns out the answer was that the certificate had to be installed in the per-user certificate store for the service account running the service, and I was looking in the machine certificate store, where someone else had also placed the certificate.

nielsm fucked around with this message at 14:27 on Jul 10, 2023

nielsm
Jun 1, 2009



epswing posted:

Control Pictures! Never heard of these, but it looks like what I'm after, thanks.

I guess my concern would be when looping through bytes, how do I know if a given byte is a single byte like 04 that I should replace with the EOT control picture, or part of a multi-byte character that happens to contain an 04 byte? If the text is ASCII encoded, probably nothing to worry about, but if it's e.g. UTF-8 couldn't there by multi-byte chars in there?

If you know the data is a specific encoding, then decode that before trying to display it.
If you don't know what the data is, then don't try to display anything above 0x7F as text since it may be anything. (The rest may also not be ASCII either, in that case, though it's rather likely.)

For UTF-8 in specific, it has an encoding scheme where the byte values 0x00 to 0x7F map directly to ASCII, and the byte values 0x80 to 0xFF have well-defined meanings as either a leading byte or a trailing byte, but never "may be either". Looking at a single standalone byte in UTF-8 encoded text, you can always know if it can be used as-is (it's a low byte), if it's a trailing byte that makes no sense without the lead byte, or if it's a lead byte and then how many more trailing bytes it needs to be decoded into a Unicode codepoint. If you haven't, do read up on how the UTF-8 encoding is designed, it's actually quite elegant.

nielsm fucked around with this message at 21:21 on Oct 12, 2023

nielsm
Jun 1, 2009



I have an API application served via Kestrel. Right now it's on .NET 5 and our team is trying to find time to upgrade to .NET 8, but I don't think that matters for this issue I'm having.

Right now the API is authenticated via Windows Authentication by Negotiate method:
C# code:
    services.AddAuthentication(NegotiateDefaults.AuthenticationScheme)
        .AddNegotiate(NegotiateDefaults.AuthenticationScheme, options =>
        {
            options.Events = new NegotiateEvents
            {
                OnAuthenticated = HttpAuthHelper.NegotiateAuthenticated
            };
        });
This works. The HttpAuthHelper.NegotiateAuthenticated delegate pulls some additional data and adds some more specific claims to the Principal in the context, and the rest of the application can access this.

Now, to support a new API consumer that can't handle Negotiate auth, I'm attempting to add HTTP Basic as an alternative. I'd prefer if it never gets advertised in the WWW-Authenticate header in 401 responses, but the main thing is that I need the application to accept either Negotiate or Basic auth, but have clients prefer Negotiate.

For this, I've added the idunno.Authentication.Basic package, and I can make it work:
C# code:
    services.AddAuthentication(BasicAuthenticationDefaults.AuthenticationScheme)
        .AddBasic(BasicAuthenticationDefaults.AuthenticationScheme, options =>
        {
            options.Realm = "MyApp";
            //options.SuppressWWWAuthenticateHeader = false;
            options.Events = new BasicAuthenticationEvents
            {
                OnValidateCredentials = HttpAuthHelper.BasicValidateCredentials
            };
        });
When I change the service configuration to add Basic instead of Negotiate, it works just fine. It calls my HttpAuthHelper.BasicValidateCredentials delegate, which receives the credentials sent, validates them, and adds the necessary claims.

However, getting the two to work at the same time has me stumped.

I've tried adding both one after each other, with two separate services.AddAuthentication calls. In that case, the last one added "wins", and the other is never initialized at all, the "options" delegate is not even called for it.

I've tried a single services.AddAuthentication(...).AddBasic(...).AddNegotiate(...), i.e. adding authentication once and then adding both methods. Then the method named as default in the AddAuthentication call wins, and the other is never initialized.

I've tried a few different setups with more complex options for AddAuthentication:
C# code:
    services.AddAuthentication(options =>
    {
        options.DefaultScheme = NegotiateDefaults.AuthenticationScheme;
        options.DefaultAuthenticateScheme = BasicAuthenticationDefaults.AuthenticationScheme;
        options.DefaultChallengeScheme = NegotiateDefaults.AuthenticationScheme;
    })
    .AddNegotiate(...)
    .AddBasic(...)
Then I can authenticate via Basic if I send then Authorization header in the initial request, and if I don't then the application responds with a WWW-Authenticate: Negotiate header. But for some reason, then a web browser trying to access the API (via an Angular application) never completes the Negotiate and just gets 401. (Both the Network tab in the browser's dev tools and a direct call with cURL -v shows that the application is sending the header.)
This seems to be the closest I've gotten to make it work. But I don't know why the browser fails to complete the Negotiate auth or where to look for any debugging related to that.

I've tried adding ForwardDefault, ForwardChallenge, ForwardForbid and more properties to the options of each of the two schemes, and it either does nothing, or gives me a variation of the above.
I've tried with and without the SuppressWWAuthenticate header in the options for Basic, it doesn't get me closer.

I've attempted to call options.AddScheme in the delegate to AddAuthentication, but I can't do it properly since while the NegotiateHandler class is public, the BasicAuthenticationHandler class is internal and not accessible in my application code... I don't know if it ought to be accessible to application code.


Do anyone know what the proper way to go about this is, or have ideas for other things to try?

nielsm
Jun 1, 2009



Maybe I won't need to solve it either way... someone else involved argued that we should find some way to get that one consumer to do Negotiate auth properly, rather than change the API, which I quite agree is preferable.



I may look at this tomorrow.

nielsm
Jun 1, 2009



Postgres can be a very good candidate for database, if you have the option to use or install the PostGIS extension. That will give you the geographic data types and operations on them to make all the map and coordinates logic much more straight forward to store and query.
To display a map with your own overlay, you can look into OpenStreetMap.

nielsm
Jun 1, 2009



You evaluate your code in production by instrumenting it with logging that captures data that can help answer your questions, without interrupting the normal request handling. What data your need to capture depends on the application.

For example, capture the total request times (request received until response ready to be sent) together with some category of the requests, so you can measure whether you have outliers where some requests take unusually long time compared to other requests of the same type, or if the request processing time increases when the service is busy.

Depending on the nature of the data you process, you can also try capturing request series and replay them in a testing environment, where you can more heavily instrument things.


Figuring out whether it's worth optimizing your program more, or throw more hardware at it, is not a science. It's just as much a business tradeoff: What's more expensive, your development time, or your increased hosting bill?
It's probably always a good idea to try to identity the cause of slowness, to decide if it's something you can reasonably improve on or not, and whether it's something that could prevent scaling to faster/wider hardware.

Adbot
ADBOT LOVES YOU

nielsm
Jun 1, 2009



CitizenKeen posted:

Is that something I can do with Serilog, or is that a separate kind of logging?

Probably? The main thing is to make sure you can retrieve the logged data so you can graph or analyze it in a useful way.
You also need to make sure the instrumentation logging doesn't drown out other operational or error logging, perhaps by logging them to separate streams/files/tables.

If you need to do instrumentation that captures lots of data, it might be better to make a database table/whatever specifically for that data, so you have it structured, efficiently packed, and indexed.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply