Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
epswing
Nov 4, 2003

Soiled Meat
I must have read https://docs.microsoft.com/en-us/ef/ef6/modeling/code-first/migrations/teams 100 times by now. EF6 Code First migrations just suck so bad for modern branchy (git) development.

Aside from the care needed to properly merge changes by two branches, which is pretty well covered in the docs above, what I'm finding now is rolling back (downgrading) is always broken after such a merge. The generated migration files are timestamped (e.g. 202104191517481_AddedJobIdColor.cs) and within the Designer.cs file this is hardcoded in a string IMigrationMetadata.Id { get { return "202104191517481_AddedJobIdColor"; } } and the entire model is stamped into the __MigrationHistory table, so you can't really modify anything there.

When you roll backwards with Update-Database -TargetMigration SomeEarlierMigration it appears to blindly apply migrations in reverse chronological order (reverse file order), which is a different order than they were applied when it was applied forwards. This means in desktop software land where you have version numbers, when you have a migration in v1.0 that comes chronologically after a migration in v2.0 (and has been merged into v2.0 using the above docs), and you have a v2.0 application that you need to roll back to v1.0 for whatever reason, when you Update-Database -TargetMigration TheLastV1Migration the command doesn't unapply the v2.0 migrations that came before TheLastV1Migration.

Just writing this out makes it sound super convoluted, I know. I'm not even sure how they'd fix this, what other choice does the update command have but to process the migrations in order? I guess I can jump through a bunch of hoops with Update-Database -Script and sorta build the exact 'downgrade' sql script I need but...drat.

I guess I'm not asking anything, just hopefully commiserating with anyone that's had to deal with this before.

Adbot
ADBOT LOVES YOU

distortion park
Apr 25, 2011


The cynical answer is to say that down migrations don't work, in general, so don't get too worked up about it.

IMO tying your migrations to your code version control is more effort than it's work. I always put them in a different repository now, and just treat the db as a separate service with its own CI etc. It's really hard to consistently make safe changes with zero downtime when the migrations and code are in the same ci pipeline.

New Yorp New Yorp
Jul 18, 2003

Only in Kenya.
Pillbug

epswing posted:

I must have read https://docs.microsoft.com/en-us/ef/ef6/modeling/code-first/migrations/teams 100 times by now. EF6 Code First migrations just suck so bad for modern branchy (git) development.

Aside from the care needed to properly merge changes by two branches, which is pretty well covered in the docs above, what I'm finding now is rolling back (downgrading) is always broken after such a merge. The generated migration files are timestamped (e.g. 202104191517481_AddedJobIdColor.cs) and within the Designer.cs file this is hardcoded in a string IMigrationMetadata.Id { get { return "202104191517481_AddedJobIdColor"; } } and the entire model is stamped into the __MigrationHistory table, so you can't really modify anything there.

When you roll backwards with Update-Database -TargetMigration SomeEarlierMigration it appears to blindly apply migrations in reverse chronological order (reverse file order), which is a different order than they were applied when it was applied forwards. This means in desktop software land where you have version numbers, when you have a migration in v1.0 that comes chronologically after a migration in v2.0 (and has been merged into v2.0 using the above docs), and you have a v2.0 application that you need to roll back to v1.0 for whatever reason, when you Update-Database -TargetMigration TheLastV1Migration the command doesn't unapply the v2.0 migrations that came before TheLastV1Migration.

Just writing this out makes it sound super convoluted, I know. I'm not even sure how they'd fix this, what other choice does the update command have but to process the migrations in order? I guess I can jump through a bunch of hoops with Update-Database -Script and sorta build the exact 'downgrade' sql script I need but...drat.

I guess I'm not asking anything, just hopefully commiserating with anyone that's had to deal with this before.

Here's the ideal way to address it: Don't ever try to migrate backwards. Version your data access so that your database schema can always be compatible with, at a minimum, two versions of the application: The current and the previous. It's a bit more management overhead on the database side of things and requires some plumbing in the application to be able to target the right version, but that lets you roll your application code forward and back without having to mess with migrations.

fuf
Sep 12, 2004

haha
I have a Blazor problem that might be more of a I don't know what I'm doing problem.

I have a component with a Stack of messages that are displayed like a log.
I want the messages to come from from a separate LoggingService that is injected into the page.
The component doesn't know to update when there's a new message in Log.Messages so I added an event that triggers "StateHasChanged" and gets invoked by the LoggingService:

C# code:
@inject LoggingService Log

@foreach(var msg in Log.Messages)
   {
      <p>@msg.Text</p>
    }

@code
{
   protected override void OnInitialized()
    {
        Log.OnNewMessage += OnNewMessageHandler;
    }

private void OnNewMessageHandler()
    {
        InvokeAsync(() => StateHasChanged());    
    }
}
The problem is that I keep getting "Collection was modified after the enumerator was instantiated" errors, because sometimes a new message is added to Log.Messages while the component is in the middle of rendering. At least I'm pretty sure that's it because it only starts happening when the Stack gets pretty big.

I tried using a new Stack within the component itself rather than iterating over the one in the LoggingService, but it has the same problem because I still have to use InvokeAsync(() => StateHasChanged()) .

Does anyone have any suggestions? Am I going about this in completely the wrong way?

distortion park
Apr 25, 2011


This seems like it could be a threading issue or something, but a super easy solution might be to use a for loop instead of foreach, assuming the collection only grows

insta
Jan 28, 2009
Quick and dirty solution: chuck a .ToList() on the Messages inside the loop.

Funking Giblet
Jun 28, 2004

Jiglightful!
Something is logging while you are doing that forloop. I wouldn't use the log collection directly, ether project it out to list of strings, or call ToList().

brap
Aug 23, 2004

Grimey Drawer
Maybe I’m missing something but it seems like you need to synchronize if different threads might read and update the collection simultaneously. I don’t see why ToList would solve your problem since it’s not atomic to do that.

insta
Jan 28, 2009
It's not, but it's a shitload faster to clone to a new List than it is to enumerate and do the TextWriting that the page is doing.

distortion park
Apr 25, 2011


brap posted:

Maybe I’m missing something but it seems like you need to synchronize if different threads might read and update the collection simultaneously. I don’t see why ToList would solve your problem since it’s not atomic to do that.

Yeah. If you are going to do locks though then cloning the collection is probably the way to go, locking while doing the UI work seems wrong

fuf
Sep 12, 2004

haha
Thanks for the suggestions guys.

I tried the ToList first:

C# code:
  @foreach(var msg in Messages.ToList())
            {
                  @msg.Text
            }
But eventually I would get this error when the log got big enough:
"Destination array was not long enough. Check the destination index, length, and the array's lower bounds."

Maybe I'm not understanding where to do the ToList()?

Then I tried using a For loop instead:

C# code:
    @for (int i = 0; i < Messages.Count; i++)
            {
                var msg = Messages[i];
                <p>
                         @msg.Text
                </p>
            }
And this works! :)

So I will stick with this.

Thanks again.

Kyte
Nov 19, 2013

Never quacked for this

fuf posted:

Thanks for the suggestions guys.
[etc]

ToList is not atomic and assumes the source list is static throughout the lifetime of the operation. What's happening here is that the source list is getting new items in between the destination list's backing store being initialized and all the items being copied over, so eventually it reaches the end, except it's not the end and there's new items, tries to copy them and discovers it's gone beyond the allocated space. Then it crashes.
You need to use a lock or similar synchronization mechanic to prevent new insertions while you're copying over, or switch to some collection that guarantees thread-safe enumeration (or maybe an immutable collection, which would be thread-safe by default).

Using a simple for works, but it'll have different behaviors based on compiler optimizations.
If the compiler checks .Length every time, then it'll loop over everything including whatever was added while the loop was executing.
If the compiler optimizes .Length into a local, it'll only loop over the length at the beginning of the loop.
And either way you're being saved from a crash because this list happens to only grow. If it ever shrinks the whole thing will blow up in your face. Better to do things properly.

Kyte fucked around with this message at 15:53 on Nov 26, 2021

New Yorp New Yorp
Jul 18, 2003

Only in Kenya.
Pillbug
Wouldn't using a thread-safe collection like ConcurrentBag or ConcurrentStack solve this problem pretty transparently?

NihilCredo
Jun 6, 2011

iram omni possibili modo preme:
plus una illa te diffamabit, quam multæ virtutes commendabunt

What you're describing is the pattern that resulted in ObservableCollection / INotifyCollectionChanged in WPF land.

Looks like Blazor doesn't have an equivalent built-in or default mechanism:

https://github.com/dotnet/aspnetcore/issues/6647

Some dude took it upon himself to reimplement it:

https://github.com/phorks/phork-blazor/blob/main/docs/reactivity/README.md

If you don't mind using a library with zero stars, zero issues (:iamafag:) and no commits in 9 months, that is.

Or you can pay Telerik a few hundred dollars for their version:

https://docs.telerik.com/blazor-ui/common-features/observable-data

Personally, I'd go with

New Yorp New Yorp posted:

using a thread-safe collection like ConcurrentBag or ConcurrentStack

because in my experience there's very very few data-flow problems that a sufficiently brutal use of ConcurrentX cannot solve.

Munkeymon
Aug 14, 2003

Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.



This is a weird thread for me to learn that wasm has thread support in

SuitcasePimp
Feb 27, 2005

fuf posted:

I have a Blazor problem that might be more of a I don't know what I'm doing problem.

I have a component with a Stack of messages that are displayed like a log.
I want the messages to come from from a separate LoggingService that is injected into the page.

Out of curiosity, what is the use case of a stack as a logging data structure? If something like this comes up again you may want to look into the System.Threading.Channels namespace: https://devblogs.microsoft.com/dotnet/an-introduction-to-system-threading-channels/

fuf
Sep 12, 2004

haha

SuitcasePimp posted:

Out of curiosity, what is the use case of a stack as a logging data structure? If something like this comes up again you may want to look into the System.Threading.Channels namespace: https://devblogs.microsoft.com/dotnet/an-introduction-to-system-threading-channels/

uhhh I just wanted to show the results of some processing of local files and I used a stack because I wanted the most recent result to be at the top :shobon:

Again, I'm a beginner so I'm basically just going with whatever random ideas I have and then messing around to see if they work

Your Channels link looks cool but sometimes it's hard to for me to tell the difference between really standard stuff that everyone uses all the time vs more specialised stuff that I should probably leave for a while

The same kind of goes for this:

New Yorp New Yorp posted:

Wouldn't using a thread-safe collection like ConcurrentBag or ConcurrentStack solve this problem pretty transparently?

NihilCredo posted:

because in my experience there's very very few data-flow problems that a sufficiently brutal use of ConcurrentX cannot solve.

Like I'm sure you guys are right and this is a helpful answer but it's weird that ConcurrentBag hasn't come up once in all the Blazor stuff I've been reading even though they talk about async / thread stuff a fair amount. Maybe I'm just still on all the real beginner examples.

fuf
Sep 12, 2004

haha
My little blazor practice project uses SQLite and entity framework and I've tried to keep the data side as simple as possible.

But now I'm trying to do a many-to-many relationship and it's all falling apart.

I have a many to many relationship between a Video and a Tag class. My naive attempt to set it all up was like this (trimmed down to what I think are the essentials):

C# code:
public class Video
{
	[Key] public int Id {get; set;}
	public string Path {get; set;}
	public List<Tag> Tags {get; set;}
}
C# code:
public class Tag
{
	[Key] public int Id {get; set;}
	public string Name {get; set;}
	public List<Video> Videos {get; set;}
}
When I created a migration this was enough for entity framework to create a new TagVideo joining table so that's cool.

The problem is when I try to add a tag to a video and then save it:

C# code:
private void AddTagToVideo(Video video, Tag tag)
{
	video.Tags.Add(tag);
	VideoDataService.UpdateVideo(video);
}
C# code:
public class VideoDataService
	{
	private readonly IDbContextFactory<AppDbContext> dbContextFactory;
	public VideoDataService(IDbContextFactory<AppDbContext> _contextFactory){
		dbContextFactory = _contextFactory;
	}

	public void UpdateVideo(Video video)
	{
		using var db = dBContextFactory.CreateDbContext();
		db.Videos.Update(video);
		db.SaveChanges();
	}
When I call AddTagToVideo I always get this error:
code:
Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred while saving the entity changes. See the inner exception for details.
 ---> Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 19: 'UNIQUE constraint failed: TagVideo.TagsId, TagVideo.VideosId'.
Weirdly when I check the database I can see that a new row in the TagVideo table was actually created.

I can call UpdateVideo to save other changes to Video objects without any issues, so I'm sure it's something to do with the many-to-many thing.

From googling it seems like it's something to do with using multiple DbContexts? Most of the blazor stuff I've read suggests this way of doing things with a dBContextFactory where you create a new context for every operation. I really have no idea where to go with fixing this.

New Yorp New Yorp
Jul 18, 2003

Only in Kenya.
Pillbug
I rarely touch EF these days but the fact that your Video and Tags have circular references probably isn't helping. A Video contains a List<Tag> and a Tag contains a List<Video>. Honestly this is why I hate ORMs; they save time right up until you have a problem, then you spend 10x the amount of time it would have taken to just use something like Dapper for mapping query results onto objects.

epswing
Nov 4, 2003

Soiled Meat

fuf posted:

code:
Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred while saving the entity changes. See the inner exception for details.
 ---> Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 19: 'UNIQUE constraint failed: TagVideo.TagsId, TagVideo.VideosId'.

Looks like EF is complaining about uniqueness, as in you already have a link (a row in the join table) between those particular Video and Tag records. Try linking another pair of records, and/or remove the existing link and try again. Also, in the AddTagToVideo function, check first if they already refer to each other and either throw an exception (“Video already tagged with this Tag”) or just do nothing (no-op) because the work’s already done.

epswing fucked around with this message at 22:20 on Dec 11, 2021

WorkerThread
Feb 15, 2012

I'll readily admit I don't have enough experience with EF, but it is frustrating when a tool makes easy problems hard and hard problems also hard. I could just write SQL and be done with this in 10 minutes.

epswing
Nov 4, 2003

Soiled Meat
I’m not one to defend EF generally, because it is a pain sometimes, but, in this case, it’s doing everything correctly and as expected. It’s telling you you’ve violated a uniqueness constraint. Writing raw SQL and executing the same statements would yield the same result.

Edit also,

WorkerThread posted:

I could just write SQL and be done with this in 10 minutes.
I used to write all my SQL by hand, and if it’s largely LOB/CRUD statements that look really similar between entities, you naturally start to abstract out the common parts, and on a long enough timeline, you basically start to build your own lovely ORM.

That said, Dapper and other mini/micro ORMs are cool and good.

epswing fucked around with this message at 02:02 on Dec 12, 2021

fuf
Sep 12, 2004

haha

epswing posted:

Looks like EF is complaining about uniqueness, as in you already have a link (a row in the join table) between those particular Video and Tag records. Try linking another pair of records, and/or remove the existing link and try again. Also, in the AddTagToVideo function, check first if they already refer to each other and either throw an exception (“Video already tagged with this Tag”) or just do nothing (no-op) because the work’s already done.

yeah it was something to do with this but definitely not in an obvious "you have a duplicate row" way because I was getting the same error when I tried to remove a tag.

the solution that seems to work is to retrieve the data to be updated first, in the same context:

C# code:
private void AddTagToVideo(Video video, Tag tag)
{
	using var db = dbContextFactory.CreateDbContext();
        var tagToAdd = db.Tags.Where(t => t.Id == tag.Id).FirstOrDefault();
        var videoToUpdate = db.Videos.Where(v => v.Id == video.Id).FirstOrDefault();
        videoToUpdate.Tags.Add(tagToAdd);
        db.Videos.Update(videoToUpdate);
        db.SaveChanges();
}
I don't claim to understand why but it's enough that I can keep hacking away for now.

Kyte
Nov 19, 2013

Never quacked for this
My guess is because you were initializing a new DbContext the change tracker started at zero so it saw all the elements in the Tags collection and thought "these are all new Tags, gotta insert them all", so the problem wasn't the tags being added or removed but rather the tags that stayed.
Loading the entity into the change tracker solves that because EF now knows which tags already exist in the DB.

This is going off memory so might be wrong but I'm pretty sure the procedure is to create your dbContext, attach the Video to the context so EF takes it as the base state, then do your change, then save.

Btw I'm fairly sure you can do db.Videos.Include(v => v.Tags) and save yourself the tag load. (Or configure the navigation property to auto-include)

Kyte fucked around with this message at 16:20 on Dec 12, 2021

fuf
Sep 12, 2004

haha

Kyte posted:

My guess is because you were initializing a new DbContext the change tracker started at zero so it saw all the elements in the Tags collection and thought "these are all new Tags, gotta insert them all", so the problem wasn't the tags being added or removed but rather the tags that stayed.

Thanks, yeah I think this must have been exactly it. When I looked at what EF was doing, even when I was trying to remove a tag, it was trying to insert data into the TagVideo table.

LOOK I AM A TURTLE
May 22, 2003

"I'm actually a tortoise."
Grimey Drawer
Many-to-many entities have always been annoying in Entity Framework, precisely because of the type of thing you're dealing with now. If I were you I would consider defining a VideoTag entity explicitly instead of relying on the inferred one.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
Is there a way to use the built-in .NET sorting algorithms with async functions? My scripting runtime is using async function calls for its callable stuff. Theoretically, any of these calls could actually block. It would be dumb to happen when comparing for a sort, but the prototype is async nonetheless. I am assuming I couldn't use the Sort functions in the collections with comparers that are awaiting on this stuff, but I thought I'd ask before having to pull out my own sorting algorithm.

Actual performance is not a big consideration here despite talking about sorting.

New Yorp New Yorp
Jul 18, 2003

Only in Kenya.
Pillbug

Rocko Bonaparte posted:

Is there a way to use the built-in .NET sorting algorithms with async functions? My scripting runtime is using async function calls for its callable stuff. Theoretically, any of these calls could actually block. It would be dumb to happen when comparing for a sort, but the prototype is async nonetheless. I am assuming I couldn't use the Sort functions in the collections with comparers that are awaiting on this stuff, but I thought I'd ask before having to pull out my own sorting algorithm.

Actual performance is not a big consideration here despite talking about sorting.

I'm not following your question. Can you post some relevant code snippets?

Like, is this what you're talking about?
code:
public async Task<IEnumerable<Foo>> SortFoos(IEnumerable<Foo> foos) {
   return Task.FromResult(foos.OrderBy(p => p.Property));
}

New Yorp New Yorp fucked around with this message at 17:50 on Dec 16, 2021

raminasi
Jan 25, 2005

a last drink with no ice
Is the question "Can I somehow plug async comparators into BCL structures and algorithms that expect synchronous ones?"

Kyte
Nov 19, 2013

Never quacked for this

Rocko Bonaparte posted:

Is there a way to use the built-in .NET sorting algorithms with async functions? My scripting runtime is using async function calls for its callable stuff. Theoretically, any of these calls could actually block. It would be dumb to happen when comparing for a sort, but the prototype is async nonetheless. I am assuming I couldn't use the Sort functions in the collections with comparers that are awaiting on this stuff, but I thought I'd ask before having to pull out my own sorting algorithm.

Actual performance is not a big consideration here despite talking about sorting.

I can't begin to imagine how'd you have say, a quicksort, work without preloading all the data it needs. Which is effectively a ToListAsync().

Jabor
Jul 16, 2010

#1 Loser at SpaceChem
I think the idea is that these comparators are being written in an embedded scripting language by someone using Rocko Bonaparte's software, and while it's okay for the sort to run dogshit slow if the user writes a bad comparator that blocks on something, it's not really okay for it to block a thread and stop other operations from proceeding.

NihilCredo
Jun 6, 2011

iram omni possibili modo preme:
plus una illa te diffamabit, quam multæ virtutes commendabunt

I think the solution is to force the async sorting functions to run synchronously, then wrap the whole OrderBy sorting call in an async so it doesn't block. You can use Task.Run to wrap non-IO-bound code (i.e. non async methods) to have it execute in a separate thread from the pool.

Something like this (sorry for the atrocious type signatures, haven't written c# in a while):

code:

static class Extensions {

	public static async Task<IOrderedEnumerable<T>> OrderByAsync<T, TKey>(this IEnumerable<T> collection, Func<T, Task<TKey>> asyncSorter) {
		return await Task.Run<IOrderedEnumerable<T>>(() => collection.OrderBy(elem => (asyncSorter(elem)).Result));
	}
}		

// ...

var shitSorter = async (int x) => {
	await Task.Delay(1000);
	return x.GetHashCode();
};			
		
var arr = new[] {1,2,3};
		
Task<IOrderedEnumerable<int>> sortedArr = arr.OrderByAsync(shitSorter);

zokie
Feb 13, 2006

Out of many, Sweden
Is it possible to not have your callers pass a Func<T, Task<int or some sortable value>> and then you can turn the ienumerable to an ienumerable<Tuple<T, int>> in an async way and use the “sort/compare value” in the sync Sort call?

Calidus
Oct 31, 2011

Stand back I'm going to try science!
Use Polly to wrap user code and apply timeouts. If people start to complain deal with it then.

EssOEss
Oct 23, 2006
128-bit approved
You need to create a DTO and apply AutoMapper to this problem to truly solve it without excessive coupling. It mostly has async support nowadays.

Rocko Bonaparte
Mar 12, 2002

Every day is Friday!
The sorting problem is that I need to specify an IComparable that invokes functions in the scripting runtime that do the comparisons. The functions are async categorically, even if I don't expect them to actually block in this function. However, that's the interface. The problem is I don't know if I can reasonably switch the IComparables prototype to become async. I'm assuming I have to write my own sort implementation, which isn't the end of the world.

Edit: The data is all there. The comparison should normally have everything it needs. However, the internals for invoking the call to do the comparison is async.

distortion park
Apr 25, 2011


I feel like any simple solution is going to be dogshit slow for a collection of any size, so you might as well just wrap the comparators in blocking calls on each invocation and get on with your day.

Pennywise the Frown
May 10, 2010

Upset Trowel
Cross post from wrong thread.

Pennywise the Frown posted:

Do you guys do C# in here? I'm just looking for some (hopefully free) online tutorials to help me continue my learning. I finished a semester in school and we have this stupid loving "book" where we have access to it virtually for 4 months (full price) and then we lose access at the end of the semester so I don't have that resource anymore. :patriot:

I'm going to check out W3 Schools I think first. Any other guides/tutorials?

Eldred
Feb 19, 2004
Weight gain is impossible.

Pennywise the Frown posted:

Cross post from wrong thread.

I’d check out MS Learn, they have C# and .NET courses. MS’ docs in general are good if you know what you want to build and just need guidance on how to do it.

Pluralsight has decent .NET content too if you’re willing to pay for it.

Adbot
ADBOT LOVES YOU

Pennywise the Frown
May 10, 2010

Upset Trowel
I'll check that out. Thank you!

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply