Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
aBagorn
Aug 26, 2004
Has anyone tried out ASP.NET vNext on a Linux/Mac box yet?

Unfortunately I don't have any box to work on right now (can't spin up a VM on the work PC and don't have the RAM to do it at home) but depending on how well it works I'd consider switching up the hosting for my personal site, as well as offering non-windows hosted web applications for people.

Adbot
ADBOT LOVES YOU

aBagorn
Aug 26, 2004
So as part of a side project, I've decided to spin off scraping NFL GameCenter's JSON data into a full fledged .NET API, which I will throw on GitHub and allow for open source usage.

Where I'm running into some trouble, from a conceptual perspective, is storage. The other API that I've seen for this has a script that pulls live game data every 15 minutes (or on request) into a flat file system and then all queries are done off of that. I think there's something to be said for locally storing the data, but I can't decide what storage solution to use. SQL Compact? Flat File JSON? I need to come up with a system that will just allow anyone to grab my DLL(s) and integrate without them having to setup the storage, but, like I said, I'm stuck.

If I'm not explaining this right or anyone needs clarification, let me know.

e:\/ SQLite looks like a pretty nice fit. Looking at sqlite-net (https://github.com/praeclarum/sqlite-net) it looks like it has enough functionality that I could do without a standalone ORM. Thanks a bunch!

aBagorn fucked around with this message at 23:00 on Aug 7, 2014

aBagorn
Aug 26, 2004
So I realized that I forgot to cross post from project.log over to here. Being that it's a .NET specific project, I think that you guys will be more interested than goons in general

http://forums.somethingawful.com/showthread.php?threadid=3657015

aBagorn
Aug 26, 2004
So potential can of worms here, but I'm considering taking an app that is little more than CRUD (and maybe some 'reporting' down the line) and making it an SPA with Web API on the server end.

What is everyone's favorite javascript framework? Angular, Backbone, Ember, React, Knockout + Require + Durandal (or sammy), etc?

I'm in the middle of watching a session on Angular on top of ASP.NET Web API and it looks like there's a steep learning curve. Am I right in my assumption there? Obviously everyone is going to have their opinion (as I've seen from googling) but I am interested in what the .NET megathread consensus is, since you guys are awesome

aBagorn
Aug 26, 2004
Looks like you can't without going raw SQL, or using a different DbContext for the initial import

http://stackoverflow.com/questions/7151710/how-do-i-temporarily-turn-off-an-identity-column-in-ef-code-first

aBagorn
Aug 26, 2004

Che Delilas posted:

ASP.NET MVC5 Identity problem.

I've been beating my head against the wall trying to understand Identity and how to make it play well with anything beyond the default MVC application. I am at a complete loss on how to proceed.

I have an MVC5 project where users can add Products into a database. Each Product entry must keep track of the user who added that Product in the first place. The default MVC5 project includes an IdentityModel.cs file with ApplicationUser and ApplicationDbContext defined. So okay, we've got a DbContext already, so let's fold the rest of my domain model into it to keep things "simple." My question is, how do I get the Identity-related stuff to play nice with my business entities? (Code and specific error follows).

Product.cs
C# code:
public class Product
{
    public int Id { get; set; }
    public string Name { get; set; }
    public ApplicationUser AddedByUser { get; set; }
}
IdentityModel.cs (Generated by default mvc project with my single addition at the end (DbSet<Product>))
C# code:
public class ApplicationUser : IdentityUser
{
    public async Task<ClaimsIdentity> GenerateUserIdentityAsync(UserManager<ApplicationUser> manager)
    {
        var userIdentity = await manager.CreateIdentityAsync(this, DefaultAuthenticationTypes.ApplicationCookie);
        return userIdentity;
    }
}

public class ApplicationDbContext : IdentityDbContext<ApplicationUser>
{
    public ApplicationDbContext()
        : base("DefaultConnection", throwIfV1Schema: false)
    {
    }

    public static ApplicationDbContext Create()
    {
        return new ApplicationDbContext();
    }

    public DbSet<Product> Products { get; set; } //I added this line
}
ProductController Add method
C# code:
[HttpPost]
[Authorize]
public ActionResult Add(Product p)
{
    if (ModelState.IsValid)
    {
	//I got this line from the AccountController generated by the default mvc project
        var usrmgr = HttpContext.GetOwinContext().GetUserManager<ApplicationUserManager>();
  
        p.AddedByUser = usrmgr.FindById(User.Identity.GetUserId());
        using (var db = new ApplicationDbContext())
        {
            db.Products.Add(p);
            db.SaveChanges();
        }
        return RedirectToAction("Index");
    }
    return View();
}
The error I get when I log in and attempt to add a product is an InvalidOperationException with the message, "An entity object cannot be referenced by multiple instances of IEntityChangeTracker." It's on the db.Products.Add(p); line.

I know, or at least I think I know, that the problem is coming from the fact that there are two conflicting contexts: one from the OwinContext that I need to get my UserManager in order to find the logged in user, and one from the new ApplicationDbContext that I'm using to try and add the Product into the database.

How do I get these things to work together? I've been trying to learn ASP.NET MVC for a little while now, and while I understand the basics of controllers and views and routing, the auth/Identity stuff is so twisty and obfuscated that I can't penetrate it. I certainly don't understand enough about auth to just ditch Identity and roll my own solution; even if I could get something to work I'm sure it would be full of security holes.

Additionally, if someone knows a resource that really explains auth and/or Identity, that would be really helpful in general.

This may not be 100% best practices, but I've never stored an actual user entity for audit tracking. You can just as easily just do this:

C# code:
public class Product
{
    public int Id { get; set; }
    public string Name { get; set; }
    public string AddedByUserId { get; set; }
}
And then in your controller:

C# code:
[HttpPost]
[Authorize]
public ActionResult Add(Product p)
{
    if (ModelState.IsValid)
    {
        p.AddedByUserId = User.Identity.GetUserId();
        using (var db = new ApplicationDbContext())
        {
            db.Products.Add(p);
            db.SaveChanges();
        }
        return RedirectToAction("Index");
    }
    return View();
}
And you will have a Product updated with the Id of the user that updated it. Put a FK constraint on your product table and then when you are querying do a simple join on the Id field to get back the username.

aBagorn
Aug 26, 2004

Bognar posted:

The problem is that the user manager has its own DbContext and you are trying to add an item from that context onto the new one that you're creating. Using the Id instead is a good way of doing it, but you need to make sure you still have a foreign key set up. This model should tell EF Code First to make AddedByUserId the foreign key for the AddedByUser relationship.

C# code:

public class Product
{
    public int Id { get; set; }
    public string Name { get; set; }
    public ApplicationUser AddedByUser { get; set; }
    public int AddedByUserId { get; set; }
}

You also could add a Data Annotation to my solution to force the foreign key relationship.

Either way, what he's saying is why it's not working. The user entity can't exist in both contexts at once.

aBagorn
Aug 26, 2004
Unit Testing and Web API 2 questions ahoy!!!

I have a basic framework laid out for how I want to start writing my tests (which is good, I'm learning TDD on the fly), but I'm kind of stumped with testing the put functionality.

As taken from here:
http://www.asp.net/web-api/overview/testing-and-debugging/mocking-entity-framework-when-unit-testing-aspnet-web-api-2#tests

The most basic PUT unit test should be something like:

C# code:
        public void PutProduct_ShouldReturnStatusCode()
        {
            var controller = new ProductController(new TestStoreAppContext());

            var item = GetDemoProduct();

            var result = controller.PutProduct(item.Id, item) as StatusCodeResult;
            Assert.IsNotNull(result);
            Assert.IsInstanceOfType(result, typeof(StatusCodeResult));
            Assert.AreEqual(HttpStatusCode.NoContent, result.StatusCode);
        }
which seems simple enough and does what it's supposed to, but what if I want/need to get deeper into testing to make sure the modified properties are correctly being modified, or am I going into the realm of validation instead of unit testing?

For example, we have a tenant object and a building object, and then there is a history mapping table that maps the start date and end date of a tenant's assignment to a building.

I want to check against that table (or mock it) when a tenant is edited to make sure that the new start date is greater than the previous start and end date (and so on and so forth).

Is this testing or validation is basically my big question...

aBagorn
Aug 26, 2004

Ithaqua posted:

If you're thinking that far ahead, you're not doing TDD. In TDD, you should be thinking about the next tiny, logical step that moves you closer to your end goal.

Right. That makes sense. I guess I'm mixing up my "project manager" and "lead developer" hats that I'm wearing for this particular project.

aBagorn
Aug 26, 2004
Random Entity Framework question:

When doing Code First, is there any tangible drawback to not including navigation properties on one end of a relationship?

For example:

C# code:

public class Foo
{
    public int Id {get;set;}
    public string FooThingy {get;set;}
   
    public virtual ICollection<Bar> Bars {get;set;}
}

public class Bar
{
    public int Id {get;set;}
    public int FooId {get;set;}
    public string BarThingy {get;set;}
}


By not having a virtual Foo in my Bar class, I miss out on being able to call Bar.Foo, but is that it? Keeping the properties one sided allows for easier JSON serialization, and the way the data is setup Bar should not be accessed directly anyway, always through Foo.

Am I making a huge boo boo?

aBagorn
Aug 26, 2004
Hey, just looking for a couple different opinions here. At my annual review, one of the goals in the next year (in line with me taking more of a leadership role here) is to start actually implementing a peer code review process, and I'm wondering how any of you that have more formalized processes go through with it?

e: v Exactly the type of answer I was looking for! Although 1 and 2 will be hard to implement currently as I am also in charge of ramping up the unit testing enforcement as well.

We are (again, slowly) migrating over to TFS, but we still have some old stuff using SVN

aBagorn fucked around with this message at 19:46 on Dec 1, 2014

aBagorn
Aug 26, 2004
Design question!

I currently have a project (in the baby stages) that has this basic setup


Ember JS
Web API 2
Service/Business Logic Layer
Entity Framework

I've obviously been trying to keep EF model objects away from my Web API layer, but when trying to flatten everything into nice DTOs for Ember JS, it's becoming increasingly annoying to do so. Should I be flattening in the service layer? I am mainly using that for data access and some calculations that need to be done before objects are saved to the DB.

I had been serializing to JSON at that layer and passing up to the Web API in that format and had hoped to just pass that right along to ember without touching it, but based on the requirements of the views, that is no longer an option (the relationships between the classes are all many to many due to needing historical data, and the view only needs to grab the current relationship).

Forgive me if I'm explaining this incorrectly.

aBagorn
Aug 26, 2004

Opulent Ceremony posted:

We do all our DTO flattening at the Service/EF layer because we use AutoMapper projections to select only the fields we want from the database.


What? You are serializing before you get to the WebAPI layer? One of the main points of WebAPI is so you can have a Controller Action like
code:
public IEnumerable<Junk> GetJunks()
and the client can decide what format they would like it serialized in via headers. If your WebAPI Actions look like
code:
public string GetJunks()
because the Service GetJunks uses already turns the data into a JSON string, that is wrong.

See, there you go. I was doing it wrong, I figured. Luckily I'm not too far enough along that refactoring is not an issue

aBagorn
Aug 26, 2004
So when you guys (in the last thread, or maybe it was somewhere else) said that at some point unit testing would just "click" and all of a sudden completely make sense and you will never want to code a different way again...


You guys were right. Holy poo poo. TDD is like crack. Every time another test goes green I get a little buzz.


Thanks again goons :glomp:

aBagorn
Aug 26, 2004
Evil Sagan I'll put together a couple examples tomorrow morning.

I started with really easy stuff, unit testing CRUD stuff (make sure GetAllFoos actually gets all the Foos) and then work on actual business logic like making sure the Foos are Frobbled and like.

aBagorn
Aug 26, 2004
Ok as promised. I'm using Microsoft Test and Moq. And like I said, this is where I started. It made doing the harder ones easier:

Here's a simple test asserting that calling the GetActiveFoos method will only return Foos who have isActive = true
C# code:
[TestMethod]
public void GetActiveFoos_ShouldNotReturnInactive()
{
    //setup
    var mockSet = SetupMockFooDbSet();
    mockContext.SetupGet(c => c.Foos).Returns(mockSet.Object);
    var service = new FooService(mockContext.Object);

    //act
    var buildings = service.GetActiveFoos();

    //assert
    Assert.AreEqual(4, foos.Count);
}
The SetupMockFooDbSet takes a prefilled list of Foos (10 in total, 4 active) and sets up all the EF behind the scenes stuff to mock it out.


Here's a test asserting that an Add method will fail correctly if passed an empty sring (the real method takes a JSON object)
C# code:
[TestMethod]
public void Add_ShouldThrowExceptionWhenNoData()
{
    //setup
    var service = new BuildingService(mockContext.Object);
    var expected = ResponseMessages.Exception.GetDescription();

    //act
    var actual = service.Add(string.Empty);

    //assert
    Assert.AreEqual(expected, actual);
}
Lastly, this is a test to make sure that the Foos are flattened with their relational objects before being sent up to the API, as well as the inverse, that a flattened object coming down should be turned into 3 distinct EF objects before being saved to the database.

C# code:
[TestMethod]
public void Flatten_ShouldCreateValidDTO()
{
    var service = new FooService(mockContext.Object);
    var dto = service.FlattenFooObjects(MockFoo());

    Assert.IsTrue(dto is FooDTO);
}

[TestMethod]
public void Unflatten_ShouldCreateDatabaseObjectsFromDTO()
{
    var service = new FooService(mockContext.Object);

    var tuple = service.UnflattenDTO(MockDTO());

    Assert.IsTrue(tuple.Item1 is Foo);
    Assert.IsTrue(tuple.Item2 is FooBazzRelationalObject);
    Assert.IsTrue(tuple.Item3 is FooBarRelationalObject);
}

aBagorn
Aug 26, 2004

Calidus posted:

I generally just gave up on WPF when trying to switch form WinForms, and went straight to MVC web apps.

This is what I did too. And now I'm further down the web rabbit hole with all the JS frameworks with Web API 2 backends.

aBagorn
Aug 26, 2004
Does anyone have any experience with Entity Framework in a load balancing environment?

And if so, what sort of pitfalls should I be looking out for with regard to db locking / results caching?

aBagorn
Aug 26, 2004

Ithaqua posted:

Stored procedures encourage you to cram business logic in your data access layer, which is never a good thing. I'd tend to agree with your architect, even taking SSDT into account. Have you ever renamed a column used in several dozen stored procedures? It can be a big pain in the rear end.

SSDT doesn't care what you use for source control, it just represents your database schema in a source control-friendly fashion.

The client's DBA I'm working with is the opposite.

He wants everything done with stored procs, and I'm trying to convince him to let us use EF, hence my above question. He said something at our last meeting about EF caching and locking data in a load balanced environment, and that it scales horribly because of that, but prefaced that he last worked with EF 4. I can't find any definitive definitions either way about EF6.

If my application is being used by 2+ servers simultaneously to hit the same database, how is EF6 going to handle concurrency? Should I be disposing context every chance I get to avoid race conditions?

aBagorn
Aug 26, 2004
Follow up, so if I'm reading these correctly, EF does NOT by default lock DB rows?

This is something that he's hung up on and the more ammo I can take in the better.

aBagorn
Aug 26, 2004

bpower posted:

I recently started web dev with .net mvc c# and its been great. With VS and ReSharper my productivity has been really really high. In the last month I started using Angularjs and my productivity has tanked. The intellisense for JS seems useless, it reminds of writing Java in Notepad. How do you discover anything? Can you guys describe your workflow / tools you use for front end dev? Is there anything comparable to vs/resharper for javascript?

Sup "just got into Angular" buddy!

What version of Visual Studio?

The Web Essentials add on is amazing for VS2012-2013. Improved JS Intellisense (including most angular built in stuff).

aBagorn
Aug 26, 2004

Dreadrush posted:

I use Visual Studio for C# dev, and Sublime Text with a bunch of plugins for front-end work.

When writing a single page app, I return html/js/css statically, not requiring ASP.NET MVC.

I started doing this today, and so far it's pretty great.

What ST plugins do you recommend?

aBagorn
Aug 26, 2004

Ithaqua posted:

Since this is the catch-all Microsoft thread: I finally got a chance to hunker down and play with the "vNext" build stuff that's coming to Visual Studio Online / TFS vNext, and it's awesome. I set up a build agent on a Linux box and got it to clone Spring from Github and build it with Gradle in about 10 minutes.

Then I repeated the exercise, except for Windows. It's crazy easy.

Blog post on vNext build (not mine!):
http://geekswithblogs.net/jakob/archive/2015/01/15/tfs-build-vnext-ndash-a-preview.aspx

This makes me very excited.

aBagorn
Aug 26, 2004

epalm posted:

drat, that's hot. :allears:

Seriously, that's sexy stuff.

I'm going to build a couple of proof of concepts with this to show my co workers.

aBagorn
Aug 26, 2004

Drastic Actions posted:

Or if it works live, see some pretty cool Azure and Xamarin app interaction.

Go on... :allears:

aBagorn
Aug 26, 2004

VSCode is pretty sweet. I've been playing around with it on a Ubuntu machine at work.

aBagorn
Aug 26, 2004
Ok I hope I'm wording this right, but I want to figure out how to do something, and I'm really dumb when it comes to async/await stuff in general.

Currently we are processing csv files (that could contain upwards of 5 million rows) and populating the results into objects that will eventually live in the database. The methods are pretty convoluted and involve multiple steps, which all have to be done in order (for now). Some samples of the code I've inherited below. I feel like it belongs in the coding horrors thread.

C# code:
public bool ProcessInputFile(string inputFilePath)
{
    using (Stream stream = File.Open(inputFilePath, FileMode.Open, FileAccess.Read, FileShare.None))
	{
		var rawDataList = GetRawDataList(stream);
		var rawDataRowList = ProcessRawDataRowList(rawDataList);
		
		List<dbObject> objectList = new List<dbObject>();
		
		var count = 0;
		
		using (var _context = new ApplicationContext)
		{
			foreach (var rawDataRow in rawDataRowList)
			{
				//create new dbObject 
				var newDbObject = CreateNewDbObjectFromRawDataRow(rawDataRow);
				var dupeCount = 0;
				
				//here be logic here to check against the database to see if this is a duplicate
				
				if (dupeCount == 0)
				{
					objectList.Add(newDbObject);
					count++;
				}
				else
				{
					//logic here that writes to a logger that this record was flagged as a duplicate
					dupeCount = 0;
				}
				
				if (count > 1 && count % 30000 == 0)
				{
					_context.dbObjects.AddRange(objectList);
					_context.SaveChanges()
					objectList.Clear();
				}
			}			
			if (objectList.Count > 0)
			{
				_context.dbObjects.AddRange(objectList)
			}
		}
	}
}

private List<RawData> GetRawDataList(Stream stream)
{
	//logic here to turn each line into a RawDataObject (contains data as string and row number from csv file)
	return rawDataList; 
}

private List<RawDataRow> ProcessRawDataRowList(List<RawDataRow> rawDataList)
{
	List<RawDataRow> rawDataRowList = new List<RawDataRow>();
	foreach (var rawData in rawDataList)
	{
		//split rawData on '|' delimeter and validate each field
		var rawDataRow = ConstructRawDataRowObject(rawData.data);
		rawDataRowList.Add(rawDataRow);
	}
	
	return rawDataRow;
}

private dbObject CreateNewDbObjectFromRawDataRow(RawDataRow row)
{
	//logic here to transform RawDataRow to dbObject	
}
So the process basically flows file -> list of rawDataObjects -> foreach loop to make list of rawDataRow objects -> foreach loop to transform rawDataRow to dbObjects and save them via EF in batches.

I don't have too much leeway to completely gut everything (i.e., we save those rawData objects to their own table at one point and FK the dbObjects to them) so I can't really skip any of the steps.

What I'd like to do, however, is potentially run the first foreach loop until I hit some arbitrary number of rawDataRow objects (say 50k) and immediately kickoff the foreach(var rawDataRow in rawDataRowList) loop with that set while the next 50k rawDatas get transformed into rawDataRows.

This should be possible, right?

aBagorn
Aug 26, 2004

ljw1004 posted:

Just as a sanity check -- why?

I assume you're running some kind of nightly batch processor that does this work? And you have more CPU/RAM on your processing machine to throw at the problem? Are you looking to have the overall job to COMPLETE in a shorter time? Or are you looking to have partial results available immediately for some reason?

(I ask because async/await is normally unrelated to these kinds of problems, and it's important to know exactly what+why you're doing...)

I'm looking to have the overall job to complete in a shorter time. We're running this in Azure so shorter processing times = less $$$

(I figured I didn't know what I was talking about)

As far as resources go, I cranked up the VM this service was running on to a D13 (8 cores 56GB RAM) and we're 3 days into processing a file with 5 million records, which is unacceptable to the client.

I have another strong feeling that reaching out to the DB (which lives on another server) during the foreach loops is also potentially a root cause of the problem, as well as improper management of the EF Context

aBagorn
Aug 26, 2004

gariig posted:

I haven't done much EF but it doesn't seem to be built to load millions of records.

It's not. I'd love to decouple this all from EF but that would require gutting the service, which isn't an option right now.

I could probably use a raw SQL query to pull a selection of Rows that are likely candidates for duplicates at the beginning of the process and put them in memory.

aBagorn
Aug 26, 2004
So I think I'm going to make a recommendation to ditch EF if at all possible.

This service was written before I got here and dealt with files that contained at most a few thousand rows, and the fact that it was not designed to scale is showing.

The only problem I forsee is that the original dev did things like this with EF relationships to the dbObject before inserting.

C# code:


dbObject.dbObjectOwner = new dbObjectOwner(); 

dbObject.ListOfThingsRelated = ThingListCreatedBefore;


And inserts with foreign keys and multiple join tables for many to many relationships.

I started looking into BULK insert but it seems like it's going to be multiple steps, especially if we are getting away from inserting these fully hydrated EF objects

aBagorn fucked around with this message at 22:51 on Jun 10, 2015

aBagorn
Aug 26, 2004

Ithaqua posted:

It was awesome for node, though.

:agreed:

It's really great with Node. I found it more intuitive than the Node Tools for Visual Studio add on, though I only used that once

aBagorn
Aug 26, 2004

Mr Shiny Pants posted:

This happens frequently....

Yeah this is pretty much every day for me, especially in non C# code (Javascript frameworks and SQL mostly).

aBagorn
Aug 26, 2004

iron buns posted:

You mean this? It's pretty cool, even if it's not the full Visual Studio.

This is my main IDE these days, now that I'm apparently a React/Node/Express dev with a MacBook

Adbot
ADBOT LOVES YOU

aBagorn
Aug 26, 2004

Uziel posted:

Yes, or figure out the equivalent if I can't do that? Intellisense is giving me the impression that I can though.

I haven't written C# in a year or so but you should be able to use .Any() on the list instead of .Contains()

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply