Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Hughmoris
Apr 21, 2007
Let's go to the abyss!
Here is a very vague question... How is C# for text parsing and regex expressons? I'm a non-programmer trying to help out at work, and we have a 60,000-line data log that I'd like to write a tool for and distribute to my coworkers. They'd input a user ID, and the program would parse the log pulling out relevant information. I'm a Python rookie and I think I could accomplish it in Python but I'd like to learn a language that is easily distributed in the Windows environment.

Hughmoris fucked around with this message at 01:47 on Feb 28, 2015

Adbot
ADBOT LOVES YOU

Jewel
May 2, 2009

Hughmoris posted:

Here is a very vague question... How is C# for text parsing and regex expressons? I'm a non-programmer trying to help out at work, and we have a 60,000-line data log that I'd like to write a tool for and distribute to my coworkers. They'd input a user ID, and the program would parse the log pulling out relevant information. I'm a Python rookie and I think I could accomplish it in Python but I'd like to learn a language that is easily distributed in the Windows environment.

Incredibly good. Python and C# are my two favorite languages because they both feel just as flexible as eachother out of the box (C# has the wonderful enumeration library and generally a lot of helper libraries/functions) and I use C# primarily now for the same reason you just said, it's easily distributed. Also typed languages are so much nicer than untyped at this point, not to mention the intellisense is great and the compiler errors are helpful.

New Yorp New Yorp
Jul 18, 2003

Only in Kenya.
Pillbug

Hughmoris posted:

Here is a very vague question... How is C# for text parsing and regex expressons? I'm a non-programmer trying to help out at work, and we have a 60,000-line data log that I'd like to write a tool for and distribute to my coworkers. They'd input a user ID, and the program would parse the log pulling out relevant information. I'm a Python rookie and I think I could accomplish it in Python but I'd like to learn a language that is easily distributed in the Windows environment.

It's fine, but PowerShell might be more up your alley as a non-programmer. PowerShell has regex operators built right in.

http://www.powershelladmin.com/wiki/Powershell_regular_expressions

Bognar
Aug 4, 2011

I am the queen of France
Hot Rope Guy

EssOEss posted:

That is foo.SomethingHappened += bar.EventHandler makes foo hold a reference to bar but not the other way around. If bar sets all references to foo to null, it will go away (provided that no other references exist). In case the handler is a closure, foo just gains a reference to the closure object that the compiler generates, which may in turn contain references to other objects.

And if your closure is referencing the event publisher (or other objects that reference it), you've created a cycle which can cause a memory leak. The garbage collector in .NET is generally pretty good about culling reference cycles that are disconnected from the object graph but, for example, the garbage collector in Mono isn't.

RICHUNCLEPENNYBAGS
Dec 21, 2010

rarbatrol posted:

Okay, cool. Thanks. I wasn't sure how tasks crossed that boundary.

I imagine something like:

var data = await controller.Action();
return this.Serialize(data);

EssOEss
Oct 23, 2006
128-bit approved

Bognar posted:

And if your closure is referencing the event publisher (or other objects that reference it), you've created a cycle which can cause a memory leak. The garbage collector in .NET is generally pretty good about culling reference cycles that are disconnected from the object graph but, for example, the garbage collector in Mono isn't.

Ouch, really? Can you give some useful links that explain the trouble spots? We are just startng to use Mono at work and this sounds like something that we need to become aware of!

mastersord
Feb 15, 2001

Gold Card Putty Fan Club
Member Since 2017!
Soiled Meat

crashdome posted:

I'll disagree with RangerAce a bit. I've taken to using EF Entities as models in my viewmodels directly because they are perfect.

I've also not had a problem using multiple entity sets to a single table. Can you post your code? or an example?

e: Are there relations to other tables? This might be the problem. I have a legacy datatable where there were multiple columns for different row types but, there are no foreign keys. Alternatively, you could also try mapping entities to unique SQL views or sprocs named differently.

I'll try. I have 3 tables in my model: tblEmployers, tblShops, and tblEmployees. An employer can have 1 or more shops. A shop can have 1 or more employees.

I have a form with 2 ComboBoxes, a dataGrid, and a search button. ComboBox 1 (me.comEmployers) has a list of employers. I want to populate comboBox 2 (me.comShops) with a list of shops belonging to the selected employer by ID. I could populate the box through box 1s "selectedIndexChanged" event, but I want the display property to show both the number (not the shopID primary key)and name with a hyphen between them. Something like this:

code:
'Created entity ShopComboEntries in model with int32 nShopID as primary key and string sName

'Conn is my model which holds my objects
Private Sub comEmployer_SelectedIndexChanged(sender As Object, e As EventArgs) Handles comEmployer.SelectedIndexChanged
    Dim Shops As New Conn
    Dim SQL As String
    SQL = ""
    SQL = SQL & "Select [nShopID]," & vbCrLf
    SQL = SQL & "       [sName] = Cast(IsNull([nNumber], '') As nVarchar) + ' - ' + [sName]" & vbCrLf
    SQL = SQL & "  From [tblShops]" & vbCrLf
    SQL = SQL & " Where [nEmployerID] = " & Me.comEmployers.SelectedValue
    With Me.comShops
        .DataSource = Shops.ShopComboEntities.SqlQuery(SQL).ToList
        .ValueMember = "nShopID"
        .DisplayMember = "sName"
    End With
End sub
I could create a query that fits in the tblShops entity where I select Null for all the other fields I don;t need. I could create a view in the database and add that to the model and query that. What I would like to do is create an entity to hold the results of this query. I don't need a big object that holds 30 fields if I just need 2 of them. I am also not doing much with the results other than using the "nShopID" to query tblEmployees into my dataGrid.

I could create the objects in the database but I was wondering if I could avoid doing that.

mastersord fucked around with this message at 22:23 on Feb 28, 2015

Hughmoris
Apr 21, 2007
Let's go to the abyss!

Jewel posted:

Incredibly good. Python and C# are my two favorite languages because they both feel just as flexible as eachother out of the box (C# has the wonderful enumeration library and generally a lot of helper libraries/functions) and I use C# primarily now for the same reason you just said, it's easily distributed. Also typed languages are so much nicer than untyped at this point, not to mention the intellisense is great and the compiler errors are helpful.

Thanks, that's good to hear.

Ithaqua posted:

It's fine, but PowerShell might be more up your alley as a non-programmer. PowerShell has regex operators built right in.

http://www.powershelladmin.com/wiki/Powershell_regular_expressions

I don't know a ton about powershell but would there be an issue deploying powershell scripts on Windows XP computers? Or would I have to do some tinkering to get it to work?

Scaramouche
Mar 26, 2001

SPACE FACE! SPACE FACE!

Hughmoris posted:

Thanks, that's good to hear.


I don't know a ton about powershell but would there be an issue deploying powershell scripts on Windows XP computers? Or would I have to do some tinkering to get it to work?

Yeah you'd have to install powershell on the older machines, set up permissions, etc. Might be easier to just do a one-click installer or spread an exe around if you're comfortable with deploying that yourself.

Mr Shiny Pants
Nov 12, 2012

Scaramouche posted:

Yeah you'd have to install powershell on the older machines, set up permissions, etc. Might be easier to just do a one-click installer or spread an exe around if you're comfortable with deploying that yourself.

Might be worth to deploy PS. Once you've used it you'll probably see all kinds of possibilities :)

bobua
Mar 23, 2003
I'd trade it all for just a little more.

I have to be making this harder than it is...

I want to take a strongly typed datatable and group\order it into another datatable. So far, without grouping I do this...

code:
var res = (from table in datatable
		select table).ToList();
res.OrderBy(x => x.FirstName);
res.ForEach(x =>
{
	var newrow = OtherDatatable.NewOtherDatatableRow();
	if(!x.isFirstNameNull()) { newrow.FirstName = x.FirstName; }
	if(!x.isCompanyNameNull()) { newrow.Company = x.Company; }
	OtherDatatable.Rows.Add(newrow);
}
This feels stupid enough... but it works.

Now that I want to do a GroupBy and Sum, I'm lost. I tried
code:
var res = (from table in datatable
		group table by new
		{
			FirstName = table.FirstName,
			etc
		} into group
		select new
		{
			FirstName = group.Key.FirstName,
			Total = group.Sum(x= > x.Cost)
		}
Which works... but then I lose EF's helper funtions that check for DBNull, so I can't map the data back in to another strongly typed datatable. Please tell me I'm going about this completely wrong and there is a better way.

RICHUNCLEPENNYBAGS
Dec 21, 2010
If you're using EF why do you have to even do it with data tables?

crashdome
Jun 28, 2011
Entity Framework uses Entities/Entity Sets not strongly-typed Datatables/Datarows.

Simply use linq to Enumerate/Sort at the DB level instead of.... and.. Why clone the data into datatables of all things?

bobua posted:

I have to be making this harder than it is...

I want to take a strongly typed datatable and group\order it into another datatable. So far, without grouping I do this...

code:
var res = (from table in datatable
		select table).ToList();
res.OrderBy(x => x.FirstName);
res.ForEach(x =>
{
	var newrow = OtherDatatable.NewOtherDatatableRow();
	if(!x.isFirstNameNull()) { newrow.FirstName = x.FirstName; }
	if(!x.isCompanyNameNull()) { newrow.Company = x.Company; }
	OtherDatatable.Rows.Add(newrow);
}

Do you realize this code above takes a generalized SQL statement (e.g. "select * from myTable") and runs it on the database and then below that you spend a great deal of time sorting it and then cloning it to some other table when you could just...

code:
//Generates a query with all your requirements
var res = from table in datatable
		orderby (x => x.FirstName)
		group table by table.FirstName into g
            	select new
            	{
              		FirstName = g.Key.FirstName,
			Total = g.Sum(x= > x.Cost)
            	};
		
//Executes Query and give you just the list you want
res.ToList();
Untested of course but, should be the general direction you should go.

bobua
Mar 23, 2003
I'd trade it all for just a little more.

crashdome posted:

Entity Framework uses Entities/Entity Sets not strongly-typed Datatables/Datarows.

Simply use linq to Enumerate/Sort at the DB level instead of.... and.. Why clone the data into datatables of all things?


Do you realize this code above takes a generalized SQL statement (e.g. "select * from myTable") and runs it on the database and then below that you spend a great deal of time sorting it and then cloning it to some other table when you could just...

code:
//Generates a query with all your requirements
var res = from table in datatable
		orderby (x => x.FirstName)
		group table by table.FirstName into g
            	select new
            	{
              		FirstName = g.Key.FirstName,
			Total = g.Sum(x= > x.Cost)
            	};
		
//Executes Query and give you just the list you want
res.ToList();
Untested of course but, should be the general direction you should go.

I couldn't figure out how to check if the field was null when trying to do the orderby within the query, kept getting strongtypingexceptions. Doing the orderby afterwards on the list seemed to get around that.

I'm working with datatables because the original program did all of it's work in stored procedures while storing all of the results in the database, the new program needs to continue to use the stored procedures\results to database, but also import\export the data and run the same calculations without the use of the stored procedures.

RICHUNCLEPENNYBAGS
Dec 21, 2010

bobua posted:

I couldn't figure out how to check if the field was null when trying to do the orderby within the query, kept getting strongtypingexceptions. Doing the orderby afterwards on the list seemed to get around that.

I'm working with datatables because the original program did all of it's work in stored procedures while storing all of the results in the database, the new program needs to continue to use the stored procedures\results to database, but also import\export the data and run the same calculations without the use of the stored procedures.

I think there's a better way to work with SPs, but don't quote me.

Anyway, yeah, for the null thing, why not just .Where( x => x.sortProp != null).OrderBy(x => x.sortProp.Value) or whatever?

biznatchio
Mar 31, 2001


Buglord

crashdome posted:

Yeah, BindingSource is great. A list doesn't have any INotify ....blah blah... so you'd have to call Suspend, Resume, Refresh on the listbox control manually during the update. Thanks for reminding me how much I don't miss WinForms.

For Windows Forms data binding, you can use BindingList<T> (or the IBindingList interface), which supports all the list changed events you'd care about, and is natively supported by BindingSource and all the built-in controls.

It's basically the WinForms version of WPF's ObservableCollection.

mastersord
Feb 15, 2001

Gold Card Putty Fan Club
Member Since 2017!
Soiled Meat

bobua posted:

EF stuff

I am still learning EF so I could be talking straight out of my rear end here.

It's been suggested to use LINQ but why not use straight SQL and the "SqlQuery" method? In SQL server we use "IsNull" in the Order By clause to substitute a value into Null valued-columns when we need to sort them (i.e. "0" or some ridiculously high number depending on where we want them sorted to).

EssOEss
Oct 23, 2006
128-bit approved
I am pretty sure you can just do "myrow.mycolumn ?? 0" in LINQ to Entities queries, as well. Granted, I do not specifically have proof of this scenario in front of me but I have worked with null values before using EF and do not recall any particular problems or missing features.

Drastic Actions
Apr 7, 2009

FUCK YOU!
GET PUMPED!
Nap Ghost
Since we don't have a Windows Store specific thread, I'll guess I post this stuff here :shurg:. Microsoft talked a little bit about Windows 10 universal apps at MWC. Some of the Windows Phone APIs are going to be deprecated and made consistent with Windows itself. So DoSomethingAndContinue will be going away :woop:. It also seems like there will be more to have true cross-platform XAML support, with hopefully less conditional "#if WINDOWS_PHONE_APP" logic and more XAML solutions. They also showed off Xbox One Universal apps as well.

Gets me a bit amped for Build. Just months away from Awful Forums Reader on XB1 :getin:

Inverness
Feb 4, 2009

Fully configurable personal assistant.
Speaking of apps, it irritates me that WinRT and its special C++/CX language even exists. Why couldn't Microsoft have dropped their internal conflict bullshit before W8.

bobua
Mar 23, 2003
I'd trade it all for just a little more.

RICHUNCLEPENNYBAGS posted:

I think there's a better way to work with SPs, but don't quote me.

Anyway, yeah, for the null thing, why not just .Where( x => x.sortProp != null).OrderBy(x => x.sortProp.Value) or whatever?

Well, I don't know what I'm talking about but as far as I know, checking the field for null throws the exception, so you can't do that... you have to use the generated helper method to check if it is null instead of checking it directly. This works fine for me in most cases, but you lose that helper method when you group the data. I also don't know how to use that helper method when doing an order by\group by within the query.

My guess at the syntax was something like
code:
from table in mytable
orderby
{
(table.isCompanyNull()) ?? "" : table.Company
}
select table
But that didn't work out. I probably shouldn't have mentioned EF, because I really am treating this whole thing like the sql database doesn't exist, and I just have datatables to work with.

brap
Aug 23, 2004

Grimey Drawer
Are you serious? EF entities don't just return null if you access a nullable property that contains null? Also, table is not a good name in that context. You are working with individual records, not tables, so it's better to say something like for chair in chairs or for butt in butts.

You're treating this like the SQL database doesn't exist? What do you mean by that?

bobua
Mar 23, 2003
I'd trade it all for just a little more.

fleshweasel posted:

Are you serious? EF entities don't just return null if you access a nullable property that contains null? Also, table is not a good name in that context. You are working with individual records, not tables, so it's better to say something like for chair in chairs or for butt in butts.

You're treating this like the SQL database doesn't exist? What do you mean by that?

If you have the property set to nullable, yes. Properties like datetime can't be set to nullable though.

Because while the program originallly get's it's data from an sql database, it saves(and reimports that data to xml files). Each strongly typed datatable is a different step in a long process. Sometimes the tables need to be mixed an matched... table 1 from live database, table 2 from an xml file, resulting table pushed back up to the database... etc. It's always different.

RICHUNCLEPENNYBAGS
Dec 21, 2010

fleshweasel posted:

Are you serious? EF entities don't just return null if you access a nullable property that contains null? Also, table is not a good name in that context. You are working with individual records, not tables, so it's better to say something like for chair in chairs or for butt in butts.

You're treating this like the SQL database doesn't exist? What do you mean by that?

They do but he's not using them apparently.

EssOEss
Oct 23, 2006
128-bit approved

bobua posted:

If you have the property set to nullable, yes. Properties like datetime can't be set to nullable though.

If your database column is a nullable datetime but in code you have a non-nullable datetime, your model is wrong.

gariig
Dec 31, 2004
Beaten into submission by my fiance
Pillbug

bobua posted:

If you have the property set to nullable, yes. Properties like datetime can't be set to nullable though.

Because while the program originallly get's it's data from an sql database, it saves(and reimports that data to xml files). Each strongly typed datatable is a different step in a long process. Sometimes the tables need to be mixed an matched... table 1 from live database, table 2 from an xml file, resulting table pushed back up to the database... etc. It's always different.

Can you stop using DataTable? It's a very leak abstraction and I think you are having trouble because you are using three different technologies, DataTable, EF, and LINQ-to-objects. My suggestion is to make a POCO (Plain Old CLR Object) model, have each step project the data into your POCO model, do your model mutations, then project that into the database. At the very least dumping DataTable will probably help out. Also, posting more code could be helpful but I think is more of an architecture problem than a coding error problem.

You could also move to something like SSIS (SQL Server Integration Services) or some other ETL (Extract, Transform, Load) system.

bobua
Mar 23, 2003
I'd trade it all for just a little more.

gariig posted:

Can you stop using DataTable? It's a very leak abstraction and I think you are having trouble because you are using three different technologies, DataTable, EF, and LINQ-to-objects. My suggestion is to make a POCO (Plain Old CLR Object) model, have each step project the data into your POCO model, do your model mutations, then project that into the database. At the very least dumping DataTable will probably help out. Also, posting more code could be helpful but I think is more of an architecture problem than a coding error problem.

You could also move to something like SSIS (SQL Server Integration Services) or some other ETL (Extract, Transform, Load) system.

I probably can stop using datatable in the long term. I think you're right on this just being an architecture problem, but I needed something done in a week without being allowed any changes to the database, and it doesn't help that this is really the first time I've used sql, datatables, or linq...

In any case, I got it working last night with a mix of 'sanitizing' the table before grouping and removing some of the original(stored procedure) author's grouping that wasn't actually necessary.

MonkeyMaker
May 22, 2006

What's your poison, sir?
Anyone here interested in teaching .NET/C#? We looking for a new teacher at Treehouse. I can vouch for it being an awesome place to work. PM me with any questions.

Sorry if this isn't the right place to post.

crashdome
Jun 28, 2011
App.Config question:

I have a WPF Application with a single project for ViewsVMs/Services/Etc. I also have two additional projects referenced by the main project. Each sub-project is basically an EF Project each with it's own connection string and data models.

When I build the solution, the connection strings are not merged up in to my main project App.Config. What am I doing wrong?

Optionally, is there a way I can have a single App.Config all projects can reference without EF wizard complaining a connection doesn't exist?

gariig
Dec 31, 2004
Beaten into submission by my fiance
Pillbug

crashdome posted:

App.Config question:

I have a WPF Application with a single project for ViewsVMs/Services/Etc. I also have two additional projects referenced by the main project. Each sub-project is basically an EF Project each with it's own connection string and data models.

When I build the solution, the connection strings are not merged up in to my main project App.Config. What am I doing wrong?

Optionally, is there a way I can have a single App.Config all projects can reference without EF wizard complaining a connection doesn't exist?

Building doesn't merge the app.configs. However, you can have one app.config reference an external file (example).

RICHUNCLEPENNYBAGS
Dec 21, 2010

bobua posted:

I probably can stop using datatable in the long term. I think you're right on this just being an architecture problem, but I needed something done in a week without being allowed any changes to the database, and it doesn't help that this is really the first time I've used sql, datatables, or linq...

In any case, I got it working last night with a mix of 'sanitizing' the table before grouping and removing some of the original(stored procedure) author's grouping that wasn't actually necessary.

You don't need to change the table to use real entities.

Hughmoris
Apr 21, 2007
Let's go to the abyss!
For performance on a script that parses a 60,000+ line text file, does it really matter what method I use to read in the file and iterate over each line? Poking around C# tonight and it looks like there are quite a few different ways to read in files. Is there a standard method I should be using to read and write files, excluding special cases?

Inverness
Feb 4, 2009

Fully configurable personal assistant.

Hughmoris posted:

For performance on a script that parses a 60,000+ line text file, does it really matter what method I use to read in the file and iterate over each line? Poking around C# tonight and it looks like there are quite a few different ways to read in files. Is there a standard method I should be using to read and write files, excluding special cases?
Use a stream and text reader. Do not be one of those idiots that loads the entire file into memory at once.

Factor Mystic
Mar 20, 2006

Baby's First Post-Apocalyptic Fiction

Drastic Actions posted:

Since we don't have a Windows Store specific thread, I'll guess I post this stuff here :shurg:. Microsoft talked a little bit about Windows 10 universal apps at MWC. Some of the Windows Phone APIs are going to be deprecated and made consistent with Windows itself. So DoSomethingAndContinue will be going away :woop:. It also seems like there will be more to have true cross-platform XAML support, with hopefully less conditional "#if WINDOWS_PHONE_APP" logic and more XAML solutions. They also showed off Xbox One Universal apps as well.

Gets me a bit amped for Build. Just months away from Awful Forums Reader on XB1 :getin:

Is there a source what what WP APIs are dead?

Drastic Actions
Apr 7, 2009

FUCK YOU!
GET PUMPED!
Nap Ghost

Factor Mystic posted:

Is there a source what what WP APIs are dead?

Not yet, they just mentioned that the APIs are going to be folded together at MWC. So we'll find out more at build.

Inverness
Feb 4, 2009

Fully configurable personal assistant.

Drastic Actions posted:

Gets me a bit amped for Build. Just months away from Awful Forums Reader on XB1 :getin:
"Xbox, show me something awful."

New Yorp New Yorp
Jul 18, 2003

Only in Kenya.
Pillbug

Inverness posted:

Use a stream and text reader. Do not be one of those idiots that loads the entire file into memory at once.

Doesn't File.ReadLines give you an iterator that streams the file into memory one line at a time as an IEnumerable? That's probably a more straightforward choice than messing with streams directly, especially since streams aren't IEnumerable and thus won't play nice with LINQ without some extra hoop-jumping.

That's just from memory though, so I might be wrong.

Inverness
Feb 4, 2009

Fully configurable personal assistant.

Ithaqua posted:

Doesn't File.ReadLines give you an iterator that streams the file into memory one line at a time as an IEnumerable? That's probably a more straightforward choice than messing with streams directly, especially since streams aren't IEnumerable and thus won't play nice with LINQ without some extra hoop-jumping.

That's just from memory though, so I might be wrong.
It seems you're correct. ReadLines() was added in .NET 4. That explains that. So yes, you can use that and it will read each line on demand.

chippy
Aug 16, 2006

OK I DON'T GET IT
Another (VB) WinForms question for you. I found out yesterday that a custom control (a sort of file browser thing cobbled together out of a treeview) was throwing null reference exceptions in multiple places. However, none of the users had mentioned it to me. After some fiddling I realised that they are only thrown when I'm debugging, and when the app is running in deployment, something is swallowing them.

The exceptions are being thrown by code in event handlers for drag and drop events, there is no Try/Catch block in the handlers... where exactly 'above' the form can they be being caught? My googling so far has suggested that they might be being caught by a Try/Catch in the 'Main' sub, but I haven't explicitly implemented one so it seems that in Winforms that means it would be getting injected by the compiler when built - or, that it could be being caught by global exception handling, but ApplicationEvents.vb is empty.

Does anyone know where this exception could be going, or how I can found out?

update: I placed a button on the form that holds the control that's throwing the exceptions, that does nothing but throw a new NullReferenceException. If I click this in deployment, the unhandled exception is being alerted and the program exits, so it's not that somethign is catching all such exceptions. What is the difference here? The custom controls are in a seperate project in the same solution, could this have anything to do with it?

another update: I put a TreeView on the form next to the custom control and wrote drag-drop event handlers for it that just throw exceptions. They just disappear when running in deployment just like the custom control ones do. So I guess it's not to do with it being in a different project. Could it be something to do with it being a drag event?

chippy fucked around with this message at 11:29 on Mar 4, 2015

Adbot
ADBOT LOVES YOU

chippy
Aug 16, 2006

OK I DON'T GET IT
Ah. It's because it's in a drag drop event. Something to do with boundaries between managed and unmanaged code.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply