Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
stevefukkinc
Oct 10, 2004
TUB-THUMPING,
PAROCHIAL
McKNOB

Victor posted:

There's also the issue of needing to verify data integrity/quality before you shove it in a production table.

Very, very true. Keep an eye out for fields being cropped and row separators not performing correctly. I use the BCP tool a fair amount in ASE and SQL2005, I would recommend always using a format file rather than the -c switch. It's slight more laborious but it's helpful when you are trying to import a CSV (or what have you) that was supplied rather than generated personally.

Also, when working in Sybase IQ, the LoadTable command is a total bitch, try and avoid where at all possible.

Adbot
ADBOT LOVES YOU

stevefukkinc
Oct 10, 2004
TUB-THUMPING,
PAROCHIAL
McKNOB

dangerous.hotdog posted:

I'm having a hell of a time with a particular query. There is a USERNAME and SYSTEMDATE field and I need to develop a query that filters out records with a USERNAME/SYSTEMDATE pair that falls within 30 seconds of another USERNAME/SYSTEMDATE pair (with identical USERNAME). Basically, if a user connects again within the same 30 second window of their first connect, we want to omit that record in our analysis.

I don't even know where to begin doing this in SQL (Oracle). I wrote up a VB app, but it's projected to take more than 9 hours to run through 300k records (and this is a very streamlined version of the app). Plus, since I can't create or update tables, I'll have to use that VB app to output some 300k records into an .XLS or .CSV file and then run the actual data analysis on this file (rather than querying the database directly). I imagine this'll just add even more overhead to the entire process.

So, long story short, is there any way I can do this type of query within SQL?

Assuming i've not missed something very obvious... Yes. If you are running on a fairly decent box it would take 5-10 minutes tops with a couple of procedures/functions nested. What if they try again after thirty seconds, you want that kept? Here is how I would do it... Assuming can create a temp table.

code:
-- work table
create table #process (
			id		int identity
			USERNAME	varchar(32)
			SYSTEMDATE	datetime
		       )

-- fill it with your data
insert into #process
select username, systemdatae
from ORIGINAL_TABLE

-- get your poo poo together
declare @x int
		, @y int
		, @currentuser varchar(32)
		, @currentdate datetime
		, @currentdate30 datetime

-- start and end points
select @x = (select min(id) from #process)
select @y = 1+(select max(id) from #process)


while @x < @y

BEGIN

select @currentuser = USERNAME
		,@currentdate = SYSTEMDATE
from #process
where id = @x

select @currentdate30 = dateadd(30, ss, @currentdate)

-- Now, remove the stuff you don't care about from your result set!

delete from #process
where USERNAME = @currentuser
and systemdate between @currentdate and @currentdate30

-- not just the next value, as you may have deleted it!
select @x = (select min(id) from #process where id > @x)

END
Obviously loving around with the precision of the datetime would be expected...

stevefukkinc
Oct 10, 2004
TUB-THUMPING,
PAROCHIAL
McKNOB

Triple Tech posted:

Or do you keep the goons table generic and make tables to join off to like goons_tfr, goons_gws, etc that consist only of the goon pk plus the exclusive attributes?

Without a shadow of doubt, this is the way to do it.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply