Ask General Programming Questions Not Worth Their Own Thread

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›3 »

Computer viking: May 30, 2011; Now with less breakage.

Biowarfare posted:

Not really a programming question but something I've been curious about for a while

how would you make a large MMO save state?

I mean, you'd be dealing with huge amounts of changes per second that need to be saved (every skill addition, every experience gain, every item move or transfer) that would completely gently caress over iowait if you did them realtime, do you just throw huge arrays of SSDs and hardware at the issue or do you do something like save state in memory then do a mass-dumpout to transaction log/database once every x seconds?

At a guess, you push the rarer events (item gains, achievements, quest flags, experience) to the DB server immediately, while the constant-stream events (positions, mostly) can be polled on a timer. As for how bad it is, hmm. Say you've got a WoW server just after an expansion. They let something like 2000 people on at the time, and I don't think they will kill more than one creature every 10 sec on average. That's 200 exp gain updates/second, plus about the same amount of gold and item updates, and somewhat fewer quest flag updates. If you poll positions every 10 sec as well, you get something like 1000 updates/second. That sounds manageable enough for a decent DB server?

# ¿ Jan 23, 2012 21:34

Adbot: ADBOT LOVES YOU

# ¿ May 3, 2024 21:47

Computer viking: May 30, 2011; Now with less breakage.

GrumpyDoctor posted:

Am I really in the minority here in assuming that the pseudocode indicated fixed-width integers?

Well, you're at least not alone. I guess the question boils down to "what's the complexity of this in the typical implementation" vs "in the general case".

edit: As for the memoization, hmm. I can't really be bothered to work it out right now (I'm at work and should really be doing something else), but I have a feeling it will get it down below O(N�). I did some empirical testing, though. Given a simple python implementation of the two variants and some timing, I got absolutely beautiful timing results for the recursive case, closely approximating T(n) = T(n-1)+T(n-2), starting around n=4.

code:

0: 0.02
1: 0.02
2: 0.06
3: 0.10
4: 0.18
5: 0.29
6: 0.51
7: 0.80
8: 1.31
9: 2.21
10: 3.50
11: 5.67
12: 9.19
13: 14.68

The memoized variant was harder to test, since I ran into the recursion limit around n=1900. It's also got a ... less obvious shape. Plotted, it looks like this: The black line is the time, the red line is just a straight line from T(0) to T(max(n)), the dots is the difference between them (arbitrarily rescaled; the absolute values aren't important), and the blue line is the 0-axis for the dots.

.
N on the X axis, time on the Y axis. The idea was that if the actual function was e.g. O(N + smallConstant * N�), subtracting out the O(N)-component would make it easier to see the shape of the rest ... but I'm not sure how much that helped.

edit2: Of course, this is as much a test of the dictionary class in python as anything else. The code I used was this:

code:

import timeit

def fibRec(i):
	if (i==0 or i==1):
		return 1
	else:
		return fibRec(i-1) + fibRec(i-2)

def fibRecMemo(i, cache):
	if i in cache:
		return cache[i]
	if i==0 or i==1:
		return 1
	res = fibRecMemo(i-1, cache) + fibRecMemo(i-2,cache)
	cache[i] = res
	return res


def runFibRec(i):
	cache={}
	return fibRecMemo(i,cache)

rRes={}
mRes={}

for i in range(100):
	rTime = timeit.Timer("fibRec(%i)" % (i), setup = "from __main__ import fibRec")
	mTime = timeit.Timer("runFibRec(%i)" % (i), setup = "from __main__ import runFibRec")
	rRes[i] = rTime.timeit(number=100000)
	mRes[i] = mTime.timeit(number=100000)
	print("%i: %.2f / %.2f" % (i,rRes[i], mRes[i]) )

edit3: Having thought about it for a bit, etc. Assume that the recursive calls are done with the n-1 first. The first chain of recursion to hit the "bottom" will have cached all the relevant values, so all the (n-2)-calls will just return a cached value. This effectively reduces the problem to O(C*n), where C is the cost of two function calls, the addition, a cache insertion, and a cache lookup. If you draw out the call tree for the non-memoized case, the memoization prunes everything except the leftmost depth-first branch and one right-hand call for each level.

If we operate with O(1) addition and memoization operations (e.g. fixed-width ints and a preallocated zeroed array of fixed-width cells in storage with real random access) the entire thing seems to be O(N). Given variable-width ints it's O(N*log(N)).

edit4: drat, I was supposed to be doing things with research data at work.

Computer viking fucked around with this message at 16:48 on Feb 8, 2012

# ¿ Feb 8, 2012 13:42

Computer viking: May 30, 2011; Now with less breakage.

gwar3k1 posted:

I didn't know if this was too basic for the game thread or not. If I have a tool that reads text files and converts characters to a MapTiles collection, do I keep those text files as part of my solution or do I store the final MapTiles collection in some manner?

Whichever is more convenient for you and/or the user? Storing the serialized MapTiles will probably give you shorter loading times, but it means you need to re-build them if you change the text files (and it makes it harder for others to change things - which could be good or bad).

Computer viking fucked around with this message at 23:35 on Feb 10, 2012

# ¿ Feb 10, 2012 11:50

Computer viking: May 30, 2011; Now with less breakage.

Combat Pretzel posted:

Is there a point in writing a software design document for myself? I mean the text sort of type, not UML diagrams or similar.

--edit: Another question, if I were to design a personal finance application, should I bother writing a delicate SQL backend, or just load everything into memory and write out as needed? Should I favor "good" design over KISS? --edit: I'm going by an assumption that there might be 100 transactions a month. Trying to keep 10 years of data would result in 12000 rows.

You could use text files as the storage system, but an in-memory SQL engine. That would let you get the nice things of SQL (it is a rather nice language for querying data) without tying yourself to any fragile on-disk formats. Write it right, and it'll even be possible to plug it into another SQL backend later if you suddenly decide you really want to use BerkleyDB or SQLite or something.

# ¿ Feb 13, 2012 22:08

Computer viking: May 30, 2011; Now with less breakage.

For the cases where you're filling out a UI-table from stored data, I don't think you'll gain anything by selecting out all the data and doing the calculations in your programming language - just do the math in SQL. It won't be much slower, it'll probably be easier to read, and it'll handle the Decimal math for you.

No comment on the opposite case (data filled in, but not yet stored to the DB), except "make sure you're not doing floating point math on dollar values".

Computer viking fucked around with this message at 23:13 on Feb 13, 2012

# ¿ Feb 13, 2012 23:10

Computer viking: May 30, 2011; Now with less breakage.

Combat Pretzel posted:

Hmmm, OK. I think I can accelerate things quite a bit by maintaining cached values of total in- and outflows for each month in the database. Keeping each month updated should be trivial using UPDATE statements where needed. If I need total balances and such, I could run queries on these few rows instead of large tables. Unless there's issues with this, I guess I'm set for the storage design. Thanks.

Keep in mind that summing ten thousand values in memory takes a handful of miliseconds - I wouldn't put in the caching until you know it's needed. (Start off without it, generate test data on the large side of realistic, and see how bad it is.)

edit: It's certainly a possibility, of course. If it does work out to be a bottleneck, you could either keep the numbers in a separate table or generate them on loading (which would take a bit longer but reduces the danger of out-of-sync values). The neatest would probably be to select them into a temporary table; that's a single SQL statement (with some fairly simple group by - clauses). Updating is going to be "fun" - as mobby says, you could use triggers, or remember to do it by hand whenever you update or insert anything.

Computer viking fucked around with this message at 23:46 on Feb 13, 2012

# ¿ Feb 13, 2012 23:33

Computer viking: May 30, 2011; Now with less breakage.

I just found something neat and relevant here (pdf : "The use of a Kolmogorov-Smirnov type statistic in testing hypotheses about seasonal variation", L.S. Freedman 1979, Journal of Epidemiology and Community Health). I'll try to summarize it, but beware that I'm very definitely not a statistician.

First, calculate what fraction of sales for a customer happened each month, then, calculate the cumulative sums for that list (That is, make a new list where the value for a month is the sum of that month plus all earlier months). December ought to get 1.

Now, make another list of how large a fraction of sales you'd expect to have happened if sales were perfectly even (and for the sake of simplicity, the months equally long) : 1/12 at the end of jan, 2/12 at the end of feb, up to 12/12 at the end of dec. Subtract one list from the other, so difference = actual-expected.

If the data had not been seasonal, you could then find the largest absolute value in that list of differences, and use that as a score: Larger means more different. However, since these are months and wrap around, a small bit of compensation is needed.

Find the largest number in the list of differences, and call it Dmax. Find the smallest (quite possibly negative) number, take the absolute of it, and call it Dmin. The interesting score is Dmin+Dmax - which stays constant if you rotate the months around.

To illustrate, consider these distributions:

code:

Raw numbers:
random:    26  6  6 17 12 24 28 19 25 13 29 25
christmas:  1  1  1  1  1  1  1  1  1  1 10 20
sales:     20  1  1  1  1  1  1  1  1  1  1 10

Fractions:
random    : 0.11 0.03 0.03 0.07 0.05 0.10 0.12 0.08 0.11 0.06 0.13 0.11
christmas : 0.03 0.03 0.03 0.03 0.03 0.03 0.03 0.03 0.03 0.03 0.25 0.50
sales     : 0.50 0.03 0.03 0.03 0.03 0.03 0.03 0.03 0.03 0.03 0.03 0.25

Cumulative:
random    : 0.11 0.14 0.17 0.24 0.29 0.40 0.52 0.60 0.71 0.77 0.89 1.00
christmas : 0.03 0.05 0.08 0.10 0.12 0.15 0.18 0.20 0.23 0.25 0.50 1.00
sales     : 0.50 0.53 0.55 0.57 0.60 0.62 0.65 0.68 0.70 0.72 0.75 1.00

If I plot them, they look like this:

The blue line is the cumulative sales fraction for the theoretical "every month is identical"-case, the red line is the actual cumulative sales, and the green dots the difference each month. The score in the simple non-seasonal case is the maximum of the green dots. Dmin is the green value where the red is the furthest below (or least above) the blue, and Dmax is the green value where the red is the furthest above the blue.

As for the scores:

code:

random:    0.1550725
christmas: 0.5833333
sales:     0.5833333

As for how to interpret those numbers, uuuhm. Let me read a bit further.

edit: Aha, there's a simple enough lookup table (calculated from 10000 random tests, which I guess was a bit of work in 1979). Take the score, multiply it with sqrt(12), and look up here:

code:

10%  :  0.58
20%  :  0.67
30%  :  0.75
40%  :  0.82
50%  :  0.89
60%  :  0.96
70%  :  1.03
80%  :  1.14
90%  :  1.21
95%  :  1.41
99%  :  1.66

In my test numbers, that means:

code:

christmas : 0.5833333 * sqrt(12) = 2.021 : >99%
random    : 0.1550725 * sqrt(12) = 0.537 : <10%

The percentages are something like "how often will a result like this not be because of random chance".

Computer viking fucked around with this message at 20:45 on Feb 22, 2012

# ¿ Feb 22, 2012 20:02

Computer viking: May 30, 2011; Now with less breakage.

I'm not entirely sure if I agree with that line of thought. If we make the (flawed, but I suspect less so each year) assumption that CS students and non-CS students start off with the same understanding (or lack of such), why should their introductions be different? If we also assume (I know...) that the introduction a non-CS student gets is a subset of what a CS student is supposed to learn the first year ... wouldn't the best language for that subset be the same for the two?

In other words. Teach the basic concepts (splitting problems into implementable chunks, statements/functions/variables/flow, debugging, maybe a small bit of OO) in whatever language makes that the simplest, then move the CS students over to another language in their next, more advanced, course. Besides, the training in jumping between languages is useful in itself.

Computer viking fucked around with this message at 21:01 on Feb 22, 2012

# ¿ Feb 22, 2012 20:57

Computer viking: May 30, 2011; Now with less breakage.

etcetera08 posted:

I really do think that's a pretty valid way to think about it too. The point at which I differ, however, is that there are some things that are necessary to spend time talking about when you teach, for example, Ada as a first language as opposed to Python (to use the example of my program). Those things, while almost always very important if you are going to be spending 3+ years studying CS, probably don't matter to people that are going to be spending the rest of their lives studying other things or working in unrelated fields. Teaching Python allows these latter people to be exposed to the very basics of CS and gives them an idea of what kinds of concepts are important to the field without forcing them to get too specific. CS majors, on the other hand, get exposed to these same things but also get a head start into the specifics and more advanced topics.

Also, this discussion began in regards to Java's relatively more difficult syntax being a barrier of entry. I do think that also can come into play in this discussion. I don't think many (any) people would argue that Python's syntax is more difficult than Java or Ada. You can make things happen with a Python program (or even with the REPL) with much less of a time investment. The problem is that the simpler syntax generally means you're being sheltered from some concepts that may or may not be incredibly important.

The question is then if it's harmful or time-wasting in the long term to start out with a language where you can ignore those things (only to bring them up later), compared to plowing straight into them from the beginning. Personally, I don't think it's a bad idea to start out by easing them into programming as an activity first, and moving into the complications and underpinnings later. It's easier to add more details later than to absorb everything at once, in my experience - even if the "details" are large and fundamental.

I don't teach programming or CS, though; maybe it would lead to annoying habits and weird misunderstandings that had to be polished out later.

# ¿ Feb 22, 2012 21:18

Computer viking: May 30, 2011; Now with less breakage.

It's only a terrible idea if there's a remote chance that you'll use the file as a module, though. (In other words, skipping it is fine if you know that you are doing it, and why).

Admittedly, my day-to-day python tends to be ugly little once-off scripts.

# ¿ Feb 22, 2012 22:18

Computer viking: May 30, 2011; Now with less breakage.

rolleyes posted:

Just to really hammer it home most languages have built-in types specifically for dealing with situations where factions are required but which would go to hell using normal floating point representations, such handling monetary values.

I've no idea what that is in Java as I've never used it, but in C# the relevant type is decimal.

As usual, Java isn't very different from C#: It's java.math.BigDecimal.

# ¿ Feb 26, 2012 16:25

Computer viking: May 30, 2011; Now with less breakage.

Bob Morales posted:

We use stuff like jQplot, and we found a iPhone library that does it (looks better too), but I'm still interested from an academic standpoint

I can't offhand get you any good documentation or discussion of them. However, pie charts sounds like they should be a fun project to code up by hand- the basic idea isn't hard, so the messy part is the corner cases. And the label placement, ugh. (Labels go in a box somewhere separate. There, done. :colbert:

)

edit: Actually, have something. The ggplot2 package in R is generally a neat piece of plotting software, and there are sometimes interesting discussions around it.

Computer viking fucked around with this message at 04:01 on Feb 29, 2012

# ¿ Feb 29, 2012 03:55

Computer viking: May 30, 2011; Now with less breakage.

The Aphasian posted:

I have a stupid MS Access question.

This is the basic data:
code:
Company       Individual Type     Individual
ABC Widgets   Primary             John Smith
Acme Inc      Primary             Jane Doe
Acme Inc      President           Pete Williams
I want to return a list of one person for each company, with a preference for President. So if there is a President, return that person, if there isn't return the Primary.
code:
Company       Individual Type     Individual
ABC Widgets   Primary             John Smith
Acme Inc      President           Pete Williams
Is it something I put in criteria, or do I need to do something in SQL?

Thanks. I have no idea how to word these types of questions for Google.

Hmm. It's ugly, but you could do something like this in SQL:

code:

select * from 
    companies as c 
    join individuals as i on (
        i.indID = (
            select indID from individuals 
            where individuals.employer=c.compID 
            order by individuals.type="President" desc limit 1
        )
    )
;

That is - for each company, you take all their people, sort them by the result of the comparison "type='President'", and use the top row. If there is a president, he sorts on top, otherwise you get some random guy.

# ¿ Feb 29, 2012 17:11

Computer viking: May 30, 2011; Now with less breakage.

Ah, that's sort of the same idea - I cast the result of the comparison to something sortable, while you use a case to do the same thing. (I guess you could eek out a bit more performance by using a fixed number instead of rand(), too.)

I think depending on group by to return the top row is a mysql-only hack, though: AFAIK most other DBs only allow you to return aggregate functions and the field(s) you grouped on.

Computer viking fucked around with this message at 22:11 on Feb 29, 2012

# ¿ Feb 29, 2012 22:08

Computer viking: May 30, 2011; Now with less breakage.

Bob Morales posted:

I notice people on IRC can be huge assholes ~~if you don't use a certain one~~.

Obviously.

What do the IRC people prefer - or is that different in every channel?

# ¿ Mar 7, 2012 17:21

Computer viking: May 30, 2011; Now with less breakage.

shrughes posted:

Python has disturbingly weird scoping rules, is really slow, and is especially bad at multithreading, and is too verbose for shell scripting.

It's a really nice "get poo poo done"-language, for those problems that require a bit more than a compact shellscript but a bit less than an application. (I just used it to do some array-of-heatmaps-with-other-junk-hung-on-the-side visualization for work). It's a useful niche, but not an end-all/be-all.

# ¿ Apr 2, 2012 14:10

Computer viking: May 30, 2011; Now with less breakage.

shrughes posted:

Yes, certainly. It outmatches Perl or Ruby, going by the get-poo poo-done metric. There is a reason I use it, after all

If you want an unpleasant thought, imagine doing everything in R. Most of the people at work know some R and not much else programming-wise, and would prefer me to leave them scripts in a language they can read. Now, R is a really nice language for its niche, but as a general tool? Uurgh.

# ¿ Apr 3, 2012 10:18

Computer viking: May 30, 2011; Now with less breakage.

Internet Janitor posted:

Forth is a low-level systems programming language that is essentially a glorified assembler with a simple subroutine calling convention and the ability to add new syntax or compile-time behaviors. This means you can build it out into whatever language is most appropriate for your task. Factor is more of an applications-level language with features like garbage collection and object orientation baked in. Factor gives you more capabilities out of the box and is much "safer" (for example, like StrongForth it statically verifies your stack effects), but it has a great optimizing compiler and is wicked fast. Factor also replaces most stack-twiddling words with stack combinators that distribute arguments and apply them to anonymous words stored on the stack, leading to a more functional style of programming.

Regarding Forth: I found this an interesting enough read.

YosefK posted:

This is a personal account of my experience implementing and using the Forth programming language and the stack machine architecture. �Implementing and using� - in that order, pretty much; a somewhat typical order, as will become apparent.

It will also become clear why, having defined the instruction set of a processor designed to run Forth that went into production, I don�t consider myself a competent Forth programmer (now is the time to warn that my understanding of Forth is just that - my own understanding; wouldn�t count on it too much.)

# ¿ Apr 8, 2012 18:06

Computer viking: May 30, 2011; Now with less breakage.

Lareous posted:

Maybe someone can help with this.

I have a project involving an online employment application where the HR person wants the hiree to fill out the form online, hit submit, and have a pre-populated PDF sent to her email with the info that the hiree entered already filled out.

Is this possible to do without a database or should I just bite the bullet and learn some SQL?

It's entirely possible; you can store the entries in a text file or something and look up the right line by email/SSID/phone number/whatever reasonably unique information you both store and can ask the candidate to reproduce. Mind your locking when writing to it, though.

That said, I would use a database; it will be easier in the long run.

As for PDF, it depends on what tools you can put on the server; one possibility is to fill the info into a LaTeX document and render a PDF from that. (Which would be entirely ignorant of how you store your data.)

Computer viking fucked around with this message at 18:04 on Feb 11, 2014

# ¿ Feb 11, 2014 18:00

Computer viking: May 30, 2011; Now with less breakage.

I think he wants to merge the form data with pre-stored info about the candidate?

# ¿ Feb 11, 2014 18:06

Computer viking: May 30, 2011; Now with less breakage.

Lareous posted:

Alright good deal, I'll try it out. Thanks!

Right - I guessed "pre-populated" referred to things you already knew about the candidate before they filled out the form.

In that case, it really is just a case of stuffing the form fields into a PDF generator of your choice; no database required. If you want to store it, text files are fine, though databases handle locking and such for you if there might be simultaneous submits.

# ¿ Feb 11, 2014 19:40

Computer viking: May 30, 2011; Now with less breakage.

First, to fix your bug: put a sort in there before the uniq. Uniq is a fairly stupid tool, and only works with sorted input.

code:

genlab4: /tmp > zcat 9606.tsv.gz | tail -n+4 | cut -f6 | uniq | wc -l
75131
genlab4: /tmp > zcat 9606.tsv.gz | tail -n+4 | cut -f6 | sort | uniq | wc -l
5523
genlab4: /tmp > zcat 9606.tsv.gz | tail -n+4 | cut -f6 | sort | uniq -u | wc -l
899

BTW, you are counting "IDs that only occur once", not "number of different IDs". (The -u flag to unique means "drop things that occur more than once"). That's fine if that was what you meant to do.

The entire line from zcat to wc is valid sh (and bash, and probably most shell) code, so you can either run it on the command line, or put it in a .sh file and execute that.

If you put it in a .sh file, you could easily make it take the filename as an argument instead of hard-coding it. Say you have a file test.sh that contains "echo $3 $2 $1", and run it as "./test.sh one two three": It will print "three two one". Given that you should be able to write something that would let you do "./humangenome.sh 9606.tsv.gz" .

Oh, and a personal thing: Using + with tail doesn't work everywhere; it fails on my FreeBSD machines (since -n+ is a GNU extension and not implemented in BSD tail). A more robust solution is to use grep -v '^#' , which drops all lines that start with a # . (For grep, -v means "everything except", and ^ is interpreted as meaning "the beginning of the line" ... so it reads as "everything except lines that start with #"). If you specifically need to delete the first 4 lines, you could also use "sed 1,4d" , which I think should work everywhere.

Computer viking fucked around with this message at 16:20 on Feb 20, 2014

# ¿ Feb 20, 2014 16:05

Computer viking: May 30, 2011; Now with less breakage.

Yeah, if you can point Access at a separate SQL database ,that would be a great start - that way, you could start replacing one part at the time without worrying about keeping data in sync. You wouldn't even have to use the same tech for the entire solution - maybe some things makes more sense as a desktop app, and some as a Web page, talking to the same DB?

I believe there are other tools that can do Access-like things, like LibreOffice Base ... but don't go there. I agree that it seems to be time to just accept that they need something custom. Going back to Django is not a horrible idea, or I guess this is kind of what Ruby on Rails is for? VB.net or C# would not be a bad idea either, I believe the .net stack has good tools for working with databases.

# ¿ Aug 20, 2021 19:43

Computer viking: May 30, 2011; Now with less breakage.

My position started as "take over this database of sample tubes, and expand it to a way to track the boxes we keep them in, so we can notice if we lose any when we move to another building" - also in a medical research group.

That was meant to be a 3-month kind of deal, but they kind of discovered how useful it is to have a general IT person around. They have scraped together the money to keep me for a decade, now - and while that tracking system still lives, I've also drifted into bioinformatics. (Genetics and cancer has proven to be sort of data-heavy.)

Not saying that "hire a fresh CS student and hope for the best" is the right solution for you, but it worked for us.

# ¿ Aug 21, 2021 14:01

Computer viking: May 30, 2011; Now with less breakage.

Yeah I was about to say - it's like the old quote about he who tries to parse HTML with regex living in a state of sin. Using a proper parser is both easier and more robust to format changes.

Edit: uh that sounds "fun", and I'll leave it to someone who has used that specific solution

# ¿ Aug 25, 2021 20:22

Computer viking: May 30, 2011; Now with less breakage.

Google domains seems to be fine enough, though I have a reflex to try and keep my domains independent of the big companies - I use Gandi for my domains, since they are just a registrar. On the other hand, I suspect this is needlessly paranoid.

# ¿ Aug 27, 2021 12:08

Computer viking: May 30, 2011; Now with less breakage.

I use 12, since it allows me to cleanly split it into sub-indents in so many different ways.

# ¿ Oct 13, 2021 20:52

Computer viking: May 30, 2011; Now with less breakage.

I have just hammered together a Java/Swing GUI like it was 2007 - it's for an ImageJ plugin, so Java is the path of least resistance.

The really bad part is that it felt so good. The documentation is decent and complete, the base libraries are big and useful, I ran into zero weird gotchas, the on the fly error checking in Eclipse is perhaps the best of any language I've written, the GUI builder I used was simple and sensible, and everything just ... works. Sure, it's a bit verbose, but given an IDE that's way overblown as a complaint.

It looks incredibly dated, but compared to the labyrinthine experience I had last time I wrote a javascript application this was downright relaxing.

# ¿ Oct 21, 2021 16:04

Computer viking: May 30, 2011; Now with less breakage.

Or to give a much more bare bones and incomplete explanation: It's the process of going from the driver acknowledging that you have a network card, to that network card being configured with an IP address and the system knowing what traffic to use it for.

# ¿ Nov 25, 2021 00:01

Computer viking: May 30, 2011; Now with less breakage.

Ranzear posted:

Yeah, that, sorta. jQuery is overkill here if not plainly bloat, and this code doesn't work without it, and including it might not be obvious to someone asking about this. I meant it as a direct caveat for the posted code. It's trivial without jq, but still different enough to matter.

There is nothing that rustles my jimmies like early programming courses teaching with jQuery.

I mean, the first programming course I did taught us PHP 4. That may still have been the lesser sin, mind you.

(And then we had the "socket programming on linux with C" course shortly after, in parallel with "Flash and ActionScript", IIRC. Fun few years.)

# ¿ Dec 25, 2021 00:45

Computer viking: May 30, 2011; Now with less breakage.

Oh that's kind of an ugly document - the nesting to get in to the value you want is what, fifteen levels deep?

Anyway. I'd suggest using something like Greasemonkey, which is a tool for managing and writing custom javascript that runs on a page. You can then write some JS to check the value, sleep for a while, reload, and check again - and then alert you if it drops below your threshold. I've never used greasemonkey, though - I have no idea what sort of notifications are available, or how it handles page reloads. It's a decade old by now, so presumably there are some tools available?

The actual scripting to pull out and compare the value is easy enough, if ugly. I'm using Firefox, but presumably you can do something similar in Chrome.
First, get the xpath to the value you want to monitor:
- Right click, inspect. This should pop up the inspector panel.
- Press the box+arrow in the top left of the inspector panel, or press ctrl+shift+C. This should put you in item selector mode, where hovering over things will highlight them. Click the price on the day you're interested in.
- This should highlight a line in the page source in the inspector panel - hopefully a line that ends with " 1,310,000</div>" or something like it.
- Right click that line, then copy/XPath in the popup.

This should copy an XPath selector for that element on the page - something like /html/body/div[1]/div/div/div/main/div/div/div[7]/div[2]/div[2]/div[1]/div. This is effectively the directions for how to get to that element, starting from the document root.

With that, the code is effectively "get the element at that path, strip out the commas, and see if that number is small enough".

I'm terrible at javascript, but something like this - though you obviously want to replace the path:

code:

var xp = "/html/body/div[1]/div/div/div/main/div/div/div[7]/div[2]/div[2]/div[1]/div[5]/div/button/div[3]/div[1]"
var res = document.evaluate(xp, document, null, XPathResult.ANY_TYPE, null)
var node = res.iterateNext()

if (node.innerHTML.replaceAll(",", "") < 1300000) {
	alert("Yay")
}

What's left as an exercise is wrapping that up in a sleep/reload/check loop and doing a more noticeable alert than a popup within that tab.

You can test that the above works by pasting it into the Console tab of the inspector. (You'll have to convince it that you're not just blindly pasting code from random people on the internet first - but it should be self-explanatory.)

Computer viking fucked around with this message at 05:16 on Dec 28, 2021

# ¿ Dec 28, 2021 05:14

Computer viking: May 30, 2011; Now with less breakage.

I also quickly checked with my BF, who is the web person here.

His suggestions:
- Use violentMonkey, it's the still-opensource replacement for greasemonkey
- Each day is a Button element. Those button elements contain both "which date is this about" (in a field called "data-testid") and a screen reader hint that contains the points value (aria-label)
- This means you can use a querySelector to ask for "a button where the data-testid is a given date".

Putting it all together, this mess pulls out the points value for the 6th:

code:

var price = document.querySelector('button[data-testid="arrival-2022-12-06"]').
    getAttribute("aria-label").match("[0-9,]{3,15}").toString().replaceAll(",", "")

The .match part is a regexp that looks for "a string made up of only digits or commas, 3-15 characters long" - the full aria-label is something like "December 6 through December 7, 1,310,000 Points per night for 1 night. Premium Room Rewards. Choose room".

He also insisted that you use a long refresh timer, like 30 minutes, just to be polite. And that you put in a sanity check of the type "if it's 2022, stop" - so it doesn't accidentally keep running forever.

Computer viking fucked around with this message at 05:46 on Dec 28, 2021

# ¿ Dec 28, 2021 05:43

Computer viking: May 30, 2011; Now with less breakage.

Gin_Rummy posted:

Can anyone recommend a really good primer/tutorial on creating a SQL database and linking it to a webapp?

I've been working on web-based a board game using JS/React, and I think I have my core game and its mechanics all programmatically established, but I want to store a huge array of trivia questions/answers in a database that can be called from my javascript functions.

I'd use python and Flask, which is a fairly simple framework for handling HTTP requests. Basically, you set up a python function to handle requests to a specific URL, use python code to talk to the database, and return JSON. However, it's hard to find a clean tutorial for that - they are all "how to orchestrate a docker image in the cloud while using this ORM layer to avoid writing SQL".

If you do want to go that way, you can piece it together yourself by looking at the python+database code separate from the Flask part: It's not hard to query a database from python.
There's also a json package that makes it trivial to create json text from a python object, so the complete flow would be something like this:

- receive a GET request on /question/
- Use SQL to get one or more rows from the database
- Put the results in a suitable python object - probably a list where each entry is a dictionary that represents a question
- Convert that to JSON text
- return a HTTP response with the JSON as the body

I'd use PostgreSQL as the database and the psycopg package in python to talk to it, but that is probably wildly overkill in this case - you may be better off with sqlite.
It's also very common to use SQLAlchemy - it's a python package that abstracts away which specific database you're using, which is useful if you ever want to migrate. It also does ORM - basically, you create python classes that represent the database tables, and it generates the SQL queries for you in the background. It seems nice, but it is one more thing to learn. Then again, so is SQL.

e: Django fills some of the same niche as Flask here, but it does a lot more for you, for good or bad. Which could be a good thing.

Computer viking fucked around with this message at 00:26 on Dec 30, 2021

# ¿ Dec 30, 2021 00:21

Computer viking: May 30, 2011; Now with less breakage.

Try the quickstart, tutorial, and documentation from the flask project itself, it's not bad.

# ¿ Dec 30, 2021 01:08

Computer viking: May 30, 2011; Now with less breakage.

Yeah, sqlite is great for this sort of thing. Postgres is a lot more work to set up, but also has some clear advantages - none of which matter at all for what you're doing.

# ¿ Dec 30, 2021 02:08

Computer viking: May 30, 2011; Now with less breakage.

Flask is great if you just want to write a layer that makes a database available over http - django feels like overkill if you're not doing much beyond that. Besides, I don't like ORMs, but that's more of a personal falling.

# ¿ Dec 31, 2021 20:23

Computer viking: May 30, 2011; Now with less breakage.

I've just spent some time doing random bugfixes in a Qt application I once wrote for internal use at work - it prints labels to a network connected label printer.

One of the things I spent some time on was moving it from Qt 4.something to 6.2, to try to keep up with the supported versions.

We still have people using Windows 7, because it's the only thing officially supported on the relevant internal network.

Qt6 has removed support for windows 7. Sigh.

Computer viking fucked around with this message at 14:45 on Jan 7, 2022

# ¿ Jan 7, 2022 14:42

Computer viking: May 30, 2011; Now with less breakage.

Though that said, being able to flag it when cars do break out of their path, intersect, and/or release fragments could perhaps be useful on its own.

# ¿ Jan 8, 2022 16:31

Computer viking: May 30, 2011; Now with less breakage.

Volmarias posted:

Relevant:

https://www.youtube.com/watch?v=k7YVxLLIuGM

# ¿ Jan 9, 2022 07:14

Adbot: ADBOT LOVES YOU

# ¿ May 3, 2024 21:47

Computer viking: May 30, 2011; Now with less breakage.

I kind of hate how something that incredibly obviously involves at least two different kinds of "server" is described as "serverless", even if I understand the point.

# ¿ Mar 12, 2022 02:30

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›3 »