<?PHP questions that don\\'t need their own thread ?>

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > <?PHP questions that don\\'t need their own thread ?>

Ranzear: Jul 25, 2013

I strongly recommend dumping Apache entirely and forever for nginx+PHP-FPM instead.

Yes, Apache and mod_php can allegedly be a little faster... ^{_{in single core environments ... with limited memory ... and avoiding simply changing default file handler limits,}} but it takes days of tuning to beat out an out-of-the-box -FPM stack. Then you have very nice nginx static serving too and a single point of configuration instead of htaccess vomit everywhere.

I might be venting. Apache likes to completely poo poo the bed on the really braindead API stuff we do and I greatly enjoy being able to say 'no Apache' on all our future projects.

Ranzear fucked around with this message at 04:23 on Mar 12, 2018

# ¿ Mar 12, 2018 04:09

Adbot: ADBOT LOVES YOU

# ¿ May 15, 2024 16:07

Ranzear: Jul 25, 2013

Hoof, that's an old one and smells of grognard. It sets cgi.fix_pathinfo=0 though, so it's on the right track.

If you don't know you need PHP 5.x, move up to PHP 7.2, and you should probably be on Xenial by now.

(This one is a bit better, but trash their mysql for MariaDB instead if you wanna be a cool FOSS kid, and PHP 7.2 instead of 7.0.)

That nginx config is really slapdash and rudimentary. Work from the stock configuration instead. Nginx configuration looks complex at first, but it's 'set it and forget it' most of the time. Dump the ipv6 lines. Dump the extra location blocks until you know what you need them for. If you want https, certbot can alter your configuration automatically, but a lot of tutorials are overwrought; you just install and do 'certbot' and pick nginx (or run certbot --nginx). Use your real domain and/or IP in server_name, not the wildcard underscore, so you dump all other requests. snippets/fastcgi-php.conf is likely very wrong - you'll have a fastcgi-params include in the stock config already.

I don't know why it has you change from a unix socket to localhost:9000, don't do that part.

Don't install APC, use the native -opcache package.

The mysql support section is kinda wrong. You don't need to install all of those individually. Just do php7-mysqlnd I think, which is the native driver. Install -pdo if you need it (you should need it).

So your install is more like:

code:

sudo apt-get remove apache2
sudo apt-get install nginx php7.2-cli php7.2-fpm php7.2-pdo php7.2-mysqlnd php7.2-opcache

and for MariaDB, go here, then run mysql-secure-installation.

And for gently caress's sake don't leave a php info page hosted.

Ranzear fucked around with this message at 19:00 on Mar 12, 2018

# ¿ Mar 12, 2018 17:48

Ranzear: Jul 25, 2013

Remi is good. I go to Webtatic for some reason, I think they're faster with or just have better coverage of some weird modules and gives me a newer (and not Fedora branded?) nginx too on Centos7.

# ¿ Mar 13, 2018 22:01

Ranzear: Jul 25, 2013

Acidian posted:

cgi.fix_pathinfo=0 [...]

code:

location ~ \.php$ {
            fastcgi_pass   127.0.0.1:9123;
            fastcgi_index  index.php;
            fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
            include        fastcgi_params;
        }

What it does, ostensibly, is prevent stuff like traversal attacks (~domain.tld/../../../foo.php) and passing weird stuff like an uploaded image bar.png with php headers then accessed with ~domain.tld/bar.png/nonexistent.php getting passed to fastcgi, because that configuration would otherwise take anything and everything that ends in .php and pass it to fastcgi but then it will 'fix' the script location to where it found the first extension, leading to php running the bar.png as a php script. It just makes to so your .php files have to be found directly and accurately, like you would expect them to be.

CGI is 'common gateway interface' and in quick and dirty terms it's just a standard pipe to and from something running a script at the location passed to it. It's 'run this script/process and give me a file handler to its input and output'. FPM is a crazy beast that can basically run thousands of workers preloaded with the scripts all the time so you just hit a preexisting process and your response times are stupid low, meanwhile most scripts are in shared memory so you end up using only about 10MB of memory per worker for even the largest codebase, so not only do you get a connection handled per worker but they rip through requests even faster so your workers stay even more available.

Ranzear fucked around with this message at 22:00 on Mar 14, 2018

# ¿ Mar 14, 2018 21:49

Ranzear: Jul 25, 2013

code:

location / {
	try_files $uri $uri/ /index.php?/$request_uri;
	location ~ /\.php$ {
		include         /etc/nginx/fastcgi.conf;
		fastcgi_split_path_info ^(.+\.php)(/.+)$;
		fastcgi_param	SCRIPT_FILENAME	/var/www/scripts$fastcgi_script_name;
		fastcgi_pass    unix:/var/run/php-fpm.sock;
	}
}

This is sufficient to pass path and lets your PHP actually be wherever you want instead of the root you specify before the location block (say, /var/www/public vs /var/www/scripts), meaning your scripts can't be static served by accident. It works for CodeIgniter at least though I've made a few speculative changes (the inner location block was just '= /index.php', not sure if regex match will work there). It even supports a bare path on your domain going to index.php (CI's style and others).

Nested location blocks are the black magic of nginx.

Ranzear fucked around with this message at 22:19 on Mar 14, 2018

# ¿ Mar 14, 2018 22:15

Ranzear: Jul 25, 2013

It's the little stuff like Ctrl+D or shift-RMB selecting that makes Sublime absolutely top tier as a pure editor. It was one of the easiest 'shut up and take my money' decisions for me.

It did take until Sublime 3 for a good php linter to just work out of the box. Getting the package manager into Sublime is still a copy-paste annoyance(?), but then the availability of good stuff just skyrockets.

I know there are packages for class hierarchy and whatnot too.

# ¿ Mar 21, 2018 19:30

Ranzear: Jul 25, 2013

Should maybe establish that that's for a development environment too, lest I see it crop up as the backend for a mobile game API.

Because that'd be par for the course.

# ¿ Mar 26, 2018 18:01

Ranzear: Jul 25, 2013

Note first that all of this is when writes are involved. If you're just doing simple reads and nothing is logically dependent on them you can just read willy-nilly.

So first some boilerplate:
Every time you have a php script open a database handler, like $dbh = new PDO($dsn, $user, $password);, it opens its own connection to the db. Every one of those is separate and parallel. Think of each as a session as well. You should not be opening a new db handler for every statement, nor should you open a new one for every include either. One script invokation (be hit a web or cli) should have one database handler. With all that correct, you won't have race conditions within a single script. The lone db handler will do everything in order.

Now, within parallel scripts, the database's state changes fairly instantly on any given write. One script can sneak a read inbetween two writes from a different script, or sneak a write in that would change the logic of another script depending on when it read that entry. What this means though is that you can't have a script read out a bunch of data at the start and expect it to stay the same for the entire duration of the script. It can still be written to by other scripts. Similarly, subsequent writes can have invalid states in between that can be read by parallel scripts.

To prevent this, first learn what A.C.I.D. means. Atomicity is the main focus, but the whole script doesn't need to be atomic, which seems to be your other concern (mostly about blocking).

So there are two main things to look into:

First is when you need to write more than one thing in separate statements or otherwise need an 'all or nothing' bunch of things to happen to the db, so that you can't for instance deduct currency but not give the purchased item. These are transactions. Doing statements outside of transactions means they 'auto commit', meaning execute immediately, but opening a transaction saves committing until you say when, and everything happens then and all at once. The db does not show any changes at all until you commit, or you can throw away the changes mid process with rollback. Also, you must commit changes; your script ending automatically calls rollback too.

Side note: If you wrap stuff in a try block, your catch can check if you're in a transaction and call a rollback, meaning any thrown exception will toss out any changes to the db. Breaking out and never hitting a commit achieves this too, but it's good to be sure.

Second comes up if you have logic dependent on a read, like if a value in the database being 1 means set it to 0 or being 0 means set it to 1. If you read 1, something else may have set it to 0 before you manage to write 0. You need to invoke row locking, meaning nothing else can interact with that row until you commit, roll back, or your script ends. You do this with SELECT ... FOR UPDATE but only when you have a transaction active (ending the transaction unlocks the row, I think autocommit is triggered on reads, but locking is only useful in transactions anyway). This signals to MariaDB/MySQL that 'nobody can gently caress with this row until I'm done with it'. You can now read the 1 and be sure it's still 1 when you write 0, and anyone else trying to read from that row will wait until you're done and always read your 0. You can and should do even just a dummy SELECT FOR UPDATE on any row you might write to during your transaction so that everything is locked while your script figures out what it wants to write.

You want to keep your transactions tightly wrapped to what needs them, because SELECT FOR UPDATE will cause other scripts to block and hang if they try to read or write that row, but in a merciful world you should be using InnoDB meaning it's a row lock and not a whole table lock. Basically, start your transaction right before you do any important reads, and end it after you've done all important writes. Don't wrap a whole slow-rear end script in a single transaction unless the reads at the start are truly logically bound to the writes at the end.

Ranzear fucked around with this message at 22:04 on Apr 21, 2018

# ¿ Apr 21, 2018 21:55

Ranzear: Jul 25, 2013

Cool Matty posted:

As an aside, just to be sure, I have tested this page using PHP's built in server, and there was no issues, so I don't think it's a PHP configuration issue either, only something specific with PHP-FPM. I am running this on WSL Ubuntu 18.04.

Anyone got a clue of what might be wrong, or even where to start looking?

What's your fpm set to listen on? If it's a unix socket it could be a variety of file/permission issues. If it's just a tcp listener, try a unix socket instead. Be sure that nginx and php-fpm are running as the same user to simplify it further.

# ¿ Jun 6, 2018 19:27

Ranzear: Jul 25, 2013

It's probably a sorted tree, so I'd expect you to want an in-order traversal.

But you really need to wrap your head around the recursive approach. It's basically "Call the function that was called on me on my children first, then return my result with theirs."

# ¿ Jul 18, 2018 06:48

Ranzear: Jul 25, 2013

Run getenforce to see if SELinux is active.

If it's running, that's why. You aren't military contract or medical records, you don't need it. It's an obtuse and obfuscate pile of poo poo and we totally had this exact issue yesterday because Linode's Centos image now has it on by default.

# ¿ Sep 11, 2018 18:21

Ranzear: Jul 25, 2013

I'm bored on a Saturday too, let's do some casual golfing:

code:

<?php
$sounds = array("p", "t", "q", "'", "b", "D", "tlh", "ch", "Q", "j", "S", "H", "v", "gh", "m", "n", "ng", "w", "r", "l", "y", "a", "e", "I", "o", "u", " " , "p", "t", "q", "'", "b", "D", "tlh", "ch", "Q", "j", "S", "H", "v", "gh", "m", "n", "ng", "w", "r", "l", "y", "w'", "y'", "rgh");
for ($i=0; $i < 21*5*25; $i++) {
	echo ($syl = $sounds[$i%21] . $sounds[21+floor($i/21)%5] . $sounds[21+5+floor($i/(21*5))%25] . "<br>") && (strpos($syl, "ow") || strpos($syl, "uw")) ? "" : $syl;
}

2541 results, right?

Of course you could inline the array three times to make it just one line, and of course you could chop variable names down, but I think this is as logically compressed as it can get. I would bet there's a way to arrange ow and uw combinations to fall in a pattern in the encoding that you could exclude them mathematically, but I don't think that'd be much simpler in the end (it would certainly be faster than string comparison if you needed it stupid fast though).

Fun fact: You can modulo encode any number of dimensions of integer coordinates as long as they have fixed bounds per dimension, up to whatever size of integer you can store. You can then perform operations on the integers themselves rather than needing to store coordinates. This is great for stuff like n-dimensional chessboards.

Ranzear fucked around with this message at 22:20 on Jan 27, 2019

# ¿ Jan 27, 2019 22:12

Ranzear: Jul 25, 2013

Kraus posted:

I came to programming from linguistics instead of math, so mathematical solutions don't immediately jump out at me.

Just beware of tailored solutions like this. Most would have done a fixed shift for each coordinate to encode into one integer (each dimension gets n bits), but mine are arbitrary to make a continuous set that can be iterated over.

Also keep in mind that code golf code is never good code. You should keep the arrays separate for clarity of purpose, and then generalize the implementation using the length of each so you don't have to muck about with literals in the code just to alter any of the arrays:

code:

<?php
$onsets = array("p", "t", "q", "'", "b", "D", "tlh", "ch", "Q", "j", "S", "H", "v", "gh", "m", "n", "ng", "w", "r", "l", "y");
$nuclei = array("a", "e", "I", "o", "u");
$codas = array(" " , "p", "t", "q", "'", "b", "D", "tlh", "ch", "Q", "j", "S", "H", "v", "gh", "m", "n", "ng", "w", "r", "l", "y", "w'", "y'", "rgh");

$oc = count($onsets);
$nc = count($nuclei);
$cc = count($codas);

for ($i=0; $i < $oc*$nc*$cc; $i++) {
	$syl = $onsets[$i%$oc];
	$syl .= $nuclei[floor($i/$oc)%$nc];
	$syl .= $codas[floor($i/($oc*$nc))%$cc]."<br>";
	echo (strpos($syl, "ow") || strpos($syl, "uw")) ? "" : $syl;
}

I was also concerned that strpos( ) could return 0, which would be falsey and fail to throw something out if it started with 'uw' or 'ow', but neither of those appear in your onsets. If they did, you'd have to more strictly check.

Ranzear fucked around with this message at 01:48 on Jan 28, 2019

# ¿ Jan 28, 2019 01:30

Ranzear: Jul 25, 2013

Right, but a little bit of golfing can help. This reduced three ugly nested loops to just one, reduced literals in the block, split concatenation over more than one line. and avoided unnecessary assignments. I would call the functional parts of that final version good enough for the kind of work I do.

Anyway, because I hate string comparison, here's the changes for a true turbo(nerd) version:

code:

$codas = array(" ", "p", "t", "q", "'", "b", "D", "tlh", "ch", "Q", "j", "S", "H", "v", "gh", "m", "n", "ng", "r", "l", "y", "y'", "rgh", "w", "w'");

code:

	echo (floor($i/$oc)%$nc >= 3 && floor($i/($oc*$nc))%$cc >= 23) ? "" : $syl;

Just do the same divide-and-modulo to get the index and check the combo isn't restricted. Easiest to just move the restricted particles to the end so you only have to do one >= compare.

I've put literals back in the block again but �\_(ツ)_/� you already had two there for strpos anyway.

Ranzear fucked around with this message at 02:14 on Jan 28, 2019

# ¿ Jan 28, 2019 02:09

Ranzear: Jul 25, 2013

duz posted:

Depending on the network setup, the private source IP might be in X-Forwarded-For.

Ideally, but requires extra config and can get super messy in my own setups like nginx serving behind haproxy splitting off my websockets and so has to get passed along multiple times or even per config block.

I skimmed a bit earlier and didn't really see a situation where one would have to go this far but now that I see one: $ip = file_get_contents("https://ipecho.net/plain");

Not ideal to poke something external or a bazillion times a second, but that'll do for little stuff?

# ¿ May 20, 2022 21:55

Ranzear: Jul 25, 2013

MariaDB dynamic columns are cool for that too, and it's weirdly likely that you're running MariaDB and don't even know it. They have some special crud to run like a key-value store in a single column and can be easily spat out as JSON. It really is just JSON in a blob with native crud though and separate associative tables tend to be way better in all cases.

A follow up: Not to be confused with JSON columns which are just a syntax check on insert/update.

Ranzear fucked around with this message at 00:29 on Jul 7, 2022

# ¿ Jul 6, 2022 23:44

Ranzear: Jul 25, 2013

Agrikk posted:

am using PDO for my connection type

Might be a 'duh' question but you're using statement prep, not just query(), right?

Agrikk posted:

populate the rest of the $query string.

I'm going to guess not. This is what statement prep is for, and PDO will handle all of the escaping as long as you have the right driver by creating your handle like $dbh = new PDO('pgsql:host=...;dbname=...', $user, $pass);

With statement prep, you can just make a query with question mark params and feed it a sparse array with column keys and the value is just the value, all formatting and escaping and a bunch of security stuff not applicable here is totally automatic. It shouldn't require any imploding of keys to columns or anything, there's a way to make it work from the raw array but it's been a while for me to remember exactly.

PDO isn't the connection type, it's the entire abstraction layer for several database formats. You may also want to PDO::getAvailableDrivers() just to double check you have postgres available.

Proper edit: With sparse arrays you're gonna need the columns. You'll have to implode the keys into a list of columns to match the array being inserted after all, I think I just had a slick way of doing it at one point. Something like implode(array_keys($rowmap), ", ")) directly in the statement but then use parameter binding for the values. if the columns don't match the array's keys it'll complain.

You could just switch the PDO driver to pgsql and use a PDO equivalent of that escaping function you found, but this is the more proper and simpler way unless I've missed something.

Ranzear fucked around with this message at 00:31 on Aug 2, 2022

# ¿ Aug 1, 2022 23:53

Ranzear: Jul 25, 2013

Nope. You're doing just a raw text query and all the hard parts manually. Look at statement prep. The only string handling should be that matching column set thing I mentioned.

# ¿ Aug 2, 2022 18:00

Ranzear: Jul 25, 2013

Looks like you fed it an empty string for a double precision field, not a null. Assuming you've no use for empty strings in any field, just cull any key:value with an empty string value before preparing the statement. There's no reason to mess with your column types, just understand that an empty string is not equivalent to NULL.

# ¿ Aug 3, 2022 00:59

Ranzear: Jul 25, 2013

Everything else is deprecate since like PHP 5.x but I've still had to carry the torch at times, just the same as having to continue the beatings until the InnoDB usage is complete. It's gotten better over time as sites have just died off, but there's still a ton of outdated garbage tutorials out there and far too much trust in them to start.

I'm from MariaDB land though, so you had me double checking that numeric types were still nullable in postgres, lol. Feeding it empty strings was a guess really.

Ranzear fucked around with this message at 00:54 on Aug 4, 2022

# ¿ Aug 4, 2022 00:52

Ranzear: Jul 25, 2013

Use PDO. Period.

Manually escaping strings is caveman poo poo.

# ¿ Feb 13, 2023 00:40

Ranzear: Jul 25, 2013

If one gets their poo poo just right, PDO doesn't even need the field/column assignment boilerplate. Work within the data structure it gives you (from a SELECT ... FOR UPDATE in a transaction you heathen) and all crud involves just passing it back in.

Hence data objects. Something I wish I'd figured out sooner.

# ¿ Feb 14, 2023 06:14

Ranzear: Jul 25, 2013

Agrikk posted:

3. Loop over all connections, starting with connection 1A through 10000U (or wherever) and create a random connection to a different connection point on a different node.

[...]

But I�m not sure of the logic to assign connection IDs to a pair of connection points. Doing it randomly has the problem of taking a long time to generate the final links as the odds of finding a free connection slot on a mode decrease as the arrays gets filled.

If you have 10k nodes numbered 0-9999, just pick a number. Additionally, why still be checking against filled nodes at all?

My method:

Put references to all nodes in an array, basically a temporary index mapped to their real index. This is the arena. Also create an empty array of 'purged' nodes.

Iterate the arena. For each reference:

If the current arena index is in the purge array, skip it (continue is your friend).
Pick a number between 0 and the arena length for a target to connect.
Check the target's arena index isn't in the purge array, else skip (rerolling might be slower, but also works).
Check references aren't already connected (or isn't self-connection), else make the two-way connection.
If current node has reached 20 connections or whatever other heuristic it has, push its arena index on the purge array.
If target node has reached its heuristic, push its arena index on the purge array.

Once iterated, remove all the purged nodes from the arena, empty the purge array, and run again (PHP's unset() can just take the purge array directly iirc, could be some black magic needed.)
When the arena is empty (or manages to run without making a new connection, could be a weird case,) you're done. There's still some 'dead hits' in each loop but they all get consumed eventually.

It shouldn't be necessary to shuffle the arena at each step, but low indexes might get a little bias if you reroll in the iteration instead of skipping so throw it in if you do. Searching the purge list should be fast enough to not worry about because it isn't carried across iterations. This doesn't guarantee full connectivity, but I think that's a way harder problem.

Changing the heuristic is now easier too; it can just be a function call defined by whatever type of node itself, and you can even mix node heuristics in the same connection map. And for instance: If you want a variable limit on number of connections for a node, store the max connections in the node and the heuristic checks against that.

Ranzear fucked around with this message at 23:55 on Feb 26, 2023

# ¿ Feb 26, 2023 23:26

Ranzear: Jul 25, 2013

Jabor posted:

- For each node, add a link to a random node lower than it. This guarantees the graph is connected.

I like this a lot. I retract my statement about it being hard, though I think this generates fewer loops and looks more linear or tree like in the end as it consumes the connection limit of lower indexes early. Maybe linearity is necessary to be complete.

I also like the final shuffle, but I was under the impression the nodes had to hold their own connections in the end for things to be traversable instead of a list of edges. I suppose this matches my arena shuffling and just plug the edges in at the end.

Ranzear fucked around with this message at 00:14 on Feb 27, 2023

# ¿ Feb 27, 2023 00:06

Adbot: ADBOT LOVES YOU

# ¿ May 15, 2024 16:07

Ranzear: Jul 25, 2013

You'll spend more time futzing with it and then vetting what it spits out than it would take to write it yourself, cleaner to read, faster, and more expansible, in the first place.

My post above can be extended, relying on having both temporary and real indexes available, to make a fully connected graph:

Create a second empty array, call this the core.

Add these steps to iterating:

At start:

If core is empty, add this node's real index to core (not arena index)
After connecting nodes but before heuristic:
If this node's real index is in core , add real indexes of all connected nodes to core
Integrate with heuristic functions:
If node not yet in core list, don't pass heuristic.

In this form it'll add additional connections until the graph is complete, or the heuristic can be checked early and the connection rerolled until it finds a core node if there's only one slot left. It's just important that connected nodes are added to core before the parent node is removed from arena. This is also now a case where a pass can go without making a new connection, all core nodes already being saturated, and I'd just start it over. In most any other language core should be a tree for decent speed.

Ranzear fucked around with this message at 00:55 on Feb 28, 2023

# ¿ Feb 28, 2023 00:31

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > <?PHP questions that don\\'t need their own thread ?>