Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
beepsandboops
Jan 28, 2014
Problem description: I'm trying to help out a friend with a website, which wasn't set up by either me nor them. It's a WordPress site running on Ubuntu 14.04, and it regularly just shits the bed and throws up a "Error establishing a database connection" message on the site.

Attempted fixes: Rebooting doesn't help

Checked system resources with top, nothing seems to be running out of control

The wp-admin page is showing the same error message, so I don't think that that DB is corrupted or anything.

All of the MySQL logs are empty, and the Apache logs seem okay.

I checked wp-config.php and it seems like it has all of the right info to connect to the DB. Is there any way to verify what credentials I need to be using there?

Recent changes: Nothing that I know of

--

Operating system: Ubuntu 14.04

System specs: VPS hosted by DigitalOcean, I believe 1 GB RAM

Location: USA

I have Googled and read the FAQ: Yes

Adbot
ADBOT LOVES YOU

telcoM
Mar 21, 2009
Fallen Rib

beepsandboops posted:

Problem description: I'm trying to help out a friend with a website, which wasn't set up by either me nor them. It's a WordPress site running on Ubuntu 14.04, and it regularly just shits the bed and throws up a "Error establishing a database connection" message on the site.

Attempted fixes: Rebooting doesn't help
[...]
I checked wp-config.php and it seems like it has all of the right info to connect to the DB. Is there any way to verify what credentials I need to be using there?

Let's start with some basic checks, then.

First check: Is the MySQL database running at all?
If the MySQL database is supposed to be running on the same host (i.e. the wp-config.php file includes the setting "define('DB_HOST', 'localhost');"), and you have command line access to the host, run this command:
code:
ps -ef | grep [s]ql
It should show you one or more mysqld processes, possibly with options like --defaults-file=<some pathname> and/or --socket=<some pathname>.

(Having the first letter of the grep search string in [brackets] is just a trick that prevents the grep from picking the grep process itself when it gets the ps command output. )

The defaults file is the main configuration of the database; you might want to take a look at it later.
The socket pathname might be needed if it turns out the database password is lost and needs to be recovered.
Also note the username the mysqld process is running as.

If there are no mysqld processes listed, the MySQL database might not be configured to start at boot: in that case, something like
code:
sudo update-rc.d mysql defaults

might fix it for future reboots.


Second check: can you login to the database using the "mysql" command line tool?
code:
mysql -p -u <DB_USER> -h <DB_HOST> <DB_NAME>
Replace <DB_USER>, <DB_HOST> and <DB_NAME> with the respective values from wp-config.php, and enter the database password when prompted.

---

If it works, you should see MySQL copuright message and a MySQL command prompt. You can exit by typing "exit" or "quit", or just by pressing Control-D.
But you might want to run a few checks first while you're in there:

Are the WordPress tables still present?
Type:
code:
show tables;
If there is no custom $table_prefix set in wp-config.php, this should list a number of table names, all starting with "wp_".

How many user accounts are defined in the database engine?
While in the MySQL command prompt, type this command and press Enter:
code:
select user from mysql.user group by user;
In theory, there should only be root and whatever the WordPress DB_USER is set to.
If DB_USER is set to "root", then WordPress has full unrestricted access to the database engine, which is probably not a good idea.

---
If you cannot connect to the database with the mysql command, you'll have to login to MySQL as the MySQL root user.

Try with:
code:
sudo -u <the username mysqld was running as> mysql -u root -S <socket pathname>
If it says "access denied", and you know the MySQL root password (which can be different from the root password of the host operating system), add the -p option and type the MySQL root password when prompted.

If that does not work, check this out:
https://help.ubuntu.com/community/MysqlPasswordReset

After you've confirmed that the MySQL engine is running, you can try this:
http://www.wpbeginner.com/wp-tutorials/how-to-fix-the-error-establishing-a-database-connection-in-wordpress/

But if it looks like the site might have been hacked, the WordPress auto-repair might cause you to lose evidence of the hack.

beepsandboops posted:

All of the MySQL logs are empty, and the Apache logs seem okay.


The MySQL logs might be empty because the database has been non-functional for a while and log rotation has cleared away any old logs from before that time... or it maybe because the host has been hacked and the intruder has wiped the logs in order to hide his/her tracks. If the system has been rebooted, any deleted-but-still-open log files that might have been easily recoverable are now gone.

beepsandboops
Jan 28, 2014
Thanks for the exhaustive reply! Feeling a bit stupid, b/c it just looks like mysql wasn't running. I started it up and ran the command you recommended to have it automatically start up again.

I'll keep an eye on it but hopefully that was it! Thanks again

beepsandboops
Jan 28, 2014
OK, site worked for maybe a day before crashing again. If I'm reading everything right, it looks like the box is running out of memory and making GBS threads the bed.

I checked /var/log/mysql/error.log and this is what I see:

code:
160710  8:42:25 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
160710  8:42:25 [Note] Plugin 'FEDERATED' is disabled.
160710  8:42:25 InnoDB: The InnoDB memory heap is disabled
160710  8:42:25 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160710  8:42:25 InnoDB: Compressed tables use zlib 1.2.8
160710  8:42:25 InnoDB: Using Linux native AIO
160710  8:42:26 InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(137363456 bytes) failed; errno 12
160710  8:42:26 InnoDB: Completed initialization of buffer pool
160710  8:42:26 InnoDB: Fatal error: cannot allocate memory for the buffer pool
160710  8:42:26 [ERROR] Plugin 'InnoDB' init function returned error.
160710  8:42:26 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
160710  8:42:26 [ERROR] Unknown/unsupported storage engine: InnoDB
160710  8:42:26 [ERROR] Aborting

160710  8:42:26 [Note] /usr/sbin/mysqld: Shutdown complete
When I check htop, this is what I see (sorting by memory)



The machine only has 1GB of memory, but for a small website that doesn't get much traffic I wouldn't think that would matter too much

Is there some obvious Apache setting that I'm missing that would stop it from launching a ton of processes?

telcoM
Mar 21, 2009
Fallen Rib

beepsandboops posted:

OK, site worked for maybe a day before crashing again. If I'm reading everything right, it looks like the box is running out of memory and making GBS threads the bed.

You're reading it exactly right.

Actually I'm pretty sure that your site is getting targeted by various WordPress hackers, pingback spammers and maybe even automated worms.
Each request gets processed by Apache, then by PHP (running WordPress itself), then by the MySQL database, and then the reply goes back through the same chain. Depending on the exact nature of the request, handling it may even require several MySQL queries.

Even a single pingback spammer with a good network connection can send pingback requests faster than a small virtual machine can handle.
(Take a guess on why I know this... :flame:)

In the Linux questions thread, there was very recently a bit of discussion about securing a WordPress site and protecting it from spamming and hacking, starting here:
https://forums.somethingawful.com/showthread.php?threadid=2389159&pagenumber=611#post461799889

In a nutshell: I don't actually know how well SELinux is implemented in Ubuntu, but use it if you can. And if you don't need WordPress's XML-RPC API, block any requests for it at the webserver level. For Apache, that means something like
code:
# Block WordPress xmlrpc.php requests
<Files xmlrpc.php>
  order deny,allow
  deny from all
</Files>
With this, any pingback spam requests will be rejected by Apache, which is much simpler and way more efficient than letting them be processed by PHP and WordPress itself.

beepsandboops posted:

Is there some obvious Apache setting that I'm missing that would stop it from launching a ton of processes?

There are several, but the appropriate setting actually somewhat depends on which Apache Multi-Processing Module you're using.

MaxRequestWorkers (or MaxClients in older versions of Apache) is probably the setting you'll want to tweak.

  • Locked thread