> ## Content Index
> Fetch the complete content index at: https://davidzych.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# The investigation of a site crash
- URL: https://davidzych.com/the-investigation-of-a-site-crash/
- Published: 2014-08-19T06:33:21.000Z
- Updated: 2026-11-19T17:28:32.000Z
- Author: David Zych
- Tags: crash, mysql, out of memory, ubuntu, wordpress, #Import 2026-11-20 21:02

When I woke up this morning I noticed by site had crashed with the following error:

> Error establishing a database connection.

Awesome. What a helpful error message. WordPress is basically saying:

![shrug](https://davidzych.com/wp-content/uploads/2014/08/shrug.jpg)

Thanks WordPress. I ssh’d into the server to see if I could figure out what was wrong *myself*. Since this was complaining about a database connection, my first step was to check the status of mysql:

```
 $ sudo netstat -tap | grep mysql

```

Nothing. Ruh Roh! Time to attempt a restart…

```
$ sudo /etc/init.d/mysql restart

```

… and the site came back up. Phew! That was easy.

But why did it crash in the first place?

Since MySql was the cause it made sense to me to first check the sql logs. I opened those up and found nothing. Based on the recommendation of Google I then searched the syslogs, specifically for `memory`:

```
$ sudo grep memory /var/logs/syslog
  Aug 19 10:56:12 localhost kernel: [10664646.817182]  [<ffffffff811429d4>] out_of_memory+0x414/0x450
  Aug 19 10:56:12 localhost kernel: [10664646.819979] Out of memory: Kill process 4803 (mysqld) score 104 or sacrifice child
  Aug 19 10:56:12 localhost kernel: [10664646.831686]  [<ffffffff811429d4>] out_of_memory+0x414/0x450
  Aug 19 10:56:12 localhost kernel: [10664646.833365] Out of memory: Kill process 4826 (mysqld) score 104 or sacrifice child</ffffffff811429d4></ffffffff811429d4>

```

Aha! It looks like MySql ran out of memory and the server killed it. Okay, now on to the next question… *Why?*

Well… it ran out of memory, that’s why. (Duh.) One way to alleviate this is to create a swap file, which it turns out I forgot to do when I originally configured this server. Without that swapfile MySql had nowhere to overflow excess data to and subsequently crashed. I created and enabled a swap file:

```
$ sudo dd if=/dev/zero of=/swapfile bs=1024 count=1024k
$ sudo mkswap /swapfile
    Setting up swapspace version 1, size = 262140 KiB
    no label, UUID=XXXXX
$ sudo swapon /swapfile

```

After creating the file everything has been running great (so far).

### Next Steps

Had I not been a narcissist and checked my own website I probably wouldn’t have noticed it was down for hours, perhaps encroaching on days. My next steps are to look into monitoring software – something that alerts me when there’s a problem, or even a potential problem before it’s even there. One I have found that does just that is [Nagois](http://www.nagios.org/?ref=davidzych.com), or it’s stepchild [Icinga](https://www.icinga.org/?ref=davidzych.com).