Nazly [Web Log] » MySQL

Optimizing the MySQL tables of a WordPress Website

Nazly — Fri, 04 Feb 2011 09:14:31 +0000

If you are running a WordPress Website or a Blog where the content is updated and/or deleted frequently, you will need to optimize the MySQL tables more often than not so that you don’t run into database issues and down times. This also helps to maintain the average response time of MySQL queries.

When executed, the following PHP script will optimize the tables which requires optimization in your WordPress database. Place this file in your WordPress’s root directory.

You can run this manually or the ideal scenario would be to setup a cron job to execute it in a given time interval based on your requirement..

            
include("wp-config.php");
mysql_connect(DB_HOST, DB_USER, DB_PASSWORD);
mysql_select_db(DB_NAME);
$selQuery = "SHOW TABLE STATUS FROM `".DB_NAME."`";
$resSel = mysql_query($selQuery) or die(mysql_error());
while($row = mysql_fetch_assoc($resSel)){
    if($row["Data_free"] > 0){    
        echo date("Y-m-d H:i:s")." - Optimized : "
                .$row["Name"]." ( ".$row["Data_free"]." )\n";
        $optQuery = "OPTIMIZE TABLE `".$row["Name"]."`";
        mysql_query($optQuery) or die(mysql_error());
    }
}
echo "---------------\n\n";
?>

Basics of maintaing a WordPress site

Nazly — Thu, 18 Nov 2010 17:36:45 +0000

WordPress is simple, yet powerful. Those are the key ingredients why it is popular among the Bloggers and Web Developers alike. WordPress will always be the first choice as a blogging platform. In the recent times it has moved far from being just a blogging engine. Most Web developers choose WordPress as their primary CMS of choice ahead of other popular Content Management Systems. Regardless of you being a Blogger or a Web Developer, the manner you maintain your WordPress files and database will play an important role in running a successful website.

Maintaining and monitoring your WordPress site requires performing certain tasks on a regular basis. The term regular will depend on how often your Website gets updated. Depending on the frequency you will need to plan this out. If you run a Website that gets updated daily (eg: news), then taking daily backups is a high priority. Performing these tasks manually can become a tedious process. There are plenty of tools and techniques available to automate them.

Monitoring a Website

You might want to use a Website Monitoring Service to check whether your website is up and running. There are plenty of services over the Internet if you Google for it. These services offer monitoring your Web Server (HTTP), FTP, Mail Server (POP & SMTP), SSL, DNS and custom TCP ports on regular intervals. If any of these don’t respond, the service will alert you via Email or SMS. Most of them offer limited options for free while you need to pay based on the additional services you require. This type of monitoring is not WordPress specific and can apply to any Wesbite.

If you are a developer you can code your own monitoring system which will be cost efficient and can be expandable according to your monitoring requirements. Using socket functions and tools like curl can assist in doing that.

In any of the case above, you will need to specify the port of the service you wish to monitor. In certain cases, the port can occasionally differ from the default port that particular service would run depending on how it is configured on your Server. So you need to specify the correct one.

There is one instance that most of the Bloggers/Web Developers fail to address. While your Web Server/Apache might run smoothly, there is a possibility that your database might not respond. This can happen when your DB gets corrupted, privileges get messed up or your MySQL Server crashes for some reason. In this case WordPress will send you this ugly message on a plain white page.

It’s not suitable to show an error message of this nature to your visitors. Most importantly the HTTP status code for this response is 200, which means Search Engines will pick it up as your content for the site. If this message hangs on for long, you will get indexed on Google with this Error Message.

Here is how you can display a custom error message with your design template and get notified at the same time when your DB doesn’t respond.

Create a file called db-error.php and store that file under the wp-contents folder.
The contents of the file should be..

header("HTTP/1.0 500 Internal Server Error");
mail("webmaster@mydomain.com", "DB fail", "Down at : ".date("Y-m-d H:i:s"));
echo "Your customer error message with yout HTML template";

It’s that simple..!

WordPress will execute the above script when the DB doesn’t respond. The first line will send the HTTP response code 500 (Internal Server Error) so that Search Engines won’t index it. The next line will send an email to you, notifying the time the DB went down. Third line will output the HTML of your template where you can include a custom error message. You can extend this piece of code to fit into your needs. Instead of Email, you could use SMS/Twitter to get instantly notified.

Backup

Taking regular backups of your WordPress installation and most importantly the database regularly will help to restore your Website within minutes if any data loss occurs. A hard disk crash, server failure or even a hacker wiping out all your data can be some of the reasons how a data loss could occur. While your hosting provider might do the backup for you, its always good to have your own. If you take regular backups, you are never a loser.

One of the important folders to backup is the wp-contents folder, but I would advise you to backup the whole WordPress installation directory. You will also need to backup the MySQL database. A simple MySQL dump would do. If you are the average blogger, then there are quite a lot of WordPress plugins for backup which would do the job for you. If you have shell access, I would advice you to write a shell script that would backup the files and database. Add a cron job to execute the script based on how frequent you wish to take backups. If you don’t have shell access, you can try using the options in your Web Hosting Control Panel.

Security

When it comes to security, applying the correct patches and upgrading to the newest version of the core and plugins of WordPress will prevent the hackers from trying to exploit the vulnerabilities in your Website. While this can be automated, it is advisable you do this process manually to reduce complications that may arise after an upgrade. It will depend on the plugins you use and the purpose you use it for. Golden rule before upgrade is to backup your WordPress files and DB so that you can restore it if something goes wrong.

Before choosing the right plugin to use, make sure to check the compatibility of the plugin with the version of WordPress you are running. Also make sure to check the ratings of the plugin. All these information is available on the plugin’s page.

Good luck..!!

Protect your website against SQL injection

Nazly — Wed, 17 Feb 2010 12:07:35 +0000

SQL injection is one of the deadliest techniques attackers use to exploit the weakness in your database code of your website. Regardless of the technology/scripting language you must make sure your code is 100% perfect against SQL injection.

Here I will use PHP and MySQL examples for its wide usage and also I’m much more comfortable with it.

Here is a basic PHP code that most developers will come up with to access the MySQL DB and get the record of a particular username submitted from a form in our website.


# Get posted username value
$userName = $_POST["usname"];
# MySQL query string to get the record of the user
$queryStr = "SELECT * FROM users WHERE usname = '$userName'");
# Output the string for debugging
echo $queryStr;
# Execute the MySQL query
$result = mysql_query($queryStr) or die(mysql_error());
?>

For example if the username that was submitted is nazly the code will output the following query and execute it.

SELECT * FROM users WHERE usname = 'nazly'

While the query works perfectly and returns the record of that particular user, a attacker can exploit this code by injecting SQL using the submission form.

For example if the attacker submits ‘ OR ‘t’='t instead of the username the query will be formed like this.

SELECT * FROM users WHERE usname = '' OR 't'='t'

When this query is executed, it will return all the records in the database since t=t will be TRUE always. The impact it will have on the website will be depend on the code after executing the query. But the important thing is someone can make the query behave differently than what we actually expected from it.

It can become deadlier than that if someone submits the following instead of the username
a’;DROP TABLE users; SELECT * FROM userinfo WHERE ‘t’ = ‘t
The query for the above value will look like this

SELECT * FROM users WHERE usname = 'a';
DROP TABLE users;
SELECT * FROM userinfo WHERE 't' = 't'

If the above query is executed, it will delete the whole users table. Similarly an attacker can inject any type of SQL code to modify/delete your tables in the database.

It is a huge security flaw in your code but newbies and even some experienced developers don’t understand the depth of problem. So developers should make sure to take precautionary measures against it.

In PHP you can use the mysql_real_escape_string() function for this task. This function will escape any special characters in the string to be used in a SQL statement.


# Get posted username value by escaping special characters
$userName = mysql_real_escape_string($_POST["usname"]);
# MySQL query string to get the record of the user
$queryStr = "SELECT * FROM users WHERE usname = '$userName'");
# Output the string for debugging
echo $queryStr;
# Execute the MySQL query
$result = mysql_query($queryStr) or die(mysql_error());
?>

If you try to inject SQL to this example, it will have no affect to the Database since the use of this function

If you are a developing a WordPress plugin for your website, you must make sure to protect the site against SQL injection as well. Since WordPress has its own class for database manipulation you should use the methods available in WordPress.

The escape() function in the WPDB class is much similar to using the standard mysql_real_escape_string() function.


function myWpPluginFunc($usName){
    global $wpdb;
    $u = $wpdb->escape($usName);
    $wpdb->query("SELECT * FROM users WHERE usname = '$u'");
}
?>

But there is a better option available in WordPress. Rather than escaping individual values you can format the SQL statement and then use the prepare() function in the WPDB class to escape the special characters. The syntax is similar to using sprintf(). Using the prepare() function, the developer is sure that all values are escaped. So less chance for errors.


function myWpPluginFunc($usName){
    global $wpdb;
    $qstr = $wpdb->prepare("SELECT * FROM users WHERE usname = %s", $usName);
    $wpdb->query($qstr);
}
?>

For more check out Data Validation in WordPress.

I wish MySQL functions in PHP had a similar function like WordPress’s prepare()

Restoring Large MySQL dumps

Nazly — Mon, 26 Nov 2007 06:37:52 +0000

A lot of things influenced me to write this post. In fact I wanted to write on this sometime back but finally I was able to squeeze some time. Lately I have been moving data between servers and the main problem I had was with large MySQL dumps because I have been using PHPMyAdmin for most of the MySQL operations and its one of the best tools available and most importantly its web-based. Even on my local development environment I'm comfortable with using PHPMyAdmin and on the web servers it can be very handy if the server is a shared hosting server. I would rather recommend using the command line client utilities that MySQL offers for import/export operations because its the safest. But you will need SSH access to your server. If you do have SSH access don't hesitate to choose this method above the others.

Export/Backup

Using the mysqldump client it is possible to backup a database into a SQL file which will contain SQL statements that can recreate the database tables when restored.

The following command from shell can be executed to backup a specific database

mysqldump -u [username] -p [password] [databasename] > [backupfile.sql]

Click Here for more options on using mysqldump

Import/Restore

You can use this command from shell to restore the database using the SQL dump file

mysql -u [username] -p [password] [databasename] < [backupfile.sql]

So thats quite basic and safest ways to import/export. But then you might ask whats the options you have if you are on a shared server and do not have SSH access. Well then PHPMyAdmin is the only choice available coz its web-based. Most servers have PHPMyAdmin as an option in the Server's Control Panel. Worst case if you don't have it or cannot find it you simply can download the source from www.phpmyadmin.net and upload the files via FTP and set it up. Installation is quite simple if you follow the Documentation.txt file. Once it is setup you can create databases etc. When you have selected a database from the left panel there will be tabs called SQL and Export. Using the SQL tab you can restore the database using your SQL dump file. Similarly you can use the Export option to export the data to a SQL file.

But then again you come across problems when you have a large MySQL dump. Exporting a large database won't be a problem but there are times that the SQL dump file tends to get corrupted for various reasons. Importing a large SQL dump file would create a problems coz with default installations there is a 2MB upload limit. This is not a PHPMyAdmin limit. This limit is set in the PHP configuration. To increase this upload limit you have to change the post_max_size and upload_max_filesize directives in the php.ini and then you can restore a large SQL dump. But if you are on a shared hosting server its highly unlikely that you can change the directives in the php.ini file. So thats where most people get stuck. When you are moving from your local machine to the server this is one problem you will face. Similar problem I faced when I had to move a large database from a server that I could SSH to a shared hosting server. I dumped the database to a SQL file using mysqldump command line utility and then when I tried to restore using PHPMyAdmin there was this upload limit. Arghh.. At that time I simply split the file manually into smaller files which are less than 2MB and uploaded one by one. It came to about 7 files at that time so didn't really bother about splitting them manually. This is a dirty trick but still effective but I won't suggest you to use this method. At a later time when I came across a similar instance I planned to write a tiny PHP script that would do the job. But thankfully I got a new server that I could SSH into.

So things can get bit messy at these situations so gotta figure out ways to overcome those with the limited resources we have. Lately I found BigDump: Staggered MySQL Dump Importer which seems to do the job on the web servers with hard runtime limit. So I guess I have more choices now. I haven't tried out this yet. Hopefully can play around with this next time when I have tight limits.

Copying records in MySQL

Nazly — Thu, 17 May 2007 06:12:05 +0000

Copying records from one table to another can be a very basic requirement. But writing queries to perform this task can be bit of a work around. But there is a very simple query to get this done.

Its by using INSERT … SELECT

http://dev.mysql.com/doc/refman/5.0/en/insert-select.html

Copy one record from a table to another

1	INSERT INTO `dest_table` SELECT * FROM source_table WHERE id = '10'

It can be very simple as this. Even multiple records can be copied from a single table or several tables. If I'm not mistaken INSERT … SELECT works on MySQL versions 4.1 and above.

MySQL GUI tools

Nazly — Tue, 20 Sep 2005 11:39:45 +0000

I have been playing around with the MySQL GUI tools lately specially with the MySQL Query Browser and its really easy to play around with SQL queries. I did use PHPMyAdmin effectively though I simply like the Query Browser. MySQL Migration Toolkit and MySQL Administrator are the other two GUI tools available. If you haven't started on it, go get 'em . You can download them from http://dev.mysql.com/downloads/

Export MySQL database to an MS Excel format

Nazly — Tue, 05 Jul 2005 02:09:55 +0000

Here is an updated version of the script I had at PHP-Help.net which exports a specified MySQL table. After couple of requests I got, I made some changes to the script to download all the tables of a specific database. Let me know if u catch any bugs.


#Name of the Database to export can be sent via GET variable called 'db'
$dbToExport=(isset($_GET["db"]))?$_GET["db"]:"test";
 
mysql_connect("localhost","root","******") or die("Connection to MySQL failed");
mysql_select_db($dbToExport) or die("Couldn't connect to DB");
 
#Get all tables in the database
$mTblQuery="show tables";
$mTblResult=mysql_query($mTblQuery);
 
$dataStr="";
#Loop through the table names
while($tblRow=mysql_fetch_assoc($mTblResult)){
	#Store output of the table name
	$dataStr.="Table : \t".$tblRow["Tables_in_".$dbToExport]."\r\n";
 
	#Select all records from the table
	$mQuery="select * from `".$tblRow["Tables_in_".$dbToExport]."`";	
	$mResult=mysql_query($mQuery);
 
	#Get no of fields in the table
	$numFields=mysql_num_fields($mResult) or die(mysql_error());
 
	#Get all fields in the table
	$tblFields=array();
	for($i=0;$i<$numFields;$i++){
		$tblFields[]=mysql_field_name($mResult,$i);
	}
 
	#Store output of fieldnames
	$dataStr.=implode("\t",$tblFields);
	$dataStr.="\r\n";
 
	#Store output of all the records
	while($row=mysql_fetch_assoc($mResult)){
		$rec=array();
		foreach($tblFields as $tblField){
			$recData=str_replace("\r\n"," ",$row[$tblField]);
			$recData=str_replace("\n"," ",$recData);
			$recData=str_replace("\t"," ",$recData);
 
			$rec[]=$recData;
		}
		$dataStr.=implode("\t",$rec);
		$dataStr.="\r\n";
	}
	$dataStr.="\r\n";
	$dataStr.="\r\n";
}
 
#Force the browser to download the file
header("Content-type: application/octet-stream");
header("Content-Disposition: attachment; filename=export_$dbToExport.xls");
header("Pragma: no-cache");
header("Expires: 0");
 
echo $dataStr;//Display Stored Output
?>

Storing Images in MySQL

Nazly — Wed, 29 Jun 2005 23:42:30 +0000

I just wrote a quick example that explains how to store images in MySQL database using the BLOB fieldtype which can used to store Binary Data. But I have been used to store the images in the filesystem and have a reference to it in the database. Most of them feel storing images or other binary data in the database is a bad idea as it creates too much overhead. But there are some advantages using this method as well. Have fun..

—– Table Structure —–

 
mysql> CREATE TABLE `imgtest` (
    -> `id` int(10) unsigned NOT NULL auto_increment,
    -> `imgstr` longblob NOT NULL,
    -> PRIMARY KEY (`id`)
    -> );

Insert image to the database


#Image Path
$imagePath="/path/to/img/img3.jpg";
 
#Connect to MySQL
mysql_connect("localhost","user","******");
mysql_select_db("test");
 
#Read Image file into a String
$imgStr=addslashes(file_get_contents($imagePath));
 
#Store Image String to the database
$query="insert into imgtest (imgstr) values ('".$imgStr."')";
mysql_query($query) or die(mysql_error());
?>

Display image from database


#Set Content Type
header("Content-Type: image/jpg");
 
#Image Id
$imgId=(isset($_GET["id"]))?$_GET["id"]:1;
 
#Connect to MySQL
mysql_connect("localhost","root","*****");
mysql_select_db("test");
 
#Output Image String from database
$query="select imgstr from imgtest where id='$imgId'";
$result=mysql_query($query);
$imgStr=mysql_result($result,0,"imgstr");
echo $imgStr;
?>

MySQL 4.1 authentication issue

Nazly — Sat, 11 Jun 2005 08:30:59 +0000

I have been working on MySQL 4.0.17 till now and wanted to upgrad to MySQL 4.1. So I downloaded and installed MySQL 4.1 and the installation was successful and it was working fine. But when I tried to access it through PHPMyAdmin or from any of my PHP code it threw this error


Client does not support authentication protocol requested by server;
consider upgrading MySQL client

So I checked it out at MySQL.com to see what I was missing and found out that in MySQL 4.1 the authentication process has been upgraded by making the hashes more secure. So when trying to access it with an old client it throws the above error.

The mysql extension in PHP doesn't support the new authentication protocol. So the workaround for this will be to reset the passwords in the format prior to MySQL 4.1. This can be done by using the old_password() function. Make sure to only reset the passwords of the users that need to access using a client prior to 4.1. Here is how it can be done

1
2
3

 
mysql> SET PASSWORD FOR
    -> 'some_user'@'some_host' = OLD_PASSWORD('newpwd');

 
mysql> UPDATE mysql.user SET Password = OLD_PASSWORD('newpwd')
    -> WHERE Host = 'some_host' AND User = 'some_user';
mysql> FLUSH PRIVILEGES;

The mysqli extension (MySQL Improved) which comes with PHP 5 is compatible with the improved password hashing in MySQL 4.1 and above.