Friday, September 29, 2006

MySQL 5.1 Cluster Testing

Hi,

It has been a while since I last posted. We have been busy testing MySQL Cluster, MySQL Replication and MySQL Cluster Replication. Good news is the serious bugs are getting harder to find. This is allowing us to spend some time automating more testing like configuration and up/down grade testing for example.

The documentation guy for cluster, Jonathan "Jon" Stephens, has been doing great work to expand the Cluster on-line manual. If you have not been out there in a while to review you should have a look.

http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster.html

I feel very positive about this version of the cluster as it has had more hands on testing then ever before. The testing includes: Functional, Negative, Load/Stress, Performance, Recovery, Network Failure, etc...

The new Disk Data will allow customer with huge database to start taking advantage of the reliability and scalability that the cluster offers.

In addition the Cluster Replication opens a whole new use for cluster. This feature adds a few more 9's to the 99.999 up time for services and allows for many different strategies in database design to take place.

One example would be to replicate to another cluster so that data mining and report generation can be done with out putting a load on the main system.

In addition, you could replicate to another site so that if something goes wrong at the main site the secondary site could take over minimizing the companies down time.

I would love to here from the users out there in the world that are using the cluster and have ideas of test they would like to see done, and or want to share how they are using it so we can incorporate that into our testing.

I will attempt to post more often and keep you apprised of our efforts and share any tips or trick I run across that might run across.

Cheers!

/ Jeb

Jonathan Miller
Austin, Texas USA
Senior Lead Quality Assurance Developer
MySQL AB www.mysql.com

Jumpstart your cluster!
http://www.mysql.com/consulting/packaged/cluster.html

Get training on clusters
http://www.mysql.com/training/courses/mysql_cluster.html

All-in-one Enterprise-grade Database, Support and Services
http://www.mysql.com/network/

1 comment:

vAN pHAM said...

Jonathan,

I'm currently testing Mysql Cluster 7.0 and 7.1.3. 1) Perform rolling restart

2) Perform rolling restart on several Data/Storage nodes while running TCP-C benchmark with 20 users.I could stop and start node 3 and node 4 while running TCP-C benchmark with 20 users without a problem.

3) Doing backup while running TCP-C benchmark with 20 users
Backup started with no problem when perform with less than 20 users. When doing backup nowait with 1710 concurrent users, the backup fail with the following error:
Node 2: Backup request from 1 failed to start. Error: 1304

4) Perform rolling shutdown of the master Data/Storage node

5) Changing the number of replicas and other important parameters in the config.ini after the cluster has started is very difficult. Changing the number of replicas in a cluster such as changing the replicas from 2 to 4 requires the cluster to be restart with the –initial option which will wiped out data

6) Soft restart of the Data/Storage server while running benchmark with high number of concurrent users.

For this test I kill the master Data node by doing init 6 while running benchmark and see if the data node will automatically switch to a new Master node.
Result: Ran init 6 on the master node while doing AS3AP benchmark and the cluster was able to switch to another data node. I perform this test several times and did not encounter any problem.

7) change tables in one schema from memory based to disk based. I was able to do this without problem, but the data and rows on the disked based schema seems smaller than when it was memory based. Is this a bug ?

The TCP-C, AS3AP benchmark results shows that Mysql Cluster did much better on a more database intense benchmark like AS3AP than the typical Mysql Master/Slave. Also, the Master/Slave servers generate much higher Total CPU usage (over 90% Total CPU Usage) while running AS3AP then the Mysql Cluster.