Posts Tagged ‘mysql database’

Make MySQL existing users to have access from any or particular host after SQL migration

Tuesday, July 1st, 2014

make_mysql_existing_users_have-access-from-any-or-particular-host-after-SQL-migration
Recently I've done a migration of MySQL server from host A (running and configured to serve requests on (localhost – 127.0.0.1) to host B (server2.host.com)
There are already existing users in mysql which are allowed to only access the database server from localhost as until now the applciation was sending SQL queries straight on localhost. Now the architecture has to change to use the MySQL Database remotely.

Hence I've migrated the MySQL server by dumping all the existing the databases on MySQL host A  with:

mysqldump -u root -p --all-databases > alldbs_dump.sql


And then importing the databases on host B with

mysql -u root -p < alldbs_dump.sql

Though this migrated the data from Host A to Host B, still the application on Host A was failing to succesfully use its data from database on Host B, because of inability to properly authenticate. It couldn't authenticate because MySQL on Host B's users are not configured to have access from IP address of Host A, but only allowed the application users to be able to connect on localhost..

I've used following SQL CLI query to check Hosts allowed to connect to MySQL (in this case localhost):

# mysql -u root -p
mysql> use mysql;
mysql> select * from user where user like '%eameiotest%' and Host='localhost';

To fix that I logged on MySQL server on Host B with mysql cli and issued for each of the users the application was using:

UPDATE mysql.user SET Host='%' WHERE Host='localhost' AND User='eameiotest';
 

UPDATE mysql.user SET Host='%' WHERE Host='localhost' AND User='eameiotest2';
 

UPDATE mysql.user SET Host='%' WHERE Host='localhost' AND User='eameiotest3';

On execution, If you get errors like:
 

ERROR 1062 (23000): Duplicate entry '%-eameiotest' for key 'PRIMARY'


Don't think that there is no solution, as I've read some threads online claiming the only way to get around this issue is to dump mysql database and re-import it, this is not necessery. There is a work around to this MySQL bug.

To work-around the error, you will first have to set the user allowed access host to empty – ' ' :

UPDATE mysql.user SET Host='' WHERE Host='localhost' AND User='eameiotest';
 

UPDATE mysql.user SET Host='' WHERE Host='localhost' AND User='eameiotest2';
 

UPDATE mysql.user SET Host='' WHERE Host='localhost' AND User='eameiotest3';


And re-issue again commands:
 

UPDATE mysql.user SET Host='%' WHERE Host='localhost' AND User='eameiotest';
 

UPDATE mysql.user SET Host='%' WHERE Host='localhost' AND User='eameiotest2';
 

UPDATE mysql.user SET Host='%' WHERE Host='localhost' AND User='eameiotest3';


You might want to also issue:
 

GRANT ALL PRIVILEGES ON yourdatabase-name.* TO 'eameiotest1'@'server-host';

GRANT ALL PRIVILEGES ON yourdatabase-name.* TO 'eameiotest2'@'server-host';

GRANT ALL PRIVILEGES ON yourdatabase-name.* TO 'eameiotest3'@'server-host';
 

This should have solve the app connection issues, Cheers 🙂

Allowing MySQL users access from all hosts – Fixing mysql ERROR 1045 (28000): Access denied for user ‘root’@’remote-admin.com’ (using password: YES)

Friday, June 20th, 2014

mysql_allow_access-from-remote-any-host-fix-access-error-after-sql-migration

I recently migrated MySQL database server from host A to host B (remotesystemadministration.com), because I wanted to have the mysql database server on a separate machine (have separation of server running services and have a dedicated mysql server).

MySQL server host (running on localhost previously was set from my mysql config my.cnf to listen and serve connections on localhost with

bind-address = 127.0.0.1

). MySQL is used by a Tomcat running Java application on localhost and my task was to set the Tomcat to use the MySQL database remotely to MySQL host B (new remote hostname where MySQL is moved is  remotesystemadminsitration.com and is running on IP 83.228.93.76).

Migration from MySQL Db server 1 (host A) to MySQL Db server 2 (host B) is done by binary copying the mysql database directory which in this case is (as it is a Debian server installed MySQL), the standard directory where mysql stores its database data is /var/lib/mysql ( datadir = /var/lib/mysql in /etc/mysql/my.cnf)

Binary copying of data from MySQL db (host A) to MySQL Db (host B) is done with rsync

After migrating and trying to login on migrated mysql  database on remotesystemadministration.net with mysql cli client:

remotesysadmin:~$ mysql -u root -p

I got following error:
 

ERROR 1045 (28000): Access denied for user 'root'@'remotesystemadministration.com' (using password: YES)


To fix the issue I had to login remotely from old migration server mysql (host A) cli:

mysql:~$ mysql -u root -p -h remotesystemadministration.com

and  run SQL commands:

GRANT ALL PRIVILEGES ON *.* TO 'root'@'remotesystemadministration.com' WITH GRANT OPTION;
GRANT USAGE ON *.* TO 'root'@'remotesystemadministration.com' IDENTIFIED BY 'secret-mysql-pass';
FLUSH PRIVILEGES;

Query OK, 0 rows affected (0.03 sec)
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.00 sec)


Another way to solve the problem is to add the root user to be able to connect from any host (Enable MySQL root access from all host), to do so issue:

GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'password';
FLUSH PRIVILEGES;

Note: In newer version of MySQL, flush privileges could be omitted.

Another approach if you want to substitute access from localhost for all users and enable all users to be able to authenticate to mysql remotely is to execute SQL Query:

UPDATE USER SET host='%' WHERE host='localhost';

Allowing all users to be able to connect from anywhere on the internet is a very bad security practice anyways, if you already have a tight firewall setup and you can only access the server via specific remote IP addresses allowing MySQL access from all hosts / ips should be ok.

How to extract database from a whole MySQL database dump with sed

Tuesday, February 12th, 2013

If you have a configured mysql dumps to be prepared regularly via cronjob
and you happen to experience crash or lost of data and need to restore only a certain
database from a whole bunch of MySQL data SCRIPT file letes say mysql-fulldump.sql, you might be wondering how.
Here is how:

# sed -n '/^-- Current Database: `DATABASE_NAME`/,/^-- Current Database: `/p' mysql-fulldump.sql > DATABASE_NAME.sql

Change DATABASE_NAME with whatever your required database name is.
mysql-fulldump.sql – is the name of whole database file dump file
DATABASE_NAME.sql – will contain the extracted database from complete SQL dump.
TThat's all, now import new dump to MySQL and Enjoy 🙂

How to check and repair broken MySQL ISAM tables

Monday, July 11th, 2011

MySQL repair artistic picture

If you are stuffed with errors in /var/log/mysqld.log similar to:

110711 11:00:48 [ERROR] /usr/libexec/mysqld: Incorrect information in file: './anyboots_moncler_spaccio/zen_seo_cache.frm'
110711 11:00:48 [ERROR] /usr/libexec/mysqld: Incorrect information in file: './anyboots_moncler_spaccio/zen_sessions.frm'

This is a sure sign something terrible has happened with your mysql database tables that lead to corruption.
Having corrupt table in mysql installation can severely lead to data loss as well as significantly reduce the speed and performance of a MySQL server in this awful times mysqlcheck is the best friend of the administrator, here is how you can check and repair broken tables in MySQL server:

mysql-server:~# mysqlcheck --all-databases -u root -p
chillor_hjbgl.vn_users OK
chillor_lul.mybb_adminlog OK
chillor_lul.mybb_adminoptions OK
chillor_lul.mybb_adminsessions OK
chillor_lul.mybb_adminviews OK
chillor_lul.mybb_announcements OK
...

You will notice the corrupt sql tables will be reported as corrupt by the tool and mysqlcheck will try it’s best to recover the corrupt tables.

In most cases this should be enough to recover corrupt tables.

How to list and exclude table names from a database in MySQL (exclude table names from an show tables in MySQL) by using information_schema

Wednesday, March 30th, 2011

Listing all table names from a MySQL database is a very easy and trivial task that every sql or system administrator out there is aware of.

However excluding certain table names from a whole list of tables belonging to a database is not that commonly used and therefore I believe many people have no clue how to do it when they have to.

Today for one of my sql backup scripts it was necessary that certain tables from a database to be excluded from the whole list of tables for a database I’m backupping.
My example database has the sample name exampledatabase and usually I do list all the table contents from that database with the well known command:

mysql> SHOW tables from exampledatabase;

However as my desire was to exclude certain tables from the list (preferrably with a certain SQL query) I had to ask around in irc.freenode.net for some hints on a ways to achieve my exclude table goals.

I was adviced by some people in #mysql that what I need to achieve my goal is the information_schema mysql structure, which is available since MySQL version 5.0.

After a bit of look around in the information_schema and the respective documentation on mysql.com, thanksfully I could comprehend the idea behind the information_schema, though to be honest the first time I saw the documentation it was completly foggy on how to use this information_schema;
It seems using the information_schema is very easy and is not much different from your normal queries syntax used to do trivial operations in the mysql server.

If you wonder just like I did what is mysql’s information_schema go and use the information_schema database (which I believe is a virtual database that is stored in the system memory).

For instance:

mysql> use information_schema;
Database changed
mysql> show tables
+---------------------------------------+
| Tables_in_information_schema |
+---------------------------------------+
| CHARACTER_SETS |
| COLLATIONS |
| COLLATION_CHARACTER_SET_APPLICABILITY |
| COLUMNS |
| COLUMN_PRIVILEGES |
| KEY_COLUMN_USAGE |
| PROFILING |
| ROUTINES |
| SCHEMATA |
| SCHEMA_PRIVILEGES |
| STATISTICS |
| TABLES |
| TABLE_CONSTRAINTS |
| TABLE_PRIVILEGES |
| TRIGGERS |
| USER_PRIVILEGES |
| VIEWS |
+---------------------------------------+
17 rows in set (0.00 sec)

To get a general view on what each of the tables in the information_schema database contains I used the normal SELECT command for example

mysql> select * from TABLES limit 10;

I used the limit clause in order to prevent being overfilled with data, where I could still see the table fields name to get general and few lines of the table to get an idea what kind of information the TABLES table contains.

If you haven’t got any ecperience with using the information_schema I would advice you do follow my example select and look around through all the listed tables in the information_schema database

That will also give you a few hints about the exact way the MySQL works and comprehends it’s contained data structures.

In short information_schema virtual database and it’s existing tables provides a very thorough information and if you’re an SQL admin you certainly want to look over it every now and then.

A bit of playing with it lead me to a command which is actually a good substitute for the normal SHOW TABLES; mysql command.
To achieve a SHOW TABLES from exampledatabase via the information_schema info structure you can for example issue:

select TABLE_NAME from TABLES where TABLE_SCHEMA='exampledatabase';

Now as I’ve said a few words about information_schema let me go back to the main topic of this small article, which is How to exclude table names from a SHOW tables list

Here is how exclude a number of tables from a complete list of tables belonging to a database:

select TABLE_NAME from TABLES where TABLE_SCHEMA='exampledatabase'
AND TABLE_NAME not in
('mysql_table1_to_exlude_from_list', 'mysql_table2_to_exclude_from_list', 'table3_to_exclude');

In this example the above mysql command will list all the tables content belonging to exampledatabase and instruct the MySQL server not to list the table names with names mysql_table1_to_exlude_from_list, mysql_table2_to_exclude_from_list, table3_to_exclude

If you need to exclude more tables from your mysql table listing just add some more tables after the …’table3_to_exclude’, ‘new_table4_to_exclude’,’etc..’);

Of course this example can easily be adopted to a MySQL backup script which requires the exclusion of certain tables from a backed up database.

An example on how you can use the above table exclude command straight from the bash shell would be:

debian:~# echo "use information_schema; select TABLE_NAME from TABLES where
TABLE_SCHEMA='exampledatabase' AND TABLE_NAME not in
('mysql_table1_to_exlude_from_list', 'mysql_table2_to_exclude_from_list', 'table3_to_exclude',);"
| mysql -u root -p

Now this little bash one-liner can easily be customized to a backup script to create backups of a certain databases with a certain tables (e.g. with excluded number of tables) from the backup.

It’s seriously a pity that by default the mysqldump command does not have an option for a certain tables exclude while making a database dump.
I’ve saw the mysqldump exclude option, being suggested somewhere online as a future feature of mysqldump, I’ve also seen it being reported in the mysql.com’s bug database, I truly hope in the upcoming releases we will see the exclude option to appear as a possible mysqldump argument.
 

Optimize, check and repair tables in MySQL, howto improve work with tables in MySQL

Monday, April 12th, 2010

There are few quick tips that helps if some unexpected downtime of your SQL server occurs. Even though nowdays this won’t happen too often with servers running with a good ups, sometimes even administrator errors can cause problems with your mysql tables. If your MySQL server refuses to start, it’s quite probable that you’re experiencing a problem with a broken table or tables in MySQL. Therefore you need to go through all your mysql databases and check the consistency of your MyISAM or Innodb tables, ofcourse accordingly to your MySQL database types. To check a certain table for consistency with MySQL after you select the database, you have to execute: mysql$ CHECK TABLE your_table_name; If the above command after presumably executed with all your databases and there consequent tables reports, everytime OK then your MySQL crashes are not caused by table incosistencies. However if instead of OK the CHECK TABLE reports Corruptthen you have a broken table and you have to fix it as soon as possible, in order to be able to bring up to life the MySQL server once again. Here is an example of a broken table after a CHECK REPAIR searchindex; : +------------------+-------+----------+------------------------------------+ | Table | Op | Msg_type | Msg_text | +------------------+-------+----------+------------------------------------+ | test.searchindex | check | error | Key in wrong position at page 4096 | | test.searchindex | check | error | Corrupt | +------------------+-------+----------+------------------------------------+ To fix the CORRUPTED or BROKEN table as also known you have to issue the command: mysql$ REPAIR TABLE yourtable_name; Depending on your table size after a while, if everything is going fine you should see something like: +------------------+--------+----------+----------+ | Table | Op | Msg_type | Msg_text | +------------------+--------+----------+----------+ | test.searchindex | repair | status | OK | +------------------+--------+----------+----------+ 1 row in set (0.08 sec) Be aware that sometimes in order to fix a broken table you have to use the MySQL repair extended function. Expect The EXTENDED REPAIR function option to take a much more time, even sometimes with large databases with million of records it could take hours, especially if the MySQL server is serving other client requests as well. This terrible siutation sometimes occurs because of mysql locks, though I believe locks are probably a topic of another post. Hopefully after issuing that the table in MySQL would properly repair and your MySQL will begin starting up with the rc script once again. Apart from crashes and table repairs there are few nice things concerning MySQL that are doing me good every now and then. I’m talking about the MySQL functions: ANALYZE TABLE and OPTIMIZE TABLE ANALYZE TABLE does synchronization of the information concerning the variables within tables that has a INDEX key settled according to the database to which they belong. In other simply words, executing ANALYZE TABLE to your database tables every now and then and that would probably help in speeding up the code executed in the SQL that has JOINS involved. The second one OPTIMIZE TABLE is natively supported with MyISAM SQL database types, and secondary supported with Innodb, where the Optimize with Innodb is done in a non-traditional way. When invoked to process an Innodb table OPTIMIZE TABLE does use ALTER TABLE to achieve an Innodb table optimization. In practice what the optimize table does is defragmentation of the table unto which it’s executed. A quick example of the optimize table is for instance: OPTIMIZE TABLE your_table_name; In order to find out which tables need to be defragmented or in other words needs optimize table you have to issue the cmd: show table status where Data_free!=0; Note that you have to issue this command on each of your databases; Just because this is so boring you can of course use my script check_optimize_sql.sh which will quickly loop through all the databases and show you which tables need to be optimized. I’ve written also a second shell script that loops through all MySQL databases and lists all databases and sub tables that requires optimize and further on proceeds optimizing to download the script check_and_optimize_sql_tables.sh click here Happy optimizing 🙂