View Issue Details

IDProjectCategoryView StatusLast Update
0006782mantisbtsqlpublic2007-12-21 23:16
Reporterlstrojny Assigned Tovboctor  
PrioritynormalSeverityminorReproducibilityalways
Status closedResolutionfixed 
Product Version1.0.0 
Target Version1.1.0Fixed in Version1.1.0a4 
Summary0006782: MantisBT should do "SET NAMES $charset" on connect to database
Description

If you run MySQL 5.0 and you set init-connect="SET NAMES utf8" in your my.cf this will result in suspicious charset problems. I fixed it by adding a $g_db->Execute ("SET NAMES latin1"); in function db_connect() in core/database_api.php. I'm not sure how to provide a real fix but I'm sure developers will do.

TagsNo tags attached.
Attached Files
6782.diff (4,084 bytes)   
Index: database_api.php
===================================================================
RCS file: /cvsroot/mantisbt/mantisbt/core/database_api.php,v
retrieving revision 1.57
diff -u -r1.57 database_api.php
--- database_api.php	9 Jun 2007 15:00:03 -0000	1.57
+++ database_api.php	20 Jul 2007 06:47:11 -0000
@@ -34,27 +34,42 @@
 
 	# --------------------
 	# Make a connection to the database
-	function db_connect( $p_dsn, $p_hostname = null, $p_username = null, $p_password = null, $p_database_name = null, $p_db_schema = null ) {
+	function db_connect( $p_dsn, $p_hostname = null, $p_username = null, $p_password = null, $p_database_name = null, $p_db_schema = null, $p_pconnect = false ) {
 		global $g_db_connected, $g_db;
 
-		if(  $p_dsn === false ) {
+		if ( $p_dsn === false ) {
 			$t_db_type = config_get_global( 'db_type' );
 			$g_db = ADONewConnection( $t_db_type );
-			$t_result = $g_db->Connect( $p_hostname, $p_username, $p_password, $p_database_name );
-			if ( db_is_db2() && $p_db_schema !== null && !is_blank( $p_db_schema ) ) {
-				$result = &$g_db->execute('set schema ' . $p_db_schema);
-				if ( $result === FALSE ) {
-					db_error();
-					trigger_error( ERROR_DB_CONNECT_FAILED, ERROR );
-					return false;
-				}
+			
+			if ( $p_pconnect ) {
+				$t_result = $g_db->PConnect( $p_hostname, $p_username, $p_password, $p_database_name );
+			} else {
+				$t_result = $g_db->Connect( $p_hostname, $p_username, $p_password, $p_database_name );
 			}
 		} else {
 			$g_db = ADONewConnection( $p_dsn );
 			$t_result = $g_db->IsConnected();
 		}
 
-		if ( !$t_result ) {
+		if ( $t_result ) {
+			# For MySQL, the charset for the connection needs to be specified.
+			if ( db_is_mysql() ) {
+				$c_charset = db_prepare_string( lang_get( 'charset' ) );
+
+				# @@@ Is there a way to translate any charset name to MySQL format? e.g. remote the dashes?
+				# @@@ Is this needed for other databases?
+				if ( strtolower( $c_charset ) === 'utf-8' ) {
+					db_query( 'SET NAMES UTF8' );
+				}
+			} elseif ( db_is_db2() && $p_db_schema !== null && !is_blank( $p_db_schema ) ) {
+				$t_result2 = db_query( 'set schema ' . $p_db_schema );
+				if ( $t_result2 === false ) {
+					db_error();
+					trigger_error( ERROR_DB_CONNECT_FAILED, ERROR );
+					return false;
+				}
+			}
+		} else {
 			db_error();
 			trigger_error( ERROR_DB_CONNECT_FAILED, ERROR );
 			return false;
@@ -68,33 +83,7 @@
 	# --------------------
 	# Make a persistent connection to the database
 	function db_pconnect( $p_dsn, $p_hostname = null, $p_username = null, $p_password = null, $p_database_name = null, $p_db_schema = null ) {
-		global $g_db_connected, $g_db;
-
-		if(  $p_dsn === false ) {
-			$t_db_type = config_get_global( 'db_type' );
-			$g_db = ADONewConnection( $t_db_type );
-			$t_result = $g_db->PConnect($p_hostname, $p_username, $p_password, $p_database_name );
-	
-			if ( db_is_db2() && $p_db_schema !== null && !is_blank( $p_db_schema ) ) {
-				$result = &$g_db->execute('set schema ' . $p_db_schema);
-				if ( $result === FALSE ) {
-					db_error();
-					trigger_error( ERROR_DB_CONNECT_FAILED, ERROR );
-					return false;
-				}
-			}
-		} else {
-			$g_db = ADONewConnection( $p_dsn );
-			$t_result = $g_db->IsConnected();
-		}
-
-		if ( !$t_result ) {
-			db_error();
-			trigger_error( ERROR_DB_CONNECT_FAILED, ERROR );
-			return false;
-		}
-		$g_db_connected = true;
-		return true;
+		return db_connect( $p_dsn, $p_hostname, $p_username, $p_password, $p_database_name, $p_db_schema, /* $p_pconnect */ true );
 	}
 
 	# --------------------
@@ -106,6 +95,19 @@
 	}
 
 	# --------------------
+	# Checks if the database is MySQL
+	function db_is_mysql() {
+		$t_db_type = config_get( 'db_type' );
+
+		switch( $t_db_type ) {
+			case 'mysql':
+				return true;
+		}
+
+		return false;
+	}
+
+	# --------------------
 	# Check is the database is PostgreSQL
 	function db_is_pgsql() {
 		$t_db_type = config_get( 'db_type' );
6782.diff (4,084 bytes)   

Relationships

related to 0007779 closedgrangeway [all lang] no upgrade path to 1.10 for non-latin characters 
related to 0008227 closedsiebrand MySQL compatibility 

Activities

Martin Fuchs

Martin Fuchs

2007-07-18 18:00

reporter   ~0015097

For Mantis version 1.1.x upwards utf8 is used as default character set. So utf8 should be used instead of latin1 for the database connection.

I had to insert the following additional line into db_connect() in core/database_api.php:

$ret = db_query('set names utf8', -1, -1);

Otherwise the Umlaut characters in bug reports are not stored correctly into the database.

vboctor

vboctor

2007-07-20 02:53

manager   ~0015107

I've attached that does the following:

  1. Add db_is_mysql()
  2. In case of mysql and UTF-8 character set call SET NAMES UTF8.
  3. Refactored db_pconnect() to use db_connect().

I would like to get some feedback on the following:

  1. Does the patch fix the problem?
  2. Should we call SET NAMES on non-UTF8 character set? If so, then is there a way to translate the charset into the format understood by MySQL? For example, in the language files we UTF-8 where MySQL understands UTF8.
  3. Should we do a similar call for MS-SQL or PostgreSQL?
Martin Fuchs

Martin Fuchs

2007-07-20 17:37

reporter   ~0015115

Yes, thanks - the patch fixes the problem after I inserted an additional include statement at the top of database_api.php:
require_once( $t_core_path.'gpc_api.php' );

But what I don't really understand: Why are you using lang_get() and evaluate the s_charset constant? I think the database connection should always be encoded in UTF-8, regardless of the current language setting of the user. Of course this requires that the database is prepared for storing UTF encoded strings. So this should be guaranted by the database installation procedure.

vboctor

vboctor

2007-07-25 04:37

manager   ~0015214

I've commited the fix and added the gpc_api.php include. This patch is designed in a way to reduce possibly break non-UTF8 scenarios. If there are other issues after this patch, then we should create new issues for them.