Is it safe to migrate from utf8_general_ci to utf8mb4_unicode_ci?

[acoder2020]

My users want to use emoticons which requires utf8mb4_unicode_ci.. Is it generally safe to migrate to this new collation?

[cas]

I think that shpould be fine on MySql level but safesty option of course to test this in a seperate environment to ensure it work from both sides (Mantis/MySql) as expected.

[acoder2020]

Converting the database itself to utf8mb4_unicode_ci was easy.

It looks like the Mantis codebase is hardcoded in a ton of places to only use utf8, I'm guessing this is a relic of the past.. because utf8mb4_unicode_ci is pretty much the standard for today.

I'd hoped this would be easy... but I'd need to back out a ton of those hard-coded instances and not sure how much time I have to devout to this right now.

Code: Select all

grep -r "utf8" /data/www/mantis/mantisbt-2.27.0/

/data/www/mantis/mantisbt-2.27.0/core/database_api.php: * using native utf8 encoding, which only supports 3 bytes chars (see #20431).
/data/www/mantis/mantisbt-2.27.0/core/database_api.php:function db_mysql_fix_utf8( $p_string ) {
/data/www/mantis/mantisbt-2.27.0/core/database_api.php:			$t_utf8_offset = mb_strlen( substr( $p_query, 0, $t_match_param[1] ), mb_internal_encoding() );
/data/www/mantis/mantisbt-2.27.0/core/database_api.php:				$p_query = mb_substr( $p_query, 0, $t_utf8_offset )
/data/www/mantis/mantisbt-2.27.0/core/database_api.php:					. mb_substr( $p_query, $t_utf8_offset + mb_strlen( $t_match_param[0] ) );
/data/www/mantis/mantisbt-2.27.0/core/classes/MantisPlugin.class.php:	 *			array('mysql' => 'ENGINE=MyISAM DEFAULT CHARSET=utf8', 'pgsql' => 'WITHOUT OIDS')));
/data/www/mantis/mantisbt-2.27.0/core/email_queue_api.php:	$c_subject = db_mysql_fix_utf8( $t_email_data->subject );
/data/www/mantis/mantisbt-2.27.0/core/email_queue_api.php:	$c_body = db_mysql_fix_utf8( $t_email_data->body );
/data/www/mantis/mantisbt-2.27.0/core/bugnote_api.php:	$t_bugnote_text = db_mysql_fix_utf8( $t_bugnote_text );
/data/www/mantis/mantisbt-2.27.0/core/bugnote_api.php:	$p_bugnote_text = db_mysql_fix_utf8( $p_bugnote_text );
/data/www/mantis/mantisbt-2.27.0/core/cfdefs/cfdef_standard.php:	'#function_value_to_database' => 'db_mysql_fix_utf8',
/data/www/mantis/mantisbt-2.27.0/core/cfdefs/cfdef_standard.php:	'#function_value_to_database' => 'db_mysql_fix_utf8',
/data/www/mantis/mantisbt-2.27.0/core/string_api.php: * Calls htmlspecialchars on the specified string, handling utf8
/data/www/mantis/mantisbt-2.27.0/core/string_api.php: * @see utf8_substr
/data/www/mantis/mantisbt-2.27.0/core/string_api.php:function utf8_str_pad( $input, $length, $padStr = ' ', $type = STR_PAD_RIGHT ) {
/data/www/mantis/mantisbt-2.27.0/core/string_api.php:    trigger_error('utf8_str_pad: Unknown padding type (' . $type . ')',E_USER_ERROR);
/data/www/mantis/mantisbt-2.27.0/core/string_api.php:function utf8_strlen( $p_string ) {
/data/www/mantis/mantisbt-2.27.0/core/string_api.php:function utf8_substr( $p_string, $p_offset, $p_length = NULL ) {
/data/www/mantis/mantisbt-2.27.0/core/string_api.php:function utf8_strtolower( $p_string ) {
/data/www/mantis/mantisbt-2.27.0/core/custom_function_api.php:	echo utf8_str_pad( '', $p_issue_level * 36, ' ' );
/data/www/mantis/mantisbt-2.27.0/core/custom_function_api.php:	echo utf8_str_pad( '', $p_issue_level * 36, ' ' );
/data/www/mantis/mantisbt-2.27.0/core/bug_api.php:				$p_value = db_mysql_fix_utf8( $p_value );
/data/www/mantis/mantisbt-2.27.0/core/bug_api.php:				$p_value = db_mysql_fix_utf8( $p_value );
/data/www/mantis/mantisbt-2.27.0/core/email_api.orig:		$t_message .= utf8_str_pad( lang_get_defaulted( $t_custom_field_name ) . ': ', $t_email_padding_length );
/data/www/mantis/mantisbt-2.27.0/core/email_api.orig:			$t_message .= $t_email_separator1 . "\n" . utf8_str_pad( lang_get( 'bug_relationships' ), 20 ) . utf8_str_pad( lang_get( 'id' ), 8 ) . lang_get( 'summary' ) . "\n" . $t_email_separator2 . "\n" . $p_visible_bug_data['relations'];
/data/www/mantis/mantisbt-2.27.0/core/email_api.orig:		$t_message .= utf8_str_pad( lang_get( 'date_modified' ), 17 ) . utf8_str_pad( lang_get( 'username' ), 15 ) . utf8_str_pad( lang_get( 'field' ), 25 ) . utf8_str_pad( lang_get( 'change' ), 20 ) . " \n";
/data/www/mantis/mantisbt-2.27.0/core/email_api.orig:			$t_message .= utf8_str_pad( date( $t_normal_date_format, $t_raw_history_item['date'] ), 17 ) . utf8_str_pad( $t_raw_history_item['username'], 15 ) . utf8_str_pad( $t_localized_item['note'], 25 ) . utf8_str_pad( $t_localized_item['change'], 20 ) . "\n";
/data/www/mantis/mantisbt-2.27.0/core/email_api.orig:		return utf8_str_pad( lang_get( $p_attribute_id ) . ': ', config_get( 'email_padding_length' ) )
/data/www/mantis/mantisbt-2.27.0/core/email_api.orig:	$t_relationship_info_text = utf8_str_pad( $t_relationship_descr, 20 );
/data/www/mantis/mantisbt-2.27.0/core/email_api.orig:	$t_relationship_info_text .= utf8_str_pad( bug_format_id( $t_related_bug_id ), 8 );
/data/www/mantis/mantisbt-2.27.0/core/email_api.php:		$t_message .= utf8_str_pad( lang_get_defaulted( $t_custom_field_name ) . ': ', $t_email_padding_length );
/data/www/mantis/mantisbt-2.27.0/core/email_api.php:			$t_message .= $t_email_separator1 . "\n" . utf8_str_pad( lang_get( 'bug_relationships' ), 20 ) . utf8_str_pad( lang_get( 'id' ), 8 ) . lang_get( 'summary' ) . "\n" . $t_email_separator2 . "\n" . $p_visible_bug_data['relations'];
/data/www/mantis/mantisbt-2.27.0/core/email_api.php:		$t_message .= utf8_str_pad( lang_get( 'date_modified' ), 17 ) . utf8_str_pad( lang_get( 'username' ), 15 ) . utf8_str_pad( lang_get( 'field' ), 25 ) . utf8_str_pad( lang_get( 'change' ), 20 ) . " \n";
/data/www/mantis/mantisbt-2.27.0/core/email_api.php:			$t_message .= utf8_str_pad( date( $t_normal_date_format, $t_raw_history_item['date'] ), 17 ) . utf8_str_pad( $t_raw_history_item['username'], 15 ) . utf8_str_pad( $t_localized_item['note'], 25 ) . utf8_str_pad( $t_localized_item['change'], 20 ) . "\n";
/data/www/mantis/mantisbt-2.27.0/core/email_api.php:		return utf8_str_pad( lang_get( $p_attribute_id ) . ': ', config_get( 'email_padding_length' ) )
/data/www/mantis/mantisbt-2.27.0/core/email_api.php:	$t_relationship_info_text = utf8_str_pad( $t_relationship_descr, 20 );
/data/www/mantis/mantisbt-2.27.0/core/email_api.php:	$t_relationship_info_text .= utf8_str_pad( bug_format_id( $t_related_bug_id ), 8 );
/data/www/mantis/mantisbt-2.27.0/core/template_api.php:			$cfdata .= '<tr><td>'.utf8_str_pad( lang_get_defaulted( $t_custom_field_name, null ) . ': ', $t_email_padding_length, ' ', STR_PAD_RIGHT ).'</td>';
/data/www/mantis/mantisbt-2.27.0/core/template_api.php:			$hisdata .= '<tr><td>'.$hisDate . '</td><td>' . utf8_str_pad( $t_his['username'], 15 ). '</td><td>' . utf8_str_pad( $t_localized_item['note'], 25 ) . '</td><td>'. utf8_str_pad( $t_localized_item['change'], 20 ) . '</td></tr>';
/data/www/mantis/mantisbt-2.27.0/core/template_api.php:				$lasthisdata .= '<tr><td>'.$hisDate . '</td><td>' . utf8_str_pad( $t_his['username'], 15 ). '</td><td>' . utf8_str_pad( $t_localized_item['note'], 25 ) . '</td><td>'. utf8_str_pad( $t_localized_item['change'], 20 ) . '</td></tr>';
/data/www/mantis/mantisbt-2.27.0/core/template_api.php:	//			$lasthisdata .= $hisDate . " => " . utf8_str_pad( $t_his['username'], 15 ). " => " . utf8_str_pad( $t_localized_item['note'], 25 ). " => " . utf8_str_pad( $t_localized_item['change'], 20 ) . "<br>";
/data/www/mantis/mantisbt-2.27.0/plugins/MantisCoreFormatting/files/syntax-highlighting/components/prism-avisynth.min.js:!function(e){function a(e,a,r){return RegExp(function(e,a){return e.replace(/<<(\d+)>>/g,(function(e,r){return a[+r]}))}(e,a),r||"")}var r="bool|clip|float|int|string|val",t=[["is(?:bool|clip|float|int|string)|defined|(?:(?:internal)?function|var)?exists?","apply|assert|default|eval|import|nop|select|undefined","opt_(?:allowfloataudio|avipadscanlines|dwchannelmask|enable_(?:b64a|planartopackedrgb|v210|y3_10_10|y3_10_16)|usewaveextensible|vdubplanarhack)|set(?:cachemode|maxcpu|memorymax|planarlegacyalignment|workingdir)","hex(?:value)?|value","abs|ceil|continued(?:denominator|numerator)?|exp|floor|fmod|frac|log(?:10)?|max|min|muldiv|pi|pow|rand|round|sign|spline|sqrt","a?sinh?|a?cosh?|a?tan[2h]?","(?:bit(?:and|not|x?or|[lr]?shift[aslu]?|sh[lr]|sa[lr]|[lr]rotatel?|ro[rl]|te?st|set(?:count)?|cl(?:ea)?r|ch(?:an)?ge?))","average(?:[bgr]|chroma[uv]|luma)|(?:[rgb]|chroma[uv]|luma|rgb|[yuv](?=difference(?:fromprevious|tonext)))difference(?:fromprevious|tonext)?|[yuvrgb]plane(?:median|min|max|minmaxdifference)","getprocessinfo|logmsg|script(?:dir(?:utf8)?|file(?:utf8)?|name(?:utf8)?)|setlogparams","chr|(?:fill|find|left|mid|replace|rev|right)str|format|[lu]case|ord|str(?:cmpi?|fromutf8|len|toutf8)|time|trim(?:all|left|right)","isversionorgreater|version(?:number|string)","buildpixeltype|colorspacenametopixeltype","addautoloaddir|on(?:cpu|cuda)|prefetch|setfiltermtmode"].join("|"),["has(?:audio|video)","height|width","frame(?:count|rate)|framerate(?:denominator|numerator)","getparity|is(?:field|frame)based","bitspercomponent|componentsize|hasalpha|is(?:planar(?:rgba?)?|interleaved|rgb(?:24|32|48|64)?|y(?:8|u(?:va?|y2))?|yv(?:12|16|24|411)|420|422|444|packedrgb)|numcomponents|pixeltype","audio(?:bits|channels|duration|length(?:[fs]|hi|lo)?|rate)|isaudio(?:float|int)"].join("|"),["avi(?:file)?source|directshowsource|image(?:reader|source|sourceanim)|opendmlsource|segmented(?:avisource|directshowsource)|wavsource","coloryuv|convertbacktoyuy2|convertto(?:RGB(?:24|32|48|64)|(?:planar)?RGBA?|Y8?|YV(?:12|16|24|411)|YUVA?(?:411|420|422|444)|YUY2)|fixluminance|gr[ae]yscale|invert|levels|limiter|mergea?rgb|merge(?:chroma|luma)|rgbadjust|show(?:alpha|blue|green|red)|swapuv|tweak|[uv]toy8?|ytouv","(?:colorkey|reset)mask|layer|mask(?:hs)?|merge|overlay|subtract","addborders|(?:bicubic|bilinear|blackman|gauss|lanczos4|lanczos|point|sinc|spline(?:16|36|64))resize|crop(?:bottom)?|flip(?:horizontal|vertical)|(?:horizontal|vertical)?reduceby2|letterbox|skewrows|turn(?:180|left|right)","blur|fixbrokenchromaupsampling|generalconvolution|(?:spatial|temporal)soften|sharpen","trim|(?:un)?alignedsplice|(?:assume|assumescaled|change|convert)FPS|(?:delete|duplicate)frame|dissolve|fade(?:in|io|out)[02]?|freezeframe|interleave|loop|reverse|select(?:even|odd|(?:range)?every)","assume[bt]ff|assume(?:field|frame)based|bob|complementparity|doubleweave|peculiarblend|pulldown|separate(?:columns|fields|rows)|swapfields|weave(?:columns|rows)?","amplify(?:db)?|assumesamplerate|audiodub(?:ex)?|audiotrim|convertaudioto(?:(?:8|16|24|32)bit|float)|converttomono|delayaudio|ensurevbrmp3sync|get(?:left|right)?channel|kill(?:audio|video)|mergechannels|mixaudio|monotostereo|normalize|resampleaudio|ssrc|supereq|timestretch","animate|applyrange|conditional(?:filter|reader|select)|frameevaluate|scriptclip|tcp(?:server|source)|writefile(?:end|if|start)?","imagewriter","blackness|blankclip|colorbars(?:hd)?|compare|dumpfiltergraph|echo|histogram|info|messageclip|preroll|setgraphanalysis|show(?:framenumber|smpte|time)|showfiveversions|stack(?:horizontal|vertical)|subtitle|tone|version"].join("|")].join("|");e.languages.avisynth={comment:[{pattern:/(^|[^\\])\[\*(?:[^\[*]|\[(?!\*)|\*(?!\])|\[\*(?:[^\[*]|\[(?!\*)|\*(?!\]))*\*\])*\*\]/,lookbehind:!0,greedy:!0},{pattern:/(^|[^\\])\/\*[\s\S]*?(?:\*\/|$)/,lookbehind:!0,greedy:!0},{pattern:/(^|[^\\$])#.*/,lookbehind:!0,greedy:!0}],argument:{pattern:a('\\b(?:<<0>>)\\s+("?)\\w+\\1',[r],"i"),inside:{keyword:/^\w+/}},"argument-label":{pattern:/([,(][\s\\]*)\w+\s*=(?!=)/,lookbehind:!0,inside:{"argument-name":{pattern:/^\w+/,alias:"punctuation"},punctuation:/=$/}},string:[{pattern:/"""[\s\S]*?"""/,greedy:!0},{pattern:/"(?:\\(?:\r\n|[\s\S])|[^"\\\r\n])*"/,greedy:!0,inside:{constant:{pattern:/\b(?:DEFAULT_MT_MODE|(?:MAINSCRIPT|PROGRAM|SCRIPT)DIR|(?:MACHINE|USER)_(?:CLASSIC|PLUS)_PLUGINS)\b/}}}],variable:/\b(?:last)\b/i,boolean:/\b(?:false|no|true|yes)\b/i,keyword:/\b(?:catch|else|for|function|global|if|return|try|while|__END__)\b/i,constant:/\bMT_(?:MULTI_INSTANCE|NICE_FILTER|SERIALIZED|SPECIAL_MT)\b/,"builtin-function":{pattern:a("\\b(?:<<0>>)\\b",[t],"i"),alias:"function"},"type-cast":{pattern:a("\\b(?:<<0>>)(?=\\s*\\()",[r],"i"),alias:"keyword"},function:{pattern:/\b[a-z_]\w*(?=\s*\()|(\.)[a-z_]\w*\b/i,lookbehind:!0},"line-continuation":{pattern:/(^[ \t]*)\\|\\(?=[ \t]*$)/m,lookbehind:!0,alias:"punctuation"},number:/\B\$(?:[\da-f]{6}|[\da-f]{8})\b|(?:(?:\b|\B-)\d+(?:\.\d*)?\b|\B\.\d+\b)/i,operator:/\+\+?|[!=<>]=?|&&|\|\||[?:*/%-]/,punctuation:/[{}\[\]();,.]/},e.languages.avs=e.languages.avisynth}(Prism);
/data/www/mantis/mantisbt-2.27.0/plugins/MantisCoreFormatting/files/syntax-highlighting/components/prism-scheme.min.js:!function(e){e.languages.scheme={comment:/;.*|#;\s*(?:\((?:[^()]|\([^()]*\))*\)|\[(?:[^\[\]]|\[[^\[\]]*\])*\])|#\|(?:[^#|]|#(?!\|)|\|(?!#)|#\|(?:[^#|]|#(?!\|)|\|(?!#))*\|#)*\|#/,string:{pattern:/"(?:[^"\\]|\\.)*"/,greedy:!0},symbol:{pattern:/'[^()\[\]#'\s]+/,greedy:!0},char:{pattern:/#\\(?:[ux][a-fA-F\d]+\b|[-a-zA-Z]+\b|[\uD800-\uDBFF][\uDC00-\uDFFF]|\S)/,greedy:!0},"lambda-parameter":[{pattern:/((?:^|[^'#])[(\[]lambda\s+)(?:[^|()\[\]'\s]+|\|(?:[^\\|]|\\.)*\|)/,lookbehind:!0},{pattern:/((?:^|[^'#])[(\[]lambda\s+[(\[])[^()\[\]']+/,lookbehind:!0}],keyword:{pattern:/((?:^|[^'#])[(\[])(?:begin|case(?:-lambda)?|cond(?:-expand)?|define(?:-library|-macro|-record-type|-syntax|-values)?|defmacro|delay(?:-force)?|do|else|except|export|guard|if|import|include(?:-ci|-library-declarations)?|lambda|let(?:rec)?(?:-syntax|-values|\*)?|let\*-values|only|parameterize|prefix|(?:quasi-?)?quote|rename|set!|syntax-(?:case|rules)|unless|unquote(?:-splicing)?|when)(?=[()\[\]\s]|$)/,lookbehind:!0},builtin:{pattern:/((?:^|[^'#])[(\[])(?:abs|and|append|apply|assoc|ass[qv]|binary-port\?|boolean=?\?|bytevector(?:-append|-copy|-copy!|-length|-u8-ref|-u8-set!|\?)?|caar|cadr|call-with-(?:current-continuation|port|values)|call\/cc|car|cdar|cddr|cdr|ceiling|char(?:->integer|-ready\?|\?|<\?|<=\?|=\?|>\?|>=\?)|close-(?:input-port|output-port|port)|complex\?|cons|current-(?:error|input|output)-port|denominator|dynamic-wind|eof-object\??|eq\?|equal\?|eqv\?|error|error-object(?:-irritants|-message|\?)|eval|even\?|exact(?:-integer-sqrt|-integer\?|\?)?|expt|features|file-error\?|floor(?:-quotient|-remainder|\/)?|flush-output-port|for-each|gcd|get-output-(?:bytevector|string)|inexact\??|input-port(?:-open\?|\?)|integer(?:->char|\?)|lcm|length|list(?:->string|->vector|-copy|-ref|-set!|-tail|\?)?|make-(?:bytevector|list|parameter|string|vector)|map|max|member|memq|memv|min|modulo|negative\?|newline|not|null\?|number(?:->string|\?)|numerator|odd\?|open-(?:input|output)-(?:bytevector|string)|or|output-port(?:-open\?|\?)|pair\?|peek-char|peek-u8|port\?|positive\?|procedure\?|quotient|raise|raise-continuable|rational\?|rationalize|read-(?:bytevector|bytevector!|char|error\?|line|string|u8)|real\?|remainder|reverse|round|set-c[ad]r!|square|string(?:->list|->number|->symbol|->utf8|->vector|-append|-copy|-copy!|-fill!|-for-each|-length|-map|-ref|-set!|\?|<\?|<=\?|=\?|>\?|>=\?)?|substring|symbol(?:->string|\?|=\?)|syntax-error|textual-port\?|truncate(?:-quotient|-remainder|\/)?|u8-ready\?|utf8->string|values|vector(?:->list|->string|-append|-copy|-copy!|-fill!|-for-each|-length|-map|-ref|-set!|\?)?|with-exception-handler|write-(?:bytevector|char|string|u8)|zero\?)(?=[()\[\]\s]|$)/,lookbehind:!0},operator:{pattern:/((?:^|[^'#])[(\[])(?:[-+*%/]|[<>]=?|=>?)(?=[()\[\]\s]|$)/,lookbehind:!0},number:{pattern:RegExp(function(e){for(var r in e)e[r]=e[r].replace(/<[\w\s]+>/g,(function(r){return"(?:"+e[r].trim()+")"}));return e[r]}({"<ureal dec>":"\\d+(?:/\\d+)|(?:\\d+(?:\\.\\d*)?|\\.\\d+)(?:[esfdl][+-]?\\d+)?","<real dec>":"[+-]?<ureal dec>|[+-](?:inf|nan)\\.0","<imaginary dec>":"[+-](?:<ureal dec>|(?:inf|nan)\\.0)?i","<complex dec>":"<real dec>(?:@<real dec>|<imaginary dec>)?|<imaginary dec>","<num dec>":"(?:#d(?:#[ei])?|#[ei](?:#d)?)?<complex dec>","<ureal box>":"[0-9a-f]+(?:/[0-9a-f]+)?","<real box>":"[+-]?<ureal box>|[+-](?:inf|nan)\\.0","<imaginary box>":"[+-](?:<ureal box>|(?:inf|nan)\\.0)?i","<complex box>":"<real box>(?:@<real box>|<imaginary box>)?|<imaginary box>","<num box>":"#[box](?:#[ei])?|(?:#[ei])?#[box]<complex box>","<number>":"(^|[()\\[\\]\\s])(?:<num dec>|<num box>)(?=[()\\[\\]\\s]|$)"}),"i"),lookbehind:!0},boolean:{pattern:/(^|[()\[\]\s])#(?:[ft]|false|true)(?=[()\[\]\s]|$)/,lookbehind:!0},function:{pattern:/((?:^|[^'#])[(\[])(?:[^|()\[\]'\s]+|\|(?:[^\\|]|\\.)*\|)(?=[()\[\]\s]|$)/,lookbehind:!0},identifier:{pattern:/(^|[()\[\]\s])\|(?:[^\\|]|\\.)*\|(?=[()\[\]\s]|$)/,lookbehind:!0,greedy:!0},punctuation:/[()\[\]']/}}(Prism);
/data/www/mantis/mantisbt-2.27.0/plugins/Csv_import/pages/import_issues_inc.php:	return string_html_specialchars( utf8_encode($t_string) );
/data/www/mantis/mantisbt-2.27.0/plugins/Csv_import/pages/import_issues_inc.php:	$t_str = utf8_encode(strtolower(trim(utf8_decode($t_str))));
/data/www/mantis/mantisbt-2.27.0/plugins/Csv_import/pages/import_issues_inc.php:	$t_value = ( ($t_column === false) || (!isset( $p_row[$t_column] )) ) ? $p_default : utf8_encode(trim( $p_row[$t_column] ));
/data/www/mantis/mantisbt-2.27.0/plugins/GaugeSupport/GaugeSupport.php:					array( "mysql" => "DEFAULT CHARSET=utf8" )
/data/www/mantis/mantisbt-2.27.0/plugins/Statistics/jpgraph/jpgraph_led.php:        if(($n = mb_strlen($aValStr,'utf8')) == 0) {
/data/www/mantis/mantisbt-2.27.0/plugins/Statistics/jpgraph/jpgraph_led.php:            $d = mb_substr($aValStr, $i, 1, 'utf8');
/data/www/mantis/mantisbt-2.27.0/plugins/Statistics/jpgraph/jpgraph_ttf.inc.php:            return $this->g2312->gb2utf8($aTxt);
/data/www/mantis/mantisbt-2.27.0/plugins/Statistics/jpgraph/jpgraph_ttf.inc.php:        return utf8_encode($o);
/data/www/mantis/mantisbt-2.27.0/plugins/Statistics/jpgraph/jpgraph_gb2312.php:    function gb2utf8($gb) {
/data/www/mantis/mantisbt-2.27.0/plugins/Statistics/jpgraph/jpgraph_gb2312.php:        $utf8='';
/data/www/mantis/mantisbt-2.27.0/plugins/Statistics/jpgraph/jpgraph_gb2312.php:                $utf8 .= $this->u2utf8($this->codetable[hexdec(bin2hex($t))-0x8080]);
/data/www/mantis/mantisbt-2.27.0/plugins/Statistics/jpgraph/jpgraph_gb2312.php:                $utf8 .= $this->u2utf8($t);
/data/www/mantis/mantisbt-2.27.0/plugins/Statistics/jpgraph/jpgraph_gb2312.php:        return $utf8;
/data/www/mantis/mantisbt-2.27.0/plugins/Statistics/jpgraph/jpgraph_gb2312.php:    function u2utf8($c) {
/data/www/mantis/mantisbt-2.27.0/admin/install.php:				'mysql' => 'DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci',
/data/www/mantis/mantisbt-2.27.0/admin/schema.php:	'mysql' => 'DEFAULT CHARSET=utf8',
/data/www/mantis/mantisbt-2.27.0/admin/schema.php:# and user.email columns to 255 chars. This causes issues with utf8mb4 charset
/data/www/mantis/mantisbt-2.27.0/admin/schema.php:# Limiting column size, planning for future MySQL utf8mb4 support (see #20465)
/data/www/mantis/mantisbt-2.27.0/admin/check/check_database_inc.php:		check_is_collation_utf8( $t_collation ),
/data/www/mantis/mantisbt-2.27.0/admin/check/check_database_inc.php:				check_is_collation_utf8( $t_row['collation'] ),
/data/www/mantis/mantisbt-2.27.0/admin/check/check_database_inc.php:					check_is_collation_utf8( $t_row['collation'] ),
/data/www/mantis/mantisbt-2.27.0/admin/check/check_api.php:function check_is_collation_utf8( $p_collation ) {
/data/www/mantis/mantisbt-2.27.0/admin/check/check_api.php:	return substr( $p_collation, 0, 4 ) === 'utf8';
/data/www/mantis/mantisbt-2.27.0/vendor/phpmailer/phpmailer/src/PHPMailer.php:        $is_utf8 = static::CHARSET_UTF8 === strtolower($this->CharSet);
/data/www/mantis/mantisbt-2.27.0/vendor/phpmailer/phpmailer/src/PHPMailer.php:                            if ($is_utf8) {
/data/www/mantis/mantisbt-2.27.0/vendor/phpmailer/phpmailer/src/PHPMailer.php:                                $len = $this->utf8CharBoundary($word, $len);
/data/www/mantis/mantisbt-2.27.0/vendor/phpmailer/phpmailer/src/PHPMailer.php:                        if ($is_utf8) {
/data/www/mantis/mantisbt-2.27.0/vendor/phpmailer/phpmailer/src/PHPMailer.php:                            $len = $this->utf8CharBoundary($word, $len);
/data/www/mantis/mantisbt-2.27.0/vendor/phpmailer/phpmailer/src/PHPMailer.php:    public function utf8CharBoundary($encodedText, $maxLength)
/data/www/mantis/mantisbt-2.27.0/vendor/adodb/adodb-php/adodb.inc.php:	 * - mysqli: setConnectionParameter(MYSQLI_SET_CHARSET_NAME,'utf8mb4');
/data/www/mantis/mantisbt-2.27.0/vendor/ezyang/htmlpurifier/library/HTMLPurifier/Encoder.php:     * @note Fallback code adapted from utf8ToUnicode by Henri Sivonen and
/data/www/mantis/mantisbt-2.27.0/vendor/ezyang/htmlpurifier/library/HTMLPurifier/Encoder.php:     *       hsivonen@iki.fi at <http://iki.fi/hsivonen/php-utf8/> under the
/data/www/mantis/mantisbt-2.27.0/vendor/ezyang/htmlpurifier/library/HTMLPurifier/Encoder.php:                foreach ($ascii_fix as $utf8 => $native) {
/data/www/mantis/mantisbt-2.27.0/vendor/ezyang/htmlpurifier/library/HTMLPurifier/Encoder.php:                    $clear_fix[$utf8] = '';
/data/www/mantis/mantisbt-2.27.0/vendor/ezyang/htmlpurifier/library/HTMLPurifier/AttrDef/URI/Host.php:            $idna = new Net_IDNA2(array('encoding' => 'utf8', 'overlong' => false, 'strict' => true));
/data/www/mantis/mantisbt-2.27.0/vendor/ezyang/htmlpurifier/library/HTMLPurifier/Generator.php:                'utf8'

[acoder2020]

Should anyone else want to take a peek or work on this here are the ALTERS needed to migrate to utf8mb4_unicode_ci



ALTER DATABASE bugtracker CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

ALTER TABLE `mantis_api_token_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_file_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_history_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_monitor_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_relationship_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_revision_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_tag_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_text_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bugnote_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bugnote_text_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_category_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_config_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_custom_field_project_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_custom_field_string_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_custom_field_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_email_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_filters_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_news_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_plugin_FAQ_results_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_plugin_GaugeSupport_support_data_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_plugin_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_project_file_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_project_hierarchy_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_project_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_project_user_list_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_project_version_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_sponsorship_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_tag_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_tokens_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_pref_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_print_pref_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_profile_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_table` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

ALTER TABLE `mantis_api_token_table` MODIFY `name` varchar(128) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_api_token_table` MODIFY `hash` varchar(128) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_file_table` MODIFY `title` varchar(250) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_file_table` MODIFY `description` varchar(250) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_file_table` MODIFY `diskfile` varchar(250) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_file_table` MODIFY `filename` varchar(250) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_file_table` MODIFY `folder` varchar(250) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_file_table` MODIFY `file_type` varchar(250) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_history_table` MODIFY `field_name` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_history_table` MODIFY `old_value` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_history_table` MODIFY `new_value` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_revision_table` MODIFY `value` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_table` MODIFY `os` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_table` MODIFY `os_build` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_table` MODIFY `platform` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_table` MODIFY `version` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_table` MODIFY `fixed_in_version` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_table` MODIFY `build` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_table` MODIFY `summary` varchar(128) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_table` MODIFY `target_version` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_text_table` MODIFY `description` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_text_table` MODIFY `steps_to_reproduce` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bug_text_table` MODIFY `additional_information` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bugnote_table` MODIFY `note_attr` varchar(250) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_bugnote_text_table` MODIFY `note` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_category_table` MODIFY `name` varchar(128) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_config_table` MODIFY `config_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_config_table` MODIFY `value` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_custom_field_string_table` MODIFY `value` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_custom_field_string_table` MODIFY `text` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_custom_field_table` MODIFY `name` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_custom_field_table` MODIFY `possible_values` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_custom_field_table` MODIFY `default_value` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_custom_field_table` MODIFY `valid_regexp` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_email_table` MODIFY `email` varchar(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_email_table` MODIFY `subject` varchar(250) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_email_table` MODIFY `metadata` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_email_table` MODIFY `body` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_filters_table` MODIFY `name` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_filters_table` MODIFY `filter_string` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_news_table` MODIFY `headline` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_news_table` MODIFY `body` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_plugin_FAQ_results_table` MODIFY `question` varchar(250) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_plugin_FAQ_results_table` MODIFY `answere` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_plugin_table` MODIFY `basename` varchar(40) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_project_file_table` MODIFY `title` varchar(250) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_project_file_table` MODIFY `description` varchar(250) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_project_file_table` MODIFY `diskfile` varchar(250) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_project_file_table` MODIFY `filename` varchar(250) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_project_file_table` MODIFY `folder` varchar(250) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_project_file_table` MODIFY `file_type` varchar(250) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_project_table` MODIFY `name` varchar(128) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_project_table` MODIFY `file_path` varchar(250) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_project_table` MODIFY `description` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_project_version_table` MODIFY `version` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_project_version_table` MODIFY `description` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_sponsorship_table` MODIFY `logo` varchar(128) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_sponsorship_table` MODIFY `url` varchar(128) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_tag_table` MODIFY `name` varchar(100) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_tag_table` MODIFY `description` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_tokens_table` MODIFY `value` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_pref_table` MODIFY `bugnote_order` varchar(4) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_pref_table` MODIFY `language` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_pref_table` MODIFY `timezone` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_print_pref_table` MODIFY `print_pref` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_profile_table` MODIFY `platform` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_profile_table` MODIFY `os` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_profile_table` MODIFY `os_build` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_profile_table` MODIFY `description` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_table` MODIFY `username` varchar(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_table` MODIFY `realname` varchar(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_table` MODIFY `email` varchar(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_table` MODIFY `password` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `mantis_user_table` MODIFY `cookie_string` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

[cas]

This is what I expected hence the advice to work wit a sepaarte test environment

Converting the database itself to utf8mb4_unicode_ci was easy.

It looks like the Mantis codebase is hardcoded in a ton of places to only use utf8, I'm guessing this is a relic of the past.. because utf8mb4_unicode_ci is pretty much the standard for today.

I'd hoped this would be easy... but I'd need to back out a ton of those hard-coded instances and not sure how much time I have to devout to this right now.

[dregad]

This is a long-standing issue... Never found the time to work on it

Please read https://mantisbt.org/bugs/view.php?id=20431

[acoder2020]

I wish it were as easy as just choosing a new collation in MySQL. I didn't read all of the code history but looking at it through 2025 eyes it's weird that a collation was specifically hard-coded into an application. I'm sure there were valid reasons at the time but it looks off in 2025.