View Issue Details

IDProjectCategoryView StatusLast Update
0023214mantisbtperformancepublic2018-03-31 19:58
Reporteratrol Assigned Toatrol  
PrioritynormalSeverityminorReproducibilityhave not tried
Status closedResolutionfixed 
Target Version2.13.0Fixed in Version2.13.0 
Summary0023214: Remove usage of outdated phputf8 library
Description

Advantages of removing the library and enforcing mbstring extension

  • better performance (no wrapping of mbstring functions)
  • less code which is not maintained, e.g there are no changes since years http://sourceforge.net/projects/phputf8/
  • less libraries which are not under composer control (just rss builder will remain at the moment)
TagsNo tags attached.

Relationships

related to 0021959 closeddregad Installer fails if mbstring extension is not installed 

Activities

atrol

atrol

2017-08-13 05:20

developer   ~0057447

WIP PR https://github.com/atrol/mantisbt/tree/remove-utf8-library

Not sure we can target this to 2.x or have to wait for 3.x.

fuge

fuge

2017-08-13 08:09

reporter   ~0057448

Hi,

We use mantis on a server without support for mbstring, and one that we can not add mbstring to.
We do not have any issues with utf8. Please do not remove something that works correctly without providing alternative.

You may find that the library hasn't changed as there is no need - if the library correctly counts the length of a utf8 string the code is not going to change!!

atrol

atrol

2017-08-13 09:17

developer   ~0057449

Last edited: 2017-08-13 09:18

It's not implemented at the moment, but it's documented that mbstring is mandatory https://mantisbt.org/docs/master/en-US/Admin_Guide/html-desktop/#admin.install.requirements.software

I looked at the code and found that even the current implementation needs the extension, at least if you want to use Markdown.
The underlying 3rd party library (parsedown) uses mb_strlen from the mbstring extension.

We use mantis on a server without support for mbstring, and one that we can not add mbstring to.

@fudge, why can't you add the extension?
If you don't have root access, did you ask your provider?

fuge

fuge

2017-08-13 10:57

reporter   ~0057451

Effectively yes.

There's a couple of internal reasons (Both technical and non-technical), externally the extension has had a few security issues recently (for example 3 CVE's in June).

In terms of the parsdown library, that library currently has several utf8 handling issues, somewhat surprised you've not hit issues, but then it may be that english is the predominate language and/or that the utf8 library also processing the text is hiding them

atrol

atrol

2017-08-13 14:38

developer   ~0057452

There's a couple of internal reasons

I see no way and no reason to consider internal reasons as long as they are internal.
Maybe you can share some more details.

the extension has had a few security issues recently

I don't see at the moment that there is a special reason not to use mbstring.
There are always security related fixes in PHP core and extensions.
You never know which CVE we will see tomorrow.
You have always to keep your complete operating environment up to date if you set value on security.

atrol

atrol

2018-02-05 15:43

developer   ~0058764

@fuge meanwhile I noticed that we are checking since 2014 for mbstring as a mandatory extension, see https://github.com/mantisbt/mantisbt/commit/5c8d5827d645eb95fdbd635cea08b92dc0662930

I told many users during the last years to run admin/check/index.php when they encountered any issue.
None of the users told that there is a error message concerning this extension.
This means there can't be that much users running Mantis without having mbstring enabled.

Unfortunately you did not share any more technical details, why enabling mbstring would be a problem for you.

I will provide a pull request for this change.
If other Mantis developers agree, the next Mantis version will require mbstring extension.

So this could be your last chance to prevent this change.

atrol

atrol

2018-02-05 15:55

developer   ~0058765

PR https://github.com/mantisbt/mantisbt/pull/1283

Related Changesets

MantisBT: master 63cac7b8

2017-07-16 11:46

atrol


Details Diff
Replace utf8_strlen by mb_strlen

Issue 0023214
Affected Issues
0023214
mod - bug_actiongroup_ext_page.php Diff File
mod - bug_actiongroup_page.php Diff File
mod - bugnote_view_inc.php Diff File
mod - core/api_token_api.php Diff File
mod - core/cfdefs/cfdef_standard.php Diff File
mod - core/custom_field_api.php Diff File
mod - core/database_api.php Diff File
mod - core/filter_api.php Diff File
mod - core/helper_api.php Diff File
mod - core/install_helper_functions_api.php Diff File
mod - core/relationship_api.php Diff File
mod - core/string_api.php Diff File
mod - core/user_api.php Diff File
mod - print_all_bug_page_word.php Diff File

MantisBT: master 292c7049

2017-07-16 12:18

atrol


Details Diff
Replace utf8_substr by mb_substr

Issue 0023214
Affected Issues
0023214
mod - bug_actiongroup_ext_page.php Diff File
mod - bugnote_view_inc.php Diff File
mod - core/authentication_api.php Diff File
mod - core/cfdefs/cfdef_standard.php Diff File
mod - core/columns_api.php Diff File
mod - core/custom_field_api.php Diff File
mod - core/database_api.php Diff File
mod - core/form_api.php Diff File
mod - core/helper_api.php Diff File
mod - core/install_helper_functions_api.php Diff File
mod - core/php_api.php Diff File
mod - core/plugin_api.php Diff File
mod - core/relationship_api.php Diff File
mod - core/string_api.php Diff File
mod - core/tag_api.php Diff File
mod - csv_export.php Diff File
mod - issues_rss.php Diff File
mod - news_rss.php Diff File
mod - plugins/MantisGraph/pages/issues_trend_bystatus_table.php Diff File
mod - print_all_bug_page_word.php Diff File
mod - return_dynamic_filters.php Diff File
mod - search.php Diff File

MantisBT: master 322acccd

2017-07-16 12:29

atrol


Details Diff
Replace utf8_strtolower by mb_strtolower

Issue 0023214
Affected Issues
0023214
mod - bug_actiongroup_ext_page.php Diff File
mod - core/columns_api.php Diff File
mod - core/database_api.php Diff File
mod - core/file_api.php Diff File
mod - core/filter_api.php Diff File
mod - core/filter_form_api.php Diff File
mod - core/helper_api.php Diff File
mod - core/install_helper_functions_api.php Diff File
mod - core/lang_api.php Diff File
mod - core/tag_api.php Diff File
mod - core/version_api.php Diff File
mod - manage_proj_cat_update.php Diff File
mod - print_all_bug_options_update.php Diff File
mod - signup.php Diff File

MantisBT: master 27090d47

2017-07-16 12:33

atrol


Details Diff
Replace utf8_strtoupper by mb_strtoupper

Issue 0023214
Affected Issues
0023214
mod - manage_tags_page.php Diff File
mod - manage_user_page.php Diff File

MantisBT: master 3278e4d5

2017-07-23 05:17

atrol


Details Diff
Move function utf8_str_pad to Mantis core

Thers is still no function mb_str_pad in latest PHP
https://bugs.php.net/bug.php?id=21317

The function has been changed to use no longer utf8_*, but mb_* functions.

I didn't rename the function to mb_str_pad to avoid any problems
if there will be mb_str_pad in later PHP versions.

Issue 0023214
Affected Issues
0023214
mod - core.php Diff File
mod - core/string_api.php Diff File

MantisBT: master 3034a126

2017-07-23 05:27

atrol


Details Diff
Remove loading of UTF8 library

Issue 0023214
Affected Issues
0023214
mod - core.php Diff File

MantisBT: master 18d5214f

2017-07-24 12:27

atrol


Details Diff
Remove UTF-8 library from source

Issue 0023214
Affected Issues
0023214
mod - core.php Diff File
mod - library/README.md Diff File
rm - library/utf8/ChangeLog Diff
rm - library/utf8/LICENSE Diff
rm - library/utf8/README Diff
rm - library/utf8/TODO.tsk Diff
rm - library/utf8/exp/regexunicode.php Diff
rm - library/utf8/index.html Diff
rm - library/utf8/mbstring/core.php Diff
rm - library/utf8/native/core.php Diff
rm - library/utf8/ord.php Diff
rm - library/utf8/readme_mantis.txt Diff
rm - library/utf8/str_ireplace.php Diff
rm - library/utf8/str_pad.php Diff
rm - library/utf8/str_split.php Diff
rm - library/utf8/strcasecmp.php Diff
rm - library/utf8/strcspn.php Diff
rm - library/utf8/stristr.php Diff
rm - library/utf8/strrev.php Diff
rm - library/utf8/strspn.php Diff
rm - library/utf8/substr_replace.php Diff
rm - library/utf8/trim.php Diff
rm - library/utf8/ucfirst.php Diff
rm - library/utf8/ucwords.php Diff
rm - library/utf8/utf8.php Diff
rm - library/utf8/utils/ascii.php Diff
rm - library/utf8/utils/bad.php Diff
rm - library/utf8/utils/patterns.php Diff
rm - library/utf8/utils/position.php Diff
rm - library/utf8/utils/specials.php Diff
rm - library/utf8/utils/unicode.php Diff
rm - library/utf8/utils/validation.php Diff

MantisBT: master 74eaa355

2017-08-13 01:47

atrol


Details Diff
Enforce mbstring extension during installation

Fixes 0023214
Affected Issues
0023214
mod - admin/install.php Diff File

MantisBT: master 3258e530

2018-02-05 23:58

atrol


Details Diff
Deprecate utf8 functions

Issue 0023214
Affected Issues
0023214
mod - core/string_api.php Diff File

MantisBT: master cb593b12

2018-02-07 12:47

atrol


Details Diff
After rebase, replace utf8_strtolower by mb_strtolower

Issue 0023214
Affected Issues
0023214
mod - core/user_api.php Diff File

MantisBT: master 82ae251f

2018-02-10 12:34

atrol


Details Diff
Correct library README

Issue 0023214
Affected Issues
0023214
mod - library/README.md Diff File

MantisBT: master 01762699

2018-02-13 22:50

atrol


Details Diff
Check mbstring at an early stage in core.php

Issue 0023214
Affected Issues
0023214
mod - core.php Diff File