View Issue Details

WikiJump to Notes
IDProjectCategoryView StatusDate SubmittedLast Update
0017403mantisbtdb mssqlpublic2014-06-03 03:012021-12-11 19:19
Reporterkarlito Assigned To 
PrioritynormalSeverityminorReproducibilityalways
Status newResolutionopen 
Product Version1.2.17 
Summary0017403: Report bug summary field in DB is too small if data contains characters in UTF-8
Description

Summary field in DB too small if data contains characters in UTF-8.
UI has maxlength limit of 128 characters for summary field, the same limit is in DB. If summary contains characters, that are in UTF-8 and character count is close to or equal to maxlength, there is an error "data would be truncated" because UTF-8 characters take more space.

A temporary workaround - increase summary field size in DB to 256 characters.

Steps To Reproduce

Try to report bug, enter in summary field text like (this is Latvian) "Par personām, kurām laika periodā no 01.01.2014 līdz 19.05.2014 izbeigta/apturēta valsts finansētu ilgstošas sociālās aprūpes".

TagsNo tags attached.

Activities

atrol

atrol

2014-06-03 05:36

developer   ~0040730

Not reproducible at this installation, see #17406
dregad

dregad

2014-06-03 05:52

developer   ~0040731

I would guess this is a problem with your database definition, make sure the collation for both the table and the columns is utf8.

In any case, also not reproducible locally (1.2.x head / mysql)
karlito

karlito

2014-06-03 09:18

reporter   ~0040734

Thanks.
Probably this is the case. We use MS SQL server, collation for table and current field is "Latvian_CI_AS".
Not sure, maybe we should use nvarchar instead of varchar for UTF data.
grangeway

grangeway

2014-06-03 18:43

reporter   ~0040738

I'm going to start by saying that the mssql support is broken, and our plan is to replace the DB layer after 1.3 to fix it properly.

In the mean time, I think you'll need to make use of NVARCHAR.

The other side of this is that MSSQL stores Unicode data in UCS-2 format, which is different to UTF8. I've yet to work out in my mind whether UCS-2 and UTF16 are the same thing.

In any case, if you are using NVARCHAR fields in MSSQL, the data is automatically stored in UTF16 format regardless of the collation.

Can I assume this is a new installation?

I can generate you a SQL script to run against MSSQL that will generate a more sensible set of tables then the current DB Layer generates.

However, I think you then move on to the next problem that we'll require custom changes to the DB layer in 1.3 to support using UTF8 data with MSSQL via the php sql server driver...

Personally, if you can live with few temporary work around's to get by for now, and drop me an email to paul@mantisforge.org, once we've got 1.3 out the door (soon) and start testing new db layer (shortly afterwards), we'll be able to contact you.

Equally, from a personal point of view, once we do start looking at the new db layer and fixing this sort of stuff properly, if you are willing to help test this and especially able to assist with non-English testing, I'll be more then happy to help diagnose any database specific issues or assist in any way to ensure you get a good experience with using Mantis on Windows via Skype/teamviewer/similar and free of any charge.
karlito

karlito

2014-06-04 01:09

reporter   ~0040739

No, this is not a new installation. Actually, we started using Mantis from version 1.1.0a2.
Since then we have had upgrades to 1.1.6, 1.2.11 and only recently 1.2.17.
I think I'll stick to temporary workarounds for now, but I can not promise we will move to 1.3 soon after it's release as it's not for me to decide.

However, I think I could manage to do some testing on our db copy and probably could assist with non-English testing.