View Issue Details

WikiRelated ChangesetsJump to Notes
IDProjectCategoryView StatusDate SubmittedLast Update
0013649mantisbtscriptingpublic2011-12-06 03:542012-11-23 18:02
Reportersbs Assigned Torombert  
PrioritynormalSeverityblockReproducibilityalways
Status closedResolutionno change required 
PlatformWindows 
Product Version1.2.8 
Summary0013649: Check-In fails on logs with accented chars
Description

The attached log does not import using checking and PHP 5, on a FR-FR version of windows running the script in a command prompt

TagsNo tags attached.
Attached Files
tmp_4142_bug.txt (193 bytes)    
SVN Revision 4142 
dt
2011-12-06 09:42:46 +0100 (mar., 06 d�c. 2011)
FIX:0001448
    Added XML Schema validation of the incoming config file before loading it to detect malformed files.
tmp_4142_bug.txt (193 bytes)   

Activities

rombert

rombert

2011-12-07 14:30

reporter   ~0030497

Is this about the source integration plugin?
sbs

sbs

2011-12-07 14:52

reporter   ~0030498

As far as I know, it is not, I am just using the checkin.php function to add notes when some commits happen in our svn repo.

I did not use the plugin since I am under the impression that it needs some sort of cron to run while the checkin is run from the repository at the commit time.
rombert

rombert

2011-12-07 14:56

reporter   ~0030499

OK, I see now. Is there some sort of error message that you can add to the report?
sbs

sbs

2011-12-07 15:04

reporter   ~0030500

Here is the return message from php for that text file:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<link rel="stylesheet" type="text/css" href="http://localhostd:\Mantisbt-1.2.8\Scripts\/css/default.css" />
<script type="text/javascript"><!--
if(document.layers) {document.write("<style>td{padding:0px;}<\/style>")}
// --></script>
<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
<meta http-equiv="Pragma" content="no-cache" />
<meta http-equiv="Cache-Control" content="no-cache" />
<meta http-equiv="Pragma-directive" content="no-cache" />
<meta http-equiv="Cache-Directive" content="no-cache" />
<meta http-equiv="Expires" content="Wed, 07 Dec 2011 20:03:21 GMT" />
<link rel="shortcut icon" href="d:\Mantisbt-1.2.8\Scripts\images/favicon.ico" type="image/x-icon" />
<link rel="search" type="application/opensearchdescription+xml" title="MantisBT: Text Search" href="http://localhostd:\Manti
sbt-1.2.8\Scripts\/browser_search_plugin.php?type=text" /> <link rel="search" type="application/opensearchdescription+xml" titl
e="MantisBT: Issue Id" href="http://localhostd:\Mantisbt-1.2.8\Scripts\/browser_search_plugin.php?type=id" /> <title>MantisBT</tit
le>
<script type="text/javascript" src="d:\Mantisbt-1.2.8\Scripts\javascript/min/common.js"></script>
<script type="text/javascript">var loading_lang = "Loading...";</script><script type="text/javascript" src="d:\Mantisbt-1.2.8\Script
s\javascript/min/ajax.js"></script>
</head>
<body>
<div align="left"><a href="my_view_page.php"><img border="0" alt="Mantis Bug Tracker" src="d:\Mantisbt-1.2.8\Scripts\images/mantis_l
ogo.gif" /></a></div>
<div align="center"><table class="width50" cellspacing="1"><tr><td class="form-title">APPLICATION ERROR #
401</td></tr><tr><td><p class="center" style="color:red">Database query failed. Error received from database was 0001366: Incorrect st
ring value: '\xE9c. 20...' for column 'note' at row 1 for the query: INSERT INTO mantis_bugnote_text_table ( note ) VALUES ( ? ).</p

</td></tr><tr><td><p class="center">Please use the "Back" button in your web browser to return to the previous page. There you can
correct whatever problems were identified in this error or select another action. You can also click an option from the menu bar to
go directly to a new section.

</td></tr></table></div></body>
</html>
rombert

rombert

2011-12-07 16:13

reporter   ~0030503

  1. Is your MySQL database configured to hold UTF-8 data?

  2. The error is a bit strange, if I am looking up the correct character then the file is incorrectly read ( http://www.codetable.net/hex/e9c ) . That would suggest that the error is in how the data is passed to Mantis
sbs

sbs

2011-12-07 16:34

reporter   ~0030504

I am not fully proficient with MySql but I have found the following charset configuration :
charset_client latin1
charset_connection latin1
charset_database latin1
charset_results latin1
charset_system utf8

collation_... latin1_swedish_ci

we enter daily bugs and notes including accented chars from our web browsers
rombert

rombert

2011-12-07 16:40

reporter   ~0030505

Then I think we're left with option 2, the file is incorrectly passed to Mantis . Or incorrectly read by Mantis. But I don't have any idea how to debug that, as my experience with Windows is very limited.
sbs

sbs

2011-12-07 16:53

reporter   ~0030506

anything I can do to help debugging that?
rombert

rombert

2011-12-08 09:36

reporter   ~0030510

(In reply to comment 0013649:0030506)

anything I can do to help debugging that?

Can you insert the following lines around line 65, after the while loop?

echo 'Extracted message:';
echo '------------------';
echo "\n\n";
echo $t_comment;
sbs

sbs

2011-12-08 09:48

reporter   ~0030511

Here you go:

Extracted message:------------------

SVN Revision 4142
dt
2011-12-06 09:42:46 +0100 (mar., 06 dÚc. 2011)
FIX:0001448
Added XML Schema validation of the incoming config file before loading it to detect malformed files.

the é has been replaced by another accented char.
rombert

rombert

2011-12-08 10:11

reporter   ~0030512

Right. So this means that when reading characters from the console the conversion is incorrect.

There doesn't seem to be a way to detect the character encoding from PHP, so I tried to look for windows console encoding solutions. I'm not sure how you launch the checking process, but this might be of interest to you.

http://stackoverflow.com/questions/1259084/what-encoding-code-page-is-cmd-exe-using

You might want to try setting the console encoding to Latin-1 or UTF-8 for the checkin operation.
sbs

sbs

2011-12-08 10:56

reporter   ~0030517

I had found this page before and tried playing with the code page in my script. It did not bring any help.

I just tried chcp 850 and 1251, the script still does not import the issue.
rombert

rombert

2011-12-08 16:13

reporter   ~0030523

Perhaps chcp 65001 for utf-8?
sbs

sbs

2011-12-08 16:19

reporter   ~0030524

with chcp 65001 the script stops right after selecting the code page
rombert

rombert

2011-12-09 04:54

reporter   ~0030537

Not really sure what to say then. My windows knowledege is limited, so I can't say what steps to take.
sbs

sbs

2012-11-09 03:31

reporter   ~0034275

Hi,

It's been a long time, but I found a fix to my accented issue :)

svn log stream is ANSI encoded, while php expects a UTF-8 encoding. I have thus written a small app that convert ANSI Stream to UTF-8 and now accented commit logs can get into the bugtrack thourgh checkin.
dregad

dregad

2012-11-09 04:04

developer   ~0034276

Thanks for the feedback.
rombert

rombert

2012-11-09 17:21

reporter   ~0034279

Moving back status change from checkin, just documented the UTF-8 requirement.

Related Changesets

MantisBT: master-1.2.x 1d3b0649

2012-11-09 16:59

rombert


Details Diff		 Document UTF-8 requirement for scripts/checkin.php

Fixes 0013649: Check-In fails on logs with accented chars 		Affected Issues
0013649
mod - docbook/administration_guide/en/configuration.sgml Diff File