View Issue Details

WikiRelated ChangesetsJump to Notes
IDProjectCategoryView StatusDate SubmittedLast Update
0011868mantisbtapi soappublic2010-04-29 17:052010-07-29 10:45
ReporterWepl Assigned Torombert  
PrioritynormalSeveritymajorReproducibilityalways
Status closedResolutionfixed 
Product Version1.2.0 
Target Version1.2.2Fixed in Version1.2.2 
Summary0011868: returned compressed data corrupt
Description

While updating from 1.2.0a3 to 1.2.0 while using the same webserver and same client (perl SOAP::Lite) a call to mc_project_get_users failed always.

After a lot of debugging and browsing I discovered that it seems the SOAP message send back from mantisbt is corrupt. It cannot be decompressed on the client (zcat: out.gz: unexpected end of file).
The header of the file looks like as compressed two times.

with 1.2.0a3 (working):
000000 1f8b0800 00000000 0003b5bd db6ee438 ba2ef82a .............n.8...*
000014 9ac260ee 52e538da aeddabb7 9cb6f3e0 43dadb76 ..`.R.8.........C..v
000028 55f5aac1 2040498c 102d8a54 93528423 1e6580bd U... @I..-.T.R.#.e..

with 1.2.0 (failing, looks as there are two gzip headers):
000000 1f8b0800 00000000 000300d9 4126be1f 8b080000 ............A&......
000014 00000000 03b5bddb 6ee438ba 2ef82a9a c260ee52 ........n.8...*..`.R
000028 e538daae ddabb79c b6f3e043 dadb7655 f5aac120 .8.........C..vU...

as a work around I have disabled compression in nusoap:

--- mantisbt-1.2.0/library/nusoap/nusoap.php 2010-02-24 17:43:29.000000000 +0100
+++ mantisbt-1.2.0-wepl/library/nusoap/nusoap.php 2010-04-29 22:24:03.000000000 +0200
@@ -4142,7 +4142,7 @@
//begin code to compress payload - by John
// NOTE: there is no way to know whether the Web server will also compress
// this data.

  • if (strlen($payload) > 1024 && isset($this->headers) && isset($this->headers['accept-encoding'])) {
  • if (false && strlen($payload) > 1024 && isset($this->headers) && isset($this->headers['accept-encoding'])) {
    if (strstr($this->headers['accept-encoding'], 'gzip')) {
    if (function_exists('gzencode')) {
    if (isset($this->debug_flag) && $this->debug_flag) {

with this change it works as with 1.2.0

TagsNo tags attached.
Attached Files
repimpx1.pl (2,176 bytes)    
#!/usr/bin/perl -w

###########################################################################
# imports
###########################################################################
use Data::Dumper;
use Date::Parse;
use Encode;
use SOAP::Lite;
#use SOAP::Lite +trace;
#print "used SOAP::Lite version is $SOAP::Lite::VERSION\n";
use strict;

###########################################################################
# define constants and globals
###########################################################################
use constant {
	# soap api namespace
	SOAP_URI	=> 'http://futureware.biz/mantisconnect',
	# soap api endpoint
	SOAP_PROXY	=> 'http://m.whdload.de/api/soap/mantisconnect.php',

	# soap api login details (must have admin access)
	SOAP_USER	=> 'administrator',
	SOAP_PASS	=> 'xxx',

	# mantis constants
	ALL_PROJECTS		=> 0,
	ACCESS_ANYBODY		=> 0,
	ACCESS_REPORTER		=> 25,
	ACCESS_DEVELOPER	=> 55,
	ACCESS_MANAGER		=> 70,

};

# access the mantis api
my $mantis = SOAP::Lite -> uri(SOAP_URI) -> proxy(SOAP_PROXY, options => {compress_threshold => 10000});
$mantis or die "could not construct soap:$!";

# get list of all registered users from mantis
print "  - getting mantis users\n";
{
	my ($response,$user,$acc,$nam,$ema,$usr);
	$response = $mantis -> mc_project_get_users(SOAP_USER, SOAP_PASS, ALL_PROJECTS, ACCESS_ANYBODY);
	$response or die "soap: mc_project_get_users failed:$!";
	#&WriteFile('out.gz',$mantis->{'_transport'}->{'_proxy'}->{'_http_response'}->{'_content'});
	#use Compress::Zlib;
	#$_ = Compress::Zlib::memGunzip($mantis->{'_transport'}->{'_proxy'}->{'_http_response'}->{'_content'}) or die "memGunzip:$gzerrno";
	#s/>/>\n/g;
	#die $_;
	#delete $mantis->{'_transport'}->{'_proxy'}->{'_http_response'}->{'_content'};
	#delete $mantis->{'_deserializer'}->{'_ids'};
	#delete $mantis->{'_deserializer'}->{'_parser'}->{'_done'};
	#die "mc_project_get_users failed:$!, dumping \$mantis\n" . &Dumper($mantis);
	$response->fault and die "soap: mc_project_get_users: (".$response->faultcode.") ".$response->faultstring;
	foreach $user (@{$response->result}) {
		# returns: 'email' 'real_name' 'name' 'id'
		#print Dumper($user); exit;
	}
}
repimpx1.pl (2,176 bytes)   

Activities

rombert

rombert

2010-05-02 13:22

reporter   ~0025363

Please try a recent 1.2.x build from git master-1.2.x , as I have recently updated the NuSOAP library to the latest released version ( http://git.mantisbt.org/?p=mantisbt.git;a=commit;h=c6cc770cc0fe5ff264e424a70b95764ada4186de ) and it might contain a fix for it.
Wepl

Wepl

2010-05-04 15:12

reporter   ~0025394

I tried mantisbt-1.2.1-2010-05-04-master-1.2.x-a56df81.zip which behaves the same.
Maybe something in other parts of mantisbt have changed causing the webserver to compress the message on delivery to the client?
rombert

rombert

2010-05-08 04:05

reporter   ~0025441

Will you be able to run a git bisect to find out the faulty commit? This way I can understand the root cause and supply a proper fix.

http://book.git-scm.com/5_finding_issues_-_git_bisect.html
Wepl

Wepl

2010-05-12 11:58

reporter   ~0025487

dcf98109db64e673c8ce2f4e2ca7b53d52495aaa is first bad commit
commit dcf98109db64e673c8ce2f4e2ca7b53d52495aaa
Date: Mon Jul 6 19:31:08 2009 +0100

Alter the way we handle compression within mantis:

This only applies zlib is enabled

if zlib.output_compression is not enabled in php.ini we use ob_gzhandler, else we use output_compression

Note: we behave differently if php is >=5.2.10 as we can use zlib.output_compression with ini_set: Note php manual:
"Note: You cannot use both ob_gzhandler() and zlib.output_compression. Also note that using zlib.output_compression is preferred over ob_gzhandler(). "

COMPRESSION_DISABLED can be defined as a constant to disable compression on a specific page if required, before including core.php.

:100644 100644 7531444a0d3703b52b1c6d7f3ff8b8b23fea69bc 5a45339efcb9d0d75af525ee31ff215e14314db0 M core.php
:040000 040000 9a66159ad50335f7fcc9554a3d45826e6042e4cf ccf19df804fe5c112647d59d196da80b4dbf53db M core

maybe the change is causing the general compression to on, so it would be interesting if nuoap compression is working the same time php compression is enabled?
rombert

rombert

2010-05-13 16:44

reporter   ~0025497

This might be cause by double-compression, as you have indicated. Can you please post your compression-related PHP/MantisBT parameters, as well as the way you're making the SOAP requests on the client side?

Perl code is fine - if you're able to share it.

Thanks.
Wepl

Wepl

2010-05-15 15:05

reporter   ~0025504

test perl script attached

php config:

zlib

ZLib Support enabled
Stream Wrapper support compress.zlib://
Stream Filter support zlib.inflate, zlib.deflate
Compiled Version 1.2.3
Linked Version 1.2.3

Directive Local Value Master Value
zlib.output_compression Off Off
zlib.output_compression_level -1 -1
zlib.output_handler no value no value

Environment

Variable Value
SERVER_PORT 80
SERVER_SOFTWARE Apache/1.3 (Unix) mod_ssl/2.8.28 OpenSSL/0.9.8f AuthPG/1.3 FrontPage/5.0.2.2635
GATEWAY_INTERFACE CGI/1.1
SERVER_PROTOCOL HTTP/1.1

I think a possible solution could be to set COMPRESSION_DISABLED in mantisconnect.php
rombert

rombert

2010-05-16 05:19

reporter   ~0025505

Actually it was the zlib compression kicking in - at least in my case. I've disabled both zlib compression and defined COMPRESSION_DISABLED for the SOAP calls. Please verify that it does work for you.
Wepl

Wepl

2010-05-16 10:39

reporter   ~0025509

it works for me, thanks.

Related Changesets

MantisBT: master 6daeb19d

2010-05-16 05:12

rombert


Details Diff		 Only allow NuSOAP to compress SOAP responses

When NuSOAP receives a request which declares that it accepts a compressed (gzip,deflate)
response, it performs its own compression. Although conceptually incorrect, since this
should be handled by the web server, there is no configuration flag to disable it.

Therefore we disable all forms of compression for the SOAP API and allow NuSOAP to perform
the compression itself.

Fixes 0011868: returned compressed data corrupt
(cherry picked from commit fd65426d7c94c3f0d4afda09604d43f27024fcf6)		 Affected Issues
0011868
mod - api/soap/mantisconnect.php Diff File

MantisBT: master-1.2.x 9faf487c

2010-05-16 05:12

rombert


Details Diff		 Only allow NuSOAP to compress SOAP responses

When NuSOAP receives a request which declares that it accepts a compressed (gzip,deflate)
response, it performs its own compression. Although conceptually incorrect, since this
should be handled by the web server, there is no configuration flag to disable it.

Therefore we disable all forms of compression for the SOAP API and allow NuSOAP to perform
the compression itself.

Fixes 0011868: returned compressed data corrupt		 Affected Issues
0011868
mod - api/soap/mantisconnect.php Diff File