I've downloaded the latest version of mantisbt, 1.3.0, and deploy it on centos 5.11, with apache 2.4.23 and php 5.6.23 and openssl-1.0.2h(important).
After config the email by smtp, I found that the notification does not sent after I modified the user's profile, and the same behavior worked correctly under version 1.2.19.
So I inspect the codes, and add the log level for email on, found some error infomations from apache's error log, which content is something like 'MAIL email_api.php:1205 email_send() ERROR: Message could not be sent - SMTP Error: Could not connect to SMTP host.'.
I add 'print $e;' after line 1205 of email_api.php and finally found that,

the lastest version of mantisbt use phpmailer 5.2.15 as email sender, and this version of phpmailer will try to use tls automatically even if SMTPSecure is not set to 'tls'. See line 257, line 1612 and line 1617 for details.
I compiled php with openssl but I do not use ssl to connect my stmp server, but this codes in phpmailer will automatically use tls when 'SMTPAutoTLS' is ture & has openssl extension & did not already using SSL & the server offers STARTTLS.
Because I do not have the cert files of the smtp server, so connection faild.
Currently, I add some clause to control the logic to avoid to use tls. In file email_api.php, line 1126, I add an 'else' branch to disable 'SMTPAutoTLS'. I think it will be better to disable the 'SMTPAutoTLS' when the variable's value is blank, who's name is 'smtp_connection_mode'.The codes is something like:

if( !is_blank( config_get( 'smtp_connection_mode' ) ) ) {
$t_mail->SMTPSecure = config_get( 'smtp_connection_mode' );
} else {
$t_mail->SMTPAutoTLS = false;
}

And it worked now.
Because I am a non-php programmer, so I believe there will be an better solution to solve the issue, and I hope this logic can be amended in future versions.
This is the end.