View Issue Details
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0016070||mantisbt||public||2013-06-18 10:51||2018-04-04 13:52|
|Target Version||2.13.0||Fixed in Version||2.13.0|
|Summary||0016070: Delay due to Mantis trying sending emails to non existent address|
Accidentally some of our users where in the mantis database with an invalid email address.
For each email sent to this users Mantis writes a record to "mantis_email_table" to queue the mails.
We found that the queue maintains all this emails with invalid destinations and the send mail process always tries to resend it again and again.
When the table is about of 1000~1500 records the sendmail process takes a long time to complete so becomes a performance problem when reporting issues, change statuses and others.
Deleting all hanging email records causes Mantis to respond faster and nice again.
|Tags||No tags attached.|
|has duplicate||0023935||closed||vboctor||Email notification stop sending and freeze because mantis_email_table is full|
|has duplicate||0021128||closed||vboctor||E-mail sender should limit the number of delivery retries (opt-in)|
|related to||0016960||acknowledged||Invalid recipient stops email notifications|
|related to||0019475||closed||vboctor||Administrators should be able to bypass allow_blank_email = OFF|
|related to||0020532||closed||dregad||Mail cron job will stop on an unknown mail adress on Exchange Server|
|related to||0024224||closed||atrol||Stale emails can remain queued for years|
1) Emails should be deleted from queue if they can not be sent for a certain time.
Consider also RFC 2821 Chapter 126.96.36.199 Sending Strategy 
Retries continue until the message is transmitted or the sender gives
2) Add a new column failed_email_count in mantis_user_table , increment whenever 1) deletes an email
3) A user should get a warning after login if emails to his address have been deleted. (failed_email_count != 0)
4) Delete the email address of an user account after a certain amount (default 500) of deleted emails.
BTW there is a script admin/email_queue.php to check / delete entries from the queue
It may also be an option to notify the Administrator if the queue grows beyond a specific threshold, so he can take corrective action. What do you think ?
What do we do about $g_allow_blank_email ?
Also, what about the case where the e-mail is not sent because of system-related issues, i.e. the e-mails are valid but the smtp account / server has a problem ? I don't think we should delete or bump the failed count in such case.
Maybe my approach to have some kind of self healing system is a bit too much.
This option is to prevent users from entering blank email addresses.
This would require a more precise return value of function email_send (just true or false atm)
Apologies for the bump, but I'm wondering if this is on the roadmap for 1.2.x?
I ask because we sometimes get our users marking notifications as junk/spam (even though they've opted for them!), but this can cause the email queue to halt. Gmail and GMX.de are particular culprits of this.
Even though our BTS is internal only (no self registration) some people on the team continuously mark us as junk or block us entirely. Unfortunately this can cause Mantis to want to retry sending the email indefinitely until we manually remove it, which is whenever we realise "oh hey we haven't had any BTS emails for a while...".
I don't expect any change in near future (1.3.x)
You might consider to write a cron job which does some cleanup.
OK, I'll do that. Thanks for the clear answer. :)
The above should be simple to implement, doesn't require schema changes, and is conservative and should cover the case of temporary smtp or cronjob issues. I'm happy to implement a fix based on the above.
I've found when developing web applications with Symfony that sending mail is only as good as the MTA doing the job regardless of how well the PHP implementation is. (In my case with Symfony I'm using the SwiftMailer library.) Possibly with multi-threading, though I'm not sure if PHP even supports that in a web environment.
Having Mantis send emails through CRON as per atrol's suggestion significantly improved the performance of Mantis in the web browser. (So significantly. Suddenly pages with 50+ queries were responding in under 500ms.) Turned out we had just 2 users with invalid email addresses slowing the queue, but even so with CRON handling it emails got to valid destinations timely enough. -- Logging the shell output also made it pretty easy to determine which email addresses were slowing the queue, so I could act on this which in turn unclogged the mail queue.
@vboctor what do you mean with it?
If so, to allow administrators to fix the root cause, it would be good to have some logging like
Although it certainly helps, this is not what I suggested,
Sending emails through cron might also no longer be necessary for good performance.
MantisBT: master ff9fb8a9
2018-03-05 03:01:31Details Diff
|Delete failing emails from queue after N days
|mod - config_defaults_inc.php||Diff File|
|mod - core/email_api.php||Diff File|
|mod - core/email_queue_api.php||Diff File|
|mod - docbook/Admin_Guide/en-US/config/email.xml||Diff File|
|2013-06-18 10:51||mcalvo||New Issue|
|2013-06-19 15:20||atrol||Note Added: 0037229|
|2013-06-19 15:20||atrol||Status||new => confirmed|
|2013-06-19 15:23||atrol||Note Edited: 0037229||View Revisions|
|2013-06-19 19:00||dregad||Note Added: 0037231|
|2013-06-19 19:00||dregad||Note Edited: 0037231||View Revisions|
|2013-06-23 09:00||atrol||Note Added: 0037252|
|2013-06-23 09:02||atrol||Note Edited: 0037252||View Revisions|
|2013-06-23 09:02||atrol||Note Edited: 0037252||View Revisions|
|2014-02-15 12:16||atrol||Relationship added||related to 0016960|
|2015-03-08 00:07||vboctor||Relationship added||related to 0019475|
|2016-01-25 16:15||atrol||Relationship added||related to 0020532|
|2016-03-24 14:58||AdamR||Note Added: 0052858|
|2016-03-25 17:24||atrol||Note Added: 0052866|
|2016-03-26 08:06||AdamR||Note Added: 0052868|
|2016-03-26 15:32||atrol||Note Edited: 0052866||View Revisions|
|2018-02-06 23:45||vboctor||Relationship added||has duplicate 0023935|
|2018-02-06 23:46||vboctor||Relationship added||has duplicate 0021128|
|2018-02-06 23:52||vboctor||Note Added: 0058789|
|2018-02-07 12:26||AdamR||Note Added: 0058791|
|2018-02-07 16:20||atrol||Note Added: 0058792|
|2018-02-07 16:35||atrol||Note Added: 0058794|
|2018-02-07 16:35||atrol||Note Edited: 0058792||View Revisions|
|2018-03-05 02:41||vboctor||Assigned To||=> vboctor|
|2018-03-05 02:41||vboctor||Status||confirmed => assigned|
|2018-03-05 03:02||vboctor||Note Added: 0059079|
|2018-03-12 21:25||vboctor||Changeset attached||=> MantisBT master ff9fb8a9|
|2018-03-12 21:25||vboctor||Status||assigned => resolved|
|2018-03-12 21:25||vboctor||Resolution||open => fixed|
|2018-03-12 21:25||vboctor||Fixed in Version||=> 2.13.0|
|2018-03-12 21:29||vboctor||Target Version||=> 2.13.0|
|2018-03-31 19:58||vboctor||Status||resolved => closed|
|2018-04-04 13:52||atrol||Relationship added||related to 0024224|