View Issue Details

IDProjectCategoryView StatusLast Update
0016070mantisbtemailpublic2018-04-04 13:52
ReportermcalvoAssigned Tovboctor 
PrioritynormalSeverityminorReproducibilityalways
Status closedResolutionfixed 
Product Version1.2.14 
Target Version2.13.0Fixed in Version2.13.0 
Summary0016070: Delay due to Mantis trying sending emails to non existent address
Description

Accidentally some of our users where in the mantis database with an invalid email address.

For each email sent to this users Mantis writes a record to "mantis_email_table" to queue the mails.

We found that the queue maintains all this emails with invalid destinations and the send mail process always tries to resend it again and again.

When the table is about of 1000~1500 records the sendmail process takes a long time to complete so becomes a performance problem when reporting issues, change statuses and others.

Deleting all hanging email records causes Mantis to respond faster and nice again.

TagsNo tags attached.

Relationships

has duplicate 0023935 closedvboctor Email notification stop sending and freeze because mantis_email_table is full 
has duplicate 0021128 closedvboctor E-mail sender should limit the number of delivery retries (opt-in) 
related to 0016960 acknowledged Invalid recipient stops email notifications 
related to 0019475 closedvboctor Administrators should be able to bypass allow_blank_email = OFF 
related to 0020532 closeddregad Mail cron job will stop on an unknown mail adress on Exchange Server 
related to 0024224 closedatrol Stale emails can remain queued for years 

Activities

atrol

atrol

2013-06-19 15:20

developer   ~0037229

Last edited: 2013-06-19 15:23

View 2 revisions

1) Emails should be deleted from queue if they can not be sent for a certain time.
The time should be configurable, maybe 7 days is a good default.

Consider also RFC 2821 Chapter 4.5.4.1 Sending Strategy [1]

Retries continue until the message is transmitted or the sender gives
up; the give-up time generally needs to be at least 4-5 days. The
parameters to the retry algorithm MUST be configurable.

2) Add a new column failed_email_count in mantis_user_table , increment whenever 1) deletes an email

3) A user should get a warning after login if emails to his address have been deleted. (failed_email_count != 0)
Set failed_email_count = 0 if the user changes his address

4) Delete the email address of an user account after a certain amount (default 500) of deleted emails.
This stops adding new entries to the queue if 3) didn't take place.

[1] http://www.ietf.org/rfc/rfc2821.txt

BTW there is a script admin/email_queue.php to check / delete entries from the queue

dregad

dregad

2013-06-19 19:00

developer   ~0037231

Last edited: 2013-06-19 19:00

View 2 revisions

Good thinking.

1) Emails should be deleted from queue if they can not be sent for a certain time.
The time should be configurable, maybe 7 days is a good default.

It may also be an option to notify the Administrator if the queue grows beyond a specific threshold, so he can take corrective action. What do you think ?

3) A user should get a warning after login

How do you see that happening concretely ? Javascript popup ? Currently we don't really have a way of directly and actively interacting with users, we just respond to their actions.

4) Delete the email address of an user account after a certain amount (default 500) of deleted emails.

What do we do about $g_allow_blank_email ?

Also, what about the case where the e-mail is not sent because of system-related issues, i.e. the e-mails are valid but the smtp account / server has a problem ? I don't think we should delete or bump the failed count in such case.

atrol

atrol

2013-06-23 09:00

developer   ~0037252

Last edited: 2013-06-23 09:02

View 3 revisions

It may also be an option to notify the Administrator if the queue grows beyond a specific threshold, so he can take corrective action. What do you think ?

Maybe my approach to have some kind of self healing system is a bit too much.
Having a notification for administrators is certainly enough as a first step.

3) A user should get a warning after login
How do you see that happening concretely ? Javascript popup ? Currently we don't really have a way of directly and actively interacting with users, we just respond to their actions.

We shouldn't rely on JavaScript (at least not in 1.2.x)
We could add some red warning text in function html_login_info

4) Delete the email address of an user account after a certain amount (default 500) of deleted emails.
What do we do about $g_allow_blank_email ?

This option is to prevent users from entering blank email addresses.
I see no problem if the system itself sets it to blank.
But you are right, it's some kind of inconsistent and better to leave it unchanged.
Instead we could check failed_email_count to stop adding new entries to the queue.

Also, what about the case where the e-mail is not sent because of system-related issues, i.e. the e-mails are valid but the smtp account / server has a problem ? I don't think we should delete or bump the failed count in such case.

This would require a more precise return value of function email_send (just true or false atm)

AdamR

AdamR

2016-03-24 14:58

reporter   ~0052858

Apologies for the bump, but I'm wondering if this is on the roadmap for 1.2.x?

I ask because we sometimes get our users marking notifications as junk/spam (even though they've opted for them!), but this can cause the email queue to halt. Gmail and GMX.de are particular culprits of this.

Even though our BTS is internal only (no self registration) some people on the team continuously mark us as junk or block us entirely. Unfortunately this can cause Mantis to want to retry sending the email indefinitely until we manually remove it, which is whenever we realise "oh hey we haven't had any BTS emails for a while...".

atrol

atrol

2016-03-25 17:24

developer   ~0052866

Last edited: 2016-03-26 15:32

View 2 revisions

I'm wondering if this is on the roadmap for 1.2.x
No, there is no target version set, thus it's not on the roadmap see
https://www.mantisbt.org/bugs/roadmap_page.php?version_id=240

I don't expect any change in near future (1.3.x)

You might consider to write a cron job which does some cleanup.

AdamR

AdamR

2016-03-26 08:06

reporter   ~0052868

OK, I'll do that. Thanks for the clear answer. :)

vboctor

vboctor

2018-02-06 23:52

manager   ~0058789

@dregad @atrol I suggest going with the following option:

  1. We don't generate emails for users with empty email address - I suspect this is already the case, but we should verify.
  2. Have a configurable time frame after which we give up on retrying an email address. This can be configurable in units of days. Default can be 7 days.
  3. The submission rejection is a subset of invalid addresses (e.g. invalid format), but there is always the case where SMTP relay would accept the email then get a bounce later. We will not handle such case.
  4. I suggest not complicating logic by removing user email address or tracking number of retries.

The above should be simple to implement, doesn't require schema changes, and is conservative and should cover the case of temporary smtp or cronjob issues. I'm happy to implement a fix based on the above.

AdamR

AdamR

2018-02-07 12:26

reporter   ~0058791

I've found when developing web applications with Symfony that sending mail is only as good as the MTA doing the job regardless of how well the PHP implementation is. (In my case with Symfony I'm using the SwiftMailer library.) Possibly with multi-threading, though I'm not sure if PHP even supports that in a web environment.

Having Mantis send emails through CRON as per atrol's suggestion significantly improved the performance of Mantis in the web browser. (So significantly. Suddenly pages with 50+ queries were responding in under 500ms.) Turned out we had just 2 users with invalid email addresses slowing the queue, but even so with CRON handling it emails got to valid destinations timely enough. -- Logging the shell output also made it pretty easy to determine which email addresses were slowing the queue, so I could act on this which in turn unclogged the mail queue.

atrol

atrol

2018-02-07 16:20

developer   ~0058792

Last edited: 2018-02-07 16:35

View 2 revisions

after which we give up on retrying an email address

@vboctor what do you mean with it?
Delete from email table based on submitted column?

If so, to allow administrators to fix the root cause, it would be good to have some logging like

 log_event( LOG_EMAIL, 'Deleted notification to ' . $t_email_address . ' from e-mail queue as sending failed since ' . $t_days . ' days' );
atrol

atrol

2018-02-07 16:35

developer   ~0058794

Having Mantis send emails through CRON as per atrol's suggestion

Although it certainly helps, this is not what I suggested,
I suggested to cleanup the email queue via cron.

Sending emails through cron might also no longer be necessary for good performance.
Starting with 1.3, sending should no longer block as we send email using a shutdown function register_shutdown_function( 'email_shutdown_function' );

vboctor

vboctor

2018-03-05 03:02

manager   ~0059079

PR: https://github.com/mantisbt/mantisbt/pull/1311

Related Changesets

MantisBT: master ff9fb8a9

2018-03-05 03:01:31

vboctor

Details Diff
Delete failing emails from queue after N days

Fixes 0016070
Affected Issues
0016070
mod - config_defaults_inc.php Diff File
mod - core/email_api.php Diff File
mod - core/email_queue_api.php Diff File
mod - docbook/Admin_Guide/en-US/config/email.xml Diff File

Issue History

Date Modified Username Field Change
2013-06-18 10:51 mcalvo New Issue
2013-06-19 15:20 atrol Note Added: 0037229
2013-06-19 15:20 atrol Status new => confirmed
2013-06-19 15:23 atrol Note Edited: 0037229 View Revisions
2013-06-19 19:00 dregad Note Added: 0037231
2013-06-19 19:00 dregad Note Edited: 0037231 View Revisions
2013-06-23 09:00 atrol Note Added: 0037252
2013-06-23 09:02 atrol Note Edited: 0037252 View Revisions
2013-06-23 09:02 atrol Note Edited: 0037252 View Revisions
2014-02-15 12:16 atrol Relationship added related to 0016960
2015-03-08 00:07 vboctor Relationship added related to 0019475
2016-01-25 16:15 atrol Relationship added related to 0020532
2016-03-24 14:58 AdamR Note Added: 0052858
2016-03-25 17:24 atrol Note Added: 0052866
2016-03-26 08:06 AdamR Note Added: 0052868
2016-03-26 15:32 atrol Note Edited: 0052866 View Revisions
2018-02-06 23:45 vboctor Relationship added has duplicate 0023935
2018-02-06 23:46 vboctor Relationship added has duplicate 0021128
2018-02-06 23:52 vboctor Note Added: 0058789
2018-02-07 12:26 AdamR Note Added: 0058791
2018-02-07 16:20 atrol Note Added: 0058792
2018-02-07 16:35 atrol Note Added: 0058794
2018-02-07 16:35 atrol Note Edited: 0058792 View Revisions
2018-03-05 02:41 vboctor Assigned To => vboctor
2018-03-05 02:41 vboctor Status confirmed => assigned
2018-03-05 03:02 vboctor Note Added: 0059079
2018-03-12 21:25 vboctor Changeset attached => MantisBT master ff9fb8a9
2018-03-12 21:25 vboctor Status assigned => resolved
2018-03-12 21:25 vboctor Resolution open => fixed
2018-03-12 21:25 vboctor Fixed in Version => 2.13.0
2018-03-12 21:29 vboctor Target Version => 2.13.0
2018-03-31 19:58 vboctor Status resolved => closed
2018-04-04 13:52 atrol Relationship added related to 0024224