Can subject be rewritten via regexp?

This plugin allows you to report an issue in MantisBT by sending an email to a particular mail account

Moderators: Contributor, Developer

mushu
Posts: 113
Joined: Jan 04, 2017 12:41 pm

Can subject be rewritten via regexp?

Post by mushu » May 15, 2018 12:17 pm

Bear with me please...we the developers have revolted against management, and implemented Mantis as our bug tracking system. Management has the helpdesk/network folks using some cr@ppy expen$ive system from Remedyforce. Did I say it was cr@ppy? Well, Mantis is WAY better in every respect. Anyway, the concession was that we make it "talk" to Remedyforce. Ugh. So I set up the EmailReporting plugin and have Remedyforce send an email to our special SMTP account that Mantis monitors every time a new ticket is placed in the Developer queue in Remedyforce.

It works great. So now my question: I'm not a php coder (but can probably figure it out), which file would be best for me to modify in the EmailTReporting plugin, in order to do a regexp search of the incoming email "Subject" line looking for specific text, and REPLACE that text with something slightly different, before it gets processed by Mantis which sends we the developers a "new incident" notification email? An example:

Subject line of email that Mantis finally throws to us after processing a forwarded email from Remedyforce:

Code: Select all

[Emailed_Tickets 0000610]: Incident #00028327 Is now Assigned to Developers Queue
And I need to add code that looks for Incident # followed by a long number, and REPLACE it in the subject line of that message with (REF:IN:nnnnnnnnnn) where the n's are the same number, everything surrounded with parentheses. So for the above example, I need it to end up in the Subject line, PRIOR TO BEING PROCESSED BY MANTIS to look like this:

Code: Select all

[Emailed_Tickets 0000610]: (REF:IN:00028327) Is now Assigned to Developers Queue
Make sense? Which file in the EmailReporting plugin would best allow me to accomplish this? I don't mind getting dirty and learning php, etc, I just would like a little head start help.
Thanks!

mushu
Posts: 113
Joined: Jan 04, 2017 12:41 pm

Re: Can subject be rewritten via regexp?

Post by mushu » May 15, 2018 12:39 pm

Huh. Just found this item, will read it closely and see if it helps me: https://mantisbt.org/forums/viewtopic.p ... 317#p64498

SL-Gundam
Posts: 533
Joined: Jul 06, 2011 9:17 am

Re: Can subject be rewritten via regexp?

Post by SL-Gundam » May 15, 2018 2:20 pm

I take it that that subject line is only important for the initial ticket creation. Emails being turned into notes do not require subject modifications?

I suggest creating a new plugin that depends on EmailReporting and MantisBT
That plugin should listen for this event: https://github.com/mantisbt-plugins/Ema ... i.php#L964 (before the creation of a new ticket) or for
https://github.com/mantisbt-plugins/Ema ... .php#L1059 (after the creation a new ticket)

EVENT_ERP_REPORT_BUG_DATA: Functions the same as EVENT_REPORT_BUG_DATA but only triggers for EmailReporting created issues - http://mantisbt.org/docs/master/en-US/D ... ortbugdata
EVENT_REPORT_BUG: triggers for normal created issues and EmailReporting created issues - http://mantisbt.org/docs/master/en-US/D ... .reportbug

When that event is triggered, your plugin should modify the summary of the issue

I will post here again when i've got a valid usefull regex

SL-Gundam
Posts: 533
Joined: Jul 06, 2011 9:17 am

Re: Can subject be rewritten via regexp?

Post by SL-Gundam » May 15, 2018 2:32 pm

Used regex101.com

https://regex101.com/r/egyeDn/1

Code: Select all

$re = '/^\w+\s#(\d+)/u';
$str = 'Incident #00028327 Is now Assigned to Developers Queue';
$subst = '(REF:IN:\\1)';

$result = preg_replace($re, $subst, $str, 1);

echo "The result of the substitution is ".$result;
The advantage of creating your own plugin for this is that you can easily update MantisBT and EmailReporting without worrying about personal modifications. If you ever decide to use SOAP instead of EmailReporting for the exchange of information. You can easily integrate your plugin into that as well making that migration easier if you ever attempt it

The reason for the advise about not connecting two ticket systems is because the two could get stuck in a loop where they update each other about created notes

mushu
Posts: 113
Joined: Jan 04, 2017 12:41 pm

Re: Can subject be rewritten via regexp?

Post by mushu » May 15, 2018 3:39 pm

Here's what I've cobbled together. I decided to just make code changes in the two files of the plugin, and document them, and do a code diff whenever I need to update the EmailReporting plugin to add my changes back into the new versions.

In mail_api.php I added this if() block in the parse_content(&$p_msg) function:

Code: Select all

	#...
	$t_email[ 'Subject' ] = trim( $t_mp->subject() );
	if ( preg_match( '/\@salesforce.com/i', trim( $t_mp->sender() ) ) ) # 05/15/2018 [XYZ]
	{
	  $t_email[ 'Subject' ] = preg_replace( '/Service Request \#(\d+)/i', '\(Ref:IN:\\1)', trim( $t_mp->subject() ) );
	  $t_email[ 'Subject' ] = preg_replace( '/Incident \#(\d+)/i', '(Ref:IN:\\1)', trim( $t_mp->subject() ) );
	}
	$t_email[ 'To' ] = $this->get_emailaddr_from_string( $t_mp->to() );
	#...
And in Parser.php I added several little things all over (so these are all spread out accordingly) but commented them for easy location later:

Code: Select all

	#...
	private $_sender; # 05/15/2018 [XYZ]
#...
	public function sender() # 05/15/2018 [XYZ]
	{
		return( $this->_sender );
	}
#...
		if ( isset( $structure->headers[ 'sender' ] ) ) # 05/15/2018 [XYZ]
		{
			$this->setSender( $structure->headers[ 'sender' ] );
		}
#...
	private function setSender( $sender ) # 05/15/2018 [XYZ]
	{
		$this->_sender = $this->process_header_encoding( $sender );
	}
	#...
The additions I made to Parser.php were to expose the Sender header in the message since that contains the identifier that Remedyforce uses so I could identify only email messages that were sent from the other ticketing system. Ugh. Everything seems to work so far, perhaps some kind soul can tell me if I broke something I'm not aware of, or perhaps made something less secure, etc...?

SL-Gundam
Posts: 533
Joined: Jul 06, 2011 9:17 am

Re: Can subject be rewritten via regexp?

Post by SL-Gundam » May 15, 2018 3:52 pm

Looks good i think

Code: Select all

$t_email[ 'Subject' ] = preg_replace( '/Service Request \#(\d+)/i', '\(Ref:IN:\\1)', trim( $t_mp->subject() ) );
$t_email[ 'Subject' ] = preg_replace( '/Incident \#(\d+)/i', '(Ref:IN:\\1)', trim( $t_mp->subject() ) );
These 2 lines could be optimized into one line with the regex i provided earlier and a slight modification to accept a space... otherwise looks good

Code: Select all

^[\w\s]+\s#(\d+)
Currently there is no way to have done this without EmailReporting code adjustments. Even if the sender header would have been available in parser.php, it would not have been available in the event triggered by EmailReporting.
So your choice for how to code this is good

mushu
Posts: 113
Joined: Jan 04, 2017 12:41 pm

Re: Can subject be rewritten via regexp?

Post by mushu » May 15, 2018 5:22 pm

Thanks for that.

One more question: where do I modify code in order to change the From: and Reply-to: headers in the incoming email? I want to be able to change those so when i hit Reply in the Mantis-generated email it will go to somewhere other than the /dev/nul I've configured as the global default reply-to address.

SL-Gundam
Posts: 533
Joined: Jul 06, 2011 9:17 am

Re: Can subject be rewritten via regexp?

Post by SL-Gundam » May 15, 2018 6:39 pm

In this case you are talking about the notification emails MantisBT generates. Is that correct?

Why not just change the /dev/nul ? i expect you have other requirements for this change? Please explain those so that i can give you a proper answer

I suggest you read this thread: https://mantisbt.org/forums/viewtopic.php?f=13&t=25410
It deals with modifications to that value and problems with certain MantisBT configurations

mushu
Posts: 113
Joined: Jan 04, 2017 12:41 pm

Re: Can subject be rewritten via regexp?

Post by mushu » May 16, 2018 10:49 am

Ok. I set up a special mailbox "DevBugs" and configured EmailReporting plugin to handle the mail. It works great. Then the nasty remedyforce ("RF" from now on) thing happened, and we devs had to interface both ticketing systems. Yes, I know about the endless-loop thing...I'm sure I can handle that. Then I did the above stuff to allow RF notices to be forwarded to Mantis via email to create new tickets, since we hate RF and don't want to even log in to it. All of this works.

The final piece is that when we close an issue in Mantis, Mantis correctly sends an email notification back to the "Reply-to" (or is it "From"?) but our global variables $g_from_email and $g_return_path_email are both set to blackhole destinations. I'm reevaluating this logic, not sure why we set it up that way because it would prevent customers from simply hitting "Reply" to interact with us in a support session. Hmmm.

So I'm going to put our special mailbox in both of those variables in config. but that brings me to the point: I need to change the "Reply-to" on the email notice that Mantis generates to us letting us know a new ticket was created and received from RF, but only for RF-generated notices. Sometimes our customers send email directly to DevBugs andMantis correctly processes it and creates a new ticket, etc, etc. The "Reply-to" needs to be changed to the RF ticketing system return email address so that all responses that get generated for that specific ticket are sent back to RF instead of our own special mailbox. Did that make sense?

I was hoping something like this added to the mail_api.php code would work, in the same place I changed the "Subject" line above, but it doesn't seem to be doing anything so perhaps I have the syntax wrong or I need to escape special characters:

Code: Select all

	if ( preg_match( '/\@salesforce.com/i', trim( $t_mp->sender() ) ) ) # 05/15/2018 [XYZ]
	{
		$t_email[ 'Subject' ] = preg_replace( '/Service Request \#(\d+)/i', '\(Ref:IN:\\1)', trim( $t_mp->subject() ) );
		$t_email[ 'Subject' ] = preg_replace( '/Incident \#(\d+)/i', '(Ref:IN:\\1)', trim( $t_mp->subject() ) );
	# $t_email[ 'Reply-to' ] = 'MantisBugTracker <RFmailbox@ourdomain.com>';
	}
TL;DR: we want to receive the notice that a ticket was created in RF and have Mantis create a new ticket for us itself (and email us notification that it has done so.) We want to do ALL work on that ticket in Mantis only. When we finally close that ticket in Mantis, we want Mantis to send an email notice back to RF so that they can close that (empty) ticket. :-) Think that's doable?

SL-Gundam
Posts: 533
Joined: Jul 06, 2011 9:17 am

Re: Can subject be rewritten via regexp?

Post by SL-Gundam » May 16, 2018 11:21 am

The part you are talking about now is outside of the scope of the EmailReporting project so i'm unsure how far we'll get but we'll try

EmailReporting connects the user account based on the "from" address
MantisBT actually sends notification emails to the email address connected to the user account (which would be the from address yes of the original email that created the ticket).

$g_from_email and $g_return_path_email is the address from which MantisBT will send notification emails. /dev/nul/ in this value will do very little except make it impossible for people to respond to them and make it likely that the emails being sent will be trapped in spam filters
If you don't want people to respond to them it would be better to use something like noreply@server.com

MantisBT can handle different notification settings per project
I advise you modify your code so that emails from RF are imported into project A, while emails sent directly are imported into project B. See below for a code example

Code: Select all

if ( preg_match( '/\@salesforce.com/i', trim( $t_mp->sender() ) ) ) # 05/15/2018 [XYZ]
{
    $this->_mailbox[ 'project_id' ] = 1; // or whatever the ID is for Project A
}
else
{
    $this->_mailbox[ 'project_id' ] = 2; // or whatever the ID is for Project B
}
You can then set up different notification settings in MantisBT per project and you can also set up a different from/reply to address per project
The from/reply to address per project will only work if you make the necessary modifications since MantisBT officially does not support this
See here: https://mantisbt.org/forums/viewtopic.p ... 921#p64921

But all of this will not send an email back to RF. The only thing that will happen is that MantisBT will send a notification email to Person A who created the ticket and send the email with the from address for RF. RF will not see this or notice this

I once had a similar use-case with a custom application developer for the company i worked at. I created a plugin which would add a button to a ticket. When pressed it would create an email which i could send to the developer to escalate the issue if i couldn't solve it myself
In your case you i advise you create a plugin that would listen to this event (possibly other as well): http://mantisbt.org/docs/master/en-US/D ... .updatebug
your plugin should check whether the new status of the bug was changed and whether the new status is "closed"
If that is true then your plugin should send an email to your RF address with the notification that the ticket was resolved

mushu
Posts: 113
Joined: Jan 04, 2017 12:41 pm

Re: Can subject be rewritten via regexp?

Post by mushu » May 17, 2018 12:47 pm

I would prefer to not go through the expense of writing a complete plugin for this little change. Adding a new mailbox costs us money every month (even just a little bit) and RF would have to be changed by coding new templates or workflows or whatever they call their process workers.

If possible, can I find out how to change the "Return-Path" header in the EmailReporting plugin please? Per the above, I'm trying to do this, but it is not working. Can you tell me why? Are the email headers getting written later on in the process?

Code: Select all

private function parse_content( &$p_msg )
#...
   if ( preg_match( '/\@salesforce.com/i', trim( $t_mp->sender() ) ) ) # 05/15/2018 [XYZ]
   {
	$t_email[ 'Subject' ] = preg_replace( '/Incident \#(\d+)/i', '(Ref:IN:\\1)', trim( $t_mp->subject() ) );
	$t_email[ 'Return-Path' ] = 'helpdesk@ourdomain.com';
   }
#...
In the Parser.php and mail_api.php files I've made the appropriate changes to expose the "Return-Path" header, same as I did the "sender" header above. As you know, sometimes not every header is put in the message depending on the mail server. So perhaps I can add a new header to the message, such as "Reply-To" or "X-Reply-To" instead? Would that be easier?

SL-Gundam
Posts: 533
Joined: Jul 06, 2011 9:17 am

Re: Can subject be rewritten via regexp?

Post by SL-Gundam » May 17, 2018 2:27 pm

There is no need to create another mailbox, the mailbox configuration in EmailReporting just contains the project where the tickets are imported. That small piece of code allows you to change the selected project based on certain conditions of your choosing

You need to look at MantisBT notification emails as completely new emails. They are not replies to the emails EmailReporting is importing
So the "Return-Path" is just a header. It does not do anything per se for MantisBT or EmailReporting. If the receiving party replies to the email, then that header comes into play. But like i said, MantisBT does not reply to emails

If MantisBT sends a notification email it will be to the email address connected to the reporting user account in MantisBT
You can make it so that EmailReporting will use the "Return-Path" address instead of the "From" address if you so wish but thats about the extend of it.
You could also modify EmailReporting so that every incoming email that matches below line gets a monitoring user called RF. Modify the MantisBT notification settings and the RF account so that the end result is that the RF account receives an email about every closed issue

Code: Select all

if ( preg_match( '/\@salesforce.com/i', trim( $t_mp->sender() ) ) ) # 05/15/2018 [XYZ]
If i understand you correctly you are trying to get MantisBT to send an email to RF when an issue is closed. "Reply-To", "X-Reply-To", "sender" and "Return-Path" do not decide where an email is going, just where a reply will be directed to.
Does any of them contain the email address of RF?
Is the RF email address always the same or is it different on a per issue basis?

mushu
Posts: 113
Joined: Jan 04, 2017 12:41 pm

Re: Can subject be rewritten via regexp?

Post by mushu » May 17, 2018 2:47 pm

Yes, I understand that Mantis itself generates a new email message when it sends notifications on certain events. I also understand that the EmailReporting plugin processes ALL incoming emails to a specific email address, which we are using for purposes of "talking" to Mantis tickets. That is our internal "DevBugs" email account.

What I have done above is change the "Subject" line of ALL incoming emails to the DevBugs account that are from the RF ticketing system only. I did that because the original incoming email Subject line is saved in Mantis as the "Summary" data, and is reused as the Subject line on all outgoing emails that Mantis generates. This works well, because it allows the outgoing notifications to be stored in the other RF ticket system as notes to their existing ticket instead of creating a new ticket because they are doing the same thing we are with a different email account ("helpdesk@ourdomain.com")

The final problem I'm trying to solve is how to send a flag to that RF system so that when we close the ticket in Mantis, and it generates the email notice, that notice goes back to the RF system and has it close the original ticket in their system also. When *their* ticket gets closed, it will generate an email to the *original* customer letting them know the incident is completed. That needs to be done because as far as Mantis is concerned, its "customer" was the other ticketing system!

Now, when Mantis generates a new email on activity, it sends it to the "From" or "Reply-to" or "Return-Path" address (I'm not sure which one.) I want to change the correct one of these on the original incoming email so that when WE get the Mantis email notice in Outlook, all WE have to do is hit the [reply] button and our response will go to the correct location (which is the other ticket system.) Right now, the [reply] button sends our email response back to Mantis, because the Mantis config global variable contains the DevBugs email address...so we have to remember to change that in our response email "To" line before sending the message. Remember, we only want to do this crazy stuff for tickets received from the other RF system.

Whew! You might have to read that a few times before it makes sense lol.

SL-Gundam
Posts: 533
Joined: Jul 06, 2011 9:17 am

Re: Can subject be rewritten via regexp?

Post by SL-Gundam » May 17, 2018 4:23 pm

Am i right to assume that for MantisBT the reporting user is "other ticketing system"?

Currently EmailReporting creates the user account based on "From". So MantisBT will send the email to the original from address that created the ticket.

But now i understand it a little better. You do not want MantisBT to send the email that the ticket is closed. MantisBT sends it to you and you want to be able to easily forward it to the right address. Why not let MantisBT handle that? Why put yourself in between?

Does MantisBT never send an email to the original customer?
is the original customer even registered in MantisBT?
Do you need the original customer to be registered in MantisBT?

So to get back to my other questions
Does any of these, "Reply-To", "X-Reply-To", "sender" and "Return-Path", contain the email address of RF? or another field?
Is the RF email address always the same or is it different on a per issue basis?

I still advise you to separate the normal tickets from the RF tickets using this

Code: Select all

if ( preg_match( '/\@salesforce.com/i', trim( $t_mp->sender() ) ) ) # 05/15/2018 [XYZ]
{
    $this->_mailbox[ 'project_id' ] = 1; // or whatever the ID is for Project A
}
else
{
    $this->_mailbox[ 'project_id' ] = 2; // or whatever the ID is for Project B
}
It does not require an extra mailbox. It actually only uses one mailbox. You can add these lines with the same code you already added in parse_content(&$p_msg)

mushu
Posts: 113
Joined: Jan 04, 2017 12:41 pm

Re: Can subject be rewritten via regexp?

Post by mushu » May 18, 2018 10:57 am

Yes, for Mantis receiving ticket emails from the other system. We have configured Mantis to put ALL emailed tickets in the same category "[emailed_tickets]" because we don't know which Project any of the incoming emails will apply to, and one of us will go in and assign it to the correct category and developer to handle it. We all get notification when a new incoming emailed ticket gets processed by Mantis.

What happens if the From line says this:

Code: Select all

From: weirdname@salesforce.com On Behalf Of servicedesk@ourdomain.com
Because I'm seeing that the other system actually sends all emails outside of our network and then the salesforce.com mailer sends the message back to us. Don't know why it is set up that way, very insecure (what if passwords are in the ticket,) just another reason why Mantis is better. EmailReporting plugin will need to use the "on behalf of" address in this case, perhaps it already does?

I see what some of the confusion is. We use LDAP for authentication, and so anyone on the network that emails a ticket to our DevBugs email account will have Mantis create a new account automatically when it creates the new ticket in its database. So every time the other ticketing system (RF) sends an email to the DevBugs account, I *believe* that mantis creates a new ticket (and associated account) because the From line seems to have a number appended to it from the other system. I will need to look in the database at the User table and find out if it is creating a new user on every email...I sure hope not.

I thought that once mantis has a user account tied to a ticket then it will use that user account's email address...? Since we are using LDAP, Mantis is able to grab the email address automatically when it authenticates the user and will use that. Will that weird From address line above cause any issues?

To be clear, we DO want Mantis to send emails the way it does currently, including Close notices. Those Close notices should go back to the other ticketing system so that the helpdesk will know to close *their* ticket (because we did all the work through Mantis and closed *our* ticket for that issue.) The point is, we don't want to have to ever look at the other system, but we need to be notified when *they* receive a customer ticket for something *we* have to deal with. (Yes, company drama is not very fun...)

The problem is that Mantis will (correctly) use the other ticket system as the email address it communicates with. While *we* are dealing with the issue, we might need to contact the actual customer for more information or ask them to do something to troubleshoot. If we just send an email notice from mantis it will go to the other ticketing system and get added as a Note because that is the Reply-to address (since that is the original From address.) So when a new ticket comes in, if I can grab the customer username and put that in the Reply-to field, then all of our emails from Mantis will go to the *actual* customer. In fact, best would be to make the real customer a Monitor in Mantis or even a CC: field in the message header.

I'm sorry for the wall of text, this is a complex situation and I'm definitely not an expert in Mantis. :-(

Post Reply