Synchronization between different Mantis Instances

Author: Martin Fuchs

There has been a recent discussion on the mailing list about synchronization between different mantis instances and “bug replication”. There seem exist already a few home brewn tools to full fill the main requirements, but there is still missing a generalized solution. I have also written some utilities for our special environment, but they are implemented in C++ and Java, as I am not a PHP programmer. They are also using a Web service, but are independent from MantisConnect. This are the reasons why I guess you are not willing to take my code as starting point. So let's start again with the design phase and write down the requirements for a synchronization solution between different Mantis instances in this Wiki page. In the following descriptions there are some links to my Mantis test installation, which can be used as a reference implementation to see a living example of two connected instances with reporting functionality). Here is a screenshot of the client application I am using to manually copy the entries of one instance to another.

• A common setup is the installation of one Mantis instance (let's call it "extern") with customer access and another instance with access for developers also containing some entries hidden from the customer (let's call it "intern"), may be also containing notes generated by CVS/SVN commits. The entries of this two Mantis instances should be synchronized. The relationship between the two instances is a master/slave relation: The external instance can be used to enter new issues from the outer side. This should become immediately visible in the internal instance. But the master for the entry status is located in the internal instance. (May it would be useful to be able to configure a master detail chain with more than two instances.)

• It should be possible to host the two instances on two different machines to separate internal and external networks.

• The configuration of replication features should be on a per project base, so it's possible for example to activate synchronization only for part of the existing projects.

• There should be the possibility to configure a mapping for different attributes between the two instances like category, assignees and may be status.

• Configuration should be available via web forms for users with administrator access.

• There should be an user interface to manually synchronize entries in both directions.

• When duplicating entries from the external to the internal instance, it is sufficient to insert HTML links to attached files instead of copying the whole data.

• The following things should be synchronized automatically when altering an entry:

 . new entries from slave to master
 . newly attached notes from slave to master
 . status changes from master to slave

• The connection between entries of the two instances can be implemented by using user defined fields: (This is our current implementation without any need to change the Mantis code. Of course it would be possible for example only to store the IDs of the other instance in any of the two instances and generate the URLs on the fly in the Mantis PHP code.)

 . In the external instance there are stored the IDs of internal entries.
 . In the internal instance there are stored the URLs to immediately jump to the external entries.

• To be able to divide an external entry into several internal subtasks (maybe with different assignees), it should be possible to associate a slave entry with more than one entry in the master instance. When replicating the status from several master entries to the slave instance, the resulting status is calculated as the minimum value of all associated master status values.

• There should be an overview page (Report) in the master instance showing a cross reference with the list of related entries of both instances side by side:

. Following Attributes should be displayed on both sides: ID, Status, Severity, Summary. Other columns to display could be configured optionally.
. The list items should contain HTML links to go directly to the related Mantis entries.
. Filtering functionality at least for the status is very useful in this list, for example to get a quick overview of open issues.

• As Mantis is a PHP application, I think it would be best to implement the synchronisation features also using the same technique. Posting change messages from one instance to another should be possible by issuing HTML requests using a standard protocol like XML-RPC or SOAP. May be it's even possible to use exactly the same protocol like Bugzilla to directly communicate with this system.

• To store the configuration data of external systems there could be created a new table 'mantis_system' with the following columns in the database:

 . system_id - integer
 . name - string
 . url - string
 . username - string
 . password - string

• new columns in table 'mantis_project_table' to store the associated external systems of a project:

 . master_system_id - integer
 . slave_system_id - integer

• new columns in table 'mantis_bug_table' to associate the bug entries with external entries:

 . master_system_id - integer
 . slave_system_id - integer
 . master_bug_ids - string (list of associated master bug IDs separated by spaces)
 . slave_bug_id - integer

• There are needed one or more new tables to store the mapping definitions for automatically inserted new entries.

• more details…

Please add your comments and feedback in this section.

• The rationale for the external system is based on IT security issues. With internal and external issues in the same tracker, there is some possibility that external people might see these issues. [Glenn Henshaw]

• Perhaps this is beyond the scope of the immediate goals, but imagine an interface which any bug tracker could implement and allowed replication or query between any tool which implemented it. The Eclipse Mylar project goes part of the way toward this goal, but by creating custom Mylar connectors per bug tracker. If a standard set of interfaces for getTask(id)→task, createTask (task), etc. were created, along with standard definitions of the structure of a task and its relationships, then this would be certainly possible. Interactions could be not just between task trackers, but also between CRM or ERP systems and task trackers. Imagine being able to create a simple dashboard widget (for those OSX users out there) which allowed creating new tasks against a range of task trackers. Or completely abstracting the front end (GUI) from the database and business logic. Mantis attempts this with the API methods, but there isn't always a good separation. From my research, most bug trackers are pretty similar in terms of database structure. Sure, it is more work, but the potential benefits of creating an open generic API are huge. Several transports are possible, but possibly SOAP is one of the easiest and most adaptable. I wonder if developers from other projects would be interested in collaborating. [Aristedes Maniatis]

• I think it's important to note that this feature should be part of Mantis, so we should try to solve this with PHP. It Should be VERY interesting to approach this issue looking for a mechanism capable to replicate bugs to other bug trackers (trac ??), so we can expand this idea (in long term) to others bug trackers. A mechanism to verify bugs replication should be developed. [Matias Torchinsky]