I had a look at your change via git.leetcode.net. This seems like an incomplete solution due to the following:

1. There is two ways of keeping track of history, standard history + these revisions.
2. Notes are not tracked.

In my opinion, it would be a better solution if we implement a technique where in the history we can store a marker that points to a text blob. For example, the old value and new value would be set as follows:

a. [text id] or
b. [text id] [partial text]

Then in the issue history section we should the partial text with a hyper link to the full text. This would work for summary, description, additional information, steps to reproduce, notes, etc. It would also provide one model for tracking / viewing history.

Point 2 is what Glenn brought up on IRC. The biggest problem with what you and Glenn mention is that is increases the level of complexity by at least an order of magnitude, and I was personally hoping to keep things simple, so that I had to touch as little of the existing codebase as possible to get the feature working...

I think the biggest barrier is not what I track, but how to best integrate the tracking so that it handles both existing bugs that have no previous revisions stored, and new bugs that can have revisions stored from the start.

How would we store extra information in the history log for things being changed? This would require touching a lot of the history API, including devising a way to properly handle history entries with or without the extra revision information.

Also, just pointing out that we already track changes to the summary just as any other field, so I don't really think it necessary to include in this system...

Why not use the existing bug_text_table and add a column(s) to track back to a bug and version? It would save a table. The bug table already points to the current text entry. Only bug_update need change.

The bug_history_table already tracks these updates based on a difference, and the version information (old and new table ids) could be linked there.

Display of the old versions is new in any case.

My opinion is that I consider a bug (or at least a sub-optimal solution) the fact we have three separate fields for description, steps to reproduce and additional info. I can't also see the reason what they are in another table but that's another story...

My other concern is that if I edit a field is to correct something that was wrong. For instance, if I left my password in a copy/paste operation, I expect to be able to remove it, having history in this case is a bug...

@giallu, the problem with no history is in case of the opposite effect of what you mention. Someone changes the contents, either unintentionally or maliciously, to be incorrect or missing critical pieces, and there's no way to go back and see what the real/correct descriptions are.

However, I don't think it would be farfetched to allow someone with admin or manager access being able to remove individual revisions of history...

The fact that we have a bug table and bug text table is from the early days of MantisBT and it was done for optimization purposes. I personally like to merge this table into the main bug table when we get a chance. Hence, we should not design around the existence of such table.

The existence of description, additional information and steps to reproduce is from the early days as well. The advantage of having them (specially steps to reproduce) is that they serve as a reminder for the user submitting the bug to provide such information. An alternative implementation to this is to allow providing a default template for the description field which provides the user with all the information requested for the project.

I can see us making the "additional information" field obsolete in the future, but not allowing it for new issues and only showing it in view/update if it has a value. However, we may require to get some feedback from the community if we wish to do that.

Amen Victor ;)

I completely agree on the template stuff (Bugzilla also have this feature).

If we go toward obsoleting the two additional fields, we could also design a conversion step for the databse, taking care of merging the content into Description. Alternatively, we could consider moving them to comments.

I thing that the optimization is still valid. Ninety percent of the time, we don't need to access the descriptive information for a bug. Having them in the bug_table would imply that the SQL server (using isam) would have to index across the text fields to find new information when the record was accessed. In the current implementation, we only access the record through a simple indexed key when we need it.

Templates would be good, but this is a different bug.

The other issue with a single field is the 64k of logs that I regularly see pasted into the 'additional information' field. These would now go out with every notification email.

Uhm, I don't really know those internals, but if we can get rid of Additional Info and Steps to reproduce, it's already a win...

I opened a bug for templates in 0009911

no one should be allowed to paste a 64K in a text field. That's something that
need to go into an attachment.

OK, I've taken into account most of the suggestions here, and I've adapted my original approach to use a single value column, and an adaptable type column, which allows the system to track Description, Steps to Reproduce, Additional Information, Bugnotes, and potentially any other longtext fields we may have in the future.

Also, I've taken Victor's advice and modified the history log to be able to show links to view revisions for things, and made other various adaptions to the old methods to meet your suggestions.

The only outstanding suggestion is Giallu's request for being able to purge some revisions from the history, but I wanted to get your eyes on this implementation before I went any further forwards...

I've updated my public repo with the new code:

The following changes since commit 054bfde74e22432920f0d3ee5c80f0f208224ee3:

John Reese (1):

Fixed formatting plugin interface.

are available in the git repository at:
git://leetcode.net/mantisbt-jreese.git revisions
John Reese (5):

Base API and schema changes for tracking revisions to bug textarea fields.

Add revision API usage to the core Bug API.

Add user-interface for viewing text field revisions from bug history log.

Schema and revision API changes to support bugnote revisions.

Fully support bugnote revision tracking.
admin/schema.php            |   11 ++

bug_view_revisions_page.php |  124 ++++++++++++++++++++++

config_defaults_inc.php     |    1 +

core/bug_api.php            |   32 +++++-

core/bug_revision_api.php   |  247 +++++++++++++++++++++++++++++++++++++++++++

core/bugnote_api.php        |   16 +++-

core/constant_inc.php       |    7 ++

core/history_api.php        |   35 ++++++-

history_inc.php             |    2 +-

lang/strings_english.txt    |    7 ++

10 files changed, 474 insertions(+), 8 deletions(-)

create mode 100644 bug_view_revisions_page.php

create mode 100644 core/bug_revision_api.php

if you already have a clone of the official repo, you can do:

$ git remote add jreese git://leetcode.net/mantisbt-jreese.git

$ git branch revisions jreese/revisions

From inspection:

1) history_localize_item adds relative links to the changes. I believe that this same text is used in the emails as well.

2) The new revision table seems to supersede parts of the bug_history_table and bug_text_table. Should the original revision be moved from the bug_text_table to the new table?

3) I don't think that you can remove the entries in the bug_history_table. These are used in getting the basic bug history that are referenced in the history display (that is, the history comes from the bug_history_table, and the details from the bug_revision_table). I can't find any other dependencies on these bug_history_table entries.

@thraxisp:

1 - I will need to investigate.

2 - No, the revision table stores less information than the text table (only one field as opposed to three), less than the history table (only the field type, not the action).

3 - I hadn't planned to remove the history entries themselves for Giallu's sugesstion; only the entry in the actual revision table would be purged/altered.

OK, I've committed both a fix for incorrect revision types when creating bugs, and a fix that will fix email histories to not show links to old revisions.

To fetch/track/update your main repository:

$ git remote add jreese git://leetcode.net/mantisbt-jreese.git

$ git fetch jreese

$ git branch revisions jreese/revisions

$ git fetch jreese

$ git checkout revisions

$ git rebase jreese/revisions

As noted in 0005466, I think it's not wise to add a changeset with:
10 files changed, 474 insertions(+), 8 deletions(-)
if the purpose is to workaround the related bug.

On the other side, if the feature is unrelated and we definitely want to retain the older values for those fields, I wonder if we can avoid adding the new table.
In particular, what if we always use an INSERT on bug_text_table fields (instead of an UPDATE) and store in the history_table just the references to the old and new values?

@giallu, note that it's actually 10 files and 474 lines over a series of commits.

However, your idea was brought earlier - however the problem is then a problem of self-referential bug's linking to bug_text's linking to bugs chicken-egg issue (and same for bugnotes). Also, with my current method, others longtext items, such as future custom fields or things added by plugins, could be supported without any changes made to the existing infrastructure.

Lastly, I don't really think this issue is a real solution for 0005466, nor a problem that would be solved just by fixing 0005466; I think they are (mostly) mutually exclusive problem sets.

I've downloaded the code and did some usability testing. I would like to spend some more time to look at the code and further usability testing. But here is my feedback so far:

1. When the description field is modified multiple times, i.e. 3, clicking on one of the "View Revisions" opens the page but it is not clear which one corresponds to the link that I clicked.

2. On the view revisions page, the timestamp used is in different format from the one used in the Issue History and notes on the View Issue page.

3. Deleting an issue note, then click on it's View Revisions link in the history produces an error.

4. From the perspective of the notes history, I find it odd to have the public/private in the main page and the text in the revisions page. I don't necessarily have a better suggestion at the moment.

5. I'm wondering if the "View Revisions" should be really viewing a page with old value -> new value, or as now the full history.

6. Not directly related to this change, but the format of the issue note number used for added/edited is different from the one used in Make Private/Public in the Issue History box.

7. The term revisions may be confusing and give the impression that the feature is relating to source control. I would rather if the table name uses terminology like history_text and also use different terminology in the UI, maybe View Changes. We also need to change the url page. Let's agree on terminology before you apply the changes.

... more to come later.

1 - well, it use #rXX in the URL to auto-scroll to the revision in question, but that only works if the page is long enough to scroll. I don't think there is any "better" method to do this, without passing a redundant GPC param and (somehow) decide on a way to highlight the revision in question...

3 - I'm not quite sure how to handle this. (IMO) we should definitely be keeping revision history of deleted bugnotes, but I don't quite know a good way to handle this. Any suggestions?

4 - I don't understand what you mean, or what you're trying to say...

5 - I think that your idea is far less useful to most users. If they want to see old revisions of a field/bug/bugnote, it makes sense to have it show all and auto-scroll to older portions if need be. No need to frustrate users by making them go back and forth to see multpile revisions.

7 - I'm not quite sure that a change in name is needed. The source control integration plugins use the term 'Changeset' as many VCS systems don't like/use the term revision. I think in this case 'revision' better conveys the concept and meaning behind feature, esp for developers. Eg, bug_change_api.php sounds like it's the API for making changes to bugs, but it's actually the API for tracking changes to certain fields, etc.

Any further updates Victor?

for 3: maybe we can avoid deleting them in the first place but activate a "deleted" flag instead?

Are there any further comments or suggestions? I'd really like to get this pushed in before an a3 release, to help get any potential issues ironed out by testers...

TBH, I still have to see the value of this feature but you know, I wear glasses...

And re-reading last Victor's comment, I think point 4 refers to the private/public status of notes when combined with having history (is the history added to notes also?). I did not actually test the feature (sorry), but if I turn my note from public to private, seeing it in the history is... weird

Anyway, to break any resistance about it, can this be a plug in?

i've still not managed to get a look at this as I can't get a web url for it working. John, i've just tried pushing it to http://git.mantisforge.org/w/mantisbt/bug9856.git but can't see any changes?

Paul

I haven't reviewed any re-work here yet, but I really suggest we don't hold 1.2.0a3 for this.

I just pushed it to the 'revisions' fork on mantisforge.org, fully rebased to the latest master. http://git.mantisforge.org/w/mantisbt/revisions.git?a=shortlog;h=refs/heads/revisions

Regarding a plugin: no, as it is, it's not possible to make this a plugin due to the way in which it modifies history entries (as requested by Victor for better integration).

As for public/private, it currently just relies on the history entries to only show links to history of notes that the user has viewer access to, although it should be rather trivial (I think) to add a bit of functionality to the history viewing page to ensure the user has viewing rights to bug notes when showing note history.

Further improved the feature set using Giallu's feedback:

author  John Reese jreese@leetcode.net

Fri, 9 Jan 2009 19:42:06 +0000 (14:42 -0500)

commit  cbffad375778e71fb73c8f9356c0fc7aa6b1b428

Bug revisions obey private bugnote thresholds, and can be 'dropped' by managers.

Renamed viewing file for better separation of interests.

Added error and confirmation strings.

Still available at http://git.mantisforge.org/w/mantisbt/revisions.git - would really like to get this reviewed and approved as soon as possible, so that I'm not maintaining out-of-branch features for my employer.

Also, as a bit of clarification, bugnote history is not shown inline with other elements from a bug. The only way to see bugnote revisions is to only see revisions for a specific bugnote, which now ensures the user has appropriate access level.

Targeting future 1.2.x release. Still looking for some actual feedback on this one, unless you just want me to push it in after 1.2.0a3 and let it get ironed out before the next release...

I am eagerly waiting for this function as well.

@giallu (0009856:0020514):

the tracking changes is a very important feature for two reasons:

1. in same place understanding what changes take place in summary/description/notes is critical and leads to save time (the most important resource).

2. open mantis at the new type of use: requirements managements. The issue can be a requirements shared with my stakeholders and have a history of changes is mandatory (requirements negotiation phase - for this process will be amazing if notes has a tree view, like forum, to track down the discussion).

I know that the system scope is bug tracking but, the link from bugs and requirements is so important for maintenance of a system that manage this in two place is waste of time. The requirement is the other side of the coin.

Implementation committed to master branch.

Firstly, nice work! This will be very useful.

Formally, what is the difference between the new 'revisions' table and the existing 'history' table? A real annoyance with Mantis (in my situation) at the moment is the issue history storing a textual representation of the values. Say you update one of the version fields later on, the issue history will not be updated to reflect the change to the version name/number.

It seems to make more sense to me that bugs are stored in the database as 'bug revisions' and the 'bug table' is simply an index pointing to the latest revision of each bug number. When you update a bug, the existing bug is copied as a new bug revision, changes are made to this copy and the bug table index is updated to point to the new revision. It would also be possible to delete information from the previous revision which is the same as the new revision (to prevent duplication). If you want to see what changed between revision n-1 and n, you'd simply list the fields in the n-1 revision that are defined, as well as the fields in revision n which weren't defined in n-1.