View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update | 
|---|---|---|---|---|---|
| 0012601 | mantisbt | upgrade | public | 2010-12-10 15:41 | 2012-11-05 10:11 | 
| Reporter | vboctor | Assigned To | vboctor | ||
| Priority | normal | Severity | minor | Reproducibility | have not tried | 
| Status | closed | Resolution | fixed | ||
| Product Version | 1.2.3 | ||||
| Target Version | 1.2.4 | Fixed in Version | 1.2.4 | ||
| Summary | 0012601: Upgrading scripts sometimes fails with a server error in case of large databases | ||||
| Description | When upgrading from a MantisBT version that is prior the datetime change to "int" instead of timestamp, if the instance has a lot of rows in one of the tables, it generates too many update queries which causes the server to fail the queries at one point. I've found the failure to typically happen after 4-5K rows. The problem is that when the page is refreshed, it does the same rows and hence no progress is made. The fix is to make it only do rows that have the int timestamp = 1, which means that they are not upgraded yet. This way, the user will still get the error, but with retries, progress will be made and the step will be finished. Still not a great experience, but at least it eventually works. I've seen this happen with the issue history table, but it can potentially happen with others as well. | ||||
| Tags | No tags attached. | ||||
| Hi Victor, I agree with the patch you've committed - looks good! I was curious as to whether you felt it was necessary to take this patch and also make it work on tables which contain more than one DATETIME column that needs upgrading? Or are these tables generally not the ones which grow to 4-5k rows? | |
| Paul seems to indicate to me that the bug_history table won't be affected by this patch (because it contains multiple DATETIME columns). If this is true, wouldn't this table also cause upgrade problems due to it's inevitably large number of rows? Thanks | |
| I think it is necessary to do two things: 
 As for the history table, it only contains one date time. The fields it contains are: id, user_id, bug_id, field_name, old_value, new_value, type, date_modified. The tables that have two timestamps are: bug, bug note, news, tag, user. I think the most likely ones to possibly hit the limits are bug note, bug and user. However, in the case of the database I was upgrading, none of these hit the limit to cause a failure. | |
| Another thing to note is that we got this error because of the one query per row thing which we don't have often. If there is no work around to avoid the server error in such case and we feel we are going to have more of this kind of query per row updates, then the investment in this area would be justified, otherwise, we should be fine with workarounds like the one I've implemented. Maybe a bit better by just doing 3k rows and then asking user to refresh or something. | |
| MantisBT: master-1.2.x 93b32ea2 2010-12-10 15:48 Details Diff | Fix 0012601: Upgrading scripts sometimes fails with a server error in case of large databases. | Affected Issues 0012601 | |
| mod - admin/install_functions.php | Diff File | ||
| MantisBT: master 2a7fe6dd 2010-12-10 15:56 Details Diff | Fixes 0012601: Upgrading scripts sometimes fails with a server error in case of large databases | Affected Issues 0012601 | |
| mod - core/install_helper_functions_api.php | Diff File | ||
| MantisBT: master-1.2.x a0fbeebf 2012-11-05 10:05 Details Diff | Documentation: note on upgrading large databases Added a comment on how to handle the failure to process the date conversion in a single setup for large databases, as documented in issues 0012601, 0012735 and 0014830 | Affected Issues 0012601, 0012735, 0014830 | |
| mod - docbook/administration_guide/en/installation.sgml | Diff File | ||
| MantisBT: master f46b71f6 2012-11-05 10:05 Details Diff | Documentation: note on upgrading large databases Added a comment on how to handle the failure to process the date conversion in a single setup for large databases, as documented in issues 0012601, 0012735 and 0014830 | Affected Issues 0012601, 0012735, 0014830 | |
| mod - docbook/Admin_Guide/en-US/Installation.xml | Diff File | ||
