View Issue Details

IDProjectCategoryView StatusLast Update
0032787mantisbtadministrationpublic2023-09-28 08:27
Reporterdregad Assigned Todregad  
PrioritynormalSeverityfeatureReproducibilityN/A
Status assignedResolutionreopened 
Target Version2.26.0Fixed in Version2.26.0 
Summary0032787: Facilitate identification of user accounts sharing the same email
Description

When email_ensure_unique = ON, it is an error to have multiple user accounts with the same email address in the database, which is something that can happen when this config's value is changed from OFF to ON or when upgrading from MantisBT < 1.3.0-rc.2 (see 0009093).

As discussed in 0020647:0052620, we should make it easier for the admin to identify these accounts having duplicate emails so they can be fixed.

TagsNo tags attached.

Relationships

related to 0009093 closedvboctor Add a configuration option to enforce email uniqueness 
related to 0020647 resolvedvboctor Not able to update existing user accounts if $g_email_ensure_unique == ON 
related to 0032451 resolveddregad Email uniqueness is not enforced on case-sensitive databases 
related to 0032940 assigneddregad Add admin check to detect users without e-mail address when allow_empty_email = OFF 

Activities

dregad

dregad

2023-07-29 05:27

developer   ~0067921

PR https://github.com/mantisbt/mantisbt/pull/1898

atrol

atrol

2023-09-09 09:38

developer   ~0068077

@dregad I noticed that our "Manage User" page is slow.
I suspect it's caused by the user_is_email_unique calls you introduced to show the icon for non-unique emails.
You might want to remove it again, as there is not much benefit for installations with thousands of users.

dregad

dregad

2023-09-10 04:23

developer   ~0068078

I'll have a look if it can be optimized and will revert if not.

dregad

dregad

2023-09-10 08:03

developer   ~0068079

I ran quick local test with a dump of mantisbt.org tracker's user table (44423 rows):

  • master (0553a42): Page execution time: 0.6156 seconds Memory usage: 3,999 KiB Total queries executed: 66 Total query execution time: 0.3678 seconds
  • commit d6681928: Page execution time: 0.6105 seconds Memory usage: 4,081 KiB Total queries executed: 16 Total query execution time: 0.3592 seconds

So it does execute 50 additional queries (as expected for the uniqueness check), but the overall performance is nearly the same.

On mantisbt.org on the other hand, the same page executes in approx. 3 seconds, vs 0.2 seconds with commit d66819280.

So it seems environment specific, maybe a missing or corrupt index on the DB ?

dregad

dregad

2023-09-10 08:15

developer   ~0068080

Git bisect identifies MantisBT master bf7a3c22 as the offending commit (no surprise).

dregad

dregad

2023-09-10 08:21

developer   ~0068081

I tried optimizing and repairing the table, but that did not change anything. I don't understand why I'm not seeing the performance degradation locally. Any ideas ?

atrol

atrol

2023-09-10 08:34

developer   ~0068082

Last edited: 2023-09-10 09:56

I don't understand why I'm not seeing the performance degradation locally

I remember you are using some script to change the original email addresses for local testing purposes.
Maybe you change all addresses to the same one or clear them?

[EDIT] Forget about that, the script adds the user name to the addresses.
https://github.com/mantisbt/mantisbt-tools/blob/master/db-anonymize.sql

atrol

atrol

2023-09-10 09:11

developer   ~0068083

Maybe replacing the ILIKE by comparing with UPPER(email) works better for some databases.

dregad

dregad

2023-09-10 13:12

developer   ~0068084

I remember you are using some script to change the original email addresses for local testing purposes.

I did not use it in this specific case, to make sure I had a test case as close as possible to the original. My local database is an exact copy.

Maybe replacing the ILIKE by comparing with UPPER(email) works better for some databases

It could be, but in this specific case I don't think that's the issue as both environments are running MySQL (8.0 on the server, 8.1 on my laptop).

dregad

dregad

2023-09-16 10:25

developer   ~0068111

Last edited: 2023-09-28 08:27

PR https://github.com/mantisbt/mantisbt/pull/1919 improves the performance issue with manage_user_page.php reported by @atrol in 0032787:0068077.

With this, the page loads under 0.5 seconds on this tracker, vs approx. 3 seconds before this change. This is still about twice as slow as before MantisBT master d6681928, but I believe that's acceptable for an admin page.

Related Changesets

MantisBT: master 3a87f5d9

2023-05-27 10:59

dregad


Details Diff
Add admin check to detect duplicate e-mails

When email_ensure_unique = ON, it is an error to have multiple user
accounts with the same email address in the database, which can happen
when this config's value is changed from OFF to ON.

This check facilitates identification of the offending accounts, helping
admins to fix them.

When email_ensure_unique = OFF, duplicates are shown as a warning, so
admins can anticipate problems when planning a switch to unique emails.

Fixes 0032787
Affected Issues
0032787
mod - admin/check/check_email_inc.php Diff File

MantisBT: master 94908539

2023-07-26 16:58

dregad


Details Diff
Fix query to find users with duplicate emails

With MySQL in only_full_group_by sql_mode, the query used in admin
checks to find users with duplicate emails (which was written and tested
with a PostgreSQL database) is throwing

ERROR 1055 (42000): Expression 0000001 of HAVING clause is not in GROUP BY
clause and contains nonaggregated column 'bugtracker.mantis_user_table.email'
which is not functionally dependent on columns in GROUP BY clause; this
is incompatible with sql_mode=only_full_group_by.

Adding a sub-query fixes the problem, and the updated query works fine
with PostgreSQL too (SQL Server and Oracle not tested).

Issue 0032787
Affected Issues
0032787
mod - admin/check/check_email_inc.php Diff File

MantisBT: master bf7a3c22

2023-07-26 17:22

dregad


Details Diff
Identify duplicated email addresses

When email_ensure_unique = ON, it is an error to have multiple user
accounts with the same email address in the database, which can happen
when this config's value is changed from OFF to ON.

This helps the admin identify offending accounts by displaying

- a warning sign next to the email address on Manage Accounts page
- warning + info message on Account Edit pages (both account_page.php
and manage_user_edit_page.php)

Fixes 0032787
Affected Issues
0032787
mod - account_page.php Diff File
mod - lang/strings_english.txt Diff File
mod - manage_user_edit_page.php Diff File
mod - manage_user_page.php Diff File

MantisBT: master 18ea9a43

2023-07-29 03:38

dregad


Details Diff
Documentation: identifying duplicate emails

Fixes 0032787
Affected Issues
0032787
mod - docbook/Admin_Guide/en-US/config/email.xml Diff File

MantisBT: master f5298f14

2023-07-29 09:47

dregad


Details Diff
Use db_get_table() in SQL for duplicated email check

Without this, the check would fail when using non-default database table
prefix/suffix.

Issue 0032787
Affected Issues
0032787
mod - admin/check/check_email_inc.php Diff File