View Issue Details

IDProjectCategoryView StatusLast Update
0017826mantisbtadministrationpublic2017-02-01 22:47
Reportervboctor 
Assigned Tovboctor 
PrioritynormalSeverityminorReproducibilityhave not tried
Status assignedResolutionopen 
Product Version1.2.17 
Target Version1.3.7Fixed in Version 
Summary0017826: Support disabling per project upload path
Description

Often there is a single attachments folder for all folders. In such case, it would be useful to disable the upload path field showing up for each project. This allows the administrator to set one path to be used for all projects.

Add $g_allow_per_project_upload_path.

Tagsmantishub

Relationships

Activities

vboctor

vboctor

2014-11-01 02:07

manager   ~0041734

Pull request:
https://github.com/mantisbt/mantisbt/pull/538

dregad

dregad

2014-11-01 10:40

developer   ~0041736

Do we really need this ?

Considering that if the upload path is left blank it defaults to the global upload path, it seems somewhat unnecessary to add a new config to just hide it in a not-so-frequently used management page.

vboctor

vboctor

2014-11-05 11:21

manager   ~0041767

Last edited: 2014-11-05 11:22

View 2 revisions

This is useful in a couple of scenario:

  • Administrator doesn't want managers to mess around with upload path.
  • MantisHub scenario where users shouldn't have to worry about such details. The hoster wouldn't even want to disclose such information.
dregad

dregad

2014-11-06 10:18

developer   ~0041775

  • Administrator doesn't want managers to mess around with upload path.

We already have code in there to hide the absolute path to non-admins.

Maybe it would make sense to completely hide the field instead of just the sensitive content ?

  • MantisHub scenario where users shouldn't have to worry about such details. The hoster wouldn't even want to disclose such information.

I personally don't mind that much myself, but I can't help smiling as I imagine grangeway's reaction to a request for patching an open-source system solely to address the needs of a commercial endeavor ;)

Anyway, kind of hard for me to comment as I don't know how MantisHub works internally.

Taking things one step further (just thinking out loud here): what about getting rid of the project-specific upload folder setting completely ? I never really understood the need to manually set that up anyway... We could consider something like:

  • only use a global setting (i.e. all files go in a single dir)
  • to limit number of files per dir, automatically create project-based subdirs (e.g. based on project ID or whatever)
  • have an upgrade step or an admin page create (and move files into) the new structure as required
vboctor

vboctor

2014-11-06 11:37

manager   ~0041776

Maybe it would make sense to completely hide the field instead of just the sensitive content ?

It could be that we should extend this option to hide the field, and enable even hiding from admin. Will check.

solely to address the needs of a commercial endeavor ;)

Making our product host-able is a good thing for the open source project. Think of WordPress. If it wasn't easy to host, it wouldn't be as successful for sure. MantisHub doesn't need such features to be ported back to MantisBT since it overlays its own set of hosting-specific features. It is more work to port these changes than just to keep them MantisHub specific. So we should think about this as goodness to MantisBT.

kind of hard for me to comment as I don't know how MantisHub works internally.

You should comment from the MantisBT perspective.

what about getting rid of the project-specific upload folder setting completely?

I'm totally for removing the per project upload path. In fact that is how I run any instance that I personally manage. I never hit an issue with number of files in folder given real usage that I have seen.

dregad

dregad

2014-11-06 11:56

developer   ~0041777

I'm totally for removing the per project upload path. In fact that is how I run
any instance that I personally manage. I never hit an issue with number of
files in folder given real usage that I have seen.

In that case, this sounds like the better way forward.

I've not seen this with Mantis, but I've had experiences of similar functionality with other software (Windows-hosted) where directories with very large number of files started performing very poorly (talking 40K+ files in my case) [1]

[1] http://support.microsoft.com/kb/130694/en-us
Reference is quite old, but AFAIK still valid today

vboctor

vboctor

2014-11-06 12:15

manager   ~0041780

I know about the large directories having a big performance issue. What I've noticed is that the number of attachments that MantisBT usually has doesn't often hit these limits.

There is two parts of the work here:

  1. Drive new instances to using a single path or possibly existing instances as well for new attachments.
  2. Migrate from N folders to 1.

Number 2 is likely more work that 1. Anyway, we will dump my thoughts here as I spend more time on it. However, it is not a priority really.

atrol

atrol

2014-11-06 12:16

developer   ~0041781

Last edited: 2014-11-07 02:13

View 2 revisions

I'm totally for removing the per project upload path.
There might be use cases where this is needed to have enough disk space.

As a side note: IIRC there is a bug when cloning issues with attachments from one project to another one that has setup the same upload folder. That's why I had to setup an own directory for project "Track DBAL replacement"

dregad

dregad

2014-11-07 03:47

developer   ~0041785

I know about the large directories having a big performance issue. What I've
noticed is that the number of attachments that MantisBT usually has doesn't
often hit these limits.

The instance I ran at my previous employer had more than 25K attachments, which is enough to start noticing performance degradations when accessing the file system (at that point we decided to move attachments to DB).

Anyway, my point is that if we revisit this, we should make sure the solution we develop works well for all cases and not just the ones we know about.

In other words, if we take away from admins the possibility to specify per-project dirs we should at least make sure we don't introduce performance issues. Organizing attachments in a tree vs a single dir, and relocating them into such a structure is not adding much more overhead, so it's probably worth the effort.

We could also consider a simple strategy where we use the first few chars of the filename's MD5 hash as directories, e.g. c4ca4238a0b923820dcc509a6f75849b ==> /c/4/ca4238a0b923820dcc509a6f75849b (but need to be careful with existing old attachments not using md5 names).

noizeg

noizeg

2016-10-15 14:51

reporter   ~0054231

Hello everyone, just want to share one thing on this.
I changed the core, so each new project will get unique folder for its attachments.
The main issue was to change default mechanics for reading attaches.
If the project has own attach folder - full folder path will be written to the mantis_bug_file_table.
To open the file - system reads full path from the folder column in mantis_bug_file_table and concat it with diskfile value. Imagine that you moved attaches folder - system won't be able to find old files, even if you change upload_path in config and in project settings. To fix it you'll have to update old records in mantis_bug_file_table or rewrite core so it won't be taking full path into consideration.

vboctor

vboctor

2016-12-02 04:23

manager   ~0054661

@dregad The suggest change for this issue is to enable the administrator to turn a config ON to hide the upload project path so that managers don't see it and can't update it. This will be OFF by default, but it is a global config option that administrator can set via config_inc.php. If ON, the attachments will be saved to the default upload path. Also the move attachments to disk in such case will save the attachments extracted from the db to the default upload path.

@noizeg From memory, if a file is not found in the path saved in the db, we look for it in the default upload file path. We should also check for the project upload path (if set). This should fix the issue that you are raising where the project path is moved. In this case, we will not need to update the db file entries with the new paths.

In general, I don't like the fact that we save the full path to the file. I would rather we just save the file name and then at retrieval time construct the full path based on project upload path (if found) or based on default upload path (if found). However, that should be a separate issue.

Issue History

Date Modified Username Field Change
2014-11-01 01:34 vboctor New Issue
2014-11-01 01:34 vboctor Status new => assigned
2014-11-01 01:34 vboctor Assigned To => vboctor
2014-11-01 02:07 vboctor Note Added: 0041734
2014-11-01 02:20 vboctor Tag Attached: mantishub
2014-11-01 10:40 dregad Note Added: 0041736
2014-11-05 11:21 vboctor Note Added: 0041767
2014-11-05 11:22 vboctor Note Edited: 0041767 View Revisions
2014-11-06 10:18 dregad Note Added: 0041775
2014-11-06 11:37 vboctor Note Added: 0041776
2014-11-06 11:56 dregad Note Added: 0041777
2014-11-06 12:15 vboctor Note Added: 0041780
2014-11-06 12:16 atrol Note Added: 0041781
2014-11-07 02:13 atrol Note Edited: 0041781 View Revisions
2014-11-07 03:47 dregad Note Added: 0041785
2014-12-08 02:10 atrol Target Version 1.3.0-beta.1 => 1.3.0-beta.2
2015-03-02 08:04 vitsaadmin Issue cloned: 0019435
2015-03-15 19:59 dregad Target Version 1.3.0-beta.2 => 1.3.0-beta.3
2015-09-06 17:47 vboctoradmin Target Version 1.3.0-beta.3 => 1.3.0-rc.1
2015-12-06 02:55 vboctor Target Version 1.3.0-rc.1 => 1.3.0-rc.2
2016-06-12 02:37 atrol Target Version 1.3.0-rc.2 => 1.3.0
2016-07-10 07:57 atroladmin Target Version 1.3.0 => 1.3.1
2016-08-28 10:37 atrol Target Version 1.3.1 => 1.3.2
2016-10-02 19:36 atrol Target Version 1.3.2 => 1.3.3
2016-10-15 14:51 noizeg Note Added: 0054231
2016-10-30 23:23 vboctor Target Version 1.3.3 => 1.3.4
2016-11-27 08:22 dregad Target Version 1.3.4 => 1.3.5
2016-12-02 04:23 vboctor Note Added: 0054661
2016-12-30 16:24 atrol Target Version 1.3.5 => 1.3.6
2017-02-01 22:47 vboctor Target Version 1.3.6 => 1.3.7