0036862: Add possibility to add robots.txt

ID	Project	Category	View Status	Date Submitted	Last Update
0036862	mantisbt	administration	public	2026-02-03 02:50	2026-02-04 04:43

Reporter	lega4	Assigned To
Priority	normal	Severity	minor	Reproducibility	always
Status	feedback	Resolution	open

Summary	0036862: Add possibility to add robots.txt
Description	It would be great if there is a supported way to provide robots.txt file to Mantis. I recently came to my Mantis instance seeing a lot of GPTBot requests and I thought I could just filter them out with robots.txt... only to figure out there is not built-in way to do so...
Tags	No tags attached.

dregad 2026-02-04 04:43 developer ~0070761	seeing a lot of GPTBot requests and I thought I could just filter them out with robots.txt Good luck with that. These guys mostly just don't care about robots.txt, directives are not respected. Read: https://en.wikipedia.org/wiki/Robots.txt#Artificial_intelligence FYI, for mantisbt.org I ended up implementing blacklist rules using fail2ban as constant AI bot requests were killing the web server (totaling >90% of millions of requests per day). That being said, robots exclusion is a generic and global web server configuration, that I IMHO should NOT be implemented as part of MantisBT.