mirror of
https://codeberg.org/forgejo/forgejo.git
synced 2024-12-22 08:38:29 +00:00
544cbc6f01
This PR optimizes the SQL query and de-duplicate the labels' ids when generating the query string, on the issue page. <hr/> ### Background Some time ago, BingBot and some other crawlers have been putting my instance on its knees with requests containing a lot of label ids, like this one : ``` [07/Aug/2023:11:28:37 +0200] "GET /Dolibarr/sendrecurringinvoicebymail/issues?q=&type=all&sort=&state=closed&labels=1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c2%2c10%2c2%2c1%2c1%2c10%2c10%2c7%2c6%2c10%2c10%2c3%2c2%2c1%2c5%2c10%2c1%2c6%2c2%2c7%2c3%2c7%2c6%2c10%2c1%2c10%2c1%2c1%2c7%2c7%2c1%2c1%2c1%2c1%2c10%2c10%2c1%2c2%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c1%2c2%2c1%2c12%2c6%2c6%2c10&milestone=0&project=-1&poster=0 HTTP/1.1" 499 0 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/103.0.5060.134 Safari/537.36" ``` Since each of the label ids implies a join, it grows exponentially expensive for the database engine (at least on PostgreSQL but SQLite suffers a little too). Thus, this PR proposes two enhancements: * rewrite the database query to use only one squashed condition, * deduplicate the label ids when generating the URL. ### Performance comparison Here are some timings on Postgresql-backed, Forgejo 7.0.4 instances : ```sh $ time curl -s -o /dev/null "http://localhost:3000/toto/tata/issues?q=&type=all&sort=&labels=19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25&state=open&milestone=0&project=0&assignee=0&poster=0" real 0m10,491s user 0m0,017s sys 0m0,008s ``` ...and with the patch: ```sh $ time curl -s -o /dev/null "http://localhost:3000/toto/tata/issues?q=&type=all&sort=&labels=19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25%2c19%2c25&state=open&milestone=0&project=0&assignee=0&poster=0" real 0m0,094s user 0m0,012s sys 0m0,013s ``` ### Annex This issue was originally proposed to [Gitea](https://github.com/go-gitea/gitea/pull/26460) but didn't get much attention, and I switched to Forgejo in the meantime :) Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4228 Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org> Co-authored-by: Chl <chl@xlii.si> Co-committed-by: Chl <chl@xlii.si> |
||
|---|---|---|
| .. | ||
| assignees.go | ||
| assignees_test.go | ||
| comment.go | ||
| comment_code.go | ||
| comment_list.go | ||
| comment_test.go | ||
| content_history.go | ||
| content_history_test.go | ||
| dependency.go | ||
| dependency_test.go | ||
| issue.go | ||
| issue_index.go | ||
| issue_index_test.go | ||
| issue_label.go | ||
| issue_label_test.go | ||
| issue_list.go | ||
| issue_list_test.go | ||
| issue_lock.go | ||
| issue_project.go | ||
| issue_search.go | ||
| issue_stats.go | ||
| issue_stats_test.go | ||
| issue_test.go | ||
| issue_update.go | ||
| issue_user.go | ||
| issue_user_test.go | ||
| issue_watch.go | ||
| issue_watch_test.go | ||
| issue_xref.go | ||
| issue_xref_test.go | ||
| label.go | ||
| label_test.go | ||
| main_test.go | ||
| milestone.go | ||
| milestone_list.go | ||
| milestone_test.go | ||
| pull.go | ||
| pull_list.go | ||
| pull_test.go | ||
| reaction.go | ||
| reaction_test.go | ||
| review.go | ||
| review_list.go | ||
| review_test.go | ||
| stopwatch.go | ||
| stopwatch_test.go | ||
| tracked_time.go | ||
| tracked_time_test.go | ||