Commit graph

3884 commits

Author SHA1 Message Date
Giteabot 1a3803effd
improve possible performance bottleneck (#28547) (#28578)
Backport #28547 by @lunny

Replace #28500

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-12-21 23:37:08 +00:00
Giteabot 16263af971
Fix inperformant query on retrifing review from database. (#28552) (#28562)
Backport #28552 by @6543

can we please PLEAS PLEASE only use raw SQL statements if it is relay
needed!!!

source is https://github.com/go-gitea/gitea/pull/28544 (before
refactoring)

Co-authored-by: 6543 <m.huber@kithara.com>
2023-12-20 16:55:08 +01:00
Giteabot d9aeb1f09d
Only check online runner when detecting matching runners in workflows (#28286) (#28512)
Backport #28286 by @yp05327

Mentioned:
[#28277](https://github.com/go-gitea/gitea/issues/28277#issuecomment-1831325276)

We should only check online runner when detecting matching runners in
workflows,
as if runner is not online, the workflow will not run.


![image](https://github.com/go-gitea/gitea/assets/18380374/11855e9d-7241-4b7a-b8d7-49dbb94ba1c5)

Co-authored-by: yp05327 <576951401@qq.com>
2023-12-19 04:06:31 +00:00
Giteabot 6cbb6f303a
Refactor SSH clone URL generation code (#28421) (#28480)
Backport #28421 by wxiaoguang

Refactor the code and add tests, keep the old logic.

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-12-15 14:54:40 +08:00
Giteabot b47482d58e
Retry SSH key verification with additional CRLF if it failed (#28392) (#28464)
Backport #28392 by @nekrondev

Windows-based shells will add a CRLF when piping the token into
ssh-keygen command resulting in
verification error. This resolves #21527.

Co-authored-by: nekrondev <heiko@noordsee.de>
Co-authored-by: Heiko Besemann <heiko.besemann@qbeyond.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-12-14 12:50:26 +08:00
Lunny Xiao 87db4a47c8
Also sync DB branches on push if necessary (#28361) (#28403)
Fix #28056
Backport #28361 

This PR will check whether the repo has zero branch when pushing a
branch. If that, it means this repository hasn't been synced.

The reason caused that is after user upgrade from v1.20 -> v1.21, he
just push branches without visit the repository user interface. Because
all repositories routers will check whether a branches sync is necessary
but push has not such check.

For every repository, it has two states, synced or not synced. If there
is zero branch for a repository, then it will be assumed as non-sync
state. Otherwise, it's synced state. So if we think it's synced, we just
need to update branch/insert new branch. Otherwise do a full sync. So
that, for every push, there will be almost no extra load added. It's
high performance than yours.

For the implementation, we in fact will try to update the branch first,
if updated success with affect records > 0, then all are done. Because
that means the branch has been in the database. If no record is
affected, that means the branch does not exist in database. So there are
two possibilities. One is this is a new branch, then we just need to
insert the record. Another is the branches haven't been synced, then we
need to sync all the branches into database.
2023-12-11 06:16:56 +00:00
Giteabot 46beb7f33f
enable system users search via the API (#28013) (#28018)
Backport #28013 by @earl-warren

Refs: https://codeberg.org/forgejo/forgejo/issues/1403

(cherry picked from commit dd4d17c159)

---------

Co-authored-by: Earl Warren <109468362+earl-warren@users.noreply.github.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-12-08 21:46:08 +00:00
Giteabot 5b5f8aab19
handle repository.size column being NULL in migration v263 (#28336) (#28363)
Co-authored-by: Nate Levesque <nate@thenaterhood.com>
2023-12-05 14:51:56 +00:00
Giteabot ea9f5a57e4
Increase "version" when update the setting value to a same value as before (#28243) (#28244)
Backport #28243

Setting the same value should not trigger DuplicateKey error, and the
"version" should be increased

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-11-28 01:58:39 +08:00
Lunny Xiao bc3d8bff73
Fix comment permissions (#28213) (#28216)
backport #28213

This PR will fix some missed checks for private repositories' data on
web routes and API routes.
2023-11-25 23:43:23 +00:00
Giteabot efcbaf8fa8
Fix no ActionTaskOutput table waring (#28149) (#28152)
Backport #28149 by @yp05327

Reproduce:
- Create a new Gitea instance
- Register a runner
- Create a repo and add a workflow
- Check the log, you will see warnings:

![image](https://github.com/go-gitea/gitea/assets/18380374/5f1278e0-114b-48bc-8113-8ba1404d9975)
It comes from:

![image](https://github.com/go-gitea/gitea/assets/18380374/c2807831-e137-4229-9536-87f6114c8a5b)

The reason is that we forgot registering `ActionTaskOutput` model.
So `action_table_output` table will be missing in your db.

Co-authored-by: yp05327 <576951401@qq.com>
2023-11-21 22:23:57 +08:00
Giteabot 9f63d27ec4
Fix system config cache expiration timing (#28072) (#28090)
Backport #28072

To avoid unnecessary database access, the `cacheTime` should always be
set if the revision has been checked.

Fix #28057

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-11-16 21:37:57 +08:00
Giteabot 073d8c50dd
Restricted users only see repos in orgs which their team was assigned to (#28025) (#28051)
Backport #28025 by @6543


---
*Sponsored by Kithara Software GmbH*

Co-authored-by: 6543 <m.huber@kithara.com>
2023-11-14 16:44:46 +01:00
Giteabot 09efce9da2
enable system users for comment.LoadPoster (#28014) (#28032)
Backport #28014 by @earl-warren

System users (Ghost, ActionsUser, etc) have a negative id and may be the
author of a comment, either because it was created by a now deleted user
or via an action using a transient token.

The GetPossibleUserByID function has special cases related to system
users and will not fail if given a negative id.

Refs: https://codeberg.org/forgejo/forgejo/issues/1425 

(cherry picked from commit 6a2d2fa243)

Co-authored-by: Earl Warren <109468362+earl-warren@users.noreply.github.com>
2023-11-14 14:50:05 +01:00
Nanguan Lin 97f4239a94
Fix wrong xorm Delete usage(backport for 1.21) (#28002)
manually backport for https://github.com/go-gitea/gitea/pull/27995
The conflict is `ctx` and `db.Defaultctx`.
2023-11-12 12:58:22 +00:00
Giteabot 61d3d9205b
Fix 500 when deleting a dismissed review (#27903) (#27910)
Backport #27903 by @lng2020

Fix #27767 
Add a test to ensure its behavior

Co-authored-by: Nanguan Lin <70063547+lng2020@users.noreply.github.com>
2023-11-05 13:59:11 +00:00
Giteabot 28e3d0b0d3
Remove action runners on user deletion (#27902) (#27908)
Backport #27902 by @earl-warren

- On user deletion, delete action runners that the user has created.
- Add a database consistency check to remove action runners that have
nonexistent belonging owner.
- Resolves https://codeberg.org/forgejo/forgejo/issues/1720

(cherry picked from commit 009ca7223d)

Co-authored-by: Earl Warren <109468362+earl-warren@users.noreply.github.com>
Co-authored-by: Gusted <postmaster@gusted.xyz>
2023-11-05 13:20:00 +00:00
6543 6637bbf510
Delete repos of org when purge delete user (#27273) (#27728)
Fixes https://codeberg.org/forgejo/forgejo/issues/1514

Backports #27273

---------

Co-authored-by: JakobDev <jakobdev@gmx.de>
2023-11-01 10:03:12 +08:00
Giteabot 2d2a5657ef
Upgrade xorm to 1.3.4 (#27807) (#27813)
Backport #27807 by @lng2020

Noticeable change: 
Remove the `OrderBy("1") `
[patch](https://github.com/go-gitea/gitea/pull/27673#issuecomment-1768570142)
for mssql since xorm has [fixed
it](0f085408af).

Co-authored-by: Nanguan Lin <70063547+lng2020@users.noreply.github.com>
2023-10-27 14:17:27 +02:00
Giteabot 89d3766d22
Upgrade xorm (#27673) (#27691)
Backport #27673 by @lng2020

Related to https://gitea.com/xorm/xorm/pulls/2341

Co-authored-by: Nanguan Lin <70063547+lng2020@users.noreply.github.com>
2023-10-19 10:56:39 +00:00
Giteabot 63587a4aef
Respect SSH.KeygenPath option when calculating ssh key fingerprints (#27536) (#27551)
Backport #27536 by @picsel2

Fixes #27535

Co-authored-by: Sebastian Grabowski <sebastian@grabel.de>
2023-10-10 07:37:58 +00:00
Giteabot fb5ae2ab94
Restore warning commit status (#27504) (#27529)
Backport #27504 by @silverwind

Partial revert of https://github.com/go-gitea/gitea/pull/25839. This
commit status is used by a number of external integrations, so I think
we should not remove it (See
https://github.com/go-gitea/gitea/pull/25839#issuecomment-1729002077).
This is a rare case where an existing migration needed to be alterted to
avoid data loss.

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: delvh <dev.lh@web.de>
2023-10-09 11:56:02 +08:00
Giteabot 44aca6a65a
Don't let API add 2 exclusive labels from same scope (#27433) (#27460)
Backport #27433 by @JakobDev

Fixes #27380

Co-authored-by: JakobDev <jakobdev@gmx.de>
2023-10-06 05:06:31 +08:00
Giteabot aaf35ee49c
Refactor system setting (#27000) (#27452)
Backport #27000 by @wxiaoguang

This PR reduces the complexity of the system setting system.

It only needs one line to introduce a new option, and the option can be
used anywhere out-of-box.

It is still high-performant (and more performant) because the config
values are cached in the config system.


![image](https://github.com/go-gitea/gitea/assets/2114189/f8cdd743-1145-41ab-9f8f-3996aa97d440)

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-10-05 10:37:59 +00:00
Giteabot 290440e1ee
Add Index to action.user_id (#27403) (#27425)
Backport #27403 by @JakobDev

Another Column that needs a Index. Found at
https://codeberg.org/forgejo/discussions/issues/61#issuecomment-1258744.

Co-authored-by: JakobDev <jakobdev@gmx.de>
2023-10-04 08:53:00 +02:00
Giteabot d5da0e622c
Don't use subselect in DeleteIssuesByRepoID (#27332) (#27408)
Backport #27332 by @JakobDev

Part of https://codeberg.org/forgejo/discussions/issues/61

This is workaround for a bug in MariaDB

Co-authored-by: JakobDev <jakobdev@gmx.de>
2023-10-03 17:59:45 +08:00
Giteabot 4e824a735e
Allow get release download files and lfs files with oauth2 token format (#26430) (#27379)
Backport #26430 by @lunny

Fix #26165
Fix #25257

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-10-01 19:54:11 +08:00
Giteabot f13a294b47
More db.DefaultContext refactor (#27265) (#27347)
Backport #27265 by @JakobDev

Part of #27065

This PR touches functions used in templates. As templates are not static
typed, errors are harder to find, but I hope I catch it all. I think
some tests from other persons do not hurt.

Co-authored-by: JakobDev <jakobdev@gmx.de>
2023-09-29 13:35:01 +00:00
Giteabot f77b4cb4a2
Add logs for data broken of comment review (#27326) (#27345)
Backport #27326 by @lunny

Fix #27306

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-09-29 13:31:13 +08:00
Giteabot 7ea7f2b37f
Add Index to comment.dependent_issue_id (#27325) (#27340)
Backport #27325 by @JakobDev

This Column is missing index. It is used by
[issue_service.deleteIssue](7ea2a910ce/services/issue/issue.go (L300)).

Co-authored-by: JakobDev <jakobdev@gmx.de>
2023-09-29 02:00:14 +00:00
Giteabot c61b9c5f3c
Fix the approval count of PR when there is no protection branch rule (#27272) (#27343)
Backport #27272 by @lng2020

As title

![ksnip_20230926-115158](https://github.com/go-gitea/gitea/assets/70063547/a60be44a-06ad-421e-ba27-e4e0adfa5db7)

Co-authored-by: Nanguan Lin <70063547+lng2020@users.noreply.github.com>
2023-09-29 09:31:35 +08:00
Giteabot 9b698362a3
Redefine the meaning of column is_active to make Actions Registration Token generation easier (#27143) (#27304)
Backport #27143 by @lunny

Partially Fix #25041

This PR redefined the meaning of column `is_active` in table
`action_runner_token`.
Before this PR, `is_active` means whether it has been used by any
runner. If it's true, other runner cannot use it to register again.

In this PR, `is_active` means whether it's validated to be used to
register runner. And if it's true, then it can be used to register
runners until it become false. When creating a new `is_active` register
token, any previous tokens will be set `is_active` to false.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-09-28 04:15:53 +00:00
Giteabot 755369df56
Fix yaml test (#27297) (#27303)
Backport #27297 by @lunny

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-09-27 15:05:35 +08:00
Giteabot 283b19bad5
Fix more yaml lint errors (#27284) (#27288)
Backport #27284 by @lunny

Fix #27268

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-09-26 19:00:12 +02:00
Giteabot fc7d3f7315
Another round of db.DefaultContext refactor (#27103) (#27262)
Backport #27103 by @JakobDev

Part of #27065

Co-authored-by: JakobDev <jakobdev@gmx.de>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-09-25 19:24:35 +02:00
Giteabot 64c0a6a4e2
Quote table release in sql queries (#27205) (#27218)
Backport #27205 by @KN4CK3R

Fixes #27174

`release` is a reserved keyword in MySql. I can't reproduce the issue on
my setup and we have a test for that code but it seems there can be
setups where it fails.


a101dbaa79/tests/integration/repo_activity_test.go (L45-L46)

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-09-23 21:15:05 +00:00
Giteabot 81ae35aa4f
Fix review request number and add more tests (#27104) (#27168)
Backport #27104 by @lng2020

fix #27019 
## testfixture yml
1. add issue20(a pr issue) in repo 23, org 17
2. add user15 to team 9
3. add four reviews about issue20
## test case
add two tests that are described with code comments
the code before pr #26784 failed the first test
<img width="479" alt="image"
src="https://github.com/go-gitea/gitea/assets/70063547/1d9b5787-11b4-4c4d-931f-6a9869547f35">
current code failed the second test(as mentioned in #27019)
<img width="484" alt="image"
src="https://github.com/go-gitea/gitea/assets/70063547/05608055-7587-43d1-bae1-92c688270819">
Any advice is appreciated.

Co-authored-by: Nanguan Lin <70063547+lng2020@users.noreply.github.com>
Co-authored-by: CaiCandong <50507092+CaiCandong@users.noreply.github.com>
2023-09-21 17:01:37 +00:00
Giteabot c3b7120042
Add index to issue_user.issue_id (#27154) (#27158)
Backport #27154 by @JakobDev

This fixes a performance bottleneck. It was discovered by Codeberg.
Every where query on that table (which has grown big over time) uses
this column, but there is no index on it.

See this part of the log which was posted on Matrix:
```
2023/09/10 00:52:01 ...rs/web/repo/issue.go:1446:ViewIssue() [W] [Slow SQL Query] UPDATE `issue_user` SET is_read=? WHERE uid=? AND issue_id=? [true x y] - 51.395434887s
2023/09/10 00:52:01 ...rs/web/repo/issue.go:1447:ViewIssue() [E] ReadBy: Error 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
2023/09/10 00:52:01 ...eb/routing/logger.go:102:func1() [I] router: completed GET /Codeberg/Community/issues/1201 for [::ffff:xxx]:0, 500 Internal Server Error in 52384.2ms @ repo/issue.go:1256(repo.ViewIssue)
```

Co-authored-by: JakobDev <jakobdev@gmx.de>
2023-09-21 11:42:34 +08:00
Lunny Xiao e644cc9448
Fix wrong xorm get usage on migration (#27111)
Fix the bug on try.gitea.io

```log
2023/09/18 01:48:41 ...ations/migrations.go:635:Migrate() [I] Migration[276]: Add RemoteAddress to mirrors
2023/09/18 01:48:41 routers/common/db.go:34:InitDBEngine() [E] ORM engine initialization attempt #7/10 failed. Error: migrate: migration[276]: Add RemoteAddress to mirrors failed: exit status 128 - fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
 - fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
```

Caused by #26952

---------

Co-authored-by: Jason Song <i@wolfogre.com>
2023-09-18 22:59:19 +03:00
Lunny Xiao 47b878858a
Search branches (#27055)
Resolve #25233 

<img width="1315" alt="图片"
src="https://github.com/go-gitea/gitea/assets/81045/3ba59b58-471a-4e1b-985c-87edac2268c0">

<img width="1297" alt="图片"
src="https://github.com/go-gitea/gitea/assets/81045/b6caa12f-323b-4f70-9c44-ef91cb71a26c">
2023-09-17 08:24:40 +00:00
Lunny Xiao dcf4b9e314
Fix wrong migration for email address (#27106)
On Iterate, `sess` should not be used in the closure function body.

Caused by #26952
2023-09-17 11:15:04 +08:00
KN4CK3R c766140dad
Add RemoteAddress to mirrors (#26952)
This PR adds a new field `RemoteAddress` to both mirror types which
contains the sanitized remote address for easier (database) access to
that information. Will be used in the audit PR if merged.
2023-09-16 16:03:02 +00:00
JakobDev f91dbbba98
Next round of db.DefaultContext refactor (#27089)
Part of #27065
2023-09-16 14:39:12 +00:00
wxiaoguang 4ffc30cb84
Use db.WithTx for AddTeamMember to avoid ctx abuse (#27095)
Compare with ignoring spaces:
https://github.com/go-gitea/gitea/pull/27095/files?diff=split&w=1
2023-09-16 12:54:23 +00:00
JakobDev c548dde205
More refactoring of db.DefaultContext (#27083)
Next step of #27065
2023-09-15 06:13:19 +00:00
JakobDev 76659b1114
Reduce usage of db.DefaultContext (#27073)
Part of #27065

This reduces the usage of `db.DefaultContext`. I think I've got enough
files for the first PR. When this is merged, I will continue working on
this.

Considering how many files this PR affect, I hope it won't take to long
to merge, so I don't end up in the merge conflict hell.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-09-14 17:09:32 +00:00
Nanguan Lin 0de09d3afc
Remove the useless function GetUserIssueStats and move relevant tests to indexer_test.go (#27067)
Since the issue indexer has been refactored, the issue overview webpage
is built by the `buildIssueOverview` function and underlying
`indexer.Search` function and `GetIssueStats` instead of
`GetUserIssueStats`. So the function is no longer used.
I moved the relevant tests to `indexer_test.go` and since the search
option changed from `IssueOptions` to `SearchOptions`, most of the tests
are useless now.
We need more tests about the db indexer because those tests are highly
connected with the issue overview webpage and now this page has several
bugs.
Any advice about those test cases is appreciated.

---------

Co-authored-by: CaiCandong <50507092+CaiCandong@users.noreply.github.com>
2023-09-14 12:35:53 -04:00
Lunny Xiao 198a9ca635
Display all user types and org types on admin management UI (#27050)
Follow #24026 

<img width="1049" alt="图片"
src="https://github.com/go-gitea/gitea/assets/81045/d3fc5159-b5e7-411a-b6f8-4a111a027e6b">

---------

Co-authored-by: delvh <dev.lh@web.de>
2023-09-14 06:53:36 +00:00
Nanguan Lin da50be7360
Replace 'userxx' with 'orgxx' in all test files when the user type is org (#27052)
Currently 'userxx' and 'orgxx' are both used as username in test files
when the user type is org, which is confusing. This PR replaces all
'userxx' with 'orgxx' when the user type is org(`user.type==1`).
Some non-trivial changes
1. Rename `user3` dir to `org3` in `tests/git-repositories-meta` 
2. Change `end` in `issue reference` because 'org3' is one char shorter
than 'user3'

![ksnip_20230913-112819](https://github.com/go-gitea/gitea/assets/70063547/442988c5-4cf4-49b8-aa01-4dd6bf0ca954)
3. Change the search result number of `user/repo2` because
`user3/repo21` can't be searched now

![ksnip_20230913-112931](https://github.com/go-gitea/gitea/assets/70063547/d9ebeba4-479f-4110-9a85-825efbc981fd)
4. Change the first org name getting from API because the result is
ordered by alphabet asc and now `org 17` is before `org25`
![JW8U7NIO(J$H
_YCRB36H)T](https://github.com/go-gitea/gitea/assets/70063547/f55a685c-cf24-40e5-a87f-3a2327319548)
![)KFD411O4I8RB5ZOH7E0
Z3](https://github.com/go-gitea/gitea/assets/70063547/a0dc3299-249c-46f6-91cb-d15d4ee88dd5)

Other modifications are just find all and replace all.
Unit tests with SQLite are all passed.

---------

Co-authored-by: caicandong <1290147055@qq.com>
2023-09-14 02:59:53 +00:00
Nanguan Lin cda97a7253
Update status and code index after changing the default branch (#27018)
Fix #26723 
Add `ChangeDefaultBranch` to the `notifier` interface and implement it
in `indexerNotifier`. So when changing the default branch,
`indexerNotifier` sends a message to the `indexer queue` to update the
index.

---------

Co-authored-by: techknowlogick <matti@mdranta.net>
2023-09-13 04:43:31 +00:00