Commit graph

3173 commits

Author SHA1 Message Date
Gusted 640f0e1ddf
Only allow returned deleted branche to be on repo (#17570)
- This will only allow `GetDeletedBranchByID` to return deletedBranch
which are on the repo, and thus don't return a deletedBranch from
another repo.
- This just should prevent possible bugs in the futher when a code is
passing the wrong ID into this function.
2021-11-08 23:45:37 +08:00
Gusted ebaf4c48ea
Avoid double imports (#17569)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-08 09:04:13 +02:00
Lunny Xiao 69b61d4373
Fix bug on admin subcommand (#17533)
* Fix bug on admin subcommand

* Add signals for all initDB

Co-authored-by: Lauris BH <lauris@nix.lv>
2021-11-07 11:11:27 +08:00
Lunny Xiao fed8e01ca1
Fix database keyword quote problem on migration v161 (#17522)
* support rerun migration v161

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-11-05 00:47:01 +02:00
zeripath 9d855bd6a1
Simplify Gothic to use our session store instead of creating a different store (#17507)
* Simplify Gothic to use our session store instead of creating a different store

We have been using xormstore to provide a separate session store for our OAuth2 logins
however, this relies on using gorilla context and some doubling of our session storing.
We can however, simplify and simply use our own chi-based session store. Thus removing
a cookie and some of the weirdness with missing contexts.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per review

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per review

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Handle MaxTokenLength

Signed-off-by: Andrew Thornton <art27@cantab.net>

* oops

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-11-03 08:33:54 +08:00
Lunny Xiao 63c0dc89ef
Rename db Engines related functions (#17481)
* Rename db Engines related functions

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-10-30 22:32:11 +08:00
zeripath 76a3190b8a
Quote the table name in CountOrphanedObjects (#17487)
CountOrphanedObjects needs to quote the table it is joining with as this table may
be `user`.

Fix #17485

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-30 10:17:40 +01:00
zeripath e6e3b212b3
Run Migrate in Install rather than just SyncTables (#17475)
* Run Migrate in Install rather than just SyncTables

The underlying problem in #17328 appears to be that users are re-running the install
page during upgrades. The function that tests and creates the db did not intend for
this and thus instead the migration scripts being run - a simple sync tables occurs.

This then causes a weird partially migrated DB which causes, in this release cycle,
the duplicate column in task table error. It is likely the cause of some weird
partial migration errors in other cycles too.

This PR simply ensures that the migration scripts are also run at this point too.

Fix #17328

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-29 09:23:10 +01:00
zeripath 0b4a8be26b
Ensure that restricted users can access repos for which they are members (#17460)
There is a small bug in the way that repo access is checked in
repoAssignment: Accessibility is checked by checking if the user has a
marked access to the repository instead of checking if the user has any
team granted access.

This PR changes this permissions check to use HasAccess() which does the
correct test. There is also a fix in the release api ListReleases where
it should return draft releases if the user is a member of a team with
write access to the releases.

The PR also adds a testcase.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-28 10:54:40 +08:00
wxiaoguang 7c951fdd4a
In many cases user avatar link should be an absolute URL with http host (#17420) 2021-10-25 13:01:16 +08:00
wxiaoguang 943dc08722
Fix issue content history problems, improve UI (#17404)
* Improve: make diff result better, make the HTML element fit the full height in the content history diff dialog
* Bug fix: when edit the main issue, the poster is wrongly set to the issue poster
2021-10-23 22:47:38 +08:00
wxiaoguang 960c322586
Refactor update checker to use AppState (#17387)
We have the `AppState` module now, it can store app related data easily. We do not need to create separate tables for each feature.

So the update checker can use `AppState` instead of a one-row dedicate table.

And the code of update checker is moved from `models` to `modules`.
2021-10-21 17:10:49 +01:00
Lunny Xiao f3c7f10c9f
Fix history count failure (#17351)
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-10-21 18:06:19 +08:00
wxiaoguang 83df0caf15
Sync gitea app path for git hooks and authorized keys when starting (#17335)
Gitea writes its own AppPath into git hook scripts. If Gitea's AppPath changes, then the git push will fail.

This PR:

* Introduce an AppState module, it can persist app states into database
* During GlobalInit, Gitea will check if the current AppPath is the same as last one. If they don't match, Gitea will sync git hooks.
* Refactor some code to make them more clear.
* Also, "Detect if gitea binary's name changed" #11341 is related, we call models.RewriteAllPublicKeys to update ssh authorized_keys file
2021-10-21 17:22:43 +08:00
Lunny Xiao f494776931
Use a variable but a function for IsProd because of a slight performance increment (#17368) 2021-10-20 16:37:19 +02:00
John Olheiser c59afa752d
Allow mocking timeutil (#17354)
Signed-off-by: jolheiser <john.olheiser@gmail.com>
2021-10-18 21:12:26 +01:00
Lunny Xiao 5326f4c9c5
Move session to models/login (#17338)
Move model/session.go to models/login
2021-10-17 19:51:56 +01:00
techknowlogick 8edda8b446
Add simple update checker to Gitea (#17212)
* Add simple update checker to Gitea

* update struct and remove comments

* fix lint

* Update custom/conf/app.example.ini

* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md

Co-authored-by: delvh <dev.lh@web.de>

* Update custom/conf/app.example.ini

Co-authored-by: delvh <dev.lh@web.de>

* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md

Co-authored-by: delvh <dev.lh@web.de>

* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md

Co-authored-by: Steven <61625851+justusbunsi@users.noreply.github.com>

* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md

* Update modules/cron/tasks_extended.go

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>

* Update custom/conf/app.example.ini

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>

* take PR feedback into account and display banner on admin dashboard for alerts

* Add more detailed message

* placate lint

* update per feedback

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Steven <61625851+justusbunsi@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-10-16 02:14:34 -04:00
David Jimenez fa8b8c055d
Don't panic if we fail to parse U2FRegistration data (#17304)
* Don't panic if we fail to parse a U2FRegistration data

Downgrade logging statement from Fatal to Error so that errors parsing
U2FRegistration data does not panic; instead, the invalid key will be
skipped and we will attempt to parse the next one, if available.

Signed-off-by: David Jimenez <dvejmz@sgfault.com>
2021-10-14 18:37:35 +02:00
zeripath 5a8dd96b33
Ensure that git daemon export ok is created for mirrors (#17243)
* Ensure that git daemon export ok is created for mirrors

There is an issue with #16508 where it appears that create repo requires that the
repo does not exist. This causes #17241 where an error is reported because of this.

This PR fixes this and also runs update-server-info for mirrors and generated repos.

Fix #17241

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-13 20:47:02 +01:00
wxiaoguang 7bcbdd0707
Add user status filter to admin user management page (#16770)
It makes Admin's life easier to filter users by various status.

* introduce window.config.PageData to pass template data to javascript module and small refactor

move legacy window.ActivityTopAuthors to window.config.PageData.ActivityTopAuthors
make HTML structure more IDE-friendly in footer.tmpl and head.tmpl
remove incorrect <style class="list-search-style"></style> in head.tmpl
use log.Error instead of log.Critical in admin user search

* use LEFT JOIN instead of SubQuery when admin filters users by 2fa. revert non-en locale.

* use OptionalBool instead of status map

* refactor SearchUserOptions.toConds to SearchUserOptions.toSearchQueryBase

* add unit test for user search

* only allow admin to use filters to search users
2021-10-12 20:11:35 +02:00
Romain d0a681fbc3
[API] Add endpount to get user org permissions (#17232)
* Add endpoint

* Add swagger response + generate swagger

* Stop execution if user / org is not found

* Add tests


Co-authored-by: 6543 <6543@obermui.de>
2021-10-12 12:47:19 +02:00
wxiaoguang c5c88f2f18
Save and view issue/comment content history (#16909)
* issue content history

* Use timeutil.TimeStampNow() for content history time instead of issue/comment.UpdatedUnix (which are not updated in time)

* i18n for frontend

* refactor

* clean up

* fix refactor

* re-format

* temp refactor

* follow db refactor

* rename IssueContentHistory to ContentHistory, remove empty model tags

* fix html

* use avatar refactor to generate avatar url

* add unit test, keep at most 20 history revisions.

* re-format

* syntax nit

* Add issue content history table

* Update models/migrations/v197.go

Co-authored-by: 6543 <6543@obermui.de>

* fix merge

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-10-10 18:40:03 -04:00
a1012112796 bb39359668
Add a simple way to rename branch like gh (#15870)
- Update default branch if needed
- Update protected branch if needed
- Update all not merged pull request base branch name
- Rename git branch
- Record this rename work and auto redirect for old branch on ui

Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: delvh <dev.lh@web.de>
2021-10-08 19:03:04 +02:00
Jimmy Praet b6147152f8
Fix incorrect repository count on organization tab of dashboard (#17256)
Fixes #17249
2021-10-08 08:49:20 +02:00
Jimmy Praet 25a2000787
Fix unwanted team review request deletion (#17257)
Add missing issue_id = ? to where clause
Fixes #17251
2021-10-07 22:39:59 +02:00
zeripath 46cac263ca
Handle duplicate keys on GPG key ring (#17242)
It is possible that a keyring can contain duplicate keys on a keyring due to jpegs or
other layers. This currently leads to a confusing error for the user - where we report
a duplicate key insertion.

This PR simply coalesces keys into one key if there are duplicates.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
2021-10-07 23:10:14 +03:00
Jimmy Praet 29acb3213b
Fix broken Activities link in team dashboard (#17255)
Remove '/' suffix from organization dashboard link

Fixes #17250
2021-10-07 20:27:38 +02:00
finga 67bc04fe21
Fix problem when database ID is not incremented as expected (#17229)
Although #17124 fixed the same issue for the feed, some other parts
with the same issue were found.

Co-authored-by: finga <finga@onders.org>
2021-10-06 15:36:24 -05:00
wxiaoguang f0ba87fda8
Avatar refactor, move avatar code from models to models.avatars, remove duplicated code (#17123)
Why this refactor

The goal is to move most files from `models` package to `models.xxx` package. Many models depend on avatar model, so just move this first.

And the existing logic is not clear, there are too many function like `AvatarLink`, `RelAvatarLink`, `SizedRelAvatarLink`, `SizedAvatarLink`, `MakeFinalAvatarURL`, `HashedAvatarLink`, etc. This refactor make everything clear:

* user.AvatarLink()
* user.AvatarLinkWithSize(size)
* avatars.GenerateEmailAvatarFastLink(email, size)
* avatars.GenerateEmailAvatarFinalLink(email, size)

And many duplicated code are deleted in route handler, the handler and the model share the same avatar logic now.
2021-10-06 01:25:46 +02:00
Romain 987152ba40
Add metrics to get issues by repository (#17225) 2021-10-05 20:39:37 +02:00
pricly-yellow 20eaca6d05
Fix stange behavior of DownloadPullDiffOrPatch in incorect index (#17223)
Fix GetPullRequestByIndex by validate index > 1

Signed-off-by: Danila Kryukov <pricly_yellow@dismail.de>

Co-authored-by: a1012112796 <1012112796@qq.com>
2021-10-05 16:41:48 +02:00
Romain fc5ee1edf9
Add metrics to get issues by label (#17201)
* Add metrics to get issues by label

* Add comment on IssueByLabelCount

* Code review - Unify "AS" in SQL (#17201)

* Code review - Remove useless join (#17201)

* Code review - Disable issue_by_label by default in settings (#17201)

* use e

* restore empty line

* update docs

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-10-03 18:46:44 -04:00
Romain c64e2a319f
Add projects and project boards in exposed metrics (#17202)
* Add projects and project boards in exposed metrics

* Refactor db.GetEngine

Co-authored-by: delvh <dev.lh@web.de>
2021-10-02 03:32:15 +02:00
KN4CK3R 3b06675811
Always set a unique Message-ID header. (#17206) 2021-10-01 11:24:43 -04:00
Romain ecfac78f6e
Kanban colored boards (#16647)
Add a column Color in ProjectBoard and color picker in new / edit project board form.
2021-09-29 22:53:12 +02:00
zeripath e8574f2f7d
Nicely handle missing user in collaborations (#17049)
* Nicely handle missing user in collaborations

It is possible to have a collaboration in a repository which refers to a no-longer
existing user. This causes the repository transfer to fail with an unusual error.

This PR makes `repo.getCollaborators()` nicely handle the missing user by ghosting
the collaboration but also adds consistency check. It also adds an
Access consistency check.

Fix #17044

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2021-09-27 19:07:19 +01:00
zeripath b5856c4437
Create doctor command to fix repo_units broken by dumps from 1.14.3-1.14.6 (#17136)
There was a serious issue with the `gitea dump` command in 1.14.3-1.14.6 which led to corruption of the `config` field of the `repo_unit` table. 

This PR adds a doctor command to attempt to fix the broken repo_units. Users affected by #16961 should run:

```
gitea doctor --fix --run fix-broken-repo-units
```

Fix #16961

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-27 16:55:12 +01:00
zeripath 123f0aea00
Allow LDAP Sources to provide Avatars (#16851)
* Allow LDAP Sources to provide Avatars

Add setting to LDAP source to allow it to provide an Avatar.

Currently this is required to point to the image bytes.

Fix #4144

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Rename as Avatar Attribute (drop JPEG)

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Always synchronize avatar if there is change

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Actually get the avatar from the ldap

Signed-off-by: Andrew Thornton <art27@cantab.net>

* clean-up

Signed-off-by: Andrew Thornton <art27@cantab.net>

* use len()>0 rather than != ""

Signed-off-by: Andrew Thornton <art27@cantab.net>

* slight shortcut in IsUploadAvatarChanged

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-09-26 22:39:36 -04:00
Lunny Xiao 91e21d4fca
Move twofactor to models/login (#17143) 2021-09-25 15:00:12 +02:00
Lunny Xiao 5842a55b31
Move login related structs and functions to models/login (#17093)
* Move login related structs and functions to models/login

* Fix test

* Fix lint

* Fix lint

* Fix lint of windows

* Fix lint

* Fix test

* Fix test

* Only load necessary fixtures when preparing unit tests envs

* Fix lint

* Fix test

* Fix test

* Fix error log

* Fix error log

* Fix error log

* remove unnecessary change

* fix error log

* merge main branch
2021-09-24 19:32:56 +08:00
zeripath 9302eba971
DBContext is just a Context (#17100)
* DBContext is just a Context

This PR removes some of the specialness from the DBContext and makes it context
This allows us to simplify the GetEngine code to wrap around any context in future
and means that we can change our loadRepo(e Engine) functions to simply take contexts.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix unit tests

Signed-off-by: Andrew Thornton <art27@cantab.net>

* another place that needs to set the initial context

Signed-off-by: Andrew Thornton <art27@cantab.net>

* avoid race

Signed-off-by: Andrew Thornton <art27@cantab.net>

* change attachment error

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-23 23:45:36 +08:00
delvh b22be7f594
Fix typo skipping a migration (#17130) 2021-09-23 20:42:42 +08:00
Lunny Xiao d9c69596ff
Fix commit status index problem (#17061)
* Fix commit status index problem

* remove unused functions

* Add fixture and test for migration

* Fix lint

* Fix fixture

* Fix lint

* Fix test

* Fix bug

* Fix bug
2021-09-23 18:50:06 +08:00
Lunny Xiao d9e237e3f2
Fix problem when database id is not increment as expected (#17124) 2021-09-22 23:09:29 +03:00
Eng Zer Jun f2e7d5477f
refactor: move from io/ioutil to io and os package (#17109)
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-09-22 13:38:34 +08:00
Lunny Xiao a4bfef265d
Move db related basic functions to models/db (#17075)
* Move db related basic functions to models/db

* Fix lint

* Fix lint

* Fix test

* Fix lint

* Fix lint

* revert unnecessary change

* Fix test

* Fix wrong replace string

* Use *Context

* Correct committer spelling and fix wrong replaced words

Co-authored-by: zeripath <art27@cantab.net>
2021-09-19 19:49:59 +08:00
Norwin 0ffad31b92
Notifications API: respond with updated notifications (#17064)
* notifications api: return updated notifications in response

* make generate-swagger

* openapi fix

Co-authored-by: zeripath <art27@cantab.net>
2021-09-17 19:40:50 -04:00
Lunny Xiao e2f0ab3343
Add doctor dbconsistency check for release and attachment (#16978) 2021-09-14 20:41:40 +01:00
KN4CK3R 87505a9464
Add AbsoluteListOptions (#17028)
This PR adds a `ListOptions` type which is not paged but uses absolute values. It is implemented as discussed in Discord.
Extracted from #16510 to clean that PR.
2021-09-14 18:48:27 +01:00
zeripath 0a542dd59f
Decrement Fork Num when converting from Fork (#17035)
When converting repositories from forks to normal the root NumFork needs to be
decremented too.

Fix #17026

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-14 18:07:08 +01:00
zeripath 26ef180f46
Correctly rollback in ForkRepository (#17034)
The rollback functionality in
services/repository/repository.go:ForkRepository is incorrect and could
lead to a deadlock as it uses DeleteRepository to delete the rolled-back
repository - a function which creates its own transaction.

This PR adjusts the rollback function to only use RemoveAll as any
database changes will be automatically rolled-back. It also handles
panics and adjusts the Close within WithTx to ensure that if there is a
panic the session will always be closed.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-14 18:16:40 +02:00
Jimmy Praet 3d6cb25e31
Support unprotected file patterns (#16395)
Fixes #16381

Note that changes to unprotected files via the web editor still cannot be pushed directly to the protected branch. I could easily add such support for edits and deletes if needed. But for adding, uploading or renaming unprotected files, it is not trivial.

* Extract & Move GetAffectedFiles to modules/git
2021-09-11 16:21:17 +02:00
6543 8e18b412c3
Skip AllowedUserVisibilityModes validation on update user if it is an organisation (#16988)
if AllowedUserVisibilityModes allow only public & limited, and orgs can be private, a user can create a repo to that organisation whitch will result in an update of the user. On this call the user is validaten and will be rejected since private is not allowed, but its not an user its an valid org ...

Co-authored-by: Alexey 〒erentyev <axifnx@gmail.com>
2021-09-08 17:47:49 +02:00
Lunny Xiao ddc709ff7f
Add repo_id for attachment (#16958)
When create a new issue or comment and paste/upload an attachment/image, it will not assign an issue id before submit. So if user give up the creating, the attachments will lost key feature and become dirty content. We don't know if we need to delete the attachment even if the repository deleted.

This PR add a repo_id in attachment table so that even if a new upload attachment with no issue_id or release_id but should have repo_id. When deleting a repository, they could also be deleted.

Co-authored-by: 6543 <6543@obermui.de>
2021-09-08 17:19:30 +02:00
Lunny Xiao a807031a30
Fix storage Iterate bug and Add storage doctor to delete garbage attachments (#16971)
* Fix storage Iterate bug and Add storage doctor to delete garbage attachments

* Close object when used
2021-09-06 16:46:20 +02:00
zeripath 57b0887ab2
Correctly return the number of Repositories for Organizations (#16807)
Calculate and return the number of Repositories on the dashboard
Organization list.

This PR restores some of the logic that was removed in #14032 to
calculate the number of repos on the dashboard orgs list.

Fix #16648
Replaces #16799

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-09-01 01:31:42 -04:00
a1012112796 cbf05c3f79
Add option to update pull request by rebase (#16125)
* add option to update pull request by `rebase`

Signed-off-by: a1012112796 <1012112796@qq.com>
2021-08-31 16:03:45 +02:00
Lunny Xiao 8d7704b5a2
Fix dump and restore respository (#16698)
* Fix dump and restore

* return different error message for get commit

* Fix missing delete release attachment when deleting repository

* Fix ci and add some comments

Co-authored-by: zeripath <art27@cantab.net>
2021-08-30 21:18:15 -04:00
Kyle Evans cad70599a6
Refactor the fork service slightly to take ForkRepoOptions (#16744)
* Refactor the fork service slightly to take ForkRepoOptions

This reduces the number of places we need to change if we want to add other
options during fork time.

Signed-off-by: Kyle Evans <kevans@FreeBSD.org>

* Fix integrations and tests after ForkRepository refactor

Signed-off-by: Kyle Evans <kevans@FreeBSD.org>

* Update OldRepo -> BaseRepo

Signed-off-by: Kyle Evans <kevans@FreeBSD.org>

* gofmt pass

Signed-off-by: Kyle Evans <kevans@FreeBSD.org>
2021-08-28 16:37:14 +08:00
zeripath 1904941382
Add test to ensure that dumping of login sources remains correct (#16847)
#16831 has occurred because of a missed regression. This PR adds a simple test to
try to prevent this occuring again.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-28 04:25:27 +01:00
Clar Fon 29b971b6d5
Actually compute proper foreground color for labels (#16729) 2021-08-25 12:55:47 -05:00
Lunny Xiao b88dbe1208
Use a common quote to instead of check database type (#16817)
`` ` `` will be converted to different database quote by xorm. So check database type is unnecessary.
2021-08-25 13:23:34 +01:00
zeripath 697213bdb3
Add primary_key to issue_index (#16813)
Make the group_id a primary key in issue_index. This already has an unique index
and therefore is a good candidate for becoming a primary key.

This PR also changes all other uses of this table to add the group_id as the
primary key.

Fix #16802

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-25 09:42:51 +01:00
Lunny Xiao efaf109435
Add information for migrate failure (#16803)
Improve the reporting of errors when there is a migration failure
2021-08-24 18:46:55 +01:00
wxiaoguang b55c699c62
Alter issue/comment table TEXT fields to LONGTEXT (#16765)
* Alter issue/comment table TEXT fields to LONGTEXT

* Use If not Switch

Co-authored-by: zeripath <art27@cantab.net>
2021-08-22 23:33:05 +08:00
KN4CK3R 4debb74eda
Fix wrong user in OpenID response (#16736)
* Fixed usage of wrong user.

* Added tests.
2021-08-19 12:11:30 -04:00
zeripath c9bca8c5e0
Recreate Tables should Recreate indexes on MySQL (#16718)
The MySQL indexes are not being renamed at the same time as RENAME table despite the
CASCADE. Therefore it is probably better to just recreate the indexes instead.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-08-19 07:11:36 +03:00
zeripath e0853d4a21
Add API Token Cache (#16547)
One of the issues holding back performance of the API is the problem of hashing.
Whilst banning BASIC authentication with passwords will help, the API Token scheme
still requires a PBKDF2 hash - which means that heavy API use (using Tokens) can
still cause enormous numbers of hash computations.

A slight solution to this whilst we consider moving to using JWT based tokens and/or
a session orientated solution is to simply cache the successful tokens. This has some
security issues but this should be balanced by the security issues of load from
hashing.

Related #14668

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-08-17 14:30:42 -04:00
Jimmy Praet a4962a9440
Add filter by owner and team to issue/pulls search endpoint (#16662)
* Filter by owner and team in API issue/pulls search

* Add integration test
2021-08-13 22:47:25 +02:00
6543 3a6edd3685
Update issue_index to finish migration (#16685)
* update issue_index to finish migration

* One Func to RecalculateIssueIndexForRepo
2021-08-13 21:06:18 +08:00
Lunny Xiao 7224cfc578
Upgrade xorm to v1.2.2 (#16663)
* Upgrade xorm to v1.2.2

* Change the Engine interface to match xorm v1.2.2
2021-08-13 01:11:42 +02:00
6543 2289580bb7
[API] generalize list header (#16551)
* Add info about list endpoints to CONTRIBUTING.md

* Let all list endpoints return X-Total-Count header 

* Add TODOs for GetCombinedCommitStatusByRef

* Fix models/issue_stopwatch.go

* Rrefactor models.ListDeployKeys

* Introduce helper func and use them for SetLinkHeader related func
2021-08-12 14:43:08 +02:00
KN4CK3R d9ef43a712
Replace list.List with slices (#16311)
* Replaced list with slice.

* Fixed usage of pointer to temporary variable.

* Replaced LIFO list with slice.

* Lint

* Removed type check.

* Removed duplicated code.

* Lint

* Fixed merge.

Co-authored-by: 6543 <6543@obermui.de>
2021-08-09 14:08:51 -04:00
6543 eaa791bedd
Add Version info to migrations.go (#16653)
Gitea migrations of 1.15.0 ends at v189
2021-08-08 13:34:42 -04:00
zeripath 9c116f2bb5
Restore compatibility with SQLServer 2008 R2 in migrations (#16627)
* Restore compatibility with SQLServer 2008 R2 in migrations

`ALTER TABLE DROP ... IF EXISTS ...` is only supported in SQL Server >16.

The `IF EXISTS` here is a belt-and-braces and does not need to be present. Therefore
can be dropped.

We need to figure out some way of restricting our SQL syntax against the minimum
version of SQL Server we will support.

My suspicion is that `ALTER DATABASE database_name SET COMPATIBILITY_LEVEL = 100` may
do that but there may be other side-effects so I am not whether to do that.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* try just dropping the index only

Signed-off-by: Andrew Thornton <art27@cantab.net>

* use lowercase for system tables

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-08-08 13:34:24 +03:00
Romain 14762abf0b
Separate open and closed issue in metrics (#16637)
* Get the issue counts in one query

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-08-07 12:43:50 +03:00
a1012112796 3705168837
Add agit flow support in gitea (#14295)
* feature: add agit flow support

ref: https://git-repo.info/en/2020/03/agit-flow-and-git-repo/

example:

```Bash
git checkout -b test
echo "test" >> README.md
git commit -m "test"
git push origin HEAD:refs/for/master -o topic=test
```

Signed-off-by: a1012112796 <1012112796@qq.com>

* fix lint

* simplify code add fix some nits

* update merge help message

* Apply suggestions from code review. Thanks @jiangxin

* add forced-update message

* fix lint

* splite writePktLine

* add refs/for/<target-branch>/<topic-branch> support also

* Add test code add fix api

* fix lint

* fix test

* skip test if git version < 2.29

* try test with git 2.30.1

* fix permission check bug

* fix some nit

* logic implify and test code update

* fix bug

* apply suggestions from code review

* prepare for merge

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix permission check bug

- test code update
- apply suggestions from code review @zeripath

Signed-off-by: a1012112796 <1012112796@qq.com>

* fix bug when target branch isn't exist

* prevent some special push and fix some nits

* fix lint

* try splite

* Apply suggestions from code review

- fix permission check
- handle user rename

* fix version negotiation

* remane

* fix template

* handle empty repo

* ui: fix  branch link under the title

* fix nits

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-07-28 17:42:56 +08:00
a1012112796 9102738797
not show private user's repo in explore view (#16550)
after #16069, visibility is also usefull for user,
so this limit is not usefull.

fix #16545
2021-07-27 00:02:35 +02:00
6543 7c7771e42b
Fix session bugs (#16552)
* fix deadlog bug

* Fix models/issue_stopwatch.go

* Update models/issue_stopwatch.go

Co-authored-by: zeripath <art27@cantab.net>
2021-07-26 23:46:06 +03:00
zeripath 6a33b290a0
Fix add authentication page (#16543)
* Fix add authentication page

There is a regression in #16199 whereby the add authentication page
fails to react to the change in selected type.

This is due to the String() method on the LoginSourceType which is ameliorated
with an Int() function being added.

Following on from this there are a few other related bugs.

Fix #16541

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-25 08:09:52 +01:00
Lunny Xiao 9f31f3aa8a
Add an abstract json layout to make it's easier to change json library (#16528)
* Add an abstract json layout to make it's easier to change json library

* Fix import

* Fix import sequence

* Fix blank lines

* Fix blank lines
2021-07-24 18:03:58 +02:00
zeripath 5d2e11eedb
Refactor: Move login out of models (#16199)
`models` does far too much. In particular it handles all `UserSignin`.

It shouldn't be responsible for calling LDAP, SMTP or PAM for signing in.

Therefore we should move this code out of `models`.

This code has to depend on `models` - therefore it belongs in `services`.

There is a package in `services` called `auth` and clearly this functionality belongs in there.

Plan:

- [x] Change `auth.Auth` to `auth.Method` - as they represent methods of authentication.
- [x] Move `models.UserSignIn` into `auth`
- [x] Move `models.ExternalUserLogin`
- [x] Move most of the `LoginVia*` methods to `auth` or subpackages
- [x] Move Resynchronize functionality to `auth`
  - Involved some restructuring of `models/ssh_key.go` to reduce the size of this massive file and simplify its files.
- [x] Move the rest of the LDAP functionality in to the ldap subpackage
- [x] Re-factor the login sources to express an interfaces `auth.Source`?
  - I've done this through some smaller interfaces Authenticator and Synchronizable - which would allow us to extend things in future
- [x] Now LDAP is out of models - need to think about modules/auth/ldap and I think all of that functionality might just be moveable
- [x] Similarly a lot Oauth2 functionality need not be in models too and should be moved to services/auth/source/oauth2
  - [x] modules/auth/oauth2/oauth2.go uses xorm... This is naughty - probably need to move this into models.
  - [x] models/oauth2.go - mostly should be in modules/auth/oauth2 or services/auth/source/oauth2 
- [x] More simplifications of login_source.go may need to be done
- Allow wiring in of notify registration -  *this can now easily be done - but I think we should do it in another PR*  - see #16178 
- More refactors...?
  - OpenID should probably become an auth Method but I think that can be left for another PR
  - Methods should also probably be cleaned up  - again another PR I think.
  - SSPI still needs more refactors.* Rename auth.Auth auth.Method
* Restructure ssh_key.go

- move functions from models/user.go that relate to ssh_key to ssh_key
- split ssh_key.go to try create clearer function domains for allow for
future refactors here.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-24 11:16:34 +01:00
lengyuqu 078e2b2c39
Add support for corporate WeChat webhooks (#15910)
* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* Update templates/admin/hook_new.tmpl

Co-authored-by: a1012112796 <1012112796@qq.com>

* Update services/webhook/wechatwork.go

Co-authored-by: a1012112796 <1012112796@qq.com>

* 修善wechatwork

* 修善wechatwork

* fix

* Update locale_cs-CZ.ini

fix

* fix build

* fix

* fix build

* make webhooks.zh-cn.md

* delet unnecessary blank line

* delet unnecessary blank line

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* Update templates/admin/hook_new.tmpl

Co-authored-by: a1012112796 <1012112796@qq.com>

* Update services/webhook/wechatwork.go

Co-authored-by: a1012112796 <1012112796@qq.com>

* 修善wechatwork

* 修善wechatwork

* fix

* fix build

* fix

* fix build

* make webhooks.zh-cn.md

* delet unnecessary blank line

* delet unnecessary blank line

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* fix

* fix

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* fix wechat

* fix wechat

* fix wechat

* fix wechat

* Fix invalid params and typo of email templates (#16394)

Signed-off-by: Meano <meanocat@gmail.com>

* Add LRU mem cache implementation (#16226)

The current default memory cache implementation is unbounded in size and number of
objects cached. This is hardly ideal.

This PR proposes creating a TwoQueue LRU cache as the underlying cache for Gitea.
The cache is limited by the number of objects stored in the cache (rather than size)
for simplicity. The default number of objects is 50000 - which is perhaps too small
as most of our objects cached are going to be much less than 1kB.

It may be worth considering using a different LRU implementation that actively limits
sizes or avoids GC - however, this is just a beginning implementation.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* [skip ci] Updated translations via Crowdin

* Replace `plugins/docker` with `techknowlogick/drone-docker`in ci (#16407)

* plugins/docker -> techknowlogick/drone-docker

* It is multi-arch

* docs: rewrite email setup (#16404)

* Add intro for both the docs page and mailer methods
  * Fix numbering level in SMTP section
  * Recommends implicit TLS

Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>

* Validate Issue Index before querying DB (#16406)

* Fix external renderer (#16401)

* fix external renderer

* use GBackground context as fallback

* no fallback, return error

Co-authored-by: Lauris BH <lauris@nix.lv>

* Add checkbox to delete pull branch after successful merge (#16049)

* Add checkbox to delete pull branch after successful merge

* Omit DeleteBranchAfterMerge field in json

* Log a warning instead of error when PR head branch deleted

* Add DefaultDeleteBranchAfterMerge to PullRequestConfig

* Add support for delete_branch_after_merge via API

* Fix for API: the branch should be deleted from the HEAD repo

If head and base repo are the same, reuse the already opened ctx.Repo.GitRepo

* Don't delegate to CleanupBranch, only reuse branch deletion code

CleanupBranch contains too much logic that has already been performed by the Merge

* Reuse gitrepo in MergePullRequest

Co-authored-by: Andrew Thornton <art27@cantab.net>

* [skip ci] Updated translations via Crowdin

* Detect encoding changes while parsing diff (#16330)

* Detect encoding changes while parsing diff

* Let branch/tag name be a valid ref to get CI status (#16400)

* fix #16384#

* refactor: move shared helper func to utils package

* extend Tests

* use ctx.Repo.GitRepo if not nil

* fix

* fix

* 企业微信webhook

* 企业微信webhook

* 企业微信webhook

* fix build

* fix build

* Apply suggestions from code review

Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: myheavily <myheavily>
Co-authored-by: zhaoxin <gitea@fake.local>
Co-authored-by: Meano <Meano@foxmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: GiteaBot <teabot@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Bagas Sanjaya <bagasdotme@gmail.com>
Co-authored-by: Norwin <noerw@users.noreply.github.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Jimmy Praet <jimmy.praet@telenet.be>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-07-23 12:41:27 +08:00
zeripath 1ce4fb256f
Restore creation of git-daemon-export-ok files (#16508)
Somewhere along the line the creation of git-daemon-export-ok
files disappeared but the updating of these files when
repo visibility changes remained. The problem is that the
current state will create files even when the org or user
is private.

This PR restores creation correctly.

Fix #15521

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-22 12:53:54 +01:00
zeripath 2635778425
Add basic edit ldap auth test & actually fix #16252 (#16465)
One of the reasons why #16447 was needed and why #16268 was needed in
the first place was because it appears that editing ldap configuration
doesn't get tested.

This PR therefore adds a basic test that will run the edit pipeline.

In doing so it's now clear that #16447 and #16268 aren't actually
solving #16252. It turns out that what actually happens is that is that
the bytes are actually double encoded.

This PR now changes the json unmarshal wrapper to handle this double
encode.

Fix #16252

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-07-20 14:30:22 +03:00
zeripath fdb0e82148
Fix crash following ldap authentication update (#16447)
Unfortunately #16268 contained a terrible error, whereby there was a double
indirection taken when unmarshalling the source data. This fatally breaks
authentication configuration reading.

Fix #16342

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-07-15 20:17:51 -04:00
zeripath 33a8eec33e
Retry rename on lock induced failures (#16435)
* Retry rename on lock induced failures

Due to external locking on Windows it is possible for an
os.Rename to fail if the files or directories are being
used elsewhere.

This PR simply suggests retrying the rename again similar
to how we handle the os.Remove problems.

Fix #16427

Signed-off-by: Andrew Thornton <art27@cantab.net>

* resolve CI fail

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-07-15 11:46:07 -04:00
Meano 423a0fccb6
Fix activation of primary email addresses (#16385)
* fix: primary email cannot be activated

* Primary email should be activated together with user account when
'RegisterEmailConfirm' is enabled.

* To fix the existing error state. When 'RegisterEmailConfirm' is enabled, the
admin should have permission to modify the activations status of user email.
And the user should be allowed to send activation to primary email.

* Only judge whether email is primary from email_address table.

* Improve logging and refactor isEmailActive

Co-authored-by: zeripath <art27@cantab.net>
2021-07-13 22:59:27 +02:00
zeripath b82293270c
Add option to provide signature for a token to verify key ownership (#14054)
* Add option to provide signed token to verify key ownership

Currently we will only allow a key to be matched to a user if it matches
an activated email address. This PR provides a different mechanism - if
the user provides a signature for automatically generated token (based
on the timestamp, user creation time, user ID, username and primary
email.

* Ensure verified keys can act for all active emails for the user

* Add code to mark keys as verified

* Slight UI adjustments

* Slight UI adjustments 2

* Simplify signature verification slightly

* fix postgres test

* add api routes

* handle swapped primary-keys

* Verify the no-reply address for verified keys

* Only add email addresses that are activated to keys

* Fix committer shortcut properly

* Restructure gpg_keys.go

* Use common Verification Token code

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-13 15:28:07 +02:00
Lunny Xiao 67f135ca5d
Fix archive error when rename repo or user (#16399)
Use repo id instead of full name to generate archive path
2021-07-13 14:16:31 +02:00
Jimmy Praet 78118a3b02
Add checkbox to delete pull branch after successful merge (#16049)
* Add checkbox to delete pull branch after successful merge

* Omit DeleteBranchAfterMerge field in json

* Log a warning instead of error when PR head branch deleted

* Add DefaultDeleteBranchAfterMerge to PullRequestConfig

* Add support for delete_branch_after_merge via API

* Fix for API: the branch should be deleted from the HEAD repo

If head and base repo are the same, reuse the already opened ctx.Repo.GitRepo

* Don't delegate to CleanupBranch, only reuse branch deletion code

CleanupBranch contains too much logic that has already been performed by the Merge

* Reuse gitrepo in MergePullRequest

Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-07-13 01:26:25 +02:00
Norwin fb04cc7dbd
Validate Issue Index before querying DB (#16406) 2021-07-12 16:22:27 -04:00
luzpaz e0296b6a6d
Fix various documentation, user-facing, and source comment typos (#16367)
* Fix various doc, user-facing, and source comment typos

Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby`
2021-07-08 13:38:13 +02:00
6543 290f458d46
Reserve user/repo pattern for rss feature (#16323) 2021-07-01 23:13:20 +08:00
Adyanth H 66bf74d1b9
Escape reference to user table in models.SearchEmails (#16313)
Fix #16312

Signed-off-by: Adyanth H <adyanthh@gmail.com>
2021-06-30 18:37:20 +02:00
sebastian-sauer e8c6cead0f
Fix list_options GetStartEnd (#16303)
end is start + pageSize and not start + page
2021-06-29 22:42:23 +01:00
6543 dea7a5c5b9
just add some unit tests (#16291)
* code.gitea.io/gitea/routers/utils coverage: 100.0%

* code.gitea.io/gitea/routers/install 0% -> 5.0%

* ConvertUtf8ToUtf8mb4: make sure DBType is mysql
2021-06-29 22:00:02 +01:00
zeripath 9a0cd3af3b
Handle misencoding of login_source cfg in mssql (#16268)
* Handle misencoding of login_source cfg in mssql

Unfortunately due a bug in xorm (see https://gitea.com/xorm/xorm/pulls/1957) updating
loginsources on MSSQL causes them to become corrupted. (#16252)

Whilst waiting for the referenced PR to be merged and to handle the corrupted
loginsources correctly we need to add a wrapper to the `FromDB()` methods to look
for and ignore the misplaced BOMs that have been added.

Fix #16252

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update models/login_source.go
2021-06-27 16:21:16 -04:00
KN4CK3R 9b1b4b5433
Refactor Webhook + Add X-Hub-Signature (#16176)
This PR removes multiple unneeded fields from the `HookTask` struct and adds the two headers `X-Hub-Signature` and `X-Hub-Signature-256`.

## ⚠️ BREAKING ⚠️ 

* The `Secret` field is no longer passed as part of the payload.
* "Breaking" change (or fix?): The webhook history shows the real called url and not the url registered in the webhook (`deliver.go`@129).

Close #16115
Fixes #7788
Fixes #11755

Co-authored-by: zeripath <art27@cantab.net>
2021-06-27 20:21:09 +01:00