Commit graph

3033 commits

Author SHA1 Message Date
6543 e14b52ea36
Enable Malayalam, Greek, Persian, Hungarian & Indonesian by default (#16998) 2021-09-09 14:16:42 +08:00
Lunny Xiao efdbba4453
Fix bug of migrated repository not index (#16991)
Fix #16986, #16152
2021-09-08 18:43:19 +01:00
6543 f55cd033ed
Make TestOneDevDownloadRepo work again & more resistant (#16987)
* Update TestOneDevDownloadRepo since OneDev api changed

* Use test functions for migrations
2021-09-08 16:17:22 +02:00
6543 63d7cbcef1
Make mirror feature more configurable (#16957)
Rename`[repository]` `DISABLE_MIRRORS` to `[mirror]` `DISABLE_NEW_PULL`  and add `ENABLED` and `DISABLE_NEW_PUSH` with the below meanings:

- `ENABLED`: **true**: Enables the mirror functionality. Set to **false** to disable all mirrors.
- `DISABLE_NEW_PULL`: **false**: Disable the creation of **new** mirrors. Pre-existing mirrors remain valid.
- `DISABLE_NEW_PUSH`: **false**: Disable the creation of **new** push mirrors. Pre-existing mirrors remain valid.


Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
2021-09-07 17:49:36 +02:00
6543 cf6d398485
Resolve TODO: Enable pagination on GiteaDownloader.GetComments() & update another TODO (#16963)
* Update TODO in migrations

* Resolve TODO: enable pagination on GiteaDownloader.GetComments()
2021-09-06 18:00:50 +02:00
Lunny Xiao a807031a30
Fix storage Iterate bug and Add storage doctor to delete garbage attachments (#16971)
* Fix storage Iterate bug and Add storage doctor to delete garbage attachments

* Close object when used
2021-09-06 16:46:20 +02:00
a1012112796 82da380af7
Add missing icon set for ActionRenameRepo (#16972)
Signed-off-by: a1012112796 <1012112796@qq.com>
2021-09-06 16:11:17 +02:00
zeripath 076bb8a2a2
Close storage objects before cleaning (#16934)
Storage.Iterate provides the path and an open object. On windows using
local storage means that the objects will be locked thus preventing clean
from deleting them.

This PR simply closes the objects early.

Fix #16932

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-03 11:48:49 +01:00
zeripath 6c125e9797
Use immediate queues in integration tests and ensure that immediate (#16927)
queue type is also used for unique queues.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-03 11:20:57 +01:00
Lunny Xiao de2e96e37b
Ignore review comment when ref commit is missed (#16905) 2021-09-01 12:33:07 +01:00
6543 7062614dee
Gitlab Migrator: dont ignore reactions of last request (#16903)
Fix bug related to early breaking when migrating reactions.
2021-09-01 11:20:19 +01:00
6543 d21702475b
Fix git.Blob.DataAsync(): close pipe since we return a NopCloser (#16899)
* make sure headGitRepo is closed on err too

* refactor

* Fix git.Blob.DataAsync(): exec cancel since we already read all bytes (close pipe since we return a NopCloser)
2021-08-31 08:43:31 +01:00
zeripath f2b4b0f491
Remove ParseQueueConnStr as it is unused (#16878)
Remove ParseQueueConnStr as `modules/nosql` has taken over all of its functions.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-31 05:33:16 +02:00
Lunny Xiao 8d7704b5a2
Fix dump and restore respository (#16698)
* Fix dump and restore

* return different error message for get commit

* Fix missing delete release attachment when deleting repository

* Fix ci and add some comments

Co-authored-by: zeripath <art27@cantab.net>
2021-08-30 21:18:15 -04:00
zeripath 06b9d553bc
Timeout on flush in testing (#16864)
* Timeout on flush in testing

At the end of each test the queues are flushed. At present there is no limit on the
length of time a flush can take which can lead to long flushes.

However, if the CI task is cancelled we lose the log information as to where the long
flush was taking place.

This PR simply adds a default time limit of 2 minutes - at which point an error will
be produced. This should allow us to more easily find the culprit.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* return better error

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-08-30 00:27:51 -04:00
Lunny Xiao d985d4bc2f
Paginate releases page & set default page size to 10 (#16857)
* Add release default page and set it to 10

* use limit

Co-authored-by: 6543 <6543@obermui.de>
2021-08-29 18:25:16 +02:00
zeripath f5b0e2c9d2
Simplify split diff view generation and remove JS dependency (#16775)
Gitea has relied on some slow JS code to match up added and deleted lines on the
diff pages. This can cause a considerable slow down on large diff pages.

This PR makes a small change meaning that the matching up can occur much more simply.

Partial fix #1351

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-29 15:28:04 +01:00
zeripath c9c0475f4d
In Render tolerate not being passed a context (#16842)
* In Render tolerate not being passed a context

It is possible for RenderString to be passed to an external renderer if markdown
is set to be rendered by an external renderer. No context is currently sent to these
meaning that this will error out.

Fix #16835

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add Context to Repo calls for RenderString

All calls from routers can easily add the context - so add it.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-28 21:15:56 +01:00
Kyle Evans cad70599a6
Refactor the fork service slightly to take ForkRepoOptions (#16744)
* Refactor the fork service slightly to take ForkRepoOptions

This reduces the number of places we need to change if we want to add other
options during fork time.

Signed-off-by: Kyle Evans <kevans@FreeBSD.org>

* Fix integrations and tests after ForkRepository refactor

Signed-off-by: Kyle Evans <kevans@FreeBSD.org>

* Update OldRepo -> BaseRepo

Signed-off-by: Kyle Evans <kevans@FreeBSD.org>

* gofmt pass

Signed-off-by: Kyle Evans <kevans@FreeBSD.org>
2021-08-28 16:37:14 +08:00
zeripath ba6baff696
Report the correct number of pushes on the feeds (#16811)
* Report the correct number of pushes on the feeds

Since the number of commits in the Action table has been limited to 5
the number of commits reported on the feeds page is now incorrectly also
limited to 5. The correct number is available as the Len and this PR
changes this to report this.

Fix #16804

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update templates/user/dashboard/feeds.tmpl

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-08-25 19:04:58 -04:00
zeripath e891d68dd7
Prevent NPE on empty commit (#16812) 2021-08-24 19:33:23 -04:00
John Olheiser 648464b504
Add bundle download for repository (#14538)
* Add bundle download

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Fix fmt

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Fix build tags

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Download specific commit

Signed-off-by: jolheiser <john.olheiser@gmail.com>
2021-08-24 11:47:09 -05:00
zeripath f31e7a67cf
Just use a slice when rendering file (#16774)
Highlight currently uses a map which is memory inefficient. Switch to use a slice instead.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-23 22:23:39 +01:00
KN4CK3R cee5f7c5e2
Add migrate from OneDev (#16356)
* Use context to simplify logic.

* Added migration from OneDev.
This PR adds [OneDev](https://code.onedev.io/) as migration source.

Supported:
- [x] Milestones
- [x] Issues
- [x] Pull Requests
- [x] Comments
- [x] Reviews
- [x] Labels
2021-08-22 00:47:45 +02:00
zeripath 03937891e2
Return nil proxy function if proxy not enabled (#16742)
Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-19 16:41:20 -04:00
Lunny Xiao f9acad82ca
Add proxy settings and support for migration and webhook (#16704)
* Add proxy settings and support for migration and webhook

* Fix default value

* Add newline for example ini

* Add lfs proxy support

* Fix lint

* Follow @zeripath's review

* Fix git clone

* Fix test

* missgin http requests for proxy

* use empty

Co-authored-by: zeripath <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
2021-08-18 21:10:39 +08:00
KN4CK3R 422c30d315
Refactored and fixed migration tests. (#16714)
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-08-17 20:47:18 -04:00
zeripath e0853d4a21
Add API Token Cache (#16547)
One of the issues holding back performance of the API is the problem of hashing.
Whilst banning BASIC authentication with passwords will help, the API Token scheme
still requires a PBKDF2 hash - which means that heavy API use (using Tokens) can
still cause enormous numbers of hash computations.

A slight solution to this whilst we consider moving to using JWT based tokens and/or
a session orientated solution is to simply cache the successful tokens. This has some
security issues but this should be balanced by the security issues of load from
hashing.

Related #14668

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-08-17 14:30:42 -04:00
Lunny Xiao 74d75eb69c
Fix spelling of HookProcReceiveResult (#16690) 2021-08-14 13:17:10 +02:00
zeripath 23a87a003e
Ensure empty lines are copiable and final new line too (#16678)
* Ensure empty lines are copiable and final new line too

When files are highlighted the newline character needs to be added in a whitespace
compliant mode. Also ensure the final empty newline is rendered.

Fix #16434

* Add test and ensure spans closed

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-14 01:16:56 +02:00
6543 3a6edd3685
Update issue_index to finish migration (#16685)
* update issue_index to finish migration

* One Func to RecalculateIssueIndexForRepo
2021-08-13 21:06:18 +08:00
zeripath 5fbccad906
Fix NPE in fuzzer (#16680)
The fuzzer found an issue with the issue pattern processor where there is a spurious
path.Clean which does not need to be there. This PR also sets the default AppURL for
the fuzzer too.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-13 00:22:05 +02:00
6543 2289580bb7
[API] generalize list header (#16551)
* Add info about list endpoints to CONTRIBUTING.md

* Let all list endpoints return X-Total-Count header 

* Add TODOs for GetCombinedCommitStatusByRef

* Fix models/issue_stopwatch.go

* Rrefactor models.ListDeployKeys

* Introduce helper func and use them for SetLinkHeader related func
2021-08-12 14:43:08 +02:00
nitul1991 2d25b7d44b
Add an api endpoint to fetch git notes (#15373) (#16649)
close #15373
2021-08-11 03:01:40 +02:00
6543 c4d70a0325
Rename ctx.Form() to ctx.FormString() and move code into own file (#16571)
Followup from #16562 prepare for #16567

* Rename ctx.Form() to ctx.FormString()
* Reimplement FormX func to need less code and cpu cycles
* Move code into own file
2021-08-11 02:31:13 +02:00
6543 89245ee309
Upgrade github.com/google/go-github v32.1.0 -> v37.0.0 (#16661)
* Upgrade github.com/google/go-github vv32.1.0 -> v37.0.0

* refactor: use GetX() func to reduce code
2021-08-10 11:49:43 +02:00
KN4CK3R d9ef43a712
Replace list.List with slices (#16311)
* Replaced list with slice.

* Fixed usage of pointer to temporary variable.

* Replaced LIFO list with slice.

* Lint

* Removed type check.

* Removed duplicated code.

* Lint

* Fixed merge.

Co-authored-by: 6543 <6543@obermui.de>
2021-08-09 14:08:51 -04:00
nitul1991 23d438f565
Change the implementation of the go-git version of GetNote to mirror the non go-git version when passed a non-existent commit (#16658)
Fixes #16657
2021-08-09 16:24:34 +01:00
Romain 14762abf0b
Separate open and closed issue in metrics (#16637)
* Get the issue counts in one query

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-08-07 12:43:50 +03:00
zeripath afd88a2418
Allow setting X-FRAME-OPTIONS (#16643)
* Allow setting X-FRAME-OPTIONS

This PR provides a mechanism to set the X-FRAME-OPTIONS header.

Fix #7951

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md

Co-authored-by: John Olheiser <john.olheiser@gmail.com>

Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2021-08-06 16:47:10 -04:00
Richard Mahn fd39e6dc6a
Fixes #16557 - duplicate csv import (#16631)
Remove the unnecessary duplicate package import for `encoding/csv` in `modules/csv/csv.go`

Fix #16557
2021-08-05 17:56:11 +01:00
zeripath 7c4172ef71
Pass down SignedUserName down to AccessLogger context (#16605)
* Pass down SignedUserName down to AccessLogger context

Unfortunately when the AccessLogger was moved back before the contexters the
SignedUserName reporting was lost. This is due to Request.WithContext leading to a
shallow copy of the Request and the modules/context/Context being within that request.

This PR adds a new context variable of a string pointer which is set and handled
in the contexters.

Fix #16600

Signed-off-by: Andrew Thornton <art27@cantab.net>

* handle nil ptr issue

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-08-04 13:26:30 -04:00
zeripath 1fc7d6d6ad
Fix table alignment in markdown (#16596)
Set the TableOptions in markdown to allow alignment of the tables to work correctly

Fix #15959

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-02 17:11:18 +01:00
Lunny Xiao 33e0b38287
Rename context.Query to context.Form (#16562) 2021-07-29 03:42:15 +02:00
a1012112796 3705168837
Add agit flow support in gitea (#14295)
* feature: add agit flow support

ref: https://git-repo.info/en/2020/03/agit-flow-and-git-repo/

example:

```Bash
git checkout -b test
echo "test" >> README.md
git commit -m "test"
git push origin HEAD:refs/for/master -o topic=test
```

Signed-off-by: a1012112796 <1012112796@qq.com>

* fix lint

* simplify code add fix some nits

* update merge help message

* Apply suggestions from code review. Thanks @jiangxin

* add forced-update message

* fix lint

* splite writePktLine

* add refs/for/<target-branch>/<topic-branch> support also

* Add test code add fix api

* fix lint

* fix test

* skip test if git version < 2.29

* try test with git 2.30.1

* fix permission check bug

* fix some nit

* logic implify and test code update

* fix bug

* apply suggestions from code review

* prepare for merge

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix permission check bug

- test code update
- apply suggestions from code review @zeripath

Signed-off-by: a1012112796 <1012112796@qq.com>

* fix bug when target branch isn't exist

* prevent some special push and fix some nits

* fix lint

* try splite

* Apply suggestions from code review

- fix permission check
- handle user rename

* fix version negotiation

* remane

* fix template

* handle empty repo

* ui: fix  branch link under the title

* fix nits

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-07-28 17:42:56 +08:00
zeripath fd15fd4c67
Handle too long PR titles correctly (#16517)
The CompareAndPullRequestPost handler for POST to /compare
incorrectly handles returning errors to the user. For a start
it does not set the necessary markers to switch SimpleMDE
but it also does not immediately return to the form.

This PR fixes this by setting the appropriate values, fixing
the templates and preventing the suggestion of a too long
title.

Fix #16507

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-25 03:59:27 +01:00
Lunny Xiao 9f31f3aa8a
Add an abstract json layout to make it's easier to change json library (#16528)
* Add an abstract json layout to make it's easier to change json library

* Fix import

* Fix import sequence

* Fix blank lines

* Fix blank lines
2021-07-24 18:03:58 +02:00
techknowlogick 29a22ade82
switch to maintained lib (#16532)
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-07-24 13:00:41 +02:00
zeripath 5d2e11eedb
Refactor: Move login out of models (#16199)
`models` does far too much. In particular it handles all `UserSignin`.

It shouldn't be responsible for calling LDAP, SMTP or PAM for signing in.

Therefore we should move this code out of `models`.

This code has to depend on `models` - therefore it belongs in `services`.

There is a package in `services` called `auth` and clearly this functionality belongs in there.

Plan:

- [x] Change `auth.Auth` to `auth.Method` - as they represent methods of authentication.
- [x] Move `models.UserSignIn` into `auth`
- [x] Move `models.ExternalUserLogin`
- [x] Move most of the `LoginVia*` methods to `auth` or subpackages
- [x] Move Resynchronize functionality to `auth`
  - Involved some restructuring of `models/ssh_key.go` to reduce the size of this massive file and simplify its files.
- [x] Move the rest of the LDAP functionality in to the ldap subpackage
- [x] Re-factor the login sources to express an interfaces `auth.Source`?
  - I've done this through some smaller interfaces Authenticator and Synchronizable - which would allow us to extend things in future
- [x] Now LDAP is out of models - need to think about modules/auth/ldap and I think all of that functionality might just be moveable
- [x] Similarly a lot Oauth2 functionality need not be in models too and should be moved to services/auth/source/oauth2
  - [x] modules/auth/oauth2/oauth2.go uses xorm... This is naughty - probably need to move this into models.
  - [x] models/oauth2.go - mostly should be in modules/auth/oauth2 or services/auth/source/oauth2 
- [x] More simplifications of login_source.go may need to be done
- Allow wiring in of notify registration -  *this can now easily be done - but I think we should do it in another PR*  - see #16178 
- More refactors...?
  - OpenID should probably become an auth Method but I think that can be left for another PR
  - Methods should also probably be cleaned up  - again another PR I think.
  - SSPI still needs more refactors.* Rename auth.Auth auth.Method
* Restructure ssh_key.go

- move functions from models/user.go that relate to ssh_key to ssh_key
- split ssh_key.go to try create clearer function domains for allow for
future refactors here.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-24 11:16:34 +01:00
zeripath f135a818f5
Make Mermaid.js limit configurable (#16519)
* Make Mermaid.js limit configurable

Add `MERMAID_MAX_SOURCE_CHARACTERS` to `[markup]` settings
to make the maximum size of a mermaid render configurable.

Fix #16513

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fixup! Make Mermaid.js limit configurable

* Update custom/conf/app.example.ini

Co-authored-by: silverwind <me@silverwind.io>

* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md

Co-authored-by: silverwind <me@silverwind.io>

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-07-24 00:21:51 -04:00