Commit graph

3244 commits

Author SHA1 Message Date
zeripath ab7f701671
Make WrappedQueues and PersistableChannelUniqueQueues Pausable (#18393)
Implements the Pausable interface on WrappedQueues and PersistableChannelUniqueQueues

Reference #15928

Signed-off-by: Andrew Thornton art27@cantab.net
2022-01-24 22:54:35 +00:00
Gusted c2e13fb763
Fix partial cloning a repo (#18373)
- Pass the Global command args into serviceRPC.
- Fixes error with partial cloning.
- Add partial clone test
- Include diff

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-01-23 22:19:32 +01:00
Dr. Tobias Quathamer 3349fd8f79
Add packagist webhook (#18224)
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-01-23 21:46:30 +08:00
wxiaoguang 87141b908d
Fix mime-type detection for HTTP server (#18370)
Bypass the unstable behavior of Golang's mime.TypeByExtension
2022-01-23 20:19:49 +08:00
Lunny Xiao 35fdefc1ff
Always use git command but not os.Command (#18363) 2022-01-23 00:57:52 -05:00
Gusted 1ddfa596c8
Make gitea, gitea-vet future-proof (#18361)
* Make gitea, gitea-vet future-proof

- Ref: https://gitea.com/gitea/gitea-vet/pulls/18

* Correct order

Co-authored-by: zeripath <art27@cantab.net>
2022-01-22 21:59:34 +00:00
zeripath a82fd98d53
Pause queues (#15928)
* Start adding mechanism to return unhandled data

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Create pushback interface

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add Pausable interface to WorkerPool and Manager

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Implement Pausable and PushBack for the bytefifos

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Implement Pausable and Pushback for ChannelQueues and ChannelUniqueQueues

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Wire in UI for pausing

Signed-off-by: Andrew Thornton <art27@cantab.net>

* add testcases and fix a few issues

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix build

Signed-off-by: Andrew Thornton <art27@cantab.net>

* prevent "race" in the test

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix jsoniter mismerge

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix conflicts

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix format

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add warnings for no worker configurations and prevent data-loss with redis/levelqueue

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Use StopTimer

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-01-22 21:22:14 +00:00
Gusted 27ee01e1e8
Disable content sniffing on PlainTextBytes (#18359)
- Disable the browser's function to "sniff" for the content-type on the
provided plain text, this will prevent the possible usage of
user-controlled data being sent, which could be malicious.

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-01-22 13:32:35 -05:00
wxiaoguang 5ff899d5ac
Fix route wrap (#18360) 2022-01-22 15:05:22 +00:00
qwerty287 1f40933d38
Add config options to hide issue events (#17414)
* Add config option to hide issue events
Adds a config option `HIDE_ISSUE_EVENTS` to hide most issue events (changed labels, milestones, projects...) on the issue detail page.
If this is true, only the following events (comment types) are shown:
* plain comments
* closed/reopned/merged
* reviews

* Make configurable using a list

* Add docs

* Add missing newline

* Fix merge issues

* Allow changes per user settings

* Fix lint

* Rm old docs

* Apply suggestions from code review

* Use bitsets

* Rm comment

* fmt

* Fix lint

* Use variable/constant to provide key

* fmt

* fix lint

* refactor

* Add a prefix for user setting key

* Add license comment

* Add license comment

* Update services/forms/user_form_hidden_comments.go

Co-authored-by: Gusted <williamzijl7@hotmail.com>

* check len == 0

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: 6543 <6543@obermui.de>
2022-01-21 18:59:26 +01:00
6543 54e9ee37a7
format with gofumpt (#18184)
* gofumpt -w -l .

* gofumpt -w -l -extra .

* Add linter

* manual fix

* change make fmt
2022-01-20 18:46:10 +01:00
Gusted 1d98d205f5
Enable deprecation error for v1.17.0 (#18341)
Co-authored-by: Andrew Thornton <art27@cantab.net>
2022-01-20 18:00:38 +01:00
wxiaoguang 5bf8d5445e
Refactor Router Logger (#17308)
Make router logger more friendly, show the related function name/file/line.

[BREAKING]
This PR substantially changes the logging format of the router logger. If you use this logging for monitoring e.g. fail2ban you will need to update this to match the new format.
2022-01-20 19:41:25 +08:00
Jelle Hulter bbd30787d3
Updated Chroma to v0.10.0 (#18270) 2022-01-20 17:59:15 +08:00
Gusted 6c0fe1f3ec
refactor httplib (#18338)
- Remove a lot of unused code(most if not all were introduced in gogs
for webhooks usages).

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2022-01-19 19:31:39 -05:00
zeripath 5cb0c9aa0d
Propagate context and ensure git commands run in request context (#17868)
This PR continues the work in #17125 by progressively ensuring that git
commands run within the request context.

This now means that the if there is a git repo already open in the context it will be used instead of reopening it.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-01-19 23:26:57 +00:00
Peter Gardfjäll 89a3214352
Add MirrorUpdated field to Repository API type (#18267)
Add the last update time to the repository api type.

Close #18266
2022-01-18 13:18:30 +00:00
Ben Niemann 2871e58a01
Remove unneeded debug messages to stdout. (#18298)
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-01-16 23:42:06 +08:00
techknowlogick 84145e45c5
Remove golang vendored directory (#18277)
* rm go vendor

* fix drone yaml

* add to gitignore
2022-01-14 18:16:05 -05:00
Lunny Xiao 35c3553870
Support webauthn (#17957)
Migrate from U2F to Webauthn

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-01-14 16:03:31 +01:00
6543 ff00b8688b
Fix NPE on try to get tag reference via API (#18245)
* fix npe

* rm gitRepo from Tag
2022-01-12 20:37:46 +00:00
luzpaz af92473920
Fix source typos (#18227)
Follow-up to #18219
2022-01-10 23:46:26 +08:00
luzpaz 8c647bf0f6
Fix various typos (#18219)
Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby,te,unknwon`

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-01-10 17:32:37 +08:00
Gusted 242dddfcb7
Remove ioutil (#18222)
- Don't use `ioutil` package anymore as it doesn't anything special
anymore since Go 1.16:

```
// As of Go 1.16, the same functionality is now provided
// by package io or package os, and those implementations
// should be preferred in new code.
```

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-01-10 09:48:13 +08:00
delvh 4f7764561a
Sort locales according to their names (#18211)
* Sort locales according to their names

* Fix documentation and sort case insensitive
2022-01-08 12:18:39 +00:00
zeripath 832f987d80
Restore setting of ctx.Repo.Mirror (#18205)
In #17933 repoAssignment no longer sets the ctx.Repo.Mirror field meaning that
attempting change mirror settings results in an NPE. This PR simply restores this.

Either we should remove this field or, we should set it. At present it seems simplest
to set it instead of going looking in the Data for the value although converting the
context to a bag of things may be the correct approach in the future.

Fix #18204

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-01-08 10:03:51 +08:00
Lunny Xiao a1c12fb0b3
Don't store assets modified time into generated files (#18193) 2022-01-06 21:33:17 -05:00
zeripath 21ed4fd8da
Add warning for BIDI characters in page renders and in diffs (#17562)
Fix #17514

Given the comments I've adjusted this somewhat. The numbers of characters detected are increased and include things like the use of U+300 to make à instead of à and non-breaking spaces.

There is a button which can be used to escape the content to show it.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Gwyneth Morgan <gwymor@tilde.club>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-01-07 02:18:52 +01:00
Gusted 4b3bfd7e89
Enable partial clone by default (#18195)
- Enable partial clones(which are by default disabled from git) by
default, unless configured otherwise.
- Resolves #18190
2022-01-06 06:38:38 +01:00
zeripath 1514e13bb8
Add option to convert CRLF to LF line endings for sendmail (#18075)
It appears that several versions of sendmail require that the mail is sent to them with
LF line endings instead of CRLF endings - which of course they will then convert back
to CRLF line endings to comply with the SMTP standard.

This PR adds another setting SENDMAIL_CONVERT_CRLF which will pass the message writer
through a filter. This will filter out and convert CRLFs to LFs before writing them
out to sendmail.

Fix #18024

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-01-06 01:43:45 +01:00
Lunny Xiao 8760af752a
Team permission allow different unit has different permission (#17811)
* Team permission allow different unit has different permission

* Finish the interface and the logic

* Fix lint

* Fix translation

* align center for table cell content

* Fix fixture

* merge

* Fix test

* Add deprecated

* Improve code

* Add tooltip

* Fix swagger

* Fix newline

* Fix tests

* Fix tests

* Fix test

* Fix test

* Max permission of external wiki and issues should be read

* Move team units with limited max level below units table

* Update label and column names

* Some improvements

* Fix lint

* Some improvements

* Fix template variables

* Add permission docs

* improve doc

* Fix fixture

* Fix bug

* Fix some bug

* fix

* gofumpt

* Integration test for migration (#18124)

integrations: basic test for Gitea {dump,restore}-repo
This is a first step for integration testing of DumpRepository and
RestoreRepository. It:

runs a Gitea server,
dumps a repo via DumpRepository to the filesystem,
restores the repo via RestoreRepository from the filesystem,
dumps the restored repository to the filesystem,
compares the first and second dump and expects them to be identical

The verification is trivial and the goal is to add more tests for each
topic of the dump.

Signed-off-by: Loïc Dachary <loic@dachary.org>

* Team permission allow different unit has different permission

* Finish the interface and the logic

* Fix lint

* Fix translation

* align center for table cell content

* Fix fixture

* merge

* Fix test

* Add deprecated

* Improve code

* Add tooltip

* Fix swagger

* Fix newline

* Fix tests

* Fix tests

* Fix test

* Fix test

* Max permission of external wiki and issues should be read

* Move team units with limited max level below units table

* Update label and column names

* Some improvements

* Fix lint

* Some improvements

* Fix template variables

* Add permission docs

* improve doc

* Fix fixture

* Fix bug

* Fix some bug

* Fix bug

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Aravinth Manivannan <realaravinth@batsense.net>
2022-01-05 11:37:00 +08:00
Gusted 623c93ff46
Increase Salt randomness (#18179)
- The current implementation of `RandomString` doesn't give you a most-possible unique randomness. It gives you 6*`length` instead of the possible 8*`length` bits(or as `length`x bytes) randomness. This is because `RandomString` is being limited to a max value of 63, this in order to represent the random byte as a letter/digit.
- The recommendation of pbkdf2 is to use 64+ bit salt, which the `RandomString` doesn't give with a length of 10, instead of increasing 10 to a higher number, this patch adds a new function called `RandomBytes` which does give you the guarentee of 8*`length` randomness and thus corresponding of `length`x bytes randomness.
- Use hexadecimal to store the bytes value in the database, as mentioned, it doesn't play nice in order to convert it to a string. This will always be a length of 32(with `length` being 16).
- When we detect on `Authenticate`(source: db) that a user has the old format of salt, re-hash the password such that the user will have it's password hashed with increased salt.

Thanks to @zeripath for working out the rouge edges from my first commit 😄.

Co-authored-by: lafriks <lauris@nix.lv>
Co-authored-by: zeripath <art27@cantab.net>
2022-01-04 15:13:52 +00:00
Gusted 165346c15c
Add MP4 as default allowed attachment type (#18170) 2022-01-04 04:36:47 +01:00
Gusted 74a5d49e86
Include folders into size cost (#18158)
- Include folders for the disk consumption size, they should be included
as they are also saved on the disk :)
- Have a more accurate picture of the size of a repo.
- Mostly they are the size of the file system's block size. E.g. 4Kb on
Linux.
2022-01-03 23:24:12 +00:00
Lunny Xiao de8e3948a5
Refactor auth package (#17962) 2022-01-02 21:12:35 +08:00
wxiaoguang e61b390d54
Unify and simplify TrN for i18n (#18141)
Refer: https://github.com/go-gitea/gitea/pull/18135#issuecomment-1003246099

Now we have a unique and simple `TrN`, and make the fix of PR #18135 also use the better `TrN` logic.
2022-01-02 04:33:57 +01:00
qwerty287 7db2f110ad
Add API to get issue/pull comments and events (timeline) (#17403)
* Add API to get issue/pull comments and events (timeline)
Adds an API to get both comments and events in one endpoint with all required data.
Closes go-gitea/gitea#13250

* Fix swagger

* Don't show code comments (use review api instead)

* fmt

* Fix comment

* Time -> TrackedTime

* Use var directly

* Add logger

* Fix lint

* Fix test

* Add comments

* fmt

* [test] get issue directly by ID

* Update test

* Add description for changed refs

* Fix build issues + lint

* Fix build

* Use string enums

* Update swagger

* Support `page` and `limit` params

* fmt + swagger

* Use global slices

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-01-01 22:12:25 +08:00
Lunny Xiao 25a290e320
Upgrade bleve from v2.0.6 to v2.3.0 (#18132) 2022-01-01 16:26:27 +08:00
Gusted e4e3df6c66
Handle invalid issues (#18111)
* Handle invalid issues

- When you hover over a issue reference, and the issue doesn't exist, it
will just hang on the loading animation.
- This patch fixes that by showing them the pop-up with a "Error
occured" message.

* Add I18N

* refactor

* fix comment for lint

* fix unit test for i18n

* fix unit test for i18n

* add comments

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-12-28 21:28:27 +08:00
zeripath 16adaaeaa3
Instead of using routerCtx just escape the url before routing (#18086)
A consequence of forcibly setting the RoutePath to the escaped url is that the
auto routing to endpoints without terminal slashes fails (Causing #18060.) This
failure raises the possibility that forcibly setting the RoutePath causes other
unexpected behaviors too.

Therefore, instead we should simply pre-escape the URL in the process registering
handler. Then the request URL will be properly escaped for all the following calls.

Fix #17938
Fix #18060
Replace #18062
Replace #17997

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-12-25 00:50:49 +08:00
Aravinth Manivannan c7151c2fb6
- name: new parameter in CreateForkOption to give the forked repository (#18066)
a custom name, intended to be used when there's a name conflict
- When a fork request results in a name conflict, HTTP 409: Conflict is
  returned instead of 500
- API documentation for the above mentioned changes

Signed-off-by: realaravinth <realaravinth@batsense.net>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
2021-12-24 22:43:00 +08:00
qwerty287 7cc44491fa
Add API to manage repo tranfers (#17963) 2021-12-23 23:26:52 -05:00
zeripath a5df7ba6bf
Prevent NPE if gitea uploader fails to open url (#18080)
If http.Get() returns an error return nil and err before attempting to
use the broken file.

Thanks to walker xiong for spotting this bug.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-12-23 16:27:33 +00:00
zeripath ffc08c1914
Do not read or write git reference files directly (#18079)
Git will and can pack references into packfiles and therefore if you write/read the
files directly you will get false results. Instead you should use update-ref and
show-ref. To that end I have created three new functions in git/repo_commit.go that
will do this correctly.

Related #17191

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-12-23 21:44:00 +08:00
99rgosse e0cf3d86c4
Migrated Repository will show modifications when possible (#17191)
* Read patches to get history
2021-12-23 16:32:29 +08:00
zeripath ba6efb105a
Quote references to the user table in consistency checks (#18072)
Although #17487 ensured that the table was quoted in the join it missed that the
query part of the check also needed to be quoted.

Fix #17485

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-12-22 23:52:57 +00:00
zeripath 7cc7f0ed75
TestRepository_GetTag intermittently panics due to an NPE (#18043)
There are repeated panics in tests due to TestRepository_GetTag failing
to run properly.  This happens when we attempt to reset the internal
repo for a tag which has failed to load. The problem is - the panic that
this is causing is preventing us from finding what the real error is.

This PR simply moves the failure out so we have a chance to see what
really is failing.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-12-21 11:10:16 +08:00
zeripath bcc13f3889
Reset Session ID on login (#18018)
* Reset Session ID on login

When logging in the SessionID should be reset and the session cleaned up.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* with new session.RegenerateID function

Signed-off-by: Andrew Thornton <art27@cantab.net>

* update go-chi/session

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Ensure that session id is changed after oauth data is set and between account linking pages too

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate lint

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per review

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-12-20 22:12:26 +08:00
Gusted ff2fd08228
Simplify parameter types (#18006)
Remove repeated type declarations in function definitions.
2021-12-20 04:41:31 +00:00
zeripath fb5f7791ef
Prevent off-by-one error on comments on newly appended lines (#18029)
* Prevent off-by-one error on comments on newly appended lines

There was a bug in CutDiffAroundLine whereby if a file without a terminal new line
has a patch which appends lines to it and a comment is placed on one of those lines
the comment diff will be a line out of place.

This fixes CutDiffAroundLine to simply ignore the missing terminal newline - however,
we should really improve this rendering to add a marker to say that there was a
previously missing terminal newline.

Fix #17875

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-12-20 02:15:49 +00:00