Commit graph

3280 commits

Author SHA1 Message Date
wxiaoguang 6d4172987e
Fix markdown URL parsing (#17924)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
2021-12-11 19:21:36 +02:00
Lunny Xiao 3ca5dc7e32
Move keys to models/asymkey (#17917)
* Move keys to models/keys

* Rename models/keys -> models/asymkey

* change the missed package name

* Fix package alias

* Fix test

* Fix docs

* Fix test

* Fix test

* merge
2021-12-10 16:14:24 +08:00
Lunny Xiao 719bddcd76
Move repository model into models/repo (#17933)
* Some refactors related repository model

* Move more methods out of repository

* Move repository into models/repo

* Fix test

* Fix test

* some improvements

* Remove unnecessary function
2021-12-10 09:27:50 +08:00
wxiaoguang c7e23401a3
Fix a panic in NotifyCreateIssueComment (caused by string truncation) (#17928)
* Fix a panic in NotifyCreateIssueComment (caused by string truncation)

* more unit tests

* refactor

* fix some edge cases

* use SplitStringAtByteN for comment content
2021-12-09 05:41:17 +00:00
zeripath 9e6e1dc950
Improve checkBranchName (#17901)
The current implementation of checkBranchName is highly inefficient
involving opening the repository, the listing all of the branch names
checking them individually before then using using opened repo to get
the tags.

This PR avoids this by simply walking the references from show-ref
instead of opening the repository (in the nogogit case).

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-12-08 19:08:16 +00:00
Garionion b59875aa12
allways set a message-id on mails (#17900)
* allways set a message-id on mails
* Add unit tests for mailer & Message-ID

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-12-08 15:34:23 +08:00
wxiaoguang a6f961fba4
Refactor install page (db type) (#17919)
* Refactor install page (db type)

* set correct default DB HOST for different DB TYPE
* remove legacy TiDB from documents
* unify the usage of DB TYPE, in code we only use "mysql". "MySQL" is only shown to users for friendly name.

* Gitea can use TiDB via MySQL protocol

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-12-07 13:44:08 +08:00
mscherer f49d160447
Replace "unix" by "http+unix" for PROTOCOL (#17771) 2021-12-05 23:46:11 -05:00
wxiaoguang 11d519b385
Test cache during init (#17852) 2021-12-06 00:24:57 +08:00
KN4CK3R 6ceef87671
Removed SizeFmt. (#17890)
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-12-03 11:33:34 +08:00
KN4CK3R 87be76213a
Add migrate from Codebase (#16768)
This PR adds [Codebase](https://www.codebasehq.com/) as migration source.

Supported:
- Milestones
- Issues
- Pull Requests
- Comments
- Labels
2021-12-02 23:24:02 +01:00
delvh 4646c7c52d
Use fmt.Sprintf correctly (#17886) 2021-12-02 20:36:50 +01:00
mscherer 34b5436ae1
Refactor various strings (#17784)
Fixes #16478

Co-authored-by: Gusted <williamzijl7@hotmail.com>

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-12-02 15:28:08 +08:00
wxiaoguang 042cac5fed
Improve install code to avoid low-level mistakes. (#17779)
* Improve install code to avoid low-level mistakes.

If a user tries to do a re-install in a Gitea database, they gets a warning and double check.
When Gitea runs, it never create empty app.ini automatically.

Also some small (related) refactoring:

* Refactor db.InitEngine related logic make it more clean (especially for the install code)
* Move some i18n strings out from setting.go to make the setting.go can be easily maintained.
* Show errors in CLI code if an incorrect app.ini is used.
* APP_DATA_PATH is created when installing, and checked when starting (no empty directory is created any more).
2021-12-01 15:50:01 +08:00
zeripath 01087e9eef
Make Requests Processes and create process hierarchy. Associate OpenRepository with context. (#17125)
This PR registers requests with the process manager and manages hierarchy within the processes.

Git repos are then associated with a context, (usually the request's context) - with sub commands using this context as their base context.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-30 20:06:32 +00:00
zeripath b5a9ee94fd
Make Co-committed-by and co-authored-by trailers optional (#17848)
This PR adds another option to app.ini make co-committed-by and co-authored-by trailers
optional on a per server basis.

Fix #17194

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-29 07:09:55 +00:00
Lunny Xiao 1fee11d69a
Move accessmode into models/perm (#17828) 2021-11-28 12:58:28 +01:00
zeripath 72d82c5b6f
Handle relative unix socket paths (#17836)
Make relative unix sockets absolute by making them absolute against the AppWorkPath

Fix #17833

## ⚠️ BREAKING ⚠️ 

Prior to this PR relative unix sockets would have been asserted to be relative to the current working directory that gitea, gitea serv, hook and manager etc were running in. Hooks and Serv would have failed to work properly under this situation so we expect that although this is a technically breaking change the previous situation was already broken.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-27 20:22:03 +00:00
Gabriel Vasile 2e8fc5b034
Replace regex usage for MIME parsing (#17831)
MIME types can have multiple optional parameters, eg:
    video/webm; codecs="w/e codec"; charset="binary"
This commit replaces the usage of regex for getting the "type/subtype"
with mime.ParseMediaType.
2021-11-27 19:12:43 +08:00
Gusted 4b4997c73e
Use correct user on releases (#17806)
- Use the provided `doer` instead of `rel.Publisher`. The code will also
run on edited releases and deleted ones, which isn't necessary done by
`rel.Publisher`.

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-25 12:17:16 +08:00
Lunny Xiao a666829a37
Move user related model into models/user (#17781)
* Move user related model into models/user

* Fix lint for windows

* Fix windows lint

* Fix windows lint

* Move some tests in models

* Merge
2021-11-24 17:49:20 +08:00
Gusted 4e7ca946da
Use *PushUpdateOptions as receiver (#17724) 2021-11-24 04:08:13 -05:00
Lunny Xiao c97d66d23c
Move repofiles from modules/repofiles to services/repository/files (#17774)
* Move repofiles from modules to services

* rename services/repository/repofiles -> services/repository/files

* Fix test

Co-authored-by: 6543 <6543@obermui.de>
2021-11-24 15:56:24 +08:00
qwerty287 754fdd8f9c
Fix visibility of org avatars (#17789)
* Fix visibility of org avatar

* more clear syntax

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-11-24 11:51:08 +08:00
Gusted 21f4401f3e
Use correct Sender on webhook change title (#17791)
As title, thanks to @mscherer for pointing out the incorrect code.
2021-11-24 10:47:03 +08:00
zeripath 1dbc58f742
More pleasantly handle broken or missing git repositories (#17747)
* More pleasantly handle broken or missing git repositories

In #17742 it was noted that there a completely invalid git repository underlying a
repo on gitea.com. This happened due to a problem during a migration however, it
is not beyond the realms of possibility that a corruption could occur to another
user.

This PR adds a check to RepoAssignment that will detect if a repository loading has
failed due to an absent git repository. It will then show a page suggesting the user
contacts the administrator or deletes the repository.

Fix #17742

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update options/locale/locale_en-US.ini

Co-authored-by: techknowlogick <techknowlogick@gitea.io>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-11-22 22:32:16 +00:00
Lunny Xiao baed01f247
Remove unnecessary attributes of User struct (#17745)
* Remove unnecessary functions of User struct

* Move more database methods out of user struct

* Move more database methods out of user struct

* Fix template failure

* Fix bug

* Remove finished FIXME

* remove unnecessary code
2021-11-22 23:21:55 +08:00
techknowlogick 499b05da22
Add user settings key/value DB table (#16834) 2021-11-22 04:47:23 -05:00
Lunny Xiao 0add627182
Fix close issue but time watcher still running (#17643)
* Fix close issue but time watcher still running

* refactor stopwatch codes

* Fix test

* Fix test

* Fix typo

* Fix test
2021-11-21 17:11:48 +08:00
zeripath 0fa41b6dc4
Reset locale on login (#17734)
When logging in reset the user's locale to ensure that it matches their
preferred locale.

Fix #15612

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-20 21:32:37 +00:00
zeripath f8ec43dbc7
Extract constant names out for the ReverseProxy and Basic authentication methods (#17735)
In order to reduce load on the GC extract out the constant names of the Basic and ReverseProxy methods.

As mentioned in https://github.com/go-gitea/gitea/pull/15119#discussion_r730352176

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-20 23:33:18 +08:00
zeripath 931d0cf854
Enable show more files in diff for git <2.31 (#17733)
Unfortunately due to a misread on my behalf I missed that git diff only learned
--skip-to in version 2.31.0. Thus this functionality was not working on older versions
of git.

This PR adds a handler that simply allows for us to skip reading the diffs until
we find the correct file to skip to.

Fix #17731

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-20 13:50:00 +00:00
wxiaoguang 013fb73068
Use hostmatcher to replace matchlist, improve security (#17605)
Use hostmacher to replace matchlist.

And we introduce a better DialContext to do a full host/IP check, otherwise the attackers can still bypass the allow/block list by a 302 redirection.
2021-11-20 17:34:05 +08:00
zeripath c96be0cd98
Make SSL cipher suite configurable (#17440) 2021-11-20 01:12:43 -05:00
Gusted 9f14fe43c6
Fix correct usage of teams (#17732)
- `.Teams` isn't a field on the User type, thus using the seperate
loaded teams.
- Add a space between `PathEscape` and argument.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-11-20 11:13:24 +08:00
wxiaoguang a8fd76557b
Better builtin avatar generator (#17707)
This PR fixes the builtin avatar generator.

1. The random background color makes some images very dirty. So now we only use white background for avatars.
2. We use left-right mirror avatars to satisfy #14799
3. Fix a small padding error in the algorithm
2021-11-20 01:10:41 +08:00
zeripath 38347aa16f
Add settings to allow different SMTP envelope from address (#17479)
* Add settings to allow different SMTP envelope from address

Sometimes it may be advisable to hide or alias the from address on an SMTP mail
envelope. This PR adds two new options to the mailer to allow setting of an overriding
from address.

Fix #17477

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-19 23:35:20 +08:00
Lunny Xiao fc3d082609
Move attachment into models/repo/ (#17650)
* Move attachment into models/repo/

* Fix test

* Fix bug
2021-11-19 21:39:57 +08:00
Lunny Xiao 7a03473159
Use a standalone struct name for Organization (#17632)
* Use a standalone struct name for Organization

* recover unnecessary change

* make the code readable

* Fix template failure

* Fix template failure

* Move HasMemberWithUserID to org

* Fix test

* Remove unnecessary user type check

* Fix test

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-19 19:41:40 +08:00
KN4CK3R a09b40de8d
Prevent double sanitize (#16386)
* Prevent double sanitize.
* Use SanitizeReaderToWriter.

At the moment `actualRender` uses `SanitizeReader` to sanitize the output. But `SanitizeReader` gets called in `markup.render` too so the output gets sanitized twice.

I moved the `SanitizeReader` call into `RenderRaw` because this method does not use `markup.render`. I would like to remove the `RenderRaw`/`RenderRawString` methods too because they are only called from tests, the fuzzer and the `/markup/raw` api endpoint. This endpoint is not in use so I think we could remove them. If we really in the future need a method to render markdown without PostProcessing we could achieve this with a more flexible `renderer.NeedPostProcess` method.
2021-11-19 18:46:47 +08:00
zeripath a85e75b2b1
Prevent deadlock in TestPersistableChannelQueue (#17717)
* Prevent deadlock in TestPersistableChannelQueue

There is a potential deadlock in TestPersistableChannelQueue due to attempting to
shutdown the test queue before it is ready.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* prevent npe

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-19 01:13:25 +00:00
KN4CK3R f34151bdb2
Move user/org deletion to services (#17673) 2021-11-19 01:42:27 +08:00
Gusted d1f5584039
Simplify code for wrting SHA to name-rev (#17696) 2021-11-18 04:50:22 -05:00
Lunny Xiao 1f1ae57139
Move task from modules to services (#17680) 2021-11-18 14:47:57 +08:00
Lunny Xiao 3c3855a05c
Move user functions into user.go (#17659)
* Move user functions into user.go

* Fix test
2021-11-18 13:58:42 +08:00
Gusted c98dd7a3e0
Remove unnecessary variable assignments (#17695)
* Remove unnecessary variable assignments

As title

* enable ineffassign

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-18 09:33:06 +08:00
KN4CK3R ea42d3c04e
Fixed commit count. (#17698)
Added "Tag" label.
Unified branch, tag and commit name.
2021-11-17 23:50:17 +00:00
zeripath 3c4724d70e
Add .gitattribute assisted language detection to blame, diff and render (#17590)
Use check attribute code to check the assigned language of a file and send that in to
chroma as a hint for the language of the file.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-17 20:37:00 +00:00
Lunny Xiao 5233051e64
Move some functions into services/repository (#17677) 2021-11-17 23:17:31 +08:00
wxiaoguang 750a8465f5
A better go code formatter, and now make fmt can run in Windows (#17684)
* go build / format tools
* re-format imports
2021-11-17 20:34:35 +08:00
Lunny Xiao 29cc169d20
Fix bug when read mysql database max lifetime (#17682) 2021-11-17 18:59:23 +08:00
zeripath bbffcc3aec
Multiple Escaping Improvements (#17551)
There are multiple places where Gitea does not properly escape URLs that it is building and there are multiple places where it builds urls when there is already a simpler function available to use this.
    
This is an extensive PR attempting to fix these issues.

1. The first commit in this PR looks through all href, src and links in the Gitea codebase and has attempted to catch all the places where there is potentially incomplete escaping.
2. Whilst doing this we will prefer to use functions that create URLs over recreating them by hand.
3. All uses of strings should be directly escaped - even if they are not currently expected to contain escaping characters. The main benefit to doing this will be that we can consider relaxing the constraints on user names and reponames in future. 
4. The next commit looks at escaping in the wiki and re-considers the urls that are used there. Using the improved escaping here wiki files containing '/'. (This implementation will currently still place all of the wiki files the root directory of the repo but this would not be difficult to change.)
5. The title generation in feeds is now properly escaped.
6. EscapePound is no longer needed - urls should be PathEscaped / QueryEscaped as necessary but then re-escaped with Escape when creating html with locales Signed-off-by: Andrew Thornton <art27@cantab.net>

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-16 18:18:25 +00:00
Lunny Xiao 7e1ae38097
Move migrations into services and base into modules/migration (#17663)
* Move migrtions into services and base into modules/migration

* Fix imports

* Fix lint
2021-11-16 23:25:33 +08:00
Lunny Xiao 48ccd325a1
Move some functions into services/repository (#17660) 2021-11-16 21:30:11 +08:00
wxiaoguang 81926d61db
Decouple unit test, remove intermediate unittestbridge package (#17662)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-11-16 16:53:21 +08:00
silverwind 23bd7b1211
Add copy button to markdown code blocks (#17638)
* Add copy button to markdown code blocks

Done mostly in JS because I think it's better not to try getting buttons
past the markup sanitizer.

* add svg module tests

* fix sanitizer regexp

* remove outdated comment

* vertically center button in issue comments as well

* add comment to css

* fix undefined on view file line copy

* combine animation less files

* Update modules/markup/markdown/markdown.go

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>

* add test for different sizes

* add cloneNode and add tests for it

* use deep clone

* remove useless optional chaining

* remove the svg node cache

* unify clipboard copy string and i18n

* remove unused var

* remove unused localization

* minor css tweaks to the button

* comment tweak

* remove useless attribute

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-16 16:16:05 +08:00
Gusted ab1379743e
Fix nil checking on typed interface (#17598)
* Fix nil checking on typed interface

- Partially resoles #17596
- Resolves SA4023 errors.
- Ensure correctly that typed interface are nil.

* Remove unnecessary code

`NewBleveIndexer` will never return nil, even on errors.

* Patch `NewBleveIndexer`

* Fix low-level functions

* Remove deadcode

* Fix GetSession

* Close Elastic search when err isn't nil

* Update elastic_search.go

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-15 21:16:11 +08:00
Gusted 253d9e4158
Remove unnecassary calls to filepath.Join (#17608)
- Partialy resolvess #17596
- Resolves `badCall` errors from go-critic `badCall: suspicious Join on
1 argument`
- When only 1 argument is passed into `filepath.Join`, it won't do
anything special other than `filepath.Clean(...)` will be applied over
it.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
2021-11-15 14:02:53 +08:00
KN4CK3R 42ea0023a3
Add migration from GitBucket (#16767)
This PR adds [GitBucket](https://gitbucket.github.io/) as migration source.

Supported:
- Milestones
- Issues
- Pull Requests
- Comments
- Reviews
- Labels

There is no public usable instance so no integration tests added.
2021-11-14 20:11:10 +01:00
zeripath bab95c3a86
Correctly handle failed migrations (#17575)
* Correctly handle failed migrations

There is a bug in handling failed migrations whereby the migration task gets decoupled
from the migration repository. This leads to a failure of the task to get deleted with
the repository and also leads to the migration failed page resulting in a ISE.

This PR removes the zeroing out of the task id from the migration but also makes
the migration handler tolerate missing tasks much nicer.

Fix #17571

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-13 19:28:50 +08:00
wxiaoguang df64fa4865
Decouple unit test code from business code (#17623) 2021-11-12 22:36:47 +08:00
Lunny Xiao 90eb9fb889
Move EmailAddress & UserRedirect into models/user/ (#17607)
* Move EmailAddress into models/user/

* Fix test

* rename user_mail to user_email

* Fix test

* Move UserRedirect into models/user/

* Fix lint & test

* Fix lint

* Fix lint

* remove nolint comment

* Fix lint
2021-11-11 15:03:30 +08:00
Lunny Xiao 33fca2b537
Move webhook into models/webhook/ (#17579) 2021-11-10 13:13:16 +08:00
Lunny Xiao 99b2858e62
Move unit into models/unit/ (#17576)
* Move unit into models/unit/

* Rename unit.UnitType as unit.Type
2021-11-09 20:57:58 +01:00
Gusted ebaf4c48ea
Avoid double imports (#17569)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-08 09:04:13 +02:00
Lunny Xiao 69b61d4373
Fix bug on admin subcommand (#17533)
* Fix bug on admin subcommand

* Add signals for all initDB

Co-authored-by: Lauris BH <lauris@nix.lv>
2021-11-07 11:11:27 +08:00
Wim bd613c704c
Fix ipv6 parsing for builtin ssh server (#17561) 2021-11-06 14:23:32 +08:00
Gusted bc6df18fb3
Use correct defaultValue for stracktrace (#17552) 2021-11-05 13:25:05 +02:00
Lunny Xiao a161c2ca58
Fix zero created time bug on commit api (#17546)
Fix #17543
2021-11-05 05:30:46 +00:00
zeripath 95da01c5cd
Add QueryEscape to general funcmap (#17526)
QueryEscape was only added to the text funcmap. Add this to the main template funcmap
too.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-02 23:00:30 +08:00
Lunny Xiao e69521f029
fix email with + when active (#17518)
Co-authored-by: zeripath <art27@cantab.net>
2021-11-02 06:26:13 +00:00
zeripath cb9c8184c9
Make Repo Code Indexer an Unique Queue (#17515)
The functioning of the code indexer queue really only makes sense as an unique queue
and doing this allows use to simplify the indexer data to simply delete the data if
the repo is no longer in the db.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-02 11:14:24 +08:00
wxiaoguang 599ff1c054
Only allow webhook to send requests to allowed hosts (#17482) 2021-11-01 16:39:52 +08:00
Richard Mahn 40c8451b7d
Properly determine CSV delimiter (#17459)
* Fixes #16558 CSV delimiter determiner

* Fixes #16558 - properly determine CSV delmiiter

* Moves quoteString to a new function

* Adds big test with lots of commas for tab delimited csv

* Adds comments

* Shortens the text of the test

* Removes single quotes from regexp as only double quotes need to be searched

* Fixes spelling

* Fixes check of length as it probalby will only be 1e4, not greater

* Makes sample size a const, properly removes truncated line

* Makes sample size a const, properly removes truncated line

* Fixes comment

* Fixes comment

* tests for FormatError() function

* Adds logic to find the limiter before or after a quoted value

* Simplifies regex

* Error tests

* Error tests

* Update modules/csv/csv.go

Co-authored-by: delvh <dev.lh@web.de>

* Update modules/csv/csv.go

Co-authored-by: delvh <dev.lh@web.de>

* Adds comments

* Update modules/csv/csv.go

Co-authored-by: delvh <dev.lh@web.de>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
2021-10-30 23:50:40 +08:00
Lunny Xiao 63c0dc89ef
Rename db Engines related functions (#17481)
* Rename db Engines related functions

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-10-30 22:32:11 +08:00
zeripath 0b4a8be26b
Ensure that restricted users can access repos for which they are members (#17460)
There is a small bug in the way that repo access is checked in
repoAssignment: Accessibility is checked by checking if the user has a
marked access to the repository instead of checking if the user has any
team granted access.

This PR changes this permissions check to use HasAccess() which does the
correct test. There is also a fix in the release api ListReleases where
it should return draft releases if the user is a member of a team with
write access to the releases.

The PR also adds a testcase.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-28 10:54:40 +08:00
Richard Mahn 8f9ac439ca
Fixes #16559 - Do not trim leading spaces for tab delimited (#17442)
* Fixes #16559 - Do not trim leading spaces for tab delimited

* Adds back semicolon delimited test

* Fixes linting

* Adds nolint directive to test because uses strings starting with spaces

Co-authored-by: zeripath <art27@cantab.net>
2021-10-26 16:46:56 -05:00
Lunny Xiao 812a9daab9
Upgrade go-github to v39 (#17437) 2021-10-26 08:19:21 +01:00
wxiaoguang 7c951fdd4a
In many cases user avatar link should be an absolute URL with http host (#17420) 2021-10-25 13:01:16 +08:00
qwerty287 3676fafdac
Add API to get/edit wiki (#17278)
* Add API to get/edit wiki

* Add swagger docs, various improvements

* fmt

* Fix lint and rm comment

* Add page parameter

* Add pagination to pages

* Add tests

* fmt

* Update func names

* Update error handling

* Update type name

* Fix lint

* Don't delete Home

* Update func name

* Update routers/api/v1/repo/wiki.go

Co-authored-by: delvh <dev.lh@web.de>

* Remove unnecessary check

* Fix lint

* Use English strings

* Update integrations/api_wiki_test.go

Co-authored-by: delvh <dev.lh@web.de>

* Update func and test names

* Remove unsed check and avoid duplicated error reports

* Improve error handling

* Return after error

* Document 404 error

* Update swagger

* Fix lint

* Apply suggestions from code review

Co-authored-by: delvh <dev.lh@web.de>

* Document file encoding

* fmt

* Apply suggestions

* Use convert

* Fix integration test

* simplify permissions

* unify duplicate key Title/Name

* improve types & return UTC timestamps

* improve types pt.2

- add WikiPageMetaData.LastCommit
- add WikiPageMetaData.HTMLURL
- replace WikiPageMetaData.Updated with .LastCommit.Committer.Created

also delete convert.ToWikiPage(), as it received too many arguments and
only had one callsite anyway. sorry for bad advice earlier 🙃

* WikiPage.Content is base64 encoded

* simplify error handling in wikiContentsByName()

* update swagger

* fix & DRY findWikiRepoCommit() error handling

ListWikiPages() previously wrote error twice when repo wiki didn't exist

* rename Content -> ContentBase64

* Fix test

* Fix tests

* Update var name

* suburl -> sub_url

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Norwin <git@nroo.de>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-10-25 11:43:40 +08:00
KN4CK3R 44f2c27d28
Fix CSV render error (#17406)
closed #17378 

Both errors from #17378 were caused by  #15175.

Problem 1 (error with added file):
`ToUTF8WithFallbackReader` creates a `MultiReader` from a `byte[2048]` and the remaining reader. `CreateReaderAndGuessDelimiter` tries to read 10000 bytes from this reader but only gets 2048 because that's the first reader in the `MultiReader`. Then the `if size < 1e4` thinks the input is at EOF and just returns that.

Problem 2 (error with changed file):
The blob reader gets defer closed. That was fine because the old version reads the whole file into memory. Now with the streaming version the close needs to defer after the method.
2021-10-25 00:42:32 +02:00
KN4CK3R f99d50fc9f
Read expected buffer size (#17409)
* Read expected buffer size.

* Changed name.
2021-10-24 22:12:43 +01:00
wxiaoguang 6c49517cbd
Fix issue markdown bugs (#17411)
* Bug fix: render Markdown http://AppURL/org/repo/issues/4?a=1&b=2#comment-123 test to HTML correctly, close #17394
* Bug fix: fix the positions of checkboxes in rendered HTML, close #17395
2021-10-23 21:38:12 +08:00
zeripath af96286f22
Stop sanitizing full name in API (#17396)
The API convert.toUser function makes the incorrect assumption that full names could
be rendered as is without being escaped. It therefore runs the names through
markup.Sanitize which leads to a double escape of user full names. This
pr stops this.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-10-22 15:17:35 +08:00
wxiaoguang 960c322586
Refactor update checker to use AppState (#17387)
We have the `AppState` module now, it can store app related data easily. We do not need to create separate tables for each feature.

So the update checker can use `AppState` instead of a one-row dedicate table.

And the code of update checker is moved from `models` to `modules`.
2021-10-21 17:10:49 +01:00
wxiaoguang 83df0caf15
Sync gitea app path for git hooks and authorized keys when starting (#17335)
Gitea writes its own AppPath into git hook scripts. If Gitea's AppPath changes, then the git push will fail.

This PR:

* Introduce an AppState module, it can persist app states into database
* During GlobalInit, Gitea will check if the current AppPath is the same as last one. If they don't match, Gitea will sync git hooks.
* Refactor some code to make them more clear.
* Also, "Detect if gitea binary's name changed" #11341 is related, we call models.RewriteAllPublicKeys to update ssh authorized_keys file
2021-10-21 17:22:43 +08:00
wxiaoguang 2add8fe9be
Frontend refactor, PascalCase to camelCase, remove unused code (#17365)
* Frontend refactor, PascalCase to camelCase, remove unused code

* fix
2021-10-21 15:37:43 +08:00
zeripath 35b918f574
Offer rsa-sha2-512 and rsa-sha2-256 algorithms in internal SSH (#17281)
* Offer rsa-sha2-512 and rsa-sha2-256 algorithms in internal SSH

There is a subtle bug in the SSH library x/crypto/ssh which makes the incorrect
assumption that the public key type is the same as the signature algorithm type.

This means that only ssh-rsa signatures are offered by default.

This PR adds a workaround around this problem.

Fix #17175

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per review

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-20 15:55:33 -04:00
Richard Mahn 98f7013756
Prevent NPE in CSV diff rendering when column removed (#17018)
Fixes #16837 if a column is deleted.

We were clobbering the columns that were added by looping through the aline (base) and then when bline (head) was looped through, it clobbered what was in the "cells" array that is show in the diff, and then left a nil cell because nothing was shifted.

This fix properly shifts the cells, and properly puts the b cell either at its location or after, according to what the aline placed in the cells.

This includes test, adding a new test function since adding/removing cells works best with three columns, not two, which results in 4 columns of the resulting cells because it has a deleted column and an added column. If you try this locally, you can try those cases and others, such as adding a column.

There was no need to do anything special for the rows when `aline == 0 || bline == 0` so that was removed. This allows the same code to be used for removed or added lines, with the bcell text always being the RightCell, acell text being the LeftCell.

I still added the patch zeripath gave at https://github.com/go-gitea/gitea/issues/16837#issuecomment-913007382 so that just in case for some reason a cell is nil (which shouldn't happen now) it doesn't throw a 500 error, so the user can at least view the raw diff.

Also fixes in the [view.go](https://github.com/go-gitea/gitea/pull/17018/files#diff-43a7f4747c7ba8bff888c9be11affaafd595fd55d27f3333840eb19df9fad393L521) file how if a CSV file is empty (either created empty or if you edit it and remove all contents) it throws a huge 500 error when you then save it (when you view the file). Since we allow creating, saving and pushing empty files, we shouldn't throw an error on an empty CSV file, but just show its empty contents. This doesn't happen if it is a Markdown file or other type of file that is empty.
EDIT: Now handled in the markup/csv renderer code
2021-10-20 20:10:03 +01:00
Lunny Xiao f494776931
Use a variable but a function for IsProd because of a slight performance increment (#17368) 2021-10-20 16:37:19 +02:00
John Olheiser c59afa752d
Allow mocking timeutil (#17354)
Signed-off-by: jolheiser <john.olheiser@gmail.com>
2021-10-18 21:12:26 +01:00
Lunny Xiao 4a57c9ea17
Fix some lints (#17337)
Fix some linting problems.
2021-10-17 20:47:12 +01:00
Lunny Xiao 5326f4c9c5
Move session to models/login (#17338)
Move model/session.go to models/login
2021-10-17 19:51:56 +01:00
zeripath 7117c7774a
Make the Mirror Queue a queue (#17326)
Convert the old mirror syncing queue to the more modern queue format.

Fix a bug in the from the repo-archive queue PR - the assumption was made that uniqueness could be enforced with by checking equality in a map in channel unique queues - however this only works for primitive types - which was the initial intention but is an imperfect. This is fixed by marshalling the data and placing the martialled data in the unique map instead.

The documentation is also updated to add information about the deprecated configuration values.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-17 12:43:25 +01:00
6543 3728f1daa0
Add RSS/Atom feed support for user actions (#16002)
Return rss/atom feed for user based on rss url suffix or Content-Type header.
2021-10-16 16:21:16 +02:00
techknowlogick 8edda8b446
Add simple update checker to Gitea (#17212)
* Add simple update checker to Gitea

* update struct and remove comments

* fix lint

* Update custom/conf/app.example.ini

* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md

Co-authored-by: delvh <dev.lh@web.de>

* Update custom/conf/app.example.ini

Co-authored-by: delvh <dev.lh@web.de>

* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md

Co-authored-by: delvh <dev.lh@web.de>

* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md

Co-authored-by: Steven <61625851+justusbunsi@users.noreply.github.com>

* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md

* Update modules/cron/tasks_extended.go

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>

* Update custom/conf/app.example.ini

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>

* take PR feedback into account and display banner on admin dashboard for alerts

* Add more detailed message

* placate lint

* update per feedback

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Steven <61625851+justusbunsi@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-10-16 02:14:34 -04:00
zeripath 58cd55d353
Check for context exceeded in WalkGitLog (#17319)
There is a slight race in checking of a context deadline exceed in #16467
which leads to a 500 on the repository page.

The solution is to check the error coming back from `*LogNameStatusRepoParser.Next()`
and if it is the `ContextDeadlineExceeded` break from the loop.

Fix #17314

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-15 19:41:34 +01:00
zeripath a889d0cc8c
Add buttons to allow loading of incomplete diffs (#16829)
This PR adds two buttons to the stats and the end of the diffs list to load the (some of) the remaining incomplete diff sections.

Contains #16775
    
Signed-off-by: Andrew Thornton <art27@cantab.net>


## Screenshots

### Show more button at the end of the diff
![Screenshot from 2021-09-04 11-12-37](https://user-images.githubusercontent.com/1824502/132091009-b1f6113e-2c04-4be5-8a04-b8ecea56887b.png)

### Show more button at the end of the diff stats box
![Screenshot from 2021-09-04 11-14-54](https://user-images.githubusercontent.com/1824502/132091063-86da5a6d-6628-4b82-bea9-3655cd9f40f6.png)
2021-10-15 17:05:33 +01:00
Lunny Xiao bdfd751af8
Multiple tokens support for migrating from github (#17134)
* multiple tokens support for migrating from github

* improve code and token description

* Fix bug

* Add comment for get client
2021-10-15 13:47:15 +08:00
wxiaoguang 56362043d3
Frontend refactor: move Vue related code from index.js to components dir, and remove unused codes. (#17301)
* frontend refactor

* Apply suggestions from code review

Co-authored-by: delvh <dev.lh@web.de>

* Update templates/base/head.tmpl

Co-authored-by: delvh <dev.lh@web.de>

* Update docs/content/doc/developers/guidelines-frontend.md

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>

* fix typo

* fix typo

* refactor PageData to pageData

* Apply suggestions from code review

Co-authored-by: delvh <dev.lh@web.de>

* Simply for the visual difference.

Co-authored-by: delvh <dev.lh@web.de>

* Revert "Apply suggestions from code review"

This reverts commit 4d78ad9b0e.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
2021-10-15 10:35:26 +08:00
Lunny Xiao d47798ccc7
Upgrade chi to v5 (#17298) 2021-10-13 22:50:23 -04:00
zeripath 5a8dd96b33
Ensure that git daemon export ok is created for mirrors (#17243)
* Ensure that git daemon export ok is created for mirrors

There is an issue with #16508 where it appears that create repo requires that the
repo does not exist. This causes #17241 where an error is reported because of this.

This PR fixes this and also runs update-server-info for mirrors and generated repos.

Fix #17241

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-13 20:47:02 +01:00
zeripath 01b9d35f1a
Disable core.protectNTFS (#17300)
core.protectNTFS protects NTFS from files which may be difficult to remove or interact
with using the win32 api, however, it also appears to prevent such files from
being entered into the git indexes - fundamentally causing breakages with PRs that
affect these files. However, deliberately setting this to false may cause security
issues due to the remain sparse checkout of files in the merge pipeline.

The only sensible option therefore is to provide an optional setting which admins
could set which would forcibly switch this off if they are affected by this issue.

Fix #17092

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-13 14:20:11 -04:00
zeripath 429258cff3
Use pointer for wrappedConn methods (#17295)
Fix #17294

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-12 21:55:18 +01:00
wxiaoguang 7bcbdd0707
Add user status filter to admin user management page (#16770)
It makes Admin's life easier to filter users by various status.

* introduce window.config.PageData to pass template data to javascript module and small refactor

move legacy window.ActivityTopAuthors to window.config.PageData.ActivityTopAuthors
make HTML structure more IDE-friendly in footer.tmpl and head.tmpl
remove incorrect <style class="list-search-style"></style> in head.tmpl
use log.Error instead of log.Critical in admin user search

* use LEFT JOIN instead of SubQuery when admin filters users by 2fa. revert non-en locale.

* use OptionalBool instead of status map

* refactor SearchUserOptions.toConds to SearchUserOptions.toSearchQueryBase

* add unit test for user search

* only allow admin to use filters to search users
2021-10-12 20:11:35 +02:00
Romain d0a681fbc3
[API] Add endpount to get user org permissions (#17232)
* Add endpoint

* Add swagger response + generate swagger

* Stop execution if user / org is not found

* Add tests


Co-authored-by: 6543 <6543@obermui.de>
2021-10-12 12:47:19 +02:00
wxiaoguang d8e06a90f0
Open markdown image links in new window (#17287) 2021-10-11 20:12:06 +08:00
a1012112796 bb39359668
Add a simple way to rename branch like gh (#15870)
- Update default branch if needed
- Update protected branch if needed
- Update all not merged pull request base branch name
- Rename git branch
- Record this rename work and auto redirect for old branch on ui

Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: delvh <dev.lh@web.de>
2021-10-08 19:03:04 +02:00
zeripath 001dbf100d
Defer Last Commit Info (#16467)
One of the biggest reasons for slow repository browsing is that we wait
until last commit information has been generated for all files in the
repository.

This PR proposes deferring this generation to a new POST endpoint that
does the look up outside of the main page request.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-08 15:08:22 +02:00
99rgosse 88fa9f3fb1
Remove heads pointing to missing old refs (#17076)
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
2021-10-08 12:59:35 +03:00
techknowlogick f0bd1e9896
Add protection to disable Gitea when run as root (#17168)
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
2021-10-07 10:52:08 +02:00
pricly-yellow 4afdb1eb78
API pull's head/base have correct permission (#17214)
close #17181

* for all pull requests API return permissions of caller
* for all webhook return empty permissions

Signed-off-by: Danila Kryukov <pricly_yellow@dismail.de>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
2021-10-07 02:03:37 +02:00
wxiaoguang f0ba87fda8
Avatar refactor, move avatar code from models to models.avatars, remove duplicated code (#17123)
Why this refactor

The goal is to move most files from `models` package to `models.xxx` package. Many models depend on avatar model, so just move this first.

And the existing logic is not clear, there are too many function like `AvatarLink`, `RelAvatarLink`, `SizedRelAvatarLink`, `SizedAvatarLink`, `MakeFinalAvatarURL`, `HashedAvatarLink`, etc. This refactor make everything clear:

* user.AvatarLink()
* user.AvatarLinkWithSize(size)
* avatars.GenerateEmailAvatarFastLink(email, size)
* avatars.GenerateEmailAvatarFinalLink(email, size)

And many duplicated code are deleted in route handler, the handler and the model share the same avatar logic now.
2021-10-06 01:25:46 +02:00
Romain 987152ba40
Add metrics to get issues by repository (#17225) 2021-10-05 20:39:37 +02:00
Romain fc5ee1edf9
Add metrics to get issues by label (#17201)
* Add metrics to get issues by label

* Add comment on IssueByLabelCount

* Code review - Unify "AS" in SQL (#17201)

* Code review - Remove useless join (#17201)

* Code review - Disable issue_by_label by default in settings (#17201)

* use e

* restore empty line

* update docs

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-10-03 18:46:44 -04:00
Romain c64e2a319f
Add projects and project boards in exposed metrics (#17202)
* Add projects and project boards in exposed metrics

* Refactor db.GetEngine

Co-authored-by: delvh <dev.lh@web.de>
2021-10-02 03:32:15 +02:00
Norwin 3bbdce2601
API: add html urls to notification subjects (#17178)
* API: add html urls to notification subjects

* add "Repository"

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-09-30 07:17:39 +03:00
techknowlogick e9a9c20d52
Create pub/priv keypair for federation (#17071)
* add logic for creating pub/priv keypair for federation

* Apply suggestions from code review

Co-authored-by: delvh <dev.lh@web.de>

* make fmt

* Update modules/activitypub/keypair.go

Co-authored-by: delvh <dev.lh@web.de>

* add tests

* fix revert

* more tests

* Apply suggestions from code review

Co-authored-by: delvh <dev.lh@web.de>

* make fmt

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
2021-09-28 15:19:22 -04:00
Lunny Xiao 37b29319aa
Fix bug of get context user (#17169)
Co-authored-by: 6543 <6543@obermui.de>
2021-09-28 21:13:04 +08:00
techknowlogick babab0bf5e
Add nodeinfo endpoint for federation purposes (#16953)
Nodeinfo is a way to expose certain metadata about a server for use of discovery regarding functionality of its federation capabilities.

Two endpoints are required:
1. `/.well-known/nodeinfo` which informs client where it can find the location of the location of its metadata (including which version of the schema is used)
2. the endpoint which exposes the metadata in json format according to schema.

Notes:
* `openRegistrations` is a required field, but I propose to set to false as default in case someone writes a crawler to discover "open" gitea instances
* to limit data leakage I also propose to not include the `usage` field (note it is required so it should be included, but left as empty).

More info:
https://github.com/jhass/nodeinfo
https://github.com/jhass/nodeinfo/tree/main/schemas/2.1
http://nodeinfo.diaspora.software/protocol.html
2021-09-28 01:38:06 +02:00
Norwin f48dce3176
Don't return binary file changes in raw PR diffs by default (#17158)
* return diffs without binary file content change

* ?binary=true option to restore old behaviour

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: zeripath <art27@cantab.net>
2021-09-27 17:09:49 -04:00
zeripath e8574f2f7d
Nicely handle missing user in collaborations (#17049)
* Nicely handle missing user in collaborations

It is possible to have a collaboration in a repository which refers to a no-longer
existing user. This causes the repository transfer to fail with an unusual error.

This PR makes `repo.getCollaborators()` nicely handle the missing user by ghosting
the collaboration but also adds consistency check. It also adds an
Access consistency check.

Fix #17044

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2021-09-27 19:07:19 +01:00
zeripath b5856c4437
Create doctor command to fix repo_units broken by dumps from 1.14.3-1.14.6 (#17136)
There was a serious issue with the `gitea dump` command in 1.14.3-1.14.6 which led to corruption of the `config` field of the `repo_unit` table. 

This PR adds a doctor command to attempt to fix the broken repo_units. Users affected by #16961 should run:

```
gitea doctor --fix --run fix-broken-repo-units
```

Fix #16961

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-27 16:55:12 +01:00
Gwyneth Morgan 4e0cca3f7d
Use light/dark theme based on system preference (#17051)
Add a new default theme `auto`, which will automatically switch between
`gitea` (light) and `arc-green` (dark) themes depending on the user's
operating system settings.

Closes: #8183
2021-09-27 15:47:44 +01:00
zeripath 920608e592
Support direct comparison (git diff a..b) as well merge comparison (a...b) (#16635)
This PR changes the compare page to make the "..." in the between branches a clickable
link. This changes the comparison type from "..." to "..". Similarly it makes the
initial compare icon clickable to switch the head and base branches.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-09-27 13:19:34 +01:00
Alexey 〒erentyev 7e9bd206fd
Fix bundle creation (#17079)
Signed-off-by: Alexey Terentyev <axifnx@gmail.com>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Gwyneth Morgan <87623694+gwymor@users.noreply.github.com>
Co-authored-by: Gwyneth Morgan <gwymor@tilde.club>
2021-09-26 00:29:25 +03:00
Lunny Xiao 91e21d4fca
Move twofactor to models/login (#17143) 2021-09-25 15:00:12 +02:00
zeripath 623d2dd411
Prevent panic in Org mode HighlightCodeBlock (#17140)
When rendering source in org mode there is a mistake in the highlight code that
causes a panic.

This PR fixes this.

Fix #17139

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-24 14:29:32 +01:00
Lunny Xiao 5842a55b31
Move login related structs and functions to models/login (#17093)
* Move login related structs and functions to models/login

* Fix test

* Fix lint

* Fix lint

* Fix lint of windows

* Fix lint

* Fix test

* Fix test

* Only load necessary fixtures when preparing unit tests envs

* Fix lint

* Fix test

* Fix test

* Fix error log

* Fix error log

* Fix error log

* remove unnecessary change

* fix error log

* merge main branch
2021-09-24 19:32:56 +08:00
zeripath 9302eba971
DBContext is just a Context (#17100)
* DBContext is just a Context

This PR removes some of the specialness from the DBContext and makes it context
This allows us to simplify the GetEngine code to wrap around any context in future
and means that we can change our loadRepo(e Engine) functions to simply take contexts.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix unit tests

Signed-off-by: Andrew Thornton <art27@cantab.net>

* another place that needs to set the initial context

Signed-off-by: Andrew Thornton <art27@cantab.net>

* avoid race

Signed-off-by: Andrew Thornton <art27@cantab.net>

* change attachment error

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-23 23:45:36 +08:00
Eng Zer Jun f2e7d5477f
refactor: move from io/ioutil to io and os package (#17109)
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-09-22 13:38:34 +08:00
wxiaoguang b231d0deab
Ignore Sync errors on pipes when doing CheckAttributeReader.CheckPath, fix the hang of git cat-file (#17096)
* Ignore Sync errors on pipes when doing `CheckAttributeReader.CheckPath`

* apply env patch

* Drop the Sync and fix a number of issues with the Close function

Signed-off-by: Andrew Thornton <art27@cantab.net>

* add logs for DBIndexer and CheckPath

* Fix some more closing bugs

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add test case for language_stats

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update modules/indexer/stats/db.go

Co-authored-by: Lauris BH <lauris@nix.lv>

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
2021-09-20 20:46:51 +01:00
Lunny Xiao a4bfef265d
Move db related basic functions to models/db (#17075)
* Move db related basic functions to models/db

* Fix lint

* Fix lint

* Fix test

* Fix lint

* Fix lint

* revert unnecessary change

* Fix test

* Fix wrong replace string

* Use *Context

* Correct committer spelling and fix wrong replaced words

Co-authored-by: zeripath <art27@cantab.net>
2021-09-19 19:49:59 +08:00
silverwind d04e581f09
Switch migration icon to svg (#15954)
Followup on https://github.com/go-gitea/gitea/pull/15952, use SVG for migration icon.

<img width="541" alt="Screen Shot 2021-05-23 at 00 26 12" src="https://user-images.githubusercontent.com/115237/119242417-c1a37600-bb5d-11eb-9f97-a80aa89741ee.png">
<img width="540" alt="Screen Shot 2021-05-23 at 00 25 39" src="https://user-images.githubusercontent.com/115237/119242419-c2d4a300-bb5d-11eb-9792-1e6e4092c9f9.png">
2021-09-18 17:22:51 +01:00
zeripath 25533657f6
Add caller to cat-file batch calls (#17082)
Some people still appear to report unclosed cat-files. This PR simply adds the caller
to the process descriptor for the CatFileBatch and CatFileBatchCheck calls.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-09-17 20:54:15 -04:00
zeripath 27b351aba5
Make LDAP be able to skip local 2FA (#16954)
This PR extends #16594 to allow LDAP to be able to be set to skip local 2FA too. The technique used here would be extensible to PAM and SMTP sources.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-17 12:43:47 +01:00
zeripath 8de44d1995
Clean-up HookPreReceive and restore functionality for pushing non-standard refs (#16705)
* Clean-up HookPreReceive and restore functionality for pushing non-standard refs

There was an inadvertent breaking change in #15629 meaning that notes refs and other
git extension refs will be automatically rejected.

Further following #14295 and #15629 the pre-recieve hook code is untenably long and
too complex.

This PR refactors the hook code and removes the incorrect forced rejection of
non-standard refs.

Fix #16688

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-16 15:34:54 +02:00
Lunny Xiao 45ebcb0d1c
Fix bug of migrate comments which only fetch one page (#17055)
* Fix bug of migrate comments which only fetch one page

* add next page to trace

Co-authored-by: zeripath <art27@cantab.net>
2021-09-15 19:30:19 +08:00
zeripath 976db2a8b7
Do not show issue context popup on external issues (#17050)
The issues pop-up context cannot work for external issues - therefore do not show
these.

Fix #17047

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-15 09:45:27 +01:00
Lunny Xiao e2f0ab3343
Add doctor dbconsistency check for release and attachment (#16978) 2021-09-14 20:41:40 +01:00
zeripath 0a542dd59f
Decrement Fork Num when converting from Fork (#17035)
When converting repositories from forks to normal the root NumFork needs to be
decremented too.

Fix #17026

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-14 18:07:08 +01:00
zeripath 26ef180f46
Correctly rollback in ForkRepository (#17034)
The rollback functionality in
services/repository/repository.go:ForkRepository is incorrect and could
lead to a deadlock as it uses DeleteRepository to delete the rolled-back
repository - a function which creates its own transaction.

This PR adjusts the rollback function to only use RemoveAll as any
database changes will be automatically rolled-back. It also handles
panics and adjusts the Close within WithTx to ensure that if there is a
panic the session will always be closed.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-14 18:16:40 +02:00
Jimmy Praet 3d6cb25e31
Support unprotected file patterns (#16395)
Fixes #16381

Note that changes to unprotected files via the web editor still cannot be pushed directly to the protected branch. I could easily add such support for edits and deletes if needed. But for adding, uploading or renaming unprotected files, it is not trivial.

* Extract & Move GetAffectedFiles to modules/git
2021-09-11 16:21:17 +02:00
6543 77f604a928
Add skip and limit to git.GetTags (#16897)
* Make GetTags() api similar to GetBranches()
* Use it for Tag/Release page
2021-09-10 19:30:37 +02:00
6543 51578d6418
Calculate label URL on API (#16186)
close #8028
2021-09-10 18:03:16 +02:00
zeripath 0faf175694
Fix missing close in WalkGitLog (#17008)
When the external context is cancelled it is possible for the
GitLogReader to not itself be Closed.

This PR does three things:

1. Instead of adding a plain defer it wraps the `g.Close` in a func as
`g` may change.
2. It adds the missing explicit g.Close - although the defer fix makes
this unnecessary.
3. It passes down the external context as the base context for the
GitLogReader meaning that the cancellation of the external context will
pass down automatically.

Fix #17007

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-10 17:46:12 +08:00
zeripath 248b96d8a3
Use git attributes to determine generated and vendored status for language stats and diffs (#16773)
Replaces #16262
Replaces #16250
Replaces #14833

This PR first implements a `git check-attr` pipe reader - using `git check-attr --stdin -z --cached` - taking account of the change in the output format in git 1.8.5 and creates a helper function to read a tree into a temporary index file for that pipe reader.

It then wires this in to the language stats helper and into the git diff generation.

Files which are marked generated will be folded by default.

Fixes #14786
Fixes #12653
2021-09-09 21:13:36 +01:00
6543 e14b52ea36
Enable Malayalam, Greek, Persian, Hungarian & Indonesian by default (#16998) 2021-09-09 14:16:42 +08:00
Lunny Xiao efdbba4453
Fix bug of migrated repository not index (#16991)
Fix #16986, #16152
2021-09-08 18:43:19 +01:00
6543 f55cd033ed
Make TestOneDevDownloadRepo work again & more resistant (#16987)
* Update TestOneDevDownloadRepo since OneDev api changed

* Use test functions for migrations
2021-09-08 16:17:22 +02:00