Add fetch.writeCommitGraph to gitconfig to ensure that a commit-graph will be written
on git fetch calls.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Allow git push to work when networked file systems with mixed
ownership are used with Gitea docker images >= 1.16.6 or Gitea
binaries running alongside git versions published after 04/2022.
There are circumstances independent of Gitea (networked file systems
with various permission systems) by which the git repositories managed
by Gitea may have mixed owners. It is not a behavior that Gitea have
control over nor is it a problem as long as the permissions for Gitea to
operate are correct. Gitea instances have been operating under these
conditions for a number of years.
It is detected as a potential security risk ( see
GHSA-vw2c-22j4-2fh2
) by the most recent git versions. However, Gitea always runs git
commands with a current directory matching the repository on
which it operates. That makes Gitea immune from this security problem
and it is safe to ignore the mixed owner permission check.
This gitconfig modification is done on a file dedicated to the user
exclusively used by Gitea.
Fixes: #19455
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
* clean git support for ver < 2.0
* fine tune tests for markup (which requires git module)
* remove unnecessary comments
* try to fix tests
* try test again
* use const for GitVersionRequired instead of var
* try to fix integration test
* Refactor CheckAttributeReader to make a *git.Repository version
* update document for commit signing with Gitea's internal gitconfig
* update document for commit signing with Gitea's internal gitconfig
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
* Fix GetNote
* Only log errors if the error is not ErrNotExist
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
When Gitea is running as PID 1 git will occassionally orphan child processes leading
to (defunct) processes. This PR simply sets Setpgid to true on these child processes
meaning that these defunct processes will also be correctly reaped.
Fix#19077
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix indention
Signed-off-by: kolaente <k@knt.li>
* Add option to merge a pr right now without waiting for the checks to succeed
Signed-off-by: kolaente <k@knt.li>
* Fix lint
Signed-off-by: kolaente <k@knt.li>
* Add scheduled pr merge to tables used for testing
Signed-off-by: kolaente <k@knt.li>
* Add status param to make GetPullRequestByHeadBranch reusable
Signed-off-by: kolaente <k@knt.li>
* Move "Merge now" to a seperate button to make the ui clearer
Signed-off-by: kolaente <k@knt.li>
* Update models/scheduled_pull_request_merge.go
Co-authored-by: 赵智超 <1012112796@qq.com>
* Update web_src/js/index.js
Co-authored-by: 赵智超 <1012112796@qq.com>
* Update web_src/js/index.js
Co-authored-by: 赵智超 <1012112796@qq.com>
* Re-add migration after merge
* Fix frontend lint
* Fix version compare
* Add vendored dependencies
* Add basic tets
* Make sure the api route is capable of scheduling PRs for merging
* Fix comparing version
* make vendor
* adopt refactor
* apply suggestion: User -> Doer
* init var once
* Fix Test
* Update templates/repo/issue/view_content/comments.tmpl
* adopt
* nits
* next
* code format
* lint
* use same name schema; rm CreateUnScheduledPRToAutoMergeComment
* API: can not create schedule twice
* Add TestGetBranchNamesForSha
* nits
* new go routine for each pull to merge
* Update models/pull.go
Co-authored-by: a1012112796 <1012112796@qq.com>
* Update models/scheduled_pull_request_merge.go
Co-authored-by: a1012112796 <1012112796@qq.com>
* fix & add renaming sugestions
* Update services/automerge/pull_auto_merge.go
Co-authored-by: a1012112796 <1012112796@qq.com>
* fix conflict relicts
* apply latest refactors
* fix: migration after merge
* Update models/error.go
Co-authored-by: delvh <dev.lh@web.de>
* Update options/locale/locale_en-US.ini
Co-authored-by: delvh <dev.lh@web.de>
* Update options/locale/locale_en-US.ini
Co-authored-by: delvh <dev.lh@web.de>
* adapt latest refactors
* fix test
* use more context
* skip potential edgecases
* document func usage
* GetBranchNamesForSha() -> GetRefsBySha()
* start refactoring
* ajust to new changes
* nit
* docu nit
* the great check move
* move checks for branchprotection into own package
* resolve todo now ...
* move & rename
* unexport if posible
* fix
* check if merge is allowed before merge on scheduled pull
* debugg
* wording
* improve SetDefaults & nits
* NotAllowedToMerge -> DisallowedToMerge
* fix test
* merge files
* use package "errors"
* merge files
* add string names
* other implementation for gogit
* adapt refactor
* more context for models/pull.go
* GetUserRepoPermission use context
* more ctx
* use context for loading pull head/base-repo
* more ctx
* more ctx
* models.LoadIssueCtx()
* models.LoadIssueCtx()
* Handle pull_service.Merge in one DB transaction
* add TODOs
* next
* next
* next
* more ctx
* more ctx
* Start refactoring structure of old pull code ...
* move code into new packages
* shorter names ... and finish **restructure**
* Update models/branches.go
Co-authored-by: zeripath <art27@cantab.net>
* finish UpdateProtectBranch
* more and fix
* update datum
* template: use "svg" helper
* rename prQueue 2 prPatchCheckerQueue
* handle automerge in queue
* lock pull on git&db actions ...
* lock pull on git&db actions ...
* add TODO notes
* the regex
* transaction in tests
* GetRepositoryByIDCtx
* shorter table name and lint fix
* close transaction bevore notify
* Update models/pull.go
* next
* CheckPullMergable check all branch protections!
* Update routers/web/repo/pull.go
* CheckPullMergable check all branch protections!
* Revert "PullService lock via pullID (#19520)" (for now...)
This reverts commit 6cde7c9159a5ea75a10356feb7b8c7ad4c434a9a.
* Update services/pull/check.go
* Use for a repo action one database transaction
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: delvh <dev.lh@web.de>
* Update services/issue/status.go
Co-authored-by: delvh <dev.lh@web.de>
* Update services/issue/status.go
Co-authored-by: delvh <dev.lh@web.de>
* use db.WithTx()
* gofmt
* make pr.GetDefaultMergeMessage() context aware
* make MergePullRequestForm.SetDefaults context aware
* use db.WithTx()
* pull.SetMerged only with context
* fix deadlock in `test-sqlite\#TestAPIBranchProtection`
* dont forget templates
* db.WithTx allow to set the parentCtx
* handle db transaction in service packages but not router
* issue_service.ChangeStatus just had caused another deadlock :/
it has to do something with how notification package is handled
* if we merge a pull in one database transaktion, we get a lock, because merge infoce internal api that cant handle open db sessions to the same repo
* ajust to current master
* Apply suggestions from code review
Co-authored-by: delvh <dev.lh@web.de>
* dont open db transaction in router
* make generate-swagger
* one _success less
* wording nit
* rm
* adapt
* remove not needed test files
* rm less diff & use attr in JS
* ...
* Update services/repository/files/commit.go
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* ajust db schema for PullAutoMerge
* skip broken pull refs
* more context in error messages
* remove webUI part for another pull
* remove more WebUI only parts
* API: add CancleAutoMergePR
* Apply suggestions from code review
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* fix lint
* Apply suggestions from code review
* cancle -> cancel
Co-authored-by: delvh <dev.lh@web.de>
* change queue identifyer
* fix swagger
* prevent nil issue
* fix and dont drop error
* as per @zeripath
* Update integrations/git_test.go
Co-authored-by: delvh <dev.lh@web.de>
* Update integrations/git_test.go
Co-authored-by: delvh <dev.lh@web.de>
* more declarative integration tests (dedup code)
* use assert.False/True helper
Co-authored-by: 赵智超 <1012112796@qq.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
There is a potential rare race possible whereby the c.running channel could
be closed twice. Looking at the code I do not see a need for this c.running
channel and therefore I think we can remove this. (I think the c.running
might have been some attempt to prevent a hang but the use of os.Pipes should
prevent that.)
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
If an `os/exec.Command` is passed non `*os.File` as an input/output, go
will create `os.Pipe`s and wait for their closure in `cmd.Wait()`. If
the code following this is responsible for closing `io.Pipe`s or other
handlers then on process death from context cancellation the `Wait` can
hang.
There are two possible solutions:
1. use `os.Pipe` as the input/output as `cmd.Wait` does not wait for these.
2. create a goroutine waiting on the context cancellation that will close the inputs.
This PR provides the second option - which is a simpler change that can
be more easily backported.
Closes#19448
Signed-off-by: Andrew Thornton <art27@cantab.net>
Follows #19266, #8553, Close#18553, now there are only three `Run..(&RunOpts{})` functions.
* before: `stdout, err := RunInDir(path)`
* now: `stdout, _, err := RunStdString(&git.RunOpts{Dir:path})`
This addresses https://github.com/go-gitea/gitea/issues/18352
It aims to improve performance (and resource use) of the `SyncReleasesWithTags` operation for pull-mirrors.
For large repositories with many tags, `SyncReleasesWithTags` can be a costly operation (taking several minutes to complete). The reason is two-fold:
1. on sync, every upstream repo tag is compared (for changes) against existing local entries in the release table to ensure that they are up-to-date.
2. the procedure for getting _each tag_ involves a series of git operations
```bash
git show-ref --tags -- v8.2.4477
git cat-file -t 29ab6ce9f36660cffaad3c8789e71162e5db5d2f
git cat-file -p 29ab6ce9f36660cffaad3c8789e71162e5db5d2f
git rev-list --count 29ab6ce9f36660cffaad3c8789e71162e5db5d2f
```
of which the `git rev-list --count` can be particularly heavy.
This PR optimizes performance for pull-mirrors. We utilize the fact that a pull-mirror is always identical to its upstream and rebuild the entire release table on every sync and use a batch `git for-each-ref .. refs/tags` call to retrieve all tags in one go.
For large mirror repos, with hundreds of annotated tags, this brings down the duration of the sync operation from several minutes to a few seconds. A few unscientific examples run on my local machine:
- https://github.com/spring-projects/spring-boot (223 tags)
- before: `0m28,673s`
- after: `0m2,244s`
- https://github.com/kubernetes/kubernetes (890 tags)
- before: `8m00s`
- after: `0m8,520s`
- https://github.com/vim/vim (13954 tags)
- before: `14m20,383s`
- after: `0m35,467s`
I added a `foreachref` package which contains a flexible way of specifying which reference fields are of interest (`git-for-each-ref(1)`) and to produce a parser for the expected output. These could be reused in other places where `for-each-ref` is used. I'll add unit tests for those if the overall PR looks promising.
This follows
* https://github.com/go-gitea/gitea/issues/18553
Introduce `RunWithContextString` and `RunWithContextBytes` to help the refactoring. Add related unit tests. They keep the same behavior to save stderr into err.Error() as `RunInXxx` before.
Remove `RunInDirTimeoutPipeline` `RunInDirTimeoutFullPipeline` `RunInDirTimeout` `RunInDirTimeoutEnv` `RunInDirPipeline` `RunInDirFullPipeline` `RunTimeout`, `RunInDirTimeoutEnvPipeline`, `RunInDirTimeoutEnvFullPipeline`, `RunInDirTimeoutEnvFullPipelineFunc`.
Then remaining `RunInDir` `RunInDirBytes` `RunInDirWithEnv` can be easily refactored in next PR with a simple search & replace:
* before: `stdout, err := RunInDir(path)`
* next: `stdout, _, err := RunWithContextString(&git.RunContext{Dir:path})`
Other changes:
1. When `timeout <= 0`, use default. Because `timeout==0` is meaningless and could cause bugs. And now many functions becomes more simple, eg: `GitGcRepos` 9 lines to 1 line. `Fsck` 6 lines to 1 line.
2. Only set defaultCommandExecutionTimeout when the option `setting.Git.Timeout.Default > 0`
Strangely #19038 appears to relate to an issue whereby a tag appears to
be listed in `git show-ref --tags` but then does not appear when `git
show-ref --tags -- short_name` is called.
As a solution though I propose to stop the second call as it is
unnecessary and only likely to cause problems.
I've also noticed that the tags calls are wildly inefficient and aren't using the common cat-files - so these have been added.
I've also noticed that the git commit-graph is not being written on mirroring - so I've also added writing this to the migration which should improve mirror rendering somewhat.
Fix#19038
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
The git command by default adds a number of global arguments. These are not
helpful to be displayed in the process manager and so should be skipped for
default process descriptions.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Make SKIP_TLS_VERIFY apply to git data migrations too through adding the `-c http.sslVerify=false` option to the git clone command.
Fix#18998
Signed-off-by: Andrew Thornton <art27@cantab.net>
It appears possible that there could be a hang due to unread data from the
repo-attribute command pipes. This PR simply closes these during the defer.
Signed-off-by: Andrew Thornton <art27@cantab.net>
This code adds a simple endpoint to apply patches to repositories and
branches on gitea. This is then used along with the conflicting checking
code in #18004 to provide a basic implementation of cherry-pick revert.
Now because the buttons necessary for cherry-pick and revert have
required us to create a dropdown next to the Browse Source button
I've also implemented Create Branch and Create Tag operations.
Fix#3880Fix#17986
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Ensure git tag tests and other create test repos in tmpdir
There are a few places where tests appear to reuse testing repos which
causes random CI failures.
This PR simply changes these tests to ensure that cloning always happens
into new temporary directories.
Fix#18444
* Change log root for integration tests to use the REPO_TEST_DIR
There is a potential race in the drone integration tests whereby test-mysql etc
will start writing to log files causing make test-check fail.
Fix#18077
Signed-off-by: Andrew Thornton <art27@cantab.net>
- Pass the Global command args into serviceRPC.
- Fixes error with partial cloning.
- Add partial clone test
- Include diff
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This PR continues the work in #17125 by progressively ensuring that git
commands run within the request context.
This now means that the if there is a git repo already open in the context it will be used instead of reopening it.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Git will and can pack references into packfiles and therefore if you write/read the
files directly you will get false results. Instead you should use update-ref and
show-ref. To that end I have created three new functions in git/repo_commit.go that
will do this correctly.
Related #17191
Signed-off-by: Andrew Thornton <art27@cantab.net>
There are repeated panics in tests due to TestRepository_GetTag failing
to run properly. This happens when we attempt to reset the internal
repo for a tag which has failed to load. The problem is - the panic that
this is causing is preventing us from finding what the real error is.
This PR simply moves the failure out so we have a chance to see what
really is failing.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* Prevent off-by-one error on comments on newly appended lines
There was a bug in CutDiffAroundLine whereby if a file without a terminal new line
has a patch which appends lines to it and a comment is placed on one of those lines
the comment diff will be a line out of place.
This fixes CutDiffAroundLine to simply ignore the missing terminal newline - however,
we should really improve this rendering to add a marker to say that there was a
previously missing terminal newline.
Fix#17875
Signed-off-by: Andrew Thornton <art27@cantab.net>
The current TestPatch conflict code uses a plain git apply which does not properly
account for 3-way merging. However, we can improve things using `git read-tree -m` to
do a three-way merge then follow the algorithm used in merge-one-file. We can also use
`--patience` and/or `--histogram` to generate a nicer diff for applying patches too.
Fix#13679Fix#6417
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR contains multiple fixes. The most important of which is:
* Prevent hang in git cat-file if the repository is not a valid repository
Unfortunately it appears that if git cat-file is run in an invalid
repository it will hang until stdin is closed. This will result in
deadlocked /pulls pages and dangling git cat-file calls if a broken
repository is tried to be reviewed or pulls exists for a broken
repository.
Fix#14734Fix#9271Fix#16113
Otherwise there are a few small other fixes included which this PR was initially intending to fix:
* Fix panic on partial compares due to missing PullRequestWorkInProgressPrefixes
* Fix links on pulls pages due to regression from #17551 - by making most /issues routes match /pulls too - Fix#17983
* Fix links on feeds pages due to another regression from #17551 but also fix issue with syncing tags - Fix#17943
* Add missing locale entries for oauth group claims
* Prevent NPEs if ColorFormat is called on nil users, repos or teams.
The current implementation of checkBranchName is highly inefficient
involving opening the repository, the listing all of the branch names
checking them individually before then using using opened repo to get
the tags.
This PR avoids this by simply walking the references from show-ref
instead of opening the repository (in the nogogit case).
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR registers requests with the process manager and manages hierarchy within the processes.
Git repos are then associated with a context, (usually the request's context) - with sub commands using this context as their base context.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Use check attribute code to check the assigned language of a file and send that in to
chroma as a hint for the language of the file.
Signed-off-by: Andrew Thornton <art27@cantab.net>
There are multiple places where Gitea does not properly escape URLs that it is building and there are multiple places where it builds urls when there is already a simpler function available to use this.
This is an extensive PR attempting to fix these issues.
1. The first commit in this PR looks through all href, src and links in the Gitea codebase and has attempted to catch all the places where there is potentially incomplete escaping.
2. Whilst doing this we will prefer to use functions that create URLs over recreating them by hand.
3. All uses of strings should be directly escaped - even if they are not currently expected to contain escaping characters. The main benefit to doing this will be that we can consider relaxing the constraints on user names and reponames in future.
4. The next commit looks at escaping in the wiki and re-considers the urls that are used there. Using the improved escaping here wiki files containing '/'. (This implementation will currently still place all of the wiki files the root directory of the repo but this would not be difficult to change.)
5. The title generation in feeds is now properly escaped.
6. EscapePound is no longer needed - urls should be PathEscaped / QueryEscaped as necessary but then re-escaped with Escape when creating html with locales Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
There is a slight race in checking of a context deadline exceed in #16467
which leads to a 500 on the repository page.
The solution is to check the error coming back from `*LogNameStatusRepoParser.Next()`
and if it is the `ContextDeadlineExceeded` break from the loop.
Fix#17314
Signed-off-by: Andrew Thornton <art27@cantab.net>
core.protectNTFS protects NTFS from files which may be difficult to remove or interact
with using the win32 api, however, it also appears to prevent such files from
being entered into the git indexes - fundamentally causing breakages with PRs that
affect these files. However, deliberately setting this to false may cause security
issues due to the remain sparse checkout of files in the merge pipeline.
The only sensible option therefore is to provide an optional setting which admins
could set which would forcibly switch this off if they are affected by this issue.
Fix#17092
Signed-off-by: Andrew Thornton <art27@cantab.net>
- Update default branch if needed
- Update protected branch if needed
- Update all not merged pull request base branch name
- Rename git branch
- Record this rename work and auto redirect for old branch on ui
Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: delvh <dev.lh@web.de>
One of the biggest reasons for slow repository browsing is that we wait
until last commit information has been generated for all files in the
repository.
This PR proposes deferring this generation to a new POST endpoint that
does the look up outside of the main page request.
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR changes the compare page to make the "..." in the between branches a clickable
link. This changes the comparison type from "..." to "..". Similarly it makes the
initial compare icon clickable to switch the head and base branches.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Ignore Sync errors on pipes when doing `CheckAttributeReader.CheckPath`
* apply env patch
* Drop the Sync and fix a number of issues with the Close function
Signed-off-by: Andrew Thornton <art27@cantab.net>
* add logs for DBIndexer and CheckPath
* Fix some more closing bugs
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add test case for language_stats
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update modules/indexer/stats/db.go
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Some people still appear to report unclosed cat-files. This PR simply adds the caller
to the process descriptor for the CatFileBatch and CatFileBatchCheck calls.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Fixes#16381
Note that changes to unprotected files via the web editor still cannot be pushed directly to the protected branch. I could easily add such support for edits and deletes if needed. But for adding, uploading or renaming unprotected files, it is not trivial.
* Extract & Move GetAffectedFiles to modules/git
When the external context is cancelled it is possible for the
GitLogReader to not itself be Closed.
This PR does three things:
1. Instead of adding a plain defer it wraps the `g.Close` in a func as
`g` may change.
2. It adds the missing explicit g.Close - although the defer fix makes
this unnecessary.
3. It passes down the external context as the base context for the
GitLogReader meaning that the cancellation of the external context will
pass down automatically.
Fix#17007
Signed-off-by: Andrew Thornton <art27@cantab.net>
Replaces #16262
Replaces #16250
Replaces #14833
This PR first implements a `git check-attr` pipe reader - using `git check-attr --stdin -z --cached` - taking account of the change in the output format in git 1.8.5 and creates a helper function to read a tree into a temporary index file for that pipe reader.
It then wires this in to the language stats helper and into the git diff generation.
Files which are marked generated will be folded by default.
Fixes#14786Fixes#12653
* make sure headGitRepo is closed on err too
* refactor
* Fix git.Blob.DataAsync(): exec cancel since we already read all bytes (close pipe since we return a NopCloser)
* Add proxy settings and support for migration and webhook
* Fix default value
* Add newline for example ini
* Add lfs proxy support
* Fix lint
* Follow @zeripath's review
* Fix git clone
* Fix test
* missgin http requests for proxy
* use empty
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
* Add info about list endpoints to CONTRIBUTING.md
* Let all list endpoints return X-Total-Count header
* Add TODOs for GetCombinedCommitStatusByRef
* Fix models/issue_stopwatch.go
* Rrefactor models.ListDeployKeys
* Introduce helper func and use them for SetLinkHeader related func
* Fix modified files list in webhooks when there is a space
There is an unfortunate bug with GetCommitFileStatus where files with
spaces are misparsed and split at the space.
There is a second bug because modern gits detect renames meaning that
this function no longer works correctly.
There is a third bug in that merge commits don't have their modified
files detected correctly.
Fix#15865
Signed-off-by: Andrew Thornton <art27@cantab.net>
Following the merging of https://github.com/go-git/go-git/pull/330 we
can now add a setting to avoid go-git reading and caching large objects.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Use `..` instead of `...` with `rev-list`. In combination with #16282 the receiver can get the correct commit. The behaviour is now like Github.
fixes#11802
There is a bug with last commit cache recursive cache where the last
commit information that refers to the current tree itself will cause a
panic due to its path ("") not being included in the expected tree entry
paths.
This PR fixes this by skipping the missing entry.
Fix#16290
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
remove log() func from gogs times and switch to proper logging
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
* Fix 500 Error with branch and tag sharing the same name #15592
Fixed 500 error while create Pull request when there are more
than one sources (branch, tag) with the same name
Fix#15592
Signed-off-by: Viktor Yakovchuk <viktor@yakovchuk.net>
* fix logging
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: 6543 <6543@obermui.de>
* More efficiently parse shas for shaPostProcessor
The shaPostProcessor currently repeatedly calls git rev-parse --verify on both backends
which is fine if there is only one thing that matches a sha - however if there are
multiple things then this becomes wildly inefficient.
This PR provides functions for both backends which are much faster to use.
Fix#16092
* Add ShaExistCache to RenderContext
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
* Improve get last commit using git log --name-status
git log --name-status -c provides information about the diff between a
commit and its parents. Using this and adjusting the algorithm to use
the first change to a path allows for a much faster generation of commit
info.
There is a subtle change in the results generated but this will cause
the results to more closely match those from elsewhere.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Make modules/context.Context a context.Context
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Simplify context calls
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Set the base context for requests to the HammerContext
Signed-off-by: Andrew Thornton <art27@cantab.net>
* pass context into get-last-commit
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Make commit_info cancellable
Signed-off-by: Andrew Thornton <art27@cantab.net>
* use context as context
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
* Added type sniffer.
* Switched content detection from base to typesniffer.
* Added GuessContentType to Blob.
* Moved image info logic to client.
Added support for SVG images in diff.
* Restore old blocked svg behaviour.
* Added missing image formats.
* Execute image diff only when container is visible.
* add margin to spinner
* improve BIN tag on image diffs
* Default to render view.
* Show image diff on incomplete diff.
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Unfortunately some old repositories can have tags with empty Tagger, Commit
or Author. Go-Git variants will always have empty values for these whereas
the native git variant leaves them at nil. The simplest solution is just to
always have these set to empty Signatures.
v156 migration also makes the incorrect assumption that these cannot be empty.
Therefore add some handling to this and add logging and adjust broken
logging elsewhere in this migration.
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Improve performance when there are multiple commits in the last commit cache
* read refs directly if we can
Signed-off-by: Andrew Thornton <art27@cantab.net>
Fix#15236
* Do not do 40byte conversion within ParseTreeLine
* Missed a to40ByteSHA
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
There is a weird gotcha with GetTagCommitID that because it uses git rev-list
can cause an ambiguous argument error.
This PR simply makes tags use the same code as branches.
Signed-off-by: Andrew Thornton <art27@cantab.net>
`enry.IsVendor` is kinda slow as it simply iterates across all regexps.
This PR ajdusts the regexps to combine them to make this process a
little quicker.
Related #15143
Signed-off-by: Andrew Thornton <art27@cantab.net>
* response 404 for diff/patch of a commit that not exist
fix#15217
Signed-off-by: a1012112796 <1012112796@qq.com>
* Update routers/repo/commit.go
Co-authored-by: silverwind <me@silverwind.io>
* use ctx.NotFound()
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
* Make auto check manual merge as a chooseable mod and add manual merge way on ui
as title, Before this pr, we use same way with GH to check manually merge.
It good, but in some special cases, misjudgments can occur. and it's hard
to fix this bug. So I add option to allow repo manager block "auto check manual merge"
function, Then it will have same style like gitlab(allow empty pr). and to compensate for
not being able to detect THE PR merge automatically, I added a manual approach.
Signed-off-by: a1012112796 <1012112796@qq.com>
* make swager
* api support
* ping ci
* fix TestPullCreate_EmptyChangesWithCommits
* Apply suggestions from code review
Co-authored-by: zeripath <art27@cantab.net>
* Apply review suggestions and add test
* Apply suggestions from code review
Co-authored-by: zeripath <art27@cantab.net>
* fix build
* test error message
* make fmt
* Fix indentation issues identified by @silverwind
Co-authored-by: silverwind <me@silverwind.io>
* Fix tests and make manually merged disabled error on API the same
Signed-off-by: Andrew Thornton <art27@cantab.net>
* a small nit
* fix wrong commit id error
* fix bug
* simple test
* fix test
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Extract out the common cat-file batch calls
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Move bleve and elastic indexers to use a common cat-file --batch when indexing
Signed-off-by: Andrew Thornton <art27@cantab.net>
* move catfilebatch to batch_reader and rename to batch_reader.go
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
Unfortunately `git log revision ... --skip=x -- path` skips the number of commits
not the number of commits relating to the path.
This PR changes the function to have a reader that reads and skips the
necessary number of commits by hand instead.
Fix#8716
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Support create single tag directly
support create tag with message from create release ui
Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
* CutDiffAroundLine makes the incorrect assumption that `---` and `+++` always represent part of the header of a diff.
This PR adds a flag to its parsing to prevent this problem and adds a streaming parsing technique to CutDiffAroundLine using an io.pipe instead of just sending data to an unbounded buffer.
Fix#14711
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Handle unquoted comment patch files
When making comment patches unfortunately the patch does not always quote the filename
This makes the diff --git header ambiguous again.
This PR finally adds handling for ambiguity in to parse patch
Fix#14812
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add in testing for no error
There is no way currently for CutDiffAroundLine in this test to cause an
error however, it should still be tested.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Reduce calls to git cat-file -s
There are multiple places where there are repeated calls to git cat-file
-s due to the blobs not being created with their size.
Through judicious use of git ls-tree -l and slight adjustments to the
indexer code we can avoid a lot of these calls.
* simplify by always expecting the long format
* Also always set the sized field and tell the indexer the update is sized
* Use cat-file --batch in GetLanguageStats
This PR moves to using a single cat-file --batch in GetLanguageStats
significantly reducing the number of processes spawned during language stat
processing.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* placate lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update modules/git/repo_language_stats_nogogit.go
Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: a1012112796 <1012112796@qq.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Somehow the test for detecting if branches are equal broke
this PR restores this functionality.
Fix#14502
Signed-off-by: Andrew Thornton <art27@cantab.net>
GetCommit() assumes that all 40 char strings are SHA1s. This leads to an
error if you try to do a PR on a branch which is 40 characters long.
This PR attempts the SHA first - and if it fails will switch to using rev-parse.
Fix#14470
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR improves HasPreviousCommit to prevent the automatic and recursive loading
of previous commits using git merge-base --is-ancestor and git rev-list
Fix#13684
Signed-off-by: Andrew Thornton <art27@cantab.net>
Memcached TTL cannot be > 30 days and if it is attempted the TTL is interpreted as
a unix timestamp.
This PR ensures that the TTL is switched to a unix timestamp in those cases.
Fix#14571
Signed-off-by: Andrew Thornton <art27@cantab.net>
Breaking the pipe is a valid way of killing a piped command and any error from
a broken cat-file batch command should be passed back up to the writer any way
therefore specifically logging it is unnecessary.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add pager to the branches page
* override pageSize if bigger than max
* Make branches commit range configurable
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: silverwind <me@silverwind.io>
* remove github.com/unknwon/com from models
* dont use "com.ToStr()"
* replace "com.ToStr" with "fmt.Sprint" where its easy to do
* more refactor
* fix test
* just "proxy" Copy func for now
* as per @lunny
* Fix git.parseTagData()
close#14092
* Add Test
* add message to test
* limit readers
* git tag -m trims and terminates with a newline
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
There is a slight bug in the commit_reader introduced in #13673 whereby
commit messages which have a final unterminated line miss their final line.
This PR fixes this.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Move last commit cache back into modules/git
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Remove go-git from the interface for last commit cache
Signed-off-by: Andrew Thornton <art27@cantab.net>
* move cacheref to last_commit_cache
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Remove go-git from routers/private/hook
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Move FindLFSFiles to pipeline
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Make no-go-git variants
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Submodule RefID
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix issue with GetCommitsInfo
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix GetLastCommitForPaths
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Improve efficiency
Signed-off-by: Andrew Thornton <art27@cantab.net>
* More efficiency
Signed-off-by: Andrew Thornton <art27@cantab.net>
* even faster
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Reduce duplication
* As per @lunny
Signed-off-by: Andrew Thornton <art27@cantab.net>
* attempt to fix drone
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix test-tags
Signed-off-by: Andrew Thornton <art27@cantab.net>
* default to use no-go-git variants and add gogit build tag
Signed-off-by: Andrew Thornton <art27@cantab.net>
* placate lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @6543
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
After cancelling the context we still need to wait for the
command to finish otherwise zombie processes may occur
Fix#13987
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
* Multiple GitGraph improvements.
Add backend support for excluding PRs, selecting branches and files.
Fix#10327
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @silverwind
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @silverwind
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Only show refs in dropdown we display on the graph
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @silverwind
Signed-off-by: Andrew Thornton <art27@cantab.net>
* use flexbox for ui header
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Move Hide Pull Request button to the dropdown
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add SHA and user pictures
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix test
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix test 2
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fixes
* async
* more tweaks
* use tabs in tmpl
Signed-off-by: Andrew Thornton <art27@cantab.net>
* remove commented thing
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix linting
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update web_src/js/features/gitgraph.js
Co-authored-by: silverwind <me@silverwind.io>
* graph tweaks
* more tweaks
* add title
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix loading indicator z-index and position
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
Go-version constraints ignore pre-releases.
Rather than change the library further this PR simply changes
the git version comparison to use simple version compare ignoring the
issue of pre-releases.
Signed-off-by: Andrew Thornton <art27@cantab.net>
This PR updates golangci-lint to the latest version 1.31.0.
The upgrade introduced a new check for which I've fixed or disabled most cases.
Signed-off-by: kolaente <k@knt.li>
* Cache last commit when pushing for big repository
* Fix bug
* detect force push
* Refactor cache push
* Finish cache last commit info when push
* Some improvements
* Fix lint
* Remove unused changes
* Move pull request test before cache
* Fix test mysql
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
If a README file is a symlink to a submodule Gitea the view branch page
will return a 500.
The underlying problem is a missed conversion of an
plumbing.ErrObjectNotFound in git/tree_blob.go.
Fix#12599
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
`GetCommitsFromIDs` is only used in one place: `LoadPushCommits` where
it expects that `c.Commits` is not nil.
This potentially nil set causes a NPE in in #12953Fix#12953
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Don't automatically delete repository files if they are present
Prior to this PR Gitea would delete any repository files if they are
present during creation or migration. This can in certain circumstances
lead to data-loss and is slightly unpleasant.
This PR provides a mechanism for Gitea to adopt repositories on creation
and otherwise requires an explicit flag for deletion.
PushCreate is slightly different - the create will cause adoption if
that is allowed otherwise it will delete the data if that is allowed.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update swagger
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix tests and migrate overwrite
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @lunny
Only offer to adopt or overwrite if the user can do that.
Allow the site administrator to adopt or overwrite in all
circumstances
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Use setting.Repository.DefaultBranch for the default branch
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Always set setting.Repository.DefaultBranch
Signed-off-by: Andrew Thornton <art27@cantab.net>
* update swagger
Signed-off-by: Andrew Thornton <art27@cantab.net>
* update templates
Signed-off-by: Andrew Thornton <art27@cantab.net>
* ensure repo closed
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Rewrite of adoption as per @6543 and @lunny
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Apply suggestions from code review
* update swagger
Signed-off-by: Andrew Thornton <art27@cantab.net>
* missing not
Signed-off-by: Andrew Thornton <art27@cantab.net>
* add modals and flash reporting
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Make the unadopted page searchable
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add API
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix swagger
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix swagger
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Handle empty and non-master branched repositories
Signed-off-by: Andrew Thornton <art27@cantab.net>
* placate lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* remove commented out code
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Add configurable Trust Models
Gitea's default signature verification model differs from GitHub. GitHub
uses signatures to verify that the committer is who they say they are -
meaning that when GitHub makes a signed commit it must be the committer.
The GitHub model prevents re-publishing of commits after revocation of a
key and prevents re-signing of other people's commits to create a
completely trusted repository signed by one key or a set of trusted
keys.
The default behaviour of Gitea in contrast is to always display the
avatar and information related to a signature. This allows signatures to
be decoupled from the committer. That being said, allowing arbitary
users to present other peoples commits as theirs is not necessarily
desired therefore we have a trust model whereby signatures from
collaborators are marked trusted, signatures matching the commit line
are marked untrusted and signatures that match a user in the db but not
the committer line are marked unmatched.
The problem with this model is that this conflicts with Github therefore
we need to provide an option to allow users to choose the Github model
should they wish to.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Adjust locale strings
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @6543
Co-authored-by: 6543 <6543@obermui.de>
* Update models/gpg_key.go
* Add migration for repository
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Right now we only compare the hostname from a submodule with the prefixURL it is viewed from to check if the submodule is hosted on the same Gitea instance. This adds an additional check to compare it against SSH_DOMAIN as well since the same Gitea instance might have a different hostname for SSH and if the submodule uses that hostname we should also detect that and link to the proper DOMAIN value.
Fixes#12747, #9756
* refactor repo_stats to use os.pipe
* woops. missing reader.
* stdout not stderr, woops
* Fix copyright date and ensure that the stderr is collected
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Set the context for CreateArchive to that of the request to ensure that archives
are only built for as long as a request is requesting them
Fix#11551
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
* go1.15
* update makefile xgo version
* fix vet issue
* update docs to version of go in use
* add TODO for asyncpreemptoff
Co-authored-by: Lauris BH <lauris@nix.lv>
Replace all calls to os.Remove/os.RemoveAll by retrying util.Remove/util.RemoveAll and remove circular dependencies from util.
Fix#12339
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
#11716 reports multiple git blame processes hanging around
this was thought to be due to timeouts, however on closer look this
appears to be due to the Close() function of the BlameReader hanging
with a blocked stdout pipe.
This PR fixes this Close function to:
* Cancel the context of the cmd
* Close the StdoutReader - ensuring that the output pipe is closed
Further it makes the context of the `git blame` command a child of the
request context - ensuring that even if Close() is not called, on
cancellation of the Request the blame is command will also be cancelled.
Fixes#11716Closes#11727
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Only write to global gitconfig if necessary
Fix#11855
Signed-off-by: Andrew Thornton <art27@cantab.net>
* placate lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
#11846 Introduced feature to show exact tag on commit view. However if a repo has no tags at all git prints out a separate and unhandled error " No names found, cannot describe anything."
Adding --always to the command makes it always use the error in the style of "fatal: no tag exactly matches" even if there are no tags at all.
Fixes#11869Fixes#11868
* Handle more pathological branch and tag names
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix failing test
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Change language statistics to save size instead of percentage in database
Co-Authored-By: Cirno the Strongest <1447794+CirnoT@users.noreply.github.com>
* Do not exclude if only language
* Fix edge cases with special langauges
Co-authored-by: Cirno the Strongest <1447794+CirnoT@users.noreply.github.com>
Unfortunately #11614 introduced a bug whereby the initial commit of a
repository could not be seen due to there being no parent commit to
create a clear diff from.
Here we create a diffstat from the difference between the parentless SHA and the SHA of the empty tree - a constant known to git. (With thanks to @L0veSunshine for informing me of this SHA)
Thanks to @a1012112796 for initial attempt to fix.
Fix#11650Closes#11674
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-Authored-By: L0veSunshine <xuan199651@gmail.com>
* Refactor Cron and merge dashboard tasks
* Merge Cron and Dashboard tasks
* Make every cron task report a system notice on completion
* Refactor the creation of these tasks
* Ensure that execution counts of tasks is correct
* Allow cron tasks to be started from the cron page
* golangci-lint fixes
* Enforce that only one task with the same name can be registered
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix name check
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @guillep2k
* as per @lafriks
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add git.CommandContext variants
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Properly generate ref URLs
Tags used to not generate correct URLs (src/branch/tags/1.0.0 instead of
src/tags/1.0.0).
Also cleans up some code around it with the created helper functions.
* Fix formatting and create migration
* Add copyright head to utils_test
* Use a raw query for the ref migration
* Remove semicolon
* Quote column and table names in migration SQL
* Change || to CONCAT, since MSSQL does not support ||
* Make migration engine aware
* Add missing import
* Move ref EndName and URL to the issue service
* Fix tests
* Add test for commit refs
* Update issue.go
* Use the right command for building JavaScript bundles
* Prepare for merge
* Check for refs/* before prepending in migration
* Update services/issue/issue_test.go
* Update modules/git/utils_test.go
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: techknowlogick <matti@mdranta.net>
* Allow compare page to look up base, head, own-fork, forkbase-of-head
Signed-off-by: Andrew Thornton <art27@cantab.net>
* as per @guillep2k
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Update routers/repo/compare.go
* as per @guillep2k
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Rationalise the names a little
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Rationalise the names a little (2)
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Fix 500 with fork of fork
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Prevent 500 on compare different trees
Signed-off-by: Andrew Thornton <art27@cantab.net>
* dotdotdot is perfectly valid in both usernames and repo names
Signed-off-by: Andrew Thornton <art27@cantab.net>
* ensure we can set the head and base repos too
Signed-off-by: Andrew Thornton <art27@cantab.net>
* ensure we can set the head and base repos too (2)
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* only set headRepo == baseRepo if isSameRepo
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Manage port in submodule refurl
Fix#11304
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix lint
Signed-off-by: Andrew Thornton <art27@cantab.net>
* URLJoin is causes a cyclic dependency and possibly isn't what what we want anyway
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Protect against leading .. in scp syntax
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
* Support unicode emojis and remove emojify.js
This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea.
This works in a few ways:
First it adds emoji parsing support into gitea itself. This allows us to
* Render emojis from valid alias (😄)
* Detect unicode emojis and let us put them in their own class with proper aria-labels and styling
* Easily allow for custom "emoji"
* Support all emoji rendering and features without javascript
* Uses plain unicode and lets the system render in appropriate emoji font
* Doesn't leave us relying on external sources for updates/fixes/features
That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also)
For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method.
The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released.
I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens.
I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others.
Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary.
Fixes: https://github.com/go-gitea/gitea/issues/9182
Fixes: https://github.com/go-gitea/gitea/issues/8974
Fixes: https://github.com/go-gitea/gitea/issues/8953
Fixes: https://github.com/go-gitea/gitea/issues/6628
Fixes: https://github.com/go-gitea/gitea/issues/5130
* add new shared function emojiHTML
* don't increase emoji size in issue title
* Update templates/repo/issue/view_content/add_reaction.tmpl
Co-Authored-By: 6543 <6543@obermui.de>
* Support for emoji rendering in various templates
* Render code and review comments as they should be
* Better way to handle mail subjects
* insert unicode from tribute selection
* Add template helper for plain text when needed
* Use existing replace function I forgot about
* Don't include emoji greater than Unicode Version 12
Only include emoji and aliases in JSON
* Update build/generate-emoji.go
* Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have
* final updates
* code review
* code review
* hard code gitea custom emoji to match previous behavior
* Update .eslintrc
Co-Authored-By: silverwind <me@silverwind.io>
* disable preempt
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
As seen in trouble shooting #11032 the new feature of Go 1.14 is causing several second delays in startup in certain situations. Debugging shows it spending several seconds handling SIGURG commands during init:
```
6922:04:51.984234 trace init() ./modules/queue/unique_queue_wrapped.go
remote: ) = 69 <0.000012>
remote: [pid 15984] 22:04:51 write(1, "\ttime taken: 236.761\302\265s\n\n", 25 time taken: 236.761µs
remote:
remote: ) = 25 <0.000011>
remote: [pid 15984] 22:04:51 --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=15984, si_uid=0} ---
remote: [pid 15984] 22:04:52 --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=15984, si_uid=0} ---
remote: [pid 15984] 22:04:52 --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=15984, si_uid=0} ---
```
This causes up to 20 seconds added to a push in some cases as it happens for each call of the gitea hook command. This is likely the cause of #10661 as well and would start to effect users once we release 1.12 which would be the first release compiled with Go 1.14. I suspect this is just a slight issue with the upstream implementatation as there have been a few very similar bugs fixed and reported:
https://github.com/golang/go/issues/37741https://github.com/golang/go/issues/37942
We should revisit this in the future and see if a newer version of Go has solved it, but for now disable this option in the environment that gitea hook runs in to avoid it.
* Handle push rejections and push out-of-date in branch creation and
file upload.
* Remove the duplicated sanitize from services/pull/merge
* Move the errors Err(Merge)PushOutOfDate and ErrPushRejected to
modules/git
* Handle errors better in the upload file dialogs
Fix#10460
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Fix panic in API pulls when headbranch does not exist
* refix other reference to plumbing.ErrReferenceNotFound
Signed-off-by: Andrew Thornton <art27@cantab.net>
* refactor
* optimize
* remove Iretating function
LoadWatchUsers do not load Users into IW object and it is used only in api ... so move this logic
* remove unessesary
* Apply suggestions from code review
Thx
Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>
* make Tests more robust
* fix rebase
* restart CI
* CI no dont hit sqlites deadlock
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Move langauge detection to separate module to be more reusable
Add option to disable vendored file exclusion from file search
Allways show all language stats for search
* Implementation for calculating language statistics
Impement saving code language statistics to database
Implement rendering langauge stats
Add primary laguage to show in repository list
Implement repository stats indexer queue
Add indexer test
Refactor to use queue module
* Do not timeout for queues
* Cache last commit to accelerate the repository directory page visit
* Default use default cache configuration
* add tests for last commit cache
* Simplify last commit cache
* Revert Enabled back
* Change the last commit cache default ttl to 8760h
* Fix test
* Some refactor on git diff and ignore getting commit information failed on migrating pull request review comments
* fix test
* fix lint
* Change error log to warn
* Add require signed commit for protected branch
* Fix fmt
* Make editor show if they will be signed
* bugfix
* Add basic merge check and better information for CRUD
* linting comment
* Add descriptors to merge signing
* Slight refactor
* Slight improvement to appearances
* Handle Merge API
* manage CRUD API
* Move error to error.go
* Remove fix to delete.go
* prep for merge
* need to tolerate \r\n in message
* check protected branch before trying to load it
* Apply suggestions from code review
Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com>
* fix commit-reader
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Fix#5997.
If a push causes the patch/diff of a PR towards target branch to change, all existing reviews for the PR will be set and shown as stale.
New branch protection option to dismiss stale approvals are added.
To show that a review is not based on the latest PR changes, an hourglass is shown
* Adds functionality to change target branch of created pull requests
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Use const instead of var in JavaScript additions
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Check if branches are equal and if PR already exists before changing target branch
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Make sure to check all commits
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Print error messages for user as error flash message
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Disallow changing target branch of closed or merged pull requests
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Resolve conflicts after merge of upstream/master
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Change order of branch select fields
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Removes duplicate check
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Use ctx.Tr for translations
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Recompile JS
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Use correct translation namespace
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Remove redundant if condition
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Moves most change branch logic into pull service
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Completes comment
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Add Ref to ChangesPayload for logging changed target branches
instead of creating a new struct
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Revert changes to go.mod
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Directly use createComment method
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Return 404 if pull request is not found. Move written check up
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Remove variable declaration
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Return client errors on change pull request target errors
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Return error in commit.HasPreviousCommit
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Adds blank line
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Test patch before persisting new target branch
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Update patch before testing (not working)
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Removes patch calls when changeing pull request target
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Removes unneeded check for base name
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Moves ChangeTargetBranch completely to pull service. Update patch status.
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Set webhook mode after errors were validated
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Update PR in one transaction
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Move logic for check if head is equal with branch to pull model
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Adds missing comment and simplify return
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Adjust CreateComment method call
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Change graceful to use a singleton obtained through GetManager instead of a global.
* Graceful: Make TestPullRequests shutdownable
* Graceful: Make the cron tasks graceful
* Graceful: AddTestPullRequest run in graceful ctx
* Graceful: SyncMirrors shutdown
* Graceful: SetDefaultContext for Xorm to be HammerContext
* Avoid starting graceful for migrate commands and checkout
* Graceful: DeliverHooks now can be shutdown
* Fix multiple syncing errors in modules/sync/UniqueQueue & Make UniqueQueue closable
* Begin the process of making the repo indexer shutdown gracefully
* Save patches to temporary files
* Remove SavePatch and generate patches on the fly
* Use ioutil.TempDir
* fixup! Use ioutil.TempDir
* fixup! fixup! Use ioutil.TempDir
* RemoveAll LocalCopyPath() in initIntergrationTest
* Default to status checking on PR creation
* Remove unnecessary set to StatusChecking
* Protect against unable to load repo
* Handle conflicts
* Restore original conflict setting
* In TestPullRequests update status to StatusChecking before running TestPatch
The underlying implementation of os.exec uses channels and goroutines.
It is possible to have time-variant error values returned from Cmd.Wait
depending on which comes first.
Also, the git subcommand and options should be separated tokens.
Fixes a flaky test in modules/git/command_test.go
* Graceful: Create callbacks to with contexts
* Graceful: Say when Gitea is completely finished
* Graceful: Git and Process within HammerTime
Force all git commands to terminate at HammerTime
Force all process commands to terminate at HammerTime
Move almost all git processes to run as git Commands
* Graceful: Always Hammer after Shutdown
* ProcessManager: Add cancel functionality
* Fix tests
* Make sure that process.Manager.Kill() cancels
* Make threadsafe access to Processes and remove own unused Kill
* Remove cmd from the process manager as it is no longer used
* the default context is the correct context
* get rid of double till
In investigating #7947 it has become clear that the storage component of go-git repositories needs closing.
This PR adds this Close function and adds the Close functions as necessary.
In TransferOwnership the ctx.Repo.GitRepo is closed if it is open to help prevent the risk of multiple open files.
Fixes#7947
This PR migrates temp_repo.go to use git.NewCommand instead creating processes by itself - this fixes the problem underlying PR #8905.
There are other places that run git outside of the controlled locale defined in #8548 but temp_repo.go is the only cause of failure of local testing in cases where English is not the default - implying that error messages from those other commands are not interpreted.
Replaces #8905
There are two major components to this PR:
* This PR handles merge and rebase failures from merging a little more nicely with Flash errors rather a 500.
* All git commands are run in the LC_ALL="C" environment to ensure that error messages are in English. This DefaultLocale is defined in a way that if necessary (due to platform weirdness) it can be overridden at build time using LDFLAGS="-X "code.gitea.io/gitea/modules/git.DefaultLocale=C"" with C changed for the locale as necessary.
This PR adds basic repository LFS management UI including the ability to find all possible pointers within the repository. Locks are not managed at present but would be addable through some simple additions.
* Add basic repository lfs management
* add auto-associate function
* Add functionality to find commits with this lfs file
* Add link to find commits on the lfs file view
* Adjust commit view to state the likely branch causing the commit
* Only read Oid from database
* Ensure defaultGPGSettings not nil
* Ensure that coerced commits gain a reference to the repo
* Add warning if trying to get defaultgpgsetting on an unattached commit
This PR fixes#7598 by providing a configurable way of signing commits across the Gitea instance. Per repository configurability and import/generation of trusted secure keys is not provided by this PR - from a security PoV that's probably impossible to do properly. Similarly web-signing, that is asking the user to sign something, is not implemented - this could be done at a later stage however.
## Features
- [x] If commit.gpgsign is set in .gitconfig sign commits and files created through repofiles. (merges should already have been signed.)
- [x] Verify commits signed with the default gpg as valid
- [x] Signer, Committer and Author can all be different
- [x] Allow signer to be arbitrarily different - We still require the key to have an activated email on Gitea. A more complete implementation would be to use a keyserver and mark external-or-unactivated with an "unknown" trust level icon.
- [x] Add a signing-key.gpg endpoint to get the default gpg pub key if available
- Rather than add a fake web-flow user I've added this as an endpoint on /api/v1/signing-key.gpg
- [x] Try to match the default key with a user on gitea - this is done at verification time
- [x] Make things configurable?
- app.ini configuration done
- [x] when checking commits are signed need to check if they're actually verifiable too
- [x] Add documentation
I have decided that adjusting the docker to create a default gpg key is not the correct thing to do and therefore have not implemented this.
* Change tests to make it possible to run TestGit with 1.7.2
* Make merge run on 1.7.2
* Fix tracking and staging branch name problem
* Ensure that git 1.7.2 works on tests
* ensure that there is no chance for conflicts
* Fix-up missing merge issues
* Final rm
* Ensure LFS filters run on the tests
* Do not sign commits from temp repo
* Restore tracking fetch change
* Apply suggestions from code review
* Update modules/repofiles/temp_repo.go
* Resolve error when comparing images
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Check blob existence instead of git-ls when checking if file exists
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Show file metadata also when a file was newly added
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Fixes error in commit view
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Excludes assigning path and image infos for compare routers to service package
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Removes nil default and fixes import order
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Adds missing comments
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Moves methods for assigning compare data to context into repo router package
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Show image compare for deleted images as well. Simplify check if image should be displayed
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Adds side-by-side diff for images
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Explain blank imports
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Use complete word for width and height labels on image compare
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Update index.css from master
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Moves ImageInfo to git commit file
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Assign ImageInfo function for template and sets correct target for BeforeSourcePath
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Adds missing comment
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Return error if ImageInfo failed
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Avoid template panic when ImageInfo failed for some reason
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Show file size on image diff
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Removes unused helper function
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Reverts copyright year change
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Close file reader
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Update commit.go
Sets correct data key
* Moves reader.Close() up a few lines
* Updates index.css
* Updates CSS file
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Transfers adjustments for image compare to compare.go file
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Adjusts variable name
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Apply lesshint recommendations
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Do not show old image on image compare if it is not in index of base commit
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Change file size text
Signed-off-by: Mario Lubenka <mario.lubenka@googlemail.com>
* Fix reading notes from nested trees
The GIT documentation for notes states "Permitted pathnames have the
form ab/cd/ef/.../abcdef...: a sequence of directory names of two
hexadecimal digits each followed by a filename with the rest of
the object ID."
* Add test case
* Fix new lines
* search commits via commit hash
Signed-off-by: Gary Kim <gary@garykim.dev>
* Also include all option for hash search
Signed-off-by: Gary Kim <gary@garykim.dev>
* Remove code duplication in commit search
Signed-off-by: Gary Kim <gary@garykim.dev>
* Add case ignore to commit hash search
Signed-off-by: Gary Kim <gary@garykim.dev>
