Commit graph

26 commits

Author SHA1 Message Date
Twenty Panda 5561e80b04 fix(hook): ignore unknown push options instead of failing
If a repository has

git config --add push.pushOption submit=".sourcehut/*.yml"

it failed when pushed because of the unknown submit push
option. It will be ignored instead.

Filtering out the push options is done in an earlier stage, when the
hook command runs, before it submits the options map to the private
endpoint.

* move all the push options logic to modules/git/pushoptions
* add 100% test coverage for modules/git/pushoptions

Test coverage for the code paths from which code was moved to the
modules/git/pushoptions package:

* cmd/hook.go:runHookPreReceive
* routers/private/hook_pre_receive.go:validatePushOptions
  tests/integration/git_push_test.go:TestOptionsGitPush runs through
  both. The test verifying the option is rejected was removed and, if
  added again, will fail because the option is now ignored instead of
  being rejected.

* cmd/hook.go:runHookProcReceive
* services/agit/agit.go:ProcReceive
  tests/integration/git_test.go: doCreateAgitFlowPull runs through
  both. It uses variations of AGit related push options.

* cmd/hook.go:runHookPostReceive
* routers/private/hook_post_receive.go:HookPostReceive
  tests/integration/git_test.go:doPushCreate called by TestGit/HTTP/sha1/PushCreate
  runs through both.
  Note that although it provides coverage for this code path it does not use push options.

Fixes: https://codeberg.org/forgejo/forgejo/issues/3651
2024-07-02 21:39:01 +02:00
oliverpool 5e73c67d67 fix: hook post-receive for sha256 repos 2024-05-15 16:43:16 +02:00
Lunny Xiao eb792d9f8a
Move database operations of merging a pull request to post receive hook and add a transaction (#30805)
Merging PR may fail because of various problems. The pull request may
have a dirty state because there is no transaction when merging a pull
request. ref
https://github.com/go-gitea/gitea/pull/25741#issuecomment-2074126393

This PR moves all database update operations to post-receive handler for
merging a pull request and having a database transaction. That means if
database operations fail, then the git merging will fail, the git client
will get a fail result.

There are already many tests for pull request merging, so we don't need
to add a new one.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit ebf0c969403d91ed80745ff5bd7dfbdb08174fc7)

Conflicts:
	modules/private/hook.go
	routers/private/hook_post_receive.go
	trivial conflicts because
	  263a716cb5 * Performance optimization for git push (#30104)
	was not cherry-picked and because of
	  998a431747 Do not update PRs based on events that happened before they existed
2024-05-12 20:03:10 +02:00
silverwind 9792a377e4
Fix tautological conditions (#30735)
As discovered by https://github.com/go-gitea/gitea/pull/30729.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
(cherry picked from commit 610802df85933e7a190a705bc3f7800da87ce868)

Conflicts:
	tests/integration/git_test.go
	trivial conflict because of https://codeberg.org/forgejo/forgejo/pulls/2834
2024-05-05 08:59:40 +01:00
Earl Warren 998a431747
Do not update PRs based on events that happened before they existed
* Split TestPullRequest out of AddTestPullRequestTask
* A Created field is added to the Issue table
* The Created field is set to the time (with nano resolution) on creation
* Record the nano time repo_module.PushUpdateOptions is created by the hook
* The decision to update a pull request created before a commit was
  pushed is based on the time (with nano resolution) the git hook
  was run and the Created field

It ensures the following happens:

* commit C is pushed
* the git hook queues AddTestPullRequestTask for processing and returns with success
* TestPullRequest is not called yet
* a pull request P with commit C as the head is created
* TestPullRequest runs and ignores P because it was created after the commit was received

When the "created" column is NULL, no verification is done, pull
requests that were created before the column was created in the
database cannot be newer than the latest call to a git hook.

Fixes: https://codeberg.org/forgejo/forgejo/issues/2009
2024-04-11 11:16:23 +02:00
Jason Song 18e95af24f
Restore deleted branches when syncing (#29898)
Regression of #29493. If a branch has been deleted, repushing it won't
restore it.

Lunny may have noticed that, but I didn't delve into the comment then
overlooked it:
https://github.com/go-gitea/gitea/pull/29493#discussion_r1509046867

The additional comments added are to explain the issue I found during
testing, which are unrelated to the fixes.

(cherry picked from commit f371f84fa3456c2a71470632b6458d81e4892a54)
2024-03-26 19:04:25 +01:00
Jason Song 375222a145 Sync branches first (gitea#29714)
Follow gitea#29493.

Sync branches to DB first, then trigger push events.
2024-03-18 17:39:34 +01:00
oliverpool 0a53eb838d use gitRepo.GetCommit directly
and give it a commitID instead of a branchName (a bit more correct and faster)
2024-03-18 17:38:58 +01:00
Jason Song 286d09203f Sync branches to DB immediately when handle git hook calling (gitea#29493)
Unlike other async processing in the queue, we should sync branches to
the DB immediately when handling git hook calling. If it fails, users
can see the error message in the output of the git command.

It can avoid potential inconsistency issues, and help #29494.

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2024-03-18 17:38:01 +01:00
Lunny Xiao 894d9b2836
Move context from modules to services (#29440)
Since `modules/context` has to depend on `models` and many other
packages, it should be moved from `modules/context` to
`services/context` according to design principles. There is no logic
code change on this PR, only move packages.

- Move `code.gitea.io/gitea/modules/context` to
`code.gitea.io/gitea/services/context`
- Move `code.gitea.io/gitea/modules/contexttest` to
`code.gitea.io/gitea/services/contexttest` because of depending on
context
- Move `code.gitea.io/gitea/modules/upload` to
`code.gitea.io/gitea/services/context/upload` because of depending on
context

(cherry picked from commit 29f149bd9f517225a3c9f1ca3fb0a7b5325af696)

Conflicts:
	routers/api/packages/alpine/alpine.go
	routers/api/v1/repo/issue_reaction.go
	routers/install/install.go
	routers/web/admin/config.go
	routers/web/passkey.go
	routers/web/repo/search.go
	routers/web/repo/setting/default_branch.go
	routers/web/user/home.go
	routers/web/user/profile.go
	tests/integration/editor_test.go
	tests/integration/integration_test.go
	tests/integration/mirror_push_test.go
	trivial context conflicts
	also modified all other occurrences in Forgejo specific files
2024-03-06 12:10:43 +08:00
yp05327 480c61432a
Fix logic error from #28138 (#29417)
There's a miss in #28138:

![image](https://github.com/go-gitea/gitea/assets/18380374/b1e0c5fc-0e6e-44ab-9f6e-34bc8ffbe1cc)

https://github.com/go-gitea/gitea/pull/28138/files#diff-2556e62ad7204a230c91927a3f2115e25a2b688240d0ee1de6d34f0277f37dfeR162

@lunny
Not sure about the impact of this, but it will only effect 1.22, and
maybe we should fix it ASAP.

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
(cherry picked from commit f8c1efe944c539396402fb014bbfdb67ba199ef2)
2024-03-06 12:10:43 +08:00
Adam Majer cbf923e87b
Abstract hash function usage (#28138)
Refactor Hash interfaces and centralize hash function. This will allow
easier introduction of different hash function later on.

This forms the "no-op" part of the SHA256 enablement patch.
2023-12-13 21:02:00 +00:00
JakobDev ebe803e514
Penultimate round of db.DefaultContext refactor (#27414)
Part of #27065

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-10-11 04:24:07 +00:00
Lunny Xiao f9cfd6ce5b
Use the type RefName for all the needed places and fix pull mirror sync bugs (#24634)
This PR replaces all string refName as a type `git.RefName` to make the
code more maintainable.

Fix #15367
Replaces #23070 
It also fixed a bug that tags are not sync because `git remote --prune
origin` will not remove local tags if remote removed.

We in fact should use `git fetch --prune --tags origin` but not `git
remote update origin` to do the sync.

Some answer from ChatGPT as ref.

> If the git fetch --prune --tags command is not working as expected,
there could be a few reasons why. Here are a few things to check:
> 
>Make sure that you have the latest version of Git installed on your
system. You can check the version by running git --version in your
terminal. If you have an outdated version, try updating Git and see if
that resolves the issue.
> 
>Check that your Git repository is properly configured to track the
remote repository's tags. You can check this by running git config
--get-all remote.origin.fetch and verifying that it includes
+refs/tags/*:refs/tags/*. If it does not, you can add it by running git
config --add remote.origin.fetch "+refs/tags/*:refs/tags/*".
> 
>Verify that the tags you are trying to prune actually exist on the
remote repository. You can do this by running git ls-remote --tags
origin to list all the tags on the remote repository.
> 
>Check if any local tags have been created that match the names of tags
on the remote repository. If so, these local tags may be preventing the
git fetch --prune --tags command from working properly. You can delete
local tags using the git tag -d command.

---------

Co-authored-by: delvh <dev.lh@web.de>
2023-05-26 01:04:48 +00:00
wxiaoguang f4538791f5
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687)
# Why this PR comes

At first, I'd like to help users like #23636 (there are a lot)

The unclear "Internal Server Error" is quite anonying, scare users,
frustrate contributors, nobody knows what happens.

So, it's always good to provide meaningful messages to end users (of
course, do not leak sensitive information).

When I started working on the "response message to end users", I found
that the related code has a lot of technical debt. A lot of copy&paste
code, unclear fields and usages.

So I think it's good to make everything clear.

# Tech Backgrounds

Gitea has many sub-commands, some are used by admins, some are used by
SSH servers or Git Hooks. Many sub-commands use "internal API" to
communicate with Gitea web server.

Before, Gitea server always use `StatusCode + Json "err" field` to
return messages.

* The CLI sub-commands: they expect to show all error related messages
to site admin
* The Serv/Hook sub-commands (for git clients): they could only show
safe messages to end users, the error log could only be recorded by
"SSHLog" to Gitea web server.

In the old design, it assumes that:

* If the StatusCode is 500 (in some functions), then the "err" field is
error log, shouldn't be exposed to git client.
* If the StatusCode is 40x, then the "err" field could be exposed. And
some functions always read the "err" no matter what the StatusCode is.

The old code is not strict, and it's difficult to distinguish the
messages clearly and then output them correctly.

# This PR

To help to remove duplicate code and make everything clear, this PR
introduces `ResponseExtra` and `requestJSONResp`.

* `ResponseExtra` is a struct which contains "extra" information of a
internal API response, including StatusCode, UserMsg, Error
* `requestJSONResp` is a generic function which can be used for all
cases to help to simplify the calls.
* Remove all `map["err"]`, always use `private.Response{Err}` to
construct error messages.
* User messages and error messages are separated clearly, the `fail` and
`handleCliResponseExtra` will output correct messages.
* Replace all `Internal Server Error` messages with meaningful (still
safe) messages.

This PR saves more than 300 lines, while makes the git client messages
more clear.

Many gitea-serv/git-hook related essential functions are covered by
tests.

---------

Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 14:32:26 +08:00
Brecht Van Lommel 3ff5a6a365
Fix missing message in git hook when pull requests disabled on fork (#22625)
And also the other way around, it would show an non-working URL in the
message when pull requests are disabled on the base repository but
enabled on the fork.
2023-01-29 15:00:10 -06:00
Lunny Xiao 0a7d3ff786
refactor some functions to support ctx as first parameter (#21878)
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-12-03 10:48:26 +08:00
flynnnnnnnnnn e81ccc406b
Implement FSFE REUSE for golang files (#21840)
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2022-11-27 18:20:29 +00:00
KN4CK3R 044c754ea5
Add context.Context to more methods (#21546)
This PR adds a context parameter to a bunch of methods. Some helper
`xxxCtx()` methods got replaced with the normal name now.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-19 16:12:33 +08:00
Lunny Xiao 1a9821f57a
Move issues related files into models/issues (#19931)
* Move access and repo permission to models/perm/access

* fix test

* fix git test

* Move functions sequence

* Some improvements per @KN4CK3R and @delvh

* Move issues related code to models/issues

* Move some issues related sub package

* Merge

* Fix test

* Fix test

* Fix test

* Fix test

* Rename some files
2022-06-13 17:37:59 +08:00
Lunny Xiao fd7d83ace6
Move almost all functions' parameter db.Engine to context.Context (#19748)
* Move almost all functions' parameter db.Engine to context.Context
* remove some unnecessary wrap functions
2022-05-20 22:08:52 +08:00
a1012112796 43a22c6a21
show pull link for agit pull request also (#18235) 2022-01-17 13:56:43 +08:00
Lunny Xiao 5723240490
Some repository refactors (#17950)
* some repository refactors

* remove unnecessary code

* Fix test

* Remove unnecessary banner
2021-12-12 23:48:20 +08:00
Lunny Xiao 719bddcd76
Move repository model into models/repo (#17933)
* Some refactors related repository model

* Move more methods out of repository

* Move repository into models/repo

* Fix test

* Fix test

* some improvements

* Remove unnecessary function
2021-12-10 09:27:50 +08:00
Gusted 4e7ca946da
Use *PushUpdateOptions as receiver (#17724) 2021-11-24 04:08:13 -05:00
zeripath 8de44d1995
Clean-up HookPreReceive and restore functionality for pushing non-standard refs (#16705)
* Clean-up HookPreReceive and restore functionality for pushing non-standard refs

There was an inadvertent breaking change in #15629 meaning that notes refs and other
git extension refs will be automatically rejected.

Further following #14295 and #15629 the pre-recieve hook code is untenably long and
too complex.

This PR refactors the hook code and removes the incorrect forced rejection of
non-standard refs.

Fix #16688

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-09-16 15:34:54 +02:00