Commit graph

123 commits

Author SHA1 Message Date
oliverpool 9a94019db4 webhook: add admin-hooks tests 2024-04-09 20:02:14 +02:00
Earl Warren a015ada29a Merge pull request '[BUG] fix webhook creation payload ref' (#3055) from oliverpool/forgejo:webhook_fix_create_payload_ref into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/3055
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-04-06 07:25:20 +00:00
oliverpool 2c85a1417b [FIX] webhook creation payload ref 2024-04-05 16:21:41 +02:00
oliverpool ed9dd0e62a [FEAT] sourcehut webhooks 2024-04-05 10:20:40 +02:00
oliverpool 04a398a1af [REFACTOR] webhook shared code to prevent import cycles 2024-04-04 08:24:33 +02:00
oliverpool 120fa61a0a [REFACTOR] add Icon to webhook.Interface 2024-03-30 15:59:56 +01:00
Lunny Xiao ef800a7ff1
Fix possible data race on tests (#30093)
(cherry picked from commit 0c8b828f5d5ac7eb8e251edfb1f2536ce1c30336)
2024-03-30 07:17:30 +01:00
Gusted f579bde69d
[CHORE] Cleanup dependency
- Remove `gitea.com/lunny/dingtalk_webhook` as dependency, we only use
two structs which are small enough to be recreated in Forgejo and don't
need to rely on the dependency.
- Existing tests (thanks @oliverpool) prove that this has no effect.
2024-03-30 00:01:42 +01:00
oliverpool 4ab341e971 [REFACTOR] webhook gogs endpoint 2024-03-27 15:38:39 +01:00
oliverpool dce754cde1 [REFACTOR] webhook packagist endpoint 2024-03-27 15:38:39 +01:00
oliverpool 36a1d37532 [REFACTOR] webhook wechatwork endpoint 2024-03-27 15:38:39 +01:00
oliverpool 6b719f08d0 [REFACTOR] webhook feishu endpoint 2024-03-27 15:38:39 +01:00
oliverpool 97f0ad49ff [REFACTOR] webhook msteams endpoint 2024-03-27 15:38:39 +01:00
oliverpool 46b71ec709 [REFACTOR] webhook telegram endpoint 2024-03-27 15:38:39 +01:00
oliverpool 9dff719523 [REFACTOR] webhook dingtalk endpoint 2024-03-27 15:38:39 +01:00
oliverpool c3f8e6ed60 [REFACTOR] webhook discord endpoint 2024-03-27 15:38:39 +01:00
oliverpool 6f00821f3d [REFACTOR] webhook slack endpoints 2024-03-27 15:38:38 +01:00
oliverpool 7e0965b02c [REFACTOR] webhook forgejo/gitea endpoints 2024-03-27 15:38:38 +01:00
oliverpool 8dfbbfef07 [REFACTOR] webhook matrix endpoints 2024-03-27 15:38:38 +01:00
Earl Warren e41e18f87e Merge pull request '[REFACTOR] webhook.Handler interface' (#2758) from oliverpool/forgejo:webhook_2_interface into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/2758
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-03-27 13:45:04 +00:00
Denys Konovalov 20f839ad00
fix telegram webhook (#29864)
Fix #29837 which is a regression caused by

https://github.com/go-gitea/gitea/pull/29145/files#diff-731445ee00f0f1bf2ff731f4f96ddcf51cdc53fd2faaf406eb3536fc292ea748L48.

The line was probably removed by accident.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
(cherry picked from commit 0285b04f4ca981d7467097dbca3b281011b7798c)
2024-03-26 19:04:25 +01:00
oliverpool 702152bfde [REFACTOR] webhook.Handler interface 2024-03-24 10:35:06 +01:00
oliverpool 2536c2db89 fix webhook fixtures 2024-03-24 07:11:19 +01:00
oliverpool a47a1e0777 [BUG] Packagist webhook: support all events
Fixes #2329
2024-03-13 09:27:12 +01:00
oliverpool ed6f724a5d
add test for webhook default method
https://github.com/go-gitea/gitea/pull/29690
2024-03-11 23:37:00 +07:00
Lunny Xiao 1b8aa3ab6f
Fix broken webhooks (#29690)
Fix #29689

(cherry picked from commit 6e8762f962c5eaaee1c92e910c95c8b85b7c1e11)
2024-03-11 23:37:00 +07:00
oliverpool 53f6f62ad4
Store webhook event in database (#29145)
Refactor the webhook logic, to have the type-dependent processing happen
only in one place.

---

1. An event happens
2. It is pre-processed (depending on the webhook type) and its body is
added to a task queue
3. When the task is processed, some more logic (depending on the webhook
type as well) is applied to make an HTTP request

This means that webhook-type dependant logic is needed in step 2 and 3.
This is cumbersome and brittle to maintain.

Updated webhook flow with this PR:
1. An event happens
2. It is stored as-is and added to a task queue
3. When the task is processed, the event is processed (depending on the
webhook type) to make an HTTP request

So the only webhook-type dependent logic happens in one place (step 3)
which should be much more robust.

- the raw event must be stored in the hooktask (until now, the
pre-processed body was stored)
- to ensure that previous hooktasks are correctly sent, a
`payload_version` is added (version 1: the body has already been
pre-process / version 2: the body is the raw event)

So future webhook additions will only have to deal with creating an
http.Request based on the raw event (no need to adjust the code in
multiple places, like currently).

Moreover since this processing happens when fetching from the task
queue, it ensures that the queuing of new events (upon a `git push` for
instance) does not get slowed down by a slow webhook.

As a concrete example, the PR #19307 for custom webhooks, should be
substantially smaller:
- no need to change `services/webhook/deliver.go`
- minimal change in `services/webhook/webhook.go` (add the new webhook
to the map)
- no need to change all the individual webhook files (since with this
refactor the `*webhook_model.Webhook` is provided as argument)

(cherry picked from commit 26653b196bd1d15c532af41f60351596dd4330bd)

Conflicts:
	services/webhook/deliver_test.go
	trivial context conflict
2024-03-11 23:36:59 +07:00
6543 e2371743d5
remove util.OptionalBool and related functions (#29513)
and migrate affected code

_last refactoring bits to replace **util.OptionalBool** with
**optional.Option[bool]**_

(cherry picked from commit a3f05d0d98408bb47333b19f505b21afcefa9e7c)

Conflicts:
	services/repository/branch.go
	trivial context conflict
2024-03-06 12:10:46 +08:00
Earl Warren 094c84ed6d
Merge branch 'rebase-forgejo-dependency' into wip-forgejo 2024-02-05 18:58:23 +01:00
Gusted 2baec139fa
[GITEA] Fix test TestWebhookProxy with http proxy env
- Unset the http proxies environments for the `TestWebhookProxy`.
- Resolves #2132

(cherry picked from commit 244b9786fc)
(cherry picked from commit 8602dfa6a2)
(cherry picked from commit 8621449209)
(cherry picked from commit aefa77f917)
2024-02-05 16:09:43 +01:00
Gusted fa37a211fb
[GITEA] Drop sha256-simd in favor of stdlib
- In Go 1.21 the crypto/sha256 [got a massive
improvement](https://go.dev/doc/go1.21#crypto/sha256) by utilizing the
SHA instructions for AMD64 CPUs, which sha256-simd already was doing.
The performance is now on par and I think it's preferable to use the
standard library rather than a package when possible.

```
cpu: AMD Ryzen 5 3600X 6-Core Processor
                │  simd.txt   │               go.txt                │
                │   sec/op    │    sec/op     vs base               │
Hash/8Bytes-12    63.25n ± 1%    73.38n ± 1%  +16.02% (p=0.002 n=6)
Hash/64Bytes-12   98.73n ± 1%   105.30n ± 1%   +6.65% (p=0.002 n=6)
Hash/1K-12        567.2n ± 1%    572.8n ± 1%   +0.99% (p=0.002 n=6)
Hash/8K-12        4.062µ ± 1%    4.062µ ± 1%        ~ (p=0.396 n=6)
Hash/1M-12        512.1µ ± 0%    510.6µ ± 1%        ~ (p=0.485 n=6)
Hash/5M-12        2.556m ± 1%    2.564m ± 0%        ~ (p=0.093 n=6)
Hash/10M-12       5.112m ± 0%    5.127m ± 0%        ~ (p=0.093 n=6)
geomean           13.82µ         14.27µ        +3.28%

                │   simd.txt   │               go.txt                │
                │     B/s      │     B/s       vs base               │
Hash/8Bytes-12    120.6Mi ± 1%   104.0Mi ± 1%  -13.81% (p=0.002 n=6)
Hash/64Bytes-12   618.2Mi ± 1%   579.8Mi ± 1%   -6.22% (p=0.002 n=6)
Hash/1K-12        1.682Gi ± 1%   1.665Gi ± 1%   -0.98% (p=0.002 n=6)
Hash/8K-12        1.878Gi ± 1%   1.878Gi ± 1%        ~ (p=0.310 n=6)
Hash/1M-12        1.907Gi ± 0%   1.913Gi ± 1%        ~ (p=0.485 n=6)
Hash/5M-12        1.911Gi ± 1%   1.904Gi ± 0%        ~ (p=0.093 n=6)
Hash/10M-12       1.910Gi ± 0%   1.905Gi ± 0%        ~ (p=0.093 n=6)
geomean           1.066Gi        1.032Gi        -3.18%
```

(cherry picked from commit abd94ff5b5)
(cherry picked from commit 15e81637ab)

Conflicts:
	go.mod
	https://codeberg.org/forgejo/forgejo/pulls/1581
(cherry picked from commit 325d92917f)

Conflicts:
	modules/context/context_cookie.go
	https://codeberg.org/forgejo/forgejo/pulls/1617
(cherry picked from commit 358819e895)
(cherry picked from commit 362fd7aae1)
(cherry picked from commit 4f64ee294e)
(cherry picked from commit 4bde77f7b1)
(cherry picked from commit 1311e30a81)
(cherry picked from commit 57b69e334c)
(cherry picked from commit 52dc892fad)
(cherry picked from commit 77f54f4187)
(cherry picked from commit 0d0392f3a5)

Conflicts:
	go.mod
	https://codeberg.org/forgejo/forgejo/pulls/2034
(cherry picked from commit 92798364e8)
(cherry picked from commit 43d2181277)
(cherry picked from commit 45c88b86a3)
(cherry picked from commit a1cd6f4e3a)
(cherry picked from commit 01191dc2ad)
(cherry picked from commit 151e07f37e)
2024-02-05 16:09:40 +01:00
Earl Warren e20cd83bc5
[BRANDING] add the forgejo webhook type & update webhook docs URLs
templates/swagger/v1_json.tmpl updated with `make generate-swagger`

(cherry picked from commit 88899c492e)
(cherry picked from commit 7171bd9617)
(cherry picked from commit 1a742446c1)
(cherry picked from commit d7c189d7b2)

Conflicts:
	routers/web/web.go
(cherry picked from commit cbdea868e4)
(cherry picked from commit 6cd150483b)
(cherry picked from commit 47246da8d3)
(cherry picked from commit f2aa0e6b76)
(cherry picked from commit 5a4fc69a16)
(cherry picked from commit 48e444ca09)
(cherry picked from commit 888e537811)
(cherry picked from commit 5121f493c9)
(cherry picked from commit 9394e55fdf)
(cherry picked from commit 3a2ce51768)
(cherry picked from commit 719ead3a65)
(cherry picked from commit 83e6f82e2a)
(cherry picked from commit 494a429b21)
(cherry picked from commit 4d775db6b4)
(cherry picked from commit b68f777dc2)
(cherry picked from commit 5b934023fa)
(cherry picked from commit 3b1ed8b16c)
(cherry picked from commit 6bc4a46c9f)
(cherry picked from commit 8064bb24a3)

Conflicts:
	templates/admin/hook_new.tmpl
	templates/org/settings/hook_new.tmpl
	templates/repo/settings/webhook/base_list.tmpl
	templates/repo/settings/webhook/new.tmpl
	templates/user/settings/hook_new.tmpl
	https://codeberg.org/forgejo/forgejo/pulls/1181
(cherry picked from commit 55f5588a91)
(cherry picked from commit a428bc06b3)
(cherry picked from commit d2186eceb9)
(cherry picked from commit b4e126e9af)
(cherry picked from commit e4c7a92c2d)
(cherry picked from commit a7165c8146)

Conflicts:
	templates/admin/hook_new.tmpl
	templates/repo/settings/webhook/new.tmpl
	https://codeberg.org/forgejo/forgejo/pulls/1420

[BRANDING] add the forgejo webhook type & update webhook docs URLs (squash) gitea logo for gitea webhooks

Refs: https://codeberg.org/forgejo/forgejo/issues/1367
(cherry picked from commit 2d8c1b9373)
(cherry picked from commit 9135a5e1db)
(cherry picked from commit 270f4020b0)
(cherry picked from commit 33e5e2f0d8)
(cherry picked from commit 7af8bcf479)

Conflicts:
	tests/integration/links_test.go
	https://codeberg.org/forgejo/forgejo/pulls/1548
(cherry picked from commit ce2a434dcf)
(cherry picked from commit c2fa42b4fd)
(cherry picked from commit 3858c8d4db)
(cherry picked from commit ebb0ae4920)
(cherry picked from commit 93d354c418)
(cherry picked from commit d5b08a2a4b)
(cherry picked from commit e5e5caffaa)
(cherry picked from commit 974d405700)
(cherry picked from commit e654fbccb1)
(cherry picked from commit 85515977fb)

[BRANDING] add the forgejo webhook type & update webhook docs URLs (squash) ctx.Locale

(cherry picked from commit e192647bc0)
(cherry picked from commit 95fb1b7242)
(cherry picked from commit ea7bd8022f)
(cherry picked from commit bf8465ccd0)
(cherry picked from commit e1baa0a300)
(cherry picked from commit 01b4f4c514)
(cherry picked from commit da330b1e51)
(cherry picked from commit 7b26c27c20)
(cherry picked from commit 0c781e6f6d)
(cherry picked from commit 0f663a8414)
(cherry picked from commit 3297fd4f15)
(cherry picked from commit 2f4ef403cb)
2024-02-05 16:02:14 +01:00
Earl Warren ec8f162b33
[BRANDING] add X-Forgejo-* headers
(cherry picked from commit 0a3388f93f)
(cherry picked from commit 7eba0a440a)
(cherry picked from commit eb9646c7ef)
(cherry picked from commit f1972578f5)

Conflicts:
(cherry picked from commit 7f96222fb4)
(cherry picked from commit e3c7c9fe7b)
(cherry picked from commit 84fdead902)
(cherry picked from commit 85148e1196)
(cherry picked from commit c0086bd70d)
(cherry picked from commit d1e31ef318)
(cherry picked from commit 681d3ed5c4)
(cherry picked from commit 76a3001f5b)
(cherry picked from commit a55a9567d3)
(cherry picked from commit aa7adc167d)
(cherry picked from commit d5354cb52c)
(cherry picked from commit 472c489996)
(cherry picked from commit dc816d065b)
(cherry picked from commit 4795f9ea85)
(cherry picked from commit ddd4ae5343)
(cherry picked from commit 0e95f2a36b)
(cherry picked from commit 47a41a4e7b)
(cherry picked from commit d4416fcd3e)
(cherry picked from commit f279d153b6)
(cherry picked from commit 959f908ffd)
(cherry picked from commit 82df953101)
(cherry picked from commit 8f8c9fd9e3)
(cherry picked from commit d4a0db7706)
(cherry picked from commit 44594d6239)
(cherry picked from commit 62b1de579e)
(cherry picked from commit 507abee353)
(cherry picked from commit 8c36ac42c7)
(cherry picked from commit 72f74067f4)
(cherry picked from commit 8e5a9e187b)
(cherry picked from commit cff8f43c7e)
(cherry picked from commit 493cc93e6d)
(cherry picked from commit c94352b4d4)
(cherry picked from commit 3f622c51a7)
(cherry picked from commit 84c62ad447)
(cherry picked from commit ba186c8ee4)
(cherry picked from commit 4238ef291d)
(cherry picked from commit 3ef1bd61b9)
(cherry picked from commit f304086bb6)
(cherry picked from commit 64a2edabcb)
(cherry picked from commit 6accea6ba7)
(cherry picked from commit 2296e93a42)
(cherry picked from commit 2bf0be3b1b)
(cherry picked from commit fb4652b197)
(cherry picked from commit 7d462cf353)
(cherry picked from commit f5319660dd)
2024-02-05 16:02:14 +01:00
Adam Majer cbf923e87b
Abstract hash function usage (#28138)
Refactor Hash interfaces and centralize hash function. This will allow
easier introduction of different hash function later on.

This forms the "no-op" part of the SHA256 enablement patch.
2023-12-13 21:02:00 +00:00
Lunny Xiao df1e7d0067
Use db.Find instead of writing methods for every object (#28084)
For those simple objects, it's unnecessary to write the find and count
methods again and again.
2023-11-24 03:49:41 +00:00
KN4CK3R 58f5fa6536
Fix Matrix and MSTeams nil dereference (#28089)
Fixes #28088 
Fixes #28094

Added missing tests.

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-11-17 12:17:33 +01:00
Lunny Xiao 16d15ce087
Fix package webhook (#27839)
Fix #23742

---------

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-10-31 04:43:38 +00:00
Jason Song 4e98224a45
Support allowed hosts for webhook to work with proxy (#27655)
When `webhook.PROXY_URL` has been set, the old code will check if the
proxy host is in `ALLOWED_HOST_LIST` or reject requests through the
proxy. It requires users to add the proxy host to `ALLOWED_HOST_LIST`.
However, it actually allows all requests to any port on the host, when
the proxy host is probably an internal address.

But things may be even worse. `ALLOWED_HOST_LIST` doesn't really work
when requests are sent to the allowed proxy, and the proxy could forward
them to any hosts.

This PR fixes it by:

- If the proxy has been set, always allow connectioins to the host and
port.
- Check `ALLOWED_HOST_LIST` before forwarding.
2023-10-18 09:44:36 +00:00
JakobDev 76a85a4ce9
Final round of db.DefaultContext refactor (#27587)
Last part of #27065
2023-10-14 08:37:24 +00:00
Lunny Xiao 673cf6af76
make writing main test easier (#27270)
This PR removed `unittest.MainTest` the second parameter
`TestOptions.GiteaRoot`. Now it detects the root directory by current
working directory.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-09-28 01:38:53 +00:00
John Olheiser 7520cd678c
Fix release URL in webhooks (#27182) 2023-09-21 17:55:09 -05:00
Lunny Xiao 4f32abaf94
move repository deletion to service layer (#26948)
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-09-08 04:51:15 +00:00
Lunny Xiao 540bf9fa6d
Move notification interface to services layer (#26915)
Extract from #22266
2023-09-05 18:37:47 +00:00
谈笑风生间 5104c887d3
Modify the content format of the Feishu webhook (#25106)
close https://github.com/go-gitea/gitea/issues/24368

## what my pull request does

Since the official documentation states that custom bots do not support
hyperlink functionality, simply adding it without making some formatting
changes would result in an unappealing output. Therefore, I have
modified the formatting of the output. Currently, it is only used for
Feishu.

--- 


[docs](https://open.feishu.cn/document/uAjLw4CM/ukTMukTMukTM/im-v1/message/create_json#%E8%B6%85%E9%93%BE%E6%8E%A5%E4%BD%BF%E7%94%A8%E8%AF%B4%E6%98%8E)

<img width="641" alt="image"
src="https://github.com/go-gitea/gitea/assets/75628309/360e1c81-ab64-4ef6-851e-aa450d6e85a4">

- Issue

<img width="423" alt="image"
src="https://github.com/go-gitea/gitea/assets/75628309/519f6fae-81ab-4ec8-89b8-f6a73ff93783">


- Issue Comment


<img width="548" alt="image"
src="https://github.com/go-gitea/gitea/assets/75628309/749c0d1c-3657-431e-b787-8bf4c23cce83">


- Assign

<img width="431" alt="image"
src="https://github.com/go-gitea/gitea/assets/75628309/066f99e5-eabb-455d-91fb-a8359cc26dc7">

<img width="457" alt="image"
src="https://github.com/go-gitea/gitea/assets/75628309/c6c10f99-db83-46ef-a775-4c91979fa68f">

- Merge

<img width="408" alt="image"
src="https://github.com/go-gitea/gitea/assets/75628309/e627bf43-5954-45aa-acf6-261ee046802f">

- PullRequest

<img width="425" alt="image"
src="https://github.com/go-gitea/gitea/assets/75628309/72cfa714-d3fa-4fb9-abdd-e8508d756056">
2023-08-24 09:00:11 +08:00
Yarden Shoham 16dee4f1b2
Add branch_filter to hooks API endpoints (#26599)
We now include the branch filler in the response.

- Closes #26591 

# Before

![image](https://github.com/go-gitea/gitea/assets/20454870/73933940-c1a7-4573-abae-f340b63028b2)

# After

![image](https://github.com/go-gitea/gitea/assets/20454870/3b3c4a85-0f7c-48c7-8617-def7a66c671d)

---------

Signed-off-by: Yarden Shoham <git@yardenshoham.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-08-21 11:43:52 +00:00
Earl Warren 50fc22eecf
Add ThreadID parameter for Telegram webhooks (#25996)
Telegram has recently implemented threads (channels) for group chats.

Co-authored-by: neveraskedtoexist <matikot415@gmail.com>
2023-08-13 14:00:06 +00:00
Yarden Shoham 220f236ff5
Add pull request review request webhook event (#26401)
Add webhook events for pull request review requests

- Fixes #26371
- Added support for the "Pull request review requested" and "Pull
request review request removed" webhook events.
- Updated the `getPullRequestPayloadInfo` function in `general.go` to
handle these new webhook events.

# Before

![image](https://github.com/go-gitea/gitea/assets/20454870/bd942971-fb1d-40f3-8961-46638e3588fa)

# After

![image](https://github.com/go-gitea/gitea/assets/20454870/216e9c7d-0a4d-49f9-8492-2d14c88bbf4e)

Signed-off-by: Yarden Shoham <git@yardenshoham.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-08-09 10:30:34 +02:00
puni9869 4971a10543
Warn instead of reporting an error when a webhook cannot be found (#26039)
Attemp fix: #25744
Fixing the log level when we delete any repo then we get error hook not
found by id. That should be warn level to reduce the noise in the logs.

---------

Co-authored-by: delvh <dev.lh@web.de>
2023-07-28 17:46:48 +00:00
Lunny Xiao 0fd1672ae4
For API attachments, use API URL (#25639)
Fix #25257

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-07-10 09:31:19 +00:00
Jason Song 174213530d
Fix Permission in API returned repository struct (#25388)
The old code generates `structs.Repository.Permissions` with only
`access.Permission.AccessMode`, however, it should check the units too,
or the value could be incorrect. For example,
`structs.Repository.Permissions.Push` could be false even the doer has
write access to code unit.

Should fix
https://github.com/renovatebot/renovate/issues/14059#issuecomment-1047961128
(Not reported by it, I just found it when I was looking into this bug)

---

Review tips:

The major changes are
- `modules/structs/repo.go`
https://github.com/go-gitea/gitea/pull/25388/files#diff-870406f6857117f8b03611c43fca0ab9ed6d6e76a2d0069a7c1f17e8fa9092f7
- `services/convert/repository.go`
https://github.com/go-gitea/gitea/pull/25388/files#diff-7736f6d2ae894c9edb7729a80ab89aa183b888a26a811a0c1fdebd18726a7101

And other changes are passive.
2023-06-22 13:08:08 +00:00