Commit graph

20816 commits

Author SHA1 Message Date
Gergely Nagy f826f673d1
feat(quota): Add a terse release not about quotas
Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
2024-08-02 11:10:34 +02:00
Gergely Nagy 67fa52dedb
feat(quota): Quota enforcement
The previous commit laid out the foundation of the quota engine, this
one builds on top of it, and implements the actual enforcement.

Enforcement happens at the route decoration level, whenever possible. In
case of the API, when over quota, a 413 error is returned, with an
appropriate JSON payload. In case of web routes, a 413 HTML page is
rendered with similar information.

This implementation is for a **soft quota**: quota usage is checked
before an operation is to be performed, and the operation is *only*
denied if the user is already over quota. This makes it possible to go
over quota, but has the significant advantage of being practically
implementable within the current Forgejo architecture.

The goal of enforcement is to deny actions that can make the user go
over quota, and allow the rest. As such, deleting things should - in
almost all cases - be possible. A prime exemption is deleting files via
the web ui: that creates a new commit, which in turn increases repo
size, thus, is denied if the user is over quota.

Limitations
-----------

Because we generally work at a route decorator level, and rarely
look *into* the operation itself, `size:repos:public` and
`size:repos:private` are not enforced at this level, the engine enforces
against `size:repos:all`. This will be improved in the future.

AGit does not play very well with this system, because AGit PRs count
toward the repo they're opened against, while in the GitHub-style fork +
pull model, it counts against the fork. This too, can be improved in the
future.

There's very little done on the UI side to guard against going over
quota. What this patch implements, is enforcement, not prevention. The
UI will still let you *try* operations that *will* result in a denial.

Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
2024-08-02 11:10:34 +02:00
Gergely Nagy a414703c09
tests: Add an IsTemplate option to DeclarativeRepoOptions
This lets us use `CreateDeclarativeRepoWithOptions` to create template
repositories.

Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
2024-08-02 11:10:34 +02:00
Gergely Nagy e1fe3bbdc0
feat(quota): Humble beginnings of a quota engine
This is an implementation of a quota engine, and the API routes to
manage its settings. This does *not* contain any enforcement code: this
is just the bedrock, the engine itself.

The goal of the engine is to be flexible and future proof: to be nimble
enough to build on it further, without having to rewrite large parts of
it.

It might feel a little more complicated than necessary, because the goal
was to be able to support scenarios only very few Forgejo instances
need, scenarios the vast majority of mostly smaller instances simply do
not care about. The goal is to support both big and small, and for that,
we need a solid, flexible foundation.

There are thee big parts to the engine: counting quota use, setting
limits, and evaluating whether the usage is within the limits. Sounds
simple on paper, less so in practice!

Quota counting
==============

Quota is counted based on repo ownership, whenever possible, because
repo owners are in ultimate control over the resources they use: they
can delete repos, attachments, everything, even if they don't *own*
those themselves. They can clean up, and will always have the permission
and access required to do so. Would we count quota based on the owning
user, that could lead to situations where a user is unable to free up
space, because they uploaded a big attachment to a repo that has been
taken private since. It's both more fair, and much safer to count quota
against repo owners.

This means that if user A uploads an attachment to an issue opened
against organization O, that will count towards the quota of
organization O, rather than user A.

One's quota usage stats can be queried using the `/user/quota` API
endpoint. To figure out what's eating into it, the
`/user/repos?order_by=size`, `/user/quota/attachments`,
`/user/quota/artifacts`, and `/user/quota/packages` endpoints should be
consulted. There's also `/user/quota/check?subject=<...>` to check
whether the signed-in user is within a particular quota limit.

Quotas are counted based on sizes stored in the database.

Setting quota limits
====================

There are different "subjects" one can limit usage for. At this time,
only size-based limits are implemented, which are:

- `size:all`: As the name would imply, the total size of everything
  Forgejo tracks.
- `size:repos:all`: The total size of all repositories (not including
  LFS).
- `size:repos:public`: The total size of all public repositories (not
  including LFS).
- `size:repos:private`: The total size of all private repositories (not
  including LFS).
- `size:git:all`: The total size of all git data (including all
  repositories, and LFS).
- `size:git:lfs`: The size of all git LFS data (either in private or
  public repos).
- `size:assets:all`: The size of all assets tracked by Forgejo.
- `size:assets:attachments:all`: The size of all kinds of attachments
  tracked by Forgejo.
- `size:assets:attachments:issues`: Size of all attachments attached to
  issues, including issue comments.
- `size:assets:attachments:releases`: Size of all attachments attached
  to releases. This does *not* include automatically generated archives.
- `size:assets:artifacts`: Size of all Action artifacts.
- `size:assets:packages:all`: Size of all Packages.
- `size:wiki`: Wiki size

Wiki size is currently not tracked, and the engine will always deem it
within quota.

These subjects are built into Rules, which set a limit on *all* subjects
within a rule. Thus, we can create a rule that says: "1Gb limit on all
release assets, all packages, and git LFS, combined". For a rule to
stand, the total sum of all subjects must be below the rule's limit.

Rules are in turn collected into groups. A group is just a name, and a
list of rules. For a group to stand, all of its rules must stand. Thus,
if we have a group with two rules, one that sets a combined 1Gb limit on
release assets, all packages, and git LFS, and another rule that sets a
256Mb limit on packages, if the user has 512Mb of packages, the group
will not stand, because the second rule deems it over quota. Similarly,
if the user has only 128Mb of packages, but 900Mb of release assets, the
group will not stand, because the combined size of packages and release
assets is over the 1Gb limit of the first rule.

Groups themselves are collected into Group Lists. A group list stands
when *any* of the groups within stand. This allows an administrator to
set conservative defaults, but then place select users into additional
groups that increase some aspect of their limits.

To top it off, it is possible to set the default quota groups a user
belongs to in `app.ini`. If there's no explicit assignment, the engine
will use the default groups. This makes it possible to avoid having to
assign each and every user a list of quota groups, and only those need
to be explicitly assigned who need a different set of groups than the
defaults.

If a user has any quota groups assigned to them, the default list will
not be considered for them.

The management APIs
===================

This commit contains the engine itself, its unit tests, and the quota
management APIs. It does not contain any enforcement.

The APIs are documented in-code, and in the swagger docs, and the
integration tests can serve as an example on how to use them.

Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
2024-08-02 11:10:34 +02:00
Gergely Nagy 250f87db59
feat(api): An order_by param for user.ListMyRepos
Add an optional `order_by` parameter to the `user.ListMyRepos`
handler (which handles the `/api/v1/user/repos` route), allowing a user
to sort repos by name (the default), id, or size.

The latter will be useful later for figuring out which repos use most
space, which repos eat most into a user's quota.

Signed-off-by: Gergely Nagy <forgejo@gergo.csillger.hu>
2024-08-02 10:52:21 +02:00
Gusted b0a104d3d4 Merge pull request 'Distinguish between new tags, releases and pre-releases on activity page' (#4782) from mahlzahn/forgejo:repo_activity_releases into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4782
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-08-02 08:11:39 +00:00
Exploding Dragon 471265c4e0 Add signature support for the RPM module (#4780)
This pull request comes from https://github.com/go-gitea/gitea/pull/27069.

If the rpm package does not contain a matching gpg signature, the installation will fail. See ([gitea/gitea#27031](https://github.com/go-gitea/gitea/issues/27031)) , now auto-signing all new rpm uploads.

This option is turned off by default for compatibility.

<!--start release-notes-assistant-->

## Draft release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
  - [PR](https://codeberg.org/forgejo/forgejo/pulls/4780): <!--number 4780 --><!--line 0 --><!--description QWRkIHNpZ25hdHVyZSBzdXBwb3J0IGZvciB0aGUgUlBNIG1vZHVsZQ==-->Add signature support for the RPM module<!--description-->
<!--end release-notes-assistant-->

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4780
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Exploding Dragon <explodingfkl@gmail.com>
Co-committed-by: Exploding Dragon <explodingfkl@gmail.com>
2024-08-02 05:56:57 +00:00
Robert Wolff 2795f5bc0e feat(UI): fix links, add labels for releases on repo activity page 2024-08-02 07:56:03 +02:00
Earl Warren 35ea74576e Merge pull request 'fix(release-notes-assistant): categorize multiline drafts & cleanup & update milestones' (#4779) from earl-warren/forgejo:wip-rna-preview into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4779
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
2024-08-01 19:33:02 +00:00
Earl Warren 9597e041da
fix(release-notes-assistant): categorize multiline drafts & cleanup
Upgrade to release-notes-assistant 1.1.1:

* multiline release notes drafts were incorrectly categorized
  according the first line, instead of for each line
* when there is a backport, link the original PR first
* remove spurious </a>
2024-08-01 20:56:34 +02:00
Earl Warren 9df2fbb187 Merge pull request 'feat(UI): add links to icons in repository file list' (#4648) from mahlzahn/forgejo:add_icon_links_file_list into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4648
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-08-01 14:34:18 +00:00
Robert Wolff b7f2739dfe feat(UI): add links to icons in repository file list 2024-08-01 13:32:01 +02:00
Codeberg Translate 3d3ddd7704 [I18N] Translations update from Weblate (#4668)
Translations update from [Weblate](https://translate.codeberg.org) for [Forgejo/forgejo](https://translate.codeberg.org/projects/forgejo/forgejo/).

Current translation status:

![Weblate translation status](https://translate.codeberg.org/widget/forgejo/forgejo/horizontal-auto.svg)

Co-authored-by: earl-warren <earl-warren@users.noreply.translate.codeberg.org>
Co-authored-by: Wuzzy <Wuzzy@users.noreply.translate.codeberg.org>
Co-authored-by: Kita Ikuyo <searinminecraft@courvix.com>
Co-authored-by: hankskyjames777 <hankskyjames777@users.noreply.translate.codeberg.org>
Co-authored-by: mahlzahn <mahlzahn@posteo.de>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-authored-by: 0ko <0ko@users.noreply.translate.codeberg.org>
Co-authored-by: lotigara <lotigara@users.noreply.translate.codeberg.org>
Co-authored-by: Fjuro <fjuro@alius.cz>
Co-authored-by: Anonymous <anonymous@users.noreply.translate.codeberg.org>
Co-authored-by: caesar <caesar@users.noreply.translate.codeberg.org>
Co-authored-by: emansije <emansije@users.noreply.translate.codeberg.org>
Co-authored-by: Caesar Schinas <caesar@caesarschinas.com>
Co-authored-by: leana8959 <leana8959@users.noreply.translate.codeberg.org>
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4668
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Codeberg Translate <translate@noreply.codeberg.org>
Co-committed-by: Codeberg Translate <translate@noreply.codeberg.org>
2024-08-01 06:57:25 +00:00
forgejo-renovate-action 2021a351e5 Merge pull request 'Update dependency vue to v3.4.35 (forgejo)' (#4772) from renovate/forgejo-patch-vue-monorepo into forgejo 2024-08-01 00:39:41 +00:00
Renovate Bot c0dde47d37 Update dependency vue to v3.4.35 2024-08-01 00:02:22 +00:00
Earl Warren 6bb541e0d0 Merge pull request '[API] Add error messages to dispatch API' (#4768) from gusted/forgejo-fix-dispatch-api-error into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4768
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-07-31 21:04:01 +00:00
Gusted db78a3abed
[API] Add error messages to dispatch API
- Add a error messages to the dispatch
API (https://code.forgejo.org/api/swagger#/repository/DispatchWorkflow)
when incorrect values are given. Otherwise an incorrect error message is
shown to the user.
- Relevant https://codeberg.org/forgejo/forgejo/issues/4765#issuecomment-2125392
2024-07-31 21:09:17 +02:00
Gusted a05d4c6154 Merge pull request 'feat(UI): add package counter to repo/user/org overview pages' (#4697) from mahlzahn/forgejo:add_packages_counter into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4697
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-07-31 18:52:30 +00:00
0ko ca5a5bf120 feat: allow color and background-color style properties for table cells (#4766)
* Allow adding text color and background color to HTML table headers and cells in markdown.
* Added a few test cases.

Preview and example: https://codeberg.org/attachments/98634f30-4fa2-4a76-adb3-6086af73744f

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4766
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-07-31 18:48:46 +00:00
Earl Warren 5a59866a32 Merge pull request 'fix(action): forgejo-backport-action committer name and email' (#4764) from mahlzahn/forgejo:fix_forgejo_backport_action_comitter_info into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4764
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-07-31 16:54:49 +00:00
Robert Wolff 4d19cc3e38 fix(action): forgejo-backport-action committer name and email 2024-07-31 17:43:36 +02:00
Earl Warren 3e28df8f67 Merge pull request 'fix: use url.JoinPath to join url parts' (#4761) from viceice/tripple-slash-review-url into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4761
Reviewed-by: Panagiotis "Ivory" Vasilopoulos <git@n0toose.net>
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-07-31 14:00:32 +00:00
Michael Kriese 46357e7856
fix: use url.JoinPath to join url parts
This avoids duplicated or more slashes.

fixes #4759
2024-07-31 14:45:37 +02:00
Earl Warren 49eeec269a Merge pull request 'fix(UI): missing rebase command line instructions for rebase ff-only' (#4758) from mahlzahn/forgejo:fix_command_line_instructions_rebase_ff_only into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4758
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-07-31 11:00:54 +00:00
Earl Warren 73cef92d6d Merge pull request 'Update module github.com/meilisearch/meilisearch-go to v0.27.1 (forgejo)' (#4755) from renovate/forgejo-github.com-meilisearch-meilisearch-go-0.x into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4755
Reviewed-by: Shiny Nematoda <snematoda@noreply.codeberg.org>
2024-07-31 10:47:38 +00:00
Robert Wolff 994bd93e69 feat(UI): add package counter to repo/user/org overview pages
- add package counter to repo/user/org overview pages
    - add go unit tests for repo/user has/count packages
    - add many more unit tests for packages model
    - fix error for non-existing packages in DeletePackageByID and SetRepositoryLink
2024-07-31 12:40:24 +02:00
Robert Wolff 622ccd4654 fix(UI): missing rebase command line instructions for rebase ff-only 2024-07-31 11:36:46 +02:00
Earl Warren da97544fa0 Merge pull request '[UI] Fix admin layout' (#4754) from gusted/forgejo-fix-admin into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4754
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-07-31 04:12:44 +00:00
Renovate Bot ccdd5d375b Update module github.com/meilisearch/meilisearch-go to v0.27.1 2024-07-31 00:02:33 +00:00
Gusted 7ec6014a10
[UI] Fix admin layout
- Partially reverts a72b660cbb
- Restores the behavior of #3087
2024-07-30 23:35:22 +02:00
TheFox0x7 4de909747b Add testifylint to lint checks (#4535)
go-require lint is ignored for now

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4535
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: TheFox0x7 <thefox0x7@gmail.com>
Co-committed-by: TheFox0x7 <thefox0x7@gmail.com>
2024-07-30 19:41:10 +00:00
Earl Warren 94933470cd Merge pull request 'Implement external release assets' (#1445) from maltejur/forgejo:forgejo-external-attachments into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/1445
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-07-30 15:50:57 +00:00
Earl Warren ef7313097c Merge pull request 'Refactor repo migration items' (#4710) from 0ko/forgejo:ui-migration-items into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4710
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-07-30 15:48:29 +00:00
Earl Warren d40873e768 Merge pull request '[gitea] week 2024-31 cherry pick (gitea/main -> forgejo)' (#4716) from earl-warren/wcp/2024-31 into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4716
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
2024-07-30 15:45:27 +00:00
Earl Warren a0577572b0 Merge pull request 'Update linters (forgejo)' (#4739) from renovate/forgejo-linters into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4739
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-07-30 15:43:48 +00:00
Earl Warren aa28af85b7 Merge pull request 'docs(release-notes): 8.0.0 & 7.0.6 - updates' (#4750) from earl-warren/forgejo:wip-release-notes into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4750
Reviewed-by: Beowulf <beowulf@noreply.codeberg.org>
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
2024-07-30 14:53:33 +00:00
Earl Warren af557bfe18
docs(release-notes): 8.0.0 & 7.0.6 - updates
- remove duplicate APA line
2024-07-30 16:28:02 +02:00
Earl Warren 1e8c9beee6 Merge pull request 'docs(release-notes): 8.0.0 & 7.0.6 - updates' (#4746) from earl-warren/forgejo:wip-release-notes into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4746
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-07-30 12:11:24 +00:00
Earl Warren 578ac60f5f Merge pull request 'chore(renovate): use mirror image' (#4745) from viceice/renovate into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4745
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-07-30 10:34:10 +00:00
Renovate Bot 69edd8a4fe Update linters 2024-07-30 10:02:20 +00:00
Earl Warren 538bf07c08
docs(release-notes): 8.0.0 & 7.0.6 - updates
- add the release notes for 7.0.6
- move the two removed frontend features first in both 8.0.0 & 7.0.6
- remove extra </a>
2024-07-30 10:14:32 +02:00
Michael Kriese 707318fcc8
chore(renovate): use mirror image 2024-07-30 09:23:44 +02:00
Earl Warren f7bc512123 Merge pull request 'docs(release-notes): 8.0.0 - updates' (#4742) from earl-warren/forgejo:wip-release-notes into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4742
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
2024-07-30 05:55:39 +00:00
Earl Warren 8c2cb172fe
docs(release-notes): 8.0.0 - updates 2024-07-30 06:52:46 +02:00
Earl Warren 7b798a88ee Merge pull request 'fix(UI): issue task list numbers, fix #4431' (#4452) from mahlzahn/forgejo:fix_issue_task_list_numbers_issue_4431 into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4452
Reviewed-by: Beowulf <beowulf@noreply.codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-07-29 20:50:42 +00:00
Earl Warren 2a09f88fd6 Merge pull request 'Lock file maintenance (forgejo)' (#4723) from renovate/forgejo-lock-file-maintenance into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4723
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-07-29 20:27:30 +00:00
Malte Jürgens a61e7c7a39
Implement external assets 2024-07-29 20:35:55 +02:00
Earl Warren 2e234300a2 Merge pull request 'fix: never set to nil: poster of an issue or comment; assignee of a comment' (#4729) from earl-warren/forgejo:wip-ghost-npe into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4729
Reviewed-by: Caesar Schinas <caesar@caesarschinas.com>
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
2024-07-29 14:30:21 +00:00
Gusted be9ee471fd Merge pull request '[UI] Show AGit label on merged PR' (#4715) from gusted/agit into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4715
Reviewed-by: Otto <otto@codeberg.org>
2024-07-29 14:21:50 +00:00
Gusted 0851aca863 Merge pull request '[PORT] Enable no-jquery/no-parse-html-literal and fix violation (gitea#31684)' (#4719) from gusted/forgejo-gt-31684 into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4719
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-07-29 14:18:03 +00:00