Support for grouping RPMs using paths (#26984)

The current rpm repository places all packages in the same repository,
and different systems (el7,f34) may hit packages that do not belong to
this distribution ( #25304 ) , which now supports grouping of rpm.

![图片](https://github.com/go-gitea/gitea/assets/33776693/d1e1d99f-7799-4b2b-a19b-cb2a5c692914)

Fixes #25304 .
Fixes #27056 .

Refactor: [#25866](https://github.com/go-gitea/gitea/pull/25866)
This commit is contained in:
Exploding Dragon 2024-01-12 11:16:05 +08:00 committed by GitHub
parent 7c2f093e85
commit ba4d0b8ffb
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
9 changed files with 192 additions and 101 deletions

View file

@ -27,17 +27,18 @@ The following examples use `dnf`.
To register the RPM registry add the url to the list of known apt sources: To register the RPM registry add the url to the list of known apt sources:
```shell ```shell
dnf config-manager --add-repo https://gitea.example.com/api/packages/{owner}/rpm.repo dnf config-manager --add-repo https://gitea.example.com/api/packages/{owner}/rpm/{group}.repo
``` ```
| Placeholder | Description | | Placeholder | Description |
| ----------- | ----------- | | ----------- |----------------------------------------------------|
| `owner` | The owner of the package. | | `owner` | The owner of the package. |
| `group` | Everything, e.g. `el7`, `rocky/el9` , `test/fc38`.|
If the registry is private, provide credentials in the url. You can use a password or a [personal access token](development/api-usage.md#authentication): If the registry is private, provide credentials in the url. You can use a password or a [personal access token](development/api-usage.md#authentication):
```shell ```shell
dnf config-manager --add-repo https://{username}:{your_password_or_token}@gitea.example.com/api/packages/{owner}/rpm.repo dnf config-manager --add-repo https://{username}:{your_password_or_token}@gitea.example.com/api/packages/{owner}/rpm/{group}.repo
``` ```
You have to add the credentials to the urls in the `rpm.repo` file in `/etc/yum.repos.d` too. You have to add the credentials to the urls in the `rpm.repo` file in `/etc/yum.repos.d` too.
@ -47,19 +48,20 @@ You have to add the credentials to the urls in the `rpm.repo` file in `/etc/yum.
To publish a RPM package (`*.rpm`), perform a HTTP PUT operation with the package content in the request body. To publish a RPM package (`*.rpm`), perform a HTTP PUT operation with the package content in the request body.
``` ```
PUT https://gitea.example.com/api/packages/{owner}/rpm/upload PUT https://gitea.example.com/api/packages/{owner}/rpm/{group}/upload
``` ```
| Parameter | Description | | Parameter | Description |
| --------- | ----------- | | --------- | ----------- |
| `owner` | The owner of the package. | | `owner` | The owner of the package. |
| `group` | Everything, e.g. `el7`, `rocky/el9` , `test/fc38`.|
Example request using HTTP Basic authentication: Example request using HTTP Basic authentication:
```shell ```shell
curl --user your_username:your_password_or_token \ curl --user your_username:your_password_or_token \
--upload-file path/to/file.rpm \ --upload-file path/to/file.rpm \
https://gitea.example.com/api/packages/testuser/rpm/upload https://gitea.example.com/api/packages/testuser/rpm/centos/el7/upload
``` ```
If you are using 2FA or OAuth use a [personal access token](development/api-usage.md#authentication) instead of the password. If you are using 2FA or OAuth use a [personal access token](development/api-usage.md#authentication) instead of the password.
@ -78,21 +80,22 @@ The server responds with the following HTTP Status codes.
To delete an RPM package perform a HTTP DELETE operation. This will delete the package version too if there is no file left. To delete an RPM package perform a HTTP DELETE operation. This will delete the package version too if there is no file left.
``` ```
DELETE https://gitea.example.com/api/packages/{owner}/rpm/{package_name}/{package_version}/{architecture} DELETE https://gitea.example.com/api/packages/{owner}/rpm/{group}/package/{package_name}/{package_version}/{architecture}
``` ```
| Parameter | Description | | Parameter | Description |
| ----------------- | ----------- | |-------------------|----------------------------|
| `owner` | The owner of the package. | | `owner` | The owner of the package. |
| `package_name` | The package name. | | `group` | The package group . |
| `package_version` | The package version. | | `package_name` | The package name. |
| `architecture` | The package architecture. | | `package_version` | The package version. |
| `architecture` | The package architecture. |
Example request using HTTP Basic authentication: Example request using HTTP Basic authentication:
```shell ```shell
curl --user your_username:your_token_or_password -X DELETE \ curl --user your_username:your_token_or_password -X DELETE \
https://gitea.example.com/api/packages/testuser/rpm/test-package/1.0.0/x86_64 https://gitea.example.com/api/packages/testuser/rpm/centos/el7/package/test-package/1.0.0/x86_64
``` ```
The server responds with the following HTTP Status codes. The server responds with the following HTTP Status codes.

View file

@ -27,17 +27,18 @@ menu:
要注册RPM注册表请将 URL 添加到已知 `apt` 源列表中: 要注册RPM注册表请将 URL 添加到已知 `apt` 源列表中:
```shell ```shell
dnf config-manager --add-repo https://gitea.example.com/api/packages/{owner}/rpm.repo dnf config-manager --add-repo https://gitea.example.com/api/packages/{owner}/rpm/{group}.repo
``` ```
| 占位符 | 描述 | | 占位符 | 描述 |
| ------- | -------------- | | ------- |--------------------------------------|
| `owner` | 软件包的所有者 | | `owner` | 软件包的所有者 |
| `group` | 任何名称,例如 `centos/7`、`el-7`、`fc38` |
如果注册表是私有的请在URL中提供凭据。您可以使用密码或[个人访问令牌](development/api-usage.md#通过-api-认证) 如果注册表是私有的请在URL中提供凭据。您可以使用密码或[个人访问令牌](development/api-usage.md#通过-api-认证)
```shell ```shell
dnf config-manager --add-repo https://{username}:{your_password_or_token}@gitea.example.com/api/packages/{owner}/rpm.repo dnf config-manager --add-repo https://{username}:{your_password_or_token}@gitea.example.com/api/packages/{owner}/rpm/{group}.repo
``` ```
您还必须将凭据添加到 `/etc/yum.repos.d` 中的 `rpm.repo` 文件中的URL中。 您还必须将凭据添加到 `/etc/yum.repos.d` 中的 `rpm.repo` 文件中的URL中。
@ -47,19 +48,20 @@ dnf config-manager --add-repo https://{username}:{your_password_or_token}@gitea.
要发布RPM软件包`*.rpm`),请执行带有软件包内容的 HTTP `PUT` 操作。 要发布RPM软件包`*.rpm`),请执行带有软件包内容的 HTTP `PUT` 操作。
``` ```
PUT https://gitea.example.com/api/packages/{owner}/rpm/upload PUT https://gitea.example.com/api/packages/{owner}/rpm/{group}/upload
``` ```
| 参数 | 描述 | | 参数 | 描述 |
| ------- | -------------- | | ------- |--------------|
| `owner` | 软件包的所有者 | | `owner` | 软件包的所有者 |
| `group` | 软件包自定义分组名称 |
使用HTTP基本身份验证的示例请求 使用HTTP基本身份验证的示例请求
```shell ```shell
curl --user your_username:your_password_or_token \ curl --user your_username:your_password_or_token \
--upload-file path/to/file.rpm \ --upload-file path/to/file.rpm \
https://gitea.example.com/api/packages/testuser/rpm/upload https://gitea.example.com/api/packages/testuser/rpm/centos/el7/version/upload
``` ```
如果您使用 2FA 或 OAuth请使用[个人访问令牌](development/api-usage.md#通过-api-认证)替代密码。您无法将具有相同名称的文件两次发布到软件包中。您必须先删除现有的软件包版本。 如果您使用 2FA 或 OAuth请使用[个人访问令牌](development/api-usage.md#通过-api-认证)替代密码。您无法将具有相同名称的文件两次发布到软件包中。您必须先删除现有的软件包版本。
@ -77,12 +79,13 @@ curl --user your_username:your_password_or_token \
要删除 RPM 软件包,请执行 HTTP `DELETE` 操作。如果没有文件剩余,这也将删除软件包版本。 要删除 RPM 软件包,请执行 HTTP `DELETE` 操作。如果没有文件剩余,这也将删除软件包版本。
``` ```
DELETE https://gitea.example.com/api/packages/{owner}/rpm/{package_name}/{package_version}/{architecture} DELETE https://gitea.example.com/api/packages/{owner}/rpm/{group}/package/{package_name}/{package_version}/{architecture}
``` ```
| 参数 | 描述 | | 参数 | 描述 |
| ----------------- | -------------- | | ----------------- | -------------- |
| `owner` | 软件包的所有者 | | `owner` | 软件包的所有者 |
| `group` | 软件包自定义分组 |
| `package_name` | 软件包名称 | | `package_name` | 软件包名称 |
| `package_version` | 软件包版本 | | `package_version` | 软件包版本 |
| `architecture` | 软件包架构 | | `architecture` | 软件包架构 |
@ -91,7 +94,7 @@ DELETE https://gitea.example.com/api/packages/{owner}/rpm/{package_name}/{packag
```shell ```shell
curl --user your_username:your_token_or_password -X DELETE \ curl --user your_username:your_token_or_password -X DELETE \
https://gitea.example.com/api/packages/testuser/rpm/test-package/1.0.0/x86_64 https://gitea.example.com/api/packages/testuser/rpm/centos/el7/package/test-package/1.0.0/x86_64
``` ```
服务器将以以下HTTP状态码响应 服务器将以以下HTTP状态码响应

View file

@ -15,8 +15,7 @@ import (
) )
const ( const (
PropertyMetadata = "rpm.metadata" PropertyMetadata = "rpm.metadata"
SettingKeyPrivate = "rpm.key.private" SettingKeyPrivate = "rpm.key.private"
SettingKeyPublic = "rpm.key.public" SettingKeyPublic = "rpm.key.public"

View file

@ -4,6 +4,7 @@
package templates package templates
import ( import (
"regexp"
"strings" "strings"
"code.gitea.io/gitea/modules/base" "code.gitea.io/gitea/modules/base"
@ -25,6 +26,10 @@ func (su *StringUtils) Contains(s, substr string) bool {
return strings.Contains(s, substr) return strings.Contains(s, substr)
} }
func (su *StringUtils) ReplaceAllStringRegex(s, regex, new string) string {
return regexp.MustCompile(regex).ReplaceAllString(s, new)
}
func (su *StringUtils) Split(s, sep string) []string { func (su *StringUtils) Split(s, sep string) []string {
return strings.Split(s, sep) return strings.Split(s, sep)
} }

View file

@ -512,19 +512,7 @@ func CommonRoutes() *web.Route {
r.Get("/files/{id}/{version}/{filename}", pypi.DownloadPackageFile) r.Get("/files/{id}/{version}/{filename}", pypi.DownloadPackageFile)
r.Get("/simple/{id}", pypi.PackageMetadata) r.Get("/simple/{id}", pypi.PackageMetadata)
}, reqPackageAccess(perm.AccessModeRead)) }, reqPackageAccess(perm.AccessModeRead))
r.Group("/rpm", func() { r.Group("/rpm", RpmRoutes(r), reqPackageAccess(perm.AccessModeRead))
r.Get(".repo", rpm.GetRepositoryConfig)
r.Get("/repository.key", rpm.GetRepositoryKey)
r.Put("/upload", reqPackageAccess(perm.AccessModeWrite), rpm.UploadPackageFile)
r.Group("/package/{name}/{version}/{architecture}", func() {
r.Get("", rpm.DownloadPackageFile)
r.Delete("", reqPackageAccess(perm.AccessModeWrite), rpm.DeletePackageFile)
})
r.Group("/repodata/{filename}", func() {
r.Head("", rpm.CheckRepositoryFileExistence)
r.Get("", rpm.GetRepositoryFile)
})
}, reqPackageAccess(perm.AccessModeRead))
r.Group("/rubygems", func() { r.Group("/rubygems", func() {
r.Get("/specs.4.8.gz", rubygems.EnumeratePackages) r.Get("/specs.4.8.gz", rubygems.EnumeratePackages)
r.Get("/latest_specs.4.8.gz", rubygems.EnumeratePackagesLatest) r.Get("/latest_specs.4.8.gz", rubygems.EnumeratePackagesLatest)
@ -589,6 +577,82 @@ func CommonRoutes() *web.Route {
return r return r
} }
// Support for uploading rpm packages with arbitrary depth paths
func RpmRoutes(r *web.Route) func() {
var (
groupRepoInfo = regexp.MustCompile(`\A((?:/(?:[^/]+))*|)\.repo\z`)
groupUpload = regexp.MustCompile(`\A((?:/(?:[^/]+))*|)/upload\z`)
groupRpm = regexp.MustCompile(`\A((?:/(?:[^/]+))*|)/package/([^/]+)/([^/]+)/([^/]+)(?:/([^/]+\.rpm)|)\z`)
groupMetadata = regexp.MustCompile(`\A((?:/(?:[^/]+))*|)/repodata/([^/]+)\z`)
)
return func() {
r.Methods("HEAD,GET,POST,PUT,PATCH,DELETE", "*", func(ctx *context.Context) {
path := ctx.Params("*")
isHead := ctx.Req.Method == "HEAD"
isGetHead := ctx.Req.Method == "HEAD" || ctx.Req.Method == "GET"
isPut := ctx.Req.Method == "PUT"
isDelete := ctx.Req.Method == "DELETE"
if path == "/repository.key" && isGetHead {
rpm.GetRepositoryKey(ctx)
return
}
// get repo
m := groupRepoInfo.FindStringSubmatch(path)
if len(m) == 2 && isGetHead {
ctx.SetParams("group", strings.Trim(m[1], "/"))
rpm.GetRepositoryConfig(ctx)
return
}
// get meta
m = groupMetadata.FindStringSubmatch(path)
if len(m) == 3 && isGetHead {
ctx.SetParams("group", strings.Trim(m[1], "/"))
ctx.SetParams("filename", m[2])
if isHead {
rpm.CheckRepositoryFileExistence(ctx)
} else {
rpm.GetRepositoryFile(ctx)
}
return
}
// upload
m = groupUpload.FindStringSubmatch(path)
if len(m) == 2 && isPut {
reqPackageAccess(perm.AccessModeWrite)(ctx)
if ctx.Written() {
return
}
ctx.SetParams("group", strings.Trim(m[1], "/"))
rpm.UploadPackageFile(ctx)
return
}
// rpm down/delete
m = groupRpm.FindStringSubmatch(path)
if len(m) == 6 {
ctx.SetParams("group", strings.Trim(m[1], "/"))
ctx.SetParams("name", m[2])
ctx.SetParams("version", m[3])
ctx.SetParams("architecture", m[4])
if isGetHead {
rpm.DownloadPackageFile(ctx)
return
} else if isDelete {
reqPackageAccess(perm.AccessModeWrite)(ctx)
if ctx.Written() {
return
}
rpm.DeletePackageFile(ctx)
}
}
// default
ctx.Status(http.StatusNotFound)
})
}
}
// ContainerRoutes provides endpoints that implement the OCI API to serve containers // ContainerRoutes provides endpoints that implement the OCI API to serve containers
// These have to be mounted on `/v2/...` to comply with the OCI spec: // These have to be mounted on `/v2/...` to comply with the OCI spec:
// https://github.com/opencontainers/distribution-spec/blob/main/spec.md // https://github.com/opencontainers/distribution-spec/blob/main/spec.md

View file

@ -33,11 +33,14 @@ func apiError(ctx *context.Context, status int, obj any) {
// https://dnf.readthedocs.io/en/latest/conf_ref.html // https://dnf.readthedocs.io/en/latest/conf_ref.html
func GetRepositoryConfig(ctx *context.Context) { func GetRepositoryConfig(ctx *context.Context) {
group := ctx.Params("group")
if group != "" {
group = fmt.Sprintf("/%s", group)
}
url := fmt.Sprintf("%sapi/packages/%s/rpm", setting.AppURL, ctx.Package.Owner.Name) url := fmt.Sprintf("%sapi/packages/%s/rpm", setting.AppURL, ctx.Package.Owner.Name)
ctx.PlainText(http.StatusOK, `[gitea-`+ctx.Package.Owner.LowerName+strings.ReplaceAll(group, "/", "-")+`]
ctx.PlainText(http.StatusOK, `[gitea-`+ctx.Package.Owner.LowerName+`] name=`+ctx.Package.Owner.Name+` - `+setting.AppName+strings.ReplaceAll(group, "/", " - ")+`
name=`+ctx.Package.Owner.Name+` - `+setting.AppName+` baseurl=`+url+group+`/
baseurl=`+url+`
enabled=1 enabled=1
gpgcheck=1 gpgcheck=1
gpgkey=`+url+`/repository.key`) gpgkey=`+url+`/repository.key`)
@ -64,7 +67,7 @@ func CheckRepositoryFileExistence(ctx *context.Context) {
return return
} }
pf, err := packages_model.GetFileForVersionByName(ctx, pv.ID, ctx.Params("filename"), packages_model.EmptyFileKey) pf, err := packages_model.GetFileForVersionByName(ctx, pv.ID, ctx.Params("filename"), ctx.Params("group"))
if err != nil { if err != nil {
if errors.Is(err, util.ErrNotExist) { if errors.Is(err, util.ErrNotExist) {
ctx.Status(http.StatusNotFound) ctx.Status(http.StatusNotFound)
@ -93,7 +96,8 @@ func GetRepositoryFile(ctx *context.Context) {
ctx, ctx,
pv, pv,
&packages_service.PackageFileInfo{ &packages_service.PackageFileInfo{
Filename: ctx.Params("filename"), Filename: ctx.Params("filename"),
CompositeKey: ctx.Params("group"),
}, },
) )
if err != nil { if err != nil {
@ -145,7 +149,7 @@ func UploadPackageFile(ctx *context.Context) {
apiError(ctx, http.StatusInternalServerError, err) apiError(ctx, http.StatusInternalServerError, err)
return return
} }
group := ctx.Params("group")
_, _, err = packages_service.CreatePackageOrAddFileToExisting( _, _, err = packages_service.CreatePackageOrAddFileToExisting(
ctx, ctx,
&packages_service.PackageCreationInfo{ &packages_service.PackageCreationInfo{
@ -153,14 +157,15 @@ func UploadPackageFile(ctx *context.Context) {
Owner: ctx.Package.Owner, Owner: ctx.Package.Owner,
PackageType: packages_model.TypeRpm, PackageType: packages_model.TypeRpm,
Name: pck.Name, Name: pck.Name,
Version: pck.Version, Version: strings.Trim(fmt.Sprintf("%s/%s", group, pck.Version), "/"),
}, },
Creator: ctx.Doer, Creator: ctx.Doer,
Metadata: pck.VersionMetadata, Metadata: pck.VersionMetadata,
}, },
&packages_service.PackageFileCreationInfo{ &packages_service.PackageFileCreationInfo{
PackageFileInfo: packages_service.PackageFileInfo{ PackageFileInfo: packages_service.PackageFileInfo{
Filename: fmt.Sprintf("%s-%s.%s.rpm", pck.Name, pck.Version, pck.FileMetadata.Architecture), Filename: fmt.Sprintf("%s-%s.%s.rpm", pck.Name, pck.Version, pck.FileMetadata.Architecture),
CompositeKey: group,
}, },
Creator: ctx.Doer, Creator: ctx.Doer,
Data: buf, Data: buf,
@ -182,7 +187,7 @@ func UploadPackageFile(ctx *context.Context) {
return return
} }
if err := rpm_service.BuildRepositoryFiles(ctx, ctx.Package.Owner.ID); err != nil { if err := rpm_service.BuildRepositoryFiles(ctx, ctx.Package.Owner.ID, group); err != nil {
apiError(ctx, http.StatusInternalServerError, err) apiError(ctx, http.StatusInternalServerError, err)
return return
} }
@ -191,19 +196,20 @@ func UploadPackageFile(ctx *context.Context) {
} }
func DownloadPackageFile(ctx *context.Context) { func DownloadPackageFile(ctx *context.Context) {
group := ctx.Params("group")
name := ctx.Params("name") name := ctx.Params("name")
version := ctx.Params("version") version := ctx.Params("version")
s, u, pf, err := packages_service.GetFileStreamByPackageNameAndVersion( s, u, pf, err := packages_service.GetFileStreamByPackageNameAndVersion(
ctx, ctx,
&packages_service.PackageInfo{ &packages_service.PackageInfo{
Owner: ctx.Package.Owner, Owner: ctx.Package.Owner,
PackageType: packages_model.TypeRpm, PackageType: packages_model.TypeRpm,
Name: name, Name: name,
Version: version, Version: strings.Trim(fmt.Sprintf("%s/%s", group, version), "/"),
}, },
&packages_service.PackageFileInfo{ &packages_service.PackageFileInfo{
Filename: fmt.Sprintf("%s-%s.%s.rpm", name, version, ctx.Params("architecture")), Filename: fmt.Sprintf("%s-%s.%s.rpm", name, version, ctx.Params("architecture")),
CompositeKey: group,
}, },
) )
if err != nil { if err != nil {
@ -219,14 +225,19 @@ func DownloadPackageFile(ctx *context.Context) {
} }
func DeletePackageFile(webctx *context.Context) { func DeletePackageFile(webctx *context.Context) {
group := webctx.Params("group")
name := webctx.Params("name") name := webctx.Params("name")
version := webctx.Params("version") version := webctx.Params("version")
architecture := webctx.Params("architecture") architecture := webctx.Params("architecture")
var pd *packages_model.PackageDescriptor var pd *packages_model.PackageDescriptor
err := db.WithTx(webctx, func(ctx stdctx.Context) error { err := db.WithTx(webctx, func(ctx stdctx.Context) error {
pv, err := packages_model.GetVersionByNameAndVersion(ctx, webctx.Package.Owner.ID, packages_model.TypeRpm, name, version) pv, err := packages_model.GetVersionByNameAndVersion(ctx,
webctx.Package.Owner.ID,
packages_model.TypeRpm,
name,
strings.Trim(fmt.Sprintf("%s/%s", group, version), "/"),
)
if err != nil { if err != nil {
return err return err
} }
@ -235,7 +246,7 @@ func DeletePackageFile(webctx *context.Context) {
ctx, ctx,
pv.ID, pv.ID,
fmt.Sprintf("%s-%s.%s.rpm", name, version, architecture), fmt.Sprintf("%s-%s.%s.rpm", name, version, architecture),
packages_model.EmptyFileKey, group,
) )
if err != nil { if err != nil {
return err return err
@ -275,7 +286,7 @@ func DeletePackageFile(webctx *context.Context) {
notify_service.PackageDelete(webctx, webctx.Doer, pd) notify_service.PackageDelete(webctx, webctx.Doer, pd)
} }
if err := rpm_service.BuildRepositoryFiles(webctx, webctx.Package.Owner.ID); err != nil { if err := rpm_service.BuildRepositoryFiles(webctx, webctx.Package.Owner.ID, group); err != nil {
apiError(webctx, http.StatusInternalServerError, err) apiError(webctx, http.StatusInternalServerError, err)
return return
} }

View file

@ -125,17 +125,18 @@ type packageData struct {
type packageCache = map[*packages_model.PackageFile]*packageData type packageCache = map[*packages_model.PackageFile]*packageData
// BuildRepositoryFiles builds metadata files for the repository // BuildSpecificRepositoryFiles builds metadata files for the repository
func BuildRepositoryFiles(ctx context.Context, ownerID int64) error { func BuildRepositoryFiles(ctx context.Context, ownerID int64, compositeKey string) error {
pv, err := GetOrCreateRepositoryVersion(ctx, ownerID) pv, err := GetOrCreateRepositoryVersion(ctx, ownerID)
if err != nil { if err != nil {
return err return err
} }
pfs, _, err := packages_model.SearchFiles(ctx, &packages_model.PackageFileSearchOptions{ pfs, _, err := packages_model.SearchFiles(ctx, &packages_model.PackageFileSearchOptions{
OwnerID: ownerID, OwnerID: ownerID,
PackageType: packages_model.TypeRpm, PackageType: packages_model.TypeRpm,
Query: "%.rpm", Query: "%.rpm",
CompositeKey: compositeKey,
}) })
if err != nil { if err != nil {
return err return err
@ -194,15 +195,15 @@ func BuildRepositoryFiles(ctx context.Context, ownerID int64) error {
cache[pf] = pd cache[pf] = pd
} }
primary, err := buildPrimary(ctx, pv, pfs, cache) primary, err := buildPrimary(ctx, pv, pfs, cache, compositeKey)
if err != nil { if err != nil {
return err return err
} }
filelists, err := buildFilelists(ctx, pv, pfs, cache) filelists, err := buildFilelists(ctx, pv, pfs, cache, compositeKey)
if err != nil { if err != nil {
return err return err
} }
other, err := buildOther(ctx, pv, pfs, cache) other, err := buildOther(ctx, pv, pfs, cache, compositeKey)
if err != nil { if err != nil {
return err return err
} }
@ -216,11 +217,12 @@ func BuildRepositoryFiles(ctx context.Context, ownerID int64) error {
filelists, filelists,
other, other,
}, },
compositeKey,
) )
} }
// https://docs.pulpproject.org/en/2.19/plugins/pulp_rpm/tech-reference/rpm.html#repomd-xml // https://docs.pulpproject.org/en/2.19/plugins/pulp_rpm/tech-reference/rpm.html#repomd-xml
func buildRepomd(ctx context.Context, pv *packages_model.PackageVersion, ownerID int64, data []*repoData) error { func buildRepomd(ctx context.Context, pv *packages_model.PackageVersion, ownerID int64, data []*repoData, compositeKey string) error {
type Repomd struct { type Repomd struct {
XMLName xml.Name `xml:"repomd"` XMLName xml.Name `xml:"repomd"`
Xmlns string `xml:"xmlns,attr"` Xmlns string `xml:"xmlns,attr"`
@ -275,7 +277,8 @@ func buildRepomd(ctx context.Context, pv *packages_model.PackageVersion, ownerID
pv, pv,
&packages_service.PackageFileCreationInfo{ &packages_service.PackageFileCreationInfo{
PackageFileInfo: packages_service.PackageFileInfo{ PackageFileInfo: packages_service.PackageFileInfo{
Filename: file.Name, Filename: file.Name,
CompositeKey: compositeKey,
}, },
Creator: user_model.NewGhostUser(), Creator: user_model.NewGhostUser(),
Data: file.Data, Data: file.Data,
@ -292,7 +295,7 @@ func buildRepomd(ctx context.Context, pv *packages_model.PackageVersion, ownerID
} }
// https://docs.pulpproject.org/en/2.19/plugins/pulp_rpm/tech-reference/rpm.html#primary-xml // https://docs.pulpproject.org/en/2.19/plugins/pulp_rpm/tech-reference/rpm.html#primary-xml
func buildPrimary(ctx context.Context, pv *packages_model.PackageVersion, pfs []*packages_model.PackageFile, c packageCache) (*repoData, error) { func buildPrimary(ctx context.Context, pv *packages_model.PackageVersion, pfs []*packages_model.PackageFile, c packageCache, compositeKey string) (*repoData, error) {
type Version struct { type Version struct {
Epoch string `xml:"epoch,attr"` Epoch string `xml:"epoch,attr"`
Version string `xml:"ver,attr"` Version string `xml:"ver,attr"`
@ -372,7 +375,7 @@ func buildPrimary(ctx context.Context, pv *packages_model.PackageVersion, pfs []
files = append(files, f) files = append(files, f)
} }
} }
packageVersion := fmt.Sprintf("%s-%s", pd.FileMetadata.Version, pd.FileMetadata.Release)
packages = append(packages, &Package{ packages = append(packages, &Package{
Type: "rpm", Type: "rpm",
Name: pd.Package.Name, Name: pd.Package.Name,
@ -401,7 +404,7 @@ func buildPrimary(ctx context.Context, pv *packages_model.PackageVersion, pfs []
Archive: pd.FileMetadata.ArchiveSize, Archive: pd.FileMetadata.ArchiveSize,
}, },
Location: Location{ Location: Location{
Href: fmt.Sprintf("package/%s/%s/%s", url.PathEscape(pd.Package.Name), url.PathEscape(pd.Version.Version), url.PathEscape(pd.FileMetadata.Architecture)), Href: fmt.Sprintf("package/%s/%s/%s/%s", url.PathEscape(pd.Package.Name), url.PathEscape(packageVersion), url.PathEscape(pd.FileMetadata.Architecture), url.PathEscape(fmt.Sprintf("%s-%s.%s.rpm", pd.Package.Name, packageVersion, pd.FileMetadata.Architecture))),
}, },
Format: Format{ Format: Format{
License: pd.VersionMetadata.License, License: pd.VersionMetadata.License,
@ -431,11 +434,11 @@ func buildPrimary(ctx context.Context, pv *packages_model.PackageVersion, pfs []
XmlnsRpm: "http://linux.duke.edu/metadata/rpm", XmlnsRpm: "http://linux.duke.edu/metadata/rpm",
PackageCount: len(pfs), PackageCount: len(pfs),
Packages: packages, Packages: packages,
}) }, compositeKey)
} }
// https://docs.pulpproject.org/en/2.19/plugins/pulp_rpm/tech-reference/rpm.html#filelists-xml // https://docs.pulpproject.org/en/2.19/plugins/pulp_rpm/tech-reference/rpm.html#filelists-xml
func buildFilelists(ctx context.Context, pv *packages_model.PackageVersion, pfs []*packages_model.PackageFile, c packageCache) (*repoData, error) { //nolint:dupl func buildFilelists(ctx context.Context, pv *packages_model.PackageVersion, pfs []*packages_model.PackageFile, c packageCache, compositeKey string) (*repoData, error) { //nolint:dupl
type Version struct { type Version struct {
Epoch string `xml:"epoch,attr"` Epoch string `xml:"epoch,attr"`
Version string `xml:"ver,attr"` Version string `xml:"ver,attr"`
@ -478,11 +481,12 @@ func buildFilelists(ctx context.Context, pv *packages_model.PackageVersion, pfs
Xmlns: "http://linux.duke.edu/metadata/other", Xmlns: "http://linux.duke.edu/metadata/other",
PackageCount: len(pfs), PackageCount: len(pfs),
Packages: packages, Packages: packages,
}) },
compositeKey)
} }
// https://docs.pulpproject.org/en/2.19/plugins/pulp_rpm/tech-reference/rpm.html#other-xml // https://docs.pulpproject.org/en/2.19/plugins/pulp_rpm/tech-reference/rpm.html#other-xml
func buildOther(ctx context.Context, pv *packages_model.PackageVersion, pfs []*packages_model.PackageFile, c packageCache) (*repoData, error) { //nolint:dupl func buildOther(ctx context.Context, pv *packages_model.PackageVersion, pfs []*packages_model.PackageFile, c packageCache, compositeKey string) (*repoData, error) { //nolint:dupl
type Version struct { type Version struct {
Epoch string `xml:"epoch,attr"` Epoch string `xml:"epoch,attr"`
Version string `xml:"ver,attr"` Version string `xml:"ver,attr"`
@ -525,7 +529,7 @@ func buildOther(ctx context.Context, pv *packages_model.PackageVersion, pfs []*p
Xmlns: "http://linux.duke.edu/metadata/other", Xmlns: "http://linux.duke.edu/metadata/other",
PackageCount: len(pfs), PackageCount: len(pfs),
Packages: packages, Packages: packages,
}) }, compositeKey)
} }
// writtenCounter counts all written bytes // writtenCounter counts all written bytes
@ -545,10 +549,8 @@ func (wc *writtenCounter) Written() int64 {
return wc.written return wc.written
} }
func addDataAsFileToRepo(ctx context.Context, pv *packages_model.PackageVersion, filetype string, obj any) (*repoData, error) { func addDataAsFileToRepo(ctx context.Context, pv *packages_model.PackageVersion, filetype string, obj any, compositeKey string) (*repoData, error) {
content, _ := packages_module.NewHashedBuffer() content, _ := packages_module.NewHashedBuffer()
defer content.Close()
gzw := gzip.NewWriter(content) gzw := gzip.NewWriter(content)
wc := &writtenCounter{} wc := &writtenCounter{}
h := sha256.New() h := sha256.New()
@ -571,7 +573,8 @@ func addDataAsFileToRepo(ctx context.Context, pv *packages_model.PackageVersion,
pv, pv,
&packages_service.PackageFileCreationInfo{ &packages_service.PackageFileCreationInfo{
PackageFileInfo: packages_service.PackageFileInfo{ PackageFileInfo: packages_service.PackageFileInfo{
Filename: filename, Filename: filename,
CompositeKey: compositeKey,
}, },
Creator: user_model.NewGhostUser(), Creator: user_model.NewGhostUser(),
Data: content, Data: content,

View file

@ -4,19 +4,23 @@
<div class="ui form"> <div class="ui form">
<div class="field"> <div class="field">
<label>{{svg "octicon-terminal"}} {{ctx.Locale.Tr "packages.rpm.registry"}}</label> <label>{{svg "octicon-terminal"}} {{ctx.Locale.Tr "packages.rpm.registry"}}</label>
<div class="markup"><pre class="code-block"><code># {{ctx.Locale.Tr "packages.rpm.distro.redhat"}} <div class="markup"><pre class="code-block"><code># {{ctx.Locale.Tr "packages.rpm.distros.redhat"}}
dnf config-manager --add-repo <gitea-origin-url data-url="{{AppSubUrl}}/api/packages/{{$.PackageDescriptor.Owner.Name}}/rpm.repo"></gitea-origin-url> {{$group_name:= StringUtils.ReplaceAllStringRegex .PackageDescriptor.Version.Version "(/[^/]+|[^/]*)\\z" "" -}}
{{- if $group_name -}}
{{- $group_name = (print "/" $group_name) -}}
{{- end -}}
dnf config-manager --add-repo <gitea-origin-url data-url="{{AppSubUrl}}/api/packages/{{$.PackageDescriptor.Owner.Name}}/rpm{{$group_name}}.repo"></gitea-origin-url>
# {{ctx.Locale.Tr "packages.rpm.distro.suse"}} # {{ctx.Locale.Tr "packages.rpm.distros.suse"}}
zypper addrepo <gitea-origin-url data-url="{{AppSubUrl}}/api/packages/{{$.PackageDescriptor.Owner.Name}}/rpm.repo"></gitea-origin-url></code></pre></div> zypper addrepo <gitea-origin-url data-url="{{AppSubUrl}}/api/packages/{{$.PackageDescriptor.Owner.Name}}/rpm{{$group_name}}.repo"></gitea-origin-url></code></pre></div>
</div> </div>
<div class="field"> <div class="field">
<label>{{svg "octicon-terminal"}} {{ctx.Locale.Tr "packages.rpm.install"}}</label> <label>{{svg "octicon-terminal"}} {{ctx.Locale.Tr "packages.rpm.install"}}</label>
<div class="markup"> <div class="markup">
<pre class="code-block"><code># {{ctx.Locale.Tr "packages.rpm.distro.redhat"}} <pre class="code-block"><code># {{ctx.Locale.Tr "packages.rpm.distros.redhat"}}
dnf install {{$.PackageDescriptor.Package.Name}} dnf install {{$.PackageDescriptor.Package.Name}}
# {{ctx.Locale.Tr "packages.rpm.distro.suse"}} # {{ctx.Locale.Tr "packages.rpm.distros.suse"}}
zypper install {{$.PackageDescriptor.Package.Name}}</code></pre> zypper install {{$.PackageDescriptor.Package.Name}}</code></pre>
</div> </div>
</div> </div>

View file

@ -76,12 +76,12 @@ Mu0UFYgZ/bYnuvn/vz4wtCz8qMwsHUvP0PX3tbYFUctAPdrY6tiiDtcCddDECahx7SuVNP5dpmb5
t.Run("RepositoryConfig", func(t *testing.T) { t.Run("RepositoryConfig", func(t *testing.T) {
defer tests.PrintCurrentTest(t)() defer tests.PrintCurrentTest(t)()
req := NewRequest(t, "GET", rootURL+".repo") req := NewRequest(t, "GET", rootURL+"/el9/stable.repo")
resp := MakeRequest(t, req, http.StatusOK) resp := MakeRequest(t, req, http.StatusOK)
expected := fmt.Sprintf(`[gitea-%s] expected := fmt.Sprintf(`[gitea-%s-el9-stable]
name=%s - %s name=%s - %s - el9 - stable
baseurl=%sapi/packages/%s/rpm baseurl=%sapi/packages/%s/rpm/el9/stable/
enabled=1 enabled=1
gpgcheck=1 gpgcheck=1
gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppName, setting.AppURL, user.Name, setting.AppURL, user.Name) gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppName, setting.AppURL, user.Name, setting.AppURL, user.Name)
@ -100,7 +100,7 @@ gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppN
}) })
t.Run("Upload", func(t *testing.T) { t.Run("Upload", func(t *testing.T) {
url := rootURL + "/upload" url := rootURL + "/el9/stable/upload"
req := NewRequestWithBody(t, "PUT", url, bytes.NewReader(content)) req := NewRequestWithBody(t, "PUT", url, bytes.NewReader(content))
MakeRequest(t, req, http.StatusUnauthorized) MakeRequest(t, req, http.StatusUnauthorized)
@ -118,7 +118,7 @@ gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppN
assert.Nil(t, pd.SemVer) assert.Nil(t, pd.SemVer)
assert.IsType(t, &rpm_module.VersionMetadata{}, pd.Metadata) assert.IsType(t, &rpm_module.VersionMetadata{}, pd.Metadata)
assert.Equal(t, packageName, pd.Package.Name) assert.Equal(t, packageName, pd.Package.Name)
assert.Equal(t, packageVersion, pd.Version.Version) assert.Equal(t, fmt.Sprintf("el9/stable/%s", packageVersion), pd.Version.Version)
pfs, err := packages.GetFilesByVersionID(db.DefaultContext, pvs[0].ID) pfs, err := packages.GetFilesByVersionID(db.DefaultContext, pvs[0].ID)
assert.NoError(t, err) assert.NoError(t, err)
@ -138,7 +138,7 @@ gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppN
t.Run("Download", func(t *testing.T) { t.Run("Download", func(t *testing.T) {
defer tests.PrintCurrentTest(t)() defer tests.PrintCurrentTest(t)()
req := NewRequest(t, "GET", fmt.Sprintf("%s/package/%s/%s/%s", rootURL, packageName, packageVersion, packageArchitecture)) req := NewRequest(t, "GET", fmt.Sprintf("%s/el9/stable/package/%s/%s/%s", rootURL, packageName, packageVersion, packageArchitecture))
resp := MakeRequest(t, req, http.StatusOK) resp := MakeRequest(t, req, http.StatusOK)
assert.Equal(t, content, resp.Body.Bytes()) assert.Equal(t, content, resp.Body.Bytes())
@ -147,7 +147,7 @@ gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppN
t.Run("Repository", func(t *testing.T) { t.Run("Repository", func(t *testing.T) {
defer tests.PrintCurrentTest(t)() defer tests.PrintCurrentTest(t)()
url := rootURL + "/repodata" url := rootURL + "/el9/stable/repodata"
req := NewRequest(t, "HEAD", url+"/dummy.xml") req := NewRequest(t, "HEAD", url+"/dummy.xml")
MakeRequest(t, req, http.StatusNotFound) MakeRequest(t, req, http.StatusNotFound)
@ -201,8 +201,8 @@ gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppN
switch d.Type { switch d.Type {
case "primary": case "primary":
assert.EqualValues(t, 718, d.Size) assert.EqualValues(t, 722, d.Size)
assert.EqualValues(t, 1729, d.OpenSize) assert.EqualValues(t, 1759, d.OpenSize)
assert.Equal(t, "repodata/primary.xml.gz", d.Location.Href) assert.Equal(t, "repodata/primary.xml.gz", d.Location.Href)
case "filelists": case "filelists":
assert.EqualValues(t, 257, d.Size) assert.EqualValues(t, 257, d.Size)
@ -311,7 +311,7 @@ gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppN
assert.EqualValues(t, len(content), p.Size.Package) assert.EqualValues(t, len(content), p.Size.Package)
assert.EqualValues(t, 13, p.Size.Installed) assert.EqualValues(t, 13, p.Size.Installed)
assert.EqualValues(t, 272, p.Size.Archive) assert.EqualValues(t, 272, p.Size.Archive)
assert.Equal(t, fmt.Sprintf("package/%s/%s/%s", packageName, packageVersion, packageArchitecture), p.Location.Href) assert.Equal(t, fmt.Sprintf("package/%s/%s/%s/%s", packageName, packageVersion, packageArchitecture, fmt.Sprintf("%s-%s.%s.rpm", packageName, packageVersion, packageArchitecture)), p.Location.Href)
f := p.Format f := p.Format
assert.Equal(t, "MIT", f.License) assert.Equal(t, "MIT", f.License)
assert.Len(t, f.Provides.Entries, 2) assert.Len(t, f.Provides.Entries, 2)
@ -401,18 +401,17 @@ gpgkey=%sapi/packages/%s/rpm/repository.key`, user.Name, user.Name, setting.AppN
t.Run("Delete", func(t *testing.T) { t.Run("Delete", func(t *testing.T) {
defer tests.PrintCurrentTest(t)() defer tests.PrintCurrentTest(t)()
req := NewRequest(t, "DELETE", fmt.Sprintf("%s/package/%s/%s/%s", rootURL, packageName, packageVersion, packageArchitecture)) req := NewRequest(t, "DELETE", fmt.Sprintf("%s/el9/stable/package/%s/%s/%s", rootURL, packageName, packageVersion, packageArchitecture))
MakeRequest(t, req, http.StatusUnauthorized) MakeRequest(t, req, http.StatusUnauthorized)
req = NewRequest(t, "DELETE", fmt.Sprintf("%s/package/%s/%s/%s", rootURL, packageName, packageVersion, packageArchitecture)). req = NewRequest(t, "DELETE", fmt.Sprintf("%s/el9/stable/package/%s/%s/%s", rootURL, packageName, packageVersion, packageArchitecture)).
AddBasicAuth(user.Name) AddBasicAuth(user.Name)
MakeRequest(t, req, http.StatusNoContent) MakeRequest(t, req, http.StatusNoContent)
pvs, err := packages.GetVersionsByPackageType(db.DefaultContext, user.ID, packages.TypeRpm) pvs, err := packages.GetVersionsByPackageType(db.DefaultContext, user.ID, packages.TypeRpm)
assert.NoError(t, err) assert.NoError(t, err)
assert.Empty(t, pvs) assert.Empty(t, pvs)
req = NewRequest(t, "DELETE", fmt.Sprintf("%s/el9/stable/package/%s/%s/%s", rootURL, packageName, packageVersion, packageArchitecture)).
req = NewRequest(t, "DELETE", fmt.Sprintf("%s/package/%s/%s/%s", rootURL, packageName, packageVersion, packageArchitecture)).
AddBasicAuth(user.Name) AddBasicAuth(user.Name)
MakeRequest(t, req, http.StatusNotFound) MakeRequest(t, req, http.StatusNotFound)
}) })