Commit graph

72 commits

Author SHA1 Message Date
Dessalines 0a6186d7aa
Fixing apt-key deprecation, and using apt-get. (#4931)
* Fixing apt-key deprecation, and using apt-get.

* Try 2

* Try 3

* Try 4

* Try 5

* Try 6

* Try 7

* Try 8

* Try 9

* Try 10

* Try 11

* Try 12

* Try 13

* Try 14

* Try 15

* Try 16

* Try 17
2024-07-31 18:54:44 -04:00
SleeplessOne1917 1ef375e5c3
Replace 3rd party once_cell crate with recently stabilized standard library API (#4919)
* Replace 3rd party once_cell crate with recently stabilized standard library API

* Bump CI rust version
2024-07-30 10:11:39 -04:00
phiresky a08642f813
federation: parallel sending per instance (#4623)
* federation: parallel sending

* federation: some comments

* lint and set force_write true when a request fails

* inbox_urls return vec

* split inbox functions into separate file

* cleanup

* extract sending task code to separate file

* move federation concurrent config to config file

* off by one issue

* improve msg

* fix both permanent stopping of federation queues and multiple creation of the same federation queues

* fix after merge

* lint fix

* Update crates/federate/src/send.rs

Co-authored-by: dullbananas <dull.bananas0@gmail.com>

* comment about reverse ordering

* remove crashable, comment

* comment

* move comment

* run federation tests twice

* fix test run

* prettier

* fix config default

* upgrade rust to 1.78 to fix diesel cli

* fix clippy

* delay

* add debug to make localhost urls not valid in ap crate, add some debug logs

* federation tests: ensure server stop after test and random activity id

* ci fix

* add test to federate 100 events

* fix send 100 test

* different data every time so activities are distinguishable

* allow out of order receives in test

* lint

* comment about https://github.com/LemmyNet/lemmy/pull/4623#discussion_r1565437391

* move sender for clarity, add comment

* move more things to members

* update test todo comment, use same env var as worker test but default to 1

* remove else below continue

* some more cleanup

* handle todo about smooth exit

* add federate inboxes collector tests

* lint

* actor max length

* don't reset fail count if activity skipped

* fix some comments

* reuse vars

* format

* Update .woodpecker.yml

* fix recheck time

* fix inboxes tests under fast mode

* format

* make i32 and ugly casts

* clippy

---------

Co-authored-by: dullbananas <dull.bananas0@gmail.com>
2024-07-21 11:50:50 -04:00
Daniel Lo Nigro 8abbd56400
Switch PostgreSQL container to use pgautoupgrade (#4892)
* Switch PostgreSQL container to use pgautoupgrade

This handles automatically upgrading the data files to newer versions of PostgreSQL.

* Fixing other uses of postgres:16-alpine image.

* Simplifying upgrade scripts.

---------

Co-authored-by: Dessalines <tyhou13@gmx.com>
2024-07-17 09:59:21 +02:00
phiresky 96b7afc0b1
upgrade rust to 1.78 to fix diesel cli (#4761) 2024-05-31 08:39:45 -04:00
Dessalines 1a4aa3eaba
Stop using a diesel_cli docker image, use cargo-install in woodpecker. (#4723)
* Stop using a diesel_cli docker image, use cargo-binstall in woodpecker.

- The diesel_cli image is 500MB, and rebuilt daily. Much easier to use
  binstall to install it.

* Trying out a multiline var.

* Try sequence merges 1

* Try removing features.

* Try path fix.

* Abstracting diesel cli install.

* Try installing mysql and postgres.

* Try installing mysql and postgres 2.

* Try installing mysql and postgres 3.

* Try installing mysql and postgres 4.

* Try installing mysql and postgres 5.

* Try installing mysql and postgres 6.

* Try installing mysql and postgres 7.

* Try installing mysql and postgres 8.

* Try installing mysql and postgres 9.

* Try installing mysql and postgres 10.

* Try installing mysql and postgres 11.

* Try installing mysql and postgres 12.

* Try installing mysql and postgres 13.

* Add logging line.

* Add logging line 2.

* Add logging line 3.

* Add logging line 4.

* Removing binstall.

* Extract taplo into its own image, ignore binstall.

* Use a smaller taplo.

* taplo is the same image.
2024-05-16 16:41:36 -04:00
SleeplessOne1917 b05f221565
Remove login step from publish to crates.io (#4674) 2024-04-30 00:07:03 +00:00
dullbananas 4ba6221e04
Move SQL triggers from migrations into reusable sql file (#4333)
* stuff

* stuff including batch_upsert function

* stuff

* do things

* stuff

* different timestamps

* stuff

* Revert changes to comment.rs

* Update comment.rs

* Update comment.rs

* Update post_view.rs

* Update utils.rs

* Update up.sql

* Update up.sql

* Update down.sql

* Update up.sql

* Update main.rs

* use anyhow macro

* Create down.sql

* Create up.sql

* Create replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update utils.rs

* Update .woodpecker.yml

* Update sql_format_check.sh

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Create dump_schema.sh

* Update start_dev_db.sh

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* stuff

* Update replaceable_schema.sql

* Update .pg_format

* fmt

* stuff

* stuff (#21)

* Update replaceable_schema.sql

* Update up.sql

* Update replaceable_schema.sql

* fmt

* update cargo.lock

* stuff

* Update replaceable_schema.sql

* Remove truncate trigger because truncate is already restricted by foreign keys

* Update replaceable_schema.sql

* fix some things

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update .woodpecker.yml

* stuff

* fix TG_OP

* Psql env vars

* try to fix combine_transition_tables parse error

* Revert "try to fix combine_transition_tables parse error"

This reverts commit 75d00a4626.

* refactor combine_transition_tables

* try to fix create_triggers

* fix some things

* try to fix combined_transition_tables

* fix sql errors

* update comment count in post trigger

* fmt

* Revert "fmt"

This reverts commit a5bcd0834b.

* Revert "update comment count in post trigger"

This reverts commit 0066a4b42b.

* fix everything

* Update replaceable_schema.sql

* actually fix everything

* refactor create_triggers

* fix

* add semicolons

* add is_counted function and fix incorrect bool operator in update_comment_count_from_post

* refactor comment trigger

* refactor post trigger

* fix

* Delete crates/db_schema/src/utils/series.rs

* subscribers_local

* edit migrations

* move migrations

* remove utils::series module declaration

* fix everything

* stuff

* Move sql to schema_setup dir

* utils.sql

* delete .pg_format

* Update .woodpecker.yml

* Update sql_format_check.sh

* Update .woodpecker.yml

* Merge remote-tracking branch 'upstream/main' into bliss

* fmt

* Create main.rs

* Update lib.rs

* Update main.rs

* Update .woodpecker.yml

* Update main.rs

* Update Cargo.toml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update triggers.sql

* YAY

* Update mod.rs

* Update Cargo.toml

* a

* Update Cargo.toml

* Update Cargo.toml

* Delete crates/db_schema/src/main.rs

* Update Cargo.toml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update utils.sql

* Update utils.sql

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update down.sql

* Update up.sql

* Update triggers.sql

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update triggers.sql

* Update down.sql

* Update .woodpecker.yml

* Update Cargo.toml

* Update .woodpecker.yml

* Update Cargo.toml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update mod.rs

* Update Cargo.toml

* Update mod.rs

* make dump_schema.sh executable

* fix dump_schema.sh

* defer

* diff dumps

* fmt

* Update utils.sql

* Update .woodpecker.yml

* use correct version for pg_dump

* Update .woodpecker.yml

* Update .woodpecker.yml

* change migration date

* atomic site_aggregates insert

* temporarily repeat tests in CI

* drop r schema in CI migration check

* show ReceivedActivity::create error

* move check_diesel_migration CI step

* Update .woodpecker.yml

* Update scheduled_tasks.rs

* Update scheduled_tasks.rs

* update cargo.lock

* move sql files

* move rank functions

* filter post_aggregates update

* fmt

* cargo fmt

* replace post_id with id

* update cargo.lock

* avoid locking rows that need no change in up.sql

* only run replaceable_schema if migrations were run

* debug ci test failure

* make replaceable_schema work in CI

* Update .woodpecker.yml

* remove println

* Use migration revert and git checkout

* Update schema_setup.rs

* Fix

* Update schema_setup.rs

* Update schema_setup.rs

* Update .woodpecker.yml

---------

Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-04-17 20:58:44 -04:00
Dessalines 705e86eb4c
Fixing docker release (#4592)
* Re-add notif on tag failure.

* Upping rust version.

* Version 0.19.4-beta.1a

* Try again.

* Version 0.19.4-beta.1b

* Removing unstable inspect.

* Version 0.19.4-beta.1c

* Remove use release cache.

* Trying to fix cargo publish 1.

* Version 0.19.4-beta.1d

* Re-adding publish release

* Version 0.19.4-beta.2

* Fixing workspace for lemmy_federate
2024-04-04 16:14:59 +02:00
Dessalines a1d632e582
Re-add notif on tag failure. (#4591) 2024-04-04 10:21:31 +02:00
Dessalines aaaa362b98
Remove latest tag for pgformatter. (#4589) 2024-04-03 17:29:24 -04:00
Dessalines 067332553d
Fixing woodpecker. (#4575) 2024-03-27 11:09:44 -04:00
Dessalines 95069d7648
Fixing some clippy and woodpecker lints. (#4565)
* Fixing some clippy and woodpecker lints.

* Try fixing woodpecker 1.

* Revert "Try fixing woodpecker 1."

This reverts commit 7c2020a08d.
2024-03-26 10:17:42 +01:00
Richard Schwab 80bfd23b4d
Ensure rustfmt is installed in cargo fmt CI job (#4466) 2024-02-26 11:22:22 +01:00
Dessalines 890565ca14
Deleting denied local_users older than a week. Fixes #4434 (#4448)
* Deleting denied local_users older than a week. Fixes #4434

* Addressing PR comments.

* Upping rust to 1.76

* Delete the person rows also.
2024-02-15 07:50:53 -05:00
Dessalines 9367cbdb00
Upgrading from postgres 15 -> 16-alpine. (#4426)
- Includes an upgrade script.
- Fixes #4406
2024-02-07 11:21:02 +01:00
Dessalines eb0dc2fda4
Moving from yarn to pnpm. (#4414)
* Moving from yarn to pnpm.

* Prettier check.
2024-01-29 11:38:39 +01:00
dullbananas f481a607d0
Show server output if federation tests fail in CI (#4389)
* Show server output if federation tests fail in CI

* dummy failure

* Revert dummy failure

* Update private_message.rs

* Fix errors

* Update private_message.rs

* Update private_message.rs

* Update .woodpecker.yml

* correct exit code

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update prepare-drone-federation-test.sh

* Update .woodpecker.yml

* Update prepare-drone-federation-test.sh

* Update .woodpecker.yml

* Update prepare-drone-federation-test.sh

* Update private_message.rs

* Update lib.rs

* Update lib.rs

* Update session_middleware.rs

* Update session_middleware.rs

* Update .woodpecker.yml
2024-01-26 10:38:15 +01:00
dullbananas ade1cb1495
Remove outdated comment from .woodpecker.yml (#4404) 2024-01-26 10:33:03 +01:00
dullbananas 759f6d8a9a
Better query plan viewing experience (#4285)
* stuff

* stuff including batch_upsert function

* stuff

* do things

* stuff

* different timestamps

* stuff

* Revert changes to comment.rs

* Update comment.rs

* Update comment.rs

* Update post_view.rs

* Update utils.rs

* Update up.sql

* Update up.sql

* Update down.sql

* Update up.sql

* Update main.rs

* use anyhow macro

* replace get(0) with first()

* as_slice

* Update series.rs

* Update db_perf.sh

* Update and rename crates/db_schema/src/utils/series.rs to crates/db_perf/src/series.rs

* Update utils.rs

* Update main.rs

* Update main.rs

* Update .woodpecker.yml

* fmt main.rs

* Update .woodpecker.yml

* Instance::delete at end

* Update main.rs

* Update Cargo.toml

---------

Co-authored-by: Nutomic <me@nutomic.com>
2024-01-24 10:22:33 -05:00
Dessalines e78fe5a34c
Removing group from woodpecker, as its deprecated. (#4387)
* Removing group from woodpecker, as its deprecated.

* Removing meltwater drone cache.
2024-01-22 10:57:07 +01:00
Nutomic 92b49dea3a
Increase CI timeout for restore cache (#4358) 2024-01-09 12:37:48 -05:00
Dessalines 3d9dda4677
Woodpecker rust 1.75 (#4355)
* Upgrading woodpecker rust to 1.75

* Trying to get publish working, try 1.

* Version 0.19.2-rc.3.publish1

* Temporarily disable arm builds.

* Version 0.19.2-rc.3.publish2

* Comment out arm blocks.

* Version 0.19.2-rc.3.publish3

* Moving back publish block to below.

* Upgrade to rasky lemmy-arm v0.2.0

* Version 0.19.2-rc.3.publish4

* Test bump for upgraded woodpecker.

* Revert "Test bump for upgraded woodpecker."

This reverts commit e2a5a99f33.

* Adding link to raskys repo.
2024-01-09 11:48:29 -05:00
Nutomic 952c162dac
Fix CI cache (#4276)
* Minor CI improvements (second attempt)

* test slow check condition

* remove rebuild cache condition for testing

* trigger quick ci check

* mkdir

* ls, remove steps

* exclude cargo home from prettier

* ci

* increase timeout

* more timeout

* even higher just for testing

* disable compression

* also increase restore timeout

* cleanup

* try again

* Update .woodpecker.yml

* Update .woodpecker.yml

* cleanup

* rerun ci

* cleanup
2024-01-04 11:40:41 -05:00
Nutomic 2b9d89057d
Reenable crates.io releases (fixes #4291) (#4294)
* Reenable crates.io releases (fixes #4291)

* disable docker release for now

* fixes

* fix indent

* move to top

* install libssl

* -y

* apt update

* debian version

* install cargo, git

* use cargo install

* move to orig position
2023-12-20 08:18:54 -05:00
Dessalines e4b97ad8a9
Revert "Minor CI improvements (#4266)" (#4271)
This reverts commit af4d008ce4.
2023-12-14 18:36:33 -05:00
Nutomic af4d008ce4
Minor CI improvements (#4266)
* Speed up SQL formatting in CI with parallel processing

* mess up formatting

* check permissions

* make repo world writable

* need git

* use debian image

* apt update

* perl image for faster install

* fix format

* run restore cache in parallel to format

* add drone-cache exit code

* revert sql format changes
2023-12-14 11:18:15 -05:00
Nutomic 4a55d4f871
Quick fix for Peertube federation (fixes #4261) (#4264)
* Quick fix for Peertube federation (fixes #4261)

* prettier

* apub assets should be included in slow check paths
2023-12-14 07:25:47 -05:00
Nutomic 95130370f0
Add CI check for ignored files (#4252)
* Add CI check for ignored files

* remove ignored files
2023-12-12 08:54:06 -05:00
Kroese fc07ba2d3b
Fix entrypoint in Dockerfile (#4202)
* Fix entrypoint

* Delete docker/builders/lemmy-builder-arm64/docker-build.sh

* Delete docker/builders directory

* Remove exception for builder

* Remove publish_builder_arm64
2023-11-26 16:50:31 -05:00
Enzo Nocera 8a05c8f8be
fix: Cross-compilation to ARM64 (#4142)
* feat(docker/docs): explain how building lemmy works

Signed-off-by: Enzo NOCERA <enzo@nocera.eu>

* feat: add arm build

* review: rename script & fix typo

* feat(ci): allow cross platform compilation

* feat(ci): prettier

* fix(docker): fix base image name

* fix: add dockerfile in CI path

Signed-off-by: Enzo Nocera <enzo@nocera.eu>

* fix(docker): fix runner name

* fix(docker): fix builder base image

* fix(docker): fix builder base image platform

* fix(docker): avoid using the wrapper adduser/addgroup

* feat: avoid adding the whole docker directory in the build context

---------

Signed-off-by: Enzo NOCERA <enzo@nocera.eu>
Signed-off-by: Enzo Nocera <enzo@nocera.eu>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
2023-11-24 12:52:19 -05:00
Nutomic 934fe7d1cb
Remove pictrs config section old_db (#4199)
* Remove pictrs config section `old_db` (ref #4194)

* ci

* remove cargo config comment

* ci

* Revert "remove cargo config comment"

This reverts commit 9389e93649.

* rename .cargo to .cargo_home
2023-11-24 11:26:21 -05:00
dullbananas 30f7135f32
Use rust 1.74.0 in CI and move clippy lints to Cargo.toml (#4170)
* ci rust 1.74.0

* do it

* Update .woodpecker.yml

* Update Cargo.toml

* Update lint.sh

* Update lib.rs

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-11-21 08:51:22 -05:00
Nutomic 9007d49324
Remove unused deps (#4111)
* Remove unused deps

* Adding cargo-machete to woodpecker.

---------

Co-authored-by: Dessalines <tyhou13@gmx.com>
2023-10-31 11:11:12 +01:00
Dessalines 645bf21d54
Moving to debian-based docker image. Fixes #3972 (#4004)
* Moving to debian-based docker image. Fixes #3972

* Fix cargo.toml format.

* Try nightly rust for cargo_fmt.

* Trying to fix cargo build.

* Trying to fix cargo build 2.

* Trying to fix federation tests.

* Trying to fix federation tests 2.

* Trying to fix federation tests 3.

* Try to fix aarch 1.

* Try to fix aarch 2.

* Try to fix aarch 3.

* Removing aarch

* Adding tests back in.

* Adding ca-certs, and debian non-root user

* Commenting arm image building.

* Fix platform on runner.

* Version 0.19.0-beta.6
2023-10-11 11:57:05 -04:00
Nutomic 5ef4807d09
Dont create :latest docker tag (fixes #3996) (#4016)
* Dont create :latest docker tag (fixes #3996)

* ci

* Update .woodpecker.yml

Co-authored-by: Richard Schwab <mail@w.tf-w.tf>

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Richard Schwab <mail@w.tf-w.tf>
2023-10-04 18:34:56 -04:00
Ellie bd3f39973f
Add ARM64 Docker image support (#3810)
* Add ARM64 Docker image support

* Fix

* Fix Docker build

* Fix typo

* Use OpenSSL 3 and PostgreSQL Lib v15 in Docker builder

* Use simple comments in Dockerfile

* Add comment explaining Docker image cross build

* Move docker step above tests for testing

* Move docker build step back to normal position

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-08-31 09:02:59 -04:00
Nutomic 7fd14b3d2a
Make remove content optional during account deletion (fixes #1617) (#3817)
* Make remove content optional during account deletion (fixes #1617)

* simplify purge params by passing context

* update js client

* add delete content

* update woodpecker config

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-08-28 12:23:45 +02:00
Maarten Vercruysse 9b710a2ed3
Update .woodpecker.yml use steps instead (#3858) 2023-08-10 08:15:30 -04:00
Dessalines 654bc513ea
Fixing woodpecker env var. (#3854) 2023-08-09 11:14:09 +02:00
Nutomic ae438e148e
Speedup CI (#3852)
* Speedup CI

* run tests in parallel

* restore cp

* main push

* no self update

* parallel format
2023-08-08 10:35:03 -04:00
Nutomic 33b2d67be6
Reorder woodpecker config to avoid ci failures (#3851) 2023-08-08 13:09:27 +02:00
Felix Ableitner bed9474cf0 Revert "Attempt to fix CI building wrong commits (#3830)"
This reverts commit 5e9930bdcc.
2023-08-07 13:19:07 +02:00
dullbananas 5e9930bdcc
Attempt to fix CI building wrong commits (#3830)
* Attempt to fix CI building wrong commits

* Update .woodpecker.yml

* Update .woodpecker.yml
2023-08-07 06:40:50 -04:00
Dessalines be1389420b
Adding SQL format checking via pg_format / pgFormatter (#3740)
* SQL format checking, 1.

* SQL format checking, 2.

* SQL format checking, 3.

* SQL format checking, 4.

* SQL format checking, 5.

* Running pg_format

* Getting rid of comment.

* Upping pg_format version.

* Using git ls-files for sql format check.

* Fixing sql lints.

* Addressing PR comments.
2023-08-02 12:44:51 -04:00
Nutomic 58388f2ce8
Only run slow CI checks for actual code changes (attempt 2) (#3759)
* Only run slow CI checks for actual code changes (attempt 2)

* use when path include

* change rust

* update

* x

* y

* zsad

* remove println

* fmt

* readme

* Revert "readme"

This reverts commit d58b6ad973.

* ci

* gitmodules
2023-08-02 12:29:21 -04:00
Felix Ableitner 963d04b352 Revert "Only run slow CI checks for actual code changes (#3750)"
This reverts commit a34e0d477e.
2023-07-28 22:44:45 +02:00
Nutomic a34e0d477e
Only run slow CI checks for actual code changes (#3750)
* Only run slow CI checks for actual code changes

Its a waste of time to compile Rust and run tests when only
something like the readme was changed.

* also make cache restore/rebuild conditional

* remove unused condition
2023-07-28 09:33:23 -04:00
Dessalines 417c4d2311
Adding diesel migration check. Fixes #3676 (#3677)
* Adding diesel migration check. Fixes #3676

* Fixing prettier.
2023-07-21 11:46:10 +02:00
Nutomic 5cd4c6c586
Dont publish releases to crates.io (fixes #3272) (#3664)
This is not needed anymore as from scratch installation now uses
`git clone` instead of `cargo install`.

https://github.com/LemmyNet/lemmy-docs/pull/241/files
2023-07-19 10:25:46 -04:00