Commit graph

278 commits

Author SHA1 Message Date
Nutomic a09027c4c0
Silence warnings from ts-rs (#4415)
* Silence warnings from ts-rs

https://github.com/Aleph-Alpha/ts-rs/issues/108

* fmt
2024-01-29 09:22:53 -05:00
Nutomic e8a52d3a5c
Rewrite images to use local proxy (#4035)
* Add markdown rule to add rel=nofollow for all links

* Add markdown image rule to add local image proxy (fixes #1036)

* comments

* rewrite markdown image links working

* add comment

* perform markdown image processing in api/apub receivers

* clippy

* add db table to validate proxied links

* rewrite link fields for avatar, banner etc

* sql fmt

* proxy links received over federation

* add config option

* undo post.url rewriting, move http route definition

* add tests

* proxy images through pictrs

* testing

* cleanup request.rs file

* more cleanup (fixes #2611)

* include url content type when sending post over apub (fixes #2611)

* store post url content type in db

* should be media_type

* get rid of cache_remote_thumbnails setting, instead automatically
take thumbnail from federation data if available.

* fix tests

* add setting disable_external_link_previews

* federate post url as image depending on mime type

* change setting again

* machete

* invert

* support custom emoji

* clippy

* update defaults

* add image proxy test, fix test

* fix test

* clippy

* revert accidental changes

* address review

* clippy

* Markdown link rule-dess (#4356)

* Extracting opengraph_data to its own type.

* A few additions for markdown-link-rule.

---------

Co-authored-by: Nutomic <me@nutomic.com>

* fix setting

* use enum for image proxy setting

* fix test configs

* add config backwards compat

* clippy

* machete

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 09:22:11 -05:00
Dessalines 1782aafd10
Upgrading deps. (#4401) 2024-01-25 10:24:07 +01:00
dullbananas d8f9e8a64c
Post view: move cursor pagination to separate library, add backward pagination to PostQuery (#4320)
* stuff

* stuff

* crates.io

* Update up.sql

* Rerun federation tests

* Update post_view.rs

* Update post_view.rs

* Update up.sql

* Update utils.rs

* Fix precision loss

* Update up.sql

* Update down.sql

* remove unwrap

* Update post_view.rs

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-24 10:50:11 -05:00
dullbananas 759f6d8a9a
Better query plan viewing experience (#4285)
* stuff

* stuff including batch_upsert function

* stuff

* do things

* stuff

* different timestamps

* stuff

* Revert changes to comment.rs

* Update comment.rs

* Update comment.rs

* Update post_view.rs

* Update utils.rs

* Update up.sql

* Update up.sql

* Update down.sql

* Update up.sql

* Update main.rs

* use anyhow macro

* replace get(0) with first()

* as_slice

* Update series.rs

* Update db_perf.sh

* Update and rename crates/db_schema/src/utils/series.rs to crates/db_perf/src/series.rs

* Update utils.rs

* Update main.rs

* Update main.rs

* Update .woodpecker.yml

* fmt main.rs

* Update .woodpecker.yml

* Instance::delete at end

* Update main.rs

* Update Cargo.toml

---------

Co-authored-by: Nutomic <me@nutomic.com>
2024-01-24 10:22:33 -05:00
Dessalines 2133bcea4e Version 0.19.3 2024-01-22 08:56:08 -05:00
Dessalines f652513030 Version 0.19.3-rc.1 2024-01-19 11:25:45 -05:00
Dessalines 0d35c247f9 Version 0.19.2 2024-01-10 10:18:09 -05:00
Nutomic 2d16d12cb7
Move publish = false to correct place in lemmy_server (fixes #4359) (#4361) 2024-01-10 09:07:09 -05:00
Dessalines ea0b856f1e Version 0.19.2-rc.5 2024-01-09 12:45:54 -05:00
Dessalines 2f09ad8e5b Version 0.19.2-rc.4 2024-01-09 09:25:52 -05:00
Nutomic e3b715002b
Handle federated reports from Mastodon, Kbin (#4323)
* Test Kbin/Mbin federation

* Handle reports from Mastodon/Kbin (fixes #4217)

* prettier

* revert

* add mastodon activity

* ci

* revert

* ci
2024-01-05 11:03:13 -05:00
Nutomic 4ca63c5641
Move apub context to join-lemmy.org (fixes #4234) (#4302)
* Move apub context to join-lemmy.org (fixes #4234)

* Dont store federation context in sent_activity table

* include basic activitypub context

* update lib

* ci
2024-01-05 15:42:46 +01:00
Felix Ableitner 346ff11795 Version 0.19.2-rc.2 2024-01-05 12:33:20 +01:00
Felix Ableitner d3efebfa4e Version 0.19.2-rc.1 2024-01-05 11:33:37 +01:00
Dessalines 4a740ee80a Version 0.19.1 2024-01-04 12:32:26 -05:00
ALEX11BR a5289dd4cf
Use pretty_assertions for assertions (#4347)
* Use `pretty_assertions` for assertions

* fixed ordering of `use`s

* ci

---------

Co-authored-by: Felix Ableitner <me@nutomic.com>
2024-01-04 04:47:18 -05:00
SleeplessOne1917 4ef00e068f
Prevent crates that shouldn't be published from being published (#4309)
* Prevent crates that shouldn't be published from being published

* Make dependent crates publishable

* Make dependent crates publishable

* fix toml formatting

* Removing publish=true

* Removing versioned deps.

---------

Co-authored-by: SleeplessOne1917 <insomnia-void@protonmail.com>
Co-authored-by: Dessalines <tyhou13@gmx.com>
2023-12-20 09:11:58 -05:00
Felix Ableitner 08b7e0f03d Version 0.19.1-rc.2 2023-12-20 10:32:51 +01:00
Dessalines 8583a85607 Version 0.19.1-rc.1 2023-12-18 14:20:42 -05:00
Dessalines aab3ca4eb0 Version 0.19.0 2023-12-15 06:53:26 -05:00
Dessalines 59eef85bb7 Version 0.19.0-rc.16 2023-12-14 07:59:27 -05:00
Dessalines 0b2df3980f Version 0.19.0-rc.15 2023-12-13 09:24:24 -05:00
Nutomic 60ffa2a599
Upgrade pictrs to 0.5.0-rc.2, remove cargo unstable flags (#4258) 2023-12-13 07:05:13 -05:00
Dessalines 5ad881f3df Version 0.19.0-rc.14 2023-12-12 14:58:17 -05:00
Nutomic c7ee53026f
Upgrade dependencies (#4248)
* Upgrade dependencies

* clippy
2023-12-12 08:56:33 -05:00
Dessalines c85e680aba Version 0.19.0-rc.13 2023-12-06 18:20:52 -05:00
Nutomic a5386187e3
Enable missing code for prometheus actix-web stats (#4230)
* Enable missing code for prometheus actix-web stats

* enable middleware conditionally
2023-12-04 09:53:53 -05:00
Dessalines 3f79eacb53 Version 0.19.0-rc.12 2023-12-01 09:29:44 -05:00
Dessalines 70003407a7 Version 0.19.0-rc.11 2023-11-30 05:02:18 -05:00
Dessalines 7ef6476520 Version 0.19.0-rc.10 2023-11-29 10:04:42 -05:00
Dessalines 23b266ec12
Upgrade deps, fix issue with wrong diesel-async in Cargo.lock (#4211)
Co-authored-by: Nutomic <me@nutomic.com>
2023-11-29 11:06:34 +01:00
dullbananas d95df3a46f
Update diesel-async (#4203)
* Update diesel-async

* Fix
2023-11-27 10:31:19 +01:00
Dessalines 7972dd0fcf Version 0.19.0-rc.8 2023-11-26 22:26:57 -05:00
Nutomic 7d489f1e3f
Upgrade pictrs to 0.5.0-beta.2 (#4194)
* Upgrade pictrs to 0.5.0-beta.2

* Update src/main.rs

Co-authored-by: asonix <asonix@asonix.dog>

* add comment

---------

Co-authored-by: asonix <asonix@asonix.dog>
2023-11-24 07:44:17 -05:00
Dessalines 2b5a31b411
Upgrading deps. (#4196)
* Upgrading deps.

* Try to use native node-fetch function.

* Fixing image upload tests.
2023-11-24 10:29:41 +01:00
Dessalines d795c54a7a Version 0.19.0-rc.7 2023-11-23 21:09:56 -05:00
Dessalines 58258414cc Version 0.19.0-rc.6 2023-11-22 09:59:04 -05:00
dullbananas 30f7135f32
Use rust 1.74.0 in CI and move clippy lints to Cargo.toml (#4170)
* ci rust 1.74.0

* do it

* Update .woodpecker.yml

* Update Cargo.toml

* Update lint.sh

* Update lib.rs

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-11-21 08:51:22 -05:00
cetra3 1c03a86c17
Upgrade activitypub to 0.5.0-beta.5 (#4179)
* Remove anyhow test

* Update lock

* Fix lints hopefully!
2023-11-21 10:26:12 +01:00
Dessalines 7cb20200d8 Version 0.19.0-rc.5 2023-11-16 23:30:09 -05:00
Dessalines adef3479c0 Version 0.19.0-rc.4 2023-11-03 10:02:30 -04:00
Nutomic 9007d49324
Remove unused deps (#4111)
* Remove unused deps

* Adding cargo-machete to woodpecker.

---------

Co-authored-by: Dessalines <tyhou13@gmx.com>
2023-10-31 11:11:12 +01:00
Nutomic 45bed71c36
Include prometheus in default build, remove build feature (fixes #3558) (#4071)
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-25 06:54:58 -04:00
Dessalines bd9739b5b7
Upgrading deps. (#4103) 2023-10-25 05:32:21 -04:00
Nutomic d55bd2f2bc
Allow Arabic and Cyrillic usernames/community names (fixes #1764) (#4083)
* Allow Arabic and Cyrillic usernames/community names (fixes #1764)

* update comment
2023-10-24 16:25:52 -04:00
Dessalines 6bcb12b14f Version 0.19.0-rc.3 2023-10-19 20:34:48 -04:00
dullbananas a14657d124
Refactor rate limiter and improve rate limit bucket cleanup (#3937)
* Update rate_limiter.rs

* Update mod.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update mod.rs

* Update scheduled_tasks.rs

* Shrink `RateLimitBucket`

* Update rate_limiter.rs

* Update mod.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update mod.rs

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* rerun ci

* Update rate_limiter.rs

* Undo changes to  fields

* Manually undo changes to RateLimitBucket fields

* fmt

* Bucket cleanup loop in rate_limit/mod.rs

* Remove rate limit bucket cleanup from scheduled_tasks.rs

* Remove ;

* Remove UNINITIALIZED_TOKEN_AMOUNT

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* stuff

* MapLevel trait

* fix merge

* Prevent negative numbers in buckets

* Clean up MapLevel::check

* MapLevel::remove_full_buckets

* stuff

* Use remove_full_buckets to avoid allocations

* stuff

* remove tx

* Remove RateLimitConfig

* Rename settings_updated_channel to rate_limit_cell

* Remove global rate limit cell

* impl Default for RateLimitCell

* bucket_configs doc comment to explain EnumMap

* improve test_rate_limiter

* rename default to with_test_config

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
2023-10-19 15:31:51 +02:00
Dessalines a675fecacd Version 0.19.0-rc.2 2023-10-17 14:13:44 -04:00
Dessalines 645bf21d54
Moving to debian-based docker image. Fixes #3972 (#4004)
* Moving to debian-based docker image. Fixes #3972

* Fix cargo.toml format.

* Try nightly rust for cargo_fmt.

* Trying to fix cargo build.

* Trying to fix cargo build 2.

* Trying to fix federation tests.

* Trying to fix federation tests 2.

* Trying to fix federation tests 3.

* Try to fix aarch 1.

* Try to fix aarch 2.

* Try to fix aarch 3.

* Removing aarch

* Adding tests back in.

* Adding ca-certs, and debian non-root user

* Commenting arm image building.

* Fix platform on runner.

* Version 0.19.0-beta.6
2023-10-11 11:57:05 -04:00
Nutomic dc327652a5
Add db table for login tokens which allows for invalidation (#3818)
* wip

* stuff

* fmt

* fmt 2

* fmt 3

* fix default feature

* use Authorization header

* store ip and user agent for each login

* add list_logins endpoint

* serde(skip) for token

* fix api tests

* A few suggestions for login_token (#3991)

* A few suggestions.

* Fixing SQL format.

* review

* review

* rename cookie

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-09 12:46:12 +02:00
Dessalines a5b8583aab Version 0.19.0-beta.7 2023-10-05 16:01:12 -04:00
Dessalines 9275041f42 Version 0.19.0-rc.1 2023-09-26 23:01:19 -04:00
SleeplessOne1917 13b20b0502 Address PR feedback 2023-09-25 06:37:45 -04:00
SleeplessOne1917 ed8dae99f1 Merge branch 'main' into bearer-token 2023-09-24 17:47:17 -04:00
SleeplessOne1917 519e800b33 feat: Replace ad hoc auth header with internet standard bearer token
auth header
2023-09-22 21:26:27 -04:00
Dessalines b16661a17d
Merge branch 'main' into comment-fix 2023-09-21 11:20:49 -04:00
Nutomic f858d8cbce
Remove explicit auth params (#3946)
* Remove explicit auth params (ref #3725)

Only take auth via header or cookie. This requires a new version
of lemmy-js-client for api tests to pass.

* rework api_crud

* remove remaining auth params, move logic to session middleware

* fmt, fix test

* update js client

* remove auth param from api tests

* Pass auth as header

* add !

* url vars, setHeader

* cleanup

* fmt

* update

* Updating for new lemmy-js-client.

---------

Co-authored-by: Dessalines <tyhou13@gmx.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-09-21 06:42:28 -04:00
SleeplessOne1917 553c5bda88 Update versions to correct version 2023-09-18 22:07:31 -04:00
phiresky 375d9a2a3c
Persistent, performant, reliable federation queue (#3605)
* persistent activity queue

* fixes

* fixes

* make federation workers function callable from outside

* log federation instances

* dead instance detection not needed here

* taplo fmt

* split federate bin/lib

* minor fix

* better logging

* log

* create struct to hold cancellable task for readability

* use boxfuture for readability

* reset submodule

* fix

* fix lint

* swap

* remove json column, use separate array columns instead

* some review comments

* make worker a struct for readability

* minor readability

* add local filter to community follower view

* remove separate lemmy_federate entry point

* fix remaining duration

* address review comments mostly

* fix lint

* upgrade actitypub-fed to simpler interface

* fix sql format

* increase delays a bit

* fixes after merge

* remove selectable

* fix instance selectable

* add comment

* start federation based on latest id at the time

* rename federate process args

* dead instances in one query

* filter follow+report activities by local

* remove synchronous federation

remove activity sender queue

* lint

* fix federation tests by waiting for results to change

* fix fed test

* fix comment report

* wait some more

* Apply suggestions from code review

Co-authored-by: SorteKanin <sortekanin@gmail.com>

* fix most remaining tests

* wait until private messages

* fix community tests

* fix community tests

* move arg parse

* use instance_id instead of domain in federation_queue_state table

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: SorteKanin <sortekanin@gmail.com>
2023-09-09 12:25:03 -04:00
SorteKanin 722027ce2e
Bump ts_rs version (#3947)
Co-authored-by: Nutomic <me@nutomic.com>
2023-09-06 13:04:24 -04:00
Nutomic fed6542055
Implement remote follow (#3738)
* Add remote follow url to webfinger response

* update apub lib
2023-08-31 09:00:41 -04:00
phiresky 514f2222e0
Fix time zone handling (#3496)
* convert naive time to utc time

* compounding fixes

* cargo fmt

* fix the rest

* fix down migration

* fix migrations

* fix after merge

* clippy fix

* ap-fed 0.5.0

---------

Co-authored-by: Nutomic <me@nutomic.com>
2023-08-24 11:27:00 -04:00
Felix Ableitner 91c024fd98 Revert "Version 0.18.4-rc.2"
This reverts commit 48466b6826.
2023-08-07 17:12:09 +02:00
Felix Ableitner 48466b6826 Version 0.18.4-rc.2 2023-08-07 17:09:10 +02:00
Nutomic 2d0f77af59
Dont use sha hash for password reset token (fixes #3491) (#3795) 2023-08-02 13:02:53 -04:00
dullbananas 9b123f45ec
Enable gzip for reqwest (#3696)
This will reduce bandwidth used by federation

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-07-26 12:22:14 +02:00
Nutomic 3753a3bf54
Upgrade activitypub library to 0.4.6 (fixes #3222) (#3675) 2023-07-20 11:33:19 -04:00
Dessalines 164f4b93d9
Make lemmy_api_common wasm-compatible (#3587)
* Fixing lemmy_api_common optionals

* Adding taplo for .toml files.

* Versioning taplo in woodpecker.

* Addressing PR comments.
2023-07-14 10:45:18 +02:00
Nutomic 39f28c1a40
Add dev profile to strip symbols and disable debug info (ref #3610) (#3611)
* Add dev profile to strip symbols and disable debug info (ref #3610)

This significantly speeds up builds:
- with strip symbols and debug 0: 43s
- without: 169s

* add comment, no strip symbols
2023-07-13 14:50:17 -04:00
Nutomic ff26bc21af
Upgrade all dependencies (#3526)
* Upgrade all dependencies

* as base64
2023-07-10 15:00:55 +02:00
Dessalines 0c82f4e660 Version 0.18.1 2023-07-07 09:24:49 -04:00
Felix Ableitner 22cdcc6b60 Version 0.18.1-rc.10 2023-07-06 14:46:46 +02:00
Andrew Fields 1e99e8b9dc
Add Prometheus endpoint (#3456)
Add a server for serving Prometheus metrics. Include a configuration
block in the config file. Provide HTTP metrics on the API, along with
process-level metrics and DB pool metrics.
2023-07-05 13:25:19 +02:00
Lemmus.org ff47d97bd3
Add feature for enabling json logging (#3462) 2023-07-04 07:11:47 -04:00
Dessalines 7a04971ecd Version 0.18.1-rc.9 2023-07-03 17:16:48 -04:00
phiresky cb28af508d
don't strip, log trace if requested (#3425) 2023-06-30 09:32:43 -04:00
Dessalines 3159eedd99 Version 0.18.1-rc.4 2023-06-29 10:45:59 -04:00
Dessalines 6c64cb5233
Fixing release script. (#3398)
* Fixing release script.

* Version 0.18.1-rc.2

* Removing cargo update from release script.

* Fixing topdir location.
2023-06-29 16:17:59 +02:00
Dessalines ad6f244b61 Version 0.18.1-rc.1 2023-06-27 08:14:46 -04:00
Sander Saarend 211e76dc27
Batch hot rank updates (#3175) 2023-06-27 10:13:51 +02:00
dullbananas b2a9d4a335
Use compression middleware (#3343) 2023-06-26 12:54:41 +02:00
Felix Ableitner a39e948b52 Remove unused actix_rt dependency 2023-06-26 12:14:47 +02:00
Sander Saarend 6d67f88603
Add support for sslmode=require for diesel-async DB connections (#3189) 2023-06-26 10:25:38 +02:00
cetra3 d7da911a48
Remove actix_rt & use standard tokio spawn (#3158)
* Remove `actix_rt` & use standard tokio spawn

* Adjust rust log back down

* Format correctly

* Update cargo lock

* Add DB settings

* Change name and update to latest rev

* Clean up formatting changes

* Move `worker_count` and `worker_retry_count` to settings

* Update defaults

* Use `0.4.4` instead of git branch
2023-06-26 10:24:11 +02:00
Dessalines 63d3759c48 Version 0.18.0 2023-06-23 08:41:20 -04:00
Dessalines cb3b9b1a66 Version 0.18.0-rc.8 2023-06-23 07:07:51 -04:00
Dessalines 60baebcb14 Version 0.18.0-rc.6 2023-06-22 08:55:08 -04:00
Nutomic ffce66abb8
Update apub library to 0.4.4 (#3258)
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-06-22 08:35:12 -04:00
Dessalines 289052f261 Version 0.18.0-rc.5 2023-06-21 16:02:42 -04:00
Dessalines 3775418952 Version 0.18.0-rc.4 2023-06-21 08:05:15 -04:00
Nutomic b5dba17426
Update all dependencies (#3091) 2023-06-15 11:37:05 +02:00
Nutomic 25275b79ee
Upgrade activitypub-federation to 0.4.1 (#3039)
This improves logging so that its not necessary to enable debug
logs to find out that activity worker count needs to be increased.
2023-06-12 14:12:15 -04:00
Charles Hall f50325e78a
update doku (#3003)
Fixes malformed documentation generated by previous version.
2023-06-12 05:55:47 -04:00
Carlrs 24323e17b2
update diesel to 2.1.0 and diesel-async to 0.3.0 (issue #2882) (#2895)
* update diesel to 2.1.0 and diesel-async to 0.3.0 (issue #2882)

* get rid of deprecated RawValue type alias usage, explicitly import diesel types instead of using wildcard import

* Update diesel-derive-enum and diesel-derive-newtype libraries

* Revert manual changes to schema.rs

* Update to newer diesel-cli image in CI, run fix-clippy before schema comparison, update schema to one generated with diesel-cli 2.1.0 and clippyfixed

* Remove wildcard import lint, fix diesel schema pipeline

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-06-07 12:54:40 +02:00
Nutomic 3565ad984a
Remove chatserver (#2919)
* Remove chatserver

* fix clippy

* Remove captchas (fixes #2922)

* fix prettier

* fix api_common build

* cargo fmt
2023-06-06 12:27:22 -04:00
Dessalines af3eb6e27b
Add CORS support for debug mode. (#2884) 2023-05-29 17:14:00 -04:00
Nutomic 4240af86ca
Remove websocket code (#2878)
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-05-29 10:44:20 -04:00
Felix Ableitner df57795595 Dont retry outgoing HTTP requests (ref #2865) 2023-05-18 16:34:52 +02:00
Dessalines 88a0d2feec
Adding typescript generation for API. Fixes #2824 (#2827)
* Adding typescript generation for API. Fixes #2824

* Try to fix Ltree issue 1.

* Forgot a few types.

* Fixing api tests.

* Removing url_serde line.

* Manually deriving TS for some types.
2023-04-26 00:26:10 -04:00