Commit graph

111 commits

Author SHA1 Message Date
Nutomic e8a52d3a5c
Rewrite images to use local proxy (#4035)
* Add markdown rule to add rel=nofollow for all links

* Add markdown image rule to add local image proxy (fixes #1036)

* comments

* rewrite markdown image links working

* add comment

* perform markdown image processing in api/apub receivers

* clippy

* add db table to validate proxied links

* rewrite link fields for avatar, banner etc

* sql fmt

* proxy links received over federation

* add config option

* undo post.url rewriting, move http route definition

* add tests

* proxy images through pictrs

* testing

* cleanup request.rs file

* more cleanup (fixes #2611)

* include url content type when sending post over apub (fixes #2611)

* store post url content type in db

* should be media_type

* get rid of cache_remote_thumbnails setting, instead automatically
take thumbnail from federation data if available.

* fix tests

* add setting disable_external_link_previews

* federate post url as image depending on mime type

* change setting again

* machete

* invert

* support custom emoji

* clippy

* update defaults

* add image proxy test, fix test

* fix test

* clippy

* revert accidental changes

* address review

* clippy

* Markdown link rule-dess (#4356)

* Extracting opengraph_data to its own type.

* A few additions for markdown-link-rule.

---------

Co-authored-by: Nutomic <me@nutomic.com>

* fix setting

* use enum for image proxy setting

* fix test configs

* add config backwards compat

* clippy

* machete

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 09:22:11 -05:00
Nutomic e3b715002b
Handle federated reports from Mastodon, Kbin (#4323)
* Test Kbin/Mbin federation

* Handle reports from Mastodon/Kbin (fixes #4217)

* prettier

* revert

* add mastodon activity

* ci

* revert

* ci
2024-01-05 11:03:13 -05:00
ALEX11BR a5289dd4cf
Use pretty_assertions for assertions (#4347)
* Use `pretty_assertions` for assertions

* fixed ordering of `use`s

* ci

---------

Co-authored-by: Felix Ableitner <me@nutomic.com>
2024-01-04 04:47:18 -05:00
Nutomic 4a55d4f871
Quick fix for Peertube federation (fixes #4261) (#4264)
* Quick fix for Peertube federation (fixes #4261)

* prettier

* apub assets should be included in slow check paths
2023-12-14 07:25:47 -05:00
Nutomic fc56d0aa05
Use result instead of unwrap in apub tests (#4168) 2023-11-16 22:51:33 -05:00
Nutomic a00313e680
Merge /site_inbox into /inbox, remove unique constraint for inboxes (#4138)
* Merge /site_inbox into /inbox (fixes #4137)

Get rid of different inboxes, only use /inbox

Remove shared_inbox_url db columns

add code migration

move to db migration, fixes

machete

fix sql

drop inbox url unique constraints

Dont create auth cookie in backend (#4136)

dont change individual inboxes to shared inbox

Dont send comment reply to user who has community blocked. Fixes #3684 (#4096)

* Dont send comment reply to user who has community blocked. Fixes #3684

* Adding source instance block check.

* Adding api test.

* Addressing PR comments.

* move site inbox rewrite to db

* fix test

* clippy

* clippy 2

* fix test
2023-11-16 14:22:40 +01:00
Nutomic 45bed71c36
Include prometheus in default build, remove build feature (fixes #3558) (#4071)
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-25 06:54:58 -04:00
Nutomic 56322c75f0
Read community follower count from home instance (fixes #1440) (#4013)
* Read community follower count from home instance (fixes #1440)

* fmt

* prettier

* fix tests

* fmt

* rename fn

* fmt

* Run prettier

* increase timeout

* ci

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Dessalines <tyhou13@gmx.com>
2023-10-17 10:34:38 -04:00
Nutomic 291ff19718
Only sanitize strings when generating RSS feeds and emails (fixes #4003) (#4024)
* Only sanitize strings when generating RSS feeds and emails (fixes #4003)

* clippy

* fix test
2023-10-11 16:48:19 +02:00
Nutomic 71d61138bc
Replace ammonia lib with manual html escaping (fixes #3774) (#3938)
* Replace ammonia lib with manual html escaping (fixes #3774)

* prettier

* clippy

* remove sanitize unit test

* fix tests

* fix schema
2023-09-06 10:56:26 -04:00
Dessalines 56e26fc3d4
Adding a post_view mode. Fixes #3730 (#3731)
* Adding a post_view mode. Fixes #3730

* Fix test.

* Addressing PR comments.

* Adding a post_view mode. Fixes #3730

* Fix test.

* Addressing PR comments.

* Fixing column order.

* Fix default Ok returns.

* Removing return Err(... where feasible.
2023-08-31 09:01:08 -04:00
Nutomic 7fd14b3d2a
Make remove content optional during account deletion (fixes #1617) (#3817)
* Make remove content optional during account deletion (fixes #1617)

* simplify purge params by passing context

* update js client

* add delete content

* update woodpecker config

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-08-28 12:23:45 +02:00
phiresky 514f2222e0
Fix time zone handling (#3496)
* convert naive time to utc time

* compounding fixes

* cargo fmt

* fix the rest

* fix down migration

* fix migrations

* fix after merge

* clippy fix

* ap-fed 0.5.0

---------

Co-authored-by: Nutomic <me@nutomic.com>
2023-08-24 11:27:00 -04:00
Nutomic 3471f3533c
Sanitize html (#3708)
* HTML sanitization in apub code

* Sanitize API inputs

* fmt

* Dont allow html a, img tags

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-07-26 14:01:15 -04:00
cetra3 9256895635
Cache & Optimize Woodpecker CI (#3450)
* Try using drone cache plugin

* Try another path

* Include volume

* Fix formatting

* Include fmt

* Exclude cargo dir from prettier

* Don't override cargo

* Just do check

* Add cache key

* Use different cache plugin

* Add clippy

* Try minio

* Add quotes

* Try adding secrets

* Try again

* Again

* Use correct secret formation

* Add back clippy

* Use secret for the root bucket name

* Try drone cache instead

* Add region

* Add path-style option

* Include cargo clippy

* Include everything again

* Fix formatting

* Don't run clippy twice

* Add `allow` statements for tests to pass

* Adjust endpoint to be a secret

* Fix prettier

* Merge & fix tests

* Try to restart the woodpecker test

* Change the ENV var name

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-07-17 11:04:14 -04:00
dullbananas 1d38aad9d3
Make functions work with both connection and pool (#3420)
* a lot

* merge

* Fix stuff broken by merge

* Get rid of repetitive `&mut *context.conn().await?`

* Add blank lines under each line with `conn =`

* Fix style mistakes (partial)

* Revert "Fix style mistakes (partial)"

This reverts commit 48a033b87f.

* Revert "Add blank lines under each line with `conn =`"

This reverts commit 773a6d3beb.

* Revert "Get rid of repetitive `&mut *context.conn().await?`"

This reverts commit d2c6263ea1.

* Use DbConn for CaptchaAnswer methods

* DbConn trait

* Remove more `&mut *`

* Fix stuff

* Re-run CI

* try to make ci start

* fix

* fix

* Fix api_common::utils

* Fix apub::activities::block

* Fix apub::api::resolve_object

* Fix some things

* Revert "Fix some things"

This reverts commit 2bf8574bc8.

* Revert "Fix apub::api::resolve_object"

This reverts commit 3e4059aabb.

* Revert "Fix apub::activities::block"

This reverts commit 3b02389abd.

* Revert "Fix api_common::utils"

This reverts commit 7dc73de613.

* Revert "Revert "Fix api_common::utils""

This reverts commit f740f115e5.

* Revert "Revert "Fix apub::activities::block""

This reverts commit 2ee206af7c.

* Revert "Revert "Fix apub::api::resolve_object""

This reverts commit 96ed8bf2e9.

* Fix fetch_local_site_data

* Fix get_comment_parent_creator

* Remove unused perma deleted text

* Fix routes::feeds

* Fix lib.rs

* Update lib.rs

* rerun ci

* Attempt to create custom GetConn and RunQueryDsl traits

* Start over

* Add GetConn trait

* aaaa

* Revert "aaaa"

This reverts commit acc9ca1aed.

* Revert "Revert "aaaa""

This reverts commit 443a2a00a5.

* still aaaaaaaaaaaaa

* Return to earlier thing

Revert "Add GetConn trait"

This reverts commit ab4e94aea5.

* Try to use DbPool enum

* Revert "Try to use DbPool enum"

This reverts commit e4d1712646.

* DbConn and DbPool enums (db_schema only fails to compile for tests)

* fmt

* Make functions take `&mut DbPool<'_>` and make db_schema tests compile

* Add try_join_with_pool macro and run fix-clippy on more crates

* Fix some errors

* I did it

* Remove function variants that take connection

* rerun ci

* rerun ci

* rerun ci
2023-07-11 09:09:59 -04:00
Nutomic 93225e5ddf
Error enum fixed (#3487)
* Create error type enum

* Replace magic string slices with LemmyErrorTypes

* Remove unused enum

* Add rename snake case to error enum

* Rename functions

* clippy

* Fix merge errors

* Serialize in PascalCase instead of snake_case

* Revert src/lib

* Add serialization tests

* Update translations

* Fix compilation error in test

* Fix another compilation error

* Add code for generating typescript types

* Various fixes to avoid breaking api

* impl From<LemmyErrorType> for LemmyError

* with_lemmy_type

* trigger ci

---------

Co-authored-by: SleeplessOne1917 <abias1122@gmail.com>
2023-07-10 16:50:07 +02:00
phiresky 45b1a0d4fb
improve performance of community followers inbox query (#3482)
* improve performance of community followers inbox query

* nightly format

* force woodpecker to retry

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-07-05 11:50:26 -04:00
Nutomic ebaf69bd70
Cache federation blocklist (#3486)
* Cache federation blocklist

* revert submodule change
2023-07-05 11:08:02 -04:00
Alex Maras becf75d1f9
Fix Hidden communities showing in community list (#3094)
* Only show hidden communities when explicitly searching for them rather then in "all"

* dont set hidden to false when creating and updating - let DB set default

* lint

---------

Co-authored-by: Alex Maras <alexmaras@gmail.com>
2023-06-15 13:39:58 +02:00
Neshura 558035b128
Fix Issue #3075 by using None for local fields instead of Some(false) (#3088)
* Set insert and update form "local" to None

* Extra space to retrigger woodpecker

* Resolve code review request

https://github.com/LemmyNet/lemmy/pull/3088#discussion_r1229336958

* Remove extra space after None,
2023-06-14 08:34:45 -04:00
Nutomic 2423b89ced
Activitypub breaking changes for Lemmy 0.18 (#2705)
* Remove deprecated field `group.moderators`

* Drop column local_site.federation_debug

* Remove backwards compatibility code for feature/lock post

https://github.com/LemmyNet/lemmy/pull/2716

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-06-07 15:18:17 -04:00
vpzomtrrfrt 09099e7b90
Fix handling of follows addressed to single value (#2920)
* Fix handling of follows addressed to single value

* Switch to deserialize_skip_error for Follow to

* Also use deserialize_skip_error for AcceptFollow and UndoFollow

* actually import deserialize_skip_error
2023-06-06 18:33:38 -04:00
Nutomic a25faf3e97
Add to field in follow activities for better compatibility (#2829)
* Add `to` field in follow activities for better compatibility (fixes #2744)

* fix tests
2023-04-24 17:19:08 -04:00
Nutomic 6f513793cb
Activitypub crate rewrite (#2782)
* update activitypub-federation crate to 0.4.0

* fixes

* apub compiles!

* everything compiling!

* almost done, federated follow failing

* some test fixes

* use release

* add code back in
2023-03-21 16:03:05 +01:00
Nutomic f0e1627824
Ignore undetermined language when calculating post default language (#2769)
This will make it much more likely to successfully get a default,
as most users and communities have undetermined selected.
2023-03-01 01:49:31 +01:00
Nutomic 62663a9f2e
Implement separate mod activities for feature, lock post (#2716)
* Implement separate mod activities for feature, lock post

Also includes collection for featured posts. Later we also need
to do the same for Comment.distinguished

* some changes

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-02-18 09:50:28 -05:00
Dessalines 36aaa50644
Organize utils into separate files. Fixes #2295 (#2736)
* Organize utils into separate files. Fixes #2295

* Moving tests.

* Fix test.

* Fix test 2
2023-02-16 05:05:14 +01:00
Nutomic eb78af9b02
Fix data fetching from Peertube (fixes #2689) (#2703)
- Other platforms can support additional language tags. Treat those
as None instead of throwing error
- deserialize_skip_error was implemented incorrectly and failed on
array values (6d9682f4e6)

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-02-05 00:38:08 -05:00
Nutomic 7e3d3839b6
Post creation from Mastodon (fixes #2590) (#2651)
* Post creation from Mastodon (fixes #2590)

* better logic for page title

* add deserialize helper

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-01-20 12:43:23 -05:00
Anon 9dfd819691
Add support for Featured Posts (#2585)
* Add support for Featured Posts

* Fix rebase

* More fixes
2022-12-12 11:17:10 +00:00
Nutomic 0ecf256ce3
Remove federation backwards compatibility with 0.16.x (#2183)
* Breaking: remove compatiblity with page.url field (ref #2182)

* Breaking: change type of Instance to `Application` (ref #2200)

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2022-12-09 11:21:17 -05:00
Felix Ableitner c6c52ab9cc Add SendActivity trait so that api crates compile in parallel with lemmy_apub 2022-12-02 10:46:49 +01:00
Felix Ableitner 201fa97769 Move code to generate apub urls into lemmy_api_common 2022-12-02 10:46:49 +01:00
Felix Ableitner 030afbc2e7 Merge websocket crate into api_common 2022-12-02 10:46:49 +01:00
Nutomic bcf5c91f81
Use audience field to federate items in groups (fixes #2464) (#2584) 2022-12-01 15:52:49 -05:00
Nutomic 41d4852efc
Federate group moderators using attributedTo field (#2588)
* Federate group moderators using attributedTo field

* fix tests

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2022-11-25 09:01:58 -05:00
Nutomic d39bc14c43
Add some more clippy lints (#2586)
* Add some more clippy lints

* restore config

* no allow dirty

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2022-11-24 16:38:00 +00:00
Nutomic d20d2b9218
Implement federated user following (fixes #752) (#2577)
* Implement federated user following (fixes #752)

* rewrite send_activity_in_community and add docs, remove default for column pending

* improve migration

* replace null values in db migration
2022-11-23 18:40:47 -05:00
Nutomic 6f3bf4634b
Various pedantic clippy fixes (#2568)
* Various pedantic clippy fixes

* more clippy pedantic fixes

* try to fix ci

* add fix clippy script, use rust 1.65

* fix clippy
2022-11-19 04:33:54 +00:00
Nutomic a0fed24cee
When announcing incoming activities, keep extra fields (#2550) 2022-11-12 08:52:57 -05:00
Nutomic e3bb43542c
Mobilizon federation (#2544)
* Mobilizon federation

* Also accept document attachments from mobilizon
2022-11-12 08:51:08 -05:00
Dessalines 5d837780f5
Add diesel_async, get rid of blocking function (#2510)
* Moving settings to Database.

- Moves many settings into the database. Fixes #2285
- Adds a local_site and instance table. Fixes #2365 . Fixes #2368
- Separates SQL update an insert forms, to avoid runtime errors.
- Adds TypedBuilder to all the SQL forms, instead of default.

* Fix weird clippy issue.

* Removing extra lines.

* Some fixes from suggestions.

* Fixing apub tests.

* Using instance creation helper function.

* Move forms to their own line.

* Trying to fix local_site_data, still broken.

* Testing out async

* Testing out async 2

* Fixing federation tests.

* Trying to fix check features 1.

* Starting on adding diesel async. 1/4th done.

* Added async to views and schema.

* Adding some more async

* Compiling now.

* Added diesel async. Fixes #2465

* Running clippy --fix

* Trying to fix cargo test on drone.

* Trying new muslrust.

* Trying a custom dns

* Trying a custom dns 2

* Trying a custom dns 3

* Trying a custom dns 4

* Trying a custom dns 5

* Trying a custom dns 6

* Trying a custom dns 7

* Addressing PR comments.

* Adding check_apub to all verify functions.

* Reverting back drone.

* Fixing merge

* Fix docker images.

* Adding missing discussion_languages.

* Trying to fix federation tests.

* Fix site setup user creation.

* Fix clippy

* Fix clippy 2

* Test api faster

* Try to fix 1

* Try to fix 2

* What are these lines about

* Trying to fix 3

* Moving federation test back to top.

* Remove logging cat.
2022-11-09 10:05:00 +00:00
Nutomic b5cd732372
Mark objects as not deleted when received via apub (fixes #2507) (#2528) 2022-11-03 09:39:30 -04:00
Dessalines 235cc8b228
Moving settings to Database. (#2492)
* Moving settings to Database.

- Moves many settings into the database. Fixes #2285
- Adds a local_site and instance table. Fixes #2365 . Fixes #2368
- Separates SQL update an insert forms, to avoid runtime errors.
- Adds TypedBuilder to all the SQL forms, instead of default.

* Fix weird clippy issue.

* Removing extra lines.

* Some fixes from suggestions.

* Fixing apub tests.

* Using instance creation helper function.

* Move forms to their own line.

* Trying to fix local_site_data, still broken.

* Fixing federation tests.

* Trying to fix check features 1.

* Addressing PR comments.

* Adding check_apub to all verify functions.
2022-10-27 09:24:07 +00:00
Nutomic aaaea2006c
Federate votes as private (#2501) 2022-10-18 03:13:18 +00:00
Nutomic 2ef0f8f5f8
implement language tags for site/community in db and api (#2434)
* implement language tags for site/community in db and api

* add api checks for valid languages

* during db migration, update existing users, sites, communities to have all languages enabled

* init new users/communities with site languages (not all languages)

* federate site/community languages

* fix tests

* when updating site languages, limit community languages to this subset

also, when making a new post and subset of user lang, community lang
contains only one item, use that as post lang

* add tests for actor_language db functions

* include language list in siteview/communityview

* Fix some of the review comments

* Some more review changes

* Add todo about boxed query

* Add default_post_language to GetCommunityResponse
2022-10-06 14:27:58 -04:00
Dessalines c9f1407429
Diesel 2.0.0 upgrade (#2452)
* Initial commit to bump diesel to 2.0.0-rc.0 and see what happens

* Add chrono feature from diesel

* db_schema crate is close to building?

* Upgrade diesel-derive-newtype

* Mostly modifying references to connections to be mutable ones; also used
new way to do migrations as suggested by the migration guide; a lot more
compiles now, though I can't figure out this tricky ToSql issue at the
moment

* Running clippy --fix

* Trying to fix drone clippy 1

* Fix clippy

* Upgrade clux-musl

* Trying to fix drone clippy 2

* Trying to fix drone clippy 3

* Trying to fix drone clippy 5

* Adding diesel table aliases, removing sql view hack. Fixes #2101

Co-authored-by: Steven Chu <stevenc1@gmail.com>
Co-authored-by: Nutomic <me@nutomic.com>
2022-09-26 14:09:32 +00:00
Felix Ableitner 3d0d8796ad Add language tags for comments 2022-09-02 10:49:54 -04:00
Nutomic e4a49b6eab
Tag posts and comments with language (fixes #440) (#2269)
* Tag posts and comments with language (fixes #440)

* Untangle PostView tests

* Implement test for PostView language query

* Store languages directly in database

* finish moving languages into db, it compiles

* update post_view

* serde skip Language.id field

* add local_user_language table, other changes suggested in review

* add code for local_user_discussion_language_view

* Remove unnecessary clones in db view converteres

* Fixing up some table and join issues.

* Clearing the current languages.

* Fix formatting.

* update user languages in single transaction

* proper test for user language queries

* Some fixes for all / missing user languages. (#2404)

* Some fixes for all / missing user languages.

* Adding back in transaction.

* fix test

Co-authored-by: Dessalines <tyhou13@gmx.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2022-08-18 15:11:19 -04:00