* When banning a federated user, also remove their content from our local
communities.
- This works by:
- Before a site ban, find all posts and comments to local communities
- Send a federated community ban action for each local comm.
- This also removes their content in the apub receive code.
- Adding back in federated community ban api tests.
- Adding in two more api tests for site bans.
- Fixes#4118
* Add local community ban, and nonlocal person check.
* Ignoring errors.
* Move local check into function.
* Addressing PR comments 2
* Add support for local only community (fixes#1576)
* add filters and tests to db views
* dont federate local only community
* test get apub community http
* tests
* more checks
* wip
* api test
* fix tests
* change community.local_only column to visibility enum
(for private communities)
* sql fmt
* rename vars
* clippy
* fix tests
* update lib
* review
* fix js client version
* update client
* Add markdown rule to add rel=nofollow for all links
* Add markdown image rule to add local image proxy (fixes#1036)
* comments
* rewrite markdown image links working
* add comment
* perform markdown image processing in api/apub receivers
* clippy
* add db table to validate proxied links
* rewrite link fields for avatar, banner etc
* sql fmt
* proxy links received over federation
* add config option
* undo post.url rewriting, move http route definition
* add tests
* proxy images through pictrs
* testing
* cleanup request.rs file
* more cleanup (fixes#2611)
* include url content type when sending post over apub (fixes#2611)
* store post url content type in db
* should be media_type
* get rid of cache_remote_thumbnails setting, instead automatically
take thumbnail from federation data if available.
* fix tests
* add setting disable_external_link_previews
* federate post url as image depending on mime type
* change setting again
* machete
* invert
* support custom emoji
* clippy
* update defaults
* add image proxy test, fix test
* fix test
* clippy
* revert accidental changes
* address review
* clippy
* Markdown link rule-dess (#4356)
* Extracting opengraph_data to its own type.
* A few additions for markdown-link-rule.
---------
Co-authored-by: Nutomic <me@nutomic.com>
* fix setting
* use enum for image proxy setting
* fix test configs
* add config backwards compat
* clippy
* machete
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Add upload timeout to PictrsConfig
* Bad space 🤔
* Update PictrsConfig upload timeout to include units.
* Add local_subscribers field to CommunityAggregates
struct and schema
* sql format
* local_subscribers test
* fix local_subscribers test
* Revert "fix local_subscribers test"
This reverts commit 4bbac5ce4a.
* Revert "local_subscribers test"
This reverts commit 735107e1f7.
* Create trigger for local_subscribers
* Rename variable
* re-trigger ci
* re-trigger ci
* Add local_subscribers count to follow.spec.ts
* Rename local_subscribers to subscribers_local
* Add subscribers_local to community_aggregates
* added subscribers_local to the aggregate tests
* Check if person exists on community_follower trigger
* Delete community follows before deleting person
* Update lemmy-js-client in api_tests
* Refactor local_subscriber migration
* fix format
* Move migration files date to now
* Fix test to wait for aggregates to federate
* re-trigger ci
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Fix fetching of community posts (fixes#4283)
Also use spawn_try_task to fetch community outbox, mods etc to avoid
delay/timeout when fetching new community.
* prettier
* fix test
* fix api test
* prettier
* add delay
* Update run-federation-test.sh
* fix test
* Merge /site_inbox into /inbox (fixes#4137)
Get rid of different inboxes, only use /inbox
Remove shared_inbox_url db columns
add code migration
move to db migration, fixes
machete
fix sql
drop inbox url unique constraints
Dont create auth cookie in backend (#4136)
dont change individual inboxes to shared inbox
Dont send comment reply to user who has community blocked. Fixes#3684 (#4096)
* Dont send comment reply to user who has community blocked. Fixes#3684
* Adding source instance block check.
* Adding api test.
* Addressing PR comments.
* move site inbox rewrite to db
* fix test
* clippy
* clippy 2
* fix test
* Dont send comment reply to user who has community blocked. Fixes#3684
* Adding source instance block check.
* Adding api test.
* Addressing PR comments.
* Fixing problem with SaveUserSettings, when either the Person or LocalUser update is null.
- Fixes#4076
- Also upgrading api_test deps
* Move function into test.
* Revert "Convert more responses to SuccessResponse. Fixes#2860 (#4058)"
This reverts commit 8deb4e5752.
* Removing purgeitem and deletecustomemoji response.
* Adding back in utils building.
* Ignore activities in remote communities without local followers (fixes#3568)
* x
* comments
* prettier
* fix api test
* fix test
* cleanup
* fix remaining test
* clippy
* decrease delay
* wip
* stuff
* fmt
* fmt 2
* fmt 3
* fix default feature
* use Authorization header
* store ip and user agent for each login
* add list_logins endpoint
* serde(skip) for token
* fix api tests
* A few suggestions for login_token (#3991)
* A few suggestions.
* Fixing SQL format.
* review
* review
* rename cookie
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Remove explicit auth params (ref #3725)
Only take auth via header or cookie. This requires a new version
of lemmy-js-client for api tests to pass.
* rework api_crud
* remove remaining auth params, move logic to session middleware
* fmt, fix test
* update js client
* remove auth param from api tests
* Pass auth as header
* add !
* url vars, setHeader
* cleanup
* fmt
* update
* Updating for new lemmy-js-client.
---------
Co-authored-by: Dessalines <tyhou13@gmx.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* User can block instances (fixes#2397)
* update comments
* review comments
* use route
* update
* add api test
* update tests
* fix
* fix test
* ci
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* persistent activity queue
* fixes
* fixes
* make federation workers function callable from outside
* log federation instances
* dead instance detection not needed here
* taplo fmt
* split federate bin/lib
* minor fix
* better logging
* log
* create struct to hold cancellable task for readability
* use boxfuture for readability
* reset submodule
* fix
* fix lint
* swap
* remove json column, use separate array columns instead
* some review comments
* make worker a struct for readability
* minor readability
* add local filter to community follower view
* remove separate lemmy_federate entry point
* fix remaining duration
* address review comments mostly
* fix lint
* upgrade actitypub-fed to simpler interface
* fix sql format
* increase delays a bit
* fixes after merge
* remove selectable
* fix instance selectable
* add comment
* start federation based on latest id at the time
* rename federate process args
* dead instances in one query
* filter follow+report activities by local
* remove synchronous federation
remove activity sender queue
* lint
* fix federation tests by waiting for results to change
* fix fed test
* fix comment report
* wait some more
* Apply suggestions from code review
Co-authored-by: SorteKanin <sortekanin@gmail.com>
* fix most remaining tests
* wait until private messages
* fix community tests
* fix community tests
* move arg parse
* use instance_id instead of domain in federation_queue_state table
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: SorteKanin <sortekanin@gmail.com>
* update api tests for new moderator view
* chage moderator view to be a listing type in get posts
Note: Internally, the listing type is called ListingType.ModeratorView,
but it's called "Moderator View" in the api endpoint
* fix formatting
* add support for moderator view to list comments
* add api test for moderator view when listing comments
* fix api test formatting
* retry tests
* don't filter out blocked users and communities when using moderator view
* fix cargo tests failing
* fix formatting
* fix previous merge
* Adding ModeratorView to listing_type_enums
* Fixing fmt.
* Adding a default to ListingType.
* Upgrading to use new lemmy-js-client.
---------
Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Dessalines <tyhou13@gmx.com>
* Lowercase domain on db query filters (#3849)
* Add test to get a community on different cased domain (#3849)
* Lowercase the identity for webfinger (#3849)
* Lowercase both sides of the domain comparison (#3849)
* Format api_tests (#3849)
* Lowercase domain lookup on Instance and Person (#3849)
---------
Co-authored-by: Freek van Zee <freek.van.zee@mediamonks.com>
Co-authored-by: Freakazoid182 <>
* shared.ts first test of getReplies
* comment testing now validates reply notifications and mentions, some code comment cleanup in other functions
* comments revised
* first use of getUnreadCount in testing
* test notification of new comment replies, clarify usage of getReplies
* killall moved earlier in bash script
* api-test jest run does not need directory prefix, make consistent with other jest runs
* do not put my testing system password into script
* fix, killall exits script when no process found
* killall now moved to parent script to release locks before database create
* need to run killall a second time, before database drop
* first use of getReplies getPosts saveUserSettings
* accidental duplication of functions, removed
* try to sync shared library with main
* Nutomic feedback: Better to rename the var instead of putting a comment which can easily get outdated.
* Correct logic to meet join-lemmy requirement, don't have closed signups. Allows Open and Applications. (#3761)
Co-authored-by: Josh Bernardini <josh.bernardini@cologix.com>
* Fix fetch instance software version from nodeinfo (#3772)
Fixes#3771
* remove unused code, revert killall change
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: figure-0e <133478007+figure-0e@users.noreply.github.com>
Co-authored-by: Josh Bernardini <josh.bernardini@cologix.com>
Co-authored-by: Denis Dzyubenko <denis@ddenis.info>
Co-authored-by: Felix Ableitner <me@nutomic.com>
* more robust test of unlike a comment, confirm replication to instance downstream from community home
* more robust 'delete a comment' test, confirm replication
* Far more robust "Report a comment" test. Many comments about situation, this is currently failing because gamma does not get the report
* typo and actually have Gamma comment check use gamma, not alpha
* prepare-drone-federation-test.sh has some more echo output and note about the LEMMY_DATABASE_URL format (#3651)
* Add http cache for webfingers (#3317)
* Add http cache for webfingers
* Remove the outgoing cache middleware & adjust the cache headers directive
* Use 1h & 3day cache header
* Update routes and adjust the cache headers location
* revert apub caching
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Felix Ableitner <me@nutomic.com>
* Rewrite activity lists to fix delete federation (fixes#3625)
* Revert "typo and actually have Gamma comment check use gamma, not alpha"
This reverts commit 7dfb6ee0f4.
* Revert "Far more robust "Report a comment" test. Many comments about situation, this is currently failing because gamma does not get the report"
This reverts commit 7bd3b20ae0.
* prettier TypeScript
* revised comments, as ResolveObject isn't using routine replication
* fmt
* fix api tests
* remove comment
---------
Co-authored-by: cetra3 <cetra3@hotmail.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Felix Ableitner <me@nutomic.com>
* HTML sanitization in apub code
* Sanitize API inputs
* fmt
* Dont allow html a, img tags
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* add option to only show posts from moderated communities
* rename moderated_only to moderator_view and show blocked users in moderator view
* add test for moderator view
* bump lemmy-js-client version for moderation view tests
* fix yarn lockfile
* retry build
* Delete logfile
* retry checks
* remove unused select statement from list posts
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Skip fragile API tests
These tests fail very often for no reason, eg in #3712 and #3696.
Better to disable them until they can be fixed.
* killall -s1
* add new flag to api
* add new ui settings for local user
* remove extraneous def
* add props to application reg.
* fix clippy updated these
* re-order db schema entries
* remove dupe
* update lemmy sdk
* update lemmy js client
---------
Co-authored-by: Nutomic <me@nutomic.com>
* Adding typescript generation for API. Fixes#2824
* Try to fix Ltree issue 1.
* Forgot a few types.
* Fixing api tests.
* Removing url_serde line.
* Manually deriving TS for some types.
* Adding diesel enums for SortType and ListingType
- Uses diesel-derive-enum.
- Adds diesel.toml , so we can again use the auto-generated schema.rs
- Fixes a lot of DB null issues and column ordering issues.
- Fixes#1136
- Also replaces RegistrationMode boilerplate.
* Fixing unit tests 1.
* Remove comment line.
* Before patch.
* Before again.
* Using patch file to fix diesel_ltree issue with diesel.toml
* Adding some yalc ignores
* Fixing RegistrationMode enums
* Adding woodpecker diesel schema check.
* Try adding openssl 1.
* Try using diesel-cli image 1
* Try using diesel-cli image 2
* Try using diesel-cli image 3
* Try using diesel-cli image 4
* Try using diesel-cli image 5
* Try using diesel-cli image 6
* Try using diesel-cli image 7
* Try using diesel-cli image 8
* Try using diesel-cli image 9
* Try using diesel-cli image 10
* Try using diesel-cli image 11
* Try using diesel-cli image 12
* Try using diesel-cli image 13
* Show deleted and removed posts for profile views. Fixes#2624
* Only showing non-deleted/removed posts for creator.
* Add a admin or mod check to views, to show deleted and removed posts.
- Also removed the pointless "blanking" functions
* Fix clippy
* Make hidden posts comment clearer.
* Fixing federation tests.
* Fixing fmt.
* Moving settings to Database.
- Moves many settings into the database. Fixes#2285
- Adds a local_site and instance table. Fixes#2365 . Fixes#2368
- Separates SQL update an insert forms, to avoid runtime errors.
- Adds TypedBuilder to all the SQL forms, instead of default.
* Fix weird clippy issue.
* Removing extra lines.
* Some fixes from suggestions.
* Fixing apub tests.
* Using instance creation helper function.
* Move forms to their own line.
* Trying to fix local_site_data, still broken.
* Fixing federation tests.
* Trying to fix check features 1.
* Addressing PR comments.
* Adding check_apub to all verify functions.
* First pass at adding comment trees.
- Extracted comment replies into its own table.
- Added ltree column to comment
- Added parent_id param to GetComments to fetch a tree branch
- No paging / limiting yet
* Adding child_count to comment_aggregates.
* Adding parent comment update counts
* Fix unit tests.
* Comment tree paging mostly done.
* Fix clippy
* Fix drone tests wrong postgres version.
* Fix unit tests.
* Add back in delete in unit test.
* Add postgres upgrade script.
* Fixing some PR comments.
* Move update ltree into Comment::create
* Updating based on comments.
* Fix send soft fail.
* Adding check for requests with no id or name, adding max limit.
* Consolidating a few functions.
* Fix page min
* Adding more websocket rate limits.
* Add check to GetCommunity
* Use a default message rate limit check.
* Adding a page and limit checker
* Fix clippy
* Fix clippy again
* Adding check for requests with no id or name, adding max limit.
* Consolidating a few functions.
* Fix page min
* Adding more websocket rate limits.
* Add check to GetCommunity
* Use a default message rate limit check.
* Adding a page and limit checker
* Fix clippy
* Fix clippy again
* Fix limit request.
* Move checks to inside limit_and_offset
* Fixing API tests.
* Change NotFound diesel errors to QueryBuilderError
* Create example for apub lib
* some rewriting of apub lib
* Add LocalInstance struct for apub lib to avoid using Lemmy Settings
* Move ActorType trait to lemmy_apub, because its not needed in library
* Use reqwest_retry instead of custom impl, dont specify timeout on every send()
* Some improvements to example
* Moved inbox handling to library
* bug fixes
* Move context and serde helpers into library
* wip: example changes
* Add lemmy_utils feature to build only LemmyError
* Rename to activitypub_federation
* Remove lemmy_utils dep from activitypub_federation using generic error type
* Finish activitypub example
* Cleanup and fix tests
* Reorganize library files
* Remove ApubObject.to_tombstone()
* Extract activitypub library into separate git repository
* Implement instance actor
* wip: make site bans federate
* finish implementation and unit tests for federated bans
* start adding api tests
* fix api test
* remve site from GetCommunityResponse
* only federate site bans originating from user's home instance
* dont expose site.private_key in api
