* Split activity table into sent and received parts (fixes#3103)
The received activities are only stored in order to avoid processing
the same incoming activity multiple times. For this purpose it is
completely unnecessary to store the data. So we can split the
table into sent_activity and received_activity parts, where
only sent_activity table needs to store activity data. This should
reduce storage use significantly.
Also reduces activity storage duration to three months, we can reduce
this further if necessary.
Additionally the id columns of activity tables are removed because
they are completely unused and risk overflowing (fixes#3560).
* address review
* move insert_received_activity() methods to verify handlers
* remove unnecessary conflict line
* clippy
* use on conflict, add tests
Two bugs were found and fixed:
- previously elements removal and deletion were counted as two separate disappearances
- removing comments did not affect post aggregations
* improve admin and mod check
* fix clippy
* move admin index to existing code
* Revert "move admin index to existing code"
This reverts commit d0c58d5f4021e1775d0c1d30d8df6c7df87557c4.
* third attempt at the migration
* fix formatting
* rebuild
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Remove `actix_rt` & use standard tokio spawn
* Adjust rust log back down
* Format correctly
* Update cargo lock
* Add DB settings
* Change name and update to latest rev
* Clean up formatting changes
* Move `worker_count` and `worker_retry_count` to settings
* Update defaults
* Use `0.4.4` instead of git branch
* Adding hot_rank columns in place of function sorting.
- Creates hot_rank columns for post, comment, and community.
- Fixes#2932
* Updating all hot ranks on startup.
* Fixing post.url migration.
* Removing update_instance_software from startup.
* Adding post_rank query
* Adding diesel enums for SortType and ListingType
- Uses diesel-derive-enum.
- Adds diesel.toml , so we can again use the auto-generated schema.rs
- Fixes a lot of DB null issues and column ordering issues.
- Fixes#1136
- Also replaces RegistrationMode boilerplate.
* Fixing unit tests 1.
* Remove comment line.
* Before patch.
* Before again.
* Using patch file to fix diesel_ltree issue with diesel.toml
* Adding some yalc ignores
* Fixing RegistrationMode enums
* Adding woodpecker diesel schema check.
* Try adding openssl 1.
* Try using diesel-cli image 1
* Try using diesel-cli image 2
* Try using diesel-cli image 3
* Try using diesel-cli image 4
* Try using diesel-cli image 5
* Try using diesel-cli image 6
* Try using diesel-cli image 7
* Try using diesel-cli image 8
* Try using diesel-cli image 9
* Try using diesel-cli image 10
* Try using diesel-cli image 11
* Try using diesel-cli image 12
* Try using diesel-cli image 13
* Combine prod and dev docker setups using build-arg
- Fixes#2603
* Dont use cache for release build.
* Adding 2FA / TOTP support.
- Fixes#2363
* Changed name to totp_2fa for clarity.
* Switch to sha256 for totp.
* Implement separate mod activities for feature, lock post
Also includes collection for featured posts. Later we also need
to do the same for Comment.distinguished
* some changes
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Speeding up comment-ltree migration, fixing index creation. Fixes#2664
* Adding some logging lines, fixing for missing posts.
* Adding more postgres config
* Implement federated user following (fixes#752)
* rewrite send_activity_in_community and add docs, remove default for column pending
* improve migration
* replace null values in db migration
* Moving settings to Database.
- Moves many settings into the database. Fixes#2285
- Adds a local_site and instance table. Fixes#2365 . Fixes#2368
- Separates SQL update an insert forms, to avoid runtime errors.
- Adds TypedBuilder to all the SQL forms, instead of default.
* Fix weird clippy issue.
* Removing extra lines.
* Some fixes from suggestions.
* Fixing apub tests.
* Using instance creation helper function.
* Move forms to their own line.
* Trying to fix local_site_data, still broken.
* Fixing federation tests.
* Trying to fix check features 1.
* Addressing PR comments.
* Adding check_apub to all verify functions.
* implement language tags for site/community in db and api
* add api checks for valid languages
* during db migration, update existing users, sites, communities to have all languages enabled
* init new users/communities with site languages (not all languages)
* federate site/community languages
* fix tests
* when updating site languages, limit community languages to this subset
also, when making a new post and subset of user lang, community lang
contains only one item, use that as post lang
* add tests for actor_language db functions
* include language list in siteview/communityview
* Fix some of the review comments
* Some more review changes
* Add todo about boxed query
* Add default_post_language to GetCommunityResponse
* Initial commit to bump diesel to 2.0.0-rc.0 and see what happens
* Add chrono feature from diesel
* db_schema crate is close to building?
* Upgrade diesel-derive-newtype
* Mostly modifying references to connections to be mutable ones; also used
new way to do migrations as suggested by the migration guide; a lot more
compiles now, though I can't figure out this tricky ToSql issue at the
moment
* Running clippy --fix
* Trying to fix drone clippy 1
* Fix clippy
* Upgrade clux-musl
* Trying to fix drone clippy 2
* Trying to fix drone clippy 3
* Trying to fix drone clippy 5
* Adding diesel table aliases, removing sql view hack. Fixes#2101
Co-authored-by: Steven Chu <stevenc1@gmail.com>
Co-authored-by: Nutomic <me@nutomic.com>
* Tag posts and comments with language (fixes#440)
* Untangle PostView tests
* Implement test for PostView language query
* Store languages directly in database
* finish moving languages into db, it compiles
* update post_view
* serde skip Language.id field
* add local_user_language table, other changes suggested in review
* add code for local_user_discussion_language_view
* Remove unnecessary clones in db view converteres
* Fixing up some table and join issues.
* Clearing the current languages.
* Fix formatting.
* update user languages in single transaction
* proper test for user language queries
* Some fixes for all / missing user languages. (#2404)
* Some fixes for all / missing user languages.
* Adding back in transaction.
* fix test
Co-authored-by: Dessalines <tyhou13@gmx.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* First pass at adding comment trees.
- Extracted comment replies into its own table.
- Added ltree column to comment
- Added parent_id param to GetComments to fetch a tree branch
- No paging / limiting yet
* Adding child_count to comment_aggregates.
* Adding parent comment update counts
* Fix unit tests.
* Comment tree paging mostly done.
* Fix clippy
* Fix drone tests wrong postgres version.
* Fix unit tests.
* Add back in delete in unit test.
* Add postgres upgrade script.
* Fixing some PR comments.
* Move update ltree into Comment::create
* Updating based on comments.
* Fix send soft fail.
* First pass at adding admin purge. #904#1331
* Breaking out purge into 4 tables for the 4 purgeable types.
* Using CommunitySafe instead in view
* Fix db_schema features flags.
* Attempting to pass API key.
* Adding pictrs image purging
- Added pictrs_config block, for API_KEY
- Clear out image columns after purging
* Remove the remove_images field from a few of the purge API calls.
* Fix some suggestions by @nutomic.
* Add separate pictrs reqwest client.
* Update defaults.hjson
Co-authored-by: Nutomic <me@nutomic.com>
* Update community statistics after post or comment is deleted by user
* Update triggers for site and person aggregates
* Restore comment counts when post is restored
* Simplify insertion/deletion trigger checks before updating statistics
* Initial working of hiding communities and adding a db entry for mod log
* Return mod log for hidden communities
* Clean up hidding communities PR
* use lower case like other migration files
* Formatting fix
* pass in admin id to list, make match logic the same in post_view as community_view. Dont force non null for reason
* Clean PR review stuff
* Change person_id to mod_person_id on hide community table
* Make bools optional, add a space for formating
Co-authored-by: Thor Odinson <odinson@asgard.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Implement instance actor
* wip: make site bans federate
* finish implementation and unit tests for federated bans
* start adding api tests
* fix api test
* remve site from GetCommunityResponse
* only federate site bans originating from user's home instance
* dont expose site.private_key in api
* First pass at invite-only migration.
* Implement email verification (fixes#219)
* remove unwrap
* Adding views and functionality to registration application. #209
* Add private instance site column, and back end checks.
* Adding some message fields to LoginResponse
* Adding private instance to site setup.
* A few additions:
- Add a DeleteAccount response.
- RegistrationApplicationView now has the safe LocalUserSettings.
- Adding VerifyEmail to websocket API, added a proper response type.
* Adding and reorganizing some email helpers.
* A few fixes for private sites:
- Added a check_registration_application function.
- Only send a verification email if its been changed.
- VerifyEmail now returns LoginResponse.
- Deleting the old tokens after a successful email verify.
- If port is missing on email config, display a better error message.
* Version 0.15.0-rc.3
* Adding published to email_verification table.
* Adding fixes from comments.
* Version 0.15.0-rc.4
* Adding modlog private site check.
* Version 0.15.0-rc.6
Co-authored-by: Felix Ableitner <me@nutomic.com>
* A first pass at user / community blocking. #426
* Adding unit tests for person and community block.
* Moving migration
* Fixing creator_blocked for comment queries, added tests.
* Don't let a person block themselves
* Fix post creator_blocked
* Adding creator_blocked to PersonMentionView
* Moving blocked and follows to MyUserInfo
* Rename to local_user_view
* Add moderates to MyUserInfo
* Adding BlockCommunityResponse
* Fixing name, and check_person_block
* Fixing tests.
* Using type in Blockable trait.
* Changing recipient to target, adding unfollow to block action.
* Make length of user/community name configurable (fixes#1306)
* use single config value, fix migrations, increase displayname length
* remove debug statements
* add helper method for setting actor_name_max_length
* move function
* Fixing some defaults
Co-authored-by: Felix Ableitner <me@nutomic.com>
* Mark account as bot. Fixes#1357
* Fix clippy
* Federate bot status using actor `type` field
* fix clippy
Co-authored-by: Dessalines <tyhou13@gmx.com>
* origin/main:
revert Compose file version from 3.3 to 2.2
Adding more mem limits
bump memory limit of iframely
Remove extra category_id s . Fixes#1429
Fixing wrong user_ and community icon and banner urls.
Remove category from activitypub context
Adding a password length check to other API actions. (#1474)
Update test script
Use URL type in most outstanding struct fields (#1468)
Forbid usage of unwrap
Upgrade Rust version
Rewrite settings implementation. Fixes#1270 (#1433)
Rename `lemmy_structs` to `lemmy_api_structs`
# Conflicts:
# crates/db_schema/src/source/user.rs
* Use URL type in most outstanding struct fields
This fixes all known remaining cases where url fields are stored as
plain strings, with the exception of form fields where empty strings
are used as sentinels (see `diesel_option_overwrite_to_url`).
Tested for regressions in the federated docker setup attempting to
exercise all changed fields, including through apub federation.
Fixes#1385
* Add migration to fix blank-string post.url values to be null
This also then fixes#602
* Address review feedback
- Fixed some unwraps and err message formatting
- Bumped the `url` library to 2.2.1 to fix a bug with serde error
messages
- Add unit tests for the two diesel option override functions
- Fix migration teardown by adding a no-op
* Rename lemmy_db_queries::Url to lemmy_db_queries::DbUrl
* fix compile error
* box PostOrComment variants
* Adding a new comment sort. Fixes#1294
* Fixing a migration comment.
* Adding a comment for newest_comment_time_necro
* Make sure federated items set correct aggregates fields in trigger.
- Fixes#1402
* Adding some recurring lemmy tasks.
- Add active users by day, week, month, and half year to site and
community. Fixes#1195
- Periodically re-index the aggregates tables that use hot_rank.
Fixes#1384
- Clear out old activities (> 6 months). Fixes#1133
* Some cleanup, recalculating actives every hour.
Forgot to add migrations
Add `sensitive` column to activities table, so PMs arent served over HTTP
Remove user_id column from actvity table
Co-authored-by: Felix Ableitner <me@nutomic.com>
Reviewed-on: https://yerbamate.ml/LemmyNet/lemmy/pulls/127
* Adding a way to GetComments for a community given its name only.
* Adding getcomments to api docs.
* A first pass at locally working isomorphic integration.
* Testing out cargo-husky.
* Testing a fail hook.
* Revert "Testing a fail hook."
This reverts commit 0941cf1736.
* Moving server to top level, now that UI is gone.
* Running cargo fmt using old way.
* Adding nginx, fixing up docker-compose files, fixing docs.
* Trying to re-add API tests.
* Fixing prod dockerfile.
* Redoing nightly fmt
* Trying to fix private message api test.
* Adding CommunityJoin, PostJoin instead of joins from GetComments, etc.
- Fixes#1122
* Fixing fmt.
* Fixing up docs.
* Removing translations.
* Adding apps / clients to readme.
* Fixing main image.
* Using new lemmy-isomorphic-ui with better javascript disabled.
* Try to fix image uploads in federation test
* Revert "Try to fix image uploads in federation test"
This reverts commit a2ddf2a90b.
* Fix post url federation
* Adding some more tests, some still broken.
* Don't need gitattributes anymore.
* Update local federation test setup
* Fixing tests.
* Fixing travis build.
* Fixing travis build, again.
* Changing lemmy-isomorphic-ui to lemmy-ui
* Error in travis build again.
Co-authored-by: Felix Ableitner <me@nutomic.com>
