fedi/akkoma
1
0
Fork 0
mirror of https://akkoma.dev/AkkomaGang/akkoma.git synced 2024-12-13 06:08:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
14630 commits 27 branches 108 tags 316 MiB
Commit graph

14630 commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
FloatingGhost 1ecdb19de5 Refactor ES on top of search behaviour 2022-06-30 16:28:31 +01:00
Ekaterina Vaartis 69d5d1a01b Update meilisearch docs 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis 563b964690 Change updateId to uid because apparently that's the new name 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis b7462040cc Change the meilisearch key auth to conform to 0.25.0 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis 09ea3bb694 Actually, unlisted posts are indexed 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis 774b4e165a Change search_indexing = 10 and retries for indexing = 2 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis 7f53aa400b Don't try removing deleted users and such from index as posts 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis 7aebff799b Fix meilisearch tests and jobs for oban 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis 5def4a7d49 Use oban for search indexing 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis 54b2a86f47 Add a search backend behaviour 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis 80e52f4d86 Add description for initial_indexing_chunk_size 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis a4914add8c Don't support meilisearch < 0.24.0, since it breaks things 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis cc3319ac1d Make chunk size configurable 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis 426cff3372 Update search.md documentation with meilisearch indexing steps 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis 58cc5d13a2 Add config description for meilisearch 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis bac70a2bc1 Implement suggestions from the Meilisearch MR 
- Index unlisted posts
- Move version check outside of the streaming and only do it once
- Use a PUT request instead of checking manually if there is need to insert
- Add error handling, sort of
 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis c128798418 Rename Activity.Search to Search.DatabaseSearch 2022-06-29 20:49:44 +01:00
Ekaterina Vaartis ea582fbf93 Rename search.ex to database_search.ex and add search/2 2022-06-29 20:49:00 +01:00
Ekaterina Vaartis 0769f06bd1 Style fixes 2022-06-29 20:49:00 +01:00
Ekaterina Vaartis 5dd908552c Move the search.ex file so credo doesn't complain 2022-06-29 20:48:59 +01:00
Ekaterina Vaartis 7a9d9cf457 Fix a typo in search docs 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis 86971fceaa Support reindexing meilisearch >=0.24.0 
It has has a different error code key
 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis 9e7d7ebd48 Add a reindex option 
Signed-off-by: Ekaterina Vaartis <vaartis@kotobank.ch>
 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis d99a2be351 Only add local posts to index in activity_pub 
Remote ones are already added in another place
 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis 51faa28568 Set content-type to application/json 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis afcdd2f437 Modify some meilisearch variables 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis 37913c8ddf Use proper deleted object for removing from index 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis a937a98df5 Don't try removing from index again in common_api 
It's already removed in the side effects of the pipeline
 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis 9ab0f130cc Add more documentation about rum to meilisearch docs 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis 0802c0666b Move add_to_index / remove_from_index to Pleroma.Actitivy.Search 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis 5ed1759091 Reorder ranking rules for (maybe) better results 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis b519726571 Add private_key: nil to default meilisearch options 2022-06-29 20:48:43 +01:00
Ekaterina Vaartis 88891e9d76 Add tests for local post indexing for meilisearch 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 41f3572152 Fix activity being passed to objec_to_search_data 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 375154e5de Add search/meilisearch documentation 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis d1079f1aa3 Add the meilisearch.stats command 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis d5cc272a91 Add a message with a count of posts to index 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis dbf556cdcf Implement meilisearch auth 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 5360cc1097 Make indexing logs rewrite themselves 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 0cf3654907 Rework task indexing to share code with the main module 
The code in the main module now scrubs new posts too
 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 117f525fd6 Adjust content indexing to skip more unneeded stuff 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 14ef6ce80f Mark only content as searchable for meilisearch 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis b3401ba7bd Also index incoming federated posts 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 52a872432d Make the chunk size smaller 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis a586ce0ddd Use content instead of source and scrub it 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis c3a04166a0 Tweak search ordering to hopefully return newer results 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 7b3701e6b9 Make meilisearch sort on publish date converted to unix time 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis abf82a63ec Make the indexing batch differently and more, show number indexed 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis e5ac2ffa07 Ensure only indexing public posts and implement clearing and delete 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 41db5c8653 Add logging to milisiearch index and make it use desc(id) 2022-06-29 20:48:29 +01:00