Commit graph

1048 commits

Author SHA1 Message Date
Claire 2c45859ca9
Refactor account and status formatting (#17877)
* Refactor status formatting

* Add account formatting helpers

* Remove StatusFormatter

* Fixup

* Fix copied typo
2022-03-28 01:17:17 +02:00
Claire aaa9ec340b Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/lib/formatter.rb`:
  Upstream completely refactored the formatting code and removed that file,
  while glitch-soc had code for Markdown and HTML toots.
  Took upstream code, glitch-soc changes will be re-implemented on top of the
  refactored classes in a later commit.
- `app/models/status.rb`:
  Upstream refactored status edit handling and moved code to
  `app/models/concerns/status_snapshot_concern.rb`.
  Applied glitch-soc's changes to that file.
- `app/serializers/activitypub/note_serializer.rb`:
  Not really a conflict, just a line added too close to one modified by
  glitch-soc.
  Applied upstream changes while keeping the glitch-soc-modified one.
- `app/services/update_status_service.rb`:
  Not really a conflict, upstream modified a line adjacent to one added by
  glitch-soc.
  Applied upstream changes while keeping the glitch-soc line.
- `app/views/statuses/_simple_status.html.haml`:
  Upstream refactored formatting, glitch-soc changed the markup slightly.
  Applied upstream changes.
- `spec/lib/formatter_spec.rb`:
  Upstream completely refactored the formatting code and removed that file,
  while glitch-soc had code for Markdown and HTML toots.
  Took upstream code, glitch-soc changes will be re-implemented on top of the
  refactored classes in a later commit.
2022-03-26 19:18:55 +01:00
Eugen Rochko cefa526c6d
Refactor formatter (#17828)
* Refactor formatter

* Move custom emoji pre-rendering logic to view helpers

* Move more methods out of Formatter

* Fix code style issues

* Remove Formatter

* Add inline poll options to RSS feeds

* Remove unused helper method

* Fix code style issues

* Various fixes and improvements

* Fix test
2022-03-26 02:53:34 +01:00
Eugen Rochko 71f2b95106
Fix edits with no actual changes being allowed (#17843)
* Fix edits with no actual changes being allowed locally

* Fix edits with no actual changes being allowed through ActivityPub

* Fix false positive changes caused by description processing in model

* Fix not recording poll expiration update

* Fix test

* Revert changes to ProcessStatusUpdateService

* Various fixes and improvements

* Fix code style issues

* Various changes and improvements

* Add guard clause
2022-03-26 00:38:44 +01:00
Claire f65eaa5aae
Add admin dashboard checks for Elasticsearch version (#17863) 2022-03-25 21:00:59 +01:00
Eugen Rochko b58db8f12e
Add workaround for YouTube Shorts links (#17869)
* Add workaround for YouTube Shorts links

* Update link_details_extractor_spec.rb
2022-03-25 19:31:35 +01:00
Claire 4e3c68e025 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/helpers/accounts_helper.rb`:
  Upstream fixed an issue that glitch-soc did not have, in a place with
  glitch-soc changes to hide the number of followers.
  Kept glitch-soc's code.
2022-03-17 19:23:39 +01:00
Claire c9b3cf8510
Fix searching already-known converted activities by URL (#17814)
Fixes #17813
2022-03-17 00:46:49 +01:00
Claire bb9a6fb5c2 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-12 11:51:25 +01:00
Eugen Rochko ddbe906c25
Fix not updating a status when newer version is fetched manually (#17745) 2022-03-12 09:11:36 +01:00
Claire 24e83246f9 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/status.rb`:
  Upstream updated media and edit-related code textually close to glitch-soc
  additions (local-only and content-type).
  Ported upstream changes.
- `app/models/status_edit.rb`:
  Upstream changes textually close to glitch-soc additions (content-type).
  Ported upstream changes.
- `app/serializers/activitypub/note_serializer.rb`:
  Upstream changed how media attachments are handled. Not really a conflict,
  but textually close to glitch-soc additions (directMessage attribute).
  Ported upstream changes.
- `app/services/remove_status_service.rb`:
  Upstream changed how media attachments are handled. Not really a conflict,
  but textually close to glitch-soc additions (DM timeline).
  Ported upstream changes.
- `app/services/update_status_service.rb`:
  Upstream fixed an issue with language selection. Not really a conflict,
  but textually close to glitch-soc additions (content-type).
  Ported upstream changes.
- `db/schema.rb`:
  Upstream added columns to the `status_edits` table, the conflict is because
  of an additional column (`content-type`) in glitch-soc.
  Ported upstream changes.
- `package.json`:
  Upstream dependency (express) textually adjacent to a glitch-soc-specific one
  (favico.js) got updated.
  Updated it as well.
2022-03-10 09:52:45 +01:00
Eugen Rochko d17fb70131
Change how changes to media attachments are stored for edits (#17696)
* Change how changes to media attachments are stored for edits

Fix not being able to re-order media attachments

* Fix not broadcasting updates when polls/media is changed through ActivityPub

* Various fixes and improvements

* Update app/models/report.rb

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Add tracking of media attachment description changes

* Change poll in status edit to have a structure closer to the real one

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-03-09 09:06:17 +01:00
Eugen Rochko bd53dd5210
Change design of federation pages in admin UI (#17704)
* Change design of federation pages in admin UI

* Fix query performance in instance media attachments measure

* Fix reblogs being included in instance languages dimension
2022-03-09 08:52:32 +01:00
Claire 1fbd1fa5c4 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/settings/preferences_controller.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/lib/user_settings_decorator.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/models/status.rb`:
  Conflict because of slight change in how glitch-soc handles the scope to
  filter out local-only posts for anonymous viewers.
  Took upstream's changes and re-applied glitch-soc's change.
- `app/models/user.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/views/directories/index.html.haml`:
  Conflict because upstream redesigned that page while glitch-soc had a minor
  change to support hiding the number of followers.
  Ported glitch-soc's change on top of upstream's redesign.

Additional changes:
- `app/models/account_statuses_filter.rb`:
  See change to `app/models/status.rb`.
2022-03-08 20:22:54 +01:00
Eugen Rochko edf09ec747
Add /api/v1/accounts/familiar_followers to REST API (#17700)
* Add `/api/v1/accounts/familiar_followers` to REST API

* Change hide network preference to be stored consistently for local and remote accounts

* Add dummy classes to migration

* Apply suggestions from code review

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-03-07 09:36:47 +01:00
Claire d9c3109bb6 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-04 01:07:49 +01:00
Eugen Rochko 2ea754b861
Fix duplicate notifications being possible after poll expiration (#17697) 2022-03-04 01:06:33 +01:00
Eugen Rochko 04c3ac896a
Fix infinite re-forwarding of update activities temporarily (#17698) 2022-03-04 01:00:13 +01:00
Eugen Rochko 085f286750
Fix returning empty <p> tag for blank account note in REST API (#17687) 2022-03-03 06:45:46 +01:00
Claire 425a6c90c4 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-02 18:02:48 +01:00
Eugen Rochko 50ea54b3ed
Change authorized applications page (#17656)
* Change authorized applications page

* Hide revoke button for superapps and suspended accounts

* Clean up db/schema.rb
2022-03-01 16:48:58 +01:00
Claire 8158b04d18 Merge branch 'main' into glitch-soc/merge-upstream 2022-02-26 21:15:37 +01:00
Eugen Rochko 0dc57ab6ed
Fix status updates not being forwarded like deletes through ActivityPub (#17648)
Fix #17521
2022-02-26 17:51:59 +01:00
Claire be493b6c0d Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/account.rb`:
  Not a real conflict, just upstream getting rid of unused constants too close
  to glitch-soc-specific contents.
  Removed unused constants like upstream did.
- `app/models/trends.rb`:
  Conflict because glitch-soc disabled email notifications for trending links.
  Upstream has refactored this quite a bit and added trending posts.
  Took upstream code, but disabling the extra trending stuff will come in
  another commit.
- `app/views/admin/trends/links/index.html.haml`:
  Conflict due to glitch-soc's theming system.
  Ported upstream changes accordingly.
2022-02-26 09:29:23 +01:00
Eugen Rochko 27965ce5ed
Add trending statuses (#17431)
* Add trending statuses

* Fix dangling items with stale scores in localized sets

* Various fixes and improvements

- Change approve_all/reject_all to approve_accounts/reject_accounts
- Change Trends::Query methods to not mutate the original query
- Change Trends::Query#skip to offset
- Change follow recommendations to be refreshed in a transaction

* Add tests for trending statuses filtering behaviour

* Fix not applying filtering scope in controller
2022-02-25 00:34:14 +01:00
Claire 3d25fe0426 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/workflows/build-image.yml`:
  Upstream changed the workflow a bit.
  Conflict comes from us pushing to ghcr while upstream pushes to dockerhub.
  Ported the upstream changes while still pushing to ghcr.
2022-02-23 18:13:12 +01:00
luzpaz 73f5e4a1d9
Fix various typos (#17621)
Found via `codespell -q 3 -S ./CHANGELOG.md,./AUTHORS.md,./config/locales,./app/javascript/mastodon/locales -L ba,keypair,medias,ro`
2022-02-22 20:14:17 +01:00
Claire 954e052b07 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/media_attachment.rb`:
  Upstream moved hardcoded values around, while in glitch-soc they are
  configurable.
  Moved them like upstream, but keeping glitch-soc's ability to configure
  them through env vars.
2022-02-22 18:06:29 +01:00
Claire 166f6e4b50
Fix some media attachments being converted with too high framerates (#17619)
Video files with variable framerates are converted to constant framerate videos
and the output framerate picked by ffmpeg is based on the original file's
container framerate (which can be different from the average framerate).

This means that an input video with variable framerate with about 30 frames per
second on average, but a maximum of 120 fps will be converted to a constant 120
fps file, which won't be processed by other Mastodon servers.

This commit changes it so that input files with VFR and a maximum framerate
above the framerate threshold are converted to VFR files with the maximum frame
rate enforced.
2022-02-22 17:11:22 +01:00
Eugen Rochko b377022cf9
Add caching layer to metrics (#17617) 2022-02-22 15:27:08 +01:00
Claire f224237862 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `db/schema.rb`:
  Conflict due to glitch-soc adding the `content_type` column on status edits
  and thus having a different schema version number.
  Solved by taking upstream's schema version number, as it is higher than
  glitch-soc's.
2022-02-17 10:58:44 +01:00
Claire 8f537a1168
Change relays handling to not record boosts (#17571)
* Change relays handling to not record boosts

* Update tests
2022-02-16 14:36:44 +01:00
Claire 73fce8d311
Fix performance of server-side filtering (#17575)
Fixes #17567
2022-02-16 14:28:45 +01:00
Jeong Arm 9a015e43ef
Add from: query operator to search syntax (#16526)
* Add 'by:userhandle' parameter to search api

* Use search syntax for "by" prefix

* Codeclimate

* Use 'from' instead of 'by'
2022-02-14 00:17:09 +01:00
Claire 322e907e04 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/settings/preferences/appearance/show.html.haml`:
  Upstream renamed some helper functions that were used in a part of the
  settings page which glitch-soc slightly changed the layout of.
  Ported the change.
2022-02-09 17:28:33 +01:00
Eugen Rochko 3aebe711fd
Change languages to be listed under standard instead of native name in admin UI (#17485) 2022-02-09 04:15:38 +01:00
Eugen Rochko fd3a45e348
Add edit history to web UI (#17390)
* Add edit history to web UI

* Change history reducer to store items per status

* Fix missing loading prop
2022-02-09 01:17:07 +01:00
Claire 692963d43b Merge branch 'main' into glitch-soc/merge-upstream 2022-02-08 18:23:53 +01:00
Eugen Rochko b6d7726ecb
Remove language detection through cld3 (#17478)
* Remove language detection through cld3

* Update app/helpers/languages_helper.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2022-02-08 02:41:17 +01:00
Eugen Rochko f1f6ddd536
Fix structured data parsing from links choking on bad data (#17403)
* Fix structured data parsing from links choking on bad data

- Fix og:url meta tag being prioritized over canonical link tag
- Fix structured data parsing choking on commented-out CDATA declarations
- Fix HTML entities in title, description, provider_name, author_name
- Change structured data parsing to attempt every JSON-LD script tag

* Remove unnecessary slash escapes from CDATA regex pattern
2022-02-07 18:16:31 +01:00
Claire d1ecc323e7
Compact JSON-LD signed incoming activities (#17426)
Co-authored-by: Puck Meerburg <puck@puck.moe>
2022-02-03 14:07:29 +01:00
Puck Meerburg 63da32468c Compact JSON-LD signed incoming activities 2022-02-02 16:13:11 +01:00
Claire b2915613fb Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Upstream-updated lib textually too close to glitch-soc-only dep.
  Updated like upstream.
2022-01-28 08:58:32 +01:00
Claire 03d59340da
Fix Sidekiq warnings about JSON serialization (#17381)
* Fix Sidekiq warnings about JSON serialization

This occurs on every symbol argument we pass, and every symbol key in hashes,
because Sidekiq expects strings instead.

See https://github.com/mperham/sidekiq/pull/5071

We do not need to change how workers parse their arguments because this has
not changed and we were already converting to symbols adequately or using
`with_indifferent_access`.

* Set Sidekiq to raise on unsafe arguments in test mode

In order to more easily catch issues that would produce warnings in production
code.
2022-01-28 00:43:56 +01:00
Claire 61ef81c548 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `spec/models/status_spec.rb`:
  Upstream added tests too close to glitch-soc-specific tests.
  Kept both tests.
2022-01-23 18:24:01 +01:00
Claire a63495230a
Change percent to rate in retention metrics API (#16910) 2022-01-23 16:01:25 +01:00
Claire fe89554a54 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/lib/activitypub/activity/create.rb`:
  Upstream refactored how `Create` activities are handled and how values are
  extracted from `Create`d objects. This conflicted with how glitch-soc
  supported the `directMessage` flag to explicitly distinguish between
  limited and direct messages.
  Ported glitch-soc's changes to latest upstream changes.
- `app/services/fan_out_on_write_service.rb`:
  Upstream largely refactored that file and changed some of the logic.
  This conflicted with glitch-soc's handling of the direct timeline and
  the options to allow replies and boosts in public feeds.
  Ported those glitch-soc changes on top of latest upstream changes.
- `app/services/process_mentions_service.rb`:
  Upstream refactored to move mention-related ActivityPub deliveries to
  `ActivityPub::DeliveryWorker`, while glitch-soc contained an extra check
  to not send local-only toots to remote mentioned users.
  Took upstream's version, as the check is not needed anymore, since it is
  performed at the `ActivityPub::DeliveryWorker` call site already.
- `app/workers/feed_insert_worker.rb`:
  Upstream added support for `update` toot events, while glitch-soc had
  support for an extra timeline support, `direct`.
  Ported upstream changes and extended them to the `direct` timeline.

Additional changes:
- `app/lib/activitypub/parser/status_parser.rb`:
  Added code to handle the `directMessage` flag and take it into account
  to compute visibility.
- `app/lib/feed_manager.rb`:
  Extended upstream's support of `update` toot events to glitch-soc's
  `direct` timeline.
2022-01-19 23:52:48 +01:00
Eugen Rochko 1060666c58
Add support for editing for published statuses (#16697)
* Add support for editing for published statuses

* Fix references to stripped-out code

* Various fixes and improvements

* Further fixes and improvements

* Fix updates being potentially sent to unauthorized recipients

* Various fixes and improvements

* Fix wrong words in test

* Fix notifying accounts that were tagged but were not in the audience

* Fix mistake
2022-01-19 22:37:27 +01:00
Claire b209e919bd
Merge pull request #1662 from ClearlyClaire/glitch-soc/merge-upstream
Merge upstream changes
2022-01-19 14:22:59 +01:00
Claire f6acc039ec
Merge pull request #1660 from ClearlyClaire/glitch-soc/features/themes-multiple-packs
Refactor glitch-soc's theme handling
2022-01-18 00:18:07 +01:00
Claire b3bf32a21e Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/reports/show.html.haml`:
  Conflicts due to glitch-soc's theming system.
2022-01-17 10:45:25 +01:00
Eugen Rochko 14f436c457
Add notifications for statuses deleted by moderators (#17204) 2022-01-17 09:41:33 +01:00
Claire 2d4b4b0b45 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/accounts_controller.rb`:
  Upstream introduced support for private pinned toots, but glitch-soc's query
  was a bit different as it filtered out local-only toots.
  Used upstream's query, while adding local-only filtering back.
- `app/controllers/activitypub/collections_controller.rb`:
  Same thing with regards to local-only posts.
- `app/validators/status_pin_validator.rb`:
  Not a real conflict, but the line below was different in glitch-soc due to
  the configurable pinned toots limit.
2022-01-17 08:28:52 +01:00
Claire d5c9feb7b7
Add support for private pinned posts (#16954)
* Add support for private pinned toots

* Allow local user to pin private toots

* Change wording to avoid "direct message"
2022-01-17 00:49:55 +01:00
Claire 1b386c881c Refactor theme config loading 2022-01-16 22:26:18 +01:00
Claire 09c61e1376 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  We have completely different contents. Kept our version.
- `package.json`:
  Not a real conflict, just an upstream dependency udpated
  textually too close to a glitch-soc-only dependency.
  Updated dependencies like upstream.
- `streaming/index.js`:
  Conflict due to code style changes on parts that were
  modified in glitch-soc to handle local-only toots.
  Changed style according to upstream.
2021-12-27 15:16:18 +01:00
Eugen Rochko fe71548844
Fix warnings on Rails boot (#16946) 2021-12-27 00:47:20 +01:00
Claire 5bf3a09c5b Merge branch 'main' into glitch-soc/merge-upstream 2021-11-30 12:13:54 +01:00
Jeong Arm 4a109ec1ba
Fix server graph on admin/tags/:id (#17066) 2021-11-28 23:13:07 +01:00
Claire 9b861d56a9 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.env.production.sample`:
  Copied upstream changes.
- `app/controllers/settings/identity_proofs_controller.rb`:
  Minor conflict due to glitch-soc's extra “enable_keybase” setting.
  Upstream removed keybase support altogether, so did the same.
- `app/controllers/well_known/keybase_proof_config_controller.rb`:
  Minor conflict due to glitch-soc's extra “enable_keybase” setting.
  Upstream removed keybase support altogether, so did the same.
- `lib/mastodon/statuses_cli.rb`:
  Minor conflict due to an optimization that wasn't shared between
  the two versions. Copied upstream's version.
2021-11-26 22:53:55 +01:00
Eugen Rochko 7de0ee7aba
Remove Keybase integration (#17045) 2021-11-26 05:58:18 +01:00
Claire 443ec4f8ba Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/tags/index.html.haml`:
  Removed upstream while it had changes in glitch-soc to accomodate for the
  theming system.

Additional changes to accomodate for the theming system:
- `app/views/admin/trends/links/preview_card_providers/index.html.haml`
- `app/views/admin/trends/links/index.html.haml`
- `app/views/admin/trends/tags/index.html.haml`
- `app/views/admin/tags/show.html.haml`
2021-11-25 23:50:35 +01:00
Eugen Rochko 6e50134a42
Add trending links (#16917)
* Add trending links

* Add overriding specific links trendability

* Add link type to preview cards and only trend articles

Change trends review notifications from being sent every 5 minutes to being sent every 2 hours

Change threshold from 5 unique accounts to 15 unique accounts

* Fix tests
2021-11-25 13:07:38 +01:00
Claire 2b4c7beb73 Merge branch 'main' into glitch-soc/merge-upstream 2021-11-19 18:51:41 +01:00
Claire 3517867b76
Fix confusing error when webfinger request returns empty document (#16986)
For some reason, some misconfigured servers return an empty document when
queried over webfinger. Since an empty document does not lead to a parse
error, the error is not caught properly and triggers uncaught exceptions
later on.

This PR fixes that by immediately erroring out with `Webfinger::Error` on
getting an empty response.
2021-11-14 21:55:40 +01:00
Eugen Rochko 5d7c852283
Fix no link previews being generated for pages with invalid structured data (#16979)
Fix #16955
2021-11-13 23:07:13 +01:00
Claire 2cda8be805 Merge branch 'main' into glitch-soc/merge-upstream 2021-11-13 09:40:13 +01:00
Claire ef8deb0878
Fix OpenGraph/LinkedData embeds having incorrectly-generated iframes (#16978) 2021-11-13 03:30:27 +01:00
Claire 4bb6b1a1e7 Merge branch 'main' into glitch-soc/merge-upstream 2021-11-06 00:15:38 +01:00
Eugen Rochko 39cdf61ab7
Add support for structured data and more OpenGraph tags to link cards (#16938)
Save preview cards under their canonical URL

Increase max redirects to follow from 2 to 3
2021-11-05 23:23:05 +01:00
Claire 6964952d5f Merge branch 'main' into glitch-soc/merge-upstream 2021-10-14 22:22:39 +02:00
Claire a184c860ca
Fix retention rate computation (#16854) 2021-10-14 22:20:37 +02:00
Claire 694c073d1f Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/admin/dashboard_controller.rb`:
  Upstream completely redesigned the admin dashboard.
  glitch-soc tracked extra features, but that list is
  gone.
  Followed upstram.
- `app/views/admin/dashboard/index.html.haml`
  Upstream completely redesigned the admin dashboard.
  glitch-soc tracked extra features, but that list is
  gone.
  Followed upstram.
2021-10-14 21:44:57 +02:00
Eugen Rochko 07341e7aa6
Add graphs and retention metrics to admin dashboard (#16829) 2021-10-14 20:44:59 +02:00
Claire 29b5c5f483 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/custom_emoji.rb`:
  Slight refactor upstream, next to a line that was different in glitch-soc
  because of our local configurable limits on custom emoji size.
  Ported upstream changes.
- `yarn.lock`:
  Not really a conflict, upstream dependency textually too close to a
  glitch-soc-only dependency.
  Updated upstream dependency as upstream.
2021-09-30 00:01:20 +02:00
Claire fc3ae1343d
Switch from unmaintained paperclip to kt-paperclip (#16724)
* Switch from unmaintained paperclip to kt-paperclip

* Drop some compatibility monkey-patches not required by kt-paperclip

* Drop media spoof check monkey-patching

It's broken with kt-paperclip and hopefully it won't be needed anymore

* Fix regression introduced by paperclip 6.1.0

* Do not rely on pathname to call FastImage

* Add test for ogg vorbis file with cover art

* Add audio/vorbis to the accepted content-types

This seems erroneous as this would be the content-type for a vorbis stream
without an ogg container, but that's what the `marcel` gem outputs, so…

* Restore missing for_as_default method

* Refactor Attachmentable concern and delay Paperclip's content-type spoof check

Check for content-type spoofing *after* setting the extension ourselves, this
fixes a regression with kt-paperclip's validations being more strict than
paperclip 6.0.0 and rejecting some Pleroma uploads because of unknown
extensions.

* Please CodeClimate

* Add audio/vorbis to the unreliable set

It doesn't correspond to a file format and thus has no extension associated.
2021-09-29 23:52:36 +02:00
Claire 3622110778 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `streaming/index.js`:
  Filtering code for streaming notifications has been refactored upstream, but
  glitch-soc had similar code for local-only toots in the same places.
  Ported upstream changes, but did not refactor local-only filtering.
2021-09-26 18:28:59 +02:00
Eugen Rochko 52e5c07948
Change routing paths to use usernames in web UI (#16171) 2021-09-26 05:46:13 +02:00
Claire f1e6884d6b Merge branch 'main' into glitch-soc/merge-upstream 2021-09-14 13:50:30 +02:00
Claire 12cd097e7c
Fix addressing of remote groups' followers (#16700)
Fixes #16699
2021-09-08 23:33:23 +02:00
Claire 8ec4be4233 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Not a real conflict, upstream-updated dependency (redis) textually too
  close to glitch-soc-only dependecy.
  Updated redis gem like upstream did.
2021-08-09 23:25:49 +02:00
Takeshi Umeda 709876bd6c
Fix invalid blurhash handling in Create activity (#16583) 2021-08-09 13:33:19 +02:00
Claire b5e10d07ff Merge branch 'main' into glitch-soc/merge-upstream 2021-07-25 12:48:48 +02:00
Claire d8629e7b86
Add logging of S3-related errors (#16381) 2021-07-21 18:34:39 +02:00
Claire d1c87be8cf Merge branch 'main' into glitch-soc/merge-upstream 2021-07-17 17:45:35 +02:00
Claire 211d5c3c30
Fix inefficiencies in auto-linking code (#16506)
The auto-linking code basically rewrote the whole string escaping non-ascii
characters in an inefficient way, and building a full character offset map
between the unescaped and escaped texts before sending the contents to
TwitterText's extractor.

Instead of doing that, this commit changes the TwitterText regexps to include
valid IRI characters in addition to valid URI characters.
2021-07-15 15:56:58 +02:00
Claire 5f334807d2 Merge branch 'main' into glitch-soc/merge-upstream 2021-05-22 13:11:31 +02:00
Claire 9a19227f17
Fix some RedisLocks auto-releasing too fast (#16276)
* Fix Delete and Create-related locks expiring too fast

Fixes #16238

By default, RedisLock expires after 10 seconds, which may not be enough to
process statuses, especially when those have attached media files.

This commit extends those 10 seconds to 15 minutes, which should be plenty
enough to handle any status, while being short enough to not waste many
sidekiq job retries in the exceedingly rare case in which a sidekiq process
would crash when processing a `Create` or `Delete`.

* Fix other RedisLock autorelease durations

Fixes #15645

- things that only perform a few simple database queries (e.g. finding and
  saving a record) have been left unchanged, so they'll still use the default
  10s duration
- things that perform significantly more complex database queries have been
  changed to a 5 minutes timeout
- things that perform multiple HTTP queries have been changed to a 15 minutes
  timeout
2021-05-19 23:52:08 +02:00
Claire dc58d02192 Merge branch 'main' into glitch-soc/merge-upstream 2021-05-12 11:03:05 +02:00
abcang b5ad787ebf
Fix rubocop warning (#16214) 2021-05-11 14:19:01 +02:00
Claire 7ac8bcf209 Merge branch 'main' into glitch-soc/merge-upstream 2021-05-10 21:45:32 +02:00
Claire afb7882189
Fix blocking someone not clearing up list feeds (#16205) 2021-05-10 17:31:55 +02:00
Claire 50b430d9a2 Merge branch 'main' into glitch-soc/merge-upstream
- `app/views/statuses/_simple_status.html.haml`:
  Small markup change in glitch-soc, on a line that has been modified by
  upstream. Ported upstream changes.
2021-05-07 18:21:59 +02:00
Eugen Rochko 74081433d0
Change trending hashtags to be affected be reblogs (#16164)
If a status with a hashtag becomes very popular, it stands to
reason that the hashtag should have a chance at trending

Fix no stats being recorded for hashtags that are not allowed
to trend, and stop ignoring bots

Remove references to hashtags in profile directory from the code
and the admin UI
2021-05-07 14:33:43 +02:00
Takeshi Umeda 7cb34b32f8
Add management of delivery availability in Federation settings (#15771)
* Add management of delivery availavility in Federation settings

* fix translate

* Remove useless object creation

* Fix DeepSource issue

* Add shortcut for all

* Fix DeepSource(skipcq)

* Change 'remove' to 'clear'

* Fix style

* Change class method name (exhausted_deliveries_key_by)
2021-05-05 23:39:02 +02:00
Eugen Rochko 036556d350
Fix media processing getting stuck on too much stdin/stderr (#16136)
* Fix media processing getting stuck on too much stdin/stderr

See thoughtbot/terrapin#5

* Remove dependency on paperclip-av-transcoder gem

* Remove dependency on streamio-ffmpeg gem

* Disable stdin on ffmpeg process
2021-05-05 19:44:01 +02:00
Eugen Rochko daccc07dc1
Change auto-following admin-selected accounts, show in recommendations (#16078) 2021-04-24 17:01:43 +02:00
Eugen Rochko f4b7c6b619
Fix nil error when removing status caused by race condition (#16099) 2021-04-24 13:35:39 +02:00
Eugen Rochko 2eb17360df
Fix delete of local reply to local parent not being forwarded (#16096) 2021-04-22 14:26:11 +02:00
Claire 536892b8ae Merge branch 'main' into glitch-soc/merge-upstream 2021-04-21 13:52:03 +02:00