* Fix media attachment size validation not correctly accounting for file type
Fixes a regression introduced in #16724 caused by the fact that kt-paperclip
now correctly runs validations before processing, meaning that file size
verification could not rely on our before_post_processing hook.
Moved the `before_post_processing` hooks to `before_validate` to make sure
the media attachment type is set correctly before the file gets validated.
* Add tests
Conflicts:
- `app/models/custom_emoji.rb`:
Slight refactor upstream, next to a line that was different in glitch-soc
because of our local configurable limits on custom emoji size.
Ported upstream changes.
- `yarn.lock`:
Not really a conflict, upstream dependency textually too close to a
glitch-soc-only dependency.
Updated upstream dependency as upstream.
* Switch from unmaintained paperclip to kt-paperclip
* Drop some compatibility monkey-patches not required by kt-paperclip
* Drop media spoof check monkey-patching
It's broken with kt-paperclip and hopefully it won't be needed anymore
* Fix regression introduced by paperclip 6.1.0
* Do not rely on pathname to call FastImage
* Add test for ogg vorbis file with cover art
* Add audio/vorbis to the accepted content-types
This seems erroneous as this would be the content-type for a vorbis stream
without an ogg container, but that's what the `marcel` gem outputs, so…
* Restore missing for_as_default method
* Refactor Attachmentable concern and delay Paperclip's content-type spoof check
Check for content-type spoofing *after* setting the extension ourselves, this
fixes a regression with kt-paperclip's validations being more strict than
paperclip 6.0.0 and rejecting some Pleroma uploads because of unknown
extensions.
* Please CodeClimate
* Add audio/vorbis to the unreliable set
It doesn't correspond to a file format and thus has no extension associated.
* Fix followers synchronization mechanism not working when URI has empty path
To my knowledge, there is no current implementation on the fediverse
that can use bare domains (e.g., actor is at https://example.org instead of
something like https://example.org/actor) that also plans to support the
followers synchronization mechanism. However, Mastodon's current implementation
would exclude such accounts from followers list.
Also adds tests and rename them to reflect the proper method names.
* Move url prefix regexp to its own constant
Conflicts:
- `Gemfile.lock`:
Not a real conflict, upstream-updated dependency (redis) textually too
close to glitch-soc-only dependecy.
Updated redis gem like upstream did.
* Add account statuses cleanup policy model
* Record last inspected toot to delete to speed up successive calls to statuses_to_delete
* Add service to cleanup a given account's statuses within a budget
* Add worker to go through account policies and delete old toots
* Fix last inspected status id logic
All existing statuses older or equal to last inspected status id must be
kept by the current policy. This is an invariant that must be kept so that
resuming deletion from the last inspected status remains sound.
* Add tests
* Refactor scheduler and add tests
* Add user interface
* Add support for discriminating based on boosts/favs
* Add UI support for min_reblogs and min_favs, rework UI
* Address first round of review comments
* Replace Snowflake#id_at_start with with_random parameter
* Add tests
* Add tests for StatusesCleanupController
* Rework settings page
* Adjust load-avoiding mechanisms
* Please CodeClimate
Conflicts:
- `app/helpers/accounts_helper.rb`:
Conflict due to upstream changing how followers count is displayed while we
have an option to hide followers count.
Ported upstream change.
- `app/views/accounts/_header.html.haml`:
Conflict due to upstream changing how followers count is displayed while we
have an option to hide followers count.
Ported upstream change.
- `app/views/directories/index.html.haml`:
Conflict due to upstream changing how followers count is displayed while we
have an option to hide followers count.
Ported upstream change.
- `app/views/statuses/_simple_status.html.haml`:
Small markup change in glitch-soc, on a line that has been modified by
upstream. Ported upstream changes.
If a status with a hashtag becomes very popular, it stands to
reason that the hashtag should have a chance at trending
Fix no stats being recorded for hashtags that are not allowed
to trend, and stop ignoring bots
Remove references to hashtags in profile directory from the code
and the admin UI
* Fix issues with POSIX::Spawn, Terrapin and Ruby 3.0
Also improve the Terrapin monkey-patch for the stderr/stdout issue.
* Fix keyword argument handling throughout the codebase
* Monkey-patch Paperclip to fix keyword arguments handling in validators
* Change validation_extensions to please CodeClimate
* Bump microformats from 4.2.1 to 4.3.1
* Allow Ruby 3.0
* Add Ruby 3.0 test target to CircleCI
* Add test for admin dashboard warnings
* Fix admin dashboard warnings on Ruby 3.0
Conflicts:
- `README.md`:
Upstream updated copyright year, we don't mention it so kept our version.
- `app/controllers/admin/dashboard_controller.rb`:
Not really a conflict, upstream change (removing the spam checker) too close
to glitch-soc changes. Ported upstream changes.
- `app/models/form/admin_settings.rb`:
Same.
- `app/services/remove_status_service.rb`:
Same.
- `app/views/admin/settings/edit.html.haml`:
Same.
- `config/settings.yml`:
Same.
- `config/environments/production.rb`:
Not a real conflict, upstream added a default HTTP header, but we have
extra headers in glitch-soc.
Added the header.
Prevent new accounts from being created using the same underlying
e-mail as a suspended account using extensions and period
permutations. Stores e-mails as a SHA256 hash
* Update devise-two-factor to unreleased fork for Rails 6 support
Update tests to match new `rotp` version.
* Update nsa gem to unreleased fork for Rails 6 support
* Update rails to 6.1.3 and rails-i18n to 6.0
* Update to unreleased fork of pluck_each for Ruby 6 support
* Run "rails app:update"
* Add missing ActiveStorage config file
* Use config.ssl_options instead of removed ApplicationController#force_ssl
Disabled force_ssl-related tests as they do not seem to be easily testable
anymore.
* Fix nonce directives by removing Rails 5 specific monkey-patching
* Fix fixture_file_upload deprecation warning
* Fix yield-based test failing with Rails 6
* Use Rails 6's index_with when possible
* Use ActiveRecord::Cache::Store#delete_multi from Rails 6
This will yield better performances when deleting an account
* Disable Rails 6.1's automatic preload link headers
Since Rails 6.1, ActionView adds preload links for javascript files
in the Links header per default.
In our case, that will bloat headers too much and potentially cause
issues with reverse proxies. Furhermore, we don't need those links,
as we already output them as HTML link tags.
* Switch to Rails 6.0 default config
* Switch to Rails 6.1 default config
* Do not include autoload paths in the load path
* Bypass MX validation for explicitly allowed domains
This spares some lookups and prevent issues in some edge cases with
local domains.
* Add tests
* Fix test
* Improve account counters handling
* Use ActiveRecord::Base::sanitize_sql to pass values instead of interpolating them
Keep using string interpolation for `key` as it is safe and using
“ActiveRecord::Base::sanitize_sql_hash_for_assignment” would require stitching
bits of SQL in a way that is not more easily checked for safety.
* Add migration hook to catch PostgreSQL versions earlier than 9.5
* Fix admin being able to suspend their own instance account
* Add text about the instance's own actor in admin view
* Change instance actor notice from flash message to template
* Do not list local instance actor in account moderation list
Conflicts:
- `app/models/form/admin_settings.rb`:
New setting added upstream. Ported it.
- `app/views/statuses/_simple_status.html.haml`:
Upstream removed RTL classes. Did the same.
- `config/settings.yml`:
New setting added upstream. Ported it.
Conflicts:
- `app/controllers/relationships_controller.rb`:
Upstream changed a line too close to a glitch-soc only line related to
glitch-soc's theming system.
Applied upstream changes accordingly.
Conflicts:
- `.github/dependabot.yml`:
Updated upstream, we deleted it to not be flooded by Depandabot.
Kept deleted.
- `Gemfile.lock`:
Puma updated on both sides, went for the most recent version.
- `app/controllers/api/v1/mutes_controller.rb`:
Upstream updated the serializer to support timed mutes, while
glitch-soc added a custom API ages ago to get information that
is already available elsewhere.
Dropped the glitch-soc-specific API, went with upstream changes.
- `app/javascript/core/admin.js`:
Conflict due to changing how assets are loaded. Went with upstream.
- `app/javascript/packs/public.js`:
Conflict due to changing how assets are loaded. Went with upstream.
- `app/models/mute.rb`:
🤷
- `app/models/user.rb`:
New user setting added upstream while we have glitch-soc-specific
user settings. Added upstream's user setting.
- `config/settings.yml`:
Upstream added a new user setting close to a user setting we had
changed the defaults for. Added the new upstream setting.
- `package.json`:
Upstream dependency updated “too close” to a glitch-soc-specific
dependency. No real conflict. Updated the dependency.
* Add support for followers synchronization on the receiving end
Check the `collectionSynchronization` attribute on `Create` and `Announce`
activities and synchronize followers from provided collection if possible.
* Add tests for followers synchronization on the receiving end
* Add support for follower synchronization on the sender's end
* Add tests for the sending end
* Switch from AS attributes to HTTP header
Replace the custom `collectionSynchronization` ActivityStreams attribute by
an HTTP header (`X-AS-Collection-Synchronization`) with the same syntax as
the `Signature` header and the following fields:
- `collectionId` to specify which collection to synchronize
- `digest` for the SHA256 hex-digest of the list of followers known on the
receiving instance (where “receiving instance” is determined by accounts
sharing the same host name for their ActivityPub actor `id`)
- `url` of a collection that should be fetched by the instance actor
Internally, move away from the webfinger-based `domain` attribute and use
account `uri` prefix to group accounts.
* Add environment variable to disable followers synchronization
Since the whole mechanism relies on some new preconditions that, in some
extremely rare cases, might not be met, add an environment variable
(DISABLE_FOLLOWERS_SYNCHRONIZATION) to disable the mechanism altogether and
avoid followers being incorrectly removed.
The current conditions are:
1. all managed accounts' actor `id` and inbox URL have the same URI scheme and
netloc.
2. all accounts whose actor `id` or inbox URL share the same URI scheme and
netloc as a managed account must be managed by the same Mastodon instance
as well.
As far as Mastodon is concerned, breaking those preconditions require extensive
configuration changes in the reverse proxy and might also cause other issues.
Therefore, this environment variable provides a way out for people with highly
unusual configurations, and can be safely ignored for the overwhelming majority
of Mastodon administrators.
* Only set follower synchronization header on non-public statuses
This is to avoid unnecessary computations and allow Follow-related
activities to be handled by the usual codepath instead of going through
the synchronization mechanism (otherwise, any Follow/Undo/Accept activity
would trigger the synchronization mechanism even if processing the activity
itself would be enough to re-introduce synchronization)
* Change how ActivityPub::SynchronizeFollowersService handles follow requests
If the remote lists a local follower which we only know has sent a follow
request, consider the follow request as accepted instead of sending an Undo.
* Integrate review feeback
- rename X-AS-Collection-Synchronization to Collection-Synchronization
- various minor refactoring and code style changes
* Only select required fields when computing followers_hash
* Use actor URI rather than webfinger domain in synchronization endpoint
* Change hash computation to be a XOR of individual hashes
Makes it much easier to be memory-efficient, and avoid sorting discrepancy issues.
* Marginally improve followers_hash computation speed
* Further improve hash computation performances by using pluck_each
Conflicts:
- `Gemfile.lock`:
Not a real conflict, upstream updated dependencies that were too close to
glitch-soc-only ones in the file.
- `app/controllers/oauth/authorized_applications_controller.rb`:
Upstream changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's theming system.
Ported upstream changes.
- `app/controllers/settings/base_controller.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's theming system.
Ported upstream changes.
- `app/controllers/settings/sessions_controller.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's theming system.
Ported upstream changes.
- `app/models/user.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc not preventing moved accounts from logging
in.
Ported upstream changes while keeping the ability for moved accounts to log
in.
- `app/policies/status_policy.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's local-only toots.
Ported upstream changes.
- `app/serializers/rest/account_serializer.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's ability to hide followers count.
Ported upstream changes.
- `app/services/process_mentions_service.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's local-only toots.
Ported upstream changes.
- `package.json`:
Not a real conflict, upstream updated dependencies that were too close to
glitch-soc-only ones in the file.
* Add bell button
Fix#4890
* Remove duplicate type from post-deployment migration
* Fix legacy class type mappings
* Improve query performance with better index
* Fix validation
* Remove redundant index from notifications
Conflicts:
- app/controllers/api/v1/timelines/public_controller.rb
- app/lib/feed_manager.rb
- app/models/status.rb
- app/services/precompute_feed_service.rb
- app/workers/feed_insert_worker.rb
- spec/models/status_spec.rb
All conflicts are due to upstream refactoring feed management and us having
local-only toots on top of that. Rewrote local-only toots management for
upstream's changes.
Conflicts:
- `app/controllers/accounts_controller.rb`:
Upstream change too close to a glitch-soc change related to
instance-local toots. Merged upstream changes.
- `app/services/fan_out_on_write_service.rb`:
Minor conflict due to glitch-soc's handling of Direct Messages,
merged upstream changes.
- `yarn.lock`:
Not really a conflict, caused by glitch-soc-only dependencies
being textually too close to updated upstream dependencies.
Merged upstream changes.
* feat: add possibility of adding WebAuthn security keys to use as 2FA
This adds a basic UI for enabling WebAuthn 2FA. We did a little refactor
to the Settings page for editing the 2FA methods – now it will list the
methods that are available to the user (TOTP and WebAuthn) and from
there they'll be able to add or remove any of them.
Also, it's worth mentioning that for enabling WebAuthn it's required to
have TOTP enabled, so the first time that you go to the 2FA Settings
page, you'll be asked to set it up.
This work was inspired by the one donde by Github in their platform, and
despite it could be approached in different ways, we decided to go with
this one given that we feel that this gives a great UX.
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
* feat: add request for WebAuthn as second factor at login if enabled
This commits adds the feature for using WebAuthn as a second factor for
login when enabled.
If users have WebAuthn enabled, now a page requesting for the use of a
WebAuthn credential for log in will appear, although a link redirecting
to the old page for logging in using a two-factor code will also be
present.
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
* feat: add possibility of deleting WebAuthn Credentials
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
* feat: disable WebAuthn when an Admin disables 2FA for a user
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
* feat: remove ability to disable TOTP leaving only WebAuthn as 2FA
Following examples form other platforms like Github, we decided to make
Webauthn 2FA secondary to 2FA with TOTP, so that we removed the
possibility of removing TOTP authentication only, leaving users with
just WEbAuthn as 2FA. Instead, users will have to click on 'Disable 2FA'
in order to remove second factor auth.
The reason for WebAuthn being secondary to TOPT is that in that way,
users will still be able to log in using their code from their phone's
application if they don't have their security keys with them – or maybe
even lost them.
* We had to change a little the flow for setting up TOTP, given that now
it's possible to setting up again if you already had TOTP, in order to
let users modify their authenticator app – given that now it's not
possible for them to disable TOTP and set it up again with another
authenticator app.
So, basically, now instead of storing the new `otp_secret` in the
user, we store it in the session until the process of set up is
finished.
This was because, as it was before, when users clicked on 'Edit' in
the new two-factor methods lists page, but then went back without
finishing the flow, their `otp_secret` had been changed therefore
invalidating their previous authenticator app, making them unable to
log in again using TOTP.
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
* refactor: fix eslint errors
The PR build was failing given that linting returning some errors.
This commit attempts to fix them.
* refactor: normalize i18n translations
The build was failing given that i18n translations files were not
normalized.
This commits fixes that.
* refactor: avoid having the webauthn gem locked to a specific version
* refactor: use symbols for routes without '/'
* refactor: avoid sending webauthn disabled email when 2FA is disabled
When an admins disable 2FA for users, we were sending two mails
to them, one notifying that 2FA was disabled and the other to notify
that WebAuthn was disabled.
As the second one is redundant since the first email includes it, we can
remove it and send just one email to users.
* refactor: avoid creating new env variable for webauthn_origin config
* refactor: improve flash error messages for webauthn pages
Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
Conflicts:
- `app/javascript/packs/public.js`:
Conflict because part of that file has been split to
`app/javascript/core/settings.js`. Ported those changes
there.
Conflicts:
- `.env.production.sample`:
Upstream deleted it, I decided to keep it.
- `package.json`:
Upstream updated a dependency textually too close to wavesurfer.js
which has been deleted from upstream but is kept in glitch-soc for now.
- Change audio files to not be stripped of metadata
- Automatically extract cover art from audio if it exists
- Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id`
- Add `icon` to represent it in attachments in ActivityPub
- Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null
- Fix duration of audio not being displayed on public pages until the file is loaded
Conflicts:
- `app/controllers/activitypub/collections_controller.rb`:
Conflict due to glitch-soc having to take care of local-only
pinned toots in that controller.
Took upstream's changes and restored the local-only special
handling.
- `app/controllers/auth/sessions_controller.rb`:
Minor conflicts due to the theming system, applied upstream
changes, adapted the following two files for glitch-soc's
theming system:
- `app/controllers/concerns/sign_in_token_authentication_concern.rb`
- `app/controllers/concerns/two_factor_authentication_concern.rb`
- `app/services/backup_service.rb`:
Minor conflict due to glitch-soc having to handle local-only
toots specially. Applied upstream changes and restored
the local-only special handling.
- `app/views/admin/custom_emojis/index.html.haml`:
Minor conflict due to the theming system.
- `package.json`:
Upstream dependency updated, too close to a glitch-soc-only
dependency in the file.
- `yarn.lock`:
Upstream dependency updated, too close to a glitch-soc-only
dependency in the file.
Conflicts:
- `Gemfile.lock`:
Not a real conflict, just a glitch-soc-only dependency too close to a
dependency that got updated upstream. Updated as well.
- `app/models/status.rb`:
Not a real conflict, just a change too close to glitch-soc-changed code
for optionally showing boosts in public timelines.
Applied upstream changes.
- `app/views/layouts/application.html.haml`:
Upstream a new, static CSS file, conflict due to glitch-soc's theming
system, include the file regardless of the theme.
- `config/initializers/content_security_policy.rb`:
Upstream dropped 'unsafe-inline' from the 'style-src' directive, but
both files are very different. Removed 'unsafe-inline' as well.
* Improve RSS entries for statuses
- Render polls in both accounts and tags serializers
- Refactor RSS serializers
- Change title preview to include ellipsis when truncated
- Change title preview to show CW instead of toot text
- Add tests
* Remove title from OEmbed serialization
Twitter doesn't serialize title either, and tihs allows us to move the
title formatting code to the RSS serializers.
Conflicts:
- `app/controllers/statuses_controller.rb`:
Upstream disabled the embed controller for reblogs.
Not a real conflict, but glitch-soc has an extra line to deal
with its theming system.
Ported upstream changes.
- `app/javascript/packs/public.js`:
Upstream made changes to get rid of most inline CSS, this changes
javascript for public pages, which in glitch are split between
different files. Ported those changes.
- `app/models/status.rb`:
Upstream changed the block check in `Status#permitted_for` to
include domain-block checks. Not a real conflict with glitch-soc,
but our scope is slightly different, as our scope for
unauthenticated access do not include instance-local toots.
Ported upstream changes.
- `app/serializers/rest/instance_serializer.rb`:
Not a real conflict, upstream added a new field to the instance
serializer, the conflict is one line above since we added more of
that.
Ported upstream changes.
- `app/views/settings/profiles/show.html.haml`:
Upstream got rid of most inline CSS and moved hidden elements
to data attributes in the process, in fields were we have
different values.
Ported upstream changes while keeping our glitch-specific
values.
- `app/views/statuses/_simple_status.html.haml`:
Upstream got rid of inline CSS on an HAML line we treat
differently, stripping empty text nodes.
Ported upstream changes to the style attribute, keeping
the empty text node stripping behavior.
* Add announcements
Fix#11006
* Add reactions to announcements
* Add admin UI for announcements
* Add unit tests
* Fix issues
- Add `with_dismissed` param to announcements API
- Fix end date not being formatted when time range is given
- Fix announcement delete causing reactions to send streaming updates
- Fix announcements container growing too wide and mascot too small
- Fix `all_day` being settable when no time range is given
- Change text "Update" to "Announcement"
* Fix scheduler unpublishing announcements before they are due
* Fix filter params not being passed to announcements filter
Conflicts:
- `Gemfile.lock`:
No real conflict, glitch-soc-only dependency (redcarpet) too close to an
upstream one (rdf-normalize)
- `README.md`:
we have different READMEs, discarded upstream's changes
- `app/views/admin/custom_emojis/index.html.haml`:
No real conflict, different context because of glitch-soc theming
- `lib/mastodon/statuses_cli.rb`:
Upstream added code to keep bookmarked statuses, we were already doing so
with slightly different code. Discarded upstream's changes.
- `package.json`:
No real conflict, glitch-soc-only dependency (favico.js) too close to
an upstream one
Fixes#12795
It was already possible to create domain blocks for TLDs, but those
weren't enforced, nor editable. This commit changes it so that they
are enforced and editable.
Conflicts:
- `app/controllers/application_controller.rb`:
Conflict due to theming system.
- `app/controllers/oauth/authorizations_controller.rb`:
Conflict due to theming system.
Conflicts:
- package.json
Not really a conflict, caused by an additional dependency in glitch-soc.
- yarn.lock
Not really a conflict, caused by an additional dependency in glitch-soc.
* Add follow_request notification type
The notification type already existed in the backend but was never pushed
to the front-end. This also means translation strings were also available
for the backend, from the notification mailer.
Unlike other notification types, these are off by default, to match what
I remember of Gargron's view on the topic: that follow requests should not
clutter notifications and should instead be reviewed at the user's own
leisure in the dedicated column.
Since follow requests have their own column, I've deemed it unnecessary to
add a specific tab for them in the notification quick filter.
* Show follow request link in single-column if there are pending requests, even if account isn't locked
* Push follow requests from notifications to the follow_requests list
* Offer to accept or reject follow request from the notification
* Redesign follow request notification
Change the behaviour of remotable concern. Previously, it would skip
downloading an attachment if the stored remote URL is identical to
the new one. Now it would not be skipped if the attachment is not
actually currently stored by Paperclip.
Fetching statuses from all followed accounts at once takes too long
within Postgres. Fetching them one by one and merging in Ruby
could be a lot less resource-intensive
Because the query for dynamically fetching the home timeline is so
heavy, we can no longer offer it when the home timeline is missing
Conflicts:
- app/models/media_attachment.rb
Upstream raised max image size from 8MB to 10MB while our limit is
configurable. Raised the default to 10MB.
Conflicts:
- Gemfile
- app/controllers/api/v1/search_controller.rb
Conflict because we changed the number of default results to be
configurable
- app/lib/settings/scoped_settings.rb
Addition of a new “noindex” site-wide setting,
conflict due to our change of the two other site-wide settings
(default flavour and skin instead of theme)
- spec/controllers/application_controller_spec.rb
Addition of a new “noindex” site-wide setting,
conflict due to our change of the two other site-wide settings
(default flavour and skin instead of theme)
