1
0
Fork 1
mirror of https://github.com/mastodon/mastodon.git synced 2024-12-22 17:17:32 +00:00
Commit graph

34 commits

Author SHA1 Message Date
Mike Dalessio a0ea2fa3b0
Change fetch link card service to parse as HTML5 () 2024-09-09 10:59:42 +00:00
David Roetzel 2ea9336b68
Do not pass unknown encoding names to nokogiri. () 2024-07-10 14:25:39 +00:00
David Roetzel 967505ee9b
Add size limit for all PreviewCard URLs () 2024-07-09 13:11:34 +00:00
David Roetzel 97eddb5906
Fix details extraction when no title exists. () 2024-07-05 13:28:52 +00:00
David Roetzel 016c1e4e78
Improve handling of encoding problems when creating link previews () 2024-07-05 11:54:38 +00:00
David Roetzel ff08d99d4d
Catch encoding errors when creating link previews. () 2024-06-27 14:41:03 +00:00
David Roetzel 42adb6eaee
Add size limit for link preview URLs () 2024-06-27 14:40:19 +00:00
David Roetzel 2cab1c7b09
Improve encoding detection for link cards () 2024-06-21 12:51:10 +00:00
Matt Jankowski 469028b6d3
Remove unneeded type: :service from spec/services files () 2024-03-13 08:39:26 +00:00
Eugen Rochko 0d14fcebae
Change link previews to keep original URL from the status () 2023-11-13 09:58:28 +00:00
Christian Schmidt f96e4b3047
Use original URL in preview if it redirects to 4xx page () 2023-07-28 23:02:08 +02:00
Matt Jankowski 7d62e3b198
Reformat large text arg in FetchLinkCardService spec () 2023-07-26 09:44:51 +02:00
Matt Jankowski 6c0e3f490a
Fix RSpec/MissingExampleGroupArgument cop () 2023-06-06 15:51:42 +02:00
Renaud Chaput 942d850b0a
Allow carets in URL search params () 2023-06-01 12:14:49 +02:00
Matt Jankowski 710745e16b
Fix RSpec/ContextWording cop () 2023-05-04 05:49:08 +02:00
Nick Schonning 84cc805cae
Enable Style/FrozenStringLiteralComment for specs () 2023-02-22 09:55:31 +09:00
Nick Schonning 81ad6c2e39
Autofix Rubocop Style/StringLiterals () 2023-02-19 07:38:14 +09:00
Eugen Rochko 39cdf61ab7
Add support for structured data and more OpenGraph tags to link cards ()
Save preview cards under their canonical URL

Increase max redirects to follow from 2 to 3
2021-11-05 23:23:05 +01:00
Claire 5614e6724e
Fix URL scanning in note length validator and preview card fetching ()
* Add tests

* Fix URL scanning in note length validator and preview card fetching
2021-03-04 00:12:26 +01:00
ThibG c656cc2191 Fix FetchLinkCardServices crashing on a tags without a target ()
* Add test for links without targets

* Fix FetchLinkCardServices crashing on a tags without a target
2019-11-21 16:04:52 +01:00
BSKY fccf83e1f2 Add noopener and/or noreferrer () 2019-10-24 22:44:42 +02:00
Eugen Rochko 9ba40a6bfd
Remove HEAD request from fetching link previews ()
It is not really necessary and we need to reduce requests
2019-10-01 04:54:10 +02:00
ThibG e709b8da0d Ignore low-confidence CharlockHolmes guesses when parsing link cards ()
* Add failing test for windows-1251 link cards

* Ignore low-confidence CharlockHolmes guesses

Fixes 

* Fix no method error when charlock holmes cannot detect charset
2018-12-17 19:19:45 +01:00
Eugen Rochko cb5b5cb5f7
Slightly reduce RAM usage ()
* No need to re-require sidekiq plugins, they are required via Gemfile

* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile

* Replace ruby-oembed with FetchOEmbedService

Reduce startup by 45382 allocated objects

* Remove preloaded JSON-LD in favour of caching HTTP responses

Reduce boot RAM by about 6 MiB

* Fix tests

* Fix test suite by stubbing out JSON-LD contexts
2018-05-02 18:58:48 +02:00
Daniel King 6ef3874b2e Fix URLs incorrectly having trailing hyphen removed ()
In cases where a URL has a trailing hyphen the FetchLinkCardService incorrectly removes the hyphen when it is parsed

The hyphen is not a reserved character in the URI spec https://tools.ietf.org/html/rfc3986#section-2.2
2018-02-11 23:49:18 +01:00
ふぁぼ原 3816943e6b Enable to recognize most kinds of characters as URL paths () 2017-09-14 18:03:20 +02:00
Eugen Rochko e7adbf572a Switch to static URIs, new URI format in both protocols for new statuses ()
* Decouple Status#local? from uri being nil

* Replace on-the-fly URI generation with stored URIs

- Generate URI in after_save hook for local statuses
- Use static value in TagManager when available, fallback to tag format
- Make TagManager use ActivityPub::TagManager to understand new format
- Adjust tests

* Use other heuristic for locality of old statuses, do not perform long query

* Exclude tombstone stream entries from Atom feed

* Prevent nil statuses from landing in Pubsubhubbub::DistributionWorker

* Fix URI not being saved ()

* Add more specs for Status

* Save generated uri immediately

and also fix method order to minimize diff.

* Fix alternate HTML URL in Atom

* Fix tests

* Remove not-null constraint from statuses migration to speed it up
2017-09-06 19:01:28 +02:00
Eugen Rochko 7dc5035031 Make PreviewCard records reuseable between statuses ()
* Make PreviewCard records reuseable between statuses

**Warning!** Migration truncates preview_cards tablec

* Allow a wider thumbnail for link preview, display it in horizontal layout ()

* Delete preview cards files before truncating

* Rename old table instead of truncating it

* Add mastodon:maintenance:remove_deprecated_preview_cards

* Ignore deprecated_preview_cards in schema definition

* Fix null behaviour
2017-09-01 16:20:16 +02:00
nullkal 007ab330e6 Use charlock_holmes instead of nkf at FetchLinkCardService ()
* Specs for language detection

* Use CharlockHolmes instead of NKF

* Correct mistakes

* Correct style

* Set hint_enc instead of falling back and strip_tags

* Improve specs

* Add dependencies
2017-07-08 22:44:31 +02:00
abcang 5e6acf9601 Fix Nokogiri::HTML at FetchLinkCardService () 2017-07-05 14:54:21 +02:00
Eugen Rochko 1efda1c453 Fix - Resolve preview cards for remote statuses as well as local ones () 2017-05-17 00:41:15 +02:00
Eugen Rochko a231f915a0 Fix - Send HEAD request ahead of GET when fetching URL previews () 2017-05-10 23:30:07 +02:00
Eugen Rochko 88725d6ce8 OEmbed support for PreviewCard ()
* OEmbed support for PreviewCard

* Improve ProviderDiscovery code failure treatment

* Do not crawl links if there is a content warning, since those
don't display a link card anyway

* Reset db schema

* Fresh migrate

* Fix rubocop style issues
Fix  - return existing access token when applicable instead of creating new

* Fix test

* Extract http client to helper

* Improve oembed controller
2017-04-27 14:42:22 +02:00
Eugen 17c591ffba Punycode URI normalization ()
* Fix  - Whenever about to send a HTTP request, normalize the URI

* Add test for IDN request in FetchLinkCardService

* Perform IDN normalization on domains before they are stored in the DB
2017-04-25 02:47:31 +02:00