Commit graph

96 commits

Author SHA1 Message Date
rinpatch f70335002d RichMedia: Do a HEAD request to check content type/length
This shouldn't be too expensive, since the connections are pooled,
but it should save us some bandwidth since we won't fetch non-html
files and files that are too large for us to process (especially
since you can't cancel a request without closing the connection
with HTTP1).
2020-09-14 14:45:58 +03:00
rinpatch f66a15c4a5 RichMedia parser: do not set a cache TTL for unchanging errors 2020-09-14 14:44:25 +03:00
Alexander Strizhakov 696bf09433
passing adapter options directly without adapter key 2020-09-07 19:59:17 +03:00
Alexander Strizhakov a83916fdac
adapter options unification
not needed options deletion
2020-09-07 19:59:17 +03:00
lain fdab01ab56 Merge branch 'fix/rich-media-fake-statuses' into 'develop'
Rich Media: Do not cache URLs for preview statuses

Closes #1987

See merge request pleroma/pleroma!2956
2020-09-07 10:19:19 +00:00
rinpatch 170599c390 RichMedia: do not log webpages missing metadata as errors
Also fixes the return value of Parser.parse on errors, previously
was just `:ok` due to the logger call in the end
2020-09-05 22:05:35 +03:00
rinpatch e198ba492e Rich Media: Do not cache URLs for preview statuses
Closes #1987
2020-09-05 20:53:46 +03:00
rinpatch 19691389b9 Rich media: Add failure tracking 2020-09-02 14:59:52 +03:00
rinpatch 47ff425cfd Merge branch 'fix/2047-rich-media-parser' into 'develop'
RichMedia parser fix

Closes #2047

See merge request pleroma/pleroma!2941
2020-09-02 09:38:43 +00:00
Alexander Strizhakov 79f65b4374
correct pool and uniform headers format 2020-09-02 09:16:51 +03:00
Alexander Strizhakov 03d06062ab
don't fail on url fetch 2020-09-01 19:39:07 +03:00
Mark Felder 016d8d6c56 Consolidate construction of Rich Media Parser HTTP requests 2020-08-03 12:37:31 -05:00
lain 781b270863 ChatMessageReferenceView: Display preview cards. 2020-07-30 19:57:26 +02:00
lain 5b1eeb06d8 Revert "Merge branch 'revert-2b5d9eb1' into 'develop'"
This reverts merge request !2784
2020-07-21 22:18:17 +00:00
lain 696c13ce54 Revert "Merge branch 'linkify' into 'develop'"
This reverts merge request !2677
2020-07-21 22:17:34 +00:00
Alex Gleason 38425ebdbf
Merge remote-tracking branch 'upstream/develop' into linkify 2020-07-16 14:51:36 -05:00
Mark Felder 18438a9bf0 Add "Bot" to User Agent to coerce Twitter into serving OGP <meta> tags. 2020-07-07 11:21:05 -05:00
Alex Gleason 8daacc9114
AutoLinker --> Linkify, update to latest version
https://git.pleroma.social/pleroma/elixir-libraries/linkify
2020-06-30 16:39:15 -05:00
Egor Kislitsyn 58e4e3db8b
Merge remote-tracking branch 'origin/develop' into merge-ogp-twitter-parsers 2020-06-15 16:03:40 +04:00
Egor Kislitsyn 520367d6fd Fix atom leak in Rich Media Parser 2020-06-13 12:08:46 +03:00
Egor Kislitsyn 09d31d24de
Return an empty map from Pleroma.Web.RichMedia.Parsers.OGP.parse/2 2020-06-12 18:39:51 +04:00
Egor Kislitsyn 2419776e19
Deprecate Pleroma.Web.RichMedia.Parsers.OGP 2020-06-11 23:11:46 +04:00
Egor Kislitsyn 1f35acce54
Merge OGP parser with TwitterCard 2020-06-11 17:57:31 +04:00
Mark Felder 3bf78f2be7 Fix Oban not receiving :ok from RichMediaHelper job 2020-04-14 11:43:53 -05:00
Mark Felder 4427161ca3 Merge branch 'develop' into gun 2020-03-03 17:15:49 -06:00
Mark Felder 05da5f5cca Update Copyrights 2020-03-03 16:44:49 -06:00
Alexander Strizhakov 509c81e4b1
Merge branch 'develop' into gun 2020-03-03 10:08:07 +03:00
Haelwenn (lanodan) Monnier 6da6540036
Bump copyright years of files changed after 2020-01-07
Done via the following command:
git diff fcd5dd259a --stat --name-only | xargs sed -i '/Pleroma Authors/c# Copyright © 2017-2020 Pleroma Authors <https:\/\/pleroma.social\/>'
2020-03-02 06:08:45 +01:00
Mark Felder cf94349287 Merge branch 'develop' into gun 2020-02-18 09:06:27 -06:00
Alexander Strizhakov 514c899275
adding gun adapter 2020-02-18 08:19:01 +03:00
rinpatch 472132215e Use floki's new APIs for parsing fragments 2020-02-16 01:55:26 +03:00
feld 237b2068f9 Revert "Merge branch 'feat/floki-fasthtml' into 'develop'"
This reverts merge request !2194
2020-02-11 16:55:18 +00:00
rinpatch ea1631d7e6 Make Floki use fast_html 2020-02-11 16:17:21 +03:00
Alexander Strizhakov 1f4fbe9d98
title parse improvement 2020-01-29 11:13:34 +03:00
Alexander Strizhakov 7bd4c14581
meta tag parser respect first title header 2020-01-28 19:29:27 +03:00
Maksim Pechnikov b4cf74c106 added prepare html for RichMedia.Parser 2019-09-15 14:53:58 +03:00
Maksim 139b196bc0 [#1150] fixed parser TwitterCard 2019-08-06 20:19:28 +00:00
Ariadne Conill b93498eb52 constants: add as_public constant and use it everywhere 2019-07-29 02:43:19 +00:00
Ariadne Conill d3bdb8e704 rich media: parser: splice the given URL into the result 2019-07-23 23:51:29 +00:00
rinpatch 3368174785 Fix rich media parser failing when no TTL can be found by image TTL
setters
2019-07-21 18:22:22 +03:00
Sachin Joshi de9906ad56 change the structure of image ttl parsar 2019-07-19 11:43:42 +05:45
Sachin Joshi 18234cc44e add the rich media ttl based on image exp time 2019-07-17 00:20:34 +05:45
Alex S f4447d82b8 parsers configurable 2019-07-14 09:21:56 +03:00
feld 93a0eeab16 Add license/copyright to all project files 2019-07-10 05:13:23 +00:00
Maksim Pechnikov 5c0f646cef fix validate_page_url 2019-06-26 06:27:17 +03:00
Maksim Pechnikov 4ad15ad2a9 add ignore hosts and TLDs for rich_media 2019-06-25 22:25:37 +03:00
Maksim Pechnikov 0276cf5a02 fix validate_url for private ip 2019-06-25 17:44:24 +03:00
lain 0e415921cd Rich Media Parser: Do not return just a title if nothing else is there. 2019-06-22 16:22:59 +02:00
lain 58c4d5312b Revert "Revert "Merge branch 'fix/ogp-title' into 'develop'""
This reverts commit b6af80f769.
2019-06-22 15:12:57 +02:00
feld b6af80f769 Revert "Merge branch 'fix/ogp-title' into 'develop'"
This reverts merge request !1277
2019-06-21 11:36:32 +00:00