Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop

This commit is contained in:
sadposter 2019-07-06 18:41:06 +01:00
commit ed709aec2c
106 changed files with 842 additions and 400 deletions

View file

@ -35,6 +35,7 @@ docs-build:
- develop@pleroma/pleroma
variables:
MIX_ENV: dev
PLEROMA_BUILD_ENV: prod
script:
- mix deps.get
- mix compile

View file

@ -3,6 +3,23 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## [Unreleased]
### Added
- MRF: Support for priming the mediaproxy cache (`Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy`)
- Mastodon API: Support for the [`tagged` filter](https://github.com/tootsuite/mastodon/pull/9755) in [`GET /api/v1/accounts/:id/statuses`](https://docs.joinmastodon.org/api/rest/accounts/#get-api-v1-accounts-id-statuses)
- Admin API: Return users' tags when querying reports
- Admin API: Return avatar and display name when querying users
### Fixed
- Not being able to pin unlisted posts
- Mastodon API: Handling of search timeouts (`/api/v1/search` and `/api/v2/search`)
### Changed
- Configuration: Filter.AnonymizeFilename added ability to retain file extension with custom text
### Changed
- NodeInfo: Return `skipThreadContainment` in `metadata` for the `skip_thread_containment` option
## [1.0.0] - 2019-06-29
### Security
- Mastodon API: Fix display names not being sanitized

View file

@ -38,7 +38,9 @@ Authentication is required and the user must be an admin.
"moderator": bool
},
"local": bool,
"tags": array
"tags": array,
"avatar": string,
"display_name": string
},
...
]
@ -331,6 +333,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
"pleroma": {},
"sensitive": false
},
"tags": ["force_unlisted"],
"statuses_count": 3,
"url": "https://pleroma.example.org/users/user",
"username": "user"
@ -366,6 +369,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
"pleroma": {},
"sensitive": false
},
"tags": ["force_unlisted"],
"statuses_count": 1,
"url": "https://pleroma.example.org/users/lain",
"username": "lain"

View file

@ -36,7 +36,7 @@ No specific configuration.
This filter replaces the filename (not the path) of an upload. For complete obfuscation, add
`Pleroma.Upload.Filter.Dedupe` before AnonymizeFilename.
* `text`: Text to replace filenames in links. If empty, `{random}.extension` will be used.
* `text`: Text to replace filenames in links. If empty, `{random}.extension` will be used. You can get the original filename extension by using `{extension}`, for example `custom-file-name.{extension}`.
## Pleroma.Emails.Mailer
* `adapter`: one of the mail adapters listed in [Swoosh readme](https://github.com/swoosh/swoosh#adapters), or `Swoosh.Adapters.Local` for in-memory mailbox.
@ -98,6 +98,7 @@ config :pleroma, Pleroma.Emails.Mailer,
* `Pleroma.Web.ActivityPub.MRF.RejectNonPublic`: Drops posts with non-public visibility settings (See ``:mrf_rejectnonpublic`` section)
* `Pleroma.Web.ActivityPub.MRF.EnsureRePrepended`: Rewrites posts to ensure that replies to posts with subjects do not have an identical subject and instead begin with re:.
* `Pleroma.Web.ActivityPub.MRF.AntiLinkSpamPolicy`: Rejects posts from likely spambots by rejecting posts from new users that contain links.
* `Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy`: Crawls attachments using their MediaProxy URLs so that the MediaProxy cache is primed.
* `public`: Makes the client API in authentificated mode-only except for user-profiles. Useful for disabling the Local Timeline and The Whole Known Network.
* `quarantined_instances`: List of ActivityPub instances where private(DMs, followers-only) activities will not be send.
* `managed_config`: Whenether the config for pleroma-fe is configured in this config or in ``static/config.json``
@ -279,7 +280,7 @@ config :pleroma, :mrf_subchain,
## Pleroma.Web.Endpoint
`Phoenix` endpoint configuration, all configuration options can be viewed [here](https://hexdocs.pm/phoenix/Phoenix.Endpoint.html#module-dynamic-configuration), only common options are listed here
* `http` - a list containing http protocol configuration, all configuration options can be viewed [here](https://hexdocs.pm/plug_cowboy/Plug.Cowboy.html#module-options), only common options are listed here
* `http` - a list containing http protocol configuration, all configuration options can be viewed [here](https://hexdocs.pm/plug_cowboy/Plug.Cowboy.html#module-options), only common options are listed here. For deployment using docker, you need to set this to `[ip: {0,0,0,0}, port: 4000]` to make pleroma accessible from other containers (such as your nginx server).
- `ip` - a tuple consisting of 4 integers
- `port`
* `url` - a list containing the configuration for generating urls, accepts

View file

@ -203,12 +203,12 @@ sudo -Hu pleroma MIX_ENV=prod mix pleroma.user new <username> <your@emailaddress
#### Further reading
* [Admin tasks](Admin tasks)
* [Backup your instance](Backup-your-instance)
* [Configuration tips](General tips for customizing pleroma fe)
* [Hardening your instance](Hardening-your-instance)
* [How to activate mediaproxy](How-to-activate-mediaproxy)
* [Small Pleroma-FE customizations](Small customizations)
* [Updating your instance](Updating-your-instance)
* [Backup your instance](backup.html)
* [Configuration tips](general-tips-for-customizing-pleroma-fe.html)
* [Hardening your instance](hardening.html)
* [How to activate mediaproxy](howto_mediaproxy.html)
* [Small Pleroma-FE customizations](small_customizations.html)
* [Updating your instance](updating.html)
## Questions

View file

@ -201,12 +201,12 @@ sudo -Hu pleroma MIX_ENV=prod mix pleroma.user new <username> <your@emailaddress
#### Further reading
* [Admin tasks](Admin tasks)
* [Backup your instance](Backup-your-instance)
* [Configuration tips](General tips for customizing pleroma fe)
* [Hardening your instance](Hardening-your-instance)
* [How to activate mediaproxy](How-to-activate-mediaproxy)
* [Small Pleroma-FE customizations](Small customizations)
* [Updating your instance](Updating-your-instance)
* [Backup your instance](backup.html)
* [Configuration tips](general-tips-for-customizing-pleroma-fe.html)
* [Hardening your instance](hardening.html)
* [How to activate mediaproxy](howto_mediaproxy.html)
* [Small Pleroma-FE customizations](small_customizations.html)
* [Updating your instance](updating.html)
## Questions

View file

@ -265,12 +265,12 @@ sudo -Hu pleroma MIX_ENV=prod mix pleroma.user new <username> <your@emailaddress
#### Further reading
* [Admin tasks](Admin tasks)
* [Backup your instance](Backup-your-instance)
* [Configuration tips](General tips for customizing pleroma fe)
* [Hardening your instance](Hardening-your-instance)
* [How to activate mediaproxy](How-to-activate-mediaproxy)
* [Small Pleroma-FE customizations](Small customizations)
* [Updating your instance](Updating-your-instance)
* [Backup your instance](backup.html)
* [Configuration tips](general-tips-for-customizing-pleroma-fe.html)
* [Hardening your instance](hardening.html)
* [How to activate mediaproxy](howto_mediaproxy.html)
* [Small Pleroma-FE customizations](small_customizations.html)
* [Updating your instance](updating.html)
## Questions

View file

@ -191,12 +191,12 @@ sudo -Hu pleroma MIX_ENV=prod mix pleroma.user new <username> <your@emailaddress
#### Further reading
* [Admin tasks](Admin tasks)
* [Backup your instance](Backup-your-instance)
* [Configuration tips](General tips for customizing pleroma fe)
* [Hardening your instance](Hardening-your-instance)
* [How to activate mediaproxy](How-to-activate-mediaproxy)
* [Small Pleroma-FE customizations](Small customizations)
* [Updating your instance](Updating-your-instance)
* [Backup your instance](backup.html)
* [Configuration tips](general-tips-for-customizing-pleroma-fe.html)
* [Hardening your instance](hardening.html)
* [How to activate mediaproxy](howto_mediaproxy.html)
* [Small Pleroma-FE customizations](small_customizations.html)
* [Updating your instance](updating.html)
## Questions

View file

@ -180,9 +180,13 @@ mix set_moderator username [true|false]
#### コンフィギュレーションとカスタマイズ
* [Configuration tips](General tips for customizing pleroma fe)
* [Small Pleroma-FE customizations](Small customizations)
* [Admin tasks](Admin tasks)
* [Backup your instance](backup.html)
* [Configuration tips](general-tips-for-customizing-pleroma-fe.html)
* [Hardening your instance](hardening.html)
* [How to activate mediaproxy](howto_mediaproxy.html)
* [Small Pleroma-FE customizations](small_customizations.html)
* [Updating your instance](updating.html)
## 質問ある?

View file

@ -284,12 +284,12 @@ If you opted to allow sudo for the `pleroma` user but would like to remove the a
#### Further reading
* [Admin tasks](Admin tasks)
* [Backup your instance](Backup-your-instance)
* [Configuration tips](General tips for customizing pleroma fe)
* [Hardening your instance](Hardening-your-instance)
* [How to activate mediaproxy](How-to-activate-mediaproxy)
* [Small Pleroma-FE customizations](Small customizations)
* [Updating your instance](Updating-your-instance)
* [Backup your instance](backup.html)
* [Configuration tips](general-tips-for-customizing-pleroma-fe.html)
* [Hardening your instance](hardening.html)
* [How to activate mediaproxy](howto_mediaproxy.html)
* [Small Pleroma-FE customizations](small_customizations.html)
* [Updating your instance](updating.html)
## Questions

View file

@ -49,7 +49,7 @@ mkdir -p /var/lib/pleroma/static
chown -R pleroma /var/lib/pleroma
# If you use the local uploader with default settings your uploads should be located in `~pleroma/uploads`
mv ~pleroma/uploads /var/lib/pleroma/uploads
mv ~pleroma/uploads/* /var/lib/pleroma/uploads
# If you have created the custom public files directory with default settings it should be located in `~pleroma/instance/static`
mv ~pleroma/instance/static /var/lib/pleroma/static
@ -122,13 +122,15 @@ su pleroma -s $SHELL -lc "./bin/pleroma stop"
## Setting up a system service
OTP releases have different service files than from-source installs so they need to be copied over again.
**Warning:** The service files assume pleroma user's home directory is `/opt/pleroma`, please make sure all paths fit your installation.
Debian/Ubuntu:
```sh
# Copy the service into a proper directory
cp ~pleroma/installation/pleroma.service /etc/systemd/system/pleroma.service
# Reload service files
systemctl reload-daemon
systemctl daemon-reload
# Reenable pleroma to start on boot
systemctl reenable pleroma

View file

@ -207,7 +207,7 @@ certbot renew --cert-name yourinstance.tld --webroot -w /var/lib/letsencrypt/ --
# Add it to the daily cron
echo '#!/bin/sh
certbot renew --cert-name yourinstance.tld --webroot -w /var/lib/letsencrypt/ --dry-run --post-hook "systemctl reload nginx"
certbot renew --cert-name yourinstance.tld --webroot -w /var/lib/letsencrypt/ --post-hook "systemctl reload nginx"
' > /etc/cron.daily/renew-pleroma-cert
chmod +x /etc/cron.daily/renew-pleroma-cert
@ -228,7 +228,7 @@ certbot renew --cert-name yourinstance.tld --webroot -w /var/lib/letsencrypt/ --
# Add it to the daily cron
echo '#!/bin/sh
certbot renew --cert-name yourinstance.tld --webroot -w /var/lib/letsencrypt/ --dry-run --post-hook "rc-service nginx reload"
certbot renew --cert-name yourinstance.tld --webroot -w /var/lib/letsencrypt/ --post-hook "rc-service nginx reload"
' > /etc/periodic/daily/renew-pleroma-cert
chmod +x /etc/periodic/daily/renew-pleroma-cert

View file

@ -149,7 +149,7 @@ defmodule Mix.Tasks.Pleroma.Instance do
uploads_dir =
get_option(
options,
:upload_dir,
:uploads_dir,
"What directory should media uploads go in (when using the local uploader)?",
Pleroma.Config.get([Pleroma.Uploaders.Local, :uploads])
)

View file

@ -38,7 +38,7 @@ defmodule Pleroma.Config do
def put([parent_key | keys], value) do
parent =
Application.get_env(:pleroma, parent_key)
Application.get_env(:pleroma, parent_key, [])
|> put_in(keys, value)
Application.put_env(:pleroma, parent_key, parent)

View file

@ -10,10 +10,19 @@ defmodule Pleroma.Upload.Filter.AnonymizeFilename do
"""
@behaviour Pleroma.Upload.Filter
def filter(upload) do
extension = List.last(String.split(upload.name, "."))
name = Pleroma.Config.get([__MODULE__, :text], random(extension))
{:ok, %Pleroma.Upload{upload | name: name}}
alias Pleroma.Config
alias Pleroma.Upload
def filter(%Upload{name: name} = upload) do
extension = List.last(String.split(name, "."))
name = predefined_name(extension) || random(extension)
{:ok, %Upload{upload | name: name}}
end
@spec predefined_name(String.t()) :: String.t() | nil
defp predefined_name(extension) do
with name when not is_nil(name) <- Config.get([__MODULE__, :text]),
do: String.replace(name, "{extension}", extension)
end
defp random(extension) do

View file

@ -836,15 +836,12 @@ defmodule Pleroma.User do
def mutes?(nil, _), do: false
def mutes?(user, %{ap_id: ap_id}), do: Enum.member?(user.info.mutes, ap_id)
def blocks?(user, %{ap_id: ap_id}) do
blocks = user.info.blocks
domain_blocks = user.info.domain_blocks
def blocks?(%User{info: info} = _user, %{ap_id: ap_id}) do
blocks = info.blocks
domain_blocks = info.domain_blocks
%{host: host} = URI.parse(ap_id)
Enum.member?(blocks, ap_id) ||
Enum.any?(domain_blocks, fn domain ->
host == domain
end)
Enum.member?(blocks, ap_id) || Enum.any?(domain_blocks, &(&1 == host))
end
def subscribed_to?(user, %{ap_id: ap_id}) do

View file

@ -43,6 +43,8 @@ defmodule Pleroma.User.Search do
defp search_query(query_string, for_user, following) do
for_user
|> base_query(following)
|> filter_blocked_user(for_user)
|> filter_blocked_domains(for_user)
|> search_subqueries(query_string)
|> union_subqueries
|> distinct_query()
@ -55,6 +57,25 @@ defmodule Pleroma.User.Search do
defp base_query(_user, false), do: User
defp base_query(user, true), do: User.get_followers_query(user)
defp filter_blocked_user(query, %User{info: %{blocks: blocks}})
when length(blocks) > 0 do
from(q in query, where: not (q.ap_id in ^blocks))
end
defp filter_blocked_user(query, _), do: query
defp filter_blocked_domains(query, %User{info: %{domain_blocks: domain_blocks}})
when length(domain_blocks) > 0 do
domains = Enum.join(domain_blocks, ",")
from(
q in query,
where: fragment("substring(ap_id from '.*://([^/]*)') NOT IN (?)", ^domains)
)
end
defp filter_blocked_domains(query, _), do: query
defp paginate(query, limit, offset) do
from(q in query, limit: ^limit, offset: ^offset)
end

View file

@ -0,0 +1,56 @@
# Pleroma: A lightweight social networking server
# Copyright © 2019 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy do
@moduledoc "Preloads any attachments in the MediaProxy cache by prefetching them"
@behaviour Pleroma.Web.ActivityPub.MRF
alias Pleroma.HTTP
alias Pleroma.Web.MediaProxy
require Logger
@hackney_options [
pool: :media,
recv_timeout: 10_000
]
def perform(:prefetch, url) do
Logger.info("Prefetching #{inspect(url)}")
url
|> MediaProxy.url()
|> HTTP.get([], adapter: @hackney_options)
end
def perform(:preload, %{"object" => %{"attachment" => attachments}} = _message) do
Enum.each(attachments, fn
%{"url" => url} when is_list(url) ->
url
|> Enum.each(fn
%{"href" => href} ->
PleromaJobQueue.enqueue(:background, __MODULE__, [:prefetch, href])
x ->
Logger.debug("Unhandled attachment URL object #{inspect(x)}")
end)
x ->
Logger.debug("Unhandled attachment #{inspect(x)}")
end)
end
@impl true
def filter(
%{"type" => "Create", "object" => %{"attachment" => attachments} = _object} = message
)
when is_list(attachments) and length(attachments) > 0 do
PleromaJobQueue.enqueue(:background, __MODULE__, [:preload, message])
{:ok, message}
end
@impl true
def filter(message), do: {:ok, message}
end

View file

@ -5,8 +5,11 @@
defmodule Pleroma.Web.AdminAPI.AccountView do
use Pleroma.Web, :view
alias Pleroma.HTML
alias Pleroma.User
alias Pleroma.User.Info
alias Pleroma.Web.AdminAPI.AccountView
alias Pleroma.Web.MediaProxy
def render("index.json", %{users: users, count: count, page_size: page_size}) do
%{
@ -17,9 +20,14 @@ defmodule Pleroma.Web.AdminAPI.AccountView do
end
def render("show.json", %{user: user}) do
avatar = User.avatar_url(user) |> MediaProxy.url()
display_name = HTML.strip_tags(user.name || user.nickname)
%{
"id" => user.id,
"avatar" => avatar,
"nickname" => user.nickname,
"display_name" => display_name,
"deactivated" => user.info.deactivated,
"local" => user.local,
"roles" => Info.roles(user.info),

View file

@ -8,7 +8,6 @@ defmodule Pleroma.Web.AdminAPI.ReportView do
alias Pleroma.HTML
alias Pleroma.User
alias Pleroma.Web.CommonAPI.Utils
alias Pleroma.Web.MastodonAPI.AccountView
alias Pleroma.Web.MastodonAPI.StatusView
def render("index.json", %{reports: reports}) do
@ -38,12 +37,19 @@ defmodule Pleroma.Web.AdminAPI.ReportView do
%{
id: report.id,
account: AccountView.render("account.json", %{user: account}),
actor: AccountView.render("account.json", %{user: user}),
account: merge_account_views(account),
actor: merge_account_views(user),
content: content,
created_at: created_at,
statuses: StatusView.render("index.json", %{activities: statuses, as: :activity}),
state: report.data["state"]
}
end
defp merge_account_views(%User{} = user) do
Pleroma.Web.MastodonAPI.AccountView.render("account.json", %{user: user})
|> Map.merge(Pleroma.Web.AdminAPI.AccountView.render("show.json", %{user: user}))
end
defp merge_account_views(_), do: %{}
end

View file

@ -11,6 +11,7 @@ defmodule Pleroma.Web.CommonAPI do
alias Pleroma.User
alias Pleroma.Web.ActivityPub.ActivityPub
alias Pleroma.Web.ActivityPub.Utils
alias Pleroma.Web.ActivityPub.Visibility
import Pleroma.Web.CommonAPI.Utils
@ -284,12 +285,11 @@ defmodule Pleroma.Web.CommonAPI do
},
object: %Object{
data: %{
"to" => object_to,
"type" => "Note"
}
}
} = activity <- get_by_id_or_ap_id(id_or_ap_id),
true <- Enum.member?(object_to, "https://www.w3.org/ns/activitystreams#Public"),
true <- Visibility.is_public?(activity),
%{valid?: true} = info_changeset <-
User.Info.add_pinnned_activity(user.info, activity),
changeset <-

View file

@ -356,6 +356,10 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
def user_statuses(%{assigns: %{user: reading_user}} = conn, params) do
with %User{} = user <- User.get_cached_by_id(params["id"]) do
params =
params
|> Map.put("tag", params["tagged"])
activities = ActivityPub.fetch_user_activities(user, reading_user, params)
conn

View file

@ -17,8 +17,8 @@ defmodule Pleroma.Web.MastodonAPI.SearchController do
plug(Pleroma.Plugs.RateLimiter, :search when action in [:search, :search2, :account_search])
def search2(%{assigns: %{user: user}} = conn, %{"q" => query} = params) do
accounts = User.search(query, search_options(params, user))
statuses = Activity.search(user, query)
accounts = with_fallback(fn -> User.search(query, search_options(params, user)) end, [])
statuses = with_fallback(fn -> Activity.search(user, query) end, [])
tags_path = Web.base_url() <> "/tag/"
tags =
@ -40,8 +40,8 @@ defmodule Pleroma.Web.MastodonAPI.SearchController do
end
def search(%{assigns: %{user: user}} = conn, %{"q" => query} = params) do
accounts = User.search(query, search_options(params, user))
statuses = Activity.search(user, query)
accounts = with_fallback(fn -> User.search(query, search_options(params, user)) end, [])
statuses = with_fallback(fn -> Activity.search(user, query) end, [])
tags =
query
@ -76,4 +76,14 @@ defmodule Pleroma.Web.MastodonAPI.SearchController do
for_user: user
]
end
defp with_fallback(f, fallback) do
try do
f.()
rescue
error ->
Logger.error("#{__MODULE__} search error: #{inspect(error)}")
fallback
end
end
end

View file

@ -162,7 +162,8 @@ defmodule Pleroma.Web.Nodeinfo.NodeinfoController do
accountActivationRequired: Config.get([:instance, :account_activation_required], false),
invitesEnabled: Config.get([:instance, :invites_enabled], false),
features: features,
restrictedNicknames: Config.get([Pleroma.User, :restricted_nicknames])
restrictedNicknames: Config.get([Pleroma.User, :restricted_nicknames]),
skipThreadContainment: Config.get([:instance, :skip_thread_containment], false)
}
}
end

55
mix.exs
View file

@ -174,10 +174,14 @@ defmodule Pleroma.Mixfile do
# Builds a version string made of:
# * the application version
# * a pre-release if ahead of the tag: the describe string (-count-commithash)
# * build info:
# * branch name
# * build metadata:
# * a build name if `PLEROMA_BUILD_NAME` or `:pleroma, :build_name` is defined
# * the mix environment if different than prod
defp version(version) do
identifier_filter = ~r/[^0-9a-z\-]+/i
# Pre-release version, denoted from patch version with a hyphen
{git_tag, git_pre_release} =
with {tag, 0} <-
System.cmd("git", ["describe", "--tags", "--abbrev=0"], stderr_to_stdout: true),
@ -198,6 +202,19 @@ defmodule Pleroma.Mixfile do
)
end
# Branch name as pre-release version component, denoted with a dot
branch_name =
with {branch_name, 0} <- System.cmd("git", ["rev-parse", "--abbrev-ref", "HEAD"]),
branch_name <- System.get_env("PLEROMA_BUILD_BRANCH") || branch_name,
true <- branch_name != "master" do
branch_name =
branch_name
|> String.trim()
|> String.replace(identifier_filter, "-")
"." <> branch_name
end
build_name =
cond do
name = Application.get_env(:pleroma, :build_name) -> name
@ -206,28 +223,26 @@ defmodule Pleroma.Mixfile do
end
env_name = if Mix.env() != :prod, do: to_string(Mix.env())
env_override = System.get_env("PLEROMA_BUILD_ENV")
build =
[build_name, env_name]
|> Enum.filter(fn string -> string && string != "" end)
|> Enum.join("-")
|> (fn
"" -> nil
string -> "+" <> string
end).()
branch_name =
with {branch_name, 0} <- System.cmd("git", ["rev-parse", "--abbrev-ref", "HEAD"]),
branch_name <- System.get_env("PLEROMA_BUILD_BRANCH") || branch_name,
true <- branch_name != "master" do
branch_name =
String.trim(branch_name)
|> String.replace(~r/[^0-9a-z\-\.]+/i, "-")
"-" <> branch_name
env_name =
case env_override do
nil -> env_name
env_override when env_override in ["", "prod"] -> nil
env_override -> env_override
end
[version, git_pre_release, branch_name, build]
# Build metadata, denoted with a plus sign
build_metadata =
[build_name, env_name]
|> Enum.filter(fn string -> string && string != "" end)
|> Enum.join(".")
|> (fn
"" -> nil
string -> "+" <> String.replace(string, identifier_filter, "-")
end).()
[version, git_pre_release, branch_name, build_metadata]
|> Enum.filter(fn string -> string && string != "" end)
|> Enum.join()
end

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.CreatePleroma.User do
use Ecto.Migration
def change do
create table(:users) do
create_if_not_exists table(:users) do
add :email, :string
add :password_hash, :string
add :name, :string

View file

@ -2,13 +2,13 @@ defmodule Pleroma.Repo.Migrations.CreatePleroma.Activity do
use Ecto.Migration
def change do
create table(:activities) do
create_if_not_exists table(:activities) do
add :data, :map
timestamps()
end
create index(:activities, [:data], using: :gin)
create_if_not_exists index(:activities, [:data], using: :gin)
end
end

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.CreatePleroma.Object do
use Ecto.Migration
def change do
create table(:objects) do
create_if_not_exists table(:objects) do
add :data, :map
timestamps()

View file

@ -2,6 +2,6 @@ defmodule Pleroma.Repo.Migrations.AddIndexToObjects do
use Ecto.Migration
def change do
create index(:objects, [:data], using: :gin)
create_if_not_exists index(:objects, [:data], using: :gin)
end
end

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.AddUniqueIndexToEmailAndNickname do
use Ecto.Migration
def change do
create unique_index(:users, [:email])
create unique_index(:users, [:nickname])
create_if_not_exists unique_index(:users, [:email])
create_if_not_exists unique_index(:users, [:nickname])
end
end

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.CreateWebsubServerSubscription do
use Ecto.Migration
def change do
create table(:websub_server_subscriptions) do
create_if_not_exists table(:websub_server_subscriptions) do
add :topic, :string
add :callback, :string
add :secret, :string

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.CreateWebsubClientSubscription do
use Ecto.Migration
def change do
create table(:websub_client_subscriptions) do
create_if_not_exists table(:websub_client_subscriptions) do
add :topic, :string
add :secret, :string
add :valid_until, :naive_datetime_usec

View file

@ -1,10 +1,12 @@
defmodule Pleroma.Repo.Migrations.AddIdContraintsToActivitiesAndObjectsPartTwo do
use Ecto.Migration
def change do
def up do
drop_if_exists index(:objects, ["(data->>\"id\")"], name: :objects_unique_apid_index)
drop_if_exists index(:activities, ["(data->>\"id\")"], name: :activities_unique_apid_index)
create unique_index(:objects, ["(data->>'id')"], name: :objects_unique_apid_index)
create unique_index(:activities, ["(data->>'id')"], name: :activities_unique_apid_index)
create_if_not_exists unique_index(:objects, ["(data->>'id')"], name: :objects_unique_apid_index)
create_if_not_exists unique_index(:activities, ["(data->>'id')"], name: :activities_unique_apid_index)
end
def down, do: :ok
end

View file

@ -6,6 +6,6 @@ defmodule Pleroma.Repo.Migrations.AddLocalFieldToActivities do
add :local, :boolean, default: true
end
create index(:activities, [:local])
create_if_not_exists index(:activities, [:local])
end
end

View file

@ -2,6 +2,6 @@ defmodule Pleroma.Repo.Migrations.AddUniqueIndexToAPID do
use Ecto.Migration
def change do
create unique_index(:users, [:ap_id])
create_if_not_exists unique_index(:users, [:ap_id])
end
end

View file

@ -1,19 +1,31 @@
defmodule Pleroma.Repo.Migrations.CaseInsensivtivity do
use Ecto.Migration
# Two-steps alters are intentional.
# When alter of 2 columns is done in a single operation,
# inconsistent failures happen because of index on `email` column.
def up do
execute ("create extension if not exists citext")
execute("create extension if not exists citext")
alter table(:users) do
modify :email, :citext
modify :nickname, :citext
modify(:email, :citext)
end
alter table(:users) do
modify(:nickname, :citext)
end
end
def down do
alter table(:users) do
modify :email, :string
modify :nickname, :string
modify(:email, :string)
end
execute ("drop extension if exists citext")
alter table(:users) do
modify(:nickname, :string)
end
execute("drop extension if exists citext")
end
end

View file

@ -1,9 +1,16 @@
defmodule Pleroma.Repo.Migrations.LongerBios do
use Ecto.Migration
def change do
def up do
alter table(:users) do
modify :bio, :text
end
end
def down do
alter table(:users) do
modify :bio, :string
end
end
end

View file

@ -2,6 +2,6 @@ defmodule Pleroma.Repo.Migrations.RemoveActivitiesIndex do
use Ecto.Migration
def change do
drop index(:activities, [:data])
drop_if_exists index(:activities, [:data])
end
end

View file

@ -3,6 +3,6 @@ defmodule Pleroma.Repo.Migrations.AddObjectActivityIndex do
def change do
# This was wrong, now a noop
# create index(:objects, ["(data->'object'->>'id')", "(data->>'type')"], name: :activities_create_objects_index)
# create_if_not_exists index(:objects, ["(data->'object'->>'id')", "(data->>'type')"], name: :activities_create_objects_index)
end
end

View file

@ -3,6 +3,6 @@ defmodule Pleroma.Repo.Migrations.AddObjectActivityIndexPartTwo do
def change do
drop_if_exists index(:objects, ["(data->'object'->>'id')", "(data->>'type')"], name: :activities_create_objects_index)
create index(:activities, ["(data->'object'->>'id')", "(data->>'type')"], name: :activities_create_objects_index)
create_if_not_exists index(:activities, ["(data->'object'->>'id')", "(data->>'type')"], name: :activities_create_objects_index)
end
end

View file

@ -2,6 +2,6 @@ defmodule Pleroma.Repo.Migrations.AddActorIndexToActivity do
use Ecto.Migration
def change do
create index(:activities, ["(data->>'actor')", "inserted_at desc"], name: :activities_actor_index)
create_if_not_exists index(:activities, ["(data->>'actor')", "inserted_at desc"], name: :activities_actor_index)
end
end

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.AddMastodonApps do
use Ecto.Migration
def change do
create table(:apps) do
create_if_not_exists table(:apps) do
add :client_name, :string
add :redirect_uris, :string
add :scopes, :string

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.CreateOAuthAuthorizations do
use Ecto.Migration
def change do
create table(:oauth_authorizations) do
create_if_not_exists table(:oauth_authorizations) do
add :app_id, references(:apps)
add :user_id, references(:users)
add :token, :string

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.CreateOAuthToken do
use Ecto.Migration
def change do
create table(:oauth_tokens) do
create_if_not_exists table(:oauth_tokens) do
add :app_id, references(:apps)
add :user_id, references(:users)
add :token, :string

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.CreateNotifications do
use Ecto.Migration
def change do
create table(:notifications) do
create_if_not_exists table(:notifications) do
add :user_id, references(:users, on_delete: :delete_all)
add :activity_id, references(:activities, on_delete: :delete_all)
add :seen, :boolean, default: false
@ -10,6 +10,6 @@ defmodule Pleroma.Repo.Migrations.CreateNotifications do
timestamps()
end
create index(:notifications, [:user_id])
create_if_not_exists index(:notifications, [:user_id])
end
end

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.CreatePasswordResetTokens do
use Ecto.Migration
def change do
create table(:password_reset_tokens) do
create_if_not_exists table(:password_reset_tokens) do
add :token, :string
add :user_id, references(:users)
add :used, :boolean, default: false

View file

@ -12,7 +12,7 @@ defmodule Pleroma.Repo.Migrations.AddActorToActivity do
end
def down do
drop index(:activities, [:actor, "id DESC NULLS LAST"])
drop_if_exists index(:activities, [:actor, "id DESC NULLS LAST"])
alter table(:activities) do
remove :actor
end

View file

@ -2,6 +2,6 @@ defmodule Pleroma.Repo.Migrations.AddLocalIndexToUser do
use Ecto.Migration
def change do
create index(:users, [:local])
create_if_not_exists index(:users, [:local])
end
end

View file

@ -6,6 +6,6 @@ defmodule Pleroma.Repo.Migrations.AddRecipientsToActivities do
add :recipients, {:array, :string}
end
create index(:activities, [:recipients], using: :gin)
create_if_not_exists index(:activities, [:recipients], using: :gin)
end
end

View file

@ -18,4 +18,6 @@ defmodule Pleroma.Repo.Migrations.FillRecipientsInActivities do
end)
end
end
def down, do: :ok
end

View file

@ -1,7 +1,7 @@
defmodule Pleroma.Repo.Migrations.MakeFollowingPostgresArray do
use Ecto.Migration
def change do
def up do
alter table(:users) do
add :following_temp, {:array, :string}
end
@ -15,4 +15,6 @@ defmodule Pleroma.Repo.Migrations.MakeFollowingPostgresArray do
end
rename table(:users), :following_temp, to: :following
end
def down, do: :ok
end

View file

@ -2,6 +2,6 @@ defmodule Pleroma.Repo.Migrations.DropLocalIndexOnActivities do
use Ecto.Migration
def change do
drop index(:users, [:local])
drop_if_exists index(:users, [:local])
end
end

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.ActuallyDropLocalIndex do
use Ecto.Migration
def change do
create index(:users, [:local])
create_if_not_exists index(:users, [:local])
drop_if_exists index("activities", :local)
end
end

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.CreateLists do
use Ecto.Migration
def change do
create table(:lists) do
create_if_not_exists table(:lists) do
add :user_id, references(:users, on_delete: :delete_all)
add :title, :string
add :following, {:array, :string}
@ -10,6 +10,6 @@ defmodule Pleroma.Repo.Migrations.CreateLists do
timestamps()
end
create index(:lists, [:user_id])
create_if_not_exists index(:lists, [:user_id])
end
end

View file

@ -2,6 +2,6 @@ defmodule Pleroma.Repo.Migrations.CreateUserTrigramIndex do
use Ecto.Migration
def change do
create index(:users, ["(nickname || name) gist_trgm_ops"], name: :users_trigram_index, using: :gist)
create_if_not_exists index(:users, ["(nickname || name) gist_trgm_ops"], name: :users_trigram_index, using: :gist)
end
end

View file

@ -2,6 +2,6 @@ defmodule Pleroma.Repo.Migrations.AddListFollowIndex do
use Ecto.Migration
def change do
create index(:lists, [:following])
create_if_not_exists index(:lists, [:following])
end
end

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.CreateUserInviteTokens do
use Ecto.Migration
def change do
create table(:user_invite_tokens) do
create_if_not_exists table(:user_invite_tokens) do
add :token, :string
add :used, :boolean, default: false

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.CreateFilters do
use Ecto.Migration
def change do
create table(:filters) do
create_if_not_exists table(:filters) do
add :user_id, references(:users, on_delete: :delete_all)
add :filter_id, :integer
add :hide, :boolean
@ -14,7 +14,7 @@ defmodule Pleroma.Repo.Migrations.CreateFilters do
timestamps()
end
create index(:filters, [:user_id])
create index(:filters, [:phrase], where: "hide = true", name: :hided_phrases_index)
create_if_not_exists index(:filters, [:user_id])
create_if_not_exists index(:filters, [:phrase], where: "hide = true", name: :hided_phrases_index)
end
end

View file

@ -7,7 +7,7 @@ defmodule Pleroma.Repo.Migrations.AddRecipientsToAndCcFieldsToActivities do
add :recipients_cc, {:array, :string}
end
create index(:activities, [:recipients_to], using: :gin)
create index(:activities, [:recipients_cc], using: :gin)
create_if_not_exists index(:activities, [:recipients_to], using: :gin)
create_if_not_exists index(:activities, [:recipients_cc], using: :gin)
end
end

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.ActivitiesAddToCcIndices do
use Ecto.Migration
def change do
create index(:activities, ["(data->'to')"], name: :activities_to_index, using: :gin)
create index(:activities, ["(data->'cc')"], name: :activities_cc_index, using: :gin)
create_if_not_exists index(:activities, ["(data->'to')"], name: :activities_to_index, using: :gin)
create_if_not_exists index(:activities, ["(data->'cc')"], name: :activities_cc_index, using: :gin)
end
end

View file

@ -1,10 +1,17 @@
defmodule Pleroma.Repo.Migrations.RemoveRecipientsToAndCcFieldsFromActivities do
use Ecto.Migration
def change do
def up do
alter table(:activities) do
remove :recipients_to
remove :recipients_cc
end
end
def down do
alter table(:activities) do
add :recipients_to, {:array, :string}
add :recipients_cc, {:array, :string}
end
end
end

View file

@ -2,6 +2,6 @@ defmodule Pleroma.Repo.Migrations.UsersAddIsModeratorIndex do
use Ecto.Migration
def change do
create index(:users, ["(info->'is_moderator')"], name: :users_is_moderator_index, using: :gin)
create_if_not_exists index(:users, ["(info->'is_moderator')"], name: :users_is_moderator_index, using: :gin)
end
end

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.CreatePushSubscriptions do
use Ecto.Migration
def change do
create table("push_subscriptions") do
create_if_not_exists table("push_subscriptions") do
add :user_id, references("users", on_delete: :delete_all)
add :token_id, references("oauth_tokens", on_delete: :delete_all)
add :endpoint, :string
@ -13,6 +13,6 @@ defmodule Pleroma.Repo.Migrations.CreatePushSubscriptions do
timestamps()
end
create index("push_subscriptions", [:user_id, :token_id], unique: true)
create_if_not_exists index("push_subscriptions", [:user_id, :token_id], unique: true)
end
end

View file

@ -1,7 +1,9 @@
defmodule Pleroma.Repo.Migrations.AddUUIDExtension do
use Ecto.Migration
def change do
def up do
execute("create extension if not exists \"uuid-ossp\"")
end
def down, do: :ok
end

View file

@ -1,7 +1,9 @@
defmodule Pleroma.Repo.Migrations.AddUUIDsToUserInfo do
use Ecto.Migration
def change do
def up do
execute("update users set info = jsonb_set(info, '{\"id\"}', to_jsonb(uuid_generate_v4()))")
end
def down, do: :ok
end

View file

@ -6,6 +6,6 @@ defmodule Pleroma.Repo.Migrations.AddTagsToUsers do
add :tags, {:array, :string}
end
create index(:users, [:tags], using: :gin)
create_if_not_exists index(:users, [:tags], using: :gin)
end
end

View file

@ -12,7 +12,7 @@ defmodule Pleroma.Repo.Migrations.UsersAndActivitiesFlakeId do
# 4- update relation pkeys with the new ids
# 5- rename the temporary column to id
# 6- re-create the constraints
def change do
def up do
# Old serial int ids are transformed to 128bits with extra padding.
# The application (in `Pleroma.FlakeId`) handles theses IDs properly as integers; to keep compatibility
# with previously issued ids.
@ -75,6 +75,8 @@ defmodule Pleroma.Repo.Migrations.UsersAndActivitiesFlakeId do
stop_clippy_heartbeats(clippy)
end
def down, do: :ok
defp start_clippy_heartbeats() do
count = from(a in "activities", select: count(a.id)) |> Repo.one!

View file

@ -37,12 +37,12 @@ defmodule Pleroma.Repo.Migrations.AddVisibilityFunction do
end
def down do
drop(
drop_if_exists(
index(:activities, ["activity_visibility(actor, recipients, data)"],
name: :activities_visibility_index
)
)
execute("drop function activity_visibility(actor varchar, recipients varchar[], data jsonb)")
execute("drop function if exists activity_visibility(actor varchar, recipients varchar[], data jsonb)")
end
end

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.CreateUserFtsIndex do
use Ecto.Migration
def change do
create index(
create_if_not_exists index(
:users,
[
"""

View file

@ -4,7 +4,7 @@ defmodule Pleroma.Repo.Migrations.FixUserTrigramIndex do
def up do
drop_if_exists(index(:users, [], name: :users_trigram_index))
create(
create_if_not_exists(
index(:users, ["(trim(nickname || ' ' || coalesce(name, ''))) gist_trgm_ops"],
name: :users_trigram_index,
using: :gist
@ -15,7 +15,7 @@ defmodule Pleroma.Repo.Migrations.FixUserTrigramIndex do
def down do
drop_if_exists(index(:users, [], name: :users_trigram_index))
create(
create_if_not_exists(
index(:users, ["(nickname || name) gist_trgm_ops"], name: :users_trigram_index, using: :gist)
)
end

View file

@ -2,6 +2,6 @@ defmodule Pleroma.Repo.Migrations.UsersAddIsAdminIndex do
use Ecto.Migration
def change do
create(index(:users, ["(info->'is_admin')"], name: :users_is_admin_index, using: :gin))
create_if_not_exists(index(:users, ["(info->'is_admin')"], name: :users_is_admin_index, using: :gin))
end
end

View file

@ -2,14 +2,14 @@ defmodule Pleroma.Repo.Migrations.CreateInstances do
use Ecto.Migration
def change do
create table(:instances) do
create_if_not_exists table(:instances) do
add :host, :string
add :unreachable_since, :naive_datetime_usec
timestamps()
end
create unique_index(:instances, [:host])
create index(:instances, [:unreachable_since])
create_if_not_exists unique_index(:instances, [:host])
create_if_not_exists index(:instances, [:unreachable_since])
end
end

View file

@ -1,9 +1,11 @@
defmodule Pleroma.Repo.Migrations.FixInfoIds do
use Ecto.Migration
def change do
def up do
execute(
"update users set info = jsonb_set(info, '{id}', to_jsonb(uuid_generate_v4())) where info->'id' is null;"
)
end
def down, do: :ok
end

View file

@ -1,9 +1,15 @@
defmodule Pleroma.Repo.Migrations.ChangePushSubscriptionsVarchar do
use Ecto.Migration
def change do
def up do
alter table(:push_subscriptions) do
modify(:endpoint, :varchar)
end
end
def down do
alter table(:push_subscriptions) do
modify(:endpoint, :string)
end
end
end

View file

@ -19,7 +19,7 @@ defmodule Pleroma.Repo.Migrations.AddCorrectDMIndex do
end
def down do
drop(
drop_if_exists(
index(:activities, ["activity_visibility(actor, recipients, data)", "id DESC"],
name: :activities_visibility_index,
concurrently: true,

View file

@ -2,11 +2,11 @@ defmodule Pleroma.Repo.Migrations.CreateThreadMutes do
use Ecto.Migration
def change do
create table(:thread_mutes) do
create_if_not_exists table(:thread_mutes) do
add :user_id, references(:users, type: :uuid, on_delete: :delete_all)
add :context, :string
end
create unique_index(:thread_mutes, [:user_id, :context], name: :unique_index)
create_if_not_exists unique_index(:thread_mutes, [:user_id, :context], name: :unique_index)
end
end

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.CreateRegistrations do
use Ecto.Migration
def change do
create table(:registrations, primary_key: false) do
create_if_not_exists table(:registrations, primary_key: false) do
add :id, :uuid, primary_key: true
add :user_id, references(:users, type: :uuid, on_delete: :delete_all)
add :provider, :string
@ -12,7 +12,7 @@ defmodule Pleroma.Repo.Migrations.CreateRegistrations do
timestamps()
end
create unique_index(:registrations, [:provider, :uid])
create unique_index(:registrations, [:user_id, :provider, :uid])
create_if_not_exists unique_index(:registrations, [:provider, :uid])
create_if_not_exists unique_index(:registrations, [:user_id, :provider, :uid])
end
end

View file

@ -2,6 +2,6 @@ defmodule Pleroma.Repo.Migrations.CreateNotificationIdIndex do
use Ecto.Migration
def change do
create index(:notifications, ["id desc nulls last"])
create_if_not_exists index(:notifications, ["id desc nulls last"])
end
end

View file

@ -2,7 +2,7 @@ defmodule Pleroma.Repo.Migrations.CreateScheduledActivities do
use Ecto.Migration
def change do
create table(:scheduled_activities) do
create_if_not_exists table(:scheduled_activities) do
add(:user_id, references(:users, type: :uuid, on_delete: :delete_all))
add(:scheduled_at, :naive_datetime, null: false)
add(:params, :map, null: false)
@ -10,7 +10,7 @@ defmodule Pleroma.Repo.Migrations.CreateScheduledActivities do
timestamps()
end
create(index(:scheduled_activities, [:scheduled_at]))
create(index(:scheduled_activities, [:user_id]))
create_if_not_exists(index(:scheduled_activities, [:scheduled_at]))
create_if_not_exists(index(:scheduled_activities, [:user_id]))
end
end

View file

@ -2,8 +2,8 @@ defmodule Pleroma.Repo.Migrations.AddOauthTokenIndexes do
use Ecto.Migration
def change do
create(unique_index(:oauth_tokens, [:token]))
create(index(:oauth_tokens, [:app_id]))
create(index(:oauth_tokens, [:user_id]))
create_if_not_exists(unique_index(:oauth_tokens, [:token]))
create_if_not_exists(index(:oauth_tokens, [:app_id]))
create_if_not_exists(index(:oauth_tokens, [:user_id]))
end
end

View file

@ -6,12 +6,12 @@ defmodule Pleroma.Repo.Migrations.CreateConversations do
use Ecto.Migration
def change do
create table(:conversations) do
create_if_not_exists table(:conversations) do
add(:ap_id, :string, null: false)
timestamps()
end
create table(:conversation_participations) do
create_if_not_exists table(:conversation_participations) do
add(:user_id, references(:users, type: :uuid, on_delete: :delete_all))
add(:conversation_id, references(:conversations, on_delete: :delete_all))
add(:read, :boolean, default: false)
@ -19,8 +19,8 @@ defmodule Pleroma.Repo.Migrations.CreateConversations do
timestamps()
end
create index(:conversation_participations, [:conversation_id])
create unique_index(:conversation_participations, [:user_id, :conversation_id])
create unique_index(:conversations, [:ap_id])
create_if_not_exists index(:conversation_participations, [:conversation_id])
create_if_not_exists unique_index(:conversation_participations, [:user_id, :conversation_id])
create_if_not_exists unique_index(:conversations, [:ap_id])
end
end

View file

@ -2,6 +2,6 @@ defmodule Pleroma.Repo.Migrations.AddParticipationUpdatedAtIndex do
use Ecto.Migration
def change do
create index(:conversation_participations, ["updated_at desc"])
create_if_not_exists index(:conversation_participations, ["updated_at desc"])
end
end

View file

@ -2,6 +2,6 @@ defmodule Pleroma.Repo.Migrations.AddIndexOnUserInfoDeactivated do
use Ecto.Migration
def change do
create(index(:users, ["(info->'deactivated')"], name: :users_deactivated_index, using: :gin))
create_if_not_exists(index(:users, ["(info->'deactivated')"], name: :users_deactivated_index, using: :gin))
end
end

View file

@ -2,13 +2,13 @@ defmodule Pleroma.Repo.Migrations.CreateBookmarks do
use Ecto.Migration
def change do
create table(:bookmarks) do
create_if_not_exists table(:bookmarks) do
add(:user_id, references(:users, type: :uuid, on_delete: :delete_all))
add(:activity_id, references(:activities, type: :uuid, on_delete: :delete_all))
timestamps()
end
create(unique_index(:bookmarks, [:user_id, :activity_id]))
create_if_not_exists(unique_index(:bookmarks, [:user_id, :activity_id]))
end
end

View file

@ -6,7 +6,7 @@ defmodule Pleroma.Repo.Migrations.MigrateOldBookmarks do
alias Pleroma.User
alias Pleroma.Repo
def change do
def up do
query =
from(u in User,
where: u.local == true,
@ -18,7 +18,7 @@ defmodule Pleroma.Repo.Migrations.MigrateOldBookmarks do
|> Enum.each(fn %{id: user_id, bookmarks: bookmarks} ->
Enum.each(bookmarks, fn ap_id ->
activity = Activity.get_create_by_object_ap_id(ap_id)
unless is_nil(activity), do: {:ok, _} = Bookmark.create(user_id, activity.id)
unless is_nil(activity), do: {:ok, _} = Bookmark.create(user_id, activity.id)
end)
end)
@ -26,4 +26,10 @@ defmodule Pleroma.Repo.Migrations.MigrateOldBookmarks do
remove(:bookmarks)
end
end
def down do
alter table(:users) do
add :bookmarks, {:array, :string}, null: false, default: []
end
end
end

View file

@ -3,6 +3,6 @@ defmodule Pleroma.Repo.Migrations.AddFTSIndexToObjects do
def change do
drop_if_exists index(:activities, ["(to_tsvector('english', data->'object'->>'content'))"], using: :gin, name: :activities_fts)
create index(:objects, ["(to_tsvector('english', data->>'content'))"], using: :gin, name: :objects_fts)
create_if_not_exists index(:objects, ["(to_tsvector('english', data->>'content'))"], using: :gin, name: :objects_fts)
end
end

View file

@ -2,6 +2,6 @@ defmodule Pleroma.Repo.Migrations.AddRefreshTokenIndexToToken do
use Ecto.Migration
def change do
create(unique_index(:oauth_tokens, [:refresh_token]))
create_if_not_exists(unique_index(:oauth_tokens, [:refresh_token]))
end
end

View file

@ -1,9 +1,15 @@
defmodule Pleroma.Repo.Migrations.ChangeHideColumnInFilterTable do
use Ecto.Migration
def change do
def up do
alter table(:filters) do
modify :hide, :boolean, default: false
end
end
def down do
alter table(:filters) do
modify :hide, :boolean
end
end
end

View file

@ -68,6 +68,6 @@ defmodule Pleroma.Repo.Migrations.AddThreadVisibilityFunction do
end
def down do
execute("drop function thread_visibility(actor varchar, activity_id varchar)")
execute("drop function if exists thread_visibility(actor varchar, activity_id varchar)")
end
end

View file

@ -2,12 +2,12 @@ defmodule Pleroma.Repo.Migrations.CreateConfig do
use Ecto.Migration
def change do
create table(:config) do
create_if_not_exists table(:config) do
add(:key, :string)
add(:value, :binary)
timestamps()
end
create(unique_index(:config, :key))
create_if_not_exists(unique_index(:config, :key))
end
end

View file

@ -1,10 +1,12 @@
defmodule Pleroma.Repo.Migrations.AddNonFollowsAndNonFollowersFieldsToNotificationSettings do
use Ecto.Migration
def change do
def up do
execute("""
update users set info = jsonb_set(info, '{notification_settings}', '{"local": true, "remote": true, "follows": true, "followers": true, "non_follows": true, "non_followers": true}')
where local=true
""")
end
def down, do: :ok
end

View file

@ -2,6 +2,6 @@ defmodule Pleroma.Repo.Migrations.AddIndexOnActivitiesLocal do
use Ecto.Migration
def change do
create(index("activities", [:local]))
create_if_not_exists(index("activities", [:local]))
end
end

View file

@ -3,6 +3,6 @@ defmodule Pleroma.Repo.Migrations.AddTagIndexToObjects do
def change do
drop_if_exists index(:activities, ["(data #> '{\"object\",\"tag\"}')"], using: :gin, name: :activities_tags)
create index(:objects, ["(data->'tag')"], using: :gin, name: :objects_tags)
create_if_not_exists index(:objects, ["(data->'tag')"], using: :gin, name: :objects_tags)
end
end

View file

@ -6,7 +6,7 @@ defmodule Pleroma.Repo.Migrations.AddGroupKeyToConfig do
add(:group, :string)
end
drop(unique_index("config", :key))
create(unique_index("config", [:group, :key]))
drop_if_exists(unique_index("config", :key))
create_if_not_exists(unique_index("config", [:group, :key]))
end
end

View file

@ -14,7 +14,7 @@ defmodule Pleroma.Repo.Migrations.AddFtsIndexToObjectsTwo do
return new;
end
$$ LANGUAGE plpgsql")
execute("create index objects_fts on objects using RUM (fts_content rum_tsvector_addon_ops, inserted_at) with (attach = 'inserted_at', to = 'fts_content');")
execute("create index if not exists objects_fts on objects using RUM (fts_content rum_tsvector_addon_ops, inserted_at) with (attach = 'inserted_at', to = 'fts_content');")
execute("CREATE TRIGGER tsvectorupdate BEFORE INSERT OR UPDATE ON objects
FOR EACH ROW EXECUTE PROCEDURE objects_fts_update()")
@ -23,12 +23,12 @@ defmodule Pleroma.Repo.Migrations.AddFtsIndexToObjectsTwo do
end
def down do
execute "drop index objects_fts"
execute "drop trigger tsvectorupdate on objects"
execute "drop function objects_fts_update()"
execute "drop index if exists objects_fts"
execute "drop trigger if exists tsvectorupdate on objects"
execute "drop function if exists objects_fts_update()"
alter table(:objects) do
remove(:fts_content, :tsvector)
end
create index(:objects, ["(to_tsvector('english', data->>'content'))"], using: :gin, name: :objects_fts)
create_if_not_exists index(:objects, ["(to_tsvector('english', data->>'content'))"], using: :gin, name: :objects_fts)
end
end

View file

@ -30,12 +30,15 @@ detect_flavour() {
detect_branch() {
version="$(cut -d' ' -f2 <"$RELEASE_ROOT"/releases/start_erl.data)"
branch="$(echo "$version" | cut -d'-' -f 4)"
# Expected format: major.minor.patch_version(-number_of_commits_ahead_of_tag-gcommit_hash).branch
branch="$(echo "$version" | cut -d'.' -f 4)"
if [ "$branch" = "develop" ]; then
echo "develop"
elif [ "$branch" = "" ]; then
echo "master"
else
# Note: branch name in version is of SemVer format and may only contain [0-9a-zA-Z-] symbols —
# if supporting releases for more branches, need to ensure they contain only these symbols.
echo "Releases are built only for master and develop branches" >&2
exit 1
fi

View file

@ -9,6 +9,12 @@ defmodule Pleroma.Tests.Helpers do
defmacro __using__(_opts) do
quote do
def collect_ids(collection) do
collection
|> Enum.map(& &1.id)
|> Enum.sort()
end
def refresh_record(%{id: id, __struct__: model} = _),
do: refresh_record(model, %{id: id})

View file

@ -0,0 +1,40 @@
defmodule Pleroma.Upload.Filter.AnonymizeFilenameTest do
use Pleroma.DataCase
alias Pleroma.Config
alias Pleroma.Upload
setup do
custom_filename = Config.get([Upload.Filter.AnonymizeFilename, :text])
on_exit(fn ->
Config.put([Upload.Filter.AnonymizeFilename, :text], custom_filename)
end)
upload_file = %Upload{
name: "an… image.jpg",
content_type: "image/jpg",
path: Path.absname("test/fixtures/image_tmp.jpg")
}
%{upload_file: upload_file}
end
test "it replaces filename on pre-defined text", %{upload_file: upload_file} do
Config.put([Upload.Filter.AnonymizeFilename, :text], "custom-file.png")
{:ok, %Upload{name: name}} = Upload.Filter.AnonymizeFilename.filter(upload_file)
assert name == "custom-file.png"
end
test "it replaces filename on pre-defined text expression", %{upload_file: upload_file} do
Config.put([Upload.Filter.AnonymizeFilename, :text], "custom-file.{extension}")
{:ok, %Upload{name: name}} = Upload.Filter.AnonymizeFilename.filter(upload_file)
assert name == "custom-file.jpg"
end
test "it replaces filename on random text", %{upload_file: upload_file} do
{:ok, %Upload{name: name}} = Upload.Filter.AnonymizeFilename.filter(upload_file)
assert <<_::bytes-size(14)>> <> ".jpg" = name
refute name == "an… image.jpg"
end
end

221
test/user_search_test.exs Normal file
View file

@ -0,0 +1,221 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.UserSearchTest do
alias Pleroma.Repo
alias Pleroma.User
use Pleroma.DataCase
import Pleroma.Factory
setup_all do
Tesla.Mock.mock_global(fn env -> apply(HttpRequestMock, :request, [env]) end)
:ok
end
describe "User.search" do
test "accepts limit parameter" do
Enum.each(0..4, &insert(:user, %{nickname: "john#{&1}"}))
assert length(User.search("john", limit: 3)) == 3
assert length(User.search("john")) == 5
end
test "accepts offset parameter" do
Enum.each(0..4, &insert(:user, %{nickname: "john#{&1}"}))
assert length(User.search("john", limit: 3)) == 3
assert length(User.search("john", limit: 3, offset: 3)) == 2
end
test "finds a user by full or partial nickname" do
user = insert(:user, %{nickname: "john"})
Enum.each(["john", "jo", "j"], fn query ->
assert user ==
User.search(query)
|> List.first()
|> Map.put(:search_rank, nil)
|> Map.put(:search_type, nil)
end)
end
test "finds a user by full or partial name" do
user = insert(:user, %{name: "John Doe"})
Enum.each(["John Doe", "JOHN", "doe", "j d", "j", "d"], fn query ->
assert user ==
User.search(query)
|> List.first()
|> Map.put(:search_rank, nil)
|> Map.put(:search_type, nil)
end)
end
test "finds users, preferring nickname matches over name matches" do
u1 = insert(:user, %{name: "lain", nickname: "nick1"})
u2 = insert(:user, %{nickname: "lain", name: "nick1"})
assert [u2.id, u1.id] == Enum.map(User.search("lain"), & &1.id)
end
test "finds users, considering density of matched tokens" do
u1 = insert(:user, %{name: "Bar Bar plus Word Word"})
u2 = insert(:user, %{name: "Word Word Bar Bar Bar"})
assert [u2.id, u1.id] == Enum.map(User.search("bar word"), & &1.id)
end
test "finds users, ranking by similarity" do
u1 = insert(:user, %{name: "lain"})
_u2 = insert(:user, %{name: "ean"})
u3 = insert(:user, %{name: "ebn", nickname: "lain@mastodon.social"})
u4 = insert(:user, %{nickname: "lain@pleroma.soykaf.com"})
assert [u4.id, u3.id, u1.id] == Enum.map(User.search("lain@ple", for_user: u1), & &1.id)
end
test "finds users, handling misspelled requests" do
u1 = insert(:user, %{name: "lain"})
assert [u1.id] == Enum.map(User.search("laiin"), & &1.id)
end
test "finds users, boosting ranks of friends and followers" do
u1 = insert(:user)
u2 = insert(:user, %{name: "Doe"})
follower = insert(:user, %{name: "Doe"})
friend = insert(:user, %{name: "Doe"})
{:ok, follower} = User.follow(follower, u1)
{:ok, u1} = User.follow(u1, friend)
assert [friend.id, follower.id, u2.id] --
Enum.map(User.search("doe", resolve: false, for_user: u1), & &1.id) == []
end
test "finds followers of user by partial name" do
u1 = insert(:user)
u2 = insert(:user, %{name: "Jimi"})
follower_jimi = insert(:user, %{name: "Jimi Hendrix"})
follower_lizz = insert(:user, %{name: "Lizz Wright"})
friend = insert(:user, %{name: "Jimi"})
{:ok, follower_jimi} = User.follow(follower_jimi, u1)
{:ok, _follower_lizz} = User.follow(follower_lizz, u2)
{:ok, u1} = User.follow(u1, friend)
assert Enum.map(User.search("jimi", following: true, for_user: u1), & &1.id) == [
follower_jimi.id
]
assert User.search("lizz", following: true, for_user: u1) == []
end
test "find local and remote users for authenticated users" do
u1 = insert(:user, %{name: "lain"})
u2 = insert(:user, %{name: "ebn", nickname: "lain@mastodon.social", local: false})
u3 = insert(:user, %{nickname: "lain@pleroma.soykaf.com", local: false})
results =
"lain"
|> User.search(for_user: u1)
|> Enum.map(& &1.id)
|> Enum.sort()
assert [u1.id, u2.id, u3.id] == results
end
test "find only local users for unauthenticated users" do
%{id: id} = insert(:user, %{name: "lain"})
insert(:user, %{name: "ebn", nickname: "lain@mastodon.social", local: false})
insert(:user, %{nickname: "lain@pleroma.soykaf.com", local: false})
assert [%{id: ^id}] = User.search("lain")
end
test "find only local users for authenticated users when `limit_to_local_content` is `:all`" do
Pleroma.Config.put([:instance, :limit_to_local_content], :all)
%{id: id} = insert(:user, %{name: "lain"})
insert(:user, %{name: "ebn", nickname: "lain@mastodon.social", local: false})
insert(:user, %{nickname: "lain@pleroma.soykaf.com", local: false})
assert [%{id: ^id}] = User.search("lain")
Pleroma.Config.put([:instance, :limit_to_local_content], :unauthenticated)
end
test "find all users for unauthenticated users when `limit_to_local_content` is `false`" do
Pleroma.Config.put([:instance, :limit_to_local_content], false)
u1 = insert(:user, %{name: "lain"})
u2 = insert(:user, %{name: "ebn", nickname: "lain@mastodon.social", local: false})
u3 = insert(:user, %{nickname: "lain@pleroma.soykaf.com", local: false})
results =
"lain"
|> User.search()
|> Enum.map(& &1.id)
|> Enum.sort()
assert [u1.id, u2.id, u3.id] == results
Pleroma.Config.put([:instance, :limit_to_local_content], :unauthenticated)
end
test "finds a user whose name is nil" do
_user = insert(:user, %{name: "notamatch", nickname: "testuser@pleroma.amplifie.red"})
user_two = insert(:user, %{name: nil, nickname: "lain@pleroma.soykaf.com"})
assert user_two ==
User.search("lain@pleroma.soykaf.com")
|> List.first()
|> Map.put(:search_rank, nil)
|> Map.put(:search_type, nil)
end
test "does not yield false-positive matches" do
insert(:user, %{name: "John Doe"})
Enum.each(["mary", "a", ""], fn query ->
assert [] == User.search(query)
end)
end
test "works with URIs" do
user = insert(:user)
results =
User.search("http://mastodon.example.org/users/admin", resolve: true, for_user: user)
result = results |> List.first()
user = User.get_cached_by_ap_id("http://mastodon.example.org/users/admin")
assert length(results) == 1
assert user == result |> Map.put(:search_rank, nil) |> Map.put(:search_type, nil)
end
test "excludes a blocked users from search result" do
user = insert(:user, %{nickname: "Bill"})
[blocked_user | users] = Enum.map(0..3, &insert(:user, %{nickname: "john#{&1}"}))
blocked_user2 =
insert(
:user,
%{nickname: "john awful", ap_id: "https://awful-and-rude-instance.com/user/bully"}
)
User.block_domain(user, "awful-and-rude-instance.com")
User.block(user, blocked_user)
account_ids = User.search("john", for_user: refresh_record(user)) |> collect_ids
assert account_ids == collect_ids(users)
refute Enum.member?(account_ids, blocked_user.id)
refute Enum.member?(account_ids, blocked_user2.id)
assert length(account_ids) == 3
end
end
end

View file

@ -1012,189 +1012,6 @@ defmodule Pleroma.UserTest do
end
end
describe "User.search" do
test "accepts limit parameter" do
Enum.each(0..4, &insert(:user, %{nickname: "john#{&1}"}))
assert length(User.search("john", limit: 3)) == 3
assert length(User.search("john")) == 5
end
test "accepts offset parameter" do
Enum.each(0..4, &insert(:user, %{nickname: "john#{&1}"}))
assert length(User.search("john", limit: 3)) == 3
assert length(User.search("john", limit: 3, offset: 3)) == 2
end
test "finds a user by full or partial nickname" do
user = insert(:user, %{nickname: "john"})
Enum.each(["john", "jo", "j"], fn query ->
assert user ==
User.search(query)
|> List.first()
|> Map.put(:search_rank, nil)
|> Map.put(:search_type, nil)
end)
end
test "finds a user by full or partial name" do
user = insert(:user, %{name: "John Doe"})
Enum.each(["John Doe", "JOHN", "doe", "j d", "j", "d"], fn query ->
assert user ==
User.search(query)
|> List.first()
|> Map.put(:search_rank, nil)
|> Map.put(:search_type, nil)
end)
end
test "finds users, preferring nickname matches over name matches" do
u1 = insert(:user, %{name: "lain", nickname: "nick1"})
u2 = insert(:user, %{nickname: "lain", name: "nick1"})
assert [u2.id, u1.id] == Enum.map(User.search("lain"), & &1.id)
end
test "finds users, considering density of matched tokens" do
u1 = insert(:user, %{name: "Bar Bar plus Word Word"})
u2 = insert(:user, %{name: "Word Word Bar Bar Bar"})
assert [u2.id, u1.id] == Enum.map(User.search("bar word"), & &1.id)
end
test "finds users, ranking by similarity" do
u1 = insert(:user, %{name: "lain"})
_u2 = insert(:user, %{name: "ean"})
u3 = insert(:user, %{name: "ebn", nickname: "lain@mastodon.social"})
u4 = insert(:user, %{nickname: "lain@pleroma.soykaf.com"})
assert [u4.id, u3.id, u1.id] == Enum.map(User.search("lain@ple", for_user: u1), & &1.id)
end
test "finds users, handling misspelled requests" do
u1 = insert(:user, %{name: "lain"})
assert [u1.id] == Enum.map(User.search("laiin"), & &1.id)
end
test "finds users, boosting ranks of friends and followers" do
u1 = insert(:user)
u2 = insert(:user, %{name: "Doe"})
follower = insert(:user, %{name: "Doe"})
friend = insert(:user, %{name: "Doe"})
{:ok, follower} = User.follow(follower, u1)
{:ok, u1} = User.follow(u1, friend)
assert [friend.id, follower.id, u2.id] --
Enum.map(User.search("doe", resolve: false, for_user: u1), & &1.id) == []
end
test "finds followers of user by partial name" do
u1 = insert(:user)
u2 = insert(:user, %{name: "Jimi"})
follower_jimi = insert(:user, %{name: "Jimi Hendrix"})
follower_lizz = insert(:user, %{name: "Lizz Wright"})
friend = insert(:user, %{name: "Jimi"})
{:ok, follower_jimi} = User.follow(follower_jimi, u1)
{:ok, _follower_lizz} = User.follow(follower_lizz, u2)
{:ok, u1} = User.follow(u1, friend)
assert Enum.map(User.search("jimi", following: true, for_user: u1), & &1.id) == [
follower_jimi.id
]
assert User.search("lizz", following: true, for_user: u1) == []
end
test "find local and remote users for authenticated users" do
u1 = insert(:user, %{name: "lain"})
u2 = insert(:user, %{name: "ebn", nickname: "lain@mastodon.social", local: false})
u3 = insert(:user, %{nickname: "lain@pleroma.soykaf.com", local: false})
results =
"lain"
|> User.search(for_user: u1)
|> Enum.map(& &1.id)
|> Enum.sort()
assert [u1.id, u2.id, u3.id] == results
end
test "find only local users for unauthenticated users" do
%{id: id} = insert(:user, %{name: "lain"})
insert(:user, %{name: "ebn", nickname: "lain@mastodon.social", local: false})
insert(:user, %{nickname: "lain@pleroma.soykaf.com", local: false})
assert [%{id: ^id}] = User.search("lain")
end
test "find only local users for authenticated users when `limit_to_local_content` is `:all`" do
Pleroma.Config.put([:instance, :limit_to_local_content], :all)
%{id: id} = insert(:user, %{name: "lain"})
insert(:user, %{name: "ebn", nickname: "lain@mastodon.social", local: false})
insert(:user, %{nickname: "lain@pleroma.soykaf.com", local: false})
assert [%{id: ^id}] = User.search("lain")
Pleroma.Config.put([:instance, :limit_to_local_content], :unauthenticated)
end
test "find all users for unauthenticated users when `limit_to_local_content` is `false`" do
Pleroma.Config.put([:instance, :limit_to_local_content], false)
u1 = insert(:user, %{name: "lain"})
u2 = insert(:user, %{name: "ebn", nickname: "lain@mastodon.social", local: false})
u3 = insert(:user, %{nickname: "lain@pleroma.soykaf.com", local: false})
results =
"lain"
|> User.search()
|> Enum.map(& &1.id)
|> Enum.sort()
assert [u1.id, u2.id, u3.id] == results
Pleroma.Config.put([:instance, :limit_to_local_content], :unauthenticated)
end
test "finds a user whose name is nil" do
_user = insert(:user, %{name: "notamatch", nickname: "testuser@pleroma.amplifie.red"})
user_two = insert(:user, %{name: nil, nickname: "lain@pleroma.soykaf.com"})
assert user_two ==
User.search("lain@pleroma.soykaf.com")
|> List.first()
|> Map.put(:search_rank, nil)
|> Map.put(:search_type, nil)
end
test "does not yield false-positive matches" do
insert(:user, %{name: "John Doe"})
Enum.each(["mary", "a", ""], fn query ->
assert [] == User.search(query)
end)
end
test "works with URIs" do
user = insert(:user)
results =
User.search("http://mastodon.example.org/users/admin", resolve: true, for_user: user)
result = results |> List.first()
user = User.get_cached_by_ap_id("http://mastodon.example.org/users/admin")
assert length(results) == 1
assert user == result |> Map.put(:search_rank, nil) |> Map.put(:search_type, nil)
end
end
test "auth_active?/1 works correctly" do
Pleroma.Config.put([:instance, :account_activation_required], true)

Some files were not shown because too many files have changed in this diff Show more