lemmy/crates/federate/src/federation_queue_state.rs
phiresky 375d9a2a3c
Persistent, performant, reliable federation queue (#3605)
* persistent activity queue

* fixes

* fixes

* make federation workers function callable from outside

* log federation instances

* dead instance detection not needed here

* taplo fmt

* split federate bin/lib

* minor fix

* better logging

* log

* create struct to hold cancellable task for readability

* use boxfuture for readability

* reset submodule

* fix

* fix lint

* swap

* remove json column, use separate array columns instead

* some review comments

* make worker a struct for readability

* minor readability

* add local filter to community follower view

* remove separate lemmy_federate entry point

* fix remaining duration

* address review comments mostly

* fix lint

* upgrade actitypub-fed to simpler interface

* fix sql format

* increase delays a bit

* fixes after merge

* remove selectable

* fix instance selectable

* add comment

* start federation based on latest id at the time

* rename federate process args

* dead instances in one query

* filter follow+report activities by local

* remove synchronous federation

remove activity sender queue

* lint

* fix federation tests by waiting for results to change

* fix fed test

* fix comment report

* wait some more

* Apply suggestions from code review

Co-authored-by: SorteKanin <sortekanin@gmail.com>

* fix most remaining tests

* wait until private messages

* fix community tests

* fix community tests

* move arg parse

* use instance_id instead of domain in federation_queue_state table

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: SorteKanin <sortekanin@gmail.com>
2023-09-09 12:25:03 -04:00

64 lines
1.8 KiB
Rust

use crate::util::ActivityId;
use anyhow::Result;
use chrono::{DateTime, TimeZone, Utc};
use diesel::prelude::*;
use diesel_async::RunQueryDsl;
use lemmy_db_schema::{
newtypes::InstanceId,
utils::{get_conn, DbPool},
};
#[derive(Queryable, Selectable, Insertable, AsChangeset, Clone)]
#[diesel(table_name = lemmy_db_schema::schema::federation_queue_state)]
#[diesel(check_for_backend(diesel::pg::Pg))]
pub struct FederationQueueState {
pub instance_id: InstanceId,
pub last_successful_id: ActivityId, // todo: i64
pub fail_count: i32,
pub last_retry: DateTime<Utc>,
}
impl FederationQueueState {
/// load state or return a default empty value
pub async fn load(
pool: &mut DbPool<'_>,
instance_id_: InstanceId,
) -> Result<FederationQueueState> {
use lemmy_db_schema::schema::federation_queue_state::dsl::{
federation_queue_state,
instance_id,
};
let conn = &mut get_conn(pool).await?;
Ok(
federation_queue_state
.filter(instance_id.eq(&instance_id_))
.select(FederationQueueState::as_select())
.get_result(conn)
.await
.optional()?
.unwrap_or(FederationQueueState {
instance_id: instance_id_,
fail_count: 0,
last_retry: Utc.timestamp_nanos(0),
last_successful_id: -1, // this value is set to the most current id for new instances
}),
)
}
pub async fn upsert(pool: &mut DbPool<'_>, state: &FederationQueueState) -> Result<()> {
use lemmy_db_schema::schema::federation_queue_state::dsl::{
federation_queue_state,
instance_id,
};
let conn = &mut get_conn(pool).await?;
state
.insert_into(federation_queue_state)
.on_conflict(instance_id)
.do_update()
.set(state)
.execute(conn)
.await?;
Ok(())
}
}