Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Fix 2FA/sign-in token sessions being valid after password change (#14802)

If someone tries logging in to an account and is prompted for a 2FA
code or sign-in token, even if the account's password or e-mail is
updated in the meantime, the session will show the prompt and allow
the login process to complete with a valid 2FA code or sign-in token

* Fix Move handler not being triggered when failing to fetch target (#15107)

When failing to fetch the target account, the ProcessingWorker fails
as expected, but since it hasn't cleared the `move_in_progress` flag,
the next attempt at processing skips the `Move` activity altogether.

This commit changes it to clear the flag when encountering any
unexpected error on fetching the target account. This is likely to
occur because, of, e.g., a timeout, when many instances query the
same actor at the same time.

* Fix slow distinct queries where grouped queries are faster (#15287)

About 2x speed-up on inboxes query

* Fix possible inconsistencies in tag search (#14906)

Do not downcase the queried tag before passing it to postgres when searching:
- tags are not downcased on creation
- `arel_table[:name].lower.matches(pattern)` generates an ILIKE anyway
- if Postgres and Rails happen to use different case-folding rules,
  downcasing before query but not before insertion may mean that some
  tags with some casings are not searchable

* Fix updating account counters when account_stat is not yet created (#15108)

* Fix account processing failing because of large collections (#15027)

Fixes #15025

* Fix downloading remote media files when server returns empty filename (#14867)

Fixes #14817

* Fix webfinger redirect handling in ResolveAccountService (#15187)

* Fix webfinger redirect handling in ResolveAccountService

ResolveAccountService#process_webfinger! handled a one-step webfinger
redirection, but only accepting the result if it matched the exact URI passed
as input, defeating the point of a redirection check.

Instead, use the same logic as in `ActivityPub::FetchRemoteAccountService`,
updating the resulting `acct:` URI with the result of the first webfinger
query.

* Add tests

* Remove dependency on unused and unmaintained http_parser.rb gem (#14574)

It seems that years ago, the “http” gem dependend on the “http_parser.rb” gem
(it now depends on the “http-parser” gem), and, still years ago, we pulled
it from git in order to benefit from a bugfix that wasn't released yet (#7467).

* Add tootctl maintenance fix-duplicates (#14860, #15201, #15264, #15349, #15359)

* Fix old migration script not being able to run if it fails midway (#15361)

* Fix old migration script not being able to run if it fails midway

Improve the robustness of a migration script likely to fail because of database
corruption so it can run again once database corruptions are fixed.

* Display a specific error message in case of index corruption

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Add support for latest HTTP Signatures spec draft

https://www.ietf.org/id/draft-ietf-httpbis-message-signatures-00.html

- add support for the “hs2019” signature algorithm (assumed to be equivalent
  to RSA-SHA256, since we do not have a mechanism to specify the algorithm
  within the key metadata yet)
- add support for (created) and (expires) pseudo-headers and related
  signature parameters, when using the hs2019 signature algorithm
- adjust default “headers” parameter while being backwards-compatible with
  previous implementation
- change the acceptable time window logic from 12 hours surrounding the “date”
  header to accepting signatures created up to 1 hour in the future and
  expiring up to 1 hour in the past (but only allowing expiration dates up to
  12 hours after the creation date)
  This doesn't conform with the current draft, as it doesn't permit accounting
  for clock skew.
  This, however, should be addressed in a next version of the draft:
  https://github.com/httpwg/http-extensions/pull/1235

* Add additional signature requirements

* Rewrite signature params parsing using Parslet

* Make apparent which signature algorithm Mastodon on verification failure

Mastodon uses RSASSA-PKCS1-v1_5, which is not recommended for new applications,
and new implementers may thus unknowingly use RSASSA-PSS.

* Add workaround for PeerTube's invalid signature header

The previous parser allowed incorrect Signature headers, such as
those produced by old versions of the `http-signature` node.js package,
and seemingly used by PeerTube.

This commit adds a workaround for that.

* Fix `signature_key_id` raising an exception

Previously, parsing failures would result in `signature_key_id` being nil,
but the parser changes made that result in an exception.

This commit changes the `signature_key_id` method to return `nil` in case
of parsing failures.

* Move extra HTTP signature helper methods to private methods

* Relax (request-target) requirement to (request-target) || digest

This lets requests from Plume work without lowering security significantly.

Follow-up to #14359

In the case of limited toots, the receiver may not be explicitly part of the
audience. If a specific user's inbox URI was specified, it makes sense to
dereference the toot from the corresponding user, instead of trying to find
someone in the explicit audience.

* use custom private boost icon for detail status

* only use className

* Add support for inlined objects in activity audience

* Add tests

Fixes #14523

* Fix contrast calculation for thumbnail color extraction

Luminance calculation was using 0-255 RGB values instead of 0-1 sRGB values,
leading to incorrectly-computed contrast values.

Since we use ColorDiff already, just use its XYZ colorspace conversion code
to get the value.

* Require at least 3:1 contrast for both accent and foreground colors

* Lower required contrast for the accent color

* Change content-type to be always computed from file data

Restore previous behavior, detecting the content-type isn't very
expensive, and some instances may serve files as application/octet-stream
regardless of their true type, making fetching media from them fail, while
it used to work pre-3.2.0.

* Add test

Fixes #14462

Fix #14668

There are edge cases where requests to certain hosts timeout when
using the vanilla HTTP.rb gem, which the goldfinger gem uses. Now
that we no longer need to support OStatus servers, webfinger logic
is so simple that there is no point encapsulating it in a gem, so
we can just use our own Request class. With that, we benefit from
more robust timeout code and IPv4/IPv6 resolution.

Fix #14091

* Add tests

* Fix handling of Reject Follow when a matching follow relationship exists

Regression from #12199

* Do not serve account actors at all in limited federation mode

When an account is fetched without a signature from an allowed instance,
return an error.

This isn't really an improvement in security, as the only information that was
previously returned was required protocol-level info, and the only personal bit
was the existence of the account. The existence of the account can still be
checked by issuing a webfinger query, as those are accepted without signatures.

However, this change makes it so that unallowed instances won't create account
records on their end when they find a reference to an unknown account.

The previous behavior of rendering a limited list of fields, instead of not
rendering the actor at all, was in order to prevent situations in which two
instances in Authorized Fetch mode or Limited Federation mode would fail to
reach each other because resolving an account would require a signed query…
from an account which can only be fetched with a signed query itself. However,
this should now be fine as fetching accounts is done by signing on behalf of
the special instance actor, which does not require any kind of valid signature
to be fetched.

* Fix tests

Fix regression from #14212

* New translations en.yml (Kazakh)
[ci skip]

* New translations en.json (Norwegian Nynorsk)
[ci skip]

* New translations simple_form.en.yml (Norwegian Nynorsk)
[ci skip]

* New translations en.yml (Latvian)
[ci skip]

* New translations simple_form.en.yml (Kazakh)
[ci skip]

* New translations en.json (Latvian)
[ci skip]

* New translations doorkeeper.en.yml (Estonian)
[ci skip]

* New translations simple_form.en.yml (Estonian)
[ci skip]

* New translations en.yml (Estonian)
[ci skip]

* New translations en.yml (Norwegian Nynorsk)
[ci skip]

* New translations en.yml (Slovak)
[ci skip]

* New translations doorkeeper.en.yml (Croatian)
[ci skip]

* New translations en.yml (Spanish, Argentina)
[ci skip]

* New translations doorkeeper.en.yml (Portuguese, Brazilian)
[ci skip]

* New translations en.json (Indonesian)
[ci skip]

* New translations simple_form.en.yml (Indonesian)
[ci skip]

* New translations doorkeeper.en.yml (Indonesian)
[ci skip]

* New translations simple_form.en.yml (Persian)
[ci skip]

* New translations doorkeeper.en.yml (Persian)
[ci skip]

* New translations en.json (Tamil)
[ci skip]

* New translations en.yml (Tamil)
[ci skip]

* New translations simple_form.en.yml (Tamil)
[ci skip]

* New translations doorkeeper.en.yml (Tamil)
[ci skip]

* New translations simple_form.en.yml (Spanish, Argentina)
[ci skip]

* New translations simple_form.en.yml (Croatian)
[ci skip]

* New translations doorkeeper.en.yml (Spanish, Argentina)
[ci skip]

* New translations en.json (Bengali)
[ci skip]

* New translations en.yml (Bengali)
[ci skip]

* New translations simple_form.en.yml (Bengali)
[ci skip]

* New translations en.json (Marathi)
[ci skip]

* New translations en.yml (Marathi)
[ci skip]

* New translations doorkeeper.en.yml (Marathi)
[ci skip]

* New translations en.json (Croatian)
[ci skip]

* New translations en.yml (Welsh)
[ci skip]

* New translations en.yml (Croatian)
[ci skip]

* New translations en.json (Welsh)
[ci skip]

* New translations simple_form.en.yml (Asturian)
[ci skip]

* New translations simple_form.en.yml (Welsh)
[ci skip]

* New translations en.yml (Taigi)
[ci skip]

* New translations en.yml (Corsican)
[ci skip]

* New translations simple_form.en.yml (Corsican)
[ci skip]

* New translations doorkeeper.en.yml (Corsican)
[ci skip]

* New translations doorkeeper.en.yml (Kabyle)
[ci skip]

* New translations en.json (Ido)
[ci skip]

* New translations en.yml (Ido)
[ci skip]

* New translations simple_form.en.yml (Ido)
[ci skip]

* New translations doorkeeper.en.yml (Ido)
[ci skip]

* New translations en.json (Taigi)
[ci skip]

* New translations doorkeeper.en.yml (Serbian (Latin))
[ci skip]

* New translations en.json (Silesian)
[ci skip]

* New translations en.yml (Silesian)
[ci skip]

* New translations en.json (Uyghur)
[ci skip]

* New translations en.yml (Uyghur)
[ci skip]

* New translations en.json (Sorani (Kurdish))
[ci skip]

* New translations en.yml (Sorani (Kurdish))
[ci skip]

* New translations simple_form.en.yml (Serbian (Latin))
[ci skip]

* New translations doorkeeper.en.yml (Welsh)
[ci skip]

* New translations en.yml (Breton)
[ci skip]

* New translations en.json (Esperanto)
[ci skip]

* New translations simple_form.en.yml (Esperanto)
[ci skip]

* New translations doorkeeper.en.yml (Esperanto)
[ci skip]

* New translations en.json (Chinese Traditional, Hong Kong)
[ci skip]

* New translations en.yml (Chinese Traditional, Hong Kong)
[ci skip]

* New translations simple_form.en.yml (Chinese Traditional, Hong Kong)
[ci skip]

* New translations doorkeeper.en.yml (Chinese Traditional, Hong Kong)
[ci skip]

* New translations en.json (Malayalam)
[ci skip]

* New translations en.yml (Malayalam)
[ci skip]

* New translations simple_form.en.yml (Malayalam)
[ci skip]

* New translations doorkeeper.en.yml (Malayalam)
[ci skip]

* New translations simple_form.en.yml (Breton)
[ci skip]

* New translations en.yml (Serbian (Latin))
[ci skip]

* New translations doorkeeper.en.yml (Breton)
[ci skip]

* New translations en.json (Kannada)
[ci skip]

* New translations en.yml (Kannada)
[ci skip]

* New translations en.json (Asturian)
[ci skip]

* New translations en.yml (Asturian)
[ci skip]

* New translations en.yml (Portuguese, Brazilian)
[ci skip]

* New translations doorkeeper.en.yml (Asturian)
[ci skip]

* New translations en.yml (Occitan)
[ci skip]

* New translations simple_form.en.yml (Occitan)
[ci skip]

* New translations doorkeeper.en.yml (Occitan)
[ci skip]

* New translations en.json (Serbian (Latin))
[ci skip]

* New translations simple_form.en.yml (Portuguese, Brazilian)
[ci skip]

* New translations doorkeeper.en.yml (Turkish)
[ci skip]

* New translations en.json (Portuguese, Brazilian)
[ci skip]

* New translations en.json (Georgian)
[ci skip]

* New translations doorkeeper.en.yml (Hungarian)
[ci skip]

* New translations en.yml (Armenian)
[ci skip]

* New translations simple_form.en.yml (Armenian)
[ci skip]

* New translations doorkeeper.en.yml (Armenian)
[ci skip]

* New translations en.yml (Italian)
[ci skip]

* New translations simple_form.en.yml (Italian)
[ci skip]

* New translations doorkeeper.en.yml (Italian)
[ci skip]

* New translations simple_form.en.yml (Japanese)
[ci skip]

* New translations doorkeeper.en.yml (Japanese)
[ci skip]

* New translations en.yml (Georgian)
[ci skip]

* New translations simple_form.en.yml (Hebrew)
[ci skip]

* New translations simple_form.en.yml (Georgian)
[ci skip]

* New translations doorkeeper.en.yml (Georgian)
[ci skip]

* New translations en.yml (Korean)
[ci skip]

* New translations simple_form.en.yml (Korean)
[ci skip]

* New translations doorkeeper.en.yml (Korean)
[ci skip]

* New translations en.json (Lithuanian)
[ci skip]

* New translations en.yml (Lithuanian)
[ci skip]

* New translations en.json (Macedonian)
[ci skip]

* New translations en.yml (Macedonian)
[ci skip]

* New translations doorkeeper.en.yml (Hebrew)
[ci skip]

* New translations en.yml (Hebrew)
[ci skip]

* New translations en.json (Occitan)
[ci skip]

* New translations en.json (Sardinian)
[ci skip]

* New translations en.yml (Sardinian)
[ci skip]

* New translations en.json (Slovenian)
[ci skip]

* New translations en.yml (Slovenian)
[ci skip]

* New translations en.yml (Esperanto)
[ci skip]

* New translations en.yml (Russian)
[ci skip]

* New translations en.json (Hebrew)
[ci skip]

* New translations en.json (Slovak)
[ci skip]

* New translations simple_form.en.yml (Russian)
[ci skip]

* New translations simple_form.en.yml (Ukrainian)
[ci skip]

* New translations en.yml (Hungarian)
[ci skip]

* New translations simple_form.en.yml (Hungarian)
[ci skip]

* New translations en.yml (Japanese)
[ci skip]

* New translations simple_form.en.yml (Chinese Simplified)
[ci skip]

* New translations en.yml (Persian)
[ci skip]

* New translations en.json (Ukrainian)
[ci skip]

* New translations en.yml (Ukrainian)
[ci skip]

* New translations simple_form.en.yml (Finnish)
[ci skip]

* New translations doorkeeper.en.yml (Finnish)
[ci skip]

* New translations en.json (Dutch)
[ci skip]

* New translations doorkeeper.en.yml (Icelandic)
[ci skip]

* New translations en.yml (Urdu (Pakistan))
[ci skip]

* New translations doorkeeper.en.yml (Swedish)
[ci skip]

* New translations en.json (Turkish)
[ci skip]

* New translations en.yml (Turkish)
[ci skip]

* New translations simple_form.en.yml (Turkish)
[ci skip]

* New translations en.yml (Indonesian)
[ci skip]

* New translations doorkeeper.en.yml (Ukrainian)
[ci skip]

* New translations en.yml (Chinese Traditional)
[ci skip]

* New translations simple_form.en.yml (Chinese Traditional)
[ci skip]

* New translations doorkeeper.en.yml (Chinese Traditional)
[ci skip]

* New translations en.json (Urdu (Pakistan))
[ci skip]

* New translations en.yml (Swedish)
[ci skip]

* New translations en.yml (Galician)
[ci skip]

* New translations simple_form.en.yml (Galician)
[ci skip]

* New translations doorkeeper.en.yml (Galician)
[ci skip]

* New translations en.json (Icelandic)
[ci skip]

* New translations en.yml (Icelandic)
[ci skip]

* New translations simple_form.en.yml (Icelandic)
[ci skip]

* New translations simple_form.en.yml (Swedish)
[ci skip]

* New translations en.json (Swedish)
[ci skip]

* New translations en.yml (Dutch)
[ci skip]

* New translations simple_form.en.yml (Portuguese)
[ci skip]

* New translations simple_form.en.yml (Dutch)
[ci skip]

* New translations doorkeeper.en.yml (Dutch)
[ci skip]

* New translations en.json (Norwegian)
[ci skip]

* New translations en.yml (Norwegian)
[ci skip]

* New translations simple_form.en.yml (Norwegian)
[ci skip]

* New translations doorkeeper.en.yml (Norwegian)
[ci skip]

* New translations en.json (Polish)
[ci skip]

* New translations en.yml (Polish)
[ci skip]

* New translations simple_form.en.yml (Polish)
[ci skip]

* New translations doorkeeper.en.yml (Polish)
[ci skip]

* New translations en.yml (Portuguese)
[ci skip]

* New translations doorkeeper.en.yml (Portuguese)
[ci skip]

* New translations doorkeeper.en.yml (Serbian (Cyrillic))
[ci skip]

* New translations doorkeeper.en.yml (Russian)
[ci skip]

* New translations simple_form.en.yml (Slovak)
[ci skip]

* New translations doorkeeper.en.yml (Slovak)
[ci skip]

* New translations simple_form.en.yml (Slovenian)
[ci skip]

* New translations doorkeeper.en.yml (Slovenian)
[ci skip]

* New translations en.yml (Albanian)
[ci skip]

* New translations simple_form.en.yml (Albanian)
[ci skip]

* New translations doorkeeper.en.yml (Albanian)
[ci skip]

* New translations en.json (Serbian (Cyrillic))
[ci skip]

* New translations en.yml (Serbian (Cyrillic))
[ci skip]

* New translations simple_form.en.yml (Serbian (Cyrillic))
[ci skip]

* New translations en.json (Portuguese, Brazilian)
[ci skip]

* New translations en.yml (Thai)
[ci skip]

* New translations en.json (French)
[ci skip]

* New translations en.json (Vietnamese)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* New translations en.json (Vietnamese)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* New translations doorkeeper.en.yml (Vietnamese)
[ci skip]

* New translations doorkeeper.en.yml (Vietnamese)
[ci skip]

* New translations simple_form.en.yml (Vietnamese)
[ci skip]

* New translations doorkeeper.en.yml (Vietnamese)
[ci skip]

* New translations en.yml (Chinese Simplified)
[ci skip]

* New translations en.yml (Chinese Simplified)
[ci skip]

* New translations en.yml (Chinese Simplified)
[ci skip]

* New translations en.yml (Chinese Simplified)
[ci skip]

* New translations en.json (Vietnamese)
[ci skip]

* New translations en.json (Vietnamese)
[ci skip]

* New translations en.json (Vietnamese)
[ci skip]

* New translations en.json (Vietnamese)
[ci skip]

* New translations en.json (Spanish)
[ci skip]

* New translations en.json (Vietnamese)
[ci skip]

* New translations en.json (Vietnamese)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* New translations en.json (Finnish)
[ci skip]

* New translations en.json (Finnish)
[ci skip]

* New translations en.yml (Finnish)
[ci skip]

* New translations en.yml (Finnish)
[ci skip]

* New translations en.yml (Finnish)
[ci skip]

* New translations simple_form.en.yml (Finnish)
[ci skip]

* New translations en.yml (Finnish)
[ci skip]

* New translations simple_form.en.yml (Finnish)
[ci skip]

* New translations en.yml (Finnish)
[ci skip]

* New translations en.yml (Finnish)
[ci skip]

* New translations en.yml (Finnish)
[ci skip]

* New translations en.yml (Finnish)
[ci skip]

* New translations en.json (Finnish)
[ci skip]

* New translations en.json (Finnish)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* New translations en.json (Vietnamese)
[ci skip]

* New translations en.json (Dutch)
[ci skip]

* New translations en.json (Dutch)
[ci skip]

* New translations en.yml (Dutch)
[ci skip]

* New translations en.json (Thai)
[ci skip]

* New translations devise.en.yml (Thai)
[ci skip]

* New translations en.json (Thai)
[ci skip]

* New translations en.json (Korean)
[ci skip]

* New translations en.json (Thai)
[ci skip]

* New translations en.yml (Thai)
[ci skip]

* New translations en.yml (Thai)
[ci skip]

* New translations en.yml (Thai)
[ci skip]

* New translations en.yml (Thai)
[ci skip]

* New translations en.json (Vietnamese)
[ci skip]

* New translations en.json (Spanish)
[ci skip]

* New translations en.yml (Spanish)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* New translations simple_form.en.yml (Vietnamese)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* New translations simple_form.en.yml (Vietnamese)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* New translations simple_form.en.yml (Vietnamese)
[ci skip]

* New translations en.yml (Vietnamese)
[ci skip]

* i18n-tasks normalize

* yarn manage:translations

* Fix normalization

* Change disabled retweet icon

* Hide fontawesome version of boost icon in both reduce-motion and no-reduce-motion

* Fix not handling Undo on some activity types when they aren't inlined

When receiving an Undo for a non-inlined activity, try looking it up in
database using the URI. The queries are ad-hoc because we don't have a global
index of object URIs, and not all activity types are stored in database with
an index on their URI.

Announces are just statuses, and have an index on URIs, so this check can
be done efficiently.

Accepts cannot be handled at all because we don't record their URI at any
point.

Follows don't have an index on URI, but they have an index on the issuing
account, which should make such queries largely manageable.

Likes don't have an index on URI, they have an index on the issuing account,
but the number of favs per account may be very high, so I decided not to
handle that.

Blocks don't have an index on URI, but they have an index on the issuing
account, which should make such queries largely manageable.

In all cases, if an Undo could not be handled properly, we call `delete_later!`
because that does not require us to know more than the URI of the undone
property.

* Add tests

* Make newer blocks overwrite older ones

Allows re-synchronizing block info by re-blocking and un-blocking again
when the original Undo Block has been lost.

* Add tests for some cachable responses

This only covers responses that we should have managed to make cachable
so far. It's not the case of all responses that should be cachable in
the end.

* Fix RSS feeds not being cachable

* Dereference object URIs in Create and Update messages

Fixes #14353

Signed-off-by: Thibaut Girka <thib@sitedethib.com>

* Refactor, and perform origin check *before* attempting to fetch object

Co-authored-by: Fire Demon <firedemon@creature.cafe>

* Changed the number of retries and rescued exceptions in ActivityPub::ProcessingWorker

* Remove RecordNotUnique from rescue

* Fix movie width and frame_rate returning nil

* Add StreamValidationError and raise

* Fix code style