IT Notes - mastodon

Aggressive caching for a Mastodon reverse proxy: what to cache, what to never cache, and why content negotiation will eventually betray you

Stefano Marinelli — Fri, 05 Jun 2026 08:44:00 +0000

I have written before about putting a cache in front of snac, and more recently about the HAProxy layer in front of FediMeteo. The general idea is always the same: the reverse proxy should absorb the repetitive, public work that has no business reaching the application server.

This post is the same idea applied to a much louder neighbour: a Mastodon instance. The instance is mastodon.bsd.cafe, the proxy is nginx on FreeBSD, and the configuration below is what I am currently running in production.

Mastodon is heavier than snac in every direction. It has Puma and Sidekiq behind it, more endpoints, more streaming, more federation patterns, and one specific characteristic that complicates everything: it serves multiple representations on the same URLs. The same path returns HTML to a browser, ActivityPub JSON to a remote instance, and sometimes plain JSON to an API client. If the proxy treats the URL as one thing, sooner or later it will return the wrong thing to the wrong client.

Most of the work below comes from that single observation.

If I had to summarize this whole post in a single sentence, it would be this:

Mastodon is not a website. It is a website, an API, and an ActivityPub server, all sharing the same URLs.

Everything else in this configuration - cache keys, variants, bypass rules, the diagnostic headers - is decoration around that one fact.

A popular toot from a friend gets boosted. Twenty federated instances ask for the same ActivityPub object within the same second. Browsers fetch the HTML version of the same URL. If the proxy sees only "a URL", it will eventually betray you: a remote instance will receive HTML, a browser will receive ActivityPub JSON, and you will spend an afternoon wondering why your timeline looks broken on three different servers. I have spent that afternoon. I do not recommend it.

Assumptions before anything else

Before any directive, this configuration assumes a few things about the instance. If any of these does not match your setup, the directives still make sense, but you must read the caveats at the end before adapting them.

The first assumption is that AUTHORIZED_FETCH (secure mode) is disabled. With secure mode off, all ActivityPub GET responses cached at the proxy layer are public and identical regardless of the requesting actor. With secure mode on, Mastodon can legitimately return different bodies depending on which remote actor is asking, and caching them blindly at the proxy becomes at best wasteful, at worst a cache-poisoning surface.

This is not a hypothetical. CVE-2026-25540, fixed in Mastodon 4.3.19, 4.4.13, and 4.5.6, is exactly this kind of mistake, but inside Mastodon's own Rails.cache: the pinned posts and featured hashtags endpoints had actor-dependent ActivityPub responses but were keyed without the actor. The CVE does not directly apply to nginx caches, but the underlying lesson does. Do not cache what depends on the caller unless the caller is part of the cache key. Keep this rule in mind every time you are tempted to cache a federation endpoint "just in case".

The second assumption is that no signed-URL storage backend sits behind /system/ or /media_proxy/. If those paths ever redirect to short-lived presigned S3 or SeaweedFS URLs, my TTLs below are too long: nginx will happily cache a redirect to a URL that has already expired.

The third assumption is that federation traffic uses HTTP Signatures, not the HTTP Authorization header. Mastodon signs federated GETs with the Signature header. The Authorization-based skip-cache rule further down catches API tokens, not signed federation traffic. If you enable AUTHORIZED_FETCH, you must add an explicit skip rule for $http_signature.

I am being deliberate about these assumptions because the configuration that follows is internally consistent only as long as they hold.

The proxy in front of mastodon.bsd.cafe has three jobs:

TLS termination, microcaching of expensive endpoints (especially federation-heavy collections and default public routes), and long-lived caching of immutable assets and user media.

The point is not to replace Mastodon's internal Rails cache. The point is to absorb spiky federation traffic and repetitive asset fetches that would otherwise hit Puma and Rails for every single request.

The strategy is deliberately layered: very long TTL on fingerprinted assets, medium TTL on user-uploaded media, very short microcache on dynamic pages and federation endpoints that get hammered, and explicit bypass rules for anything private, authenticated, actor-dependent, or otherwise unsafe.

Every cacheable layer is keyed correctly for content negotiation. That is the part that matters most.

The cache zone

A single cache zone is shared across all Mastodon locations:

proxy_cache_path /var/cache/nginx/mastodon
                 levels=1:2
                 keys_zone=mastodon_cache:200m
                 max_size=20g
                 inactive=24h
                 use_temp_path=off;

200m of keys zone holds metadata for roughly 1.6 million entries in RAM. The body can grow up to 20g on disk. The two numbers are independent: keys live in shared memory, bodies live on the filesystem, and the cache key is what links them.

inactive=24h evicts anything not requested for a day, even if there is free space. This is intentional. I do not want a long, cold tail of stale entries to squat in the cache forever. I want the working set to remain hot, and I want the rest to fade.

use_temp_path=off is small but important. By default nginx writes a cached response to a temporary file and then renames it into place. If the temp path and cache path are on different filesystems, that cheap rename becomes a real copy. Setting use_temp_path=off puts temporary files directly under the cache directory and avoids that trap. It is the kind of detail nobody mentions until something is suspiciously slow.

Of all the maps in this configuration, only one really earns its place. This one:

map $http_accept $mastodon_cache_variant {
    default                          "default";
    "~*application/activity\+json"   "activitypub";
    "~*application/ld\+json"         "activitypub";
    "~*application/json"             "json";
    "~*text/html"                    "html";
}

Mastodon serves the same URL with different bodies depending on the Accept header. A status URL like /@user/123456789 returns rendered HTML to a browser and an ActivityPub object to another federated instance. If you cache by URL alone, the first request that comes in wins and the next request receives the wrong content type. Instances start federating HTML, browsers start downloading JSON, and the failure is subtle enough to waste hours.

The map normalizes Accept into four buckets - activitypub, json, html, and default - and the result is folded into the cache key in every location that does content negotiation:

proxy_cache_key "$scheme$host$request_uri|accept=$mastodon_cache_variant";

Coalescing equivalent MIME types is intentional. application/activity+json and application/ld+json both map to activitypub, because splitting them across two cache buckets would fragment the cache for no useful operational gain.

A subtle point I want to be explicit about: I do not include $request_method in the cache key. nginx already converts HEADinto GET for caching purposes by default, which is what I want here. A HEAD request on /@user/123 should hit the same cache entry as a GET request on the same URL. Adding the method would only separate them for no benefit.

During rollout I also expose the selected variant as a response header:

add_header X-Cache-Variant $mastodon_cache_variant always;

The header is there to verify the behaviour in production. It can come off once the configuration has proved itself, but I tend to leave it on. A cache that works should be visible. A cache that is invisible can be correct, but it can also be silently wrong, and I would rather know.

This is the first real gotcha, and I want to spend a moment on it because it caught me out the first time I configured a similar setup.

nginx honors the upstream Vary response header in addition to proxy_cache_key. If Mastodon emits Vary: Accept, or worse, Vary: Accept, Cookie, ..., my carefully normalized variant key gets paired with nginx's native Vary handling. The result is that the cache may still fragment on the full, un-normalized Accept header - which defeats the entire point of the variant map.

There is another, very specific failure mode on older or unpatched nginx builds. nginx stores the Vary value in a fixed-size cache metadata field. Historically that field was 42 bytes, which is famously short and almost charmingly suspicious of being a Douglas Adams reference. Modern nginx raised the limit to 128 bytes, which is enough for the common cases but still surprisingly small. If your upstream emits a long Vary header, anything beyond the limit is treated as Vary: *, which means the response is not cached at all. The only signal you get is a critical line in the error log, and unless you are looking for it, you will not see it.

The operational lesson is the same in both cases: if you rely on your own normalized variant key, do not assume upstream Vary is harmless. Check your nginx version, check your error log, and verify cache behaviour via X-Cache-Statusand X-Cache-Variant.

On the locations where the variant map is the cache dimension I care about, I take responsibility explicitly:

proxy_ignore_headers Vary;

This tells nginx to stop using upstream Vary to protect me. That is fine only if my own cache key and request normalization cover every response dimension that matters. In particular, I make sure the backend is not also varying on Accept-Encoding in a way that would create compressed and uncompressed variants behind my back. The cleanest way to avoid that is not to forward Accept-Encoding to the backend at all, and let frontend nginx handle compression itself:

proxy_set_header Accept-Encoding "";

This is the kind of decision I prefer to be explicit about. Ignoring Vary is not magic. It is a responsibility, and it should be paired with the rules that take its place.

Rather than build one giant boolean to decide what bypasses cache, I prefer to decompose the logic into small orthogonal maps. Each map is 1 when caching must be skipped, and the final decision is an OR of all of them.

map $request_method $skip_cache_method {
    default 1;
    GET     0;
    HEAD    0;
}

map $http_authorization $skip_cache_auth {
    default 1;
    ""      0;
}

map $http_cookie $skip_cache_cookie {
    default 1;
    ""      0;
}

map $uri $skip_cache_uri {
    default                  0;
    ~^/auth                  1;
    ~^/oauth                 1;
    ~^/settings              1;
    ~^/admin                 1;
    ~^/api/v1/custom_emojis$ 0;
    ~^/api/v1/instance$      0;
    ~^/api/v2/instance$      0;
    ~^/api/v1/trends/tags$   0;
    ~^/api/oembed$           0;
    ~^/api/                  1;
}

The reasoning is straightforward. Only GET and HEAD are cacheable; everything else, including POST, DELETE, PUT, and ActivityPub deliveries, must pass through. Any request carrying an Authorization header is an API call with a token, and those are never public. Any request with a cookie is potentially logged-in traffic, and caching logged-in pages would leak personal timelines across users. Auth flows, settings, admin, and most of the API bypass the cache by URI, while a small, carefully chosen set of slow-changing public API endpoints is allowed through.

The important caveat I want to underline: the Authorization map does not catch signed federated GETs. Mastodon federation uses HTTP Signatures, which means the relevant request header is Signature. If AUTHORIZED_FETCH is enabled, you must add a parallel map:

map $http_signature $skip_cache_signature {
    default 1;
    ""      0;
}

and then include it in both proxy_cache_bypass and proxy_no_cache. Do this before enabling secure mode, not after.

The maps are used together in each cacheable location:

proxy_cache_bypass $skip_cache_method $skip_cache_auth $skip_cache_cookie $skip_cache_uri;
proxy_no_cache     $skip_cache_method $skip_cache_auth $skip_cache_cookie $skip_cache_uri;

Both directives are necessary. proxy_cache_bypass means "do not read from cache for this request". proxy_no_cache means "do not write this response to cache". Without proxy_no_cache, a logged-in user's response could still poison the anonymous cache. Without proxy_cache_bypass, a request that should have gone straight to the backend might still receive a cached anonymous response. I keep both, every time.

Most locations share a common proxy baseline. There is nothing clever here, but if any of these lines is missing the rest of the configuration quietly does less than expected.

proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Connection "";
proxy_set_header Accept-Encoding "";

proxy_http_version 1.1 and proxy_set_header Connection "" matter for upstream keepalive. Without them, nginx may use HTTP/1.0 semantics upstream and send Connection: close on every request, which makes the keepalive directive on the upstream block far less useful than it looks.

proxy_set_header Accept-Encoding "" keeps backend responses uncompressed so nginx can cache a single representation and handle client-facing compression itself. It also prevents accidental cache fragmentation through Vary: Accept-Encoding, which would otherwise creep in despite the variant map.

These settings are not exciting, and they should not be. The interesting parts of an infrastructure are not always the parts that should be unusual.

The Mastodon server block in my configuration ends up with seven distinct request profiles. Six of them cache; one explicitly does not, because streaming is not a cacheable workload.

I do not group them under one location / with a giant if block. I prefer to keep each profile in its own location, even if some of them look similar. When something goes wrong in production, I want to be able to point at one location and reason about it without holding the rest of the configuration in my head.

location ~ ^/(assets|packs|emoji)/ {
    proxy_cache mastodon_cache;
    proxy_cache_key "$scheme$host$request_uri";
    proxy_ignore_headers Vary;

    proxy_cache_valid 200 301 302 7d;
    proxy_cache_valid 404 10m;

    proxy_cache_lock on;
    proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
    proxy_cache_background_update on;

    proxy_cache_bypass $skip_cache_method $skip_cache_auth $skip_cache_cookie $skip_cache_uri;
    proxy_no_cache     $skip_cache_method $skip_cache_auth $skip_cache_cookie $skip_cache_uri;

    proxy_pass http://$custom_upstream;
}

These paths are content-addressed. Webpack fingerprints filenames with hashes, so a new deploy publishes new URLs while the old URLs remain valid. A 7-day TTL is safe because /packs/js/common-abc123.js will never become different content under the same URL. If it does, it has a new URL.

404s get a short 10-minute TTL so a temporarily missing asset can recover quickly.

proxy_cache_lock on is the thundering-herd guard. When a popular asset is not cached and ten clients ask for it at once, nine wait for the first request to populate the cache instead of all ten hammering the backend. I like this directive a lot. It is the kind of small switch that quietly removes a class of problems.

proxy_cache_use_stale together with proxy_cache_background_update is the stale-while-revalidate pattern. If an entry has expired but Mastodon is slow or briefly down, nginx can serve the stale copy and refresh it asynchronously. For static assets this is almost always the right trade-off. The asset has not actually changed under the same URL, and a few extra hours of stale data hurt nobody.

location ~ ^/system/(accounts/avatars|media_attachments/files|custom_emojis/images)/ {
    proxy_cache mastodon_cache;
    proxy_cache_key "$scheme$host$request_uri";
    proxy_ignore_headers Vary;

    proxy_cache_valid 200 302 6h;
    proxy_cache_valid 404 5m;

    proxy_cache_lock on;
    proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
    proxy_cache_background_update on;

    proxy_cache_bypass $skip_cache_method $skip_cache_auth $skip_cache_cookie $skip_cache_uri;
    proxy_no_cache     $skip_cache_method $skip_cache_auth $skip_cache_cookie $skip_cache_uri;

    proxy_pass http://$custom_upstream;
}

Avatars, attachment thumbnails, and custom emoji are also effectively content-addressed, because the file path contains an ID. They can still be replaced or deleted, so the TTL is more conservative than for assets: six hours instead of seven days.

The 302 status is also cached, because Mastodon may redirect to another storage location, and the redirect is usually stable enough to cache for hours.

This is also where the caveat about signed URLs really matters. If you ever put a signed-URL backend behind /system/, this TTL must be shorter than the signed URL lifetime, or nginx will eventually serve a redirect to a URL that no longer works. On mastodon.bsd.cafe I do not use signed URLs, so six hours is fine.

location ~ ^/(users|ap/users)/[^/]+/statuses/[0-9]+/replies {
    proxy_cache mastodon_cache;
    proxy_cache_key "$scheme$host$request_uri|accept=$mastodon_cache_variant";
    proxy_ignore_headers Vary;

    proxy_cache_valid 200 30s;
    proxy_cache_valid 404 10s;

    proxy_cache_lock on;
    proxy_cache_lock_timeout 1s;
    proxy_cache_lock_age 5s;

    proxy_cache_bypass $skip_cache_method $skip_cache_auth $skip_cache_cookie $skip_cache_uri;
    proxy_no_cache     $skip_cache_method $skip_cache_auth $skip_cache_cookie $skip_cache_uri;

    proxy_pass http://$custom_upstream;
}

This is the location I tuned most carefully. When a status starts going viral, dozens of federated instances poll /replies to build their thread view, often within the same second. The same URL must serve an HTML thread view to browsers and an ActivityPub OrderedCollection to remote instances, so the variant key is essential here.

A 30-second microcache absorbs the spike without serving meaningfully stale data. A reply that appears 30 seconds late in a federated thread is usually invisible to humans, while the backend relief is very visible.

The lock settings keep backend load and latency bounded. proxy_cache_lock_timeout 1s bounds how long queued requests wait behind the lock. If the timeout expires, they go to the upstream directly, but their responses are not stored in the cache, which prevents a runaway thundering herd from clogging the cache fill path. proxy_cache_lock_age 5s prevents one slow cache-populating request from monopolizing the fill path forever; if the request holding the lock has not completed after 5 seconds, nginx may let another request reach the upstream to retry.

I have currently left proxy_cache_use_stale off on this location while I am still validating the deployment. This is a deliberate debugging stance, not a permanent choice. Stale-while-revalidate is useful in production, but during rollout it can hide upstream issues while I am trying to understand the system. Once the behaviour is stable, the production version will be:

proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
proxy_cache_background_update on;

location ^~ /media_proxy/ {
    proxy_cache mastodon_cache;
    proxy_cache_key "$scheme$host$request_uri";

    proxy_cache_valid 200 10m;
    proxy_cache_valid 301 302 10m;
    proxy_cache_valid 404 1m;

    proxy_ignore_headers Cache-Control Expires Vary;

    proxy_cache_lock on;
    proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
    proxy_cache_background_update on;

    proxy_cache_bypass $skip_cache_method $skip_cache_auth $skip_cache_cookie $skip_cache_uri;
    proxy_no_cache     $skip_cache_method $skip_cache_auth $skip_cache_cookie $skip_cache_uri;

    proxy_pass http://$custom_upstream;
}

Mastodon's /media_proxy/ fetches remote media so clients do not leak their IP address to remote servers. The response is the same regardless of Accept, so the cache key intentionally omits the variant. Splitting media proxy responses across html, json, activitypub, and default buckets would only waste storage.

proxy_ignore_headers Cache-Control Expires Vary is deliberate here. Mastodon may emit conservative cache headers, or none at all, and I want the proxy to enforce a short local 10-minute policy regardless of what the backend says.

Set-Cookie is not in the ignore list. nginx's default refusal to cache responses carrying Set-Cookie still applies, and I want it to. It is a safety net I do not want to disable just to win a few cache hits.

The ^~ prefix is a small useful detail. Once this location matches, nginx stops evaluating regex locations. Media proxy traffic can be heavy, and skipping further regex matching is a tiny but free win.

location ~ ^/(users|ap/users)/[^/]+/(followers|following) {
    proxy_pass http://$custom_upstream;
}

This one is a pure proxy, no cache. I want to be explicit that this is a decision, not an omission.

/users/<name>/followers and /users/<name>/following are pagination-heavy, change frequently as people follow and unfollow, and are queried by federation crawlers in ways that would make the cache key proliferate through pages and cursors. The likely hit ratio is poor, the risk of serving stale social graph data is non-trivial, and the cost of caching them - in storage and in mental overhead - is not worth it.

If a remote instance starts hammering these endpoints, the right answer is rate limiting with limit_req_zone, not retrofitting cache as a rate limiter.

Default location: the microcache and streaming without cache

location / {
    proxy_cache mastodon_cache;
    proxy_cache_key "$scheme$host$request_uri|accept=$mastodon_cache_variant";
    proxy_ignore_headers Vary;

    proxy_cache_valid 200 10s;
    proxy_cache_valid 301 302 1m;
    proxy_cache_valid 404 10s;

    proxy_cache_lock on;
    proxy_cache_lock_timeout 5s;

    proxy_cache_bypass $skip_cache_method $skip_cache_auth $skip_cache_cookie $skip_cache_uri;
    proxy_no_cache     $skip_cache_method $skip_cache_auth $skip_cache_cookie $skip_cache_uri;

    proxy_next_upstream error timeout http_502 http_503 http_504;
    proxy_next_upstream_tries 2;

    proxy_pass http://$custom_upstream;
}

Everything not matched by a more specific location falls here: profiles, individual statuses, the about page, the public timeline, and many ActivityPub object fetches.

The TTL is only 10 seconds for 200 responses. That is enough to deduplicate the wave of requests when a popular toot gets boosted or linked from elsewhere, without making the page feel stale to a human visitor.

It is worth being honest that short TTLs still cost CPU. A 10-second microcache on a sustained-traffic URL means the backend regenerates the entry six times per minute. That is vastly better than serving every request from Rails, but it is not free. If your backend cannot comfortably handle that, raise the TTL, or enable stale-while-revalidate on these dynamic paths.

proxy_next_upstream with proxy_next_upstream_tries 2 is the failover trigger. If the primary returns 502, 503, 504, or times out, nginx retries on the backup. The chain is capped at two attempts so a sick upstream cannot hold the request indefinitely.

At the http level:

map $http_upgrade $connection_upgrade {
    default upgrade;
    ""      close;
}

In the server block:

location /api/v1/streaming {
    proxy_buffering off;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;
    proxy_read_timeout 3600s;
    proxy_send_timeout 3600s;
    proxy_pass http://$custom_upstream;
}

Streaming is a WebSocket and SSE-style endpoint. Buffering must be off, otherwise the proxy may hold messages while waiting for buffers to fill. The Upgrade and Connection headers are driven by $connection_upgrade, which is upgradeonly when the client actually sent an Upgrade header. That way a non-WebSocket request to the same path does not get its Connection header mangled.

The hour-long read and send timeouts allow long-lived streams to stay open through quiet periods.

There is no cache here. Streaming is not a cacheable workload, and trying to make it one is one of those ideas that sounds clever for about thirty seconds.

Upstream and failover

upstream mastodonbsdcafe {
    server 192.168.123.33  max_fails=3 fail_timeout=30s;
    server 192.168.122.133 backup;
    keepalive 64;
}

The primary backend is on another VPS; the backup is in a jail next to the reverse proxy. After three consecutive failures, the primary is marked down for 30 seconds. Traffic flips to the backup, then nginx retries the primary after the window.

keepalive 64 holds up to 64 idle TCP connections to the upstream per worker. On a busy instance, this saves real handshake overhead, but only if the proxied connection can actually stay open. That is why the shared proxy settings include proxy_http_version 1.1 and proxy_set_header Connection "". Without those, upstream keepalive does much less than it looks like it should.

I also use an indirection layer:

map $remote_addr $custom_upstream {
    default mastodonbsdcafe;
}

Today everything defaults to the main upstream group. The map exists so that specific client IPs can be pinned to a specific upstream when I am debugging, or so an admin connection can be routed to the backup while the primary is being tested. It costs nothing to have it sitting there, and it has saved me time more than once.

What I log and why

log_format detailed '$remote_addr - $remote_user [$time_local] '
                    '"$request" $status $body_bytes_sent '
                    '"$http_referer" "$http_user_agent" '
                    'rt=$request_time '
                    'uct=$upstream_connect_time '
                    'uht=$upstream_header_time '
                    'urt=$upstream_response_time '
                    'us=$upstream_status '
                    'ua=$upstream_addr '
                    'cache=$upstream_cache_status '
                    'variant=$mastodon_cache_variant';

access_log /var/log/nginx/access.mastodon.bsd.cafe.log detailed;

add_header X-Cache-Status  $upstream_cache_status always;
add_header X-Cache-Variant $mastodon_cache_variant always;

This log format is purpose-built for the cache layer. For each request it records total request time, upstream connect time, upstream header time, upstream response time, upstream status, which backend served the request, cache status, and which content-negotiation variant was selected.

The cache status is one of the values nginx exposes through $upstream_cache_status: HIT, MISS, BYPASS, EXPIRED, STALE, UPDATING, or REVALIDATED. The response headers expose the same information to the client, which makes it trivial to verify behaviour with curl -I or browser dev tools.

The always qualifier matters. Without it, nginx only adds these headers to a subset of responses, so a 502 from the backend might arrive without the diagnostic headers you need most. I want them on every response, no exceptions.

There is also a small operational detail I find pleasant: a custom 502 page.

error_page 502 /502.html;
location = /502.html {
    root /usr/local/www/mastodon_errors;
    internal;
}

It is not part of the cache strategy, but it makes backend hiccups less ugly. And I block some abusive user agents with 444, which closes the connection without sending any response at all:

if ($http_user_agent ~* "bytespider") {
    return 444;
}

This is not a general bot strategy. It is just a cheap refusal path for traffic I know I do not want.

How I check it actually works

A configuration that I cannot verify is a configuration I do not trust. Here is the short set of commands I keep in a paste buffer for this proxy.

The first verification is variant separation. Three requests to the same URL with different Accept headers should produce three independent cache entries:

for v in 'text/html' \
         'application/activity+json' \
         'application/ld+json; profile="https://www.w3.org/ns/activitystreams"'; do
  printf '%-75s -> ' "$v"
  curl -s -o /dev/null -D - -H "Accept: $v" \
    https://mastodon.bsd.cafe/@someuser/123456789 \
    | awk '/^[Xx]-[Cc]ache/ { printf "%s ", $0 } END { print "" }'
done

On the first pass, every variant should be a MISS. On the second pass, every variant should be a HIT, with X-Cache-Variantshowing the expected bucket.

The second verification is that cookies and Authorization always trigger BYPASS:

curl -I -H 'Cookie: _mastodon_session=test' \
  https://mastodon.bsd.cafe/@someuser

curl -I -H 'Authorization: Bearer fake' \
  https://mastodon.bsd.cafe/api/v1/timelines/home

Both should return X-Cache-Status: BYPASS. If they do not, the skip-cache rules are wrong, and the entire setup is unsafe.

If you intend to enable AUTHORIZED_FETCH, the third verification is for signed GETs. A quick synthetic check that the nginx map fires correctly:

curl -I -H 'Signature: fake' \
       -H 'Accept: application/activity+json' \
       https://mastodon.bsd.cafe/users/someuser

If you added $skip_cache_signature, the result should be X-Cache-Status: BYPASS.

Finally, the logs themselves tell me how the cache is performing in production. Cache status distribution:

awk '{
  for (i = 1; i <= NF; i++)
    if ($i ~ /^cache=/) c[$i]++
}
END {
  for (k in c) print k, c[k]
}' /var/log/nginx/access.mastodon.bsd.cafe.log

A healthy instance shows cache=HIT and cache=BYPASS doing most of the work, with cache=MISS accounting for cold paths and short-TTL refreshes. The same trick works for the variant distribution:

awk '{
  for (i = 1; i <= NF; i++)
    if ($i ~ /^variant=/) v[$i]++
}
END {
  for (k in v) print k, v[k]
}' /var/log/nginx/access.mastodon.bsd.cafe.log

This tells me what my traffic actually looks like. A federation-heavy instance shows a lot of activitypub. An instance with many human visitors shows more html. On mastodon.bsd.cafe the balance shifts depending on what is happening in the wider Fediverse on any given day.

Caveats worth being honest about

I do not like presenting configurations as magic, so I want to be explicit about the conditions under which this one is appropriate.

Short TTLs cost CPU. A 10-second microcache on a sustained-traffic URL means six backend regenerations per minute. That is much better than no cache, but it is not free. If the backend cannot comfortably handle that, raise the TTL or enable stale-while-revalidate on the dynamic paths.

Dynamic stale-while-revalidate is powerful but it hides problems. I currently keep proxy_cache_use_stale off on the dynamic locations because I am still validating behaviour. In steady-state production, stale-while-revalidate is usually the right choice. During rollout, it can quietly hide upstream errors and make debugging harder. Be honest with yourself about which mode you are in.

AUTHORIZED_FETCH changes the threat model. With secure mode disabled, public ActivityPub GET responses are safe to cache as public content, provided your cache key handles content negotiation correctly. With secure mode enabled, ActivityPub responses can become actor-dependent. At that point you must either bypass cache for signed GETs or include the signing actor in the key. The latter usually destroys the hit ratio, so bypassing is the practical answer.

The variant map is a compromise. It covers application/activity+json, application/ld+json, application/json, and text/html. Everything else falls into the default bucket. That is intentional, but the default bucket is still a bucket. If you discover a real client type that matters on your instance, add it explicitly.

Ignoring Vary is a responsibility. proxy_ignore_headers Vary is not magic; it tells nginx to stop protecting you based on upstream Vary. That is fine only if your own cache key and request normalization cover every dimension Vary was protecting. For this configuration that means normalizing Accept into a variant, avoiding backend Accept-Encodingvariation, never caching cookies or authorization, and never caching signed GETs if secure mode is enabled.

Followers and following are uncached on purpose. They are pagination-heavy and change frequently. Caching them would create many low-value entries with questionable freshness. If a remote instance hammers these endpoints, use limit_req_zone. Do not retrofit cache as a rate limiter.

Signed-URL redirects require shorter TTLs. Caching 302s is useful when redirects are stable. It is dangerous when redirects point to short-lived signed URLs. If your media storage returns presigned URLs, your nginx redirect TTL must be shorter than the URL lifetime.

Set-Cookie must remain special. Do not add Set-Cookie to proxy_ignore_headers unless you are absolutely sure the location cannot produce user-specific responses. nginx's default refusal to cache Set-Cookie responses is a safety net. Keep it.

A good configuration is a written form of the assumptions behind a service. When the assumptions change, the configuration must change too.

There is no single brilliant directive in this configuration. The trick is combining long TTLs for immutable assets, medium TTLs for media, tiny TTLs for dynamic public pages, cache locking for thundering-herd protection, strict bypass rules for private or actor-dependent traffic, a normalized content-negotiation key, and enough logging to prove the system is doing what I think it is.

What this layer buys me, in one sentence: fewer requests reach Puma and Rails.

That is the metric I care about. Mastodon is not slow, but it is heavy, and the bigger the instance grows the more it benefits from a layer in front that quietly absorbs the work that does not need to be done by the application. A reverse proxy that caches Mastodon safely has to remember, with every request, that the same URL might mean three different things to three different clients. Once it does, even a very short microcache can remove a surprising amount of load without changing the user-visible behaviour of the instance.

Self-hosting your Mastodon media with SeaweedFS

Stefano Marinelli — Thu, 06 Nov 2025 11:30:02 +0000

Mastodon 4.5.0 is here, and with it come some interesting changes that, in my opinion, might encourage more people to consider it for self-hosting their Fediverse community.

While it may not be as lightweight and simple as other solutions (like snac or GoToSocial or many others), I believe it remains one of the best platforms for managing a medium-sized Fediverse community, thanks in part to the direct feedback that many admins have provided to the developers.

I have previously written about how to install Mastodon in a FreeBSD jail and how to modify its character and poll limits.

One of the most critical initial decisions (which can be changed later, but with extra work) is where to store your media files. Mastodon downloads and re-processes all media it encounters from other instances for three main reasons:

Local Caching: Your users connect to your media server, reducing the load on the original instance.
Security: Re-processing media helps to remove any potential "impurities" before they reach the user's device.
Privacy: It prevents disclosing your users' IP addresses to other instances. A user will only connect to their own instance to fetch all data, including remote content.

At least initially, media files will be the largest part of your instance's storage footprint. It is therefore essential to plan where to store them and to add a regular cleanup script; otherwise, their growth will be exponential.

Mastodon supports uploading media to external S3-compatible solutions, and many admins use the usual commercial providers, paying for data uploads and transfers.

I am a firm believer in "Own Your Data", so I have always used my own self-hosted S3 servers. I initially started with Minio, but over time, I realized that, by design, it doesn't perform well with a multitude of small files (performance degrades). After running some tests, I decided to switch to SeaweedFS.

SeaweedFS "is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek..." - this, combined with the fact that it is a mature and proven piece of software, was enough for me to give it a try. The result? Excellent. The I/O and CPU load on my media server dropped drastically, making SeaweedFS an incredibly suitable solution. Furthermore, some of its features (like the ability to run a filer.sync) allow for efficient and fast replication to other storage, another host, or... anything else.

SeaweedFS works perfectly with Mastodon, and I will explain the steps to get it into production.

I will install SeaweedFS in a dedicated jail and use a dedicated subdomain. This ensures that the media server can be moved to another host at any time without reconfiguring everything or changing domains. SeaweedFS has its own FreeBSD package, installable via pkg, or can be downloaded directly from the project's website.

In either case, I will describe a "test" setup - which can also be used in production without issues. However, I highly recommend diving deeper into the tool, as it is incredibly powerful and flexible and can solve many more problems than one might imagine.

Setting up the SeaweedFS Jail

First, let's create a dedicated jail with BastilleBSD:

bastille create media 14.3-RELEASE 10.0.0.66 bastille0

Now, let's enter the jail and install SeaweedFS (and tmux, which can be useful):

bastille console media
pkg install -y tmux seaweedfs

I suggest launching SeaweedFS in a tmux session so you can monitor its output. Later, you should configure an automatic startup method, such as using the included rc.d file or any other method you prefer.

Create a directory for the data and start SeaweedFS as the "seaweedfs" user:

mkdir -p /seaweedfs/data
chown -R seaweedfs /seaweedfs
su -m seaweedfs
cd /seaweedfs/
/usr/local/bin/weed server -dir /seaweedfs/data -s3

At this point, SeaweedFS will start and create everything it needs to function, including the S3 server.

Configuring Buckets and Users

Now, let's open the weed shell to create the necessary bucket and users:

weed shell
s3.bucket.create -name mastomedia

Still in the weed shell, create a user for Mastodon and grant read permissions for unauthenticated users (which is necessary to serve media to the world):

s3.configure -access_key=mastomedia -secret_key=CHANGEME -buckets=mastomedia -user=mastodon -actions=Read,Write,List,Tagging,Admin -apply
s3.configure -buckets=mastomedia -user=anonymous -actions=Read -apply
s3.configure -buckets=mastomedia -actions=Read -apply

Security Tip: For the -secret_key, avoid using a simple password. You can generate a strong, random key directly from your shell with a command like openssl rand -base64 32.

Done. SeaweedFS is now ready to receive (and serve) media. The next step is to set up a reverse proxy to serve everything over HTTPS. My preferred approach is to configure the system as if it were external, even if the services are in adjacent jails. This might use slightly more resources, but the time and trouble it saves in the future are well worth it.

Nginx Reverse Proxy Configuration

The reverse proxy can be configured something like this:

[...]

server {
   server_name  media.mastodon.example.com;

   ignore_invalid_headers off;
   client_max_body_size 0; # Allow large file uploads without Nginx limits

   location / {
      proxy_set_header Host $http_host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;

      proxy_connect_timeout 300;
      proxy_http_version 1.1;
      proxy_set_header Connection "";
      chunked_transfer_encoding off;

      expires 1y;
      add_header Cache-Control public;

      add_header X-Cache-Status $upstream_cache_status;
      add_header X-Content-Type-Options nosniff;

      proxy_pass http://10.0.0.66:8333;
   }

# ... other server configurations like SSL ...

}

Mastodon Configuration

Now let's configure Mastodon. If you are running the setup wizard for the first time, here is a summary of the options:

[...]
Do you want to store uploaded files on the cloud? yes
Provider Minio
Minio endpoint URL: https://media.mastodon.example.com
Minio bucket name: mastomedia
Minio access key: mastomedia
Minio secret key: CHANGEME
Do you want to access the uploaded files from your own domain? Yes
Domain for uploaded files: media.mastodon.example.com

If Mastodon is already active, or once the setup is complete, the options in your .env.prod file should be modified to be consistent with what SeaweedFS expects:

S3_ENABLED=true
S3_PROTOCOL=https
S3_REGION=us-east-1
S3_ENDPOINT=https://media.mastodon.example.com
S3_HOSTNAME=media.mastodon.example.com
S3_BUCKET=mastomedia
AWS_ACCESS_KEY_ID=mastomedia
AWS_SECRET_ACCESS_KEY=CHANGEME
S3_FORCE_SINGLE_REQUEST=true
# remove the S3_ALIAS_HOST if it is set

IMPORTANT NOTE: If both services are in jails on the same host (i.e., SeaweedFS is on the same host as Mastodon), you should ensure that the Mastodon jail can reach the SeaweedFS jail through the reverse proxy and not via the external IP. To do this, add the following line to the /etc/hosts file of the Mastodon jail:

10.0.0.1        media.mastodon.example.com

In this example, the reverse proxy is at 10.0.0.1. If you are not using a separate reverse proxy but are exposing Nginx directly from the jail (as described in my Mastodon installation article), use the IP of the Mastodon jail itself instead (e.g., 10.0.0.42).

With this setup, Mastodon will be able to upload media to the SeaweedFS server and generate the correct links for other instances, public visitors, and users of your own instance.

Have fun with SeaweedFS!

Increasing or Modifying Character Limits and Poll Options in Mastodon 4.3, 4.4 and 4.5

Stefano Marinelli — Wed, 09 Oct 2024 14:53:00 +0200

Tested with Mastodon 4.5.x

Mastodon 4.3.0 was released just a few hours ago, and many instances have already started updating.

During my tests, I noticed that some things have changed in the core files, and the procedures normally described for changing the character limit of posts no longer work. So, I decided to write this short blog post to document the updated procedure.

Increasing or Modifying the Character Limit

At BSD Cafe, the character limit is set to 5000 (up from the 500 originally allowed by Mastodon). No one has ever used all the available characters, but I prefer that the limit be decided by the poster, not by the platform.

To customize this limit, you need to modify two files:

live/app/javascript/mastodon/features/compose/containers/compose_form_container.js - find the line that contains:

maxChars: state.getIn(['server', 'server', 'configuration', 'statuses', 'max_characters'], 500),

and change 500 to your desired value.

Next, you need to modify another file: live/app/validators/status_length_validator.rb - find the line that contains:

MAX_CHARS = 500

and change it accordingly.

Once done, run the following command:

RAILS_ENV=production bundle exec rails assets:precompile

and restart the Mastodon services to apply the changes.

Increasing or Modifying the Number of Poll Options

Edit the file live/app/validators/poll_options_validator.rb and modify the line:

MAX_OPTIONS = 4

to the value you want. There are also other customizable options in this file.

Again, once done, run the following command:

RAILS_ENV=production bundle exec rails assets:precompile

and restart the Mastodon services to apply the changes.

Changes might not be immediately visible because of caching (browser, web server, etc.), but once these caches expire, the new limits will take effect.

These modifications might be overridden by subsequent Mastodon updates, so be sure to check after every update that they are still valid.

Using a Permanent WebFinger Address for My Fediverse Profile

Stefano Marinelli — Tue, 08 Oct 2024 12:53:00 +0200

Decentralized technologies have always been of great interest to me. In my opinion, the decentralization of the Internet was one of the keys to its success, but year after year, we are gradually losing it. This is happening because we increasingly rely on "mainstream" services for convenience, habit, or trend.

I often hear colleagues say, "I've put my client's email on Y (Y = one of the major players in the email world) to avoid problems." The problems remain, but the colleague can "pass the buck" to the big player of the moment. And, in my experience, clients tend to be much more forgiving when the problem is with a big player.

Once, people used to say: "Nobody ever got fired for buying IBM." Today, I would say, "Nobody ever got fired for putting email on [Google, Microsoft, etc.]."

A few days ago, a colleague mentioned that he struggles to find some of his friends and colleagues on the fediverse because they are spread across various instances, and a simple search doesn't always yield the desired results.

Own Your Data

The fediverse is an extremely effective, resilient, autonomous decentralized communication technology. Unlike email, the many software implementations (such as Mastodon, GoToSocial, Mitra, Akkoma, Pleroma, etc. - soon snac will also support this type of operation) support the ability to "move" one's account. This means that when a user decides to relocate, the server will notify all followers that the user has moved. The followers will automatically stop following the original account and start following the new one. As of today, it is not possible to move past content (which will still be available on the old server).

In my opinion, every government, public entity, association, foundation, etc., that needs to communicate with the public should have its own communication channel, with full control over its data and the messages it delivers. When I read "my Discord server," I feel like responding "there's nothing 'yours' about it - tomorrow morning they could shut everything down, and you would have lost EVERYTHING.". Own your data!

Sometimes, I read that instances are not opened because of "costs not balanced by the number of users." But even public television channels are often economically unprofitable, yet they are considered an essential service for public communication. Open, decentralized technologies that ensure control over one's data should be treated the same way.

The problem, as with email, is tied to the address. A server change necessarily implies an address change, which can sometimes cause problems.

A Permanent Address

For this reason, I decided to implement a kind of "permanent" address that will provide the correct data based on my main server. I decided that this address should be linked to my email address, and since I control the entire domain (including the web server), I can do this quite simply. In this way, if I change servers, users who already follow me will be moved by the "move" operation, while those who do not yet follow me will still be able to find me using my "permanent" address, which will, in turn, provide the new user on the new server.

My setup will be very similar to what I found in an interesting blog post that I based my approach on.

In Practice

My email address is stefano@dragas.it, and I want anyone from any fediverse instance to be able to type it in and find me.
My main account is currently on BSD Cafe: https://mastodon.bsd.cafe/@stefano.

To achieve this, I modified the nginx configuration on the server that serves the "dragas.it" website.

Nginx Configuration

Before the server directive for dragas.it, I added this mapping:

map $query_string $account_name {
    ~resource=acct:stefano@dragas.it$ stefano;
    ~resource=acct%3Astefano%40dragas.it$ stefano;
}

This ensures that requests for those two resources (some clients send the second one, so I wanted to handle that situation) will assign the value "stefano" to the $account_name variable.

WebFinger Exception

Within the server directive of the dragas.it virtual host, I added an exception for webfinger (the protocol used to find user data):

location = /.well-known/webfinger {
    root  /usr/local/www/webfinger;

    if ($account_name) {
      rewrite ^(.*)$ /$account_name.json break;
    }

    try_files $uri = 404;
}

JSON File for WebFinger

At this point, I created the /usr/local/www/webfinger directory and added a file named stefano.json (matching $account_name) with the following content:

{
  "subject": "acct:stefano@dragas.it",
  "aliases": [
    "https://mastodon.bsd.cafe/@stefano",
    "https://mastodon.bsd.cafe/users/stefano"
  ],
  "links": [
    {
      "rel": "http://webfinger.net/rel/profile-page",
      "type": "text/html",
      "href": "https://mastodon.bsd.cafe/@stefano"
    },
    {
      "rel": "self",
      "type": "application/activity+json",
      "href": "https://mastodon.bsd.cafe/users/stefano"
    },
    {
      "rel": "http://ostatus.org/schema/1.0/subscribe",
      "template": "https://mastodon.bsd.cafe/authorize_interaction?uri={uri}"
    }
  ]
}

In practice, this will return a reference to my main Mastodon account.

Testing the Setup

After reloading the nginx configurations, it will be enough to test it:

# curl "https://dragas.it/.well-known/webfinger?resource=acct:stefano@dragas.it"

{
  "subject": "acct:stefano@dragas.it",
  "aliases": [
    "https://mastodon.bsd.cafe/@stefano",
    "https://mastodon.bsd.cafe/users/stefano"
  ],
  "links": [
    {
      "rel": "http://webfinger.net/rel/profile-page",
      "type": "text/html",
      "href": "https://mastodon.bsd.cafe/@stefano"
    },
    {
      "rel": "self",
      "type": "application/activity+json",
      "href": "https://mastodon.bsd.cafe/users/stefano"
    },
    {
      "rel": "http://ostatus.org/schema/1.0/subscribe",
      "template": "https://mastodon.bsd.cafe/authorize_interaction?uri={uri}"
    }
  ]
}

In this case, everything works perfectly. When you search for my email address from any fediverse instance, my BSD Cafe profile will appear.

Conclusion

This approach allows you to define multiple users for each domain. You just need to add the corresponding mapping in the nginx configuration and the .json file with all the data.

A Small Compendium of Fediverse Platforms I Use

Stefano Marinelli — Thu, 12 Sep 2024 18:45:00 +0200

Following my old article about the Fediverse software I've experimented and use, I've decided to convert a Mastodon post into this small blog post.

In the past few days, I revisited several of my old Fediverse instances after some friends asked me to help them set up a new one. While I was at it, I took the opportunity to perform maintenance on some leftover instances I still manage. Here’s a summary of my experience with various platforms:

Akkoma

This is my oldest instance, still running since 2022. It was offline for about 3 or 4 months, but I recently updated it to the latest version and restarted it. After upgrading the software and the database, it didn't show any problem. Akkoma is a very good solution, supports quote posts and emoji reactions.

GoToSocial

I helped a friend update their GoToSocial instance. While the software itself was up-to-date, the underlying system needed an update. I noticed that when the number of followings exceeds 2000, the instance becomes a bit sluggish. PostgreSQL isn’t the issue in this case – it's the GoToSocial process itself that seems to get heavy on the VPS. Despite this, GoToSocial remains very usable, and I see a lot of potential in it. The Mastodon API is well-implemented, and it works seamlessly with major apps.

Mitra

Mitra is another Fediverse platform I’ve been exploring. I helped someone with around 1000 followers and followings migrate from a large Mastodon instance to Mitra. There were no speed issues, though sending messages does make the server slightly "heavier" for a short time. The Mastodon API is partially implemented, but the software is evolving quickly. I find its native interface quite user-friendly, and it’s a platform worth keeping an eye on.

Snac2

I’ve always had a soft spot for Snac2. It doesn’t use a database, and its design choices make it ideal for small instances. One feature I particularly like is how it sends posts to all known instances, which increases visibility and interaction. The interface is basic, with no JavaScript, which is a nice change, but it might feel too minimalistic for users coming from Mastodon. However, the Mastodon API support is steadily improving with each release. Snac2 does struggle with larger numbers, but this is more due to the underlying file system than the software itself. Snac2 now supports moving in/out, so it's easy top test it. I highly recommend it for anyone looking to self-host a small or single-user instance.

Mastodon

My old personal instance of Mastodon was stuck on version 4.1.x and had been offline for a few months. I updated the FreeBSD Jail, upgraded Mastodon to 4.2.12, and then to 4.3.0 without any issues. I also helped a friend migrate their Pleroma-based instance to Mastodon. This user has about 5000 followers and followings, and the instance runs on FreeBSD on an arm64 VPS for around 3 euros per month. Aside from media storage (which isn’t Mastodon’s fault), there were no significant issues. Although Mastodon is sometimes criticized for being resource-intensive, its modular design ensures that even during high load, queues might slow down, but the local timeline and navigation remain reasonably fast. This makes it a strong contender for larger-scale use.

Conclusion

Overall, I feel that these platforms are evolving in the right direction. The developers are doing a fantastic job, and the Fediverse is growing stronger with each new release. Well done to all the devs working hard behind the scenes!

Deploying a piece of the Fediverse

Stefano Marinelli — Sun, 15 Jan 2023 09:29:45 +0000

After Elon Musk’s Twitter deal, many users decided to “fly away” from the "traditional" commercial Social Networks. Some for good, some just decided to increase their presence in other, alternative Social Network. That's what I'm doing.

Many of those users decided to join the Fediverse - even if many of them just call it Mastodon, as they don’t understand that Mastodon is just a Software that allows to join the Fediverse.

The Fediverse is composed by thousands of “instances”, some are bigger (like mastodon.social), some are personal (aka: single user instances), many are normal communities with their members. Many of them communicate using the same open protocol, ActivityPub.

Because of this, there’s no “one size fits all” so I’ve started to explore the different solutions. I’ve been mainly focusing on running them on FreeBSD, but I’ve had to fire up Linux for some tests. Here’s what I’ve found out.

Backends / Complete Solutions

Mastodon

I won’t spend too much time on Mastodon as you may find almost everything, everywhere, about it. Tons of articles have been written about Mastodon, so this one would be just another one, surely not the best one. Many consider it to be “the Fediverse” (they just say “Mastodon” to refer to the whole “Fediverse”, and they’re wrong), it’s by far the most installed solution. It’s so popular that there are plenty of clients (both for Android and iOS) that perfectly work with it. Mastodon has its own APIs - and many other Fediverse solutions are using them, just to be able to be compatibile with the Mastodon apps. It also supports backend based “Hide replies” (only show new posts, not all the replies to other posts) in timeline. It’s good as you can filter the replies from any frontend or mobile app as the backend won’t provide them at all, while the client doesn't need to be aware that you're filtering them.

Why I am suggesting it: It’s stable and well done, there’s a lot of documentation and if you install and manage it correctly, you shouldn’t notice anything strange or unexpected. Remember, no software solution can be considered “set and forget” and Mastodon is not an exception. Please, don’t forget that running an instance is not just installing the software. Moderation is a serious issue. More about it later.

Please, consider that: Mastodon is also quite heavy, not easy to scale (even if there’s documentation around) and, by default, is caching everything it sees and knows about. This means that both a single user instance or a thousands of users’ one, will (rapidly) grow because any media will be locally cached.

My first, single user installation grew, in a week, well over 100 GB of occupied storage because of all this caching. It can’t be avoided (you can just tell Mastodon to delete the cache after x days). While it may make sense for a big instance, it can be considered an overkill for a single user one. This is a "known problem", but mainly considered as a feature: all the media will be locally processed, all the contents will be locally stored. So an evil content hidden in a media file will be reprocessed by local ffmpeg or ImageMagick and will be cleared.

Why I’m suggesting another solution: Monopoly is bad and Mastodon is becoming, for many, a synonym of Fediverse . More, it requires much space and it’s resource-hungry. It’s easy to install Mastodon and experience a huge resource drain in just a few days, especially if you’re not a skilled system administrator. While Mastodon is the best solution for a complete microblogging experience, other solutions exist.

To sum up: Installing Mastodon on FreeBSD was easy. Even if I had read about problems, I’ve documented how to do it and it’s stable and reliable. Mastodon is, IMHO, a good piece of software but keep in mind that it could not be the best solution for you and a small, single user instance can become huge in a few weeks. Also, keep in mind that it's the most deployed Fediverse software, so any mobile app, any web app, any hint will work perfectly with Mastodon. Your Fediverse experience will be smooth.

Akkoma (Pleroma fork)

I’ve read about Akkoma in a reddit thread about how difficult was to install Mastodon on FreeBSD. It was described as a Pleroma fork, but actively developed and maintained, faster and with more advanced features. That’s why I decided to try it and - at least for now - stick with it (and not Pleroma, but many of the things I’ll point out here apply to Pleroma, too).

Akkoma (as all the Pleroma forks) is much, much lighter than Mastodon. It’s perfectly able to run a single user instance on a Raspberry PI. Moreover, Christine Lemmer-Webber, coauthor of the ActivityPub protocol, said that Akkoma is a good solution - and I think that her opinion is a qualified one.

Why I am suggesting it: First of all, for its documentation. As for Pleroma, there are installation instructions for a lot of operating systems (yes, also FreeBSD, NetBSD, OpenBSD). It’s light and fast. It doesn’t cache remote media by default, so that’s perfect for a single user instance. You can enable it (both pre-fetching media as soon as the server gets the status, like Mastodon, or just downloading them and caching when the first user meets them), you can also proxy your local media. S3 storage and remote CDNs are supported and everything is customisable. Message size limit is set to 5000 characters by default, but can be adjusted. Everything is configurable and you can choose your favourite frontend. It has quote posts (while Mastodon doesn’t allow them, even if they’re perfectly visible if created by Akkoma).

Please, consider that: Akkoma is not Mastodon. They “talk” using the same language but are different pieces of software. There’s less activity around it (the documentation is good, the support forum is good, but the number of Akkoma installations can’t be compared to Mastodon’s ones). Many Mastodon mobile apps seem to have problems with Akkoma and at the moment there’s a bug (probably it's a Mastodon bug, but users will think it's Akkoma's fault) that, if you’re posting an hashtag, a link will be shown on Mastodon. More, if you want to hide boosts or replies from your timeline, remember that Akkoma won’t perform that at backend’s level, but it should be done by the frontend. Not all frontends and mobile apps support it.

Why I’m suggesting another solution: Well, not exactly. Actually, I’m suggesting to try Akkoma. It’s a good piece of software, developed by friendly people, well accepted by the Fediverse instances’ administrators and has a lot of happy users and instance administrators. I've used Akkoma as my main instance software for more or less one month. The bug I've described and some problems here and there made me move back to Mastodon. I'm keeping my Akkoma instance up, even if not actively used.

To sum up: Akkoma installation is easy and well documented, you have a lot of settings to customise your instance and can fine-tune your installation for your hardware capabilities.

Misskey (and its forks like Firefish, etc.)

Misskey is a very nice piece of software. I had some troubles to run it on FreeBSD (but I didn’t try that much) so I decided to fire up a Linux machine and use Docker.

The interface is nice - it’s Japanese design, so it’s fancy and rich of emojis and effects. I’ve tried it just for a few hours and I appreciated it but decided it wasn’t ok for my needs (at least, for now).

Why I am suggesting it: if you’re building a community, Misskey (or one of its forks) is a very good choice. It’s eye candy, complete and usable. It’s a part of the Fediverse, so no problems to talk to Mastodon, Akkoma, etc. It also has a “drive” feature, useful for many users that want to exchange files.

Why I’m suggesting another solution: the main reason why I had to look at another solution is that it doesn’t support Mastodon APIs so no Mastodon Android or iOS app is working with a Misskey instance. While it may be ok for many users, this could be a problem for others.

To sum up: Misskey is nice, worth trying and a very good solution if it fits your community’s needs. For me, it doesn’t give any advantage over other lighter solutions.

Friendica

Friendica is a project that aims to be similar to Facebook. It’s federated and actively maintained, the interface is nice and familiar to Facebook users. I’ve been able to install it on a FreeBSD jail (as it’s in PHP) and everything worked as expected. I didn’t spend too much time on Friendica but I’m planning to do a deeper test as I’m working on a community of former Facebook users and this could be the right choice.

Why I am suggesting it: If you’re creating a community for (former) Facebook users, they’ll have a familiar feeling in Friendica. The interface is clean and usable, it supports a lot of protocols (ActivityPub, OStatus, diaspora), it supports plugins, can import websites via rss (so automatic post is easy).

Please, consider that: Moderation tools are different from the ones you have on Mastodon or Akkoma (Pleroma, etc.) and you can’t easily report users, especially remote ones as support for Mastodon API is limited. With the current growth of users, it’s easy to find a bad person trying to disturb. Not having effective ways to deal with it may be frustrating.

Why I’m suggesting another solution: I didn’t try Friendica long enough to find some big problems with it.

To sum up: Generally speaking, it is considered a solid and stable solution, actively maintained and, being in php, portable. If you want to create a Facebook-like community, that's the way to go.

snac2

snac2 is a simple, minimalistic ActivityPub instance that supports the Mastodon API. This makes it compatible with platforms like Pleroma, Akkoma, and Mastodon itself. It's written in portable C and it's been created to be light, easy to deploy, and with only two dependencies: openssl and curl. It heavily relies on hard links and doesn't need any database. This is a big, big plus for me. I've performed many tests and found that this is one of the best lightweight solutions to join the Fediverse.

Why I am suggesting it: snac2 is clean and polished. The federation with the other solutions is good, it works beautifully with Tusky (on Android) and tooot (both on Android and iOS) - also consider Enafore as a PWA or web interface - and its integrated web interface is minimal but effective. No javascript, no cookies - clean web. More, the dev is responsive and open to patches and contributions. I've helped with some stress tests and contributed with instructions and patches to make it work on FreeBSD and NetBSD, and they've been merged immediately.

Please, consider that: It's not Mastodon. Some of the Mastodon API features are (currently) not supported so the experience could be different from the other solutions. Some Mastodon apps don't work (the official Mastodon app, for example, can't login). There's no open registration option (users should be manually registered from the cli) and account migration is not supported, at the moment. More, while it's not caching external media, locally published media will stay on the local drives (no S3 upload option), so be prepared to serve those files as well. Testing it from my FTTC connection, I almost DDoSed my internet connection for 15 minutes, publishing a photo.

Why I’m suggesting another solution: Actually, I'm suggesting to try snac2. I think it could be a great solution for a single user instance or for instances managed by tech people, as you can run it just 1 minute after the download. It's light, easy, straightfoward and the dev is a nice person. I'd suggest other solutions if the priority is to offer a full, feature rich Fediverse experience, with registration, big and featureful configuration panels, etc

To sum up: snac2 is a lightweight and effective way to join the Fediverse. Currently, it's my favourite solution for small communities as the "file only" approach and no dependencies are coherent with my ideas. I've migrated my instance from FreeBSD to NetBSD, from external datacenters to my home network and it's just been a matter of a single rsync (-H)

GoToSocial

GoToSocial is a new microblogging platform. Its target it to be a light, customisable, easy to manage and integrated software. It is still at its early alpha stage, it should evolve into a beta at some point in ~~2023~~ 2024. It’s developed in Go and installation in easy and fast, it supports Postgres, Mysql and Sqlite. Basic functionalities have already been integrated and it can federate with (almost) all the other ActivityPub implementations, even if with some small problems. FreeBSD installation was easy and fast, as they provide a amd64 FreeBSD binary. Being written in go, it shouldn't be difficult to self-compile it for other architectures.

Why I am suggesting it: While it doesn’t have an integrated frontend (but Mastodon mobile apps or other frontends can be used, as Enafore Mangane, etc.), it already supports many of the features you’d expect from an ActivityPub implementation. It's fast, suitable for installation on a low end hardware, can be deployed without a reverse proxy as has integrated support for Letsencrypt certificates. But…

Please, consider that: It’s still at alpha stage. Things can still break and it doesn’t support any kind of account migration from Mastodon (while it’s supported by Pleroma and its forks).

Why I’m suggesting another solution: While I think it could be a game changer, I think it’s bit early to deploy it unless you’re a very skilled and experienced administrator. You should understand its “quirks and features”, mainly tied to its alpha status. While I’ve installed and will keep installed a GoToSocial instance, at the moment I’ll just keep it in a test stage server in order to follow its development status.

To sum up: GoToSocial could potentially become one of the most interesting ActivityPub microblogging platforms, at least for small and medium sized communities. It's a bit early to consider it ready for a production deployment, but it's already worth testing.

Frontends

All the Fediverse backend implementations have their own specific features but the users will just interact via a frontend. While Mastodon - but also Pixelfed, Peertube, etc. - are presented as a specific stack (backend + frontend), there are many frontends that can interact via the Mastodon API.

Of course, being the “Mastodon” API, not all the backends are perfectly compatibile/supported by all the frontends. Mobile apps use the Mastodon API, that’s why their compatibility with other implementations like snac2, Pleroma, Akkoma, etc. may not be perfect at all the times.

I won’t describe them all. I’ll just enumerate the ones I’m using, with some notes:

Akkoma’s Pleroma-FE

With the 2022.12 release of Akkoma, Pleroma-FE has been evolved into a proper, nice looking PWA. After an initial configuration of its many options, I found it quite nice and effective. The only problem (common with many PWAs on iOS) is that when you suspend the app, it doesn’t detect it, so if you open it again after two hours, you’ll just see the recent posts, not all the posts of the last two hours. This is because of the way iOS deals with app suspend/resume and many PWAs don’t seem to understand they’ve been suspended.

Mangane

Mangane is a Soapbox fork that aims to improve compatibility with Akkoma. It is nice and clean and the developers are improving it to support Akkoma's features, so it's great. At the moment I've noticed some visual problems, on iOS, with the icons - but Guérin contacted me to ask for information (after reading this article) and opened an issue on to fix it. Guérin has been nice and helpful, making Mangane even more appealing. I’m using it for planned posts on Akkoma, as Pleroma-FE doesn’t support them, yet. I'm also using Mangane as a daily driver, from time to time, as I like it.

Enafore

A light, fast, complete and usable web frontend. I’ve been using it when I needed a simple, clear frontend. It supports “hide replies”, which is great to improve timeline quality. It's among my favourite choices when using a webapp and is a good choice for snac2.

Elk

Elk is a frontend for Mastodon API. It’s nice, clear, eye candy, intelligent. The developers are nice people, open to suggestions. Development is a bit slower compared to the initial pace, but the app is already very complete and stable.

Elk is definitely a very good piece of software, and I recommend to try it.

Moderation and final considerations

One of the things you should be considering is that deploying a piece of the Fediverse isn’t just installing a software and interacting with others. Actually, that’s just the easiest part of the experience, at least if you’re not creating a single user instance.

Many of the people that joined the Fediverse in its early days decided to do it as they felt attacked on other social networks. In the last years, the commercial socials have proven to be the perfect place for negative people, attacking others without being blocked/stopped in an efficient way. Hate causes addiction and the owners of those commercial socials make a lot of money if people interact, showing them ads every time they open the app/website. They make money (also) through people hating each other.

The Fediverse gives the possibility to mute and block users, but also to mute and block entire instances. One of the main tasks of an instance’s administrator is to make sure that everything is ok. While you can define your own instance’s rules, other instances’ admins may block you if they find you’re federating by sending messages agains their rules.

As an administrator, you’re also responsible of keeping your users’ data safe, to avoid sharing/providing illegal contents or offensive stuff. So you’re free to set your rules, but others are free to “defederate” you, if they don’t like the contents your instance is providing. While it’s not an issue for a single user instance, you must be quite careful when opening the registrations as you may find out you've been defederated because of (your) lack of moderation.

There’s some strong criticism against the Fediverse because of this, as many see it as more “censored” than the traditional, commercial social networks. I don’t think it’s true, as you’re free to fire up your instance, decide your rules and act as you want. But you can’t impose others and their instances to follow you, even if you think you’re right. In a free world, everybody should be free to decide if they want to listen to you. But everybody should also be free to create a new space and start sharing their ideas. There’s not a central authority of contents, so only the single admins and users can choose what they want to see or avoid. There's not a commercially driven algorithm that may decide that you should see contents that will cause you anger and hate just because it generates traffic (and money) to the social's owner.

The Fediverse can be a beautiful place to stay.

Installing Mastodon inside a FreeBSD jail: A Comprehensive Guide

Stefano Marinelli — Wed, 23 Nov 2022 07:52:02 +0000

Note: Updated for Mastodon 4.5

Introduction

Mastodon and the Fediverse have gained significant popularity, especially during times of uncertainty with traditional social media platforms. As users seek alternative spaces, many are discovering the decentralized nature of Mastodon instances. However, this influx of users has led to challenges for unprepared instances, including performance issues and moderation difficulties.

This guide aims to provide a comprehensive walkthrough for installing Mastodon on a FreeBSD jail, managed by BastilleBSD. While Mastodon documentation tends to be Linux-centric, this tutorial fills the gap for FreeBSD users.

Note: This guide describes a simple, single-jail installation. For production environments, it's recommended to separate services (Valkey, PostgreSQL, etc.) into individual jails. This tutorial assumes a basic understanding of FreeBSD and Unix-like systems.

Prerequisites

A FreeBSD system with BastilleBSD installed
Basic knowledge of FreeBSD jail management
Familiarity with command-line operations

Step 1: Creating the Jail

Let's start by creating a new jail using BastilleBSD:

bastille create mdontest 14.3-RELEASE 10.0.0.42 bastille0

Step 2: Configuring the Jail

As we'll be installing PostgreSQL in the jail, we need to add some configurations to the jail's jail.conf:

sysvmsg=new;
sysvsem=new;
sysvshm=new;

After adding these lines, restart the jail:

bastille restart mdontest
bastille console mdontest

Step 3: Installing Dependencies

Now, let's install the necessary packages:

pkg install -y curl wget gnupg gmake git-lite vips node22 yarn-node22 postgresql18-server postgresql18-contrib ImageMagick7 ffmpeg autoconf nginx valkey py311-certbot py311-certbot-nginx sudo rubygem-bundler rubygem-posix-spawn

Step 4: Enabling and Configuring Services

Enable Valkey, Nginx, and PostgreSQL:

service valkey enable
service nginx enable
service postgresql enable

Valkey Configuration

For simplicity in this jail environment, we'll disable Valkey's protected mode. However, this is not recommended for production environments without proper security measures.

Edit /usr/local/etc/valkey.conf and set:

protected-mode no

Important: In a production environment, ensure proper authentication and network security measures are in place before disabling protected mode.

PostgreSQL Initialization and Configuration

Initialize the PostgreSQL database:

service postgresql initdb

Modify PostgreSQL to accept connections from the jail's services. Edit /var/db/postgres/data18/pg_hba.conf and add:

host    all    all    10.0.0.42/32    trust

Note: In a production environment, consider using more restrictive authentication methods.

Start PostgreSQL and Valkey:

service postgresql start
service valkey start

Step 5: Database Setup

Create the Mastodon database user:

sudo -u postgres psql
CREATE USER mastodon CREATEDB;
\q

Step 6: Creating the Mastodon User

Create a dedicated user for Mastodon:

pw add user mastodon -m
echo 'export LC_ALL="en_US.UTF-8"' >> /home/mastodon/.profile

Step 7: Installing Mastodon

Enable corepack, switch to the Mastodon user and install the software:

corepack enable
su -l mastodon
git clone https://github.com/mastodon/mastodon.git live && cd live
git checkout $(git tag -l | grep '^v[0-9.]*$' | sort -V | tail -n 1)
corepack prepare

Install Ruby and Node dependencies:

export CONFIGURE_ARGS="--with-cflags=\"-Wno-error=incompatible-function-pointer-types\""
export NODE_OPTIONS="--openssl-legacy-provider"
bundle config deployment 'true'
bundle config without 'development test'
bundle install -j$(getconf _NPROCESSORS_ONLN)
yarn install --immutable

Step 8: Mastodon Setup

Run the Mastodon setup command:

RAILS_ENV=production bundle exec rake mastodon:setup

When prompted for the PostgreSQL host, enter 127.0.0.1 (or 10.0.0.42).

Mastodon can now use libvips as a lighter and more modern alternative to ImageMagick. ImageMagick support is being deprecated, so it's suggested to switch to libvips.

To use libvips instead of ImageMagick, set the MASTODON_USE_LIBVIPS environment variable to true into the .env.production:

[...]
MASTODON_USE_LIBVIPS=true

Step 9: Nginx Configuration

In the dist/ directory, you'll find an nginx.conf file. This is not a complete Nginx configuration but a partial one for Mastodon. Integrate this with your existing Nginx setup based on your specific requirements.

Note: Many administrators advise against exposing Mastodon through Cloudflare, as it may interfere with some APIs and disrupt Fediverse interactions.

Step 10: Creating FreeBSD RC Scripts

To manage Mastodon services on FreeBSD, we'll create custom rc scripts. You can find the scripts for mastodon_sidekiq, mastodon_web, and mastodon_streaming at the provided links.

Place these scripts in the /usr/local/etc/rc.d/ directory, make them executable (chmod a+rx /usr/local/etc/rc.d/mastodon_*) and enable them:

service mastodon_sidekiq enable
service mastodon_web enable
service mastodon_streaming enable

Conclusion

Restart the jail or start the services individually. Logs will be appended to /var/log/messages.

You now have a functioning Mastodon instance running in a FreeBSD jail. All services are run by the "daemon" user and are supervised.

Remember to regularly update your Mastodon instance and monitor its performance. For production environments, consider implementing additional security measures and potentially separating services into individual jails.

If you want to change the characters or poll limits, you can refer to this article.

Enjoy your new Mastodon instance!

IT Notes - mastodon

Aggressive caching for a Mastodon reverse proxy: what to cache, what to never cache, and why content negotiation will eventually betray you

Assumptions before anything else

The cache zone

Default location: the microcache and streaming without cache

Upstream and failover

What I log and why

How I check it actually works

Caveats worth being honest about

Self-hosting your Mastodon media with SeaweedFS

Setting up the SeaweedFS Jail

Configuring Buckets and Users

Nginx Reverse Proxy Configuration

Mastodon Configuration

Increasing or Modifying Character Limits and Poll Options in Mastodon 4.3, 4.4 and 4.5

Increasing or Modifying the Character Limit

Increasing or Modifying the Number of Poll Options

Using a Permanent WebFinger Address for My Fediverse Profile

Own Your Data

A Permanent Address

In Practice

Nginx Configuration

WebFinger Exception

JSON File for WebFinger

Testing the Setup

Conclusion

A Small Compendium of Fediverse Platforms I Use

Conclusion

Deploying a piece of the Fediverse

Backends / Complete Solutions

Akkoma (Pleroma fork)

Misskey (and its forks like Firefish, etc.)

Other solutions

Frontends

Akkoma’s Pleroma-FE

Moderation and final considerations

Installing Mastodon inside a FreeBSD jail: A Comprehensive Guide

Introduction

Prerequisites

Step 1: Creating the Jail

Step 2: Configuring the Jail

Step 3: Installing Dependencies

Step 4: Enabling and Configuring Services

Valkey Configuration

PostgreSQL Initialization and Configuration

Step 5: Database Setup

Step 6: Creating the Mastodon User

Step 7: Installing Mastodon

Step 8: Mastodon Setup

Step 9: Nginx Configuration

Step 10: Creating FreeBSD RC Scripts

Conclusion