Kamden Bolhuis 6872049e71 Feature: External S3 Media Storage (#5813)
* Stamp deleted attachments with their deletion time on every delete path

The socket handlers carried their own copies of moveUploadToDeleted that
skipped the utimes stamp, so an old attachment deleted from chat kept its
upload time and was purged at the next 15 minute sweep instead of after the
retention window. All callers now share the one implementation in
src/uploadFiles.js.

* Route every upload move and delete through a storage layer

src/storage holds the driver for uploads/ (local disk for now) and keeps a
new media_objects index in step with each write, so a bucket driver can sit
behind the same calls later. Soft deletes, permanent deletes, purges and
restores all go through it; src/uploadFiles.js is gone.

Storage calls are asynchronous, and every caller now awaits them before it
answers or emits: message, bulk, channel, sub-channel, DM and group DM
deletes, kicks from a group, bot deletes, auto-cleanup, the encrypted file
release, the uploads trim, self-destruct, and the sound, emoji, sticker and
badge icon deletes. Self-destruct sweeps no longer overlap when a timer fires
mid-sweep, and stop() keeps a running sweep from re-arming the timer.

The storage layer accepts any path that stays inside uploads/, so names that
keep an uploader's unusual extension still move and delete as before.

* Serve /uploads from the storage layer instead of express.static

src/storage/serve.js reads each file through the storage driver and streams
it out, so the same safety headers will hold for files kept in a bucket. It
answers as express.static did: the same weak ETag, Last-Modified, the 7 day
immutable cache, conditional GETs and 304s, a single byte range with 206,
HEAD, the dotfile refusal, and falling through to the app's 404, with a
failed precondition or an unsatisfiable range still handed to the error
handler. The guard that refuses deleted-attachments/ and bot-audio/ in every
encoding moved into the same module unchanged.

test/uploadsServe.test.js serves one folder through the old express.static
setup and the new handler side by side and compares status, headers and
body for each file type, conditional request, range and refused path.

One difference: a folder path without a trailing slash (/uploads/stickers)
now gets the 404 directly instead of a 301 to the same 404.

* Admit uploads to storage only after their route has checked them

A cleaned upload now waits in unprocessed/staging/ (never served) while its
route checks the magic bytes and fixes the extension, and is then admitted
to storage with storage.admitUpload before anything stores its URL. A
failed admit answers 500 and leaves nothing behind; a restart sweeps any
staged file a cut-short request left.

Covers attachments and files, avatars, borders, bot and persona avatars,
sounds, emoji and stickers (single and bulk), badge icons, the server icon
and banner, role icons, sign-in avatars and the starter sticker pack. Bulk
emoji and sticker uploads report a file that could not be stored as an error
and carry on with the rest. A persona avatar refused as someone else's
persona is now removed instead of left in uploads.

* Read upload sizes from the media index instead of walking the folder

The moderator usage report, the uploads trim, the Files & Media sizes, the
check for files a bot delete may move, and the deleted-attachments purge now
read media_objects rather than stat'ing or walking uploads/, so none of them
needs a listing of the bucket once storage can be one. The report's numbers
are unchanged; the trim still counts the temporary bot audio kept on disk,
as before. The purge counts the retention window from the deletion time the
index records, and the local driver still prunes the sub-folders it empties.

The server only starts listening once storage.prepare has checked storage
answers; a failure exits with a message rather than starting a server that
cannot serve media. On the first start after this change it builds the index
from what is already in uploads/ (src/storage/reconcile.js), once.
scripts/reconcile-media.js runs the same repair by hand. The privacy sweep
keeps the index in step when it rewrites, holds back or returns a file.

* Add an S3-compatible storage driver for uploads

HAVEN_EXTERNAL_STORAGE=true keeps media in a bucket (HAVEN_S3_ENDPOINT,
_BUCKET, _REGION, _ACCESS_KEY, _SECRET_KEY, _PREFIX, _FORCE_PATH_STYLE)
through src/storage/s3.js, behind the same calls as the local driver: put
(multipart over 5 MB), head, ranged reads, soft delete and restore by copy
and delete, and paged listing. Haven still streams every file itself, so the
safety headers are unchanged. Missing settings or a bucket that does not
answer stop the server at boot with the endpoint, bucket and reason.

HAVEN_S3_LOCAL_FALLBACK=true serves files not yet moved from uploads/ on
disk; deleting then removes the local copy too, so it cannot come back.
Without it, files left on disk are counted in a warning at boot.

Checksums are only sent and checked where the API needs them: the SDK's
default CRC32 is rejected by stores such as Garage for files sent in parts.

test/storageS3.integration.test.js runs against a real bucket when
HAVEN_TEST_S3_ENDPOINT is set (passes on Garage 1.1.0) and is skipped
otherwise. Not documented for operators yet: Ferry, badges, templates and
backups still read uploads/ directly.

* Clean photos in the bucket, release replaced profile pictures, reconcile on demand

The privacy sweep has a pass of its own for media kept in a bucket: it goes
by the media index, cleans each photo and video in unprocessed/ and writes
it back only when cleaning changed it, and holds back a file it cannot clean
the same way it does on disk. Its progress is kept per driver, so moving to
a bucket checks everything there once.

A replaced or removed avatar or border, and a persona's avatar when it is
replaced or the persona is deleted, now go to deleted-attachments instead of
staying in uploads forever, but only when the file is that person's own
upload and nothing else still shows it. A persona can point at anyone's
upload, so the ownership check is what keeps one person from taking another
person's file. Border uploads now record their owner as avatars do.

Admins can rebuild the media index from storage with
POST /api/admin/media/reconcile, the same repair as
scripts/reconcile-media.js.

* Read uploads through storage everywhere, and move servers to a bucket

The last readers of uploads/ on disk now go through src/storage:

- Badge icons are keyed from storage (imageIconKey); built-in and emoji keys
  stay synchronous.
- Ferry checks Discord's size limits against the media index, then reads each
  file's bytes once, so a retry no longer reads the files again.
- Server templates read and write images through storage. exportTemplate and
  applyTemplate are now async; images still go in before the database
  changes and are removed if those fail. tools/template.js uses the target
  server's own storage, bucket included.
- Backups with files stream them out of the bucket one at a time. A restore
  marks the restored database so the next start rebuilds the media index,
  and on a bucket first copies the restored files into it
  (media_restore_pending), so the restore no longer has to send gigabytes
  in the moment before it restarts.

scripts/migrate-uploads-to-s3.js moves an existing server's files into the
bucket while it stays up (with HAVEN_S3_LOCAL_FALLBACK=true), skips what is
already there, and with --delete-local frees the disk only once the bucket
holds everything. GUIDE.md, docker-compose.yml and .env.example now document
external storage.

Tested against Garage 1.1.0: a live move from disk to the bucket with
nothing going missing, and a backup taken on the bucket restored back into it.

* Let the media scripts read the server's .env

migrate-uploads-to-s3.js and reconcile-media.js only saw HAVEN_S3_* when
they were set in the shell, so on an install that keeps its settings in the
data folder's .env, as a non-Docker install does, the migration refused to
run. They now load that .env the way server.js does; anything set in the
shell still wins.

* More documentation in .env.example
2026-10-20 18:03:57 -04:00
2026-10-17 21:13:13 -04:00
2026-10-17 21:13:13 -04:00
2026-10-17 21:13:13 -04:00
2026-10-17 17:40:36 -04:00
2026-10-17 17:40:36 -04:00

⬡ HAVEN

A private Discord alternative that runs on your own machine. No cloud, no company account, no telemetry, no paid tier. Free forever.

Version License Node Platform

One person runs the server. Everyone else joins with a code. Messages, files and voice never touch anybody else's infrastructure.

Haven's default look: channels and DMs on the left, the member list on the right

Get started in about a minute

Docker

docker run -d -p 3000:3000 -v haven_data:/data ghcr.io/ancsemi/haven:latest

Windows, no Docker: download the zip, unzip it, double-click Start Haven.bat. It installs Node.js for you if you do not have it.

Linux / macOS, no Docker: chmod +x start.sh && ./start.sh

Then open https://localhost:3000, register with the username admin, make a channel, share the code. The certificate warning on first load is expected, click Advanced then Proceed.

One-click cloud deploy on Zeabur · on RepoCloud · Full setup guide


Three ways in

🖥️ Haven Desktop Windows and Linux. Per-app audio sharing (send one app’s sound, nothing else), device switching mid-call, native notifications, tray. It can also run the server itself.
📱 Haven for Android (Google Play)
💠 Android (via Obtainium)
On Google Play, or as a direct download through Obtainium. Native, not a web wrapper. Full chat, voice and push notifications. Built by Amnibro.
🍎 Haven for iOS In testing through TestFlight. Native, by Amnibro as well. Until it is on the App Store, the browser works too: open your server in Safari and Add to Home Screen.
🌐 Any browser Nothing to install. Send a link, they are in. Works on phones.

Apps and browser talk to the same server and see the same thing, so nobody has to install anything to be included.

Want to look around first? The community server is open, no code needed, just sign up. Volunteer-hosted, thanks MutantRabbit.


Why bother

Discord Haven
Hosting Their cloud Your machine
Account Email and phone No email, no verification
Your messages Stored by Discord Inc. Never leave your server
Cost Nitro, boosts Free, no paid tier
Telemetry Analytics and tracking None
Source Closed Open, AGPL-3.0

What you get

Coming from Discord

  • Import your whole server history from inside Haven. Every channel, thread, forum post, reaction, pin and attachment. Paste a token, pick a server, go.
  • Ferry, a two-way Discord bridge. Messages pass both ways between a Haven channel and a Discord channel, under each person's real name. Friends who are not ready to leave Discord can still talk to the ones who did.

Talking

  • Voice chat, peer to peer by default so audio doesn't pass through the server (an optional voice relay is there for big servers). Per-user volume, mute, deafen, noise suppression, talking indicators.
  • Screen sharing, several people at once in a tiled grid. Watch parties work.
  • Listen together with synced playback from Spotify, YouTube and SoundCloud, or a music player right in the voice channel.
  • Threads, replies, reactions, pins, editing, mentions, polls, search, link previews, code blocks, spoilers.
  • Direct messages, end-to-end encrypted. Private keys never leave the browser, so not even the person hosting can read them.
  • Rich presence for what people are playing and listening to, via Steam, Last.fm or Spotify. Off until you turn it on, never written to the database.

Making it yours

  • 26 built-in themes, a colour picker, an RGB cycle, and stackable visual effects like CRT, Matrix rain and snowfall.
  • A plugin and theme system, so you can drop in your own without forking anything.
  • Custom stickers, custom emoji, personas (send as an alternate character), soundboard, avatar shapes and borders.
  • Server branding, sidebar layout options, font size and density controls.
  • 11 languages out of the box: English, Français, Deutsch, Español, Polski, Русский, 中文, Português, فارسی, 日本語 and 한국어.

Running it

  • One-click updates from inside the app. Haven takes a pre-update backup and restarts itself.
  • Built-in backups, scheduled or on demand, with restore.
  • Single sign-on via OpenID Connect, so Authentik, Keycloak, Authelia and Auth0 all work as your login.
  • Bots and webhooks with a REST API. Send messages, play soundboard sounds, register slash commands, join voice, receive signed callbacks.
  • Roles with per-channel permissions. Kick, timed mute, ban, IP bans, slow mode, automod link filtering, an audit log, and a moderation API.
  • Guest accounts, invite links, per-channel join codes with automatic rotation.
  • Push notifications, including UnifiedPush if you would rather leave Google out.
  • Built-in Cloudflare and LocalTunnel support, so friends can reach you without port forwarding.
  • Bcrypt passwords, JWT auth, HTTPS, MFA, rate-limited logins, and a disk reserve so a full drive never wedges the server.
A community channel in the Braid theme

Documentation

Everything in detail lives in GUIDE.md: setup, Docker, remote access (port forwarding, Tailscale, Cloudflare Tunnel, reverse proxies), themes, voice and TURN, Discord import, Ferry, encryption, backups, configuration, translations, the bot API, and troubleshooting.

Release notes are in CHANGELOG.md.


Contributing

Issues and PRs are welcome, including translations and themes. If you are reporting a bug, the version number from https://your-server:3000/api/version helps.

AI usage

Haven is built with AI. I'm not a professional programmer. I came up with Haven, I decide what goes in, and I test it and run it with my own community. Most of the code, the server logic included, was written by AI coding tools (mainly Claude Code) working from my direction, and most code review is AI-assisted too. Commits don't carry a per-commit AI label, so assume any part of the code may have been written by AI. Contributors sometimes use AI as well, and pull requests are welcome either way.

What keeps it in check:

  • An automated test suite (over 650 tests) runs before every push.
  • Every pull request is read in full and checked for security problems before it's merged.
  • Logins, permissions, encryption and uploads have had dedicated security reviews, and the fixes ship in public releases.
  • Bugs are reported and fixed in the open, here on GitHub.

License

AGPL-3.0. Free to use, modify and share. If you deploy a modified version as a network service, you have to publish its source. See LICENSE.

A DM window and profile card in the CRT theme Screen sharing with voice chat and more than one stream The login screen with the theme picker

⬡ Haven

Languages
JavaScript 79.8%
HTML 10.4%
CSS 8.6%
PowerShell 0.4%
Shell 0.4%
Other 0.4%