* Stamp deleted attachments with their deletion time on every delete path
The socket handlers carried their own copies of moveUploadToDeleted that
skipped the utimes stamp, so an old attachment deleted from chat kept its
upload time and was purged at the next 15 minute sweep instead of after the
retention window. All callers now share the one implementation in
src/uploadFiles.js.
* Route every upload move and delete through a storage layer
src/storage holds the driver for uploads/ (local disk for now) and keeps a
new media_objects index in step with each write, so a bucket driver can sit
behind the same calls later. Soft deletes, permanent deletes, purges and
restores all go through it; src/uploadFiles.js is gone.
Storage calls are asynchronous, and every caller now awaits them before it
answers or emits: message, bulk, channel, sub-channel, DM and group DM
deletes, kicks from a group, bot deletes, auto-cleanup, the encrypted file
release, the uploads trim, self-destruct, and the sound, emoji, sticker and
badge icon deletes. Self-destruct sweeps no longer overlap when a timer fires
mid-sweep, and stop() keeps a running sweep from re-arming the timer.
The storage layer accepts any path that stays inside uploads/, so names that
keep an uploader's unusual extension still move and delete as before.
* Serve /uploads from the storage layer instead of express.static
src/storage/serve.js reads each file through the storage driver and streams
it out, so the same safety headers will hold for files kept in a bucket. It
answers as express.static did: the same weak ETag, Last-Modified, the 7 day
immutable cache, conditional GETs and 304s, a single byte range with 206,
HEAD, the dotfile refusal, and falling through to the app's 404, with a
failed precondition or an unsatisfiable range still handed to the error
handler. The guard that refuses deleted-attachments/ and bot-audio/ in every
encoding moved into the same module unchanged.
test/uploadsServe.test.js serves one folder through the old express.static
setup and the new handler side by side and compares status, headers and
body for each file type, conditional request, range and refused path.
One difference: a folder path without a trailing slash (/uploads/stickers)
now gets the 404 directly instead of a 301 to the same 404.
* Admit uploads to storage only after their route has checked them
A cleaned upload now waits in unprocessed/staging/ (never served) while its
route checks the magic bytes and fixes the extension, and is then admitted
to storage with storage.admitUpload before anything stores its URL. A
failed admit answers 500 and leaves nothing behind; a restart sweeps any
staged file a cut-short request left.
Covers attachments and files, avatars, borders, bot and persona avatars,
sounds, emoji and stickers (single and bulk), badge icons, the server icon
and banner, role icons, sign-in avatars and the starter sticker pack. Bulk
emoji and sticker uploads report a file that could not be stored as an error
and carry on with the rest. A persona avatar refused as someone else's
persona is now removed instead of left in uploads.
* Read upload sizes from the media index instead of walking the folder
The moderator usage report, the uploads trim, the Files & Media sizes, the
check for files a bot delete may move, and the deleted-attachments purge now
read media_objects rather than stat'ing or walking uploads/, so none of them
needs a listing of the bucket once storage can be one. The report's numbers
are unchanged; the trim still counts the temporary bot audio kept on disk,
as before. The purge counts the retention window from the deletion time the
index records, and the local driver still prunes the sub-folders it empties.
The server only starts listening once storage.prepare has checked storage
answers; a failure exits with a message rather than starting a server that
cannot serve media. On the first start after this change it builds the index
from what is already in uploads/ (src/storage/reconcile.js), once.
scripts/reconcile-media.js runs the same repair by hand. The privacy sweep
keeps the index in step when it rewrites, holds back or returns a file.
* Add an S3-compatible storage driver for uploads
HAVEN_EXTERNAL_STORAGE=true keeps media in a bucket (HAVEN_S3_ENDPOINT,
_BUCKET, _REGION, _ACCESS_KEY, _SECRET_KEY, _PREFIX, _FORCE_PATH_STYLE)
through src/storage/s3.js, behind the same calls as the local driver: put
(multipart over 5 MB), head, ranged reads, soft delete and restore by copy
and delete, and paged listing. Haven still streams every file itself, so the
safety headers are unchanged. Missing settings or a bucket that does not
answer stop the server at boot with the endpoint, bucket and reason.
HAVEN_S3_LOCAL_FALLBACK=true serves files not yet moved from uploads/ on
disk; deleting then removes the local copy too, so it cannot come back.
Without it, files left on disk are counted in a warning at boot.
Checksums are only sent and checked where the API needs them: the SDK's
default CRC32 is rejected by stores such as Garage for files sent in parts.
test/storageS3.integration.test.js runs against a real bucket when
HAVEN_TEST_S3_ENDPOINT is set (passes on Garage 1.1.0) and is skipped
otherwise. Not documented for operators yet: Ferry, badges, templates and
backups still read uploads/ directly.
* Clean photos in the bucket, release replaced profile pictures, reconcile on demand
The privacy sweep has a pass of its own for media kept in a bucket: it goes
by the media index, cleans each photo and video in unprocessed/ and writes
it back only when cleaning changed it, and holds back a file it cannot clean
the same way it does on disk. Its progress is kept per driver, so moving to
a bucket checks everything there once.
A replaced or removed avatar or border, and a persona's avatar when it is
replaced or the persona is deleted, now go to deleted-attachments instead of
staying in uploads forever, but only when the file is that person's own
upload and nothing else still shows it. A persona can point at anyone's
upload, so the ownership check is what keeps one person from taking another
person's file. Border uploads now record their owner as avatars do.
Admins can rebuild the media index from storage with
POST /api/admin/media/reconcile, the same repair as
scripts/reconcile-media.js.
* Read uploads through storage everywhere, and move servers to a bucket
The last readers of uploads/ on disk now go through src/storage:
- Badge icons are keyed from storage (imageIconKey); built-in and emoji keys
stay synchronous.
- Ferry checks Discord's size limits against the media index, then reads each
file's bytes once, so a retry no longer reads the files again.
- Server templates read and write images through storage. exportTemplate and
applyTemplate are now async; images still go in before the database
changes and are removed if those fail. tools/template.js uses the target
server's own storage, bucket included.
- Backups with files stream them out of the bucket one at a time. A restore
marks the restored database so the next start rebuilds the media index,
and on a bucket first copies the restored files into it
(media_restore_pending), so the restore no longer has to send gigabytes
in the moment before it restarts.
scripts/migrate-uploads-to-s3.js moves an existing server's files into the
bucket while it stays up (with HAVEN_S3_LOCAL_FALLBACK=true), skips what is
already there, and with --delete-local frees the disk only once the bucket
holds everything. GUIDE.md, docker-compose.yml and .env.example now document
external storage.
Tested against Garage 1.1.0: a live move from disk to the bucket with
nothing going missing, and a backup taken on the bucket restored back into it.
* Let the media scripts read the server's .env
migrate-uploads-to-s3.js and reconcile-media.js only saw HAVEN_S3_* when
they were set in the shell, so on an install that keeps its settings in the
data folder's .env, as a non-Docker install does, the migration refused to
run. They now load that .env the way server.js does; anything set in the
shell still wins.
* More documentation in .env.example
4.5.0 taught the server to make its certificate itself, but Start Haven.bat, Install Haven.ps1 and the web installer still went looking for openssl.exe first and reported a skipped certificate as done. They now call the same generator through scripts/gen-cert.js, the certificate carries the CA and server-auth flags phones expect when you import it, and the guide and support page stop telling people to install OpenSSL. Reported by MutantRabbit767.
- Sidebar panel no longer auto-opens on app launch when sidebar mode was
previously enabled. Panel always starts hidden; the toggle arrow on the
right edge must be clicked to open it.
- Manage tab: custom sounds now appear above built-in sounds.
- Manage tab: both sections (CUSTOM and BUILT-IN) are now collapsible
via native <details>/<summary> elements.
- Soundboard tile eye icon changed from display:none/inline to
visibility:hidden/visible so it always occupies space. Hovering no
longer shifts the tile content.
- Disabling sidebar mode now closes the panel immediately rather than
leaving it open in the background.
- Soundboard sidebar collapse now shrinks the panel to header height only
(bottom: auto when collapsed) so the whole panel folds up, not just the
sound grid.
- Users/voice sidebar collapse arrow repositioned to vertical center
(top: 50% / translateY(-50%) instead of fixed top: 56px).
- Built-in sounds (AOL set) can now be removed from the admin Manage tab.
Deleting a built-in adds it to a new disabled_builtin_sounds DB table
so it no longer appears in the soundboard or event-assign selects.
- Removed the lock icon on built-in sounds and gave them a delete button.
- Fixed garbled emoji characters (pencil and trash can) in the Manage tab
caused by double-encoded UTF-8 bytes; replaced with HTML entity forms
that are immune to encoding corruption.
Restored ~70 corrupted emojis across docs/index.html and website/index.html (matched contextually: nav, hero, channel-code, platform tags, feature icons, footer).
Added four-layer prevention so this doesn't ship again:
- UTF-8 BOM on the landing pages (first 3 bytes EF BB BF)
- .editorconfig pinning charset=utf-8-bom for docs/website index + support
- .gitattributes working-tree-encoding=UTF-8 for landing pages, CHANGELOG, README, GUIDE
- scripts/fix-emoji-encoding.py reusable repair script
Verified hashes match between docs and website copies. Google Ads tag AW-17950126697 already present (2 occurrences in each file).
The ask was to decouple translations from the release cycle so
community translators don't have to wait on a maintainer to merge
strings. Loading translations from a CDN was offered, but pulling
untrusted JSON into the UI at runtime adds attack surface and breaks
the offline-friendly self-hosted story.
Instead, automate the merge path that already exists:
- New scripts/validate-locales.js checks every public/locales/*.json
for JSON validity, _meta fields, and shape compatibility with
en.json (same key paths, same value types). Missing keys are
warnings (partial translations are OK, English is the fallback).
Also runnable locally with 'npm run validate-locales'.
- New .github/workflows/translations-autovalidate.yml runs that
validator on every PR that touches public/locales/**. If the PR is
translations-only (no en.json, no code) and validation passes,
it labels the PR 'translations-only' and turns on GitHub's
auto-merge so it goes in as soon as required checks are green.
- Workflow runs on pull_request (not pull_request_target) so it
cannot read secrets from fork PRs.
Kamden Bolhuis