- Updated Go version from 1.24 to 1.26.5 in Dockerfile and CI configuration.
- Upgraded various dependencies in go.mod, including AWS SDK and OpenTelemetry packages, to their latest versions for enhanced features and security.
- Refactored Docker client imports from the Docker library to the Moby library for better compatibility with the updated API.
- Adjusted Docker client usage throughout the codebase to align with the new Moby client structure.
- Replaced client.IsErrNotFound with cerrdefs.IsNotFound for consistent error handling across container operations.
- Updated probe image version to alpine:3.23.5.
- Enhanced logging for container removal failures in flow tools, ensuring better traceability of issues.
- Improved context usage in tests for better cancellation handling.
ContainerInspect returns a not-found error once a container is gone from
the daemon, and IsContainerRunning wrapped it as an inspection failure, so
callers could not tell a missing container apart from an unreachable
daemon. StopContainer and RemoveContainer already special-case
client.IsErrNotFound; do the same here.
- Updated the documentation in `docker.md` to provide a detailed explanation of the explicit capability allow-list used for primary containers, emphasizing the rationale behind the selected capabilities and the deliberate omission of `MKNOD`.
- Revised comments in `client.go` and `tools.go` to reflect the decision against using `no-new-privileges`, clarifying its impact on privilege escalation testing and container security.
- Enhanced the overall clarity and completeness of the capability management section to aid understanding of security measures in the Docker-in-Docker setup.
- Added new environment variables `DOCKER_INSIDE_HOST`, `DOCKER_INSIDE_TLS_VERIFY`, and `DOCKER_INSIDE_CERT_PATH` to facilitate secure Docker access for worker containers.
- Updated `.env.example` and `docker-compose.yml` to reflect these new configurations.
- Enhanced backend logic to support the new Docker settings, ensuring proper isolation and security for worker containers.
- Expanded documentation to clarify the usage and implications of the new Docker access configurations.
- Introduced `TENANT_ID` to namespace resources for multi-instance deployments, preventing collisions in shared services.
- Added new environment variables in `.env.example` and `docker-compose.yml` for Docker daemon configurations (`DOCKER_INSIDE_HOST`, `DOCKER_INSIDE_TLS_VERIFY`, `DOCKER_INSIDE_CERT_PATH`).
- Updated backend logic to handle tenant-specific configurations, ensuring proper isolation of resources across instances.
- Enhanced documentation to clarify the usage and implications of `TENANT_ID` and related Docker settings.
- demuxExecStdout errors on a header cut short mid-frame (was a silent EOF that
dropped the tail) and surfaces a docker systemerr frame instead of discarding it.
- ftester drains telemetry on exit through the bounded observer.Drain instead of
two unbounded ForceFlush calls, so an unreachable collector can't hang it at exit.
- container-listing failures are sorted for a deterministic skipped-entries preview,
logged per-entry at Debug (the detail is already in the response and the endpoint
is hit on every navigation) with names quoted so control bytes in a hostile
filename can't inject into a log line; corrected the stat-failure comment
(a dangling symlink lstats fine, it doesn't fail).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The truncation slice and the demux stdout byte-cap had no docker-layer tests —
only the handler's Truncated wiring was exercised through the fake, so a
mis-slice or a dropped cap would have gone unnoticed. Extract find-output
parsing into a pure parseFindEntries and take the byte cap as a demuxExecStdout
parameter, then unit-test both boundaries (at cap / cap+1 / over-limit stream).
Also bound how many paths one container-files request may list
(maxContainerListPaths), so the per-path entry cap can't be multiplied by an
attacker-chosen path count into a large fan-out or response body.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Container-listing polish on top of the partial-success work:
- A directory whose entries all fail to stat no longer 500s. The find exec
already proved the container alive, so ListContainerDir returns the readable
entries (possibly none) plus the per-entry failures, and the handler serves a
200 partial listing; only a cancelled request or a dir that can't be listed at
all still errors.
- Over the entry cap, list the first page and set a Truncated flag rather than
erroring with end-user copy from the docker layer. The flag flows through to
the Pull dialog, which now warns the user the directory was truncated instead
of silently showing a subset.
- The Pull dialog no longer flashes a skeleton on an in-place refetch (Refresh,
or after a Pull): the listing hook clears its rows only when the path actually
changes, so a same-path reload keeps the current rows under the loading guard.
Tests: an all-entries-failed listing returns 200 with the failures; the
Truncated flag surfaces in the response. Swagger regenerated for the new field.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three defects on the shutdown and container-listing paths:
- Telemetry drain ran the langfuse and otel sinks serially under one deadline.
langfuse's ForceFlush/Shutdown ignore the caller ctx and block on their own
timeout, so a langfuse outage burned the whole budget and the healthy otel
sink's final batch was never flushed. Add observer.Drain, which flushes then
shuts down each sink in its own goroutine bounded by the deadline; main uses
it, retiring the inline flush-then-shutdown dance.
- The partial-success listing put the raw stat/list error into the 200 body's
Failures[].Message, exposing docker-layer detail (container ids, the daemon
address) that response.Error only reveals under develop mode. Send a generic
message to the client and keep the raw text dev-gated, logged server-side.
- A path read successfully by one query could also appear in Failures when
another query failed on it, order-dependent. Reconcile at assembly: drop any
failure whose path was read successfully, so no path is in both arrays.
Also bound the exec stdout buffered before parsing so a compromised sandbox
can't stream unbounded output into memory.
Tests: Drain delivers a healthy otel sink while a langfuse sink blocks and
returns within its deadline; the listing failure message is dev-gated; a path
never lands in both Files and Failures. Live: unreachable collector -> shutdown
stays bounded (~5s) and exits cleanly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
find -print0 already delimits with NUL, but the exec still ran under a TTY,
whose onlcr translation rewrites every \n in the stream to \r\n — including a
\n that is part of a filename — so a file literally named with a newline was
stat'd under the mangled name and reported unreadable. Run the exec without a
TTY and demux the resulting multiplexed stdout/stderr stream by hand (no new
dependency), so filename bytes reach the parser untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Parsing `ls -1` output was unsafe: under the TTY the exec allocates, GNU
coreutils shell-quotes names — so a readable file with a space, quote or
non-ASCII byte was stat'd with the quotes and reported unreadable — and
busybox wraps every entry in ANSI colour codes, so on an alpine/busybox
image the whole listing came back unreadable. Switch to `find -maxdepth 1
-print0`: literal bytes, portable across GNU and busybox, and the NUL
delimiter survives filenames containing newlines. Cap the entry count so a
directory with hundreds of thousands of files can't fan out into that many
Docker API calls, and classify a cancelled request or an all-entries-failed
listing as a directory-level error instead of a misleading empty 200.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A single unreadable directory entry (dangling symlink, a file removed
between ls and stat, a transient /proc entry) used to fail the whole
listing with HTTP 500, blanking the file browser and discarding every
readable sibling. The frontend already expects /proc/sys to not fail
spuriously, but the backend did the opposite.
ListContainerDir now returns a ContainerDirListing{Files, Failures}: per-
entry stat errors no longer abort the batch. GetFlowContainerFiles serves
the readable entries as HTTP 200, carries the failures back in a new
ContainerFiles.Failures field, and logs each skipped entry (capped) plus a
degradation summary. Directory-level faults (not a dir, ls failed, container
gone) still return 500 — there is no partial to show. statContainerEntries
returns successes + failures instead of the lowest-index error.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The deleted queue normalized workers<=0 to a default; the errgroup helper
passed the count straight to SetLimit, where 0 deadlocks the first Go() on an
unbuffered semaphore and a negative value disables the bound entirely. The
sole caller hardcodes 20 so this was latent, but clamp it back so a future
config-driven count can't hang the listing or flood the Docker daemon.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ListContainerDir was the only consumer of pkg/queue and discarded its ordered
delivery (the caller dedups and re-sorts). Replace the fan-out with a small
errgroup helper — SetLimit keeps the same 20-way bound on concurrent stats
against the Docker daemon, the caller's context propagates into each call,
results stay in input order, and the lowest-index failure is surfaced
deterministically — then delete pkg/queue. The helper was verified byte-
identical to the previous queue implementation across 600 randomized directory
shapes (sizes, failure positions, out-of-order completion) before removal.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Address the container escape risk reported in issue #337 (Docker socket
bind-mount when DOCKER_INSIDE=true).
Changes:
backend/pkg/docker/client.go — RunContainer hardening
• Add `no-new-privileges:true` to SecurityOpt on every sandbox container.
Prevents setuid/setgid and file-capability escalation inside the sandbox.
• Add PidsLimit=2048 as a cheap fork-bomb / resource exhaustion guard.
• Add startup warning (logrus.Warn) when DOCKER_INSIDE=true so operators
see the socket-escape risk at boot rather than only in documentation.
• Correct the comment to not imply the socket-escape vector is mitigated
by no-new-privileges alone (it is not; the real fix is DOCKER_INSIDE=false
or a least-privilege socket proxy).
backend/pkg/tools/tools.go — CapDrop ALL on primary terminal container
• Drop all Linux capabilities then add back only NET_RAW (always) and
NET_ADMIN (when DOCKER_NET_ADMIN=true). Removes ~14 unnecessary caps
from the default Docker set (e.g. SYS_CHROOT, AUDIT_WRITE, MKNOD) that
pentest tools should not need and that could be abused in an escape chain.
.env.example
• Flip DOCKER_INSIDE default to false (aligned with config.go envDefault).
The previous example was contradicting the code default.
• Expand the DOCKER_INSIDE comment: explains socket-escape risk, DinD
guidance, and links to docker-socket-proxy as a least-privilege alternative.
Note: DOCKER_INSIDE=false is the safest configuration; operators who need
DinD should consider fronting /var/run/docker.sock with a proxy rather than
mounting it directly.
- Added methods for non-recursive directory listing and file stat operations in the Docker client.
- Implemented a new API endpoint to retrieve files from a running container's directory.
- Updated documentation to reflect new file operations and API changes.
- Introduced data structures for container file metadata and integrated them into the flow file service.
- Enhanced flow file management capabilities with improved synchronization between local and container file systems.
Dmitry Ng