This expands `cctl run` to utilize entrypoint and cmd when present in
the OCI image, instead of always defaulting to `/bin/sh`. The user can
still provide explicit commands if necessary to override these values.
- Closes#787.
- The `blockSizeInBytes` parameter on `EXT4Unpacker`
was misleading — it does not represent an ext4 filesystem
block size.
- This change renames `blockSizeInBytes` to
`capacityInBytes`, updates the doc comment to accurately
describe the parameter and updates the name in the call
sites.
Changes the default Linux capability set for container processes from
`.allCapabilities` to `.defaultOCICapabilities`, making the library
secure-by-default. Callers that genuinely need elevated capabilities
must now opt in explicitly.
Signed-off-by: michael_crosby <michael_crosby@apple.com>
apple/containerization currently runs containers in per-container VMs on
macOS hosts via Virtualization.framework. This adds a second VMM backend
so the same Swift orchestration layer (LinuxContainer / LinuxPod /
Vminitd gRPC contract) runs on Linux hosts via cloud-hypervisor + KVM.
**CloudHypervisor Swift package** (`Sources/CloudHypervisor/`) — a thin
client for cloud-hypervisor's REST-over-UDS API, layered on
AsyncHTTPClient. Endpoints cover VMM / VM lifecycle / hotplug (disk, fs,
net, vsock, remove-device). Cross-platform (compiles on macOS for unit
tests; consumed at runtime only by the Linux side of Containerization).
**CH backend in Containerization** — one cloud-hypervisor subprocess per
VM, gated behind `#if os(Linux)`. CHVirtualMachineManager /
CHVirtualMachineInstance mirror the VZ shape behind the existing
VirtualMachineManager / VirtualMachineInstance protocol. CHProcess and
VirtiofsdProcess manage the binaries; CHHotplugProvider handles
virtio-blk and virtio-fs runtime hotplug (with one virtiofsd per unique
source-hash tag, refcounted across containers).
**Linux host networking** — BridgeManager brings up a Linux bridge with
an IPv4 subnet and (opt-in via `--enable-nat`) iptables MASQUERADE +
scoped FORWARD rules. LinuxBridgedNetwork enslaves a fresh TAP per
container to the bridge. State is recorded under `/run/containerization`
so `cctl bridge delete` reverses exactly what create did. Bridge
teardown verifies the link kind via sysfs to refuse deleting non-bridge
interfaces.
**cctl run / bridge** — end-to-end Linux container run path (image pull,
ext4 rootfs assembly, VM boot, container exec) plus `cctl bridge
create|delete` for the host network plumbing.
**Build & dist** — `make linux-build` / `make linux-integration` build
and exercise the host side inside an apple/container `--virtualization`
dev container. `make dist-x86_64` produces a deployment tarball (cctl +
cloud-hypervisor + virtiofsd + initfs + kernel) cross-compiled from the
aarch64 dev container; pipeline documented in `docs/x86_64-build.md`.
Static-musl C deps and the Zig cross compiler are pinned by SHA256.
The host orchestrator runs as root. Per-VM runtime state lives under
`/run/containerization/ch/<UUID>` with mode 0700; UDS sockets inside are
bound with mode 0600. Vminitd's gRPC channel inherits that trust
boundary — socket-file perms are the auth.
Sandbox flags are upstream-secure by default. Two per-component opt-outs
exist for the apple/container dev-container case (where the host seccomp
profile SIGSYS-kills CH and virtiofsd):
- `CONTAINERIZATION_NO_CH_SECCOMP=1` — `cloud-hypervisor --seccomp
false`.
- `CONTAINERIZATION_NO_VIRTIOFSD_SANDBOX=1` — `virtiofsd --sandbox
none`. Each emits a one-shot `logger.warning` at process start. Legacy
alias `CONTAINERIZATION_RELAXED_SANDBOX=1` flips both. cctl spawns both
binaries with `setsid` and a minimal env allowlist (PATH / HOME /
RUST_LOG / RUST_BACKTRACE) so the parent's secrets don't leak to
children.
`make linux-integration` runs the cross-platform integration suite
against a real cloud-hypervisor VM inside the dev container. Linux runs
the cross-platform subset (`process true`/`false`/`echo hi`, virtiofs
round-trip, hotplug); the macOS suite is unchanged.
Signed-off-by: michael_crosby <michael_crosby@apple.com>
Extends the network plumbing to support per-interface IPv6 address
configuration.
The `Interface` protocol supports `ipv6Address` and `ipv6Gateway`.
The agent's networking RPCs carry per-family fields via new
`InterfaceAddress`, `LinkRoute`, and `DefaultRoute` types in
`ContainerizationExtras`.
`NetlinkSession` adds IPv6 methods for address and route operations.
---------
Co-authored-by: michael_crosby <michael_crosby@apple.com>
Co-authored-by: Michael Crosby <crosbymichael@gmail.com>
To be able to test vminitd/vmexec/linux specific packages on ci it'd be
a heck of a lot easier if `make` just worked. This should be the last
bit needed. The default goal currently compiles just fine after the
linux specific `make deps` is ran. Next in line would be adding decent
unit tests/actually getting ci setup for the linux bits.
I've added an optional progress handler for rootfs unpacking so that
consumers can show progress. The Ubuntu image takes about 8 seconds to
unpack on my machine, and I'm developing an app where it would be useful
to show progress in the UI.
Total size is determined in an optional first pass that scans archive
headers. Bytes written are then reported during unpacking. The optional
first pass adds 15 (Alpine) to 115 (Ubuntu) ms to unpacking duration on
my machine, depending on image size.
The network protocol and VMNetNetwork implementation currently are
housed on ContainerManager even though they are generally useful types
even outside of this easy to use helper type. This change moves them to
not be nested types anymore, as well as exposes a new param on
VMNetNetworks constructor so we can pass the type of network.
### Summary
`rootfs` create writes proc/self/exe as a symlink, but currently passes
the previous regular file buffer to `ArchiveWriter.writeEntry`. This
change switches that symlink entry to nil, which matches archive writer
expectations.
### Test
Archive tests pass.
There are some workloads that benefit from having a "true" init process
that reaps zombies. This change implements a minimal init as part of
vminitd and exposes an API to be able to use this init process in our
containers. signals will be forwarded, any child procs will be reaped
and the exit code of the actual workload will be propagated. The
LinuxContainer/LinuxPod API is a very simple bool to ask to use this as
our init process, and the init simply spawns whatever command you want
as a child underneath this.
```
➜ ./bin/cctl run --kernel bin/kernel.arm64 --init
/ # ps aux
PID USER TIME COMMAND
1 root 0:00 /.cz-init -- /bin/sh
2 root 0:00 /bin/sh
3 root 0:00 ps aux
```
- Add a `list` function that is needed for the `container registry list`
command
- Updates to `KeychainHelper` and `KeychainQuery` variable/parameter
names
- Updates to test cases
* Admit only valid OCI layout files when loading image.
- Adds `AdmissionMapper` protocol for validating archive
member paths and normalizing them to relative paths
under the extraction root directory.
- Renames `Reader.swift` to `ArchiveReader.swift` to
match type name.
- Rework `TempDir` to address `NSString.utf8String`
deprecation warning in Swift 6.2.3.
- Rework `ArchiveReader.extractContent()` to use an
`AdmissionMapper` to validate and remap archive
members before extracting.
- Adds `IdentityAdmissionWrapper` for naive extraction.
- Adds `NoSymlinkAdmissionWrapper` that only extracts
regular files and directories under the extraction root.
- Adds `OCIImageAdmissionWrapper` that only extracts
valid OCI image layout paths.
- Use `OCIImageAdmissionWrapper` for `cctl image load`
and print rejected paths.
* PR feedback.
* Adds public init() for TrustedAdmissionMapper.
* Replace AdmissionMapper with more secure extraction.
- Adds FileDescriptor.mkdirSecure() to prevent
root escapes on member pathnames, and to prevent
symlink traversal.
- Adds FileDescriptor.unlinkRecursive() to
facilitate overwrites when there are multiple
archive entries with the same member path.
- Adds FileDescriptor.validateSymlinkTargetInRoot()
to validate that extracted symlink targets do
not escape the root.
- Rewrite ArchiveReader.extractContents() to use
secure path functions.
* Remove unneeded symlink check, rename files.
* Simplify the lexical normalizer workaround.
* Reject member paths containing parent traversal components.
* Remove unused lexical normalization workaround.
* Fix leaking fds, extract absolute members as relative.
Closely related to https://github.com/apple/container/issues/305 I would
like to override the used SSL TrustRoots via standard env variables.
This here would add this configuration and would give an entrypoint for
an implementation of 305 to provide CLI flags or similar.
This has no tests yet, as this would require setting up something like a
MITM proxy when testing against a registry.
As I am unfamiliar with the codebase, I would be willing to do this, but
would require a first nudge on where to best implement this.
To actually use this, we would need to add the allowed env variables to
the `container system start` command env filter.
Addresses apple/container#141, where containers don't receive filesystem
events on mounted volumes, preventing incremental rebuilds and other
file-watching features. This PR implements the guest-side components for
FSNotify. Host-side implementation in the container repo will complete
the pipeline.
Summary:
- Add gRPC protocol definitions for filesystem event notifications
- Implement guest-side event handler that generates Linux inotify events
- Add CLI testing tool (`cctl fsnotify`) and integration test
infrastructure
Optimize unpack a little by trying to reduce allocations in the hot
path. Today for every file we read the entire file into memory and then
pass the data blob to the ext4 writer to eventually be written to the
sparse file. Before being written to the sparse file the data is copied
*again* to a temp buffer before finally hitting write(2) in FileHandle.
This change moves things around such that we can pass an optional buffer
to the ext4 create() (so we can reuse a buffer for file writes), as well
as stops reading entire files into memory by passing the archive entry
itself (wrapped in a ReaderStream object albeit) down to the writer.
Testing with unpacking every platform for
`docker.io/jenkins/jenkins:lts` on an M1 Max:
Old Avg (5 runs): 7.43s
New Avg (5 runs): 5.31s
Today this protocols create method is just odd. Our only implementation
of it immediately casts to LinuxContainer, failing if it cannot do so. I
think what might make more sense is to pass in a configuration itself
with core parameters that we expect every vmm to be able to support, and
then in a specific implementation they can continue to cast this type to
a specific one to possibly extract some extra configuration values
(rosetta for VZ for example).
This rework will also make it simpler to support a Pod type, as the vm
setup is identical and simple.
- Motivation is to be able to add the `swift-backtrace-static` binary as
needed.
- Use singlular `--add-file` and `--label` options since both accept
multiple invocations with single values each.
Example usage:
```bash
./bin/cctl rootfs create \
--vminitd vminitd/bin/vminitd \
--vmexec vminitd/bin/vmexec \
--add-file /Users/john/.swiftpm/swift-sdks/swift-6.2-RELEASE_static-linux-0.0.1.artifactbundle/swift-6.2-RELEASE_static-linux-0.0.1/swift-linux-musl/musl-1.2.5.sdk/aarch64/usr/libexec/swift/linux-static/swift-backtrace-static:sbin/swift-backtrace \
--label org.opencontainers.image.source=https://github.com/apple/containerization \
--image vminit:latest \
bin/init.rootfs.tar.gz
```
This refactors the rootfs command to produce an image and have the
ability to create an ext4 formatted block of the init filesystem
directly from the command.
closes#220
Signed-off-by: crosbymichael <michael_crosby@apple.com>
The `allocations` property in the nested `Allocator` structure wasn't
updated. This PR resolves the problem by using mutating methods and
keeping the original types as structs.
This reverts commit 5f7dcd2a16. Rosetta is
setup at /run because the guest rootfs is ro, and unfortunately this
happens prior to LinuxContainer getting to mount any of this. I'm not
super happy about this, but we can figure something out later.
Additionally this adds a rosetta flag to cctl so we can quickly
spotcheck things like this again.
A ContainerManager is a type that handles more of the required resource
needed to create and run a container.
---------
Signed-off-by: crosbymichael <michael_crosby@apple.com>
This is a fairly large reworking, but it gets rid of something that has
plagued this since release which is the properties needing to be locked
to be Sendable compliant. This was somewhat of a copout because we
mostly know there's not a great deal of ways to have misused the setup
today, but alas we'd need to either mark the type as `@unchecked` or
just find a different route for setting the configuration. This change:
Exposes the underlying Configuration type that today only housed things
that aren't on the OCI spec. I'd love to just expose the OCI spec, but
we don't (and possibly never will) support everything on the spec, so
exposing it to be freely modified would be a bit odd. Now everything
related to the container is configured on this type, and the same goes
for execs.
Closes#206
Much like we have support for supplying DNS configurations and writing
out /etc/resolv.conf, this adds a way to write out /etc/hosts for a
given container.
Creates a new Unpacker protocol that defines a single method
```
func unpack(_ image: Image, for platform: Platform, at path: URL, progress: ProgressHandler?) async throws -> Mount
```
This change also removes the `unpack(...)` method from the Image type.
Before
```
let mount = try await image.unpack(for: platform, at: path)
```
After
```
let unpacker = EXT4Unpacker(blockSizeInBytes: 2.gib())
let mount = try await unpacker.unpack(image, for: platform, at: path)
```
---------
Signed-off-by: Aditya Ramani <a_ramani@apple.com>
Make sure we always set TERM for containers that ask for a tty. Right
now this handling was spread around in a bunch of spots, but I'd wager
setting it for the client on the host via LinuxContainer/Process is more
sane and already what we do for a lot of the other fields.
Fixes#103 (and should fix
https://github.com/apple/container/issues/108)
Today we only parse if the username field of the runtime spec is set,
but this misses if someone just supplied the integer uid and gid fields
only.
This change adds documentation to quite a few existing public types that
didn't have a blurb before.
Additionally, this fixes a couple things that I think either didn't make
sense when going to document them:
- Rename ConnectionStream to VsockConnectionStream. This type only
functions for vsock connections.
- Deletes NsLock+Closure. This was not used anywhere.
- Rename ContainerizationOCI/Config.swift to ImageConfig.swift.
Signed-off-by: Danny Canter <danny_canter@apple.com>
Noah Thornton