This is not wired up to anything right now, but it will be used for
implementing stats gathering on the host via an rpc in the future. I've
left off the rpc portion as this is already a decent amount to review.
Fixes#296
Today the kernel subdirectory is somewhat useless. The dockerfile we had
tried to use an image that isn't actually on ghcr, so everything would
kind of just crap out. This change swaps things so that we build a local
image that has all the bits needed to compile the kernel, and then uses
that image to build the resulting kernel and plop it in the current
working directory. This change does add a reliance on `container` being
installed however, but for folks that need to build a custom kernel this
doesn't seem like too much of an issue to me.
- Motivation is to be able to add the `swift-backtrace-static` binary as
needed.
- Use singlular `--add-file` and `--label` options since both accept
multiple invocations with single values each.
Example usage:
```bash
./bin/cctl rootfs create \
--vminitd vminitd/bin/vminitd \
--vmexec vminitd/bin/vmexec \
--add-file /Users/john/.swiftpm/swift-sdks/swift-6.2-RELEASE_static-linux-0.0.1.artifactbundle/swift-6.2-RELEASE_static-linux-0.0.1/swift-linux-musl/musl-1.2.5.sdk/aarch64/usr/libexec/swift/linux-static/swift-backtrace-static:sbin/swift-backtrace \
--label org.opencontainers.image.source=https://github.com/apple/containerization \
--image vminit:latest \
bin/init.rootfs.tar.gz
```
Have wait return a struct containing the exit code and the time at which
the process actually exited. This is useful so higher level tools (like
container) don't need to craft a date themselves, and it will be more
accurate as it's taken in the guest directly after the process exits.
swift-nio's public export of NIOFileSystem was removed in 2.86.1:
https://github.com/apple/swift-nio/pull/3370
NIOFileSystem was not yet supposed to be public, but _NIOFileSystem
depended on it as a public import. This made it possible for
`containerization` to see the `NIOFileSystem` package by accident.
Replacing the use of `NIOFileSystem` by `_NIOFileSystem`, as used
elsewhere, fixes the problem.
## Why does CI currently pass?
The change in `swift-nio` does not currently cause `containerization`'s
CI to fail because `Package.resolved` pins `swift-nio` to 2.83.0, before
the change was made. New versions of upstream dependencies will not be
tested until `Package.resolved` is explicitly updated.
When containerization is built as a dependency of a end-user project,
its `Package.resolved` file is ignored. Instead, the dependency
constraints from containerization's Package.swift file are combined with
those of the project and any other library dependencies, so SwiftPM or
Xcode can find a set of mutually compatible packages. This can lead to
new versions of containerization's upstream dependencies being used,
even though those versions have never been tested in CI.
The build failure can be demonstrated by creating a new package which
depends on `containerization` but does not constrain package versions:
```
% swift package init --type executable
Creating executable package: test
Creating Package.swift
Creating Sources
Creating Sources/test/test.swift
% cat > Package.swift <<EOF
heredoc> // swift-tools-version: 6.2
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription
let package = Package(
name: "test",
platforms: [
.macOS(.v26),
],
dependencies: [
.package(url: "https://github.com/apple/containerization", from: "0.7.2"),
],
targets: [
// Targets are the basic building blocks of a package, defining a module or a test suite.
// Targets can depend on other targets in this package and products from dependencies.
.executableTarget(
name: "test",
dependencies: [
.product(name: "Containerization", package: "containerization"),
]
),
]
)
EOF
% swift build
...
/private/tmp/test/.build/checkouts/containerization/Sources/ContainerizationOCI/Client/RegistryClient+Fetch.swift:25:8: error: no such module 'NIOFileSystem'
23 |
24 | #if os(macOS)
25 | import NIOFileSystem
| `- error: no such module 'NIOFileSystem'
26 | #endif
27 |
```
The project has `kernel-build/` and `kernel/image/` which contain the
same files:
```
$ diff -ru kernel/image/ kernel-build/
# no output
```
I did not find any reference to `kernel-build/` in any workflow or
README or Makefile, so I removed it in favor of `kernel/image/`.
With the changes in this PR, we'll print the Swift version before
building. This mirrors a similar change in container, introduced in
https://github.com/apple/container/pull/550.
The PR updates the toolchain and Static Linux SDK. This resolves the
compilation errors below:
```
Building vminitd and vmexec...
Building for debugging...
<unknown>:0: error: stack nearly exhausted; compilation time may suffer, and crashes due to stack overflow are likely
<module-includes>:1:10: note: in file included from <module-includes>:1:
1 | #include "SwiftMusl.h"
| `- note: in file included from <module-includes>:1:
2 |
/Users/Dmitry/Library/org.swift.swiftpm/swift-sdks/swift-6.2-DEVELOPMENT-SNAPSHOT-2025-06-25-a_static-linux-0.0.1.artifactbundle/swift-6.2-DEVELOPMENT-SNAPSHOT-2025-06-25-a_static-linux-0.0.1/swift-linux-musl/musl-1.2.5.sdk/aarch64/usr/lib/swift_static/linux-static/aarch64/SwiftMusl.h:135:10: error: could not build module 'sys_stropts'
133 | #include <sys/signalfd.h>
134 | #include <sys/statfs.h>
135 | #include <sys/stropts.h>
| `- error: could not build module 'sys_stropts'
136 | #include <sys/swap.h>
137 | #include <sys/syscall.h>
<unknown>:0: error: stack nearly exhausted; compilation time may suffer, and crashes due to stack overflow are likely
<unknown>:0: error: could not build C module 'SwiftMusl'
<unknown>:0: error: stack nearly exhausted; compilation time may suffer, and crashes due to stack overflow are likely
<module-includes>:1:10: note: in file included from <module-includes>:1:
1 | #include "SwiftMusl.h"
| `- note: in file included from <module-includes>:1:
2 |
/Users/Dmitry/Library/org.swift.swiftpm/swift-sdks/swift-6.2-DEVELOPMENT-SNAPSHOT-2025-06-25-a_static-linux-0.0.1.artifactbundle/swift-6.2-DEVELOPMENT-SNAPSHOT-2025-06-25-a_static-linux-0.0.1/swift-linux-musl/musl-1.2.5.sdk/aarch64/usr/lib/swift_static/linux-static/aarch64/SwiftMusl.h:135:10: error: could not build module 'sys_stropts'
133 | #include <sys/signalfd.h>
134 | #include <sys/statfs.h>
135 | #include <sys/stropts.h>
| `- error: could not build module 'sys_stropts'
136 | #include <sys/swap.h>
137 | #include <sys/syscall.h>
<unknown>:0: error: stack nearly exhausted; compilation time may suffer, and crashes due to stack overflow are likely
<unknown>:0: error: could not build C module 'SwiftMusl'
[719/1348] Compiling x_all.cc
make[1]: *** [all] Error 1
make: *** [vminitd] Error 2
```
To update:
- Run `make cross-prep`
- List the installed Static Linux SDKs using `swift sdk list`
- Remove the old Static Linux SDKs using `swift sdk remove <SDK>`
Optionally:
- List the installed toolchains using `swiftly list`
- Uninstalled the old toolchains using `swiftly uninstall <TOOLCHAIN>`
Today, because we don't wait for execvpe to finish to continue onwards,
it's possible that if you did an exec quick enough after starting an
init process for a container, that you could join the init processes
namespaces before pivot_root has taken place which is quite fun. Let's
wait for exec to finish (or an error to occur) to prevent this.
We had a log for the initial oci runtime spec, but we don't log anything
for execed processes. This makes it hard to look into issues that may be
due to just a mistake in the spec and nothing wrong with the actual
logic of the program.
Systemd loves to move the root cgroups processes to one it created named
/init.scope and then write the root cgs subtree_control file. Because of
this we can't just add exec processes to the cg we made for the
container anymore as we'll get EBUSY. We should follow where the init
processes cg is actually at (/proc/pid/cgroup) and add it there.
This refactors the rootfs command to produce an image and have the
ability to create an ext4 formatted block of the init filesystem
directly from the command.
closes#220
Signed-off-by: crosbymichael <michael_crosby@apple.com>
We currently weren't doing any cgroup setup whatsoever. For the most
part this doesn't matter too much, however certain images that may fool
around with cgroups don't like this :). Lets do the bare minimum these
expect which is to at least have the process running in a nested cg and
not apart of the root cg.
The `allocations` property in the nested `Allocator` structure wasn't
updated. This PR resolves the problem by using mutating methods and
keeping the original types as structs.
Clarify that container is only supported on macOS 26, and remove
language that implies that it's also designed for macOS 15.
This change also includes a couple of minor textual changes to improve
readability.
This change adds in a new rpc that we probably should swap to for the
underlying implementation of a couple rpcs we currently have. This new
rpc simply writes to an existing or new file some data. This is useful
for guest (and container) setup for things like /etc/hosts,
/etc/resolv.conf, writing to one-off cgroup files and so on.
Currently I've just implemented subtree_control toggling for the root cg
in standardSetup with this new change, but in the future I'd like to
swap the implementations of our /etc/hosts and /etc/resolv.conf logic
with this.
Improves https://github.com/apple/containerization/pull/245. A potential
deadlock risk was not proven. This PR moves the closing operation inside
the lock to prevent leaking an unclosed handle in case of an error.
Three things:
1. Timeout.run today didn't allow a throwing closure.
2. Add a new constructor for Timeout that takes in a Duration directly
so we can be more granular than seconds.
3. Add unit tests because we all love those :)
This PR replaces public computed properties with setters in the
`NATNetworkInterface` class with constants to avoid using locks in the
setters. A new instance of `NATNetworkInterface` should be created to
change these values.
While individual accesses were properly mutex-protected, the logical
operations were not atomic, leading to a potential inconsistency in a
multi-threaded environment. Multiple separate `state.withLock` calls
created race windows. With the proposed changes, single atomic
operations check and update the state together.
This reverts commit 5f7dcd2a16. Rosetta is
setup at /run because the guest rootfs is ro, and unfortunately this
happens prior to LinuxContainer getting to mount any of this. I'm not
super happy about this, but we can figure something out later.
Additionally this adds a rosetta flag to cctl so we can quickly
spotcheck things like this again.
After the container is stopped it's a bit odd how we don't allow the
object to be restarted. If someone wanted to continuously rerun the same
container they'd need to make a new object every time even though all
guest state is blown away on stop() so it's a clean slate.
This change makes it so that you can create+start again after stop
(although create stands out like a sore thumb now as stop -> create is
just strange).
Eric Ernst