There are some workloads that benefit from having a "true" init process
that reaps zombies. This change implements a minimal init as part of
vminitd and exposes an API to be able to use this init process in our
containers. signals will be forwarded, any child procs will be reaped
and the exit code of the actual workload will be propagated. The
LinuxContainer/LinuxPod API is a very simple bool to ask to use this as
our init process, and the init simply spawns whatever command you want
as a child underneath this.
```
➜ ./bin/cctl run --kernel bin/kernel.arm64 --init
/ # ps aux
PID USER TIME COMMAND
1 root 0:00 /.cz-init -- /bin/sh
2 root 0:00 /bin/sh
3 root 0:00 ps aux
```
This adds support to LinuxContainer to be able to provide an optional
writable layer. This is useful as today the rootfs size is determined by
the size of the image block itself which is set in stone at unpack time
of the image. This leaves a lot to be desired because a user might want
a larger or smaller rootfs size, but it's not configurable on a per
container basis because of this. To support this, we can pass an
additional writable block device that we can overlayfs with the image
contents in the guest. All writes will go to this writable layer, and
the rootfs size in the container is now whatever size this writable
layer is.
The block devices (via our ext4 package, but you could use whatever) are
cheap enough to generate on the fly that you can choose whatever size
you want easily.
These two stats are import to calculate the following
- wss bytes (memory usage - inactive files)
- rss bytes (anonymous memory)
- available bytes (limit - wss bytes)
Just exposing these two in `SandboxContext.proto` as we already collect
them in `readMemoryStats()` in `Cgroup2Manager`
- Downgrades a warning to the debug level in `BidirectionalRelay`
because the shutdown syscall isn't supported for certain socket types
- Rethrows an error if we can't start a socket relay
I've extracted the network cleanup step from `delete` into a new public
`releaseNetwork` method. This mirrors the existing `release` on
`VmnetNetwork` and enables stop-and-restart workflows where the network
allocation needs to be freed without removing the container's rootfs.
## Changes
- Add `ContainerManager.release(_:)` — frees network resources for a
container after it has stopped
- Refactor `delete(_:)` to call `releaseNetwork(_:)` before removing
files, preserving existing behavior
## Motivation
Currently the only way to free a container's network allocation is
`delete`, which also removes its files. This makes it impossible to stop
a container (freeing the network so the ID can be re-allocated) and
later restart it with its filesystem intact. The new `releaseNetwork `
method fills this gap.
- Adds a public method to wait for the relay to complete when both
directions close
- Improves error handling
- Moves `UnixSocketRelayManager` into a separate file
- Renames `SocketRelay` to `UnixSocketRelay`
- Adds logs at the info level after converting other logs to the debug
level previously
- Guarded against ipv4Gateway if nil and throw
`ContainerizationError(.invalidState, ...)` instead of force unwraping
- Added a unit test
`ContainerManagerTests.testCreateThrowsWhenGatewayMissing`
If we have the same source we should dedupe these. Previously
virt.framework would yell at us for a duplicate tag because we derive
the tag from hashing the mount source.
Changes:
- Refactors a bidirectional relay into a new class
- Makes the unique identifier for a socket configuration public
Bug fixes:
- Fixes a bug where partial writes didn't advance the pointer
- Fixes a buffer memory leak
- Fixes error messages
- Fixes log messages
- Fixes log levels
"clean up" is a verb, and "cleanup" is a noun. There are several
examples of the incorrect usage of "cleanup" as a verb in this
repository, including one in the public API (`cleanupOrphanedBlobs`).
For this public method, I've added a deprecation notice to the name with
the incorrect usage.
- Closesapple/container#988.
- macOS libarchive is not built with zstd support, so the workaround in
ArchiveReader is to attempt to decompress every archive as zstd. If
decompression fails, we pass the original archive to libarchive. If it
succeeds, we pass the uncompressed archive.
- Adds blob media type recognition for zstd to EXT4Unpacker. Tested zstd
blob unpack using `image pull tonistiigi/hello-world:zstd-docker`.
Co-authored-by: Aditya Ramani <a_ramani@apple.com>
I was not a huge fan of how rlimits were specified using the OCI type as
it just takes in strings for the type which isn't super ergonomic. This
adds a new type for rlimits that we can convert to oci type like we
already do for capabilities.
Upon further thought, we should probably support this at the pod level
and allow per container overrides. I've changed my mind on how tedious
it is to specify it on a per-container basis.
Fixes#79
This adds support for single file (virtiofs based) mounts to the two
main container types. This is transparent, and doesn't need to be setup
by a user, so there is no visible API change. Virtiofs does not support
single file mounts today, and it would be less than ideal from a
security standpoint to just expose the parent directory of the file to
the vm and then bind mount in the file, so we chose the following:
1. Create a temp directory the container types will manage.
2. Hardlink in the file we want.
3. Share that tempdir into the vm.
4. Finally bind mount in the file into the container.
The main goal I wanted for this is to leave the logic out of the
`VirtualMachineInstance`. I really want this to just be a little dance
the container types do, and leave the vm out of it. Because of that,
most of the logic is shoved in a new `FileMountContext` type that does
the dirty tricks of rewriting what the user asked for to be a bind mount
from a temporary holding spot we'll use in the guest for these temp
directories.
One of the downsides to this is today the tempdirs are solely on the
main volume, so cross volume will need to copy the file to the tempdir,
so writes won't get written back.
Co-authored-by: Dean Coulstock <d_coulstock@apple.com>
Co-authored-by: Jaewon Hur <hurjaewon@snu.ac.kr>
When we set a gateway outside a subnet in Linux, this fails with:
```
ip-route-add-default: netlink response indicates error, rc = -101
```
To resolve the error, we should first add a route to the gateway outside
the subnet, which this PR does.
Fixes#477
Because we start piping stdin before process launch we can fill up the
guest pipe buffer before the process even starts. We'd need some
backpressure mechanism to handle this and slow consumers (register the
other end with epoll and buffer some data etc.), but we should probably
just start piping after the process is up and running. This change does
exactly that, as well as stops holding the process mutex while draining
stdin in the guest for `CloseStdin()`.
This fixes issues where we try and write > pipe_buf bytes through stdin.
Today this hangs.
The guest agent isn't free, and today we were sizing the VMs memory
allotment and the containers cgroup to the same value. This change gives
some overhead memory to the VM (50MB for now) for the guest agent.
It's possible a user doesn't want the full stats list, and only wants
cpu/mem etc. This plumbs through the ability to filter to only what is
requested. This, while we're already here, adds in memory.event output
to the stats list. For that specifically, I think eventually we may want
a streaming variant of this so you can get alerted of changes in the
file immediately instead of polling/one off reads, but this is useful
for now.
- Simplifies processing and validation of MAC addresses for
apple/container#1005, where we need to create MAC addresses and
corresponding link local IPv6 addresses.
- Adds Codable for IPAddress and CIDR for apple/container#1006, so we
can parse and relay IP addresses regardless of address family for port
forwards.
Add a streaming rpc to copy single files in and out. This can easily be
extended to support copying directories, but we need the guest agent to
be able to tar/untar, and we'll need to figure out what that model is.
Closely related to https://github.com/apple/container/issues/305 I would
like to override the used SSL TrustRoots via standard env variables.
This here would add this configuration and would give an entrypoint for
an implementation of 305 to provide CLI flags or similar.
This has no tests yet, as this would require setting up something like a
MITM proxy when testing against a registry.
As I am unfamiliar with the codebase, I would be willing to do this, but
would require a first nudge on where to best implement this.
To actually use this, we would need to add the allowed env variables to
the `container system start` command env filter.
In a prior change I'd added a way for vminitd to double as a simple
pause container. This change wires this up by adding a new bool to the
pod config to ask for pid ns sharing.
Add a small bit of logic to monitor if vminitd goes over a (somewhat
arbitrary) memory threshold. On average, when running one container it
seems to hover around 25MiB, so this is mostly to catch cases where it's
exceeding a limit we don't deem normal.
Adds `calculateOrphanedBlobsSize()` to calculate the size of orphaned
blobs, will need this to include them under size and reclaimable space
for images in the `container system df` command so it matches up with
what `container image prune` frees up on disk.
- Fixes#417.
Rename `_prune()` to `cleanupOrphanedBlobs()` to clarify what it
actually does, and remove `prune()` method as we'll do all that logic in
container directly.
For the constructor we're using where we pass an already connected
socket, I'm not sure this even does anything, and if it does that's a
bug I'd rather just avoid altogether.
Also gets rid of the unused constructor we have to pass in a UDS.
* on glibc, swift expects that the msg_controllen is type Int
* on glibc, socket shutdown options should be an Int32
Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
Fixes#369
Per the OCI Image Spec, manifest descriptor annotations are optional.
Previously, archives without annotations would fail to import with
"Failed to import image".
**Changes:**
- Modified `getImageReferencefromDescriptor` to return digest-based
references (`untagged@sha256:...`) when annotations are missing
- Removed guard that skipped manifests without annotations
- Added test case with `scratch_no_annotations.tar`
**Testing:**
All 167 tests pass, including new test for images without annotations.
Taking in a filehandle gives the user quite a bit more freedom on how to
handle boot log output. They can set up a kqueue watch on it and
redirect output somewhere else etc etc. The implementation for this has
us take in a new BootLog type that has two options:
1. .file, which is analogous to what we had prior. Just provide a URL
and a true by default append field.
2. .fileHandle which is the new addition. Can pass any fd that is
writable, and the VMM should write serial console output to it.
This change is aimed at making forgetting to call .delete() on a
LinuxProcess less destructive than it can be. Because Virt.framework
invalidates any vsock fds it vended if the vm is stopped, trying to
perform some operations on the grpc client through any of the process
methods could trigger an ebadf, which NIO asserts on. This keeps a
reference to the execs and deletes all of them for you once the
container dies. I still think leaving .delete a public method is useful
as otherwise the stdio fds are left open, but cleanup should occur all
in one place now if you don't care about this.
This additionally:
1. Fixes two of our tests that forgot to delete() an exec.
2. Adds two new tests to verify that process.delete() is now idempotent,
and we don't need to call delete().
Danny Canter