Add `Stat` RPC to vminitd.
`Stat` RPC is used to stat file system objects in the VM. It accepts the
path to the object (resolved against VM root) and returns the stat
result.
- Closes#671.
- Adds optional journal parameter to `EXT4.Formatter.init()`, with nil
default specifying the current no-journal filesystem configuration.
Otherwise the parameter value contains the journal size and mode.
- `minDiskSize` parameter for formatter init specifies the minimum
usable capacity of the resulting filesystem. The resulting disk image
grows past this value to accommodate the filesystem on-disk structures,
including the journal if specified.
Related to https://github.com/apple/container/issues/1251
Change to sharing in the parent directory, and then bind mounting in the
file into the container. This has the unfortunate reality of being less
secure, but the current approach is burdended by a couple things,
namely:
1. You can't share in files that are on a different volume
2. There is a Virtualization bug that causes spurious errors when trying
to open the file.
I've added a doc to go over the approach we've taken, and some
workarounds if the approach is not satisfactory.
To be able to test vminitd/vmexec/linux specific packages on ci it'd be
a heck of a lot easier if `make` just worked. This should be the last
bit needed. The default goal currently compiles just fine after the
linux specific `make deps` is ran. Next in line would be adding decent
unit tests/actually getting ci setup for the linux bits.
The network protocol and VMNetNetwork implementation currently are
housed on ContainerManager even though they are generally useful types
even outside of this easy to use helper type. This change moves them to
not be nested types anymore, as well as exposes a new param on
VMNetNetworks constructor so we can pass the type of network.
Now that the static linux SDK we use has libarchive linked against it,
we can finally live our dream (/s) of supporting copying directories
in/out a little easier. This was.. kind of annoying. I think the very
simple route of writing archive to guest/host -> streaming over grpc is
simplest, but on the guest end we have a couple problems:
1. The VMs rootfs is read only today always, which is a good thing to me
and I don't want to change if we don't have to.
2. Writing the archive to the containers rootfs temporarily could work,
but it's a bit weird, and the user can make the containers rootfs
readonly which would screw that plan.
3. We could write it to /run or /tmp, but they're tmpfs and dealing with
the headache of the user possibly tarring an enormous dir is one I don't
want to care about.
So, that leaves us with the (truthfully better to me) approach of trying
to write the tar data directly to the host and skipping grpc which kinda
forces us to have a temp spot. Because of that, I made it such that we
pass a port number from host<->guest, and transfer the actual data
(either the single file or tarred dir contents) over the vsock port
instead. The stream is kinda clunky, but it just serves as a means to
exchange metadata and a "we're done" signifier.
Closes#256
If the target of the uds proxy goes through a symlink (/var/run) we had
the same bug we had with mounts previously where we'd follow this and if
the resolved path existed in the root of the VM we'd end up mounting
there. Fix this by plopping the sockets in a holding spot and then bind
mounting them in (which has the correct resolution logic).
This change fixes an issue in linux pod where the exec process
does not inherit the container process config which means any
environment variables, working directory, user, rlimits,
capabilities would be different for the exec process.
This change ensures that exec process would inherit
environment variables, working directory, user, rlimits,
and capabilities from the container process config but
resets arguments, terminal, and stdio as those should not
be inherited.
This change also adds an integration test to exercise this.
Just like we have for hosts and dns configurations, we can add a
hostname field to make it easier to setup if every container is going to
be using the same hostname.
This PR adds a `networking: Bool = true` parameter to
`ContainerManager.create()` so callers can opt out of network interface
creation on a per-container basis.
## Motivation
Currently, `ContainerManager.create()` unconditionally allocates a vmnet
network interface for every container (when the manager has a network
configured). Some use cases don't need network access and benefit from
having it disabled to reduce attack surface.
There's no way to achieve this today without either:
- Initializing the `ContainerManager` without a network (which disables
networking for *all* containers)
- Clearing `config.interfaces` in the configuration closure (which
wastes an IP allocation from the vmnet pool since `network.create(id)`
has already been called)
## Changes
- Add `networking: Bool = true` to all three `create()` overloads on
`ContainerManager`
- When `false`, `self.network?.create(id)` is skipped entirely. No
interface is allocated, and no DNS is configured.
- `releaseNetwork`/`delete` remain safe to call regardless, since
`Allocator.release` silently ignores unknown IDs.
- Add unit test `testNetworkingFalseSkipsInterfaceCreation` using the
existing `NilGatewayNetwork` fixture
- Add integration tests `testNetworkingDisabled` and
`testNetworkingEnabled` that create containers through a network-enabled
`ContainerManager` and verify the presence/absence of `eth0` via
`/sys/class/net/`
There are some workloads that benefit from having a "true" init process
that reaps zombies. This change implements a minimal init as part of
vminitd and exposes an API to be able to use this init process in our
containers. signals will be forwarded, any child procs will be reaped
and the exit code of the actual workload will be propagated. The
LinuxContainer/LinuxPod API is a very simple bool to ask to use this as
our init process, and the init simply spawns whatever command you want
as a child underneath this.
```
➜ ./bin/cctl run --kernel bin/kernel.arm64 --init
/ # ps aux
PID USER TIME COMMAND
1 root 0:00 /.cz-init -- /bin/sh
2 root 0:00 /bin/sh
3 root 0:00 ps aux
```
This adds support to LinuxContainer to be able to provide an optional
writable layer. This is useful as today the rootfs size is determined by
the size of the image block itself which is set in stone at unpack time
of the image. This leaves a lot to be desired because a user might want
a larger or smaller rootfs size, but it's not configurable on a per
container basis because of this. To support this, we can pass an
additional writable block device that we can overlayfs with the image
contents in the guest. All writes will go to this writable layer, and
the rootfs size in the container is now whatever size this writable
layer is.
The block devices (via our ext4 package, but you could use whatever) are
cheap enough to generate on the fly that you can choose whatever size
you want easily.
If we have the same source we should dedupe these. Previously
virt.framework would yell at us for a duplicate tag because we derive
the tag from hashing the mount source.
I was not a huge fan of how rlimits were specified using the OCI type as
it just takes in strings for the type which isn't super ergonomic. This
adds a new type for rlimits that we can convert to oci type like we
already do for capabilities.
Upon further thought, we should probably support this at the pod level
and allow per container overrides. I've changed my mind on how tedious
it is to specify it on a per-container basis.
Fixes#79
This adds support for single file (virtiofs based) mounts to the two
main container types. This is transparent, and doesn't need to be setup
by a user, so there is no visible API change. Virtiofs does not support
single file mounts today, and it would be less than ideal from a
security standpoint to just expose the parent directory of the file to
the vm and then bind mount in the file, so we chose the following:
1. Create a temp directory the container types will manage.
2. Hardlink in the file we want.
3. Share that tempdir into the vm.
4. Finally bind mount in the file into the container.
The main goal I wanted for this is to leave the logic out of the
`VirtualMachineInstance`. I really want this to just be a little dance
the container types do, and leave the vm out of it. Because of that,
most of the logic is shoved in a new `FileMountContext` type that does
the dirty tricks of rewriting what the user asked for to be a bind mount
from a temporary holding spot we'll use in the guest for these temp
directories.
One of the downsides to this is today the tempdirs are solely on the
main volume, so cross volume will need to copy the file to the tempdir,
so writes won't get written back.
Co-authored-by: Dean Coulstock <d_coulstock@apple.com>
Co-authored-by: Jaewon Hur <hurjaewon@snu.ac.kr>
Fixes#477
Because we start piping stdin before process launch we can fill up the
guest pipe buffer before the process even starts. We'd need some
backpressure mechanism to handle this and slow consumers (register the
other end with epoll and buffer some data etc.), but we should probably
just start piping after the process is up and running. This change does
exactly that, as well as stops holding the process mutex while draining
stdin in the guest for `CloseStdin()`.
This fixes issues where we try and write > pipe_buf bytes through stdin.
Today this hangs.
It's possible a user doesn't want the full stats list, and only wants
cpu/mem etc. This plumbs through the ability to filter to only what is
requested. This, while we're already here, adds in memory.event output
to the stats list. For that specifically, I think eventually we may want
a streaming variant of this so you can get alerted of changes in the
file immediately instead of polling/one off reads, but this is useful
for now.
Add a streaming rpc to copy single files in and out. This can easily be
extended to support copying directories, but we need the guest agent to
be able to tar/untar, and we'll need to figure out what that model is.
In a prior change I'd added a way for vminitd to double as a simple
pause container. This change wires this up by adding a new bool to the
pod config to ask for pid ns sharing.
Fixes#277
When vmexec fails, it logs to stderr and exits with code 1. Previously,
the error details were lost. This change captures stderr and converts it
into a proper ContainerizationError.
Signed-off-by: Rahul Thennarasu <rahulthennarasu07@gmail.com>
Taking in a filehandle gives the user quite a bit more freedom on how to
handle boot log output. They can set up a kqueue watch on it and
redirect output somewhere else etc etc. The implementation for this has
us take in a new BootLog type that has two options:
1. .file, which is analogous to what we had prior. Just provide a URL
and a true by default append field.
2. .fileHandle which is the new addition. Can pass any fd that is
writable, and the VMM should write serial console output to it.
This change is aimed at making forgetting to call .delete() on a
LinuxProcess less destructive than it can be. Because Virt.framework
invalidates any vsock fds it vended if the vm is stopped, trying to
perform some operations on the grpc client through any of the process
methods could trigger an ebadf, which NIO asserts on. This keeps a
reference to the execs and deletes all of them for you once the
container dies. I still think leaving .delete a public method is useful
as otherwise the stdio fds are left open, but cleanup should occur all
in one place now if you don't care about this.
This additionally:
1. Fixes two of our tests that forgot to delete() an exec.
2. Adds two new tests to verify that process.delete() is now idempotent,
and we don't need to call delete().
Add a test to ingest 200MiB of data across stdout/stderr. Useful to test
how fast we can ingest stdio. The timing will always be a tad off as it
times between start and wait returning, and there's quite a lot in the
way.. but it's a good enough metric.
Addresses apple/container#141, where containers don't receive filesystem
events on mounted volumes, preventing incremental rebuilds and other
file-watching features. This PR implements the guest-side components for
FSNotify. Host-side implementation in the container repo will complete
the pipeline.
Summary:
- Add gRPC protocol definitions for filesystem event notifications
- Implement guest-side event handler that generates Linux inotify events
- Add CLI testing tool (`cctl fsnotify`) and integration test
infrastructure
Closes#319
Introduces a new type capable of running > 1 container in the guest. The
API mostly follows LinuxContainer, and each individual container can be
addressed via any of the methods that require you to pass the
containerID as the first param. Today there's no support for namespace
sharing, but that shouldn't be terrible to support.
Closes#227
Previously, the bootlog was supplied once in the constructor to
VZVirtualMachineManager which meant that if you used this same manager
for multiple ctrs that all logs would end up going to the same file,
which becomes quite cumbersome to follow..
This change moves bootlog to be a container configuration param and also
moves it to be a VMConfiguration param, so it can be threaded through
from LinuxContainer -> vmm.create() and be truly container unique now.
The largest driver for this was the integration tests which today every
single test spits out logs to a singular file, making guest
investigations tricky to actually look into.
Result after:
```
➜ containerization git:(bootlog-per-ctr) ✗ ls -alh bin/bootlogs
total 1520
drwxr-xr-x@ 24 dcantah staff 768B Oct 22 17:34 .
drwxr-xr-x@ 8 dcantah staff 256B Oct 22 17:34 ..
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-cat-mount.log
-rw-------@ 1 dcantah staff 22K Oct 22 17:34 test-cgroup-limits.log
-rw-------@ 1 dcantah staff 249K Oct 22 17:34 test-concurrent-processes-output-stress.log
-rw-------@ 1 dcantah staff 167K Oct 22 17:34 test-concurrent-processes.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-container-devconsole.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-container-hostname.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-container-hosts-file.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-container-manager.log
-rw-------@ 1 dcantah staff 22K Oct 22 17:34 test-container-reuse.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-container-statistics.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-container-stdin.log
-rw-------@ 1 dcantah staff 0B Oct 22 17:34 test-nested-virt.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-pause-resume-io.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-pause-resume-wait.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-pause-resume.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-process-custom-home-envvar.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-process-echo-hi.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-process-false.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-process-home-envvar.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-process-true.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-process-tty-envvar.log
-rw-------@ 1 dcantah staff 38K Oct 22 17:34 test-process-user.log
```
jwhur