If we have the same source we should dedupe these. Previously
virt.framework would yell at us for a duplicate tag because we derive
the tag from hashing the mount source.
I was not a huge fan of how rlimits were specified using the OCI type as
it just takes in strings for the type which isn't super ergonomic. This
adds a new type for rlimits that we can convert to oci type like we
already do for capabilities.
Upon further thought, we should probably support this at the pod level
and allow per container overrides. I've changed my mind on how tedious
it is to specify it on a per-container basis.
Fixes#79
This adds support for single file (virtiofs based) mounts to the two
main container types. This is transparent, and doesn't need to be setup
by a user, so there is no visible API change. Virtiofs does not support
single file mounts today, and it would be less than ideal from a
security standpoint to just expose the parent directory of the file to
the vm and then bind mount in the file, so we chose the following:
1. Create a temp directory the container types will manage.
2. Hardlink in the file we want.
3. Share that tempdir into the vm.
4. Finally bind mount in the file into the container.
The main goal I wanted for this is to leave the logic out of the
`VirtualMachineInstance`. I really want this to just be a little dance
the container types do, and leave the vm out of it. Because of that,
most of the logic is shoved in a new `FileMountContext` type that does
the dirty tricks of rewriting what the user asked for to be a bind mount
from a temporary holding spot we'll use in the guest for these temp
directories.
One of the downsides to this is today the tempdirs are solely on the
main volume, so cross volume will need to copy the file to the tempdir,
so writes won't get written back.
Co-authored-by: Dean Coulstock <d_coulstock@apple.com>
Co-authored-by: Jaewon Hur <hurjaewon@snu.ac.kr>
Fixes#477
Because we start piping stdin before process launch we can fill up the
guest pipe buffer before the process even starts. We'd need some
backpressure mechanism to handle this and slow consumers (register the
other end with epoll and buffer some data etc.), but we should probably
just start piping after the process is up and running. This change does
exactly that, as well as stops holding the process mutex while draining
stdin in the guest for `CloseStdin()`.
This fixes issues where we try and write > pipe_buf bytes through stdin.
Today this hangs.
It's possible a user doesn't want the full stats list, and only wants
cpu/mem etc. This plumbs through the ability to filter to only what is
requested. This, while we're already here, adds in memory.event output
to the stats list. For that specifically, I think eventually we may want
a streaming variant of this so you can get alerted of changes in the
file immediately instead of polling/one off reads, but this is useful
for now.
Add a streaming rpc to copy single files in and out. This can easily be
extended to support copying directories, but we need the guest agent to
be able to tar/untar, and we'll need to figure out what that model is.
In a prior change I'd added a way for vminitd to double as a simple
pause container. This change wires this up by adding a new bool to the
pod config to ask for pid ns sharing.
Fixes#277
When vmexec fails, it logs to stderr and exits with code 1. Previously,
the error details were lost. This change captures stderr and converts it
into a proper ContainerizationError.
Signed-off-by: Rahul Thennarasu <rahulthennarasu07@gmail.com>
Taking in a filehandle gives the user quite a bit more freedom on how to
handle boot log output. They can set up a kqueue watch on it and
redirect output somewhere else etc etc. The implementation for this has
us take in a new BootLog type that has two options:
1. .file, which is analogous to what we had prior. Just provide a URL
and a true by default append field.
2. .fileHandle which is the new addition. Can pass any fd that is
writable, and the VMM should write serial console output to it.
This change is aimed at making forgetting to call .delete() on a
LinuxProcess less destructive than it can be. Because Virt.framework
invalidates any vsock fds it vended if the vm is stopped, trying to
perform some operations on the grpc client through any of the process
methods could trigger an ebadf, which NIO asserts on. This keeps a
reference to the execs and deletes all of them for you once the
container dies. I still think leaving .delete a public method is useful
as otherwise the stdio fds are left open, but cleanup should occur all
in one place now if you don't care about this.
This additionally:
1. Fixes two of our tests that forgot to delete() an exec.
2. Adds two new tests to verify that process.delete() is now idempotent,
and we don't need to call delete().
Add a test to ingest 200MiB of data across stdout/stderr. Useful to test
how fast we can ingest stdio. The timing will always be a tad off as it
times between start and wait returning, and there's quite a lot in the
way.. but it's a good enough metric.
Addresses apple/container#141, where containers don't receive filesystem
events on mounted volumes, preventing incremental rebuilds and other
file-watching features. This PR implements the guest-side components for
FSNotify. Host-side implementation in the container repo will complete
the pipeline.
Summary:
- Add gRPC protocol definitions for filesystem event notifications
- Implement guest-side event handler that generates Linux inotify events
- Add CLI testing tool (`cctl fsnotify`) and integration test
infrastructure
Closes#319
Introduces a new type capable of running > 1 container in the guest. The
API mostly follows LinuxContainer, and each individual container can be
addressed via any of the methods that require you to pass the
containerID as the first param. Today there's no support for namespace
sharing, but that shouldn't be terrible to support.
Closes#227
Previously, the bootlog was supplied once in the constructor to
VZVirtualMachineManager which meant that if you used this same manager
for multiple ctrs that all logs would end up going to the same file,
which becomes quite cumbersome to follow..
This change moves bootlog to be a container configuration param and also
moves it to be a VMConfiguration param, so it can be threaded through
from LinuxContainer -> vmm.create() and be truly container unique now.
The largest driver for this was the integration tests which today every
single test spits out logs to a singular file, making guest
investigations tricky to actually look into.
Result after:
```
➜ containerization git:(bootlog-per-ctr) ✗ ls -alh bin/bootlogs
total 1520
drwxr-xr-x@ 24 dcantah staff 768B Oct 22 17:34 .
drwxr-xr-x@ 8 dcantah staff 256B Oct 22 17:34 ..
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-cat-mount.log
-rw-------@ 1 dcantah staff 22K Oct 22 17:34 test-cgroup-limits.log
-rw-------@ 1 dcantah staff 249K Oct 22 17:34 test-concurrent-processes-output-stress.log
-rw-------@ 1 dcantah staff 167K Oct 22 17:34 test-concurrent-processes.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-container-devconsole.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-container-hostname.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-container-hosts-file.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-container-manager.log
-rw-------@ 1 dcantah staff 22K Oct 22 17:34 test-container-reuse.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-container-statistics.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-container-stdin.log
-rw-------@ 1 dcantah staff 0B Oct 22 17:34 test-nested-virt.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-pause-resume-io.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-pause-resume-wait.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-pause-resume.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-process-custom-home-envvar.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-process-echo-hi.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-process-false.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-process-home-envvar.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-process-true.log
-rw-------@ 1 dcantah staff 11K Oct 22 17:34 test-process-tty-envvar.log
-rw-------@ 1 dcantah staff 38K Oct 22 17:34 test-process-user.log
```
I only see a very small delta halving this (.6 seconds), and the default
will likely stress the CI machines a lot more. This change also swaps
the atomic increments to .relaxed, as there's no ordering required, just
need no torn writes.
Fixes#320
Today we don't actually setup any cgroup limits, as because there's a
1-1 mapping from container<->vm we can just use the VMs resources as the
limit (can't use more than 1GB if that's all the guest sees :) ).
However, if we ever supported > 1 container in the guest it'd be
necessary to actually setup the cg limits. This change just sets a
memory limit and cpu toggles to match whatever was specific for the
container.
Closes#138
This rounds up cgroup and network stats (via netlink) and exposes them
via an agent rpc and on LinuxContainer. While this is not an entirely
accurate view into the full resources being used by the container given
the virtualized nature (doesn't account for vcpu, device etc overhead)
it should give an accurate overview of the workload resource usage in
the guest.
This change besides the stated goal also removes the interfaceStatistics
rpc and just moves these network stats into the new containerStatistics
one. Related to that, it also renames the InterfaceStatistics struct to
NetworkStatistics and moves this into ContainerStatistics as a nested
struct.
Have wait return a struct containing the exit code and the time at which
the process actually exited. This is useful so higher level tools (like
container) don't need to craft a date themselves, and it will be more
accurate as it's taken in the guest directly after the process exits.
The `allocations` property in the nested `Allocator` structure wasn't
updated. This PR resolves the problem by using mutating methods and
keeping the original types as structs.
After the container is stopped it's a bit odd how we don't allow the
object to be restarted. If someone wanted to continuously rerun the same
container they'd need to make a new object every time even though all
guest state is blown away on stop() so it's a clean slate.
This change makes it so that you can create+start again after stop
(although create stands out like a sore thumb now as stop -> create is
just strange).
A ContainerManager is a type that handles more of the required resource
needed to create and run a container.
---------
Signed-off-by: crosbymichael <michael_crosby@apple.com>
This is a fairly large reworking, but it gets rid of something that has
plagued this since release which is the properties needing to be locked
to be Sendable compliant. This was somewhat of a copout because we
mostly know there's not a great deal of ways to have misused the setup
today, but alas we'd need to either mark the type as `@unchecked` or
just find a different route for setting the configuration. This change:
Exposes the underlying Configuration type that today only housed things
that aren't on the OCI spec. I'd love to just expose the OCI spec, but
we don't (and possibly never will) support everything on the spec, so
exposing it to be freely modified would be a bit odd. Now everything
related to the container is configured on this type, and the same goes
for execs.
Closes#206
Much like we have support for supplying DNS configurations and writing
out /etc/resolv.conf, this adds a way to write out /etc/hosts for a
given container.
This allows the user the ability to raise an EOF for the containers
stdin. Today there's no way to close stdin so something as simple as
"cat" and relying on EOF to move the process forward doesn't work
Danny Canter