- Closes#712.
- Replace synchronous `write()` calls in
`BidirectionalRelay` with non-blocking I/O
and `DispatchSourceWrite` backpressure
handling. Under concurrent vsock proxy load,
a single blocked write on the shared serial
dispatch queue would freeze all relay
connections permanently, including unrelated
new connections.
- Set relay file descriptors to `O_NONBLOCK`
and handle `EAGAIN` by suspending reads and
installing a write source to drain pending data.
- Give each `BidirectionalRelay` its own serial queue
instead of sharing one from `UnixSocketRelayManager`,
eliminating cross-connection blocking.
- Resume suspended read sources before cancelling
in `stop()` — GCD does not deliver cancel handlers
on suspended dispatch sources, which caused file
descriptor and memory leaks on teardown under
backpressure.
- Guard-unwrap `buf.baseAddress` in
`drainPendingWrite`.
- Closes#749.
- ExportOperation hardcoded the pushed index descriptor's
mediaType to the OCI image index type. RegistryClient.push
uses that descriptor's mediaType as the HTTP Content-Type
header. When the source index was in
Docker manifest.list.v2+json format (the common case for
images pulled from Docker Hub and other public registries),
the body's embedded mediaType field disagreed with the
header, and OCI registries rejected the index PUT with
HTTP 400 MANIFEST_INVALID.
- Use the source index's mediaType for the pushed descriptor
so the header always matches the body. Per-architecture
child manifests are unaffected because they were already
pushed with their actual mediaType.
- Add a parameterized unit test for ExportOperation.export
covering both Docker manifest.list
This pull request enhances the handling of file and directory copy
operations in Linux containers, particularly by improving destination
path resolution and error handling for the `copyIn` operation with the
`Stat RPC`
This PR is needed for
[container#1190](https://github.com/apple/container/pull/1190)
Today we craft the proxied sockets we create at a path that contains the
container ID. This is normally completely fine, except if the ID is
enormous, as it can cause the UDS path to be longer than the allowed
length.
I think realistically we should have a limit on this. Technically today
the ID length restriction is whatever our grpc buffer length is..
I think 64 is long enough so that folks could use an entire hex based
sha256, any uuid, and also sethostname has a length restriction of 64
anyways.
Move the MiB rounding from LinuxContainer into
`VZVirtualMachineInstance`, so all callers are covered in one place and
the alignment constraint stays with the VMM layer that owns it.
This will also ensure LinuxPod can get memory alignment for free.
If someone wanted to use a custom guest agent/guest image in general the
overhead we chose is kind of a nuisance.. This allows the values to be
configurable, and raises the default to 128, and vminitds cgroup
monitoring to 80.
This change adds support for attaching network block device (NBD) to
both LinuxContainer and LinuxPod.
For LinuxContainer, whether to use the underlying
`VZNetworkBlockDeviceStorageDeviceAttachment` is determined by the URL
of the container Mount source.
For LinuxPod, adds additional API to support pod-level volumes that can
be mounted into multiple containers. The PodVolume type provides enum to
support multiple types of volume source. LinuxContainer can reference
the pod level volume using the `Mount.sharedMount()` constructor with
the name referencing the name of the pod volume. This will allow the NBD
to be attached to the pod at the VM level and then bind-mounted into the
container.
For integration tests, added a lightweight NBD server implementation in
swift that speaks the NBD protocol to ensure there is sufficient
coverage for the changes introduced.
Apple Virtualization NBD support documentation:
https://developer.apple.com/documentation/virtualization/vznetworkblockdevicestoragedeviceattachment
Replace the old platform-gated Signals helper in ContainerizationOS with
a Signal struct in Containerization. Signal provides named constants for
common signals (.kill, .term, etc.), platform-specific namespaces
(Signal.Linux for rtmin/stkflt/pwr, Signal.Darwin for emt/info), string
parsing, and ExpressibleByIntegerLiteral support.
LinuxContainer, LinuxProcess, and LinuxPod kill APIs now take Signal
instead of raw Int32.
Add `Stat` RPC to vminitd.
`Stat` RPC is used to stat file system objects in the VM. It accepts the
path to the object (resolved against VM root) and returns the stat
result.
Make the `HTTP2ConnectBufferingHandler` class public temporarily so that
consumers of the containerization package may reuse it for other grpc
clients.
Closes: #678
There's no reason to have an idle timeout per connection as users may
use this library in many different ways. The grpc lib defaults to a 30
min idle timeout which could be trivially reached if instead of waiting
on the init process, you were waiting on a bunch of execs. NIO under the
hood asserts on ebadfs (which is good, but bad for this scenario..), so
lets turn off the timeout altogether.
- Closes#51. Again.
- No real notable API changes. The largest change code-wise is in
Vminitd. The connection loop is set up immediately in the new API by
calling runConnections() on the client, and this blocks until either the
task it's running on is cancelled or beginGracefulShutdown is called.
Because of this, either we'd have to modify our API to have
VirtualMachineAgent implementations call some run() method, or we just
put runConnections in an internal task on Vminitd. This change uses the
latter approach as it's simpler.
- Updated dependencies to pick up latest protobuf and grpc-swift-2
fixes.
- ClientBootstrap.withConnectedSocket(fd).wait() registers the vsock fd
with epoll/kqueue immediately. vminitd sends its HTTP/2 SETTINGS frame
right away upon accepting the connection, so those bytes can arrive on
the fd — and be read by NIO — before the gRPC pipeline is installed.
With a bare pipeline, NIO has nowhere to send the bytes and discards
them.
- Buffer until ClientConnectionHandler is in the pipeline so it can
process the SETTINGS frame and fire .ready, then schedule delivery of
the buffered data with assumeIsolatedUnsafeUnchecked().execute.
- To help produce more accurate merged logs, defines a
`StderrLogHandler` in the vminitd Application.swift that writes
timestamps to the boot log with millisecond precision and format
identical to that of the log files created using the `--log-root` arg on
`container`.
- Updates hawkeye to latest version (v6.5.1).
- In vminitd, use the Server event loop group for the gRPC server.
Co-authored-by: Danny Canter <danny_canter@apple.com>
Related to https://github.com/apple/container/issues/1251
Change to sharing in the parent directory, and then bind mounting in the
file into the container. This has the unfortunate reality of being less
secure, but the current approach is burdended by a couple things,
namely:
1. You can't share in files that are on a different volume
2. There is a Virtualization bug that causes spurious errors when trying
to open the file.
I've added a doc to go over the approach we've taken, and some
workarounds if the approach is not satisfactory.
To be able to test vminitd/vmexec/linux specific packages on ci it'd be
a heck of a lot easier if `make` just worked. This should be the last
bit needed. The default goal currently compiles just fine after the
linux specific `make deps` is ran. Next in line would be adding decent
unit tests/actually getting ci setup for the linux bits.
I've added an optional progress handler for rootfs unpacking so that
consumers can show progress. The Ubuntu image takes about 8 seconds to
unpack on my machine, and I'm developing an app where it would be useful
to show progress in the UI.
Total size is determined in an optional first pass that scans archive
headers. Bytes written are then reported during unpacking. The optional
first pass adds 15 (Alpine) to 115 (Ubuntu) ms to unpacking duration on
my machine, depending on image size.
The memory limit (clamp to system ram) is a VZ limitation so it's weird
that we had that logic directly in LinuxContainer. Someone could make a
VZVirtualMachineInstance that doesn't have this limitation but we
inherit this logic because it's done before passing off the memory
amount to the VMM.
## What
Add a `DNS.validate()` method that verifies all nameserver entries are
valid IPv4 or IPv6 addresses. The method is called from
`Vminitd.configureDNS()` before applying the configuration.
## Why
Closes#467. Currently, any arbitrary string can be passed as a
nameserver in `DNSConfiguration`, which silently results in an invalid
`/etc/resolv.conf` inside the container. Hostname strings like
`dns.example.com` would be written to resolv.conf but would not work as
nameservers.
## How
- Added `DNS.validate() throws` method in `DNSConfiguration.swift` that
iterates over all nameservers and attempts to parse each as either an
`IPv4Address` or `IPv6Address` (using the existing parsers in
`ContainerizationExtras`)
- Added `import ContainerizationExtras` to `DNSConfiguration.swift` (the
`Containerization` target already depends on `ContainerizationExtras`)
- Called `config.validate()` at the start of `Vminitd.configureDNS()` so
validation happens before any GRPC call
## Testing
Added 6 new unit tests in `DNSTests.swift`:
- ✅ Valid IPv4 nameservers accepted
- ✅ Valid IPv6 nameservers accepted
- ✅ Mixed IPv4/IPv6 nameservers accepted
- ✅ Empty nameserver list accepted
- ❌ Hostname rejected
- ❌ Invalid address rejected
## Checklist
- [x] Tests added
- [x] No breaking changes to existing API (validate() is a new method,
existing init is unchanged)
- [x] Follows existing patterns (uses ContainerizationExtras IP address
types, ContainerizationError for errors)
Signed-off-by: Maxime Grenu <maxime.grenu@gmail.com>
The network protocol and VMNetNetwork implementation currently are
housed on ContainerManager even though they are generally useful types
even outside of this easy to use helper type. This change moves them to
not be nested types anymore, as well as exposes a new param on
VMNetNetworks constructor so we can pass the type of network.
- No real notable API changes. The largest change code-wise is in
Vminitd. The connection loop is set up immediately in the new API by
calling runConnections() on the client, and this blocks until either the
task it's running on is cancelled or beginGracefulShutdown is called.
Because of this, either we'd have to modify our API to have
VirtualMachineAgent implementations call some run() method, or we just
put runConnections in an internal task on Vminitd. This change uses the
latter approach as it's simpler.
- Updated dependencies to pick up latest protobuf and grpc-swift-2
fixes.
- ClientBootstrap.withConnectedSocket(fd).wait() registers the vsock fd
with epoll/kqueue immediately. vminitd sends its HTTP/2 SETTINGS frame
right away upon accepting the connection, so those bytes can arrive on
the fd — and be read by NIO — before the gRPC pipeline is installed.
With a bare pipeline, NIO has nowhere to send the bytes and discards
them.
- Buffer until ClientConnectionHandler is in the pipeline so it can
process the SETTINGS frame and fire .ready, then schedule delivery of
the buffered data with assumeIsolatedUnsafeUnchecked().execute.
- To help produce more accurate merged logs, defines a
`StderrLogHandler` in the vminitd Application.swift that writes
timestamps to the boot log with millisecond precision and format
identical to that of the log files created using the `--log-root` arg on
`container`.
- Updates hawkeye to latest version (v6.5.1).
---------
Co-authored-by: Danny Canter <danny_canter@apple.com>
Now that the static linux SDK we use has libarchive linked against it,
we can finally live our dream (/s) of supporting copying directories
in/out a little easier. This was.. kind of annoying. I think the very
simple route of writing archive to guest/host -> streaming over grpc is
simplest, but on the guest end we have a couple problems:
1. The VMs rootfs is read only today always, which is a good thing to me
and I don't want to change if we don't have to.
2. Writing the archive to the containers rootfs temporarily could work,
but it's a bit weird, and the user can make the containers rootfs
readonly which would screw that plan.
3. We could write it to /run or /tmp, but they're tmpfs and dealing with
the headache of the user possibly tarring an enormous dir is one I don't
want to care about.
So, that leaves us with the (truthfully better to me) approach of trying
to write the tar data directly to the host and skipping grpc which kinda
forces us to have a temp spot. Because of that, I made it such that we
pass a port number from host<->guest, and transfer the actual data
(either the single file or tarred dir contents) over the vsock port
instead. The stream is kinda clunky, but it just serves as a means to
exchange metadata and a "we're done" signifier.
Closes#221
This contains (in the static linux sdk but the swift version and the sdk
must match) libarchive support and mimalloc! This also (unrelated to the
sdk bump it seems) raises the amount of retries we do when dialing
vminitd. It seems CI on rare occasions recently has timed out on some
tests. We could also explore just lowering the concurrency on the
integration suite.
Closes#256
If the target of the uds proxy goes through a symlink (/var/run) we had
the same bug we had with mounts previously where we'd follow this and if
the resolved path existed in the root of the VM we'd end up mounting
there. Fix this by plopping the sockets in a holding spot and then bind
mounting them in (which has the correct resolution logic).
Danny Canter