- Fixes#776
- Release 2.9.0 of `grpc-swift-nio-transport` fixes
the HTTP/2 initialization race that required a
`HTTP2ConnectBufferingHandler` for the vminitd
API. This upgrades the dependency and eliminates
the workaround.
- Closes#712.
- Replace synchronous `write()` calls in
`BidirectionalRelay` with non-blocking I/O
and `DispatchSourceWrite` backpressure
handling. Under concurrent vsock proxy load,
a single blocked write on the shared serial
dispatch queue would freeze all relay
connections permanently, including unrelated
new connections.
- Set relay file descriptors to `O_NONBLOCK`
and handle `EAGAIN` by suspending reads and
installing a write source to drain pending data.
- Give each `BidirectionalRelay` its own serial queue
instead of sharing one from `UnixSocketRelayManager`,
eliminating cross-connection blocking.
- Resume suspended read sources before cancelling
in `stop()` — GCD does not deliver cancel handlers
on suspended dispatch sources, which caused file
descriptor and memory leaks on teardown under
backpressure.
- Guard-unwrap `buf.baseAddress` in
`drainPendingWrite`.
- Closes#51. Again.
- No real notable API changes. The largest change code-wise is in
Vminitd. The connection loop is set up immediately in the new API by
calling runConnections() on the client, and this blocks until either the
task it's running on is cancelled or beginGracefulShutdown is called.
Because of this, either we'd have to modify our API to have
VirtualMachineAgent implementations call some run() method, or we just
put runConnections in an internal task on Vminitd. This change uses the
latter approach as it's simpler.
- Updated dependencies to pick up latest protobuf and grpc-swift-2
fixes.
- ClientBootstrap.withConnectedSocket(fd).wait() registers the vsock fd
with epoll/kqueue immediately. vminitd sends its HTTP/2 SETTINGS frame
right away upon accepting the connection, so those bytes can arrive on
the fd — and be read by NIO — before the gRPC pipeline is installed.
With a bare pipeline, NIO has nowhere to send the bytes and discards
them.
- Buffer until ClientConnectionHandler is in the pipeline so it can
process the SETTINGS frame and fire .ready, then schedule delivery of
the buffered data with assumeIsolatedUnsafeUnchecked().execute.
- To help produce more accurate merged logs, defines a
`StderrLogHandler` in the vminitd Application.swift that writes
timestamps to the boot log with millisecond precision and format
identical to that of the log files created using the `--log-root` arg on
`container`.
- Updates hawkeye to latest version (v6.5.1).
- In vminitd, use the Server event loop group for the gRPC server.
Co-authored-by: Danny Canter <danny_canter@apple.com>
- No real notable API changes. The largest change code-wise is in
Vminitd. The connection loop is set up immediately in the new API by
calling runConnections() on the client, and this blocks until either the
task it's running on is cancelled or beginGracefulShutdown is called.
Because of this, either we'd have to modify our API to have
VirtualMachineAgent implementations call some run() method, or we just
put runConnections in an internal task on Vminitd. This change uses the
latter approach as it's simpler.
- Updated dependencies to pick up latest protobuf and grpc-swift-2
fixes.
- ClientBootstrap.withConnectedSocket(fd).wait() registers the vsock fd
with epoll/kqueue immediately. vminitd sends its HTTP/2 SETTINGS frame
right away upon accepting the connection, so those bytes can arrive on
the fd — and be read by NIO — before the gRPC pipeline is installed.
With a bare pipeline, NIO has nowhere to send the bytes and discards
them.
- Buffer until ClientConnectionHandler is in the pipeline so it can
process the SETTINGS frame and fire .ready, then schedule delivery of
the buffered data with assumeIsolatedUnsafeUnchecked().execute.
- To help produce more accurate merged logs, defines a
`StderrLogHandler` in the vminitd Application.swift that writes
timestamps to the boot log with millisecond precision and format
identical to that of the log files created using the `--log-root` arg on
`container`.
- Updates hawkeye to latest version (v6.5.1).
---------
Co-authored-by: Danny Canter <danny_canter@apple.com>
- Closesapple/container#988.
- macOS libarchive is not built with zstd support, so the workaround in
ArchiveReader is to attempt to decompress every archive as zstd. If
decompression fails, we pass the original archive to libarchive. If it
succeeds, we pass the uncompressed archive.
- Adds blob media type recognition for zstd to EXT4Unpacker. Tested zstd
blob unpack using `image pull tonistiigi/hello-world:zstd-docker`.
Co-authored-by: Aditya Ramani <a_ramani@apple.com>
* Admit only valid OCI layout files when loading image.
- Adds `AdmissionMapper` protocol for validating archive
member paths and normalizing them to relative paths
under the extraction root directory.
- Renames `Reader.swift` to `ArchiveReader.swift` to
match type name.
- Rework `TempDir` to address `NSString.utf8String`
deprecation warning in Swift 6.2.3.
- Rework `ArchiveReader.extractContent()` to use an
`AdmissionMapper` to validate and remap archive
members before extracting.
- Adds `IdentityAdmissionWrapper` for naive extraction.
- Adds `NoSymlinkAdmissionWrapper` that only extracts
regular files and directories under the extraction root.
- Adds `OCIImageAdmissionWrapper` that only extracts
valid OCI image layout paths.
- Use `OCIImageAdmissionWrapper` for `cctl image load`
and print rejected paths.
* PR feedback.
* Adds public init() for TrustedAdmissionMapper.
* Replace AdmissionMapper with more secure extraction.
- Adds FileDescriptor.mkdirSecure() to prevent
root escapes on member pathnames, and to prevent
symlink traversal.
- Adds FileDescriptor.unlinkRecursive() to
facilitate overwrites when there are multiple
archive entries with the same member path.
- Adds FileDescriptor.validateSymlinkTargetInRoot()
to validate that extracted symlink targets do
not escape the root.
- Rewrite ArchiveReader.extractContents() to use
secure path functions.
* Remove unneeded symlink check, rename files.
* Simplify the lexical normalizer workaround.
* Reject member paths containing parent traversal components.
* Remove unused lexical normalization workaround.
* Fix leaking fds, extract absolute members as relative.
Closely related to https://github.com/apple/container/issues/305 I would
like to override the used SSL TrustRoots via standard env variables.
This here would add this configuration and would give an entrypoint for
an implementation of 305 to provide CLI flags or similar.
This has no tests yet, as this would require setting up something like a
MITM proxy when testing against a registry.
As I am unfamiliar with the codebase, I would be willing to do this, but
would require a first nudge on where to best implement this.
To actually use this, we would need to add the allowed env variables to
the `container system start` command env filter.
Make sure we always set TERM for containers that ask for a tty. Right
now this handling was spread around in a bunch of spots, but I'd wager
setting it for the client on the host via LinuxContainer/Process is more
sane and already what we do for a lot of the other fields.
This PR resolves warnings:
```
…/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Preview/Symbolic Links/SwiftDocCPluginUtilities/BuildGraph/DocumentationBuildGraph.swift:106:17: warning: capture of non-sendable type 'Target.Type' in an isolated closure
104 | // Copy the closure and the target into a block operation object
105 | let new = BlockOperation { [work, task] in
106 | work(task)
| `- warning: capture of non-sendable type 'Target.Type' in an isolated closure
107 | }
108 | operationsByID[task.id] = new
…/.build/checkouts/swift-docc-plugin/Plugins/Swift-DocC Preview/Symbolic Links/SwiftDocCPluginUtilities/BuildGraph/DocumentationBuildGraph.swift:105:41: warning: capture of non-sendable type 'Target.Type' in an isolated closure
103 | }
104 | // Copy the closure and the target into a block operation object
105 | let new = BlockOperation { [work, task] in
| `- warning: capture of non-sendable type 'Target.Type' in an isolated closure
106 | work(task)
107 | }
```
adityabagchi24