Commit Graph
194 Commits
Author SHA1 Message Date
Harry LiandDmitry Kovba 5fbae0e590 feat: add proxy utility (#288)
Signed-off-by: Harry Li <harrymingh@gmail.com>
Co-authored-by: Dmitry Kovba <dkovba@apple.com>
2025-09-12 11:23:41 -07:00
Dmitry Kovba 31bfef4a8e Print the Swift version before building (#286)
With the changes in this PR, we'll print the Swift version before
building. This mirrors a similar change in container, introduced in
https://github.com/apple/container/pull/550.
2025-09-11 10:17:24 -07:00
Dmitry Kovba a41dbd4671 Update the toolchain and Static Linux SDK (#287)
The PR updates the toolchain and Static Linux SDK. This resolves the
compilation errors below:

```
Building vminitd and vmexec...
Building for debugging...
<unknown>:0: error: stack nearly exhausted; compilation time may suffer, and crashes due to stack overflow are likely
<module-includes>:1:10: note: in file included from <module-includes>:1:
1 | #include "SwiftMusl.h"
  |          `- note: in file included from <module-includes>:1:
2 | 

/Users/Dmitry/Library/org.swift.swiftpm/swift-sdks/swift-6.2-DEVELOPMENT-SNAPSHOT-2025-06-25-a_static-linux-0.0.1.artifactbundle/swift-6.2-DEVELOPMENT-SNAPSHOT-2025-06-25-a_static-linux-0.0.1/swift-linux-musl/musl-1.2.5.sdk/aarch64/usr/lib/swift_static/linux-static/aarch64/SwiftMusl.h:135:10: error: could not build module 'sys_stropts'
133 | #include <sys/signalfd.h>
134 | #include <sys/statfs.h>
135 | #include <sys/stropts.h>
    |          `- error: could not build module 'sys_stropts'
136 | #include <sys/swap.h>
137 | #include <sys/syscall.h>

<unknown>:0: error: stack nearly exhausted; compilation time may suffer, and crashes due to stack overflow are likely
<unknown>:0: error: could not build C module 'SwiftMusl'
<unknown>:0: error: stack nearly exhausted; compilation time may suffer, and crashes due to stack overflow are likely
<module-includes>:1:10: note: in file included from <module-includes>:1:
1 | #include "SwiftMusl.h"
  |          `- note: in file included from <module-includes>:1:
2 | 

/Users/Dmitry/Library/org.swift.swiftpm/swift-sdks/swift-6.2-DEVELOPMENT-SNAPSHOT-2025-06-25-a_static-linux-0.0.1.artifactbundle/swift-6.2-DEVELOPMENT-SNAPSHOT-2025-06-25-a_static-linux-0.0.1/swift-linux-musl/musl-1.2.5.sdk/aarch64/usr/lib/swift_static/linux-static/aarch64/SwiftMusl.h:135:10: error: could not build module 'sys_stropts'
133 | #include <sys/signalfd.h>
134 | #include <sys/statfs.h>
135 | #include <sys/stropts.h>
    |          `- error: could not build module 'sys_stropts'
136 | #include <sys/swap.h>
137 | #include <sys/syscall.h>

<unknown>:0: error: stack nearly exhausted; compilation time may suffer, and crashes due to stack overflow are likely
<unknown>:0: error: could not build C module 'SwiftMusl'
[719/1348] Compiling x_all.cc
make[1]: *** [all] Error 1
make: *** [vminitd] Error 2
```

To update:

- Run `make cross-prep`
- List the installed Static Linux SDKs using `swift sdk list`
- Remove the old Static Linux SDKs using `swift sdk remove <SDK>`

Optionally:

- List the installed toolchains using `swiftly list`
- Uninstalled the old toolchains using `swiftly uninstall <TOOLCHAIN>`
2025-09-11 10:16:31 -07:00
Michael Crosby 753aa080f0 make Import and Export public types (#284)
Signed-off-by: crosbymichael <michael_crosby@apple.com>
2025-09-09 16:29:05 -04:00
Dmitry Kovba 1b1cfdbaf3 Add comments clarifying thread-safety (#282) 2025-09-08 23:15:22 -07:00
Danny Canter 9aa552593b LinuxContainer: Try and stop vm on start failure (#281)
If start fails today the vm stays running.
2025-09-05 11:14:52 -07:00
Dmitry Kovba 9319eecf14 Prevent a leak detection crash when close() fails (#280)
When an image is broken, `container-core-images` may crash with:
```
Application Specific Information:
_NIOFileSystem/SystemFileHandle.swift:131: Fatal error: Leaking file descriptor: the handle for '/Users/Dmitry/Library/Application Support/com.apple.container/content/ingest/03144D38-D98D-4833-9E0C-60229BF653D9/.tmp-MIB1oc' MUST be closed or detached with 'close()' or 'detachUnsafeFileDescriptor()' before the final reference to the handle is dropped.

Thread 9 Crashed:
0   libswiftCore.dylib            	       0x1adbfe110 _assertionFailure(_:_:file:line:flags:) + 176
1   container-core-images         	       0x105773628 closure #1 in SystemFileHandle.deinit + 424 (SystemFileHandle.swift:131)
2   container-core-images         	       0x10577365c partial apply for closure #1 in SystemFileHandle.deinit + 20
3   container-core-images         	       0x104ebd994 closure #1 in LockStorage.withLockedValue<A>(_:) + 124 (NIOLock.swift:183)
4   container-core-images         	       0x104ebde38 partial apply for closure #1 in LockStorage.withLockedValue<A>(_:) + 52
5   container-core-images         	       0x104c387fc ManagedBuffer<>.withUnsafeMutablePointers<A, B>(_:) + 152
6   container-core-images         	       0x104ebd8ec LockStorage.withLockedValue<A>(_:) + 172 (NIOLock.swift:180)
7   container-core-images         	       0x104ebe088 NIOLockedValueBox.withLockedValue<A>(_:) + 104 (NIOLockedValueBox.swift:38)
8   container-core-images         	       0x105773438 SystemFileHandle.deinit + 112 (SystemFileHandle.swift:128)
9   container-core-images         	       0x105773684 SystemFileHandle.__deallocating_deinit + 28
10  libswiftCore.dylib            	       0x1adaeef50 _swift_release_dealloc + 56
11  libswiftCore.dylib            	       0x1adaefabc bool swift::RefCounts<swift::RefCountBitsT<(swift::RefCountInlinedness)1>>::doDecrementSlow<(swift::PerformDeinit)1>(swift::RefCountBitsT<(swift::RefCountInlinedness)1>, unsigned int) + 152
12  container-core-images         	       0x104b81aac RegistryClient.fetchBlob(name:descriptor:into:progress:) + 180 (RegistryClient+Fetch.swift:200)
13  container-core-images         	       0x104b9f481 protocol witness for ContentClient.fetchBlob(name:descriptor:into:progress:) in conformance RegistryClient + 1
14  container-core-images         	       0x1049b6b29 ImageStore.ImportOperation.fetchBlob(_:) + 1 (ImageStore+Import.swift:167)
15  container-core-images         	       0x1049b5c6d ImageStore.ImportOperation.fetch(_:) + 1 (ImageStore+Import.swift:153)
16  container-core-images         	       0x1049b4f69 closure #1 in closure #1 in ImageStore.ImportOperation.fetchAll(_:) + 1 (ImageStore+Import.swift:126)
17  container-core-images         	       0x1049bb08d partial apply for closure #1 in closure #1 in ImageStore.ImportOperation.fetchAll(_:) + 1
18  libswift_Concurrency.dylib    	       0x2945e05f9 completeTaskWithClosure(swift::AsyncContext*, swift::SwiftError*) + 1
```

Replacing the `SwiftNIO` file handle with the `Foundation` file handle
resolved the crash. However, this led to a worse performance:

```
Debug version:
SwiftNIO: 1:05s
Foundation: 1:18s (1.2 times worse)

Release version:
SwiftNIO: 0:34s
Foundation: 1:00s (1.8 times worse)
```

This PR attempts to prevent the crash without switching back to the
`Foundation` file handle.
2025-09-04 23:07:11 -07:00
Danny Canter b422e036da ContainerizationOS: Rework User type (#279) 2025-09-04 18:16:17 -07:00
Dmitry Kovba 6a31184ea5 Handle warnings as errors (#271)
To handle warnings as errors when building, use `make`. To allow
warnings, use `make WARNINGS_AS_ERRORS=false test`.

Also see https://github.com/apple/container/pull/535.
2025-08-28 10:39:17 -04:00
Eric Ernst f92c0cd8c6 containerization: Add additional tests (#273)
Add additional tests, with varying levels of utility.

Before: Overall Code Coverage: 41.73%
After: Overall Code Coverage: 42.95%

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2025-08-26 13:13:05 -07:00
Agam Dua 4099bdccd6 vminitd: Removed extraneous import (#276)
We shouldn't need this import anymore

cc: @crosbymichael

Signed-off-by: Agam Dua <agam_dua@apple.com>
2025-08-26 09:36:44 -07:00
Danny Canter df2425ef7f vminitd: Wait for execvpe to return to continue (#275)
Today, because we don't wait for execvpe to finish to continue onwards,
it's possible that if you did an exec quick enough after starting an
init process for a container, that you could join the init processes
namespaces before pivot_root has taken place which is quite fun. Let's
wait for exec to finish (or an error to occur) to prevent this.
2025-08-26 09:34:13 -07:00
Danny Canter 48e2f412b3 vminitd: Log spec for exec processes (#274)
We had a log for the initial oci runtime spec, but we don't log anything
for execed processes. This makes it hard to look into issues that may be
due to just a mistake in the spec and nothing wrong with the actual
logic of the program.
2025-08-25 12:26:35 -07:00
Danny Canter 53021bec36 Cgroup2Manager: Various adjustments (#270)
Systemd loves to move the root cgroups processes to one it created named
/init.scope and then write the root cgs subtree_control file. Because of
this we can't just add exec processes to the cg we made for the
container anymore as we'll get EBUSY. We should follow where the init
processes cg is actually at (/proc/pid/cgroup) and add it there.
2025-08-19 17:31:51 -07:00
Danny Canter 2f32e36a0d Cgroup2Manager: Fix error log mistake (#269)
I accidentally made the creation log error level. Additionally adds a
log to deletion of the cg.
2025-08-18 12:19:55 -04:00
Dmitry Kovba c3f5816353 Handle the same file name in the fetchBlob method (#258)
With the suggested change, when a file exists, we'll verify whether it
has the same content before skipping it.
2025-08-15 14:02:22 -04:00
Michael Crosby 54980872d9 add ability to export initfs as block (#268)
This refactors the rootfs command to produce an image and have the
ability to create an ext4 formatted block of the init filesystem
directly from the command.

closes #220

Signed-off-by: crosbymichael <michael_crosby@apple.com>
2025-08-15 00:12:14 -07:00
Michael Crosby e08b0d4fb4 add kernel support section to readme (#267)
closes #224

Signed-off-by: crosbymichael <michael_crosby@apple.com>
2025-08-14 15:15:43 -07:00
Danny Canter 531ccd6205 vminitd: Rename CgroupManager -> Cgroup2Manager (#266)
To be more explicit.
2025-08-13 20:16:03 -04:00
Danny Canter 79e07b43ce vminitd: Add init and execs to cgroup (#265)
We currently weren't doing any cgroup setup whatsoever. For the most
part this doesn't matter too much, however certain images that may fool
around with cgroups don't like this :). Lets do the bare minimum these
expect which is to at least have the process running in a nested cg and
not apart of the root cg.
2025-08-13 12:58:49 -04:00
Dmitry Kovba 50500276e2 Update allocations in Allocator (#264)
The `allocations` property in the nested `Allocator` structure wasn't
updated. This PR resolves the problem by using mutating methods and
keeping the original types as structs.
2025-08-13 12:50:23 -04:00
Danny Canter 985dac83e5 vminitd: Proper pty setup / mount /dev/console (#248) 2025-08-12 20:03:27 -04:00
Tim Sneath 5a1975a3bf Clarify macOS version support and minor style tweaks (#263)
Clarify that container is only supported on macOS 26, and remove
language that implies that it's also designed for macOS 15.

This change also includes a couple of minor textual changes to improve
readability.
2025-08-11 13:35:38 -07:00
Danny Canter 87541cbe65 Agent: Add and implement writeFile (#259)
This change adds in a new rpc that we probably should swap to for the
underlying implementation of a couple rpcs we currently have. This new
rpc simply writes to an existing or new file some data. This is useful
for guest (and container) setup for things like /etc/hosts,
/etc/resolv.conf, writing to one-off cgroup files and so on.

Currently I've just implemented subtree_control toggling for the root cg
in standardSetup with this new change, but in the future I'd like to
swap the implementations of our /etc/hosts and /etc/resolv.conf logic
with this.
2025-08-11 12:33:30 -04:00
Dmitry Kovba 7b00f39140 Close a handle inside a lock (#257)
Improves https://github.com/apple/containerization/pull/245. A potential
deadlock risk was not proven. This PR moves the closing operation inside
the lock to prevent leaking an unclosed handle in case of an error.
2025-08-11 12:26:31 -04:00
Danny Canter 259c841591 ContainerizationExtras: Timeout adjustments (#254)
Three things:

1. Timeout.run today didn't allow a throwing closure.
2. Add a new constructor for Timeout that takes in a Duration directly
so we can be more granular than seconds.
3. Add unit tests because we all love those :)
2025-08-07 14:34:45 -04:00
Danny Canter adf5bd1866 LinuxContainer: Support pause/resume (#217)
This allows pausing the (really VM) container and any processes running
inside.
2025-08-07 13:12:11 -04:00
Danny Canter 88aeffe4db Add /dev/rtc symlink (#252)
Closes #251

We use devtmpfs today so our pal /dev/rtc0 is there. It's typical to
have a /dev/rtc symlink to /dev/rtc0.
2025-08-07 13:09:57 -04:00
Dmitry Kovba 14482784db Ensure mutable properties aren't accessed outside their intended thread-safe usage patterns (#247)
This PR modifies the access level of certain mutable properties to
ensure their thread-safe usage. It doesn't address
https://github.com/apple/containerization/issues/246 at this time.
2025-08-06 18:44:27 -07:00
Dmitry Kovba 36a5cd505a Replace public computed properties with locks in setters (#242)
This PR replaces public computed properties with setters in the
`NATNetworkInterface` class with constants to avoid using locks in the
setters. A new instance of `NATNetworkInterface` should be created to
change these values.
2025-08-06 18:29:07 -07:00
Dmitry Kovba bfc34e0da4 Eliminate race windows in the Socket class (#245)
While individual accesses were properly mutex-protected, the logical
operations were not atomic, leading to a potential inconsistency in a
multi-threaded environment. Multiple separate `state.withLock` calls
created race windows. With the proposed changes, single atomic
operations check and update the state together.
2025-08-06 18:22:54 -07:00
Danny Canter b2b2d41d77 Revert "Vminitd: Remove special cased /run mount (#233)" (#253)
This reverts commit 5f7dcd2a16. Rosetta is
setup at /run because the guest rootfs is ro, and unfortunately this
happens prior to LinuxContainer getting to mount any of this. I'm not
super happy about this, but we can figure something out later.

Additionally this adds a rosetta flag to cctl so we can quickly
spotcheck things like this again.
2025-08-06 17:34:00 -07:00
Danny Canter 4d2f73d300 LinuxContainer: Allow reuse after being stopped (#240)
After the container is stopped it's a bit odd how we don't allow the
object to be restarted. If someone wanted to continuously rerun the same
container they'd need to make a new object every time even though all
guest state is blown away on stop() so it's a clean slate.

This change makes it so that you can create+start again after stop
(although create stands out like a sore thumb now as stop -> create is
just strange).
2025-08-06 09:24:07 -07:00
Eric Ernst 14239b02c3 ContainerizationOS: minor fixes (#241)
While reviewing test coverage, I noticed a couple of minor issues:
 - 0 is non-negative - allow for zero memory
 - we should likely use lstat instead of stat for file info
 - we should check for empty name parameter

---------

Signed-off-by: Eric Ernst <eric_ernst@apple.com>
2025-07-31 23:32:10 -07:00
Danny Canter ab37e574a0 ContainerManager: Make root/imagestore suppliable (#228)
We should be able to supply an ImageStore if the user wants a different
ContentStore, or a different path.
2025-07-29 15:41:54 -04:00
Dmitry Kovba d512fcb7cc Transition to actors (#225)
Due to the reduced use of the macro, we can now fully transition to
`Mutex`.
2025-07-29 08:10:48 -04:00
Dmitry Kovba 95978185fd Standardize property names in ReadStream (#213)
This PR only changes names of private properties to align them with the
naming standards.
2025-07-28 12:26:09 -04:00
Danny Canter 71a5765d7f LinuxContainer: Remove IOHandler (#235)
Is unused after the config rework.
2025-07-28 12:15:23 -04:00
Kathryn Baldauf 41c3e4a2c3 Update xcode developer dir to new regularly updated symlink (#234)
The Xcode installation on the runners is regularly updated to the latest
release. A symlink at /Applications/Xcode-latest.app points to the most
recent installation. We should not rely on a specific version path.

Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
2025-07-25 11:38:55 -07:00
Danny Canter 5f7dcd2a16 Vminitd: Remove special cased /run mount (#233)
Revisiting this, I'm not a huge fan of having /run be special cased for
some reason
2025-07-25 09:43:48 -07:00
Danny Canter 032dcee131 Chore: Simplify AttachedFilesystem constructor (#232)
The type copy is the exact same on any of the types, just move it
outside of the cases like we have for options and destination.
2025-07-24 16:40:32 -07:00
Danny Canter a5937a8b13 Misc. capitalization/comment adjustments (#231)
The VirtualMachineInstance comment is not entirely true. It just
represents a single VM.
2025-07-24 16:40:21 -07:00
Danny Canter a73541e393 Netlink: Change logging to trace level (#229)
There's 82KiB of logs on every container launch for netlink, that may be
a bit excessive 😄.
2025-07-24 11:36:53 -07:00
Danny Canter 6071a9fe1f LinuxContainer: Remove SendableProperty usage (#223)
The state updates because of the way the macro works weren't actually
being performed under the lock.
2025-07-22 15:41:29 -07:00
Michael Crosby 0a4ff1b737 Add ContainerManager (#200)
A ContainerManager is a type that handles more of the required resource
needed to create and run a container.

---------

Signed-off-by: crosbymichael <michael_crosby@apple.com>
2025-07-22 11:53:52 -07:00
Danny Canter 636099970e LinuxContainer/LinuxProcess: Rework supplying configuration (#219)
This is a fairly large reworking, but it gets rid of something that has
plagued this since release which is the properties needing to be locked
to be Sendable compliant. This was somewhat of a copout because we
mostly know there's not a great deal of ways to have misused the setup
today, but alas we'd need to either mark the type as `@unchecked` or
just find a different route for setting the configuration. This change:

Exposes the underlying Configuration type that today only housed things
that aren't on the OCI spec. I'd love to just expose the OCI spec, but
we don't (and possibly never will) support everything on the spec, so
exposing it to be freely modified would be a bit odd. Now everything
related to the container is configured on this type, and the same goes
for execs.
2025-07-22 12:14:57 -04:00
Danny Canter fcd198e968 ContainerizationOS: Speed up fd closes (#222)
Instead of looping through from minimum fd -> rlimit max, we can
close_range(2) or read /proc/self/fd on Linux, and read /dev/fd on
macOS.
2025-07-22 10:30:10 -04:00
Danny Canter 8b93a2083a LinuxContainer: Fix build break (#218)
Crummy rebase that didn't pickup the Mutex changes
2025-07-18 08:44:16 -07:00
Danny Canter c4b6b68ce7 LinuxContainer: Add /etc/hosts writing functionality (#207)
Closes #206 

Much like we have support for supplying DNS configurations and writing
out /etc/resolv.conf, this adds a way to write out /etc/hosts for a
given container.
2025-07-18 00:54:18 -07:00
Danny Canter 3cc638624f KeychainHelper: Simplify lookup (#214) 2025-07-17 09:37:59 -07:00