This PR replaces public computed properties with setters in the
`NATNetworkInterface` class with constants to avoid using locks in the
setters. A new instance of `NATNetworkInterface` should be created to
change these values.
This reverts commit 5f7dcd2a16. Rosetta is
setup at /run because the guest rootfs is ro, and unfortunately this
happens prior to LinuxContainer getting to mount any of this. I'm not
super happy about this, but we can figure something out later.
Additionally this adds a rosetta flag to cctl so we can quickly
spotcheck things like this again.
After the container is stopped it's a bit odd how we don't allow the
object to be restarted. If someone wanted to continuously rerun the same
container they'd need to make a new object every time even though all
guest state is blown away on stop() so it's a clean slate.
This change makes it so that you can create+start again after stop
(although create stands out like a sore thumb now as stop -> create is
just strange).
A ContainerManager is a type that handles more of the required resource
needed to create and run a container.
---------
Signed-off-by: crosbymichael <michael_crosby@apple.com>
This is a fairly large reworking, but it gets rid of something that has
plagued this since release which is the properties needing to be locked
to be Sendable compliant. This was somewhat of a copout because we
mostly know there's not a great deal of ways to have misused the setup
today, but alas we'd need to either mark the type as `@unchecked` or
just find a different route for setting the configuration. This change:
Exposes the underlying Configuration type that today only housed things
that aren't on the OCI spec. I'd love to just expose the OCI spec, but
we don't (and possibly never will) support everything on the spec, so
exposing it to be freely modified would be a bit odd. Now everything
related to the container is configured on this type, and the same goes
for execs.
Closes#206
Much like we have support for supplying DNS configurations and writing
out /etc/resolv.conf, this adds a way to write out /etc/hosts for a
given container.
In general I believe it's clever to synthesize an index for
single-platform image manifest, but we still need a way to distinguish
it. Add a dedicated annotation is the slightest change I've come up
with, and it's also OCI compliant. With this change come in, we can work
around https://github.com/apple/container/issues/212 and imitate the
behavior of other runtime with `container`.
Note that since `cctl` is meant to be a dedicated tool for inspecting
the Containerization framework itself, I didn't apply the indirection
for it, and it will be as-is with the genuine storage.
- The `@SendableProperty` macro now checks for supported types to
prevent a misuse with unsupported types like structs, dictionaries, and
arrays
- The new `@SendablePropertyUnchecked` can be used with classes and
enums
This allows the user the ability to raise an EOF for the containers
stdin. Today there's no way to close stdin so something as simple as
"cat" and relying on EOF to move the process forward doesn't work
While this only supports AS, xcode's default will build all arches, so
this presents a bit nicer default for Archiving an xcode application
that utilizes containerization.
Fixes#179
Signed-off-by: crosbymichael <michael_crosby@apple.com>
Creates a new Unpacker protocol that defines a single method
```
func unpack(_ image: Image, for platform: Platform, at path: URL, progress: ProgressHandler?) async throws -> Mount
```
This change also removes the `unpack(...)` method from the Image type.
Before
```
let mount = try await image.unpack(for: platform, at: path)
```
After
```
let unpacker = EXT4Unpacker(blockSizeInBytes: 2.gib())
let mount = try await unpacker.unpack(image, for: platform, at: path)
```
---------
Signed-off-by: Aditya Ramani <a_ramani@apple.com>
Here we implement the mechanism for ensuring that `container` doesn't
set up two default routes on containers that connect to multiple
networks. We'll implement the policy in apple/container#243.
Because we were installing rosetta directly in the constructor for
VZVirtualMachineInstance, and we'd prefer to not have the constructor
async as it pollutes so much more, we had devised this gnarly callback
approach for the install that is a bit of an eyesore. This changes focus
is on moving the install flow to a method that is already async so we
can piggyback off of it, and removing the install logic from the config
-> VZConfig conversion. Now the install will occur during start() if
rosetta is not installed.
Make sure we always set TERM for containers that ask for a tty. Right
now this handling was spread around in a bunch of spots, but I'd wager
setting it for the client on the host via LinuxContainer/Process is more
sane and already what we do for a lot of the other fields.
## Changes
### README.md
- Fixed grammatical error: "when a process is ran" → "when a process is
run"
- Fixed plural form: "for instruction on" → "for instructions on"
### kernel/README.md
- Fixed numbering sequence: "4. Run `make`" → "3. Run `make`" in build
instructions
### Sources/Containerization/LinuxContainer.swift
- Fixed grammatical error in comment: "The container has ran" → "The
container has run"
We should return our own error type always for our methods. This wraps
all of the public methods on the process to return a more full featured
error than the rpcerrors we get today.
This change adds a new private method `waitIoComplete` on the
`LinuxProcess` type.
This method is called internally when the user calls `wait` for a
process - and it tries to give the IO streams some time to clear their
buffers.
Internally, this method sets up an `AsyncStream` down which an item is
sent when the vsock connection for either stdout/stderr is terminated.
We get this termination signal when the readability handler for the
associated fd fires with a no available data.
Inside the guest - once we are done relaying the IO from the process
into the socket connection, we close the socket fd which triggers the
above.
All this logic is wrapped around a timeout of 3 seconds, just to ensure
the method does not block forever.
---------
Signed-off-by: Aditya Ramani <a_ramani@apple.com>
This PR ensures that we enter a lock inside the `@SendableProperty`
implementation as soon as we access a computed property. Additionally,
it mirrors the access level of the original property. Both changes are
required for [improved
accuracy](https://github.com/apple/container/pull/144) of progress
updates in container. Additionally, it should resolve
https://github.com/apple/containerization/issues/60 that occurs on
certain configurations.
Please tag as 0.1.1 after merging.
Use scoped access to the agent connection to ensure the vsock fds are
close(2)'d.
### Note
In a follow up I'd like to change `LinuxProcess` to dialing the agent
connection inside of `.start()`. I don't like needing to pass it an
agent connection. It has a reference to a vm so it can easily dial
inside itself if it needs to.
Fixes#85
The virtualization bool on LinuxContainer mostly just forwards to
VZVirtualMachineInstance which today would silently take your boolean
and do nothing if the underlying platform doesn't have support for it.
This is (to me) arguably worse than erroring, as it gives the client a
false security that the setting is on, and they should have virt
capabilities in the container/guest now.
This change makes it so that we throw a ContainerizationError for this
case, with a code of .unsupported so it's checkable by a user if they
want more information on the "why".
## Issue
When pulling images, the download speed appears to be slower compared to
Docker.
I found that parallel chunk generation was being performed during the
layer download process.
While individual chunks allow parallel download operations, the overall
process remains sequential between chunks.
This results less performance when chunks contain both small and large
layers mixed together.
## Changes
Discontinued chunk-based segmentation to enable more efficient parallel
downloads.
## Results(in my local env)
| image | layers | old | new |
| ---|---|---|---|
| node:latest | 12 | 1m15s | 1m10s |
| ghcr.io/norio-nomura/swift_discord_bot:main | 54 | 2m45s | 2m30s |
<details>
<summary>raw terminal log</summary>
```
❯ rm -rf ~/Library/Application\ Support/com.apple.containerization && time bin/cctl_old images pull docker.io/library/node:latest
image pulled
bin/cctl_old images pull docker.io/library/node:latest 42.37s user 6.60s system 64% cpu 1:15.94 total
❯ rm -rf ~/Library/Application\ Support/com.apple.containerization && time bin/cctl_new images pull docker.io/library/node:latest
image pulled
bin/cctl_new images pull docker.io/library/node:latest 42.27s user 6.72s system 69% cpu 1:10.58 total
❯ rm -rf ~/Library/Application\ Support/com.apple.containerization && time bin/cctl_old images pull docker.io/library/node:latest
image pulled
bin/cctl_old images pull docker.io/library/node:latest 45.65s user 7.36s system 70% cpu 1:15.64 total
❯ rm -rf ~/Library/Application\ Support/com.apple.containerization && time bin/cctl_new images pull docker.io/library/node:latest
image pulled
bin/cctl_new images pull docker.io/library/node:latest 39.76s user 6.32s system 65% cpu 1:10.50 total
❯ rm -rf ~/Library/Application\ Support/com.apple.containerization && time bin/cctl_old images pull docker.io/library/node:latest
image pulled
bin/cctl_old images pull docker.io/library/node:latest 42.47s user 6.75s system 65% cpu 1:14.72 total
❯ rm -rf ~/Library/Application\ Support/com.apple.containerization && time bin/cctl_new images pull docker.io/library/node:latest
image pulled
bin/cctl_new images pull docker.io/library/node:latest 42.28s user 6.65s system 69% cpu 1:09.93 total
❯ rm -rf ~/Library/Application\ Support/com.apple.containerization && time bin/cctl_new images pull ghcr.io/norio-nomura/swift_discord_bot:main
image pulled
bin/cctl_new images pull ghcr.io/norio-nomura/swift_discord_bot:main 103.83s user 18.71s system 81% cpu 2:30.02 total
❯ rm -rf ~/Library/Application\ Support/com.apple.containerization && time bin/cctl_old images pull ghcr.io/norio-nomura/swift_discord_bot:main
image pulled
bin/cctl_old images pull ghcr.io/norio-nomura/swift_discord_bot:main 120.79s user 20.70s system 85% cpu 2:45.26 total
```
</details>
<details>
<summary>patch for download only</summary>
```diff
diff --git a/Sources/cctl/ImageCommand.swift b/Sources/cctl/ImageCommand.swift
index 84c5218..4aa4bb8 100644
--- a/Sources/cctl/ImageCommand.swift
+++ b/Sources/cctl/ImageCommand.swift
@@ -127,6 +127,7 @@ extension Application {
}
print("image pulled")
+ return
let tempDir = FileManager.default.uniqueTemporaryDirectory(create: true)
if let platform {
```
</details>
- The kqueue handlers for stdio didn't have logic to exit on zero byte
reads.
- The agent wasn't getting closed explicitly in .delete()
- In LinuxContainer we should call delete just for sanity, if for
nothing more than ensuring the agent vsock fd is closed.
Signed-off-by: Danny Canter <danny_canter@apple.com>
This is something I forgot to update ages ago. The field names, while
they do make sense, were different than the constructor names that
eventually set the fields themselves. This is kind of awkward, and I'd
much rather fix this now while it's much easier.
We'll need to update `container` after this
Signed-off-by: Danny Canter <danny_canter@apple.com>
This change adds documentation to quite a few existing public types that
didn't have a blurb before.
Additionally, this fixes a couple things that I think either didn't make
sense when going to document them:
- Rename ConnectionStream to VsockConnectionStream. This type only
functions for vsock connections.
- Deletes NsLock+Closure. This was not used anywhere.
- Rename ContainerizationOCI/Config.swift to ImageConfig.swift.
Signed-off-by: Danny Canter <danny_canter@apple.com>
Danny Canter