- When container is not running, the runtime helper
traverses the container's root fs and writes it to the
specified tar archive or stdout.
- When the container is running, the helper performs
the same operation but wraps it in freeze/thaw
to ensure data integrity for the resulting archive.
Closes https://github.com/apple/container/issues/1687
The default kernel archive is downloaded from a remote release URL
during first-run setup and via `container system kernel set
--recommended`. Previously, the archive contents were not verified after
download, so integrity depended on HTTPS and the release artifact
remaining unchanged.
This change adds digest verification for kernel archives. The
recommended/default kernel now has pinned digest metadata using an
algorithm-prefixed value such as `sha256:<hex>`. `container system
kernel set --tar` accepts `--digest`; remote tar URLs require it, and
local tar archives can also be verified before unpacking and
installation.
The system config also supports `kernel.digest`, and a custom
`kernel.url` must provide a digest for that archive.
- Closes#1812.
- The network plugin is the source of truth for the variant, if any,
that applies to the network. Resolving a missing variant configuration
option in the API server can create a situation where the variant the
runtime uses for interface selection is incorrect.
- Adds serial suites trait to tests to see whether it helps current CI
issues.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Fixes a flaw in our interface strategy logic.
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
`container` runs each workload in an ephemeral VM, so there's no
built-in way to keep a persistent Linux environment you can log into and
work in. `container machine` adds one.
A container machine is a lightweight, persistent, and integrated Linux
environments that feel like an extension of your Mac, created from
standard OCI images with a familiar UX. The login user matches your host
account with passwordless `sudo`, your home directory is mounted inside
the VM, and each machine keeps its filesystem and runs the image's own
init system (such as`systemd` or `openrc`).
```bash
container machine create alpine:3.22 --name my-machine
container machine run -n my-machine # interactive shell
container machine set -n my-machine cpus=4 memory=8G
```
Subcommands: `create`, `run`, `list` (`ls`), `inspect`, `set`,
`set-default`, `logs`, `stop`, `delete` (`rm`); `m` aliases `machine`.
Docs added to `docs/command-reference.md` (Machine Management) and
`docs/how-to.md` ("Use container machines").
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [x] Added/updated docs
Signed-off-by: Raj Aryan Singh <rajaryan_singh@apple.com>
Co-authored-by: Jaewon Hur <jaewon_hur@apple.com>
Co-authored-by: John Logan <john_logan@apple.com>
Co-authored-by: Michael Crosby <michael_crosby@apple.com>
Co-authored-by: Eric Ernst <eric_ernst@apple.com>
Co-authored-by: Danny Canter <danny_canter@apple.com>
- Closes#1647.
- `id` will become a system assigned (Docker-like) identifier
for the managed resource, and `configuration.name` is the
user-assigned name.
Today when we send a signal we don't wait for the container to exit, as
we don't know what signals the program will actually do anything with.
However, sigkill does not fit this mold, and we should wait for the
container to exit (or be removed for --rm containers).
Normalize JSON output for network and volume resources. (#1624)
- Reworks both JSON output presets to use sorted keys, ISO timestamps.
`compact` is used for `ls` output, and `pretty` is used for `inspect`.
- Extracts non-DRY option configuration into presets.
- Closes#1623.
- Closes#1526 and #1527.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
This PR fixes `system df` to report actual on-disk allocated bytes
(content blobs + snapshots) instead of summing per-image snapshot sizes.
Orphaned blobs are now included as reclaimable, and storage shared
across tags is no longer double counted. Also consolidates three
identical `calculateDirectorySize` implementations into a shared
`FileManager.allocatedSize(of:)` extension.
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [ ] Added/updated docs
- Refactor network model types: replace `NetworkState` enum and
phase-based NetworkStatus with a flat `NetworkStatus` struct.
- Simplify API server ↔ plugin protocol: plugin `status()` returns
runtime status only, API server owns configuration.
- `NetworksService` `list()`/`create()` now return `NetworkResource`
directly.
- Remove lifecycle phase checks and state machine guards throughout CLI
and API server.
- `variant` is plugin-specific, it's not a required property. This PR
replaces `NetworkPluginInfo` with a `plugin` name property on
`NetworkConfiguration` and an `options` list similar to that for
volumes.
- Moved `variant` to the option list.
- Part of #1404.
- Updates containerization to 0.33.2.
- Reorganizes network plugin targets into:
- `ContainerNetworkClient` - network plugin client and default types
- `ContainerNetworkServer` - separate protocols for `Network` which
manages the underlying virtual network, `NetworkService`, which takes a
network and implements the API, and an actor `NetworkHarness` that
marshals between the API and the XPC protocol. The service-harness
separation will help us ensure XPC protocol compatibility in both
directions as we evolve the plugin APIs.
- Removes `disableAllocator()` which is no longer used since #1545
switched over to using XPC connections between runtime and network
plugin instances to track whether a network has attached containers.
## Type of Change
- [x] New feature
## Motivation and Context
Related to https://github.com/apple/container/issues/1404.
This PR adds the initial work to have volume resources conform to
ManagedResource, in alignment with other resources such as networks (see
[here](https://github.com/apple/container/pull/1421)). Further work is
necessary to move the use of `VolumeResource` down to the APIServer (in
the VolumesService) and in the volume client. Volumes do not currently
have any plugin or runtime state, so that information is not included in
the `VolumeResource`, but could be added later if needed.
## Testing
- [x] Tested locally
Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
- Closes#1581.
- This adds stop signal support to the cli. The priority is:
1. If an explicit stop signal is passed on the cli use this.
2. If not, check if there is a stop signal in the image config.
3. Finally, use the default (TERM).
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [x] Documentation update
## Motivation and Context
Adds the `container copy` (aliased as `cp`) command to copy files
between a running container and the local filesystem.
I saw [#1023](https://github.com/apple/container/pull/1023) and the
feedback from @dcantah — the previous attempt relied on tar being
installed inside the container.
This implementation takes the recommended approach:
file transfers go through the guest agent via the existing
`copyIn`/`copyOut` methods on the core `Containerization`, with no
dependency on container tooling.
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [x] Added/updated docs
---------
Co-authored-by: jwhur <57657645+JaewonHur@users.noreply.github.com>
- This change adds two types to support opaque runtime
data passing through the APIServer:
- RuntimeConfiguration `data`
- RuntimeLinuxData type
- LinuxRuntimeData defines runtime specific information.
With this change, it will support `variant` only. The
optional RuntimeConfiguration `data` field encodes
runtime specific data to pass through the APIServer.
It is decoded as needed by the runtime.
- The idea is to eventually move all runtime specific data
into the `data` field so that the APIServer is only aware
of generic container information.
- Closes#1318.
- Closes#1378.
- Reduces the complexity and coupling for IP allocation.
- Runtimes connect to networks for the life of the running container.
The runtime shuts down on connection loss.
- Networks automatically deallocate a runtime's IP address and hostname
record on connection loss.
- Removes AllocatedAttachment as this is no longer necessary. The
`bootstrap()` XPC now takes a `NetworkBootstrapInfo` array which
parallels the attachments in the bundle config and provides the network
plugin attributes needed to create VM network interface configurations.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Simplify IP allocation and make deallocation more reliable.
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
- Part of #1318.
- Part of #1378.
- Removes network plugin `deallocate()`, and allocate takes an
`XPCServerSession` that registers an `onDisconnect` handler that
performs deallocation.
- ContainerService now tracks `networkSessions` for allocations.
- Discussion topic #1336.
- This change migrates away from using `UserDefaults`,
instead providing a TOML configuration mechanism for
user configurable settings. All existing system property
settings keys are supported in the new configuration
file. However, users will have to migrate any settings
they have configured in the `UserDefaults` into TOML
for these settings to take effect.
- Breaking changes:
* `container system property get` is removed in favor of
users directly utilizing `container system property list --format toml | jq<>`.
* `container system property set` is removed since the TOML
configuration is effectively immutable during the lifetime of the
`container` daemon. Uses can edit the TOML they have in their home
directory, however no changes will take effect until the daemon is
restarted via `container system stop && container system start`
* `container system property list --format table` is removed as
generating tabular format is non-trivial and the new TOML format is
intended to be human readable
- Part of #1404.
- Evolves API to use NetworkResource that conforms to ManagedResource.
- BREAKING CHANGE - compile time impact due to changes to Swift client
API signatures. No change to persistent data or API server XPC
protocols.
- Closes#1433.
- Renames `env` parameter to `dynamicEnv` to differentiate these
special-case environment variables from the standard container
environment.
- Use `[String: String]` instead of `[String: String]?`.
- Default to `[:]` when an down-revision client calls bootstrap without
supplying `dynamicEnv`.
This PR fixes#357, passing `SSH_AUTH_SOCK` env variable from current
terminal to the `SandboxService` so that the container can mount the
correct ssh auth socket. For that, it introduces `env` parameters to
`bootstrap` RPC of both `ContainersService` and `SandboxService`. This
parameter is used only for passing `SSH_AUTH_SOCK` now, but can be
extended to pass more runtime env variables.
This PR is a follow up PR of #1214.
## Type of Change
- [X] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Container run `--ssh` was inheriting `SSH_AUTH_SOCK` env variable from
launchd, not from current terminal.
## Testing
- [X] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
- Part of #1404.
- COMPATIBILITY: Breaking change to client API. XPC protocol and
persistent data remain fully compatibile.
- Renames `ClientNetwork` to `NetworkClient`, aligning with
`ContainerClient`.
- Client uses instance methods instead of utility functions, with an
`init(serviceIdentifier:)` for the XPC service name. The value is
currently unused.
- Closes#1359.
- Create a ResourceLabels type and extract the label validation from
NetworkConfiguration into the new type.
- Create a base AppError type that is compatible with structured logging
and delegates message presentation to the error receiver.
- Define LabelError over AppError for label validation.
- Slightly reworks NetworkConfiguration entity migration code in
NetworksService.
- Closes#1268.
- The types we were using weren't very usable with Swift 6 structured
concurrency.
- Implements just the subset of records that we use.
- Use notImplemented instead of formatError for unknown record types.
- Use pure actor for LocalhostDNSHandler now that we have sendable
types.
- Use DNSName as key for table lookups in LocalhostDNSHandler and
HostTableResolver.
- Utilize dot-suffixed domain names everywhere in the lookup chain.
- Huge thanks to @manojmahapatra and @katiewasnothere for their
diligent and patient review :D
This PR adds an option in `ContainerCreateOption` to override the root
filesystem of container. When `ContainerCreateOption.rootFsOverride` is
set, container uses that as the root fs instead of the one cloned from
image snapshot.
## Type of Change
- [ ] Bug fix
- [X] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
[Why is this change needed?]
## Testing
- [X] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
This PR reaps already running containers with auto remove flag on
container system start.
Container API server shutdown ideally should stop every running
container, and reap the containers with auto remove flag. However, crash
or unexpected exit of API server leaves the containers running. In such
case, next system start should stop running containers (i.e., launchd
services) and reap the containers with auto remove flag.
## Type of Change
- [X] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Preserve the system to be in a consistent state after unexpected
failure.
## Testing
- [X] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
---------
Co-authored-by: Raj <realrajaryan@gmail.com>
- This reverts commit 1942a399e3.
- This approach to getting variant into the runtime helper was fine for
proof of concept work but we need a persistent runtime config file so
that the variant is available for `container start`.
- Adds plugin variant selection to bootstrap() on the Container API
service in such a way that we can revert the change soon without
compatibility issues when we work out a more permanent approach, and
requires no persistent data migration.
- Restore lexical ordering on ManagementFlags (except `--runtime`, will
take care of that next PR).
- Add a couple plugin loader tests to improve coverage.
- Closes#1236.
- As with other resources, default to tabular output, with `--format
json` alternative.
- Also changes common workflow to collect logs even when test step times
out.
- Adds a a `--log-root` option to `swift system start`, propagating the
value as `CONTAINER_LOG_ROOT` to services for logging to files instead
of the OS log facility. This is not a "production" capability as it
neither merges nor rotates logs.
- Currently we don't collect logs on CI builds, and we don't have
permission to run the `log` command there. The PR adds `--log-root` to
the CI test phase, archives the results, and uploads the archive as an
artifact.
- Use FilePath from swift-system for the log root. Foundation URL is a
bit of a footgun for filesystem paths, so unless we identify a
showstopper, we should incrementally transition to this type everywhere
except where we really need network URLs.
- Output the hostname of the CI runner at the start of the test phase so
we can identify runner-specific issues where they exist.
- Fix formatting for log messages with multiple metadata items, and fix
unstructured messages on instances that weren't found using `grep -r
'log\.' Sources`.
- Adds command reference documentation for `--log-root`.
- Closes#1206.
- Closes#1185.
- Closes#507.
- Addresses existing log messages for #642.
- Nondeterministic CI errors are resulting from very slow launch times
for the first runtime helper, which causes ContainersService to be
locked for longer than our 20 sec timeout. Bumping the timeout to 60
seconds addresses this case for now.
- Since many log messages needed to be changed to troubleshoot the
issue, updated all log messages to use structured logging, and
implemented consistent entry/exit logging for all service operations.
- Added logging for ContainerService lock acquisition to help with
finding root cause for the slow service startup.
- Plumbed the `--debug` flag on both `container system start` and
`container system logs` so that the flag is actually useful.
- Updated the `install-init.sh` script so that can install in a custom
app root directory.
Closes#1202
Today it's possible to pass a memory amount that very easily will cause
the container's VM to not be able to boot. We should protect against
this to avoid weird hangs/error messages. I could be convinced that a
limit should be in Containerization as well, but I think having one in
the daemon is a decent idea regardless.
## Type of Change
- [x] New feature
- [x] Breaking change
## Motivation and Context
We want to be able to support using multiple network plugins during
`container`'s lifetime. This additionally means needing to pick an
interface strategy to interpret a network attachment based on what
network plugin was used to create that attachment. This PR will
potentially replace https://github.com/apple/container/pull/1081.
Followups:
- doc updates to include the ability to specify plugin in the network
creation cli
## Testing
- [x] Tested locally
- [x] Added/updated tests
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Closes#1046 -- Right now we're creating container bundles in
ContainersService. Move this to the SandboxService to make it easier to
support different container bundle types.
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [ ] Added/updated docs
This is not intended to be used to support `--filter` or similar on the
CLIs list yet, it's solely to clean up our rather awkward use of
`ContainerClient.list()` today in the CLI. The list RPC simply returns
all of the containers we have created. Because of this, for a LOT of our
commands we filter to what we need client side, which feels like a
waste.. This change introduces a filter struct that we can provide an
array of container IDs, labels, and the status of the containers to
filter the `list()` output from.
This additionally, because it was killing (pun not intended) me and I
was already having to change this area for the `list()` additions,
changes container kill slightly to return an error if you try and kill a
container that doesn't exist.
Co-authored-by: saehejkang <saehej.kang@gmail.com>
Co-authored-by: Anthony DePasquale <anthony@depasquale.org>
- Bump `containerization` to `0.25.0`
- Updates for parameter changes on containerization
registry access API.
- Updates for change to containerization
`cleanUpOrphanedBlobs` function.
ClientContainer was honestly extremely awkward. It could only be created
by passing either a ContainerConfiguration, or a Snapshot that had to be
obtained from calling a static method on the type itself. The type also
did not store a connection, so every single method got a new xpc
connection to the APIServer. This change aims to rework this type to be
just a generic client, that is *not* a client for one specific
container, but for any.
- Rename to ContainerClient
- Have list() return [ContainerSnapshot]
- Create a connection in the constructor
- Change all the callsites to use the new API
- Small, somewhat related, change to logs API in the APIServer. Now that
we don't need to call get() to grab a client anymore which was typically
what did "does this container exist" logic and gave a nice error
message, I added a small check in the APIServer to see if the container
exists and return mostly the same error message.
- Closes#1122.
- Adds placeholder ManagedResource and unit tests. Nothing is using
these yet.
- Adds system-defined resource labels for owning plugin and resource
role. The system discriminates the builtin network using role "builtin".
- Adds builtin role when creating builtin network at startup, and
ensures that a preexisting network with ID "default" gets updated with
the role label.
- Replace all network ID checks for "default" with the builtin role
check.
- Adds "builder" role to builder VM.
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Role and owner labels should make cross-cutting resource policy easier
to implement.
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [ ] Added/updated docs
- Fixed#892.
- By contrast with `rm`, `prune` should display
the amount of reclaimed storage, so added code
to retrieve it.
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
- Closes#977.
- Closes#1058.
- Prevents unexpected removal of containers on
bootstrapping and starting failures, by reorganizing
error handling for container `run`, `start`, and
`exec` so that error handling only unwinds that
which was done in the current scope.
- Relies on apple/containerization#495.
Kathryn Baldauf