- Closes#1528.
- Several commands (`builder status`, `image list`,
`stats`, `system df`, `system status`) advertised
`--format yaml` and `--format toml` but only handled
`json`, and every other format fell through to the
table. With this PR, we now route them through one
shared renderer with an exhaustive switch over the
format enum, so a missing format would now be a
compile error, and not just fail silently.
- Since TOML has no top level array, TOML output
now wraps list payloads under an `items` key,
because otherwise it was returning nothing for lists.
- `stats` now prints one static result for machine
readable formats instead of opening its live table
view.
- `builder status` now returns an empty list for
json/yaml/toml when no builder is running, instead
of the unparseable "builder is not running" text.
The table view keeps the message.
- with `--quiet` and no builder it now exits 0 with
no output, earlier it exited non-zero.
Normalize JSON output for network and volume resources. (#1624)
- Reworks both JSON output presets to use sorted keys, ISO timestamps.
`compact` is used for `ls` output, and `pretty` is used for `inspect`.
- Extracts non-DRY option configuration into presets.
- Closes#1623.
## Type of Change
- [x] Bug fix
## Motivation and Context
Many CLI commands need to reference the system configurations for
`container`. Previously, CLI commands would try to load the system
configurations from the default application root location, regardless of
if `container` had been started with a custom application root location.
This PR fixes that issue by having each CLI command ping the APIServer's
health check service to get the correct app root path.
Closes https://github.com/apple/container/issues/1576
## Testing
- [x] Tested locally
Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
- Closes#1557.
- Replaces `executableURL` utility function for getting app executable
path with `executablePath`.
- Adds `FilePath.resolvingSymlinks()` extension.
- Also converts for FilePath for types in `ContainerVersion` target.
- Closes#1581.
- This adds stop signal support to the cli. The priority is:
1. If an explicit stop signal is passed on the cli use this.
2. If not, check if there is a stop signal in the image config.
3. Finally, use the default (TERM).
- This adds support for reading configurations from
a three layer hierarchy:
1. User provided TOML
2. Install root TOML
3. Code defaults
- We add some code to support plugin configurations
via the ConfigurationLoader. Each plugin can provide
a struct with an accompanying id that gets used to
parse the scoped section of the TOML.
- Discussion topic #1336.
- This change migrates away from using `UserDefaults`,
instead providing a TOML configuration mechanism for
user configurable settings. All existing system property
settings keys are supported in the new configuration
file. However, users will have to migrate any settings
they have configured in the `UserDefaults` into TOML
for these settings to take effect.
- Breaking changes:
* `container system property get` is removed in favor of
users directly utilizing `container system property list --format toml | jq<>`.
* `container system property set` is removed since the TOML
configuration is effectively immutable during the lifetime of the
`container` daemon. Uses can edit the TOML they have in their home
directory, however no changes will take effect until the daemon is
restarted via `container system stop && container system start`
* `container system property list --format table` is removed as
generating tabular format is non-trivial and the new TOML format is
intended to be human readable
- Closes#1450.
- If we require that bundle-packaged installs must populate a
Unix-structured `bin` and `libexec`, we don't need the code that
searches the app bundle install root. As it stands, we enumerate both
the app bundle and the Linux paths, meaning we're looking at the same
plugins (in different paths) twice.
## Motivation and Context
This PR consolidates duplicated list-output formatting across the CLI
into shared rendering infrastructure.
Currently, each list command has its own copy of the same
json/quiet/table branching. This PR pulls that into shared rendering
infrastructure in `ContainerCommands`:
- `ListDisplayable` protocol for table + quiet output
- `renderJSON`, `renderTable`, `renderList` as pure functions that
return strings
- `emit()` as the single stdout boundary (no-ops on empty strings to
avoid blank-line regressions)
- `JSONOptions` so all JSON encoding goes through one path, including
volume inspect's pretty + ISO 8601 case
JSON encoding remains separate from display formatting: each command
still chooses its own JSON model, while `ListDisplayable` is used only
for table and quiet output. `ImageList` remains the intentional
exception for quiet mode so it can avoid unnecessary async work.
This change also replaces inline `JSONEncoder` usage with `renderJSON`,
removes the old `Codable+JSON.swift` helper, and moves `TableOutput` and
`ListFormat` into `ContainerCommands`. It also adds unit tests for the
shared rendering helpers and expands integration coverage for image,
network, and registry list formatting.
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [ ] Added/updated docs
Same as we'd done with kill.
```
➜ container git:(stop-kill-throw-error) ✗ ./bin/container stop foo bar baz
Warning! Running debug build. Performance may be degraded.
Error: internalError: "failed to stop container" (cause: "notFound: "container with ID baz not found"")
internalError: "failed to stop container" (cause: "notFound: "container with ID bar not found"")
internalError: "failed to stop container" (cause: "notFound: "container with ID foo not found"")
➜ container git:(stop-kill-throw-error) ✗ ./bin/container kill foo bar baz
Warning! Running debug build. Performance may be degraded.
Error: internalError: "failed to kill container" (cause: "notFound: "container with ID foo not found"")
internalError: "failed to kill container" (cause: "notFound: "container with ID bar not found"")
internalError: "failed to kill container" (cause: "notFound: "container with ID baz not found"")
➜ container git:(stop-kill-throw-error) ✗ ./bin/container delete foo bar baz
Warning! Running debug build. Performance may be degraded.
Error: internalError: "failed to delete container" (cause: "notFound: "container with ID foo not found"")
internalError: "failed to delete container" (cause: "notFound: "container with ID baz not found"")
internalError: "failed to delete container" (cause: "notFound: "container with ID bar not found"")
```
- Closes#1236.
- As with other resources, default to tabular output, with `--format
json` alternative.
- Also changes common workflow to collect logs even when test step times
out.
- Adds a a `--log-root` option to `swift system start`, propagating the
value as `CONTAINER_LOG_ROOT` to services for logging to files instead
of the OS log facility. This is not a "production" capability as it
neither merges nor rotates logs.
- Currently we don't collect logs on CI builds, and we don't have
permission to run the `log` command there. The PR adds `--log-root` to
the CI test phase, archives the results, and uploads the archive as an
artifact.
- Use FilePath from swift-system for the log root. Foundation URL is a
bit of a footgun for filesystem paths, so unless we identify a
showstopper, we should incrementally transition to this type everywhere
except where we really need network URLs.
- Output the hostname of the CI runner at the start of the test phase so
we can identify runner-specific issues where they exist.
- Fix formatting for log messages with multiple metadata items, and fix
unstructured messages on instances that weren't found using `grep -r
'log\.' Sources`.
- Adds command reference documentation for `--log-root`.
- Closes#1206.
- Closes#1185.
- Closes#507.
- Addresses existing log messages for #642.
- Nondeterministic CI errors are resulting from very slow launch times
for the first runtime helper, which causes ContainersService to be
locked for longer than our 20 sec timeout. Bumping the timeout to 60
seconds addresses this case for now.
- Since many log messages needed to be changed to troubleshoot the
issue, updated all log messages to use structured logging, and
implemented consistent entry/exit logging for all service operations.
- Added logging for ContainerService lock acquisition to help with
finding root cause for the slow service startup.
- Plumbed the `--debug` flag on both `container system start` and
`container system logs` so that the flag is actually useful.
- Updated the `install-init.sh` script so that can install in a custom
app root directory.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
- CI build is failing because runners don't have an /etc/resolver
directory, causing the directory monitor to fail. This occurs while the
install-kernel make target is running, so it appears that kernel
download is failing when it's just that the API server is dying. Created
#1207 for the issue.
- Fixing the initial scan for the directory just moves the problem to
the filesystem watch loop, likely because we're not testing the result
of `open()` for errors.
- Right now the priority is getting CI running and PRs merged, so just
commenting out the realhost DNS server setup.
- Also seeing that under some conditions it can take quite a while for
launchd to start the helper for the default network (8 seconds or more).
With the 10 second health check timeout after API server registration,
this means that some CI runs can exhibit this failure mode. Added a
`--timeout` option to SystemStart and set a 60 second timeout for
install-kernel and integration Makefile targets.
- Fixed a bug where `--debug` was being placed in the wrong location in
the api server startup args.
- Disabled all network CLI tests due to container bootstrap errors when
trying to run the container immediately after creating the network. The
slow network helper launch could be the reason behind the failures that
drove us to serialize these tests. Filed #1206 for this issue.
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
This is not intended to be used to support `--filter` or similar on the
CLIs list yet, it's solely to clean up our rather awkward use of
`ContainerClient.list()` today in the CLI. The list RPC simply returns
all of the containers we have created. Because of this, for a LOT of our
commands we filter to what we need client side, which feels like a
waste.. This change introduces a filter struct that we can provide an
array of container IDs, labels, and the status of the containers to
filter the `list()` output from.
This additionally, because it was killing (pun not intended) me and I
was already having to change this area for the `list()` additions,
changes container kill slightly to return an error if you try and kill a
container that doesn't exist.
Instead of logging errors, and then additionally throwing an error
stating what containers couldn't be stopped/killed/deleted, let's just
concatenate the errors and throw the single error.
ClientContainer was honestly extremely awkward. It could only be created
by passing either a ContainerConfiguration, or a Snapshot that had to be
obtained from calling a static method on the type itself. The type also
did not store a connection, so every single method got a new xpc
connection to the APIServer. This change aims to rework this type to be
just a generic client, that is *not* a client for one specific
container, but for any.
- Rename to ContainerClient
- Have list() return [ContainerSnapshot]
- Create a connection in the constructor
- Change all the callsites to use the new API
- Small, somewhat related, change to logs API in the APIServer. Now that
we don't need to call get() to grab a client anymore which was typically
what did "does this container exist" logic and gave a nice error
message, I added a small check in the APIServer to see if the container
exists and return mostly the same error message.
- Closes#346.
- This PR enables connecting host's localhost ports from
containers.
- It adds an option `--localhost <localhost>` to DNS
create command, after which the packets heading
ip address in container are redirected to localhost in
host machine. Packet filter rule is added and deleted
along with the creation and deletion of localhost domain.
- Facilitates #507, #642.
- Plumb logger with logger option access into every command.
- StderrLogHandler allows logging to be enhanced for ANSI color mode for
log levels, JSONL log output, replacing `--debug` with `--level level`.
- Global logger is now just an Application.swift fileprivate only used
for initial args processing.
- Log with timestamp at debug and trace level.
- Metadata output is crude at present; we can refine this in a
follow-up.
- Closes#461.
- Extract core types into ContainerResources target.
- Extract ContainerNetworkServiceClient from ContainerNetworkService.
- Relocate sandbox client from ContainerClient to
ContainerSandboxServiceClient.
- Relocate ContainerClient to ContainerAPIServiceClient.
- Common structure from services and clients under Source/Services.
Updated project hierarchy:
```
Sources/CAuditToken - audit token access wrapper
Sources/CLI - CLI executable
Sources/ContainerBuild - builder
Sources/ContainerCommands - CLI command implementations
Sources/ContainerLog - logging helpers
Sources/ContainerPersistence - persistent data and system property helpers
Sources/ContainerPlugin - plugin system
Sources/ContainerResource - resource (container, image, volume, network) types
Sources/ContainerVersion - version helpers
Sources/ContainerXPC - XPC helpers
Sources/CVersion - injected project version
Sources/DNSServer - container DNS resolver
Sources/Helpers - service executables
Sources/Services/*/Client - service clients
Sources/Services/*/Server - service implementations
Sources/SocketForwarder - port forwarding
Sources/TerminalProgress - progress bar
```
## Type of Change
- [ ] Bug fix
- [ ] New feature
- [x] Breaking change
- [ ] Documentation update
## Motivation and Context
The ContainerClient library was a bit of a grab bag. This refactor
applies a more sensible project and library structure for resource data
types, services, and clients.
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [ ] Added/updated docs
- Part of work for #460.
- Enable set/get of IPv6 network prefix in ReservedVmnetNetwork.
- Show IPv6 prefix in `network list` full output.
- Option for setting IPv6 prefix when creating a network.
- System property for default IPv6 prefix.
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [x] Documentation update
## Motivation and Context
See #460.
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [x] Added/updated docs
- Part of work for #460.
- With CZ release 0.17.0, the IP and CIDR address
types changed from String to IPv4Address and
CIDRv4, respectively. This PR applies the corresponding
adaptations to container.
- Closes#884.
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
This PR implements the `container system df` command to display disk
usage statistics for images, containers, and volumes, along with their
total count, active count, size, and reclaimable space for each resource
type.
Active resources are determined by container mount references and
running state, while reclaimable space is calculated from inactive or
stopped resources.
Example output:
```
~/container ❯ container system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 4 3 4.42 GB 516.5 MB (11%)
Containers 4 2 2.69 GB 1.51 GB (56%)
Local Volumes 3 2 208.5 MB 66.2 MB (32%)
```
I'll have some follow-on PRs that will add `-v/--verbose` flag for
detailed per-resource information, `--filter` flag for filtering output
by resource type, and a `--debug` flag for debug statistics like block
usage, clone counts etc.
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [ ] Added/updated docs
- Part of #653.
- No need for `defaultCommand` since ContainerService startup will write
a new launchd plist with the `start` subcommand included.
## Type of Change
- [ ] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Allows us to add commands to support post-installation, migration, etc.
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
Today if you do system stop twice (or if you forgot the server isn't
running) you get bombarded with annoying (and confusing) XPC interrupted
messages. We should skip the container stop and list dance if we can't
talk to the APIServer with a simple ping.
Raj