- This fixes the LLVM coverage data not properly being emitted for XPC
services. It requires piping the `LLVM_PROFILE_FILE` environment
variable through to all the services and plugins. The variable itself
also required the "%c" formatter to ensure that it continuously emits
coverage data, otherwise when XPC services are killed via "bootout" they
do not emit coverage.
- Part of #1833.
- CLI progress and registry test migrations were inadventently reverted
by #1857.
- Migrate TestCLINoParallelCases to TestCLIImagePruneSerial and
TestCLINetworkPruneSerial.
- Clean up test selection patterns in Makefile.
- Remove all legacy CLITests files.
- Use swift-testing `withKnownIssue` to run but ignore failures on flaky
`testCreateNameLongestValid` and `testIsolatedNetwork`.
- Extracts a fixture helper for tests requiring a retry loop.
- Part of #1833.
- Replace old targets with new ones.
- Try increased parallelism after test tweaks in #1857.
- Exclude test files from coverage analysis.
- Part of #1833.
- Adds `ContainerFixture` with scoped resource lifecycle and cleanup in
place of implementation inheritance for test support functions. The
fixture also handles resource prefixing and uses a more ergonomic
`CommandResult` in place of a tuple for return values.
- `ImageWarmup` suite pre-pulls well-known images, and
`copyWarmupImage()` tags test-local refs, keeping the canonical image
store untouched.
- Three-phase `integration-new`: warmup, followed by concurrent tests
(managed by the swift test
`--experimental-maximum-parallelization-width` flag), followed by
serialized tests.
- `coverage-new` merges unit + integration-new profraw, replacing
`coverage` in CI as a migration progress indicator.
- Updates GH workflow so non-coverage invokes both the `integration` and
`integration-new` Makefile targets, while coverage runs invoke the
`coverage-new` target.
Fixes#1738
`container cp` fails when the host source path is relative (e.g.
`container cp file foo:/root/`), because `NSString.standardizingPath`
only canonicalizes paths but does not make them absolute. The unchanged
relative path is then interpreted as `/file` (root-absolute) by
`URL(fileURLWithPath:)` on the runtime side.
Fixed by resolving relative paths against the current working directory
before use, matching the pattern already used by `container export`,
`container image save`, and `container image load`.
The same fix was also applied to the copy-out destination path (line
68), which had the same issue.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
`container cp file foo:/root/` fails with `"copyIn: source not found
'/file'"` because the relative path `file` is never expanded to an
absolute path. Using `$PWD/file` works, but relative paths should work
too — every other command in the codebase handles this correctly.
## Testing
- [x] Tested locally — builds and all existing tests pass
- [ ] Added/updated tests
- [ ] Added/updated docs
---------
Co-authored-by: jwhur <57657645+JaewonHur@users.noreply.github.com>
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
`container` runs each workload in an ephemeral VM, so there's no
built-in way to keep a persistent Linux environment you can log into and
work in. `container machine` adds one.
A container machine is a lightweight, persistent, and integrated Linux
environments that feel like an extension of your Mac, created from
standard OCI images with a familiar UX. The login user matches your host
account with passwordless `sudo`, your home directory is mounted inside
the VM, and each machine keeps its filesystem and runs the image's own
init system (such as`systemd` or `openrc`).
```bash
container machine create alpine:3.22 --name my-machine
container machine run -n my-machine # interactive shell
container machine set -n my-machine cpus=4 memory=8G
```
Subcommands: `create`, `run`, `list` (`ls`), `inspect`, `set`,
`set-default`, `logs`, `stop`, `delete` (`rm`); `m` aliases `machine`.
Docs added to `docs/command-reference.md` (Machine Management) and
`docs/how-to.md` ("Use container machines").
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [x] Added/updated docs
Signed-off-by: Raj Aryan Singh <rajaryan_singh@apple.com>
Co-authored-by: Jaewon Hur <jaewon_hur@apple.com>
Co-authored-by: John Logan <john_logan@apple.com>
Co-authored-by: Michael Crosby <michael_crosby@apple.com>
Co-authored-by: Eric Ernst <eric_ernst@apple.com>
Co-authored-by: Danny Canter <danny_canter@apple.com>
- Fixes#1642.
- The `ensure-hawkeye-exists.sh` now actually
ensures that hawkeye is installed. If it is not
installed, the script informs that the installation
uses `curl | sh` and asks the user to confirm
before proceeding.
- Automated workflows can bypass the prompt
by invoking the script with the `-y`/`--auto-install`
option, or by setting the environment variable
`HAWKEYE_AUTO_INSTALL=1`.
- Export HAWKEYE_AUTO_INSTALL=1 in every
Git workflow job that runs make check, to
ensure license/format checks don't stall.
- Closes#1641.
- This PR replaces the existing hook pathname computation
for `make pre-commit` with `git rev-parse --git-path hooks/...`,
which resolves to the shared hooks directory in both the main
checkout and any worktree.
- Closes#1526 and #1527.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
This PR fixes `system df` to report actual on-disk allocated bytes
(content blobs + snapshots) instead of summing per-image snapshot sizes.
Orphaned blobs are now included as reclaimable, and storage shared
across tags is no longer double counted. Also consolidates three
identical `calculateDirectorySize` implementations into a shared
`FileManager.allocatedSize(of:)` extension.
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [ ] Added/updated docs
## Type of Change
- [x] New feature
## Motivation and Context
When developing, there are times when I want to run a specific set of
CLI tests. This PR allows users to set what integration tests they want
to run by setting the makefile variable `INTEGRATION_TEST_SUITES`.
Example usage:
```
% INTEGRATION_TEST_SUITES="TestCLIVolumes TestCLIAnonymousVolumes" make all integration
```
## Testing
- [x] Tested locally
Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
This utilizes swift test `--skip-build` in combination with a single big
swift test compilation step. By doing this we reduce duplicated swift
compilation between unit and integration.
## Type of Change
- [ ] Bug fix
- [X] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Coverage GitHub action is taking too long, so hopefully reducing
duplicated work decreases time.
## Testing
- [X] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
This extends the github actions to by default run the coverage
generation targets. The targets will generate the coverage artifacts,
which will then be used by a seprate github action to comment on the PR
the coverage numbers.
This is a sanity check for testing information for new features and
refactors alike. It will allow reviewers to quickly identify if there
are major issues with new tests, or large changes to coverage percentage
that indicate a problem.
This improves the integration test suite speed by running all
integration tests in a single `swift test` invocation. We maintain the
one `@Suite` at a time behavior by introducing a new test trait that
uses a single mutex to limit inter-suite parallelism.
This adds make targets that generate json, html, and plain text coverage
reports for unit tests, integration tests, and an aggregation of both
suites. These new targets can be run via `make coverage` (aggregate),
`make coverage-unit` (only unit tests), and `make coverage-integration`
(only integration tests).
## Type of Change
- [ ] Bug fix
- [X] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
This provides developers a means to measure test coverage for new and
old features alike, to ensure that testing for aforementinoed features
is adequate.
## Testing
- [X] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
This change migrates plugin config format away from using JSON to use
TOML.
- Addresses the plugin portion of discussion #1336
## Type of Change
- [ ] Bug fix
- [X] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
Context within https://github.com/apple/container/discussions/1336
> We're looking to improve the user experience and overall functionality
of setting defaults for container to better enable future use cases.
>
> Today container uses macOS's
[UserDefaults](https://developer.apple.com/documentation/foundation/userdefaults)
to configure settings needed at runtime. As mentioned in
https://github.com/apple/container/issues/608, UserDefaults may be
idiomatic for macOS, but they apply globally to all sessions and do not
handle representing complex, hierarchical data well. Additionally, we
currently have two ways of setting these defaults, either directly with
macOS's defaults command or through container system property.
>
> https://github.com/apple/container/issues/608 proposes moving to use
environment variables in place of UserDefaults. However, we do not
believe this is sufficient. Environment variables are not in a
consistent location, are not sourced from data, and also do not handle
representing complex, hierarchical data well.
As part of this migration to TOML for user settings we want to also move
plugin settings to use TOML so that we can have a singular file format
for `container` configurations.
## Testing
- [X] Tested locally
- [X] Added/updated tests
- [ ] Added/updated docs
Closes#1352
Containerization has had support for a bit, it was just never brought
over here. It's exposed on the CLI via the classic `--cap-add` and
`--cap-drop` UX.
- Closes#387.
- Override the swift-argument-parser `help` command
with a version that prints the reformatted help that
includes plugin information.
---------
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
- TestCLIStatus isn't in the integration test.
- Update the output to align with the current output.
- Follow up #1193
---------
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
- Part of #1295.
- Needed for #1342.
- If APP_ROOT is defined, remove everything but kernel from data
directory before starting integration test.
- Adds `--enable-kernel-install` for system start on integration test.
Same as we'd done with kill.
```
➜ container git:(stop-kill-throw-error) ✗ ./bin/container stop foo bar baz
Warning! Running debug build. Performance may be degraded.
Error: internalError: "failed to stop container" (cause: "notFound: "container with ID baz not found"")
internalError: "failed to stop container" (cause: "notFound: "container with ID bar not found"")
internalError: "failed to stop container" (cause: "notFound: "container with ID foo not found"")
➜ container git:(stop-kill-throw-error) ✗ ./bin/container kill foo bar baz
Warning! Running debug build. Performance may be degraded.
Error: internalError: "failed to kill container" (cause: "notFound: "container with ID foo not found"")
internalError: "failed to kill container" (cause: "notFound: "container with ID bar not found"")
internalError: "failed to kill container" (cause: "notFound: "container with ID baz not found"")
➜ container git:(stop-kill-throw-error) ✗ ./bin/container delete foo bar baz
Warning! Running debug build. Performance may be degraded.
Error: internalError: "failed to delete container" (cause: "notFound: "container with ID foo not found"")
internalError: "failed to delete container" (cause: "notFound: "container with ID baz not found"")
internalError: "failed to delete container" (cause: "notFound: "container with ID bar not found"")
```
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
`TestCLIVersion` isn't in the integration test; we should include this.
And update the output to align with the real output.
Follow up #911
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [ ] Added/updated docs
---------
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
- Adds a a `--log-root` option to `swift system start`, propagating the
value as `CONTAINER_LOG_ROOT` to services for logging to files instead
of the OS log facility. This is not a "production" capability as it
neither merges nor rotates logs.
- Currently we don't collect logs on CI builds, and we don't have
permission to run the `log` command there. The PR adds `--log-root` to
the CI test phase, archives the results, and uploads the archive as an
artifact.
- Use FilePath from swift-system for the log root. Foundation URL is a
bit of a footgun for filesystem paths, so unless we identify a
showstopper, we should incrementally transition to this type everywhere
except where we really need network URLs.
- Output the hostname of the CI runner at the start of the test phase so
we can identify runner-specific issues where they exist.
- Fix formatting for log messages with multiple metadata items, and fix
unstructured messages on instances that weren't found using `grep -r
'log\.' Sources`.
- Adds command reference documentation for `--log-root`.
- Closes#1206.
- Closes#1185.
- Closes#507.
- Addresses existing log messages for #642.
- Nondeterministic CI errors are resulting from very slow launch times
for the first runtime helper, which causes ContainersService to be
locked for longer than our 20 sec timeout. Bumping the timeout to 60
seconds addresses this case for now.
- Since many log messages needed to be changed to troubleshoot the
issue, updated all log messages to use structured logging, and
implemented consistent entry/exit logging for all service operations.
- Added logging for ContainerService lock acquisition to help with
finding root cause for the slow service startup.
- Plumbed the `--debug` flag on both `container system start` and
`container system logs` so that the flag is actually useful.
- Updated the `install-init.sh` script so that can install in a custom
app root directory.
## Type of Change
- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
- CI build is failing because runners don't have an /etc/resolver
directory, causing the directory monitor to fail. This occurs while the
install-kernel make target is running, so it appears that kernel
download is failing when it's just that the API server is dying. Created
#1207 for the issue.
- Fixing the initial scan for the directory just moves the problem to
the filesystem watch loop, likely because we're not testing the result
of `open()` for errors.
- Right now the priority is getting CI running and PRs merged, so just
commenting out the realhost DNS server setup.
- Also seeing that under some conditions it can take quite a while for
launchd to start the helper for the default network (8 seconds or more).
With the 10 second health check timeout after API server registration,
this means that some CI runs can exhibit this failure mode. Added a
`--timeout` option to SystemStart and set a 60 second timeout for
install-kernel and integration Makefile targets.
- Fixed a bug where `--debug` was being placed in the wrong location in
the api server startup args.
- Disabled all network CLI tests due to container bootstrap errors when
trying to run the container immediately after creating the network. The
slow network helper launch could be the reason behind the failures that
drove us to serialize these tests. Filed #1206 for this issue.
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
- Fixed#892.
- By contrast with `rm`, `prune` should display
the amount of reclaimed storage, so added code
to retrieve it.
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
- Closes#639.
- Adds swift format configuration that removes lint checks so we can use
`swift lint` to perform format-only tests.
- Adds `check` target that invokes format and header checks.
- Adds pre-commit script that runs `make check`.
- Adds `pre-commit` target that installs the check script as a
pre-commit hook.
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [x] Documentation update
## Motivation and Context
Avoids wasting time and commit rewrites.
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [x] Added/updated docs
- TestCLIRunCommand now run so many tests concurrently that the API
server gets swamped and tests randomly time out.
- The parallelism options on `swift test` only work for XCTest, not
swift-testing.
- Work around this while retaining some parallelism (good for stress
testing) by breaking the tests into two suites.
- It is common to have `gnu-tar` alongside other GNU tools
installed and aliased for compatibility reasons. However, this
breaks the current make build.
- Use BSD-only binaries (no GNU equivalents that are
commonly aliased), making the Makefile more portable.
Often times I'll be making a change that only touches the cli and I
don't feel like sitting through the potential song and dance of the
other components building/installing.
- Updates to `image prune` for consistency with how
other `prune` commands are done. Added missing
test cases as well for the command
- Relates to the discussion from #914
- Fixes#943.
- Use images other than alpine:3.20 for image concurrency test so as not
to interfere with tests using that image.
- Rename test files to match suite names.
Closes#824
This implements statistics gathering across the various components, but
ultimately this is for implementing a new CLI command: `container
stats`. This shows memory usage, cpu usage, network and block i/o and
the number of processes in the container. The new command can inspect
stats for 1-N containers and by default continuously updates in a `top`
like stream.
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [ ] Documentation update
## Testing
- [x] Tested locally
- [x] Added/updated tests
- [x] Added/updated docs
To facilitate ability to gather code coverage metrics without slowing
down our integration target, let's add a coverage target.
From this we can run `xcrun llvm-profdata merge` to get profdata and
gather overall code coverage metrics
## Type of Change
- [ ] Bug fix
- [x] New feature
- [ ] Breaking change
- [ ] Documentation update
## Motivation and Context
[Why is this change needed?]
## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
Signed-off-by: Eric Ernst <eric_ernst@apple.com>
Noah Thornton