Commit Graph

321 Commits

Author SHA1 Message Date
Santosh Bhavani f7bcb687fd Add --max-concurrent-downloads flag for parallel layer downloads (#716)
Adds `--max-concurrent-downloads` flag to `container image pull` for
configurable concurrent layer downloads.

Fixes #715
Depends on apple/containerization#311

**Usage**:
```bash
container image pull nginx:latest --max-concurrent-downloads 6
```

**Changes**:
- Add CLI flag (default: 3)
- Thread parameter through XPC stack
- Update to use forked containerization with configurable concurrency

**Performance**: ~1.2-1.3x faster pulls for multi-layer images with
higher concurrency

**Tests**: Included standalone tests verify concurrency behavior and
parameter flow

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-07 15:56:50 -03:00
J Logan 1e19a4d6e3 Updates CZ to 0.16.0. (#927) 0.7.0 2025-12-02 20:48:37 -03:00
J Logan 856d594938 Fix incorrect creationDate for networks. (#926)
- Fixes #925.
- When the API service enumerates creation dates, the configuration
value from the network helper is used instead of the persistent config
value, so the creation date winds up being the current time. The short
term fix is for the API server to return NetworkState values that
combine the API server's persistent network config with the network
status returned from the helper.
- The longer term refinement could be to define a uniform model for
resources where each resource type has Metadata (id/Identifiable,
creationDate, labels), and type-specific configuration and state/status.

## Type of Change
- [x] Bug fix
- [ ] New feature  
- [ ] Breaking change
- [ ] Documentation update

## Motivation and Context

See above.

## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
2025-12-02 18:57:57 -03:00
Saehej Kang 6f55360def [networks]: Add creationDate field (#791)
- Closes #665 
- Existing containers that don't have `creationDate` default
  to the epoch date.
2025-12-02 10:15:26 -03:00
Saehej Kang a5fd902893 [docs]: move callout regarding documentation revisions (#921)
- Closes #920
- Makes note about main branch documentation more discoverable.
2025-12-02 10:12:42 -03:00
Ronit Sabhaya f394d87b26 fix: connect to backend immediately on channel active for server-first protocols(#794) (#813)
- Closes #794 
- TCP port forwarding currently fails for server-first protocols (SMTP,
  FTP, SSH, PostgreSQL) because the backend connection is only established
  when the client sends data.
2025-12-01 19:52:57 -03:00
Saehej Kang f0eb13176c [docs]: add volume prune to command reference (#910)
## Type of Change
- [ ] Bug fix
- [ ] New feature  
- [ ] Breaking change
- [x] Documentation update

## Motivation and Context

Looks like this got accidentally deleted in #805

## Testing
- [ ] Tested locally
- [ ] Added/updated tests
- [x] Added/updated docs
2025-11-21 15:33:50 -08:00
Raj 1c0e9888f3 Fix container image prune to actually remove images, add -a flag support, and bump cz to 0.15.0 (#909)
- Fixes #901.

## Type of Change
- [x] Bug fix
- [x] New feature  
- [ ] Breaking change
- [ ] Documentation update

## Motivation and Context
Previously `container image prune` called `ImageStore.prune()` (renamed
to `cleanupOrphanedBlobs()` in cz 0.15.0) which only removed orphaned
content blobs and never actually removed images.

This PR fixes that behavior so `container image prune` removes dangling
images by default, and with `-a` removes all unused images, not just
dangling ones.

## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
2025-11-21 10:55:00 -08:00
Dmitry Kovba 7926c317f6 Build input file cannot be found: '.../CAuditToken.o' (#908)
Resolves a build error in Xcode:
> Build input file cannot be found: '.../CAuditToken.o'. Did you forget
to declare this file as an output of a script phase or custom build rule
which produces it?
2025-11-20 16:01:25 -08:00
Raj d327a50219 Add container system df command for disk usage reporting (#902)
- Closes #884. 

## Type of Change
- [ ] Bug fix
- [x] New feature  
- [ ] Breaking change
- [ ] Documentation update

## Motivation and Context
This PR implements the `container system df` command to display disk
usage statistics for images, containers, and volumes, along with their
total count, active count, size, and reclaimable space for each resource
type.

Active resources are determined by container mount references and
running state, while reclaimable space is calculated from inactive or
stopped resources.

Example output:
```
~/container ❯ container system df
TYPE           TOTAL  ACTIVE  SIZE      RECLAIMABLE
Images         4      3       4.42 GB   516.5 MB (11%)
Containers     4      2       2.69 GB   1.51 GB (56%)
Local Volumes  3      2       208.5 MB  66.2 MB (32%)
```

I'll have some follow-on PRs that will add `-v/--verbose` flag for
detailed per-resource information, `--filter` flag for filtering output
by resource type, and a `--debug` flag for debug statistics like block
usage, clone counts etc.

## Testing
- [x] Tested locally
- [x] Added/updated tests
- [ ] Added/updated docs
2025-11-20 09:19:00 -08:00
Ronit Sabhaya c76e0f4f84 Added the split method for labeler workflow (#788)
- This can basically Help teams know what changes are made in pr
- Added the split approach @katiewasnothere suggested using openssf
- Added additional feature for colors
- Currently this is for CLI only later we can add multiple
- Labeling flow
  - TRIGGER: Pull Request Opened (untrusted code)
  - READ-ONLY stage: just save job number to artifact
  - WRITE: load job number artifact, use GH labeler
    action with `labeler.yml` from repo
2025-11-19 20:17:35 -03:00
J Logan 87862d1faa Updates to CZ 0.14.0. (#903) 2025-11-19 18:55:23 -03:00
J Logan b2f5f3ff2f Adds client uid validation to XPC server. (#896)
- When a user performs an `su` the effective UID changes but the bootstrap
  mach port does not, so that if container is running as `alice` from a
  GUI login session, it's possible to `su bob` and continue running
  container. While this doesn't pose a significant security risk as it's
  necessary for Alice to know Bob's password and manually enter it with
  `su`, this change closes the loophole by validating that client UID from
  the caller's audit token matches that of the API server.
2025-11-19 00:41:56 -03:00
Danny Canter 266b135da2 Fix container stats build break (#898)
Seems `run` was changed to return a data blob variant of stdout, which
made the function have a tuple of size 4 instead of 3 now. The stats
tests still used the old 3 wide variant which broke the build.
2025-11-18 16:29:24 -08:00
Danny Canter cf0eba495e Implement container stats (#851)
Closes #824

This implements statistics gathering across the various components, but
ultimately this is for implementing a new CLI command: `container
stats`. This shows memory usage, cpu usage, network and block i/o and
the number of processes in the container. The new command can inspect
stats for 1-N containers and by default continuously updates in a `top`
like stream.

## Type of Change
- [ ] Bug fix
- [x] New feature  
- [ ] Breaking change
- [ ] Documentation update

## Testing
- [x] Tested locally
- [x] Added/updated tests
- [x] Added/updated docs
2025-11-18 14:10:05 -08:00
ChengHao Yang 2d07d8ff96 Fix TestCLICreateCommand failing test (#897)
Signed-off-by: ChengHao Yang
2025-11-18 15:47:08 -03:00
Saehej Kang 9b435536f0 [container-build]: Support inline Dockerfile from stdin (#827)
- Closes #727
2025-11-18 12:25:11 -03:00
J Logan 739f5e5a9c Import --publish checks and data representation. (#872)
- Closes #871.
- Simplify and improve port range checks.
- Add count field to `PublishPort` to keep config
  size small for large port ranges.
2025-11-18 10:02:44 -03:00
Saehej Kang a99cb4ad08 [image-save-load]: support for stdin/stdout (#734)
- Closes #559
- Facilitates easy transfer between container applications that can use OCI tarfiles.
2025-11-18 00:42:24 -03:00
Raj 88a473e08f Fix progress bar crash on cached image rebuilds (#894)
- Fixes #883.

Changes `BuildImageResolver` to use `fetch()` instead of `pull()`. 

`fetch()` checks the cache first and only pulls if needed, preventing
incorrect progress events (e.g., "1/1 KB" metadata checks) on cached
images that caused the progress bar width calculation to produce
negative/weird padding counts, triggering "Fatal error: Negative count
not allowed" crashes during rebuilds.
2025-11-17 15:55:50 -08:00
J Logan 87b5623ecd Fixes translation from Filesystem props to Mount runtimeOptions. (#877)
- Closes #876.
- Default to safer block device sync options to prevent fs
  damage when containers aren't cleanly shut down.
2025-11-17 13:13:06 -03:00
Saehej Kang e49a563912 [tests]: refactor run function for stdin/stdout support (#830)
When working on test cases for #827 and #734 the following issues
occurred
  1. There was no support to pass in `Data` for `stdin` (input pipe added
    + new argument for the data)
  2. Pipes were getting blocked during the `try process.run()` and
    `process.waitUntilExit()` (order of operations were fixed)
  3. We need the binary data for directly piping the `stdout` data to
    `stdin`, and converting the data to a `string`, returns `""` (added a
    new value to the tuple)

I opened a new PR here for this as I did not want to bloat the other PRs
with so many updates. At first I thought of creating a new function
named `runStdinStdout`, but that seemed redundant as it pretty much was
going to do the same thing as `run`.

## Testing
- [x] Tested locally
- [x] Added/updated tests
- [ ] Added/updated docs

---------

Co-authored-by: J Logan <john_logan@apple.com>
2025-11-14 21:01:31 -03:00
J Logan 9fbf84b04e Proxy refinements (thanks to @Lukasa). (#873)
- Lock-free UDP proxy forwarding by puttng front and back end on the
same thread.
- Cleans up some Swift language awkwardness in the proxies.

## Type of Change
- [ ] Bug fix
- [x] New feature  (optimization)
- [ ] Breaking change
- [ ] Documentation update

## Motivation and Context
Proxies worked but could be made lock-free.

## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
2025-11-14 00:15:52 -03:00
Raj 827b46c672 Show image download progress during builds (#850)
- Closes #710.

## Type of Change
- [ ] Bug fix
- [x] New feature  
- [ ] Breaking change
- [ ] Documentation update

## Motivation and Context
Fetching images during builds currently show just a 0.0s timer, and no
other info making builds appear hung. This adds a realtime download
progress bar with info, when pulling base images during builds.

<img width="1133" height="640" alt="Screenshot 2025-11-04 at 2 54 33 PM"
src="https://github.com/user-attachments/assets/200d8485-a956-4ce1-bbfe-d7406bd85774"
/>

## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
2025-11-12 15:08:22 -08:00
caztanj 8685bb2c23 Add support for publish port ranges (#801)
- Adding many `-p` arguments for each port gets a bit
  annoying. Adding an entire range of ports with one `-p`
  arguments is much nicer.
2025-11-12 19:15:59 -03:00
J Logan 936c916916 Actually resolve symlink when loading bundle Info.plist. (#864)
- #859 added the traversal necessary to load the app bundle but forgot
to resolve symlinks. This fix adds the resolution, making it possible to
get default system properties from an app bundle Info.plist even if the
user invokes a command from, for example, `/usr/local/bin/container`
which is a symlink to the actual install path of the bundle.
- Also fixes bugs where an incorrect executable path was supplied in
some calls.
- Breaking change: `CommandLine.executablePathUrl` extension moved from
ContainerPlugin to ContainerVersion.
2025-11-10 17:00:18 -03:00
Ryan Govostes 651c39cf95 Add --rosetta option for arm64 images (#846)
`container` automatically enables Rosetta for amd64 containers. This
change allows Rosetta to be enabled for arm64 containers by passing the
`--rosetta` flag. This allows native containers to benefit from being
able to execute the occasional amd64 executable in niche situations,
e.g., containers used for cross-compilation.

Resolves #391.

I considered adding a warning for passing `--rosetta` when it is not
necessary, but there wasn't obvious infrastructure for conveying the
2025-11-09 15:10:58 -08:00
Danny Canter 14f1df59bf CLI: Implement exec -d (#852)
Surprised we didn't have this already..
2025-11-07 12:46:00 -08:00
J Logan c88cd2ef4f Fix Info.plist system properties for symlinked container. (#859)
- Regular `Bundle.main.infoDictionary` doesn't work in this case.
- Load bundle using location presuming binary is under `Contents/MacOS`.
2025-11-07 09:38:53 -08:00
J Logan 384e3a12ff Use container-builder-shim 0.7.0, ensures use of Rosetta. (#858)
- Addresses slow cross-platform builds from #68.
- The shim wasn't doing everything needed to ensure the use of Rosetta for
  `container build`. The new shim adds an `--enable-qemu` option that
  controls whether `buildkit-qemu-emulator-x86_64 is available; when it is
  not available, buildkitd will attempt to build natively, meaning Rosetta
  will execute amd64 binaries.
2025-11-06 13:29:55 -08:00
Raj 56f6d734f3 UX: fix typos, grammar and consistency issues (#845)
Fixes several typos, grammatical errors, and consistency issues to make
the UX smoother :)
2025-11-04 13:35:29 -08:00
Danny Canter a627c82a03 Deps: Bump Containerization to 0.13.0 (#848)
Speeds up unpacks.
2025-11-04 12:33:08 -08:00
Raj 62721e7336 Clarify container list abstract to mention it shows running containers (#840)
Clarify container list abstract to mention it shows running containers
only be default.
2025-11-03 17:05:48 -08:00
Raj e1e016b784 Fix HTTPClient crash when download fails before shutdown (#837)
Currently, when `FileDownloader.downloadFile()` encounters an error
during `client.execute()` (e.g., XPC connection interruption or network
timeout), the `HTTPClient` instance is deallocated without calling
`shutdown()`, resulting in a crash:
`AsyncHTTPClient/HTTPClient.swift:187: Fatal error: Client not shut down
before the deinit.`

This caused to a control flow issue when attempting to download the
default kernel during `container system start` and, under certain
conditions (e.g., network latency), encountering the timeout error
`HTTPClientError.connectTimeout`.

To address this, `client.execute()` is now wrapped in a do/catch to
ensure `shutdown()` is always called on both success and error paths.
Also I've added an explicit timeout configuration (30s connect, no read
timeout) to better accommodate large file downloads.
2025-11-03 13:37:50 -08:00
Saehej Kang cd7c3a12c8 [options]: Replace --disable-progress-updates with --progress (none | ansi) (#808)
Part 1 of #641
2025-11-01 14:24:50 -07:00
Rado Kubiak | Rado x Tech 745cc1813e Fix container DNS resolution broken by AAAA/IPv6 NXDOMAIN handling (#786)
- In Alpine Linux containers (commonly used as Docker base images),
  standard DNS resolution is provided by **musl**, a lightweight C
  standard library (libc). Musl implements DNS lookups via
  `getaddrinfo()`, which queries AAAA (IPv6) records first.
- Problem: DNS did not work correctly **inside containers**. Any
  system command attempting to resolve hostnames
  (e.g., `ping dynamodb-admin`) **failed** when the DNS server
  responded NXDOMAIN for AAAA records, even if A (IPv4) records
  existed. Explicitly forcing IPv4 (`ping -4dynamodb-admin`) worked
  correctly, showing the issue is specific to musl’s IPv6-first behavior.
- Consequence: In IPv4-only environments, Alpine-based containers
  cannot resolve hostnames using standard tools or libraries.
  Applications relying on `getaddrinfo()` fail with ENOTFOUND,
  breaking networking and inter-container communication.
- Root cause:  Following RFC 8305 / RFC 6724, musl treats NXDOMAIN
  for AAAA as “hostname does not exist” and does not fallback to A
  (IPv4) records.
- Fix: The Apple Container DNS engine now behaves as follows:
  * If an **A record exists**, AAAA queries return **NOERROR with empty
    answer (NODATA)**.
  * If neither **A nor AAAA** exist, NXDOMAIN is returned.
  This ensures that Alpine-based containers in IPv4-only networks can
  correctly resolve hostnames inside containers without modifying
  container images or application code.
2025-10-30 17:03:17 -07:00
Raj 13a2f1a9e3 Update builder-shim to 0.6.3 for metadata only Dockerfile support (#825)
- Fixes #736.
- BuildKit returns nil ref for Dockerfiles containing only metadata
  (`ENV/ARG/LABEL`) directives without filesystem operations
  (`RUN/COPY/ADD`). Previously, this caused builds to fail with "no build
  directives" error.
- Builder-shim 0.6.3 [fixes this](https://github.com/apple/container-builder-shim/pull/47) by
  creating a minimal marker layer when ref is nil but image config is
  valid, satisfying OCI manifest requirements.
- Also, added some tests for this behavior.
2025-10-30 16:27:38 -07:00
Danish Singh Sethi c909cb3a27 Add --mac-address flag to set custom MAC addresses for containers (#753)
- Closes #752.
- Currently, there is no way to specify a custom MAC address for a
container's network interface and the MAC address is auto-generated by
the system.
- Use Cases
  - **Network Testing**: Developers testing network-dependent applications
that need predictable MAC addresses
  - **License Management**: Running containerized software with MAC-based
license keys
  - **Network Automation**: Scripts and tools that expect specific MAC
addresses for configuration
  - **Debugging**: Consistent MAC addresses across container restarts for
easier troubleshooting
2025-10-30 16:26:06 -07:00
Saehej Kang 7197b41de8 [build-command]: refactor --progress option (#810)
Relates to #808

This PR is a refactor to add the `ProgressType` enum for the
`--progress` option + fixing a typo in the command help.
2025-10-30 11:38:25 -07:00
Dmitry Kovba 375e83c405 Improvements to Protobuf.Makefile (#821)
- Sync `Protobuf.Makefile` with other projects.
- Also see https://github.com/apple/containerization/pull/356.
2025-10-30 07:55:22 -07:00
Dmitry Kovba 7e4eaddf30 Improvements to Makefile (#822) 2025-10-29 16:41:30 -07:00
Dmitry Kovba ea26cce50a Update documentation (#805)
Updates documentation according to the currently supported commands and
displayed help strings.
2025-10-29 09:04:24 -07:00
Saehej Kang 90a716500f [how-to-docs]: Add steps for installation of completion scripts (#799)
- I tested all steps locally and developers should be able to just
  copy/paste the commands.
2025-10-29 09:03:17 -07:00
J Logan a23bcf0061 Propagate host ssh socket permissions to relay socket in guest. (#811)
- Closes #580.
- Depends on apple/containerization#362.

## Type of Change
- [x] Bug fix
- [ ] New feature  
- [ ] Breaking change
- [ ] Documentation update

## Testing
- [x] Tested locally
- [ ] Added/updated tests
- [ ] Added/updated docs
0.6.0
2025-10-27 14:46:27 -07:00
Danny Canter 9f8a0fb87d Build: Fallback to Containerfile if Dockerfile not found (#812)
Closes #782 #99

If a Dockerfile can't be found in the context dir, check for the common
alternative Containerfile. This does not implement .containerignore
also, solely Containerfile for now. This also funnily enough fixes us
not checking for the Dockerfile IN the context directory.. woops.

## Type of Change
- [x] Bug fix
- [x] New feature  
- [ ] Breaking change
- [ ] Documentation update

## Testing
- [x] Tested locally
- [x] Added/updated tests
- [x] Added/updated docs
2025-10-27 08:28:29 -07:00
Dmitry Kovba f2b3cbdd73 Lowercase stdin, stdout, stderr (#807)
Lowercased stdin, stdout, stderr.
2025-10-25 06:55:45 -07:00
Dmitry Kovba 4944fe1641 Use a consistent capitalization in log and error messages (#806)
Updates the capitalization in log and error messages to be consistent.
2025-10-24 19:18:32 -07:00
J Logan 8f2e20de5a CZ 0.12.1 - pick up relay fixes, other goodies. (#804) 2025-10-24 14:52:39 -07:00
Danny Canter b4528924dc Dns: Fix delete error message (#803) 2025-10-24 14:04:47 -07:00
J Logan 5de195f720 Bump Containerization to 0.12.0. (#802)
- Include change to bootlog API.
2025-10-23 14:26:45 -07:00