rea/docs/python-packaged-applications.md

# Python packaged applications

REA statically inventories PyInstaller executables with an appended CArchive.
`inspect_artifact` and `extract_artifact` share the same reader through CLI and
MCP. They read archive bytes without launching the application. Bytecode
source decompilation, encrypted PYZ recovery, zipapp and PEX analysis are outside
this capability.

## Facts and focused queries

The root occurrence's `format_metadata` records the CArchive layout (2.0 or
2.1), archive and cookie offsets, table extent, entry count, raw Python version,
normalized major/minor when known, library name and encryption marker. Member
metadata retains the original name, table index, type, compression flag,
relative offset and known directory record extent. Logical paths use archive
path rules; the source name retains its spelling, including backslashes.
`relative_offset` is relative to the CArchive start for CArchive rows, or to
the decoded PYZ image for PYZ rows. A CArchive directory record starts at
`archive_start + toc_offset + directory_record_offset`; its size is
`directory_record_size`. PYZ row directory extents remain unknown.

Namespace packages remain directories. Their `relative_offset` and
`compressed_size` preserve any payload declared by older producers, while
`declared_size` is zero because no bytecode is materialized for a namespace.
Every declared span must end before the PYZ table of contents.
PYZ `compression_flag` is derived from storage: nonempty payloads are compressed,
including legacy namespaces; an empty namespace has no compressed payload.

CArchive type `o` entries are runtime options. They are metadata occurrences,
with no content artifact or extractable file. Their query paths are
`__pyinstaller__/runtime-options/<table index>`; the option text is
`format_metadata.source_name`. Repeated declarations retain distinct occurrence
identities. These paths do not claim output filesystem names.

Type `d` entries are multipackage dependencies. They are also metadata, under
`__pyinstaller__/dependencies/<table index>`, and never materialize an empty file.
The reader preserves the raw reference and splits its first colon into
`dependency_archive` and `dependency_member`. `dependency_status` remains
`unresolved`, or `invalid-reference` with unknown parts for malformed references.
`external_dependency_count` summarizes these declarations. REA does not open
another executable or claim its contents are available. A real file with the
same logical path as metadata remains independently extractable.
Both declaration types require empty payloads and no compression. Their query
paths do not participate in filesystem collision checks, directory ancestry or
source-map target lookup; their parent identifies the declaring container.
Complete authenticated inventory page sets preserve producer identities in any
merge order, including metadata/file aliases. Malformed TOC diagnostics name the
table index; original names remain available in the supplied artifact.
Path failures use bounded escaped previews and report the original length;
accepted paths and retained source names are not shortened.

Request `detail: "summary"` for a compact inspection, then query retained
Evidence with `inspect_analysis_view`. An inventory view can select
`collection: "occurrences"`, `entry_kinds: ["metadata"]`, exact logical paths or
occurrence IDs, requested `fields`, and `offset`/`limit`. Filters intersect;
identities, parent Evidence and coverage survive projection. Full inventory
Evidence remains available for export. A compact reply reduces returned context;
it still performs the complete member inventory and hashing work.
Focused views add calls and tool-schema costs. A complete small inventory can
cost less overall; prefer compact delivery when the full member graph would
overwhelm the answer.

```bash
rea inspect-artifact /absolute/path/hello --detail summary --evidence_output /absolute/new/evidence.json --json
rea extract-artifact /absolute/path/hello --json
```

## Decoding and evidence limits

Inventory hashes members individually through bounded streams. Stored PYZ
images use descriptor ranges; outer-compressed images stream to private files,
which the reader owns through cleanup and any cleanup retry. Extraction binds
the selected carrier to its inventoried digest and checks source identity and
state before and after reading. It does not retain expanded module bodies.
Opaque outer-compressed images reuse that verified private image when opened,
instead of inflating the carrier again. Direct source spans describe stored
CArchive bytes; decompressed content has no direct span even when its encoded
and decoded sizes coincide. Original offsets and compression remain metadata.

Only the PYZ header and marshal directory are loaded for listing. PYZ tables
have a 16 MiB budget; CArchive tables have a 32 MiB budget. Parsed facts have a
32 MiB serialized budget. Projected entries share a 64 MiB retained budget,
including metadata, bookkeeping and conservative reservations for namespace
objects and possible case-collision annotations at each path component. These
reservations happen before normalization or trie allocation.
Outer-compressed PYZ images have a separate per-reader capacity of 128 private
files, bounding retained inodes and image identities. The complete image count
is admitted before any temporary file is created; excess images return a
capacity failure. This is a resource policy, not a PyInstaller format limit.
Only a zero-byte read establishes EOF; short successful descriptor reads fill
the bounded range. Compressed members must consume their complete declared
input span; trailing junk and concatenated streams are refused.
Plaintext expansion has an aggregate archive-derived budget. Exceeding a budget
returns a resource failure.

Supported final Python bytecode magic values cover Python 3.7 through 3.14.
Supported marshal directory values are lists, tuples, strings, integers and
bytes. An unsupported magic or directory stays an opaque member with a
limitation and `pyz_status: "opaque"`; its module contents remain unknown.
Any opaque PYZ makes producer traversal coverage unknown, including retained
summaries and exhausted queries. Per-archive reasons remain on the opaque
occurrences; the coverage observation summarizes their count without repeating
every limitation. An empty result does not establish module absence.
Observed `pyz_python_magic` bytes remain available even when their Python version
is unknown.
The directory declares compressed module lengths, so uncompressed declared
sizes and source spans inside decoded data remain unknown.

PYZ modules receive a derived 16-byte `.pyc` header from their own supported
magic. CArchive bytecode receives that header only when every sibling PYZ magic
and the cookie's Python version agree. Otherwise it retains its original bytes
and `bytecode_header: "unknown"`. `bytecode_magic` and graph derivation edges
identify synthesized headers; member digests cover the resulting derived files.
These headers do not establish successful execution or decompilation.

The nonempty CArchive Python module `pyimod00_crypto_key` (`m`/`M`) is recorded
as an encryption marker; a same-named data file or script is not. Affected
PYZ blobs are reported without decoding; extraction refuses encrypted entries.
The raw PYZ container remains readable and hashable, including when its module
table is opaque. Its marker stays in metadata and opaque child coverage stays
unknown; extracting the container does not decrypt its payloads.
The marker also applies to nonempty legacy namespace payloads. Empty namespaces
have no payload to encrypt and retain a null encryption source.

Detection searches the last 16 MiB for a cookie, allowing a trailing macOS code
signature within that bound. Member offsets are relative to the archive start:
cookie position plus cookie size minus package length. Native bootloader
analysis remains available through the native tools.