mirror of
https://github.com/morluto/rea.git
synced 2026-10-26 00:29:27 +00:00
fb6785dcafe5d4ab59eaabf503801d8e8a9767fa
13
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 morluto
|
02987d8e8c |
test(hopper): bind string assertions to observed source extents
Remove OS-based assumptions about long and non-ASCII string recognition. Require the dedicated fragmented fixture, independently verify full source bytes, and reject gaps, overlaps, incomplete reads and corrupted decoding. Report unrecognized native strings explicitly and retain the actual procedure count. Verified through 14 verifier regressions and the complete macOS arm64 Hopper 6.6.0-demo CLI/MCP lane with cleanup. |
||
|
Ariel Fikru Avicennaandmorluto
|
c86b09a109 |
feat(ghidra): admit 32-bit big-endian PowerPC ELF (#1747)
* feat(ghidra): admit 32-bit big-endian PowerPC ELF Recognize EM_PPC as a provider-neutral `powerpc` family that keeps its ELF class, byte order, type and flags, and admit ELF32 big-endian ET_EXEC through a bounded Ghidra lane. ELF64, little-endian, relocatable/shared objects and EF_PPC_EMB are refused with a specific reason; Hopper and IDA do not gain PowerPC support, and Windows Ghidra P0 is unchanged. Ghidra's ELF loader selects PowerPC:BE:32:e500:default for a plain EM_PPC header, which misreads 750-family code. The lane binds PowerPC:BE:32:default/default in the profile (language_selection "admitted-lane-v1"), imports with it and checks the reported language. A language set through REA_GHIDRA_LANGUAGE_ID still replaces the lane default. Add a source-owned fixture, fixed-encoding probe, GNU readelf cross-check and the optional `verify:ghidra:ppc` real-provider lane, plus docs/ghidra-ppc.md. Refs #1741 * style(ghidra): format shared PowerPC profile integration * fix(ghidra): preserve caller-selected PowerPC languages --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
记得吃中饭
|
2995ec6910 |
fix(ghidra): resolve file offsets in decompile (#2101)
* fix(native): expose loaded image bases in overviews * test(ghidra): retain resources when verifier cleanup fails * docs(ghidra): spell out file-offset address mapping * fix(ci): separate and retry NativeAOT dependency restoration * fix(test): resolve CI script file URLs on Windows * fix(native): retain conflict observations and actionable CLI guidance * fix(fixtures): keep NativeAOT retry logs credential-safe * fix(fixtures): redact authenticated URLs at diagnostic creation * fix(ghidra): restrict source-coordinate observations to decompilation * fix(cli): identify conflicting file-offset selectors accurately * fix(ghidra): bind file offsets to admitted target sources * fix(ghidra): preserve resolved file-offset failure context * test(ghidra): verify ELF address and file-offset round trips * test(nativeaot): align diagnostic regressions with fixture host support |
||
|
YsopwnableandLuis Taffelli
|
141d2095f5 |
test(hopper): repair Linux demo setup and native verification (#2638)
* fix(verification): align Linux Hopper setup lane with current contracts * fix(verification): retain orphan checks on clean Linux runners * fix(verification): detect Hopper through alternate launcher paths * fix(verification): report identified Hopper leaks before proc errors * fix(verification): distinguish Hopper string recognition from byte fidelity * fix(verification): report full literal decoding coverage explicitly --------- Co-authored-by: Luis Taffelli <86675342+LuisTaffelli@users.noreply.github.com> |
||
|
The one -Niuandmorluto
|
0baad58e60 |
feat(ghidra): admit RISC-V ELF analysis through a pinned generic lane (#2151)
* feat(ghidra): admit RISC-V ELF analysis through a pinned generic lane Refs #2063. Recognize `EM_RISCV` (243) as a provider-neutral family that carries its own ELF facts, and bound the first Ghidra lane to little-endian `ET_EXEC` images that declare the double-float ABI, pinned to `RISCV:LE:32:default` or `RISCV:LE:64:default` with the `gcc` compiler specification. Ghidra 12.1.4 is not reliable without the pin: its `riscv.opinion` pins `variant="default"` for the 64-bit constraint and leaves the 32-bit constraint with the comment that the loader "must choose between 'default' and 'AndeStar' variants". Measured on the installed release tree, a source-owned generic RV32 ELF with no Andes declaration imports as `RISCV:LE:32:AndeStar_v5:gcc`, while RV64 resolves to `RISCV:LE:64:default:gcc`. The lane therefore states the language per width and refuses any caller-selected processor outside it instead of inheriting the header guess. Refusals name the declaration rather than falling back to a guessed profile: big-endian images (Ghidra 12.1.4 ships no big-endian RISC-V language), non-`ET_EXEC` images, `EF_RISCV_RVE`, `EF_RISCV_TSO`, and every float ABI other than double, which the pinned generic compiler specifications model. The raw e_flags bits stay in Evidence as declarations, not as instruction semantics. Family recognition now has one owner for the ELF-metadata families: `ELF_METADATA_ARCHITECTURES`, `isElfMetadataArchitecture`, and `HeaderArchitecture` replace ten `Exclude<BinaryArchitecture, "mips">` sites that would each have needed a second hand-edited member. `ghidraPinnedLanguage` is that owner for the import commitment: the session readback previously re-derived only the MIPS pin, so a profile committing any second pinned language failed with a language-mismatch error before the bridge started. Hopper and IDA refuse RISC-V by name, matching the MIPS wording. Caller-supplied raw-binary entry keeps its existing four-family list; this lane is ELF-header admission only. Verification on macOS 15.7.7 arm64, Node 24.18.0, Ghidra 12.1.4 PUBLIC and JDK 21.0.12.1, with the host decompiler built from the release sources for `os/mac_arm_64`: `npm run check` passes; `npm run test:fast` passes 396 test files; `npm run verify:test-discovery` discovers all 917 test files exactly once. Source-owned RV32 and RV64 double-float `ET_EXEC` fixtures complete `rea analyze --provider ghidra` and `rea decompile rea_riscv_entry`, committing `RISCV:LE:32:default` and `RISCV:LE:64:default` with `language_selection: provider-pinned-v1`; the soft-float RV64 shape is refused with the named float-ABI reason, and the full refusal matrix including the RV32 and width-mismatch shapes is covered by unit tests. No `verify:ghidra:riscv` lane script is included in this change; the runs above were performed manually against the installed engine. Fork pull requests receive no CI check runs, so the real-provider scope above is the evidence available for review. * fix(ghidra): bound the RISC-V lane by declared flags and document it Address review feedback on #2151. `e_flags` bits outside `EF_RISCV_RVC | EF_RISCV_FLOAT_ABI` are now refused with the offending value named, so a reserved or newer producer declaration cannot enter the verified generic lane by falling through the RVE, TSO and float-ABI checks. Admission reads `e_type` only, so the refusal text no longer calls `ET_EXEC` static: a fixed-address declaration does not establish static linkage, and `PT_INTERP` and dynamic dependencies are not inspected. The session records that boundary as a limitation rather than inferring a load model. Add `docs/ghidra-riscv.md` in the shape of the MIPS lane guide (admission boundary, refusal table, why the language is pinned per class, Evidence fields, and what the fixtures do not establish), and correct `docs/firmware-analysis.md`, which still told users that MIPS and RISC-V decompilation had not been added. Verification after the change: `npm run check` and `npm run docs:check` pass; the RISC-V module tests pass (17 lane cases plus 9 parser cases); on Ghidra 12.1.4 with JDK 21.0.12.1 the RV32 and RV64 double-float fixtures still analyze and decompile with `RISCV:LE:32:default` / `RISCV:LE:64:default` and `language_selection: provider-pinned-v1`, the soft-float fixture is still refused by name, and the new linkage limitation appears in the emitted Evidence. * fix(ghidra): align RISC-V pinning with admission Reuse the complete lane admission boundary when selecting a language, retain unsigned unsupported flag bits in diagnostics, and derive float ABI metadata from the source declaration. * test(ghidra): verify RISC-V CLI and MCP workflows Build source-owned RV32/RV64 fixtures with Clang and LLD, establish independent ELF and symbol facts with LLVM readelf, and verify real provider evidence, instruction bytes, calls, decompilation, schema projections, snapshot replay, refusal diagnostics, source immutability and cleanup. Verified on macOS ARM64 with Ghidra 12.1.4, JDK 21 and LLVM/LLD 23.1.2. Dynamic linkage, big-endian, Andes and RVE/TSO workflows remain unverified; ISA attributes are not inspected. --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
Aurelio Bernal
|
eaf88aca2a |
test(ghidra): require pending multi-HI16 coverage in the real MIPS relocatable lane (#1974)
* test(ghidra): require pending multi-HI16 coverage in the real MIPS relocatable lane Post-merge review on #1950: the relocation oracle exposes coverage.multiple_hi16, but the real-Ghidra lane never required it, and Clang's C output never produces a LO16 that resolves several pending HI16s. Ghidra's loader could therefore mishandle that stream while the lane still passed, although most GCC-built kernel modules checked contain it. The relocatable fixture adds an assembly block with two lui %hi(rea_mips_rel_carry); LLVM's relocation sort then leaves one HI16 pending until a later LO16 that resolves it together with another HI16, including the carry. The lane now requires multiple_hi16: false on the previous fixture, true for all three variants (LE, BE, soft-float module) on real Ghidra 12.1.4. * test(ghidra): make the MIPS multi-HI16 fixture independent of relocation sorting Review on #1974: the block used two %lo of rea_mips_rel_carry, a symbol with other %lo uses, so the relocation stream depended on the assembler's HI16/LO16 sort. Clang 21 happened to leave a HI16 pending; Clang 17 (per review) and GNU as 2.45 (checked here: HI@4 LO@8 HI@0 LO@10) pair each HI16 with its own LO16, so multiple_hi16 stays false and the lane fails. Use a dedicated symbol with two %hi and exactly one %lo. Any assembler that orders each HI16 before a matching LO16 must then emit both HI16s ahead of that single LO16: Clang 21 gives HI@0 HI@4 LO@8, GNU as 2.45 gives HI@4 HI@0 LO@8. The real-Ghidra lane passes for LE, BE and the soft-float module variant with multiple_hi16 true. |
||
|
Aurelio Bernalandmorluto
|
290d301ec7 |
feat(ghidra): admit relocatable MIPS32r2 ELF objects (#1950)
* feat(ghidra): admit soft-float MIPS32r2 ELF with integer-register float storage Soft-float o32 executables (Linux kernel builds, bare-metal firmware) were refused because their ABI flags record declares no CPR1 and FP ABI 3. Ghidra's MIPS32 compiler specifications model only hard-float o32, so typed float/double signatures (DWARF) would be bound to $f12/$f14/$f0 and the decompiler loses the real dataflow: `return sel ? a : b` decompiled to `return in_f0;`, and callers passed phantom FPU inputs. Admit ELF32 ET_EXEC o32 MIPS32r2 targets whose ABI record declares soft-float with no CPR1, and commit `mips_soft_float_storage: o32-integer-registers-v1` in their analysis profile only (hard-float profile digests are unchanged). ReaGhidraMipsSoftFloat.java runs after auto-analysis and, for each function with a float/double formal type, either keeps the types with the storage Ghidra's o32 model assigns to a same-size integer (pair alignment, stack spill), or, for auto-parameters such as hidden struct-return pointers, swaps float types for same-size integers under dynamic storage. No function is left on hard-float storage. The report is written last; the provider client requires it exactly when the profile commits the policy and refuses the session before any operation otherwise. Verification: real Ghidra 12.1.4 lane (verify:ghidra:mips) now builds hard- and soft-float fixtures in both byte orders and checks exact parameter/return storage through inspect_native_api, caller constants, snapshot profile and Evidence outcome; test:fast, verify:package and an installed-tarball soft-float run pass. * fix(ghidra): model soft-float MIPS as a prototype for every typed callable Review feedback on #1949: rewriting storage per Function left external functions and indirect calls through typed function pointers on Ghidra's hard-float o32 prototype, which still read $f12/$f14/$f0 while the session reported success. A negative control confirms it: without the change, `(*rea_mips_soft_callback)(8.5f, 9.5f, sel)` decompiles to `(*rea_mips_soft_callback)(in_f12, in_f14, sel)`. ReaGhidraMipsSoftFloat.java now derives a `__rea_soft_float` prototype from the program's own default o32 model (encoded, float pentries and float-consuming rules removed, so byte-order-specific aggregate rules and the hidden struct-return pointer are kept), installs it as a specification extension, and assigns it with dynamic storage to every non-thunk function, external function and function-pointer definition whose formal types use float or double. Thunks follow their targets. Float types are kept everywhere, so the integer-retype fallback is removed. The script fails without writing its report if the derived prototype or any reassigned function still uses an FPU register. The policy becomes `o32-soft-float-prototype-v1` with a {functions, definitions} report. The soft-float fixture adds an indirect call through a typed pointer; the real-Ghidra lane requires its constants with no FPU inputs and the struct return's `float scale` in $a1. * test(ghidra): pin ODDSPREG acceptance for soft-float MIPS ABI records Review on #1949 suggested requiring flags1 == 0 for soft-float, since MIPS_AFL_FLAGS1_ODDSPREG names CPR1 registers. GCC 5.2 kernel modules, LLVM/Rust firmware and Clang -msoft-float objects all record flags1 = 1 with FP ABI soft-float: it is the MIPS32 o32 toolchain default and selects nothing without CPR1. Keep accepting it, say why at the check, and test that soft-float with ODDSPREG is admitted while any other general flag is refused. * feat(ghidra): admit relocatable MIPS32r2 ELF objects Linux MIPS kernel modules (.ko) and other ELF32 o32 MIPS32r2 ET_REL objects were refused by the Ghidra MIPS lane. Ghidra's ELF loader already places each allocated section and applies REL relocations; this admits ET_REL alongside ET_EXEC (ET_DYN and PRX stay refused) under its own support lane, `elf32-rel-o32-arch32r2-standard-v1`, so executable targets keep theirs. Relocatable sessions carry limitations: addresses are analysis coordinates, undefined symbols live in Ghidra's EXTERNAL block, and HI16 relocations report PARTIAL because Ghidra applies them with their paired LO16. Verification: new `verify:ghidra:mips-relocatable` real-Ghidra lane builds -c -G0 objects in both byte orders plus a soft-float -mlong-calls module variant. GNU readelf supplies sections, symbols and REL entries; every relocation in an allocated section must encode S + A in the loaded word (S from Ghidra placement, A from original bytes), with HI16/LO16 checked as the effective address each LO16 forms, and coverage of a shared HI16, a LO16 carry, external code/data and R_MIPS_32 is asserted. The executable lane, unit suites pass, and on 30 real .ko files 54,972 relocations matched (93 reference empty sections and are not checkable). * fix(ghidra): verify every MIPS HI16 and archive-typed soft-float externals Review feedback on #1950: - The relocation oracle kept one pending HI16, so a second HI16 before the LO16 that resolves both overwrote the first, and unrelated relocations cleared it; the earlier site's high half and carry were never compared with S + A. Every HI16 now waits for the next LO16 of its symbol, which checks all of them; later LO16s reuse those HI16s, and a HI16 left unresolved at a section end fails. On 30 real soft-float kernel modules, 24 contain several HI16s per LO16; all 54,972 placed relocations match. - External callees with float prototypes: the relocatable fixture now calls powf, whose prototype Ghidra's C library archive applies by name. Every variant must decompile `powf(1.5,2.5)` without FPU inputs, and the soft-float module variant must report that __rea_soft_float covered it (with the external step removed it decompiles as `powf(param_1,param_2)`). `-fno-builtin` keeps the call from being folded. Also updates the module variant's expected soft-float policy to `o32-soft-float-prototype-v1`. --------- Co-authored-by: morluto <76467478+morluto@users.noreply.github.com> |
||
|
Aurelio Bernal
|
5e219de441 |
feat(ghidra): admit soft-float MIPS32r2 ELF with integer-register float storage (#1949)
* feat(ghidra): admit soft-float MIPS32r2 ELF with integer-register float storage Soft-float o32 executables (Linux kernel builds, bare-metal firmware) were refused because their ABI flags record declares no CPR1 and FP ABI 3. Ghidra's MIPS32 compiler specifications model only hard-float o32, so typed float/double signatures (DWARF) would be bound to $f12/$f14/$f0 and the decompiler loses the real dataflow: `return sel ? a : b` decompiled to `return in_f0;`, and callers passed phantom FPU inputs. Admit ELF32 ET_EXEC o32 MIPS32r2 targets whose ABI record declares soft-float with no CPR1, and commit `mips_soft_float_storage: o32-integer-registers-v1` in their analysis profile only (hard-float profile digests are unchanged). ReaGhidraMipsSoftFloat.java runs after auto-analysis and, for each function with a float/double formal type, either keeps the types with the storage Ghidra's o32 model assigns to a same-size integer (pair alignment, stack spill), or, for auto-parameters such as hidden struct-return pointers, swaps float types for same-size integers under dynamic storage. No function is left on hard-float storage. The report is written last; the provider client requires it exactly when the profile commits the policy and refuses the session before any operation otherwise. Verification: real Ghidra 12.1.4 lane (verify:ghidra:mips) now builds hard- and soft-float fixtures in both byte orders and checks exact parameter/return storage through inspect_native_api, caller constants, snapshot profile and Evidence outcome; test:fast, verify:package and an installed-tarball soft-float run pass. * fix(ghidra): model soft-float MIPS as a prototype for every typed callable Review feedback on #1949: rewriting storage per Function left external functions and indirect calls through typed function pointers on Ghidra's hard-float o32 prototype, which still read $f12/$f14/$f0 while the session reported success. A negative control confirms it: without the change, `(*rea_mips_soft_callback)(8.5f, 9.5f, sel)` decompiles to `(*rea_mips_soft_callback)(in_f12, in_f14, sel)`. ReaGhidraMipsSoftFloat.java now derives a `__rea_soft_float` prototype from the program's own default o32 model (encoded, float pentries and float-consuming rules removed, so byte-order-specific aggregate rules and the hidden struct-return pointer are kept), installs it as a specification extension, and assigns it with dynamic storage to every non-thunk function, external function and function-pointer definition whose formal types use float or double. Thunks follow their targets. Float types are kept everywhere, so the integer-retype fallback is removed. The script fails without writing its report if the derived prototype or any reassigned function still uses an FPU register. The policy becomes `o32-soft-float-prototype-v1` with a {functions, definitions} report. The soft-float fixture adds an indirect call through a typed pointer; the real-Ghidra lane requires its constants with no FPU inputs and the struct return's `float scale` in $a1. * test(ghidra): pin ODDSPREG acceptance for soft-float MIPS ABI records Review on #1949 suggested requiring flags1 == 0 for soft-float, since MIPS_AFL_FLAGS1_ODDSPREG names CPR1 registers. GCC 5.2 kernel modules, LLVM/Rust firmware and Clang -msoft-float objects all record flags1 = 1 with FP ABI soft-float: it is the MIPS32 o32 toolchain default and selects nothing without CPR1. Keep accepting it, say why at the check, and test that soft-float with ODDSPREG is admitted while any other general flag is refused. |
||
|
Jay-Alexander Elliotandgithub-actions[bot]
|
bddddccdee |
feat(native): admit bounded MIPS ELF analysis through Ghidra (#1362)
* feat(native): admit bounded MIPS ELF analysis through Ghidra Retain MIPS ELF identity and interpretation through Evidence and snapshots. Keep provider admission bounded to ELF32 ET_EXEC o32 MIPS32r2, without implicitly widening Hopper, IDA or Windows P0 coverage. Add source-owned fixtures and an optional real-provider verifier; real Ghidra verification remains pending in this contribution environment. Refs #718 * fix(ghidra): bound MIPS floating-point mode admission Reject explicit EF_MIPS_FP64 and EF_MIPS_NAN2008 declarations until their semantics have a real-provider verification lane. Add PSP PRX/Allegrex, o64/EABI and floating-point-mode refusal regressions. Document that the header gate is not a full .MIPS.abiflags check and advance the profile lane identity to invalidate older interpretation bindings. Validation: 24 standalone checks against the transpiled production profile module passed; the two new flag cases failed before this change. Clang 17 and LLD compiled the source-owned fixture for both endian targets. The complete Vitest/typecheck suite and real Ghidra verification remain pending. Refs morluto/rea#718; PSP support remains follow-up morluto/rea#1330. * fix(native): validate MIPS ABI records before Ghidra admission Read bounded ELF32 PT_MIPS_ABIFLAGS and SHT_MIPS_ABIFLAGS records through the already-open target handle. Preserve declared ABI fields, reject conflicting/duplicate/truncated records, and keep missing records unknown. Wire inspection into target resolution before provider selection. Require compatible declarations for the deliberately narrow Ghidra lane and bind all ABI fields into profile identity (v3) for snapshot invalidation. Clang 17 produced eight source-owned fixtures spanning both byte orders and FPXX/FP32/FP64/soft-float modes with identical ELF header flags. The new reader and policy distinguish them. 24 focused regression test bodies passed after TypeScript transpilation using node:test in place of Vitest; strict isolated TypeScript checking of the reader and target/result types passed. These are limited checks, not full Vitest/project typechecking or real-Ghidra verification, which remain pending. Advances morluto/rea#718. PSP/Allegrex remains morluto/rea#1330 follow-up. * fix(native): repair MIPS integration boundaries and test resolver behavior Keep ELF targets out of Hopper Mach-O preparation and scope existing PE fixture types correctly. Move ABI tests into the filesystem boundary lane, use shared cleanup, add resolver regressions, apply the pinned formatter, and register the real MIPS verifier through the existing package-command pattern. Validated with repository-pinned Node/npm, check:ci, actual focused Vitest, discovery and docs:check. Real Ghidra conformance remains pending. * fix(test): honor caller-selected JDK in real MIPS verification * test(native): strengthen MIPS Ghidra conformance Cross-check the source-owned fixture with bounded GNU readelf inspection, compare production ELF/ABI interpretation and independent symbol addresses, and verify effective Ghidra language/compiler identity through existing public operations. Add a source-owned fixed-encoding probe for immediate and conditional branch checks plus global/string byte assertions. Add 16 assertion/producer-boundary regression cases without widening production admission, changing provider contracts, or introducing proprietary dependencies. Local focused suite: 99 passed, one existing root-permission test skipped. Typecheck, lint/module boundaries, formatting and 700-file discovery pass; full hosted checks and real Ghidra verification remain pending for this checkpoint. * fix(ghidra): preserve native-tool architecture checks after MIPS integration The conflict-free upstream merge failed TypeScript compilation because includes on the union of MIPS-only and other architecture arrays requires never. Use equality-based membership without casts or changing accepted native hosts. Extend the existing native-decompiler rejection table with a MIPS executable at a Linux x64 tool path; retain upstream FAT and unknown-header behavior. Hosted verification follows this commit; no success is claimed yet. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
Ysopwnable
|
33d5ae0b52 |
fix(hopper): reserve Linux demo application before launch (#1158)
* test(hopper): give locals oracle explicit stack storage * fix(hopper): reserve Linux demo application before launch * fix(hopper): serialize stale lease recovery |
||
|
morluto
|
7d7b3f961a | fix(hopper): preserve typed string values and verify real searches | ||
|
morluto
|
64717ad7da | test(hopper): cover real boundary failures and CLI parity | ||
|
morluto
|
d648bcd2e4 | test: add source-built Hopper conformance fixtures |