mirror of
https://github.com/safishamsi/graphify.git
synced 2026-10-16 08:59:26 +00:00
ayushcodes10
2226fd91b7
os._exit(1) terminates only the rebuild process itself, not any
ProcessPoolExecutor worker it spawned for a large corpus. It skips
every cleanup path, including the pool's own context-manager
shutdown, so a worker still running at the moment the watchdog fires
is orphaned, reparented to PID 1 on POSIX, and keeps going
unsupervised for as long as whatever it was doing takes. A worker
stuck in catastrophic regex backtracking, the exact shape #3341
fixed, was observed surviving 2.5 days that way on macOS.
extract.py already documents this risk for its own 1 worker special
case, but that only covers max_workers == 1, not a real multi worker
pool.
The watchdog fires on a separate timer thread with no reference to
the pool object, but multiprocessing.active_children() enumerates
every live worker process regardless of which thread asks, so the
fallback kills them before exiting itself. Kill, not terminate: a
worker stuck in a C level call like regex backtracking never gets a
chance to act on a signal a Python level handler would need to catch.
Fixes #3396.
(cherry picked from commit 319869e6c1)