mirror of
https://github.com/safishamsi/graphify.git
synced 2026-10-16 08:59:26 +00:00
Import the canonical Unicode aware id normalizer into dedup.py
Toward issue 3352: _id_prefixes reimplements id slugification with
its own ASCII only regex instead of the canonical normalize_id every
real extractor uses to mint an id, so it silently drops non Latin
characters. This adds the import with no behavior change yet; the
swap lands in the next commit.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qfdzgbA5KedGEjD1AayNh
(cherry picked from commit 237f02d135)
This commit is contained in:
1 parent
e4399cb9ca
commit
b5247b5a88
1 file changed
+1
@@ -12,6 +12,7 @@ from collections import defaultdict
|
||||
from pathlib import Path
|
||||
|
||||
from graphify._minhash import MinHash, MinHashLSH
|
||||
from graphify.ids import normalize_id
|
||||
from rapidfuzz.distance import DamerauLevenshtein, Jaro, JaroWinkler
|
||||
|
||||
|
||||
|
||||
Reference in new issue
Block a user