Ej`dZddlmZddlZddlZddlZddlZddlmZm Z m Z m Z m Z ddl mZejeZej$dZdZdZd Zd Zd Zd&d Zd'd Zd(dZd)dZ d* d+dZdZeefZ d* d,dZ ddd d-dZ!d.dZ" d/dZ# d* d0dZ$ d* d1dZ%d*d2dZ&d3dZ'dd d4dZ(d5dZ)ej$dZ*d Z+e,hd!Z-ej$d"Z.d6d#Z/d7d$Z0 d8d%Z1y)9u{G3.2 owned validation helpers — single source. producer-side normalize + consumer-side hash/sentinel build + shape validator + fail-fast guard (옛 v4 / partial v5 cp 차단). post-parse 가 cp 에 삽입하는 sentinel: {schema_version, t2i_prompt_hash, owned_hash, camera_direction_hash, validator, violations} LLM 응답 schema 에는 들어가지 않음 (CP-only). ) annotationsN)AnyDictIterableListOptionalAppErrorz [0-9a-f]{16}zscene_detail_owned_objects.v1z(scene_detail_owned_objects.v1.close_skipP)redrawanchorabsentc t}g}|xsgD]}t|ts'tdd|dt |j d|j }|sM |jdt|tkDr#tdd t|d td |||vr|j||j||j|S#t$rtdd|dwxYw) ustrip / dedupe (case-sensitive) / drop empty whitespace / 정렬. 저장 직전에 호출 — 단순 검증이 아니라 정규화 책임. case 보존 이유: 영어 canonical 안에서도 "TV" vs "tv" 같은 분리가 의도일 수 있음. Wave 6 IMPORTANT fix (feedback_no_silent_fallback.md): - 비-string entry → AppError(step.contract_violation) raise (silent drop 차단 — upstream LLM 이 schema 위반 가능). - overlength entry (>OWNED_MAX_LEN) → AppError raise (silent truncation 으로 의미 잘림 차단). - non-ASCII (Hangul / CJK / Kana 등) → AppError raise (영어 canonical 위반 surface — 옛 silent skip 패턴 폐기). - 진정한 빈/whitespace-only entry 만 silent drop (정규화 의도). step.contract_violationz)normalize_owned_list got non-string item z (type=)codemessageasciiz(normalize_owned_list got non-ASCII item uS — owned MUST be English canonical common nouns (Hangul / CJK / Kana 등 차단).z*normalize_owned_list got overlength item (z > z): )set isinstancestrr type__name__stripencodeUnicodeEncodeErrorlen OWNED_MAX_LENaddappendsort)itemsseenoutitss S/Users/manta/Documents/Projects/TheRoad-I1/backend/app/core/steps/_owned_helpers.pynormalize_owned_listr*#s&DCkrk"c".?vF!"X../q2  HHJ   HHW  q6M !.Axs=/QE;  9    1 MNHHJ J/" .>qeD88  s !C""C=cdjt|jd}tj|j ddS)uTsorted+joined sha256[:16]. caller 가 normalize 통과한 list 를 넘긴다고 가정. 빈 list 도 결정론적 hash — sentinel hash 가 항상 존재 보장. Wave 6 IMPORTANT fix: defense-in-depth — caller order 와 무관하게 내부에서 sorted 적용. docstring 의 "sorted+joined" 약속을 구현이 보장. |utf-8N)joinsortedrhashlibsha256 hexdigest)ownedpayloads r)compute_owned_hashr6_s@hhve}%,,W5G >>' " , , .s 33cx|xsdjd}tj|jddS)ucamera_direction 텍스트 sha256[:16]. close framing 판정의 input 이라 text 변경 시 sentinel drift 감지에 사용. r-Nr.rr1r2r3)camera_directionr5s r)compute_camera_direction_hashr<ks8 %2--g6G >>' " , , .s 33r7cx|xsdjd}tj|jddS)u8variation['t2i_prompt'] or "" exact string 의 sha256[:16] (round 5 BLOCKING 3). t2i_review 가 t2i_prompt 를 수정 후 sentinel 미갱신 하면 verify_completion 의 drift 검증이 stale 로 partial 마킹. case / whitespace / punctuation 변화 모두 다른 hash → drift 검출 보장. r9r-Nr.r:) t2i_promptr5s r)compute_t2i_prompt_hashr?ss8R''0G >>' " , , .s 33r7r9cPt||t|}|d|k(ry||d<y)u\sentinel 의 t2i_prompt_hash 만 갱신. 다른 hash 필드 보존. sentinel shape 검증 후 갱신 — shape 위반 시 raise. Returns: True (갱신 발생) / False (이미 일치, no-op). AC-A1 (spec §3.4): t2i_review 가 mutation 후 의무 호출. owned_hash / camera_direction_hash / validator / violations 는 보존. wheret2i_prompt_hashFT)assert_owned_sentinel_shaper?)sentinelr>rBnew_hashs r)refresh_t2i_prompt_hashrG~s7 6&z2H!"h."*H  r7)schema_versionrC owned_hashcamera_direction_hash validator violationsowned_usage_hashc t|ts&tddt|jd|g}t |D]\}}t|t s)tdd|dt|jd|dD]A}||vst||trtdd|d|d |j|d||d tvr tdd|d |d d td ||d dk7r|dstdd|d|d d||j|dt|t|k7r(tddt|dt|d |tt|t|k7rtdd|d |t|t|k7r:tddtt|dtt|d|y)uC2 v1 §4.1 — owned_object_usage[] 가 normalized owned list 를 정확히 1:1 echo 검증. 위반 시 AppError(step.contract_violation). silent miss 차단. 검증: - owned_object_usage 는 list. - 각 entry 는 dict + {owned_token, usage_kind, source_phrase}. - usage_kind ∈ {redraw, anchor, absent}. - source_phrase 는 usage_kind == absent 일 때만 빈 문자열 허용. - exact cardinality: len(owned_object_usage) == len(normalized_owned_list). - owned_token unique (duplicate 금지). - set(owned_token) == set(normalized_owned_list) (no extra / no missing). - empty owned list → owned_object_usage 는 [] (required field 존재). r%owned_object_usage must be list (got ) rowned_object_usage[] must be dict (got  owned_token usage_kind source_phrase]. must be str (got rU ].usage_kind  not in  rrV%].source_phrase empty but usage_kind=z! (empty allowed only for absent) rTzowned_object_usage cardinality z != normalized owned_list z/owned_object_usage owned_token has duplicates: z#owned_object_usage owned_token set z (no extra / no missing) N)rlistr rr enumeratedictrget_OWNED_USAGE_KINDSr"rrr0)owned_object_usagenormalized_owned_listrBtokensidxentryks r)$validate_owned_object_usage_coveragerhs& ($ /*/099:"UGE  F 23 U%&.)#/ K001E7< @A~Za#%>2-cU"QC8 % ! /r%:@  &8 8.)#mE,bI  )*I.0L 67 U%&.)#/ K001E7< @A~Za#%>2-cU"QC8 % ! /r%:@m$<( o. / /.)#mJ>J015';   !&&&'>CJJ #(",%2" L1 E  L .)#nUIF&&+W.   !-.)#/"",/LUGU   !m.)#.>}>OPIINQ !$* UW8h'  'A  q2N '  } | s H>H#ctd|Dd}tj|ddd}tj|j dj d d S) uC2 v1 §3.2 — owned_object_usage[] canonical sha256[:16]. Canonicalization: 1. sorted by owned_token (deterministic order). 2. payload per entry = {owned_token, usage_kind, source_phrase} exact (source_phrase = exact post-schema string, no strip / no normalization). 3. canonical JSON (sort_keys, no whitespace variance). caller 가 validate_owned_object_usage_coverage 를 먼저 호출했다고 가정 (build_owned_sentinel). c3:K|]}|d|d|ddyw)rTrUrVrSN).0es r) z+compute_owned_usage_hash..s5 ( !/ o!"?!3  (sc |dS)NrTrz)r|s r)z*compute_owned_usage_hash..s a &r7)keyT),:F) sort_keys separators ensure_asciir-Nr.)r0jsondumpsr1r2rr3)rborderedr5s r)compute_owned_usage_hashrsa (   ' GjjDZV[\G >>'..1 2 < < >s CCr7c t||dtt|t|t |t ||rt ntt|xsgdS)upost-parse 가 cp 에 삽입할 sentinel 생성 (C2 v1 sentinel v2). close framing 도 모든 hash 포함 (round 3 #4 + round 5 BLOCKING 3). drift 잡기 위함: - t2i_prompt 변경 → t2i_prompt_hash drift (t2i_review 수정 검출). - owned 변경 → owned_hash drift. - camera_direction 변경 → camera_direction_hash drift. - owned_object_usage 변경 → owned_usage_hash drift (C2 v1 — tampered usage detect). C2 v1 §3.2: owned_object_usage coverage 를 먼저 validate (validate_owned_object_usage_coverage against `owned`) 한 뒤 owned_usage_hash 계산. coverage 위반 시 AppError. build_owned_sentinelrA)rHrCrIrJrMrKrL) rhOWNED_SENTINEL_SCHEMA_VERSIONr?r6r<rOWNED_VALIDATOR_CLOSE_SKIPOWNED_VALIDATOR_FULLr])r4r;r>rrrLrbs r)rrs[*)E)?82:>(/!>?O!P45GH*: &@T:+,  r7c vt||t|}|d|k7rtdd|dd|d|y) uC2 v1 N-1 — sentinel owned_usage_hash vs current owned_object_usage hash 비교. mismatch → AppError(step.contract_violation) — tampered usage array detect. sentinel shape 검증 후 비교 (shape 위반 시 raise). rArMru8owned_validation.owned_usage_hash mismatch — sentinel z != current z% (tampered owned_object_usage array) rN)rDrr )rErbrB current_hashs r)assert_owned_usage_hash_matchesrsb 6+,>?L"#|3*J./2,|>NO77bI   " #B"d:b##6"@]:]*12&bI  ] b\"c",*@*@*D.'t,6E7, ]"H\23Q!T".23%8 G,,-Rw8 ABrBb#&26se2bTB $R 1 12"UG=A > lG26se< !({"UG5=4kJs H Hc |sytd|D}|dk(ry|t|k(rtd|DStdd|dt|d  ) uviolations array 가 진짜 redraw violation 을 포함하는지 판정. Schema 자동 감지 (owned-judge prompt v1 vs v2): - v2 schema (모든 entry 에 verdict present): `verdict == "redraw_violation"` 만 카운트. anchor_reference 는 LLM 이 "발견했지만 위반 아님" 으로 분류한 것 — 통과. - v1 legacy schema (모든 entry 에 verdict 없음): 옛 동작 — 비어있지 않으면 violation. - mixed schema (일부만 verdict): **strict raise** (review I1) — production 도달 불가 (LiteLLM 1차 schema 검증이 차단). 만약 도달했다면 LLM bug 또는 cp 손상 → silent fallback 대신 fail-fast 로 debug 명확성 보장. cascade fix 2026-05-05: v1 prompt 는 LLM 이 anchor reference 도 violations 배열에 포함시키고 reason 으로 self-clearing 하는 false positive 패턴 발생. v2 prompt 는 verdict enum 으로 강제 분류 → 진짜 redraw 만 contract_violation marking. Fc3LK|]}t|tsd|vsdyw)rN)rr_r{rs r)r}z'has_redraw_violation..s#aAt!4a:s $$$rTc3DK|]}|jddk(yw)rrN)r`rs r)r}z'has_redraw_violation..s N:a155#'99:s rz2owned_validation.violations mixed verdict schema: /uc entries have verdict — LLM schema bug 또는 cp 손상. silent fallback 차단 (cascade fix I1).r)sumranyr )rL verdict_counts r)has_redraw_violationrsz MJ'N:NNN  &@oQs:/0T T r7rAc|sy|td|d|jdxsd}|dkrtd|d|d |jd ixsijd ixsi}|jD]\}}t|ts|jd d k7r,|jd}|rt|t std|d|dt |D](\}} t| ts | jd*t|} | rtd|d|dy#t$rtd|d|d|d| dwxYw)uSpec 8.7 critical gap (round 4 BLOCKING 1 강화 + round 7 BLOCKING 1) — 옛 v4 / cp 부재 / partial v5 cp / 수동 편집 silent drop fail-fast. 허용 path: - bg off → cp None / 옛 cp 모두 통과 (caller 가 빈 dict 리턴). - bg on + cp schema=2 + 모든 ok background entry 에 owned ASCII 1+ entries (normalize 후) → OK. 차단 path (모두 raise): - bg on + cp None → block (round 4 BLOCKING 1 — silent {} 통과 차단). - bg on + cp schema<2 → block (옛 v4 cp 잔존). - bg on + cp ok 인데 어느 ok background entry 라도 owned 부재/빈 → block. - bg on + cp ok 인데 owned entry 에 non-ASCII 포함 → block (round 7 BLOCKING 1 — 수동 편집 / 부분 산출 stale ["문"] silent drop 차단). - bg on + cp ok 인데 normalize 후 owned 가 빈 list → block (entry 가 모두 whitespace 등으로 silent drop 되는 케이스 차단). Nruj: background_mode is on but background_prompt cp is missing. force background_prompt 먼저 실행 필요.rrHrr z&: background_prompt cp schema_version=uI (<2) — 옛 v4 cp 잔존. force background_prompt 먼저 실행 필요.data backgroundsstatusokobjects_owned_by_backgroundz: background_prompt[uA] missing or empty objects_owned_by_background — partial v5 cp.rz].objects_owned_by_background[z] zf contains non-ASCII. owned MUST be English canonical common nouns (round 4 Q2=B / round 7 BLOCKING 1).uK].objects_owned_by_background empty after normalize — silent drop 차단.) r r`r$rr_r]r^rrrr*) bp_cpbackground_mode_onrBschema_vrbidrfr4reitem normalizeds r)'assert_background_prompt_owned_contractrs.  }*'II  yy)*/aH!|*'?zJRS  99VR(.B33M2FL"K!'') U%&  99X $ &  78Jud3.g1#7EE #5)ICdC(  G$ *"*%0 .g1#7OO G**& 2 '!5cU;&&)U"TH5?? s D88#Ec g}|D]R}|jdd}|jdd}|jdd}|jd|d|d|Tdj|S) uowned judge redraw_violation evidence 를 repair prompt 용 텍스트 블록으로. 각 violation = {owned_object, violating_phrase, reason, verdict}. caller 가 이미 verdict == "redraw_violation" 로 필터링한 list 를 넘긴다. FINDING 7 (e2e-bughunt-v1): `scene_detail_owned_repair` user_template 의 {redraw_violations_block} placeholder 치환에 사용. pure 함수 — LLM 호출 없음. rr9rru- owned 객체: "u" | 위반 구절: "u " | 사유:  )r`r"r/)redraw_violationslinesrrrrs r)format_redraw_violations_blockrs E uu^R0 55!3R8x$  ~-ABRAST "  99U r7z\s+z}Deterministic prop-owned namespace reconciliation: visible prop {pp} owns token {tok}; prompt uses {pp} in the source phrase.>aanatbyinofonortoandforthefromintooverwithunderacrossbetweenz [a-z0-9]+cftjd|jjS)ulprop term normalize — strip + lowercase + collapse internal whitespace. producer (SceneContextLoader.prop_term_map prebuild) 과 consumer (`reconcile_owned_prop_namespace_overlap` 의 owned_token 비교) 양쪽에서 동일 normalize 를 사용한다. caller 가 빈 / whitespace-only 입력을 이미 거른 후 호출하는 것을 가정. r[)_PROP_TERM_NORMALIZE_REsubrlower)r(s r)_normalize_prop_termr,s$ # & &sAGGIOO,= >>r7ct|}|s tS|h}tj|D]}|tvr |j ||S)u*Wave 2 fixup (Task 14) — expand ONE prop source string into the set of normalized terms that should be added to a prop's owned-match term set. For an ASCII string: full normalized phrase plus each ASCII content token found by ``re.findall(r"[a-z0-9]+", normalized)``, minus the fixed stopword set (``_PROP_TERM_STOPWORDS``). Tokens are word- boundary exact, so ``"mapped territory"`` yields ``{"mapped territory", "mapped", "territory"}`` and NOT ``"map"``. For a non-ASCII (Korean / mixed) string: only the full normalized phrase. ``re.findall(r"[a-z0-9]+", ...)`` returns ``[]`` against a pure Korean phrase, and we do not tokenize Korean. Mixed-script strings (e.g. ``"Korean 지도 paper map"``) yield the full normalized phrase plus the ASCII tokens only; Hangul glyphs are not split into sub-tokens. Empty / whitespace-only input → empty set. No length cutoff — 2-letter ASCII tokens like ``"tv"`` survive (unless they appear in the stopword set). This util is pure (no DB / I/O / global state). Producer (``SceneContextLoader._load_entity_canon_prop_term_map``) owns token expansion; the consumer helper (``reconcile_owned_prop_namespace_overlap``) stays membership-only. Args: src: a single prop source string (entity_canon.name / description / t2i_prompt or an entity_alias.alias row). Returns: set of normalized term strings to union into the prop's ``prop_term_map`` entry. Empty set if the input normalizes to empty. )rr_PROP_TERM_ASCII_TOKEN_REfindall_PROP_TERM_STOPWORDSr!)srcrr&toks r)_expand_prop_term_variantsr7sSJ&c*J u LC(00< & &   = Jr7cBs|S|s|Stfd|D}|s|S|xsdj}|Dcic]}||j|v}}g} d} |D]} t| trd| vr| j | *| j ddk7r| j | P| j dd} t| t r| s| j | t| } | s| j | | j ddxsd}|j}| j}d}|D]Q}|j |dsj |xs t}| |vr8|j|vs||vrO|}n|| j | Ht| }d |d<tj|| }| j d d}t|t r|r |d z|z|d <n||d <| j |d } | s|S| Scc}w)uTask 14 Wave 2 — visible prop ↔ background-owned namespace overlap fix. background_prompt 가 어떤 location 의 owned 환경 객체 (e.g. ``"map"``) 를 선언했고, 같은 shot 의 visible_entities 에 같은 의미의 visible prop (e.g. ``P06 = 종이 지도``) 이 함께 있을 때, scene_detail T2I prompt 가 prop short_id (``P06``) 로 prop 을 명시적으로 anchor 하면 owned-judge 가 overlap 토큰을 redraw 로 오인하는 false-positive 가 발생한다. 이 helper 는 consumer boundary 의 결정론적 reclass — judge 가 ``redraw_violation`` 으로 찍은 entry 중, 같은 shot 의 visible prop short_id 와 source phrase 가 문법적으로 묶인 경우만 ``anchor_reference`` 로 다운그레이드한다. 재분류 4개 gate (모두 통과시에만 reclass): Gate A (visible-prop gate) ``visible_entities`` 안에 ``P##`` 접두 short_id 가 존재 AND 그 short_id 가 ``prop_term_map`` 의 key 에 존재. Gate B (prompt-literal gate) ``t2i_prompt`` 의 literal substring 에 ``P##`` 가 포함. Gate C (term-match gate) ``v["owned_object"]`` 를 ``_normalize_prop_term`` 으로 normalize 한 뒤 ``prop_term_map[P##]`` set 에 full-string membership 검사. multi-word / single-word 모두 동일 규칙 — helper 는 prop term 을 sub-word 로 split 하지 않는다 (예: owned_object ``"map"`` 은 prop term set ``{"map", "paper map"}`` 에는 matched 되지만 ``{"oil lantern"}`` (bare ``"lantern"`` 미포함) / ``{"mapped territory"}`` 에는 not matched). 캐논 prop 표현에 bare common-noun 토큰이 명시적으로 등록되어 있을 때만 reclass — 옛 background owned ASCII canonical 과 prop canon term 양쪽이 같은 어휘로 등재되어야 한다는 의도된 boundary. Gate D (violating-phrase linkage gate, W4b-tightened) ``v["violating_phrase"]`` 가 ``P##`` 와 ``v["owned_object"]`` 둘 다 literal substring 으로 포함 (case-insensitive — 양쪽 모두 ``lower()`` 후 substring 검사). After W4b: gate 4 requires ``violating_phrase`` to contain both ``P##`` and ``owned_object`` as case-insensitive substrings. ``owned_object_usage`` source_phrase echo alone is **insufficient** — the prior 4(a) branch was dropped because the LLM can declare a source_phrase containing the prop's ``P##`` even when the actual ``violating_phrase`` describes a real redraw with no ``P##`` reference. The ``owned_object_usage`` parameter is retained in the signature for input-immutability assertions and future audit; it is not consulted for the linkage gate. 여러 ``P##`` 가 후보면 ``sorted(visible_entities)`` 순서에서 처음 4 gate 를 통과하는 것 선택 (결정론). 매치된 entry 는 새 dict (shallow copy of ``v``) 로 생성 — 입력 dict 는 절대 mutate 안 함 (G9 invariant). 새 dict 의 ``verdict`` 를 ``"anchor_reference"`` 로 바꾸고, ``reason`` 끝에 정확히 한 공백 + 결정론적 note 를 append. 이미 ``verdict == "anchor_reference"`` 인 entry 는 그대로 pass-through — idempotent. Defensive short-circuit (G10) — 모두 input list 의 reference identity 보존: - ``prop_term_map is None`` 또는 ``{}`` → ``violations`` 그대로 반환. - ``violations`` 가 empty → 그대로 반환. - ``violations`` 안에 visible-prop gate 통과 후보 없음 → 그대로 반환. Args: violations: owned-judge ``violations`` list. 각 entry 는 dict (owned_object/violating_phrase/reason/[verdict]). verdict 누락 또는 ``"anchor_reference"`` 는 pass-through. owned_object_usage: 같은 shot 의 merged owned_object_usage. **After W4b: not consulted for reclassification** — gate D dropped the source_phrase echo branch. Parameter is retained in the signature for input-immutability assertions (G9 invariant) and future audit / debug visibility; the helper does not mutate it and does not index into it for gate decisions. t2i_prompt: 현재 variation 의 t2i_prompt 문자열. Gate B literal substring 검사 입력. visible_entities: shot 의 visible_entities list (C##/L##/P##/C##O## mixed). helper 가 ``P`` 접두만 추출. prop_term_map: short_id (``P##``) → 정규화된 prop term frozenset. ``SceneContextLoader._load_entity_canon_prop_term_map`` 가 main thread 에서 미리 build 한 dict. helper 는 read-only. Returns: 새 list. 재분류 발생 entry 만 새 dict (shallow copy + verdict / reason 갱신). 나머지 entry 는 원본 reference 그대로 reuse. defensive short-circuit 시엔 ``violations`` 자체 reference 를 그대로 반환. Pure 함수 — DB / I/O / 글로벌 상태 접근 없음. c3lK|]+}t|tr|jdr|vr|-yw)PN)rr startswith)r{sid prop_term_maps r)r}z9reconcile_owned_prop_namespace_overlap..s5"' c3 CNN3$7C=visible_entitiesrvisible_props_sorted _t2i_lowerrprop_in_promptr& any_reclassrrnorm_tokrviolating_phrase_lowerowned_object_lower matched_pp prop_terms new_entrynote prev_reasons ` r)&reconcile_owned_prop_namespace_overlaprgscx  ""'" "))+J'&B RXXZ: %%&!#CK !T"iq&8 JJqM  55 1 1 JJqM uu^R0 ,,L JJqM ' 5 JJqM 55!3R8>B!1!7!7!9)//1!% &B!%%b%0'**2.=)+Jz) "88%-CCJ -'0   JJqM G 1 )"))Z\)JeeHb) k3 'K"-"3d":Ih "&Ih  9 B  JcsH)r$z Iterable[Any]return List[str])r4rrr)r;rrr)r>rrr)r9)rEDict[str, Any]r>rrBrrr)rbrrcrrBrrNone) rqrrcrrrrrBrrizOptional[Dict[str, Any]]rList[Dict[str, str]])rbrrr)r4rr;rr>rrrrrLrrbrrr)rErrbrrBrrr)rbrrBrrr)rErrBrrr)rLList[Dict[str, Any]]rr)rzDict[str, Any] | NonerrrBrrr)rrrr)r(rrr)rrrzset[str]) rLrrbrr>rrrrzDict[str, frozenset] | Nonerr)2__doc__ __future__rr1rloggingretypingrrrrrapp.core.errorsr getLoggerrrmcompilerrrrr rar*r6r<r?rGrrrhrwrrrrrDrrrrrrrrrrrrzr7r)rs #  66$   8 $rzz/* !6G 39x 444=?*-69 &*+EF U U $U  U  U z,0 ii$i i  i * iiXD4" "" "  " % "-""P  ,     2 ,    >k\"R U UU  U  Up4%"**V,?!"'BJJ|4?-`~$~,~~ ~ / ~  ~r7