Oj#dZddlmZddlZddlZddlmZmZddlm Z ddl m Z m Z m Z mZmZddlmZddlmZej(eZd Zd Zdd ZGd d eZ d ddZy)uBackgroundMasterPlanStep — Phase 7 Step 2. 각 chain_bg group에 대해 LLM 1회 호출 → master plan 산출. 그룹간 ThreadPool 병렬. prev_shot_ref 그룹은 skip. 흐름: 1. background_classify + scene_save + shot_validator + shot_selection + visual_world_rules 체크포인트 로드 2. classify의 chain_bg group만 추려 그룹별 LLM 호출 (ThreadPool 병렬) 3. prev_shot_ref 그룹은 plan=null로 기록 (호출 X) 체크포인트 data: data.plans = {group_id: {status, plan | error}} ) annotationsN)ThreadPoolExecutor as_completed)Path)AnyDictListOptionalTuple) StepRunner)run_background_master_planz6.202605290248c|xsgDcgc]*}t|ts|jddk(s)|,c}Scc}w)u`background_classify` 의 building_groups → **chain_bg 그룹**(이 스텝의 구매 단위). ★술어는 여기 한 곳 — 스텝과 canary 의 사전 문(`assert_chain_group_cap_covers`)이 같은 함수를 부른다.kindchain_bg) isinstancedictget)groupsgs `/Users/manta/Documents/Projects/TheRoad-I1/backend/app/core/steps/background_master_plan_step.pychain_groups_ofr/s; " ] !At)<vR\A\A ]] ]s ;;;cleZdZd dZd dZd d dZddZ ddZ ddZddZ e ddZ y )BackgroundMasterPlanStepcddl}ddl}ddlm}|j|j t td}|j|j|djdjddS)Nrsettings)background_modemodelschema_versionprompt_versionT) sort_keysutf-8) hashlibjsonapp.core.configrr openai_modelSCHEMA_VERSIONPROMPT_VERSIONsha256dumpsencode hexdigest)selfr%_jsonrpayloads r _config_hashz%BackgroundMasterPlanStep._config_hash5se%,'77**,,   ~~ KK4K 0 7 7 @ )+cr cLddlm}t|j|jz dz dz |j z |z dz }|j r& tj|jdSy#t$r!}tjd||Yd}~yd}~wwxYw) Nrr checkpointsepisodes manifest.jsonr#encodingz+background_master_plan: %s parse failed: %s) r'rr projects_dir project_id episode_idexistsr&loads read_text Exceptionloggerwarning)r/step_idrcpexcs r_load_prev_checkpointz.BackgroundMasterPlanStep._load_prev_checkpointBs, && '$// 9 ( )+/?? ; ' ( 99; \zz",,,"@AA \LgWZ[[ \s$A99 B#BB#c  &'()*+,-./0ddlm}|jdvr;tj d|jdddt |j idS|jd}|jd}|jd0|jd /|jd }|jd }0rdd lm }|0|jd i-|rj|jdixsijdgxsgD]=}|jd} |jddxsd} | -| s0| -t| <?|xsijdixsijdgxsg} | s2tj ddddt |j diidS|xsijdixsijddxs.|xsijdixsijddxsd,|xsijdixsijdgxsg.ddl m'm&ddlm(t%| } i} d}|j'*|j+| r|j)ni)d2&'()*+,-./0f d }d}i}t+dt-dt/| }| rt1|5}| Dcic]}|j3|||d }}t5|D]q}||} |j7\}}|j?d%d}|r|jd&r|||<|| |<|jd'}|d"k(r|dz }g|d(k(sm|dz }s ddd| D]"}|jd)d*k(sd+dd,| |d <$| Dcic]}|d | vr |d | |d }}|jA|| }|jC|\}} }!}"|jED]C\}}#|#jd'd-k7r|#jd.xsi}$|jG|$|$d/<Et-dt/| |z }%|%t-d|%|z |t |j ||| |!|"||d0d1dScc}w#t8$r4}tj;d!||d"t=|dd#dd$}Yd}~d}~wwxYw#1swY^xYwcc}w)3Nrr>onfloor_plan_anchoredu6background_master_plan: skipped — background_mode=%s)applicable_countcompleted_count failed_countr config_hashdatabackground_classify scene_saveshot_validatorshot_selectionvisual_world_rulesscene_director)assert_no_failed_scenesbackground_master_plan) consumer_steprNscenes scene_indexprimary_locationbuilding_groupsu8background_master_plan: no groups from classify — skipplans rules_texttextsegments)build_master_plan_user_promptMasterPlanError)call_structuredc r |d}|jdgxsg}g}g}|D]o}|jdd}|r|vr|j|-|j||jddt|jddxsddd qt|t||d }|s0tj d ||Dcgc]}|d c}|d d|d fS|r2tj d|t||Dcgc]}|d c}|ri|d|in|} |Dcgc]}|d } }t| \} } | | } t| || | }|d||d fScc}wcc}wcc}w#$r6}tjd|||dt|ddd|dfcYd}~Sd}~wwxYw)Ngroup_idmembersloc_idr[label shot_countrno_location_profile)rgrhrireason)raw_member_countkept_member_countfiltered_membersuHmaster_plan: group %s skipped — all members lack location_profile (%s) all_filtered)statusplan pre_filterzHmaster_plan: group %s pre-filtered %d member(s) w/o location_profile: %s) scene_primary)grouprXrSlocation_profiles) user_promptexpected_group_id group_loc_idsgroup_shot_idsrucall_structured_fnproject_config opik_metadataokz master_plan: group %s failed: %sfailed)rperrorrqrr) rappendintlenrArBinfo_select_scenes_for_groupr rstr)rgid raw_members kept_membersrnmlocrrfmg_effrxrelevant_scenesryrvrqrErbrarcru opik_metar{r^rsscene_segmentsshot_selection_cpshot_validator_cps r_processz3BackgroundMasterPlanStep._execute.._processs9J-C%% 2.4"K13L57  eeHb)3"33 ''*$++"%!"w!3&)!%% a*@*EA&F"7 - !%( $4%(%6$4J  ^1AB1A2"X,1AB,d*  ^-.,<=,?,QQx[,M?.F~'8:K+/ +O^8&#-"3 K  1 +&)"/#1&7'6#1"+ tTTTTEC>@.#  ?cJ&S$3", s0 E, E1# E6E;;F6+F1+F61F6) max_workersrez'master_plan: group %s thread raised: %sr~r)rprrqrrrnrpror prev_shot_refprev_shot_only)rprqr}rq gen_order)w21b_surface_role_scope_filterlocation_profile_pre_filter)r]background_catalogshot_background_mapbg_catalog_hashshot_binding_hash diagnostics)rDict[str, Any]returnzTuple[str, Dict[str, Any]])$r'rrrArr)r2rF"app.core.steps.shot_validator_steprUr{rr+app.modules.pipeline.background_master_planrarbapp.modules.llm.llm_clientrcrbuild_opik_metadata_load_location_profilesminmaxrrsubmitrresultr@rrpop%_apply_w21b_surface_role_scope_filter_d6_post_processitems_build_gen_order)1r/moder classify_cp scene_save_cprules_cpscene_director_cprUscsiprimaryr chain_groupsr]r~rropre_filter_by_grouprpoolrfuturesfutr_gidresrEpfrp ordered_plans scope_diagrrrrgentryrq attemptedrbrarcrurr{r^rsrrrs1 @@@@@@@@@@@r_executez!BackgroundMasterPlanStep._executePsS,  # #+H H KKH((  %&#$a"0#002  001FG 22<@  667GH 667GH--.BC 667GH  R #$5t7J7JZr s)+ (,,VR8>BCCHbQWUWWVVM*&&!3R8>B>g-4M#b'* X $"))&"5;@@ARTVW][] KKR S$%#$a"0#002 "   ~2**626<"AA,PRSNR$$VR06B;;FBGNKM (.B33FB?E2JJ:WYZ`^`  ?&v. ",,. ,, /;D ( ( * J J X .0!SC $567 # <LXYLq4;;x3Qz]BLY'0C!#,CZ$'JJL c t4Bbff%7835+C0!$E#J WWX.F)! >1$) #1=,AuuV}/2BD'Qa m$ #)D"(QAjMU,B:a m(<<"( D?? 6   ! !- 0 T/BS)..0KCzz(#t+::f%+D $ 5 5d ;D  13|,|;< )"1i&&89",,,.&'9':#2%66@4G     AZ %Z %NPSUXY)1CHTcNTXYZ =<6DsUS Q?&S<RAS SS?S S )R<6S<SSScddlm}ddlm}i}|} |j |j |j |j k(|jdk(|jjdj}|D]}} tj|jxsd}|xsij%d}t'|t(sM|j%d} t'| t(so| ||j< |j+|S#ttf$r#t j#d|jYwxYw#|j+wxYw) u entity_canon.metadata_json.location.space_profile 을 loc_id → profile 로 build. D6 SOT: location.space_profile 은 entity_extractor (T2) 가 생성. master_plan 후처리의 normalize_space_key 가 본 dict 를 단일 source 로 사용. r) SessionLocal) EntityCanonlocationNz{}uHbackground_master_plan: EntityCanon %s metadata_json parse fail — skip space_profile)app.core.databaserapp.models.projectrqueryfilterr; entity_typeshort_idisnotallr&r> metadata_json TypeError ValueErrorrArBrrrclose) r/rrprofilesdbrowsrowmd loc_blockprofiles rrz0BackgroundMasterPlanStep._load_location_profiles=s2 32.0 ^ 88K(//&&$//9'':5$$**40ce  C$5$5$=>B X2NN:6 !)T2#--8gt,-4HS\\* HHJ":.NNb   HHJs7A2E #D-AE5E/E E E  EE c -ddlm}i}|D]}|jdd}|jdgxsg}|Dcic]o}t|tr]|jdrL|jddt |jddt |jd dxsdd qc}||<d/d -d0-fd } d1d } g} g} g} |jD]\}}t|ts|jddk7r-|jdxsi}t|tsSt|jdxsg}t|jdxsg}|j|i}|Dchc]F}t|jddtr$|jddr|jddH}}g}t}t}|D]&}|jdd}t|jdxsg}t|jdxsg}g}||vr|jd|s|jd|dk(r|s|jd|Dcgc]}t|tr|r||vr|}}|r|jd|rH| j| |||j-||j| ||j|)t|}d} | rd} g}!|D]}t|jdxsg}"d}#|"D]}$t|$tr|$s|$|vsd}#n|#rJ| j| |dg|j-||j| |d} |!j||!}| r|D%chc]0}|jdxsgD]}%t|%tr|%r|%2}&}}%g}'|D]}|jdd}(|jdd})|j|)xsi}*|(|&vr|'j|R| j||(|)t |*jddt |*jd dxsdddgd |'|d<||d<|r| j|d!d"d#|d<d$|d<|jD]\}}|jddk7r|jdxsi}|jdxsgDchc]}|jdd}+}|jdxsgD]j}|jdxsgD]P}%t|%ts|%s|%|+vstd%|d&|jdd'|jd(d)|%d* ld+td,|jDtd-|jDt!| t!| t!| | | | d. },|,Scc}wcc}wcc}wcc}%}wcc}w)2uW21B-wave-2 — surface-role scope filter + transitive BG cascade. This replaces the W20F6 hard filter ``indoor AND shot_count>=2 AND dep_BG_count>=1``. The old policy made exterior/transition/site surfaces disappear before rendering. The new policy is background-first: - ``interior_room`` backgrounds remain floor-plan-bound. - ``exterior_plate`` / ``transition_zone`` / ``site_surface`` backgrounds may be fp-less and are not dropped merely because the location is outdoor. - floor_plans are retained only when a kept background still references them. - dropped backgrounds cascade through exact raw ``depends_on_bg`` keys. bg_id 부재 (_d6_post_process 호출 전 trim) — diagnostic 은 raw identity (loc_id, space_key_hint, time_phase, state_class, applies_to_shots, reasons) carry. plan.floor_plans / plan.backgrounds 둘 다 in-place trim. group 의 backgrounds[] 가 0 으로 비게 되면 ``dropped_groups[]`` 에 reason ``all_filtered_after_trim`` carry (gen_order 단에서 빈 plan 은 자연 skip; defensive). Returns: diagnostics dict (manifest.data.diagnostics .w21b_surface_role_scope_filter 안으로 carry). r)SURFACE_ROLE_ENUMrer[rfrg is_indoorFri)rric t|jddt|jddt|jddt|jddfS)Nrgr[space_key_hint time_phase state_class)rr)bgs r_raw_bg_identityzXBackgroundMasterPlanStep._apply_w21b_surface_role_scope_filter.._raw_bg_identitysYBFF8R()BFF+R01BFF<,-BFF="-.  r3c2dj|S)N|)join)rrs r_raw_bg_key_stringzZBackgroundMasterPlanStep._apply_w21b_surface_role_scope_filter.._raw_bg_key_strings88,R01 1r3c |jdd|jdd|jdd|jdd|jddt|jdxsgt|dS) Nrgr[rrr surface_roleapplies_to_shots)rgrrrrrreasons)rlist)rrs r_bg_identity_dictzYBackgroundMasterPlanStep._apply_w21b_surface_role_scope_filter.._bg_identity_dictsq&&2."$&&)92"> ff\26!vvmR8 "~r :$(0B)C)Ir$J= r3rpr}rq floor_plans backgroundsfp_idrr depends_on_fpinvalid_surface_roleno_consuming_shot interior_roommissing_floor_plan_dependencydepends_on_missing_fpT depends_on_bgdepends_on_dropped_bgno_kept_background_ref)rerrgrrireferenced_by_kept_backgroundrall_filtered_after_trim)rerkroNz*W21B surface-role filter assertion: group=z background loc_id=z space=rz still references dropped fp_id=z after trim.zsurface_role-aware background-first filter; interior_room remains floor-plan-bound; exterior/transition/site may be fp-less; transitive cascade depends_on_bg exact raw key.c3K|]Y}t|trG|jddk(r3t|jdxsijdxsg[yw)rpr}rqrNrrrr.0rs r zQBackgroundMasterPlanStep._apply_w21b_surface_role_scope_filter..CS*/Aa&155?d+BQUU6](b--m<BC/AA!c3K|]Y}t|trG|jddk(r3t|jdxsijdxsg[yw)rpr}rqrNrrs rrzQBackgroundMasterPlanStep._apply_w21b_surface_role_scope_filter..Hrr) policykept_floor_plans_countkept_backgrounds_countdropped_floor_plans_countdropped_backgrounds_countdropped_groups_countdropped_floor_plansdropped_backgroundsdropped_groups)rrrzTuple[str, str, str, str])rrrr)rrr List[str]rr)rrrrrboolrrrrsetradd RuntimeErrorsumvaluesr).r/rrrmember_by_gid_locrrrfrrrr rrrrqrrloc_mapfpfp_idskept_backgrounds_pass1dropped_raw_keys_in_group dropped_raw_key_strings_in_grouprrolerdep_fpsrfmissing_fp_refskept_backgroundschanged still_keptdep_bgsmatchedreffp_refreferenced_fp_idskept_floor_plansrrgmember kept_fp_idsrrs. @rrz>BackgroundMasterPlanStep._apply_w21b_surface_role_scope_filterbs@ R CEA%% B'CeeIr*0bG ! & !Aa&155? h#!%aeeK&?!@"%aeeL!&<&A"B&! & c "$  2 5746/1 )..0KCfd+zz(#t+::f%+DdD)txx 6<"=Ktxx 6<"=K'++C4G&%BbffWb137BFF7B "-0U %47E ,!vvnb1#'/A(B(Hb#I rvvo6<"=%'00NN#9:'NN#67?*7NN#BC&#&!!!S)aAVOw ##NN#:;'../@G0.112B22FG4889KB9OP*11"51"< $$:; G35 *B"266/#:#@bAG#G&)#s33$"BB&*G! ' +223D!8 942556Fr6JK8<<=OPR=ST"&"))"-#+$$. +8+!*B!vvo6<"((=:Q:WUW:WY:W2266'2.:WKYxx .4"4!vvo6<"W*HN&&(ffX&6%9HXAY@\]>>DZ|U =5 1 B'**&--/*' '**&--/*' *--@)A),-@)A$'$7#6#6,+ .S&p*#^!XZsA4X<A Y=Y5Y 8Yc Xddlm}m}m}m}ddlm}|j}g}i} |jD]v\} } | jddk7r| jdxsi} t| jdxsgD]&\} }t|| | | f<|j|(x|j}||||}||ddlm}m}m}|jDcic] \}}|d | }}}| jD]K\\} } }|| dd| } ||d }||d |d |}||d ||d |d }|||d<M||}||}||}||||fScc}}w#t"|f$r}t%d| d| d||d}~wwxYw)uLLM raw intent → deterministic bg_id catalog + sibling fields. Returns: (background_catalog, shot_background_map, bg_catalog_hash, shot_binding_hash). 실패 시 raise — downstream (background_render) 가 빈 dict 로 silent 진행 안 함. I8 intent sort + B7 monotonic next_b 는 `assign_bg_ids` 가 처리. B3 handshake: 각 plan.backgrounds[].bg_id slot 에 새 L##B## 갱신. r) assign_bg_idsbuild_shot_background_mapcompute_bg_catalog_hashcompute_shot_binding_hash)validate_master_plan_assignedrpr}rqr)compute_semantic_keynormalize_space_keySemanticKeyError semantic_keyrgrrrzD6 handshake: group=z idx=u sem_key 재계산 실패 — Nbg_id)app.core.bg_catalogr.r/r0r1rr2rrr enumeraterr_load_prev_background_catalogr3r4r5KeyErrorr)r/rr.r/r0r1r2ru flat_intentsbackref_to_flatrrrqidxr prev_catalogcatalogr3r4r5r7entry sem_to_bgid _flat_idxr space_keysem_keyrE shot_bg_map catalog_hash binding_hashs rrz)BackgroundMasterPlanStep._d6_post_processVs   !88:.0 68(..0KCzz(#t+::f%+D$TXXm%<%BCR/2,.?c +##B' D 199;  lRT[\ .xL)R -=r-?P&g.BwK&= 08 .w7 0=  \<??+Y./ "*3%uSE9WX[W\] s/E>*,F F)F$$F)c |jdz }|jrv|jdj}|rTt j |}|xsij dixsi}|j d}t|tr|r|S |j}|t j |jd}|xsij dixsi}|j d}t|tr"tjd|j|SiS#t$r }tjd|Yd}~d}~wwxYw#t$r!}tjd |Yd}~iSd}~wwxYw) u T4-fix B2 + T4-fix2 B1: 자기 cp 또는 archive 에서 이전 background_catalog 로드. force re-run / 운영 중 master_plan 재실행 시 prev_catalog 가 비어있으면 next_b counter reset 가능 → ID 변경 → 모든 downstream chain_bg 재렌더 발생. 이전 catalog 를 prev 로 넘기면 assign_bg_ids 가 same-semantic_key 는 ID 재사용 + 새 sem_key 만 monotonic next_b 부여. T4-fix2 B1 (review iter6): explicit force (`.force_cleared` marker) 환경에서도 prev catalog 보존 의무. `load_checkpoint()` 는 marker 있으면 archive 복원 차단 → prev={} → ID reset. 본 helper 는 archive 를 직접 read (manifest restore 안 함, force 의도 보존). 조회 우선순위: 1. manifest.json (정상 resume — load_checkpoint 와 동일 결과). 2. .force_cleared marker 있어도 latest archive 직접 read (`_find_latest_archive`). 3. 둘 다 없으면 빈 dict (fresh — 첫 실행). r7r#r8rNrzCbackground_master_plan: manifest.json read failed (try archive): %sNzdbackground_master_plan: prev catalog read from archive %s (D6 ID preservation under explicit force).zrrrr@rArB_find_latest_archivername) r/ manifest_path text_dataown_cprNr@rE archive_path archive_datas rr:z6BackgroundMasterPlanStep._load_prev_background_catalogst&  LL?:M##%)33W3EKKM !ZZ 2F"Lb--fb9?RD"hh';._visitst|9$ ;D8D(z*_select_scenes_for_group..Ls q'7r3)keyS_Shot) rrr setdefaultrrXrsortrW)rtrrrrsr member_locs primary_mapsel_mapsrshots_by_scenesi_intselshshirrelevant_scene_indicesrsegrys rrrs). )R(@(FB(FG(F11X;(FKG%2K G &B++FB7=2 B B8R P VTV V UU= ! : quu%B": + W&!!4!4!67,.O WW] #"c"b0F&F wwy"-GGFB'#''B/   78,,.//B $B B4uR %&'$ (/ N **cHXs J 'J)rrj)N) rtrrrjrrgrrgrszOptional[Dict[int, str]]rz&Tuple[List[Dict[str, Any]], List[str]])__doc__ __future__rr&loggingconcurrent.futuresrrpathlibrtypingrrr r r app.core.step_runnerr rr getLoggerrkrAr)r*rrrrcr3rrs # ?33+   8 $!^ \ z\ H/3 @+ @+(@+0@+0 @+ , @+ , @+r3