W21B Wave 4 FP-as-Data / VLM Path Study

0. Current Failure And v7 Evidence

0.1 Observed failures

• 동일 공간이 run마다 투룸처럼 보였다가 원룸처럼 보이는 layout flip이 발생했다.
• 외부 계단이나 deck처럼 단순해야 할 circulation이 복잡한 미로처럼 과해졌다.
• FP PNG가 metadata와 맞지 않으면 BG가 올바른 metadata를 따라도 사용자 눈에는 FP/BG가 불일치해 보인다.
• DB/ImageAsset에는 stale FP row가 history로 누적되지만 active 개념이 명시되지 않아 review UI에서 혼동이 발생했다.

0.2 v7 evidence to preserve

1. Candidate Architecture Families

FP v8의 목표는 "예쁜 도면"이 아니라 "카메라와 BG가 참조할 수 있는 일관된 공간 정보"다. 그 공간 정보는 code-drawn SVG 좌표에서 올 수도 있고, VLM이 FP image를 읽어 추출한 좌표/공간 해석에서 올 수도 있다. 카메라뷰 10x10 projection은 FP 자체의 generic 속성이 아니라, selected shot의 camera/look-at이 확정된 뒤 계산되는 per-selected-shot downstream artifact다.

1.1 Path comparison

2. Candidate Contracts

v8 should compare two ways to obtain a trusted coordinate/spatial SOT: Path-S layout JSON and Path-V VLM-read FP coordinates. Camera-view projection consumes that SOT only after shot selection/cinematography fixes the actual camera/look-at request. The final implementation can choose one or combine them.

2.1 Path-S layout JSON top-level shape

{
  "fp_id": "fp_l05_main",
  "layout_grid": {
    "unit": "normalized",
    "width": 100,
    "height": 100,
    "display_downsample": 10
  },
  "place_semantic_tags": ["small dwelling", "compact service zone"],
  "expected_visual_density": "low|medium|high",
  "key_fixture_groups": [
    {"group_id": "service_strip", "label": "kitchenette and bathroom fixtures"}
  ],
  "element_layout": [ ... ],
  "metadata_only_elements": [ ... ],
  "validation_notes": []
}

2.2 Path-S element layout item

{
  "number": 19,
  "label": "sleeping mat or low bed",
  "base_layer_decision": "base_persistent_furniture",
  "category": "furniture",
  "importance": 0.82,
  "importance_reason": "used by selected background camera recommendation",
  "render_on_plan": true,
  "shape_kind": "rect|line|door_arc|window_line|fixture_icon|area",
  "rect": {"x": 12, "y": 8, "w": 46, "h": 14},
  "anchor_points": [{"kind": "center", "x": 35, "y": 15}],
  "adjacent_to": [9, 20],
  "opens_to": [],
  "fixture_group_id": "sleeping_zone",
  "camera_visibility_hint": "likely foreground|midground|background|not_visible"
}

2.3 Path-S required validator rules

• Every rendered marker number must exist in the source numbered_elements exact-ID set.
• Every source base structural unit and opening referenced by selected BG recommendations must have a layout item.
• rect must be within normalized grid bounds.
• Hard structural areas may touch or contain, but impossible overlaps are needs_review or fail-closed.
• Rendered element count must be capped. Low-importance props become metadata_only_elements.
• Transient/state overlay markers are never rendered on the base FP SVG/raster.
• No scenario-specific string rules. Decisions are driven by IDs, categories, base-layer decisions, and selected-shot usage.

2.4 Path-V coordinate / camera-view VLM probe shape

{
  "fp_id": "fp_l05_main",
  "bg_id": "L05B05",
  "source_image_kind": "t2i_floor_plan|svg_raster|other",
  "camera_request": {
    "source": "selected_shot_camera_recommendation",
    "camera_position_text": "near marker 19, facing marker 9",
    "look_at_targets": [9, 20],
    "lens_hint": "normal|wide|telephoto",
    "use_numbered_elements": [2, 4, 8, 9, 17, 19, 20]
  },
  "camera_view_grid_10x10": [
    [{"marker_numbers": [], "summary": "", "confidence": 0.0}, "..."],
    "..."
  ],
  "visible_elements": [
    {
      "number": 19,
      "label": "sleeping mat or low bed",
      "screen_cell": [7, 4],
      "depth_band": "foreground",
      "horizontal_band": "center",
      "evidence": "visible rectangular low bed shape near upper-left sleeping area",
      "confidence": 0.86
    }
  ],
  "place_context": {
    "place_semantic_tags": ["compact dwelling"],
    "expected_visual_density": "low",
    "key_fixture_groups_in_frame": ["sleeping_zone"]
  },
  "diagnostics": []
}

2.5 Open constants for Claude to fill during implementation brief

2.6 Open constants — Path-V & hybrid (Claude-filled)

Claude confirms the 2.5 Path-S defaults (100x100 internal grid, downsample 10x10; rendered cap 18 counting only render_on_plan=true markers, with selected-BG-referenced structural units/openings exempt from the cap; importance keep-threshold 0.55 unless required by a selected BG camera recommendation; 10 depth bands). The following Path-V / hybrid constants are filled here so A-0 can run both paths without further blocking decisions.

2.7 Projection sequencing lock

3. Phase A-0 Dual-Path Cheap Dry-Run

3.1 Scope

• Targets: fp_l05_main, fp_l09_main, and optionally fp_l04_stairs.
• Input: existing active FP prompt/dossier data and camera recommendations.
• Path-S output: layout JSON, validator report, and debug SVG/HTML preview generated from JSON.
• Path-V output: VLM-derived FP coordinate/spatial interpretation, plus shot-conditioned sample camera_view_grid_10x10, visible elements, and evidence-backed diagnostics.
• Cost: Path-S uses text LLM only. Path-V uses a small VLM probe. Image generation remains 0.

3.2 Dry-run comparison matrix

3.3 Path selection criteria

3.4 Stop conditions

• LLM cannot produce valid rectangles after two prompt attempts.
• VLM camera-view probe cannot produce evidence-backed visible elements for simple views.
• Validator catches repeated impossible adjacency for simple layouts.
• Both paths reintroduce known failures: room-count flip, maze stairs, or generic empty market.
• Hard stop (Codex-locked): if Path-S is invalid AND Path-V is < 0.70 confidence or returns many evidence-less visible elements, do NOT proceed to BG — Stop. A-0 is a path-selection experiment, not an implementation that presumes success.

3.5 A-0 result contract (locked with Codex)

The A-0 runner must emit a single path_comparison_summary.json and a side-by-side HTML, image generation 0, Path-V VLM probe within the ≤6 cap.

path_comparison_summary.json fields

• targets — the FP ids probed (fp_l05_main, fp_l09_main, optional fp_l04_stairs).
• path_s.layout_json_valid, path_s.validator_errors[], path_s.preview_artifact (debug SVG/HTML path).
• path_v.vlm_call_count, path_v.confident_cells_count, path_v.uncertain_cells_count, path_v.visible_elements, path_v.evidence_samples, and path_v.source_image_kind (= v7_t2i_fp_raster for A-0; stamped so a future svg_raster probe compares under the same schema).
• shot_conditioning — must state that projection is per-selected-shot, not a generic FP artifact, and include the camera request source used for each A-0 probe.
• decision_recommendation ∈ {Path-S, Path-V, Hybrid, Stop} and decision_reasons[].

Side-by-side HTML

• Per target: Path-S preview (rendered-from-JSON SVG/HTML) next to the Path-V coordinate readback and shot-conditioned camera-view 10x10 grid + evidence, so a human can read both paths' output together.

3.6 A-0 Result And Selected Path

3.7 Hybrid Implementation Lock

• Path-S responsibility: simple, deterministic box/SVG structure and coordinate SOT where validator passes.
• Path-V responsibility: shot-conditioned camera-view projection, visibility reasoning, place-density evidence, and fallback where Path-S is invalid or low-confidence.
• Conflict policy: validated Path-S coordinates win for geometry; Path-V wins for visual density/place evidence. Disagreements over one screen cell or more are logged as needs_review.
• L09 policy: do not block v8 on the single overlap. Use Path-V fallback now; keep a small optional prompt-calibration backlog if richer retail layouts repeatedly overlap.

4. Path-S Deterministic SVG Renderer

If Phase A-0 selects Path-S or hybrid, the renderer consumes validated layout JSON and emits SVG. It must not call LLM, image, or VLM APIs. It should be simple enough that a user can inspect the output and understand the spatial scaffold.

4.1 Rendering rules

• Use flat rectangles, lines, door arcs, window lines, and simple fixture icons.
• Use a small fixed palette by layer/category, not by scene-specific words.
• Draw marker numbers exactly once for rendered elements.
• Keep metadata-only elements out of the SVG, but list them in sidecar/debug table.
• SVG should be deterministic byte-stable for identical layout JSON.

4.2 Artifacts

5. Shot-Conditioned Camera-View Projection Candidates

This is the missing link in the user's proposal: convert top-down layout data into a coarse camera-image 10x10 map so BG/i2i knows what is likely visible in the frame. This conversion happens after a selected shot has an actual camera/look-at/lens request. The map can be computed from trusted coordinates, derived by VLM from an FP image plus shot request, or produced through a hybrid.

5.1 Path-S computed projection algorithm sketch

1. Resolve camera point C from selected camera unit or camera marker.
2. Resolve look-at point L from target unit/marker.
3. Compute forward = normalize(L - C) and right = perpendicular(forward).
4. For each rendered and metadata-only element, sample center and optional rect corners.
5. Compute depth = dot(P - C, forward) and lateral = dot(P - C, right).
6. Keep points with depth > 0 and abs(lateral / depth) <= tan(fov / 2).
7. Map lateral to screen columns 0..9 and depth bands to screen rows 0..9.

5.2 Path-V VLM-derived projection

Path-V asks a VLM to read the FP image plus camera request and emit the same output class: visible elements, camera-view grid cells, confidence, and evidence text. This is useful when coordinate JSON is unstable or when the camera-view relation is too ambiguous for deterministic projection.

5.3 Common output shape

{
  "bg_id": "L05B05",
  "fp_id": "fp_l05_main",
  "selected_shot_id": "S05_Shot06",
  "projection_timing": "after_selected_shot_camera_decision",
  "coordinate_sot_source": "path_s_layout_json|path_v_vlm_readback|hybrid",
  "camera": {
    "camera_element_number": 19,
    "look_at_element_number": 9,
    "fov_deg": 50
  },
  "projection_source": "computed_from_layout|vlm_derived|hybrid",
  "camera_view_grid_10x10": [
    [{"markers": [], "summary": ""}, "..."],
    "..."
  ],
  "visible_elements": [
    {
      "number": 19,
      "label": "sleeping mat or low bed",
      "screen_cell": [7, 4],
      "depth_band": "foreground",
      "horizontal_band": "center",
      "importance": 0.82,
      "evidence": "coordinate projection or VLM observation"
    }
  ],
  "place_context": {
    "place_semantic_tags": ["compact dwelling"],
    "expected_visual_density": "low",
    "key_fixture_groups_in_frame": ["sleeping_zone"]
  }
}

6. BG Consumption Contract

Phase C changes BG planning from "look at a detailed FP image and infer layout" to "consume an explicit spatial scaffold plus a per-selected-shot camera-view grid and place context."

6.1 What BG receives

• Optional SVG raster image or simplified FP image for visual reference, deliberately simple.
• camera_view_grid_10x10 with visible marker labels and rough screen cells, computed or VLM-derived only after the selected-shot camera/look-at decision.
• Layout JSON subset when Path-S/hybrid is selected, or VLM evidence records when Path-V is selected.
• Place richness fields: place_semantic_tags, expected_visual_density, key_fixture_groups.

6.2 Place-richness rule

6.3 Phase C acceptance

• BG anchor plates preserve the selected spatial scaffold better than T2I-FP-derived plates.
• Known sparse/generic cases retain place identity through key_fixture_groups and density hints.
• No transient/event state leaks into base FP artifact.
• No scenario-specific production literals. Scenario examples stay in fixtures/docs only.

7. Non-Goals And Separate Waves

8. Acceptance And Stop Conditions

8.1 Acceptance by phase

8.2 Stop conditions

• LLM layout JSON repeatedly violates basic geometry constraints.
• VLM camera-view probe gives non-evidence-backed or inconsistent visible element maps.
• Known failure modes reappear in preview: room-count flip, maze-like stairs, or essential place identity loss.
• Validator must rely on scenario-specific literals or substring heuristics to pass.
• BG consumption requires detailed FP art to recover place identity. That means Phase C contract is wrong.

8.3 Testing stance

• Deterministic tests cover schema, validator, SVG renderer stability if selected, and projection math if selected.
• Quality is not claimed from broad TDD. It is assessed through dry-run previews and focused visual review.
• LLM/VLM/T2I uncertainty remains experiment-driven; tests guard contracts, not aesthetics.

9. Work Split