Class 2 → 20.05.23

3D Scene Generation ABC + Text2Scene Generation

1. 3D Scene Generation: Input and Output.
2. Learn 3D Scene Generation from an Example.
3. Text2Scene: Dataset(in depth at Phase 2) and conditional generation.

Readings:

1. Angela’s Thesis: https://purl.stanford.edu/vg064sy5087

   Angela’s Thesis Stanford

2. Sceneformer: https://arxiv.org/abs/2012.09793

   SceneFormer

3. DiffuScene : https://tangjiapeng.github.io/projects/DiffuScene

   DiffuScene

Assignment:

Read all materials, write summary & challenges, deliver questions the night before the next class.

Class Notes

• We focus on Indoor scene generation → outdoor generation suffers from not enough data

• How to represent 3D Indoor Scene (3D infoor scene representation)

  • Composition
  • Arrangement → relative position (e.g., on the surface, beside, supported by)
  • Functionality
  • Asthetic Style → 美学, 通过编辑图像信息
  • Traffic Flow → 人活动的动线

• Representations

  • structural
    • segmented scenes
      • Pro: standard visual representation - eazy to work with; Con: can’t understand the underlying relationships among different elements
    • component/entity sets (*)
      • Pro: s2s analysis; Con: not understand object interaction

  

  • semetic reational graph
    • Pro: can present complex relationships; Con: algorithms on graph wasn’t good enough
  • hierarchy tree
    • Pro: human congnition is hierachical; Con: consume hierarchies inherit design
  • egocentric → represent a scene as a single entity
    • RGB (*)
    • depth (Microsoft Kinect) (*)
    • voxelized representation (can be rebuilt from depth map and point cloud) (*)
    • point cloud-subscene (3D scanner) → not so precise
    • Pro: Easy to get; Con: Relative arrangements not shown

() - used in TOMCAT*

evaluating similarities

• graph kernels
• Interaction descriptors