Fluid documentation (princeton-vl/infinigen_internal/princeton-vl#61)

* Add hello world scene

* Config tweaks

* fix the camera bug and water not simulating

* res fix

---------

Co-authored-by: pvl-bot <[email protected]>

docs/GeneratingFluidSimulations.md

@@ -1,13 +1,26 @@
 # Generating Fluid Simulations
 
-This documentation details how to generate fire and water simulations like those shown in the Infinigen launch video. It assumes you have already completed [Installation.md] and [HelloWorld.md]. Fluid simulations require *significant* computational resources.
+This documentation details how to generate fire and water simulations like those shown in the Infinigen launch video. It assumes you have already completed [Installation.md], [HelloWorld.md]. Fluid simulations require *significant* computational resources.
 
 ## Setup
 
 To generate fluids, you run install.sh with the optional FLIP-Fluids setup step (`bash install.sh flip_fluids`), or, please run `bash worldgen/tools/install/compile_flip_fluids.sh` to install flip fluids now.
 
 ## Example Commands
 
+#### Generate a video of a single scene with simulated fire generated on the fly
+```
+python -m tools.manage_datagen_jobs --specific_seed 3930249d --output_folder outputs/fire --num_scenes 1 --pipeline_config local_256GB.gin monocular_video.gin --wandb_mode online --cleanup none --config plain.gin fast_terrain_assets.gin use_on_the_fly_fire.gin
+```
+Because fluid simulation takes a long time, the fire resolution can be reduced in use_on_the_fly_fire.gin, by setting `set_obj_on_fire.resolution = {resolution}`. This will reduce the fire quality but speed up the simulation.
+
+#### Generate a video of a single valley scene with simulated river
+```
+python -m tools.manage_datagen_jobs --specific_seed 61fc881a --output_folder outputs/river --num_scenes 1 --pipeline_config local_256GB.gin monocular_video.gin opengl_gt.gin cuda_terrain.gin --pipeline_overrides iterate_scene_tasks.frame_range=[100,244] --config river.gin simulated_river.gin no_assets.gin no_creatures.gin fast_terrain_assets.gin --wandb_mode online --cleanup none 
+```
+Similar to fire, the simulation can be sped up by reducing the resolution. In simulated_river.gin, the resolution can be modified by setting `make_river.resolution = {resolution}`.
+Also, note that this command will produce a scene without assets to speed up the process. However, the liquids generally interact with the objects by splashing on them, and a scene like this can be produced by removing the `no_assets.gin` option in the above command. 
+
 #### Generate videos of random scene types, with simulated fire generated on the fly when needed
 ```
 python -m tools.manage_datagen_jobs --output_folder outputs/onthefly  --num_scenes 10 \
@@ -25,3 +38,25 @@ python -m tools.manage_datagen_jobs --output_folder /n/fs/pvl-renders/kkayan/riv
     --wandb_mode online --cleanup none --warmup_sec 12000
 ```
 
+
+
+## Using Pre-Generated Fire
+
+High-resolution fluid simulations take a long time, so assets on fire can pre-generated and imported in the scene instead of being baked on-the-fly. This allows for fire to be simulated once, instead of every time a scene is generated. To pre-generate fire assets of all types (bush, tree, creature, cactus, boulder), run:
+```
+python tools/submit_asset_cache.py -b $BLENDER -f {fire_asset_folder} -n 1 -s -40 -d 184
+```
+where `fire_asset_folder` is where you want to save the fire. The number of assets per type, start frame and duration can be adjusted.  
+
+If you only want to pre-generate one type of asset once, run:
+```
+$BLENDER --background -noaudio --python fluid/run_asset_cache.py -- -f {fire_asset_folder} -a {asset} -s {start_frame} -d {simulation_duration}
+```
+where `fire_asset_folder` is where you want to save the fire. `asset` can be one of `CachedBushFactory`, `CachedTreeFactory`, `CachedCactusFactory`, `CachedCreatureFactory`, `CachedBoulderFactory`. 
+
+### Import pre-generated fire when generating a scene
+After fire is pre-generated with one of the previous commands, edit config/use_cached_fire.gin and set the `FireCachingSystem.asset_folder` variable to `fire_asset_folder` you used when pre-generating fire. After this `use_cached_fire.gin` can be used instead of `use_on_the_fly_fire.gin` when generating a scene. This will import the fire from the folder it is saved instead of simulating it on-the-fly. 
+#### Example Command
+```
+python -m tools.manage_datagen_jobs --specific_seed 3930249d --output_folder outputs/fire --num_scenes 1 --pipeline_config local_256GB.gin monocular_video.gin --wandb_mode online --cleanup none --config plain.gin fast_terrain_assets.gin use_cached_fire.gin
+```

worldgen/config/scene_types_fluidsim/simulated_river.gin

@@ -7,7 +7,7 @@ cam/AnimPolicyRandomForwardWalk.speed = 1
 cam/AnimPolicyRandomForwardWalk.step_range = (5, 10)
 cam/AnimPolicyRandomForwardWalk.yaw_dist = ("uniform",-10, 10)
 
-keep_cam_pose_proposal.min_terrain_distance = 1
+keep_cam_pose_proposal.min_terrain_distance = 1   
 
 compose_scene.hero_boulders_chance = 0.0
 compose_scene.simulated_river_enabled = True
@@ -16,9 +16,8 @@ terrain.elements.landtiles.LandTiles.y_tilt = 0
 terrain.elements.landtiles.LandTiles.y_tilt_clip = 0
 
 camera_pose_proposal.override_loc = (0.45, -24, 8)
-camera_pose_proposal.roll = 180
-camera_pose_proposal.pitch = ("normal", -120, 5)
-camera_pose_proposal.yaw = ('normal', -180, 3)
+camera_pose_proposal.override_rot = (-120, 180, -178)
+
 
 assets.boulder.create_placeholder.boulder_scale = 1
 LandTiles.land_process = None
@@ -28,3 +27,4 @@ core.render.hide_water = True
 compute_base_views.min_candidates_ratio = 1
 walk_same_altitude.ignore_missed_rays = True
 
+make_river.resolution = 400

worldgen/config/scene_types_fluidsim/tilted_river.gin

@@ -15,9 +15,7 @@ compose_scene.hero_boulders_chance = 1.0
 terrain.elements.landtiles.LandTiles.y_tilt = 0.7
 terrain.elements.landtiles.LandTiles.y_tilt_clip = 11
 camera_pose_proposal.override_loc = (0.678843,-30.5532, 11.7858)
-camera_pose_proposal.roll = ('normal', -110, 5)
-camera_pose_proposal.pitch = 180
-camera_pose_proposal.yaw = ('normal', -180, 3)
+camera_pose_proposal.override_rot = (-110, 180, -178)
 
 compose_scene.tilted_river_enabled = True
 

worldgen/placement/camera.py

@@ -220,37 +220,40 @@ def camera_pose_proposal(
     pitch=90, 
     headspace_retries=30, 
     override_loc=None,
+    override_rot=None
 ):
 
     if override_loc is None:
         loc = np.random.uniform(*terrain_bbox)
-    else:
-        loc = Vector(random_general(override_loc))
 
-    alt = animation_policy.get_altitude(loc, terrain_bvh)
-    if alt is None: return None
-
-    headspace = animation_policy.get_altitude(loc, terrain_bvh, dir=Vector((0, 0, 1)))
-    for headspace_retry in range(headspace_retries):
-        desired_alt = random_general(altitude)
-        if desired_alt is None:
-            zoff = 0
-            break
-        zoff = desired_alt - alt
-        if headspace is None:
-            break
-        if desired_alt < headspace:
-            break
-        logger.debug(f'camera_pose_proposal failed {headspace_retry=} due to {headspace=} {desired_alt=} {alt=}')
-    else: # for-else triggers if no break, IE no acceptable voffset was found
-        logger.warning(f'camera_pose_proposal found no zoff for {loc=} after {headspace_retries=}')
-        return None
+        alt = animation_policy.get_altitude(loc, terrain_bvh)
+        if alt is None: return None
 
-    loc[2] = loc[2] + zoff
-    if loc[2] > terrain_bbox[1][2] or loc[2] < terrain_bbox[0][2]: 
-        return None
-
-    rot = np.deg2rad([random_general(pitch), random_general(roll), random_general(yaw)])
+        headspace = animation_policy.get_altitude(loc, terrain_bvh, dir=Vector((0, 0, 1)))
+        for headspace_retry in range(headspace_retries):
+            desired_alt = random_general(altitude)
+            if desired_alt is None:
+                zoff = 0
+                break
+            zoff = desired_alt - alt
+            if headspace is None:
+                break
+            if desired_alt < headspace:
+                break
+            logger.debug(f'camera_pose_proposal failed {headspace_retry=} due to {headspace=} {desired_alt=} {alt=}')
+        else: # for-else triggers if no break, IE no acceptable voffset was found
+            logger.warning(f'camera_pose_proposal found no zoff for {loc=} after {headspace_retries=}')
+            return None
+
+        loc[2] = loc[2] + zoff
+        if loc[2] > terrain_bbox[1][2] or loc[2] < terrain_bbox[0][2]: 
+            return None
+    else:
+        loc = Vector(random_general(override_loc))
+    if override_rot:
+        rot = np.deg2rad(override_rot)
+    else:
+        rot = np.deg2rad([random_general(pitch), random_general(roll), random_general(yaw)])
 
     return loc, rot
 

worldgen/surfaces/templates/blackbody_shader.py

@@ -2,6 +2,7 @@
 # This source code is licensed under the BSD 3-clause license found in the LICENSE file in the root directory of this source tree.
 
 # Authors: Karhan Kayan
+# Acknowledgement: This file draws inspiration from https://www.youtube.com/watch?v=zyIJQHlFQs0 by PolyFjord
 
 import bpy
 import mathutils

worldgen/tools/install/compile_flip_fluids.sh

@@ -1,3 +1,18 @@
+OS=$(uname -s)
+
+if [ "${OS}" = "Linux" ]; then
+    BLENDER_DIR='blender'
+    BLENDER_ADDONS="${BLENDER_DIR}/3.3/scripts/addons"
+    BLENDER_EXE="${BLENDER_DIR}/blender"
+elif [ "${OS}" = "Darwin" ]; then
+    BLENDER_DIR='./Blender.app'
+    BLENDER_ADDONS="${BLENDER_DIR}/Contents/Resources/3.3/scripts/addons"
+    BLENDER_EXE="${BLENDER_DIR}/Contents/MacOS/Blender"
+else
+    echo "Unsupported OS"
+    exit -1
+fi
+
 FLIP_FLUIDS="https://github.com/rlguy/Blender-FLIP-Fluids"
 FLIP_FLUIDS_FOLDER="Blender-FLIP-Fluids"
 FLIP_FLUIDS_ADDON_FOLDER="${FLIP_FLUIDS_FOLDER}/build/bl_flip_fluids/flip_fluids_addon"

worldgen/tools/submit_asset_cache.py

@@ -27,7 +27,7 @@ def get_slurm_banned_nodes(config_path=None):
     'CachedBushFactory',
     'CachedTreeFactory',
     'CachedCactusFactory',
-    # 'CachedCreatureFactory',
+    'CachedCreatureFactory',
     'CachedBoulderFactory'
 ])
 parser.add_argument('-n', '--number', type=int, default=1)