Update NanoVDB with the latest changes by matthewdcong · Pull Request #2031 · AcademySoftwareFoundation/openvdb

matthewdcong · 2025-04-18T20:58:26Z

This PR introduces and includes several improvements to NanoVDB. Changes include:

Add missing constructor for GridBlindMetaData required for C++20
Fixed incorrect accessor recursion for non-leaf nodes
Fixed signedFloodFill for the (rare) case when tiles are missing in the root node
Added IndexGrid support to nanovdb_convert
Introduced new magic numbers for grids and files (backwards and forwards compatible)

From @matthewdcong

Added a DeviceMesh utility class for (multi-)GPU stream and communication management
Added a multi-GPU implementation of PointsToGrid
Fixed TBB template deduction in C++20
Clean up const-correctness in UnifiedBuffer
Fixed missing relocatable device flag for CUDA unit tests

kmuseth

looks good to me though this is so huge I can't say I carefully studied every single change

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* improved change notes and updated version number Signed-off-by: Ken Museth <ken.museth@gmail.com> * improved documentation Signed-off-by: Ken Museth <ken.museth@gmail.com> --------- Signed-off-by: Ken Museth <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* improved nanovdb_convert Signed-off-by: Ken Museth <ken.museth@gmail.com> * fixed typo Signed-off-by: Ken Museth <ken.museth@gmail.com> * added unit tests Signed-off-by: Ken Museth <ken.museth@gmail.com> * review feedback Signed-off-by: Ken Museth <ken.museth@gmail.com> --------- Signed-off-by: Ken Museth <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Minor fix to unit-test of NanoVDB Signed-off-by: Ken Museth <ken.museth@gmail.com> * minor change Signed-off-by: Ken Museth <ken.museth@gmail.com> --------- Signed-off-by: Ken Museth <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add device mesh Signed-off-by: Matthew Cong <mcong@nvidia.com> * Restore current device in constructor and destructor Signed-off-by: Matthew Cong <mcong@nvidia.com> * Enable range-based for loops Signed-off-by: Matthew Cong <mcong@nvidia.com> * Use structured bindings inside for loop Signed-off-by: Matthew Cong <mcong@nvidia.com> * Remove uneeded accessors and modularize functions Signed-off-by: Matthew Cong <mcong@nvidia.com> * Statically initialize entry point Signed-off-by: Matthew Cong <mcong@nvidia.com> * Encapsulate Signed-off-by: Matthew Cong <mcong@nvidia.com> * Switch to lightweight wrapper class for DeviceNodes vector Signed-off-by: Matthew Cong <mcong@nvidia.com> * Cleanup Signed-off-by: Matthew Cong <mcong@nvidia.com> * Minor changes needed for DistributedPointsToGrid Signed-off-by: Matthew Cong <mcong@nvidia.com> * Add RAII DeviceGuard Signed-off-by: Matthew Cong <mcong@nvidia.com> * More cleanup Signed-off-by: Matthew Cong <mcong@nvidia.com> * Clean up NCCL dependency Signed-off-by: Matthew Cong <mcong@nvidia.com> * Remove parallel_for to modularize Signed-off-by: Matthew Cong <mcong@nvidia.com> * Implement and test move constructor/assignment Signed-off-by: Matthew Cong <mcong@nvidia.com> * Add docs Signed-off-by: Matthew Cong <mcong@nvidia.com> * Expand test and documentation Signed-off-by: Matthew Cong <mcong@nvidia.com> * Move non-reinit comment to function description Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com> * Switch to using aliases and modern type names Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com> * Use size_type instead of int for return Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com> * Separate out DeviceGuard and clean up API Signed-off-by: Matthew Cong <mcong@nvidia.com> --------- Signed-off-by: Matthew Cong <mcong@nvidia.com> Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Convert MGPU convolution example to use DeviceMesh Signed-off-by: Matthew Cong <mcong@nvidia.com> * Fix typos and clarify Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com> * Check for errors and switch to for_each Signed-off-by: Matthew Cong <mcong@nvidia.com> --------- Signed-off-by: Matthew Cong <mcong@nvidia.com> Co-authored-by: Mark Harris <783069+harrism@users.noreply.github.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Fixed issue in signedfloodfill Signed-off-by: Ken <ken.museth@gmail.com> * partially addressed review comments Signed-off-by: Ken <ken.museth@gmail.com> * added RootData::TileIterator Signed-off-by: Ken <ken.museth@gmail.com> * cleanup Signed-off-by: Ken <ken.museth@gmail.com> * fixed assert bug Signed-off-by: Ken <ken.museth@gmail.com> * snapshot Signed-off-by: Ken <ken.museth@gmail.com> * fixed bug and improved unit-test Signed-off-by: Ken <ken.museth@gmail.com> * improved unit test and documentation Signed-off-by: Ken <ken.museth@gmail.com> --------- Signed-off-by: Ken <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Signed-off-by: Matthew Cong <mcong@nvidia.com> Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

Signed-off-by: Ken <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

…dBlindMetaData Signed-off-by: Matthew Cong <mcong@nvidia.com> Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

* major improvements to GridBlindMetaData Signed-off-by: Ken <ken.museth@gmail.com> * minor changes to fix clang issue Signed-off-by: Ken <ken.museth@gmail.com> --------- Signed-off-by: Ken <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

* minor simplification in CreateNanoGrid Signed-off-by: Ken <ken.museth@gmail.com> * fixed typo Signed-off-by: Ken <ken.museth@gmail.com> --------- Signed-off-by: Ken <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

* introducing new magic numbers for grids and files Signed-off-by: Ken <ken.museth@gmail.com> * removed two redundant magic numbers Signed-off-by: Ken <ken.museth@gmail.com> --------- Signed-off-by: Ken <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

Signed-off-by: Ken <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add distributed implementation of PointsToGrid Signed-off-by: Matthew Cong <mcong@nvidia.com> * Fix race condition in merge and single GPU case Signed-off-by: Matthew Cong <mcong@nvidia.com> * Fix kernel/cudaFree race conditions Signed-off-by: Matthew Cong <mcong@nvidia.com> * Add DistributedPointsToGrid unittest Signed-off-by: Matthew Cong <mcong@nvidia.com> * Clean up example Signed-off-by: Matthew Cong <mcong@nvidia.com> * Use range-based for loops Signed-off-by: Matthew Cong <mcong@nvidia.com> * Use structured bindings inside for loop Signed-off-by: Matthew Cong <mcong@nvidia.com> * Fix Windows build and some warnings Signed-off-by: Matthew Cong <mcong@nvidia.com> * Update for refactored DeviceMesh Signed-off-by: Matthew Cong <mcong@nvidia.com> * Fix copyright/include Signed-off-by: Matthew Cong <mcong@nvidia.com> * Add parallelForEach helper Signed-off-by: Matthew Cong <mcong@nvidia.com> * Templatize kernels to align with CUB better Signed-off-by: Matthew Cong <mcong@nvidia.com> * Shorten TemporaryDevicePool to TempDevicePool Signed-off-by: Matthew Cong <mcong@nvidia.com> * Refactor to use TempDevicePool Signed-off-by: Matthew Cong <mcong@nvidia.com> * Switch to class Signed-off-by: Matthew Cong <mcong@nvidia.com> * Parallelize kernel dispatch in build Signed-off-by: Matthew Cong <mcong@nvidia.com> * Fix race condition wrt to root node processing Signed-off-by: Matthew Cong <mcong@nvidia.com> * Speed up synchronization Signed-off-by: Matthew Cong <mcong@nvidia.com> * Add comments and fix deprecated TransformInputIterator Signed-off-by: Matthew Cong <mcong@nvidia.com> * Address review comments Signed-off-by: Matthew Cong <mcong@nvidia.com> * Address more review comments * Add more cudaCheck calls and restrict EqualityIndicator type Signed-off-by: Matthew Cong <mcong@nvidia.com> * Fix license identifiers and remove unused code Signed-off-by: Matthew Cong <mcong@nvidia.com> * Add missing include Signed-off-by: Matthew Cong <mcong@nvidia.com> --------- Signed-off-by: Matthew Cong <mcong@nvidia.com> Co-authored-by: = <=> Signed-off-by: Matthew Cong <mcong@nvidia.com>

Signed-off-by: Ken <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Silence CMake warning about FindBoost module deprecation Signed-off-by: Matthew Cong <mcong@nvidia.com> * Fix typo Signed-off-by: Matthew Cong <mcong@nvidia.com> --------- Signed-off-by: Matthew Cong <mcong@nvidia.com> Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

…uild failures * Adding /bigobj to TestNanoVDB.cu compilation options to fix Windows build failure --------- Signed-off-by: Jonathan Swartz <jonathan@jswartz.info> Signed-off-by: Matthew Cong <mcong@nvidia.com>

Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Start support for more than two GPUs in key merge Signed-off-by: Matthew Cong <mcong@nvidia.com> * Flip flop storage Signed-off-by: Matthew Cong <mcong@nvidia.com> * Generalize concurrent leftIntervals/rightIntervals usage Signed-off-by: Matthew Cong <mcong@nvidia.com> * Fix median search Signed-off-by: Matthew Cong <mcong@nvidia.com> * Fix rebalancing Signed-off-by: Matthew Cong <mcong@nvidia.com> * Fix recursive MGPU merge Signed-off-by: Matthew Cong <mcong@nvidia.com> * Update TODO Signed-off-by: Matthew Cong <mcong@nvidia.com> * Add guards for zero-tile GPUs Signed-off-by: Matthew Cong <mcong@nvidia.com> --------- Signed-off-by: Matthew Cong <mcong@nvidia.com>

…corresponding tests) * Fix memory leaks in PointsToGrid tests Signed-off-by: Matthew Cong <mcong@nvidia.com> * Fix leapfrogging across recursion levels Signed-off-by: Matthew Cong <mcong@nvidia.com> * Parallelize kernel dispatch for different levels Signed-off-by: Matthew Cong <mcong@nvidia.com> --------- Signed-off-by: Matthew Cong <mcong@nvidia.com>

* minor cleanup Signed-off-by: Ken <ken.museth@gmail.com> * fixed issue in get/set random access methods Signed-off-by: Ken <ken.museth@gmail.com> * cleanup Signed-off-by: Ken <ken.museth@gmail.com> * deleted white space Signed-off-by: Ken <ken.museth@gmail.com> * cleanup Signed-off-by: Ken <ken.museth@gmail.com> * added unit tests Signed-off-by: Ken <ken.museth@gmail.com> * improved unit tests Signed-off-by: Ken <ken.museth@gmail.com> * improved unit tests Signed-off-by: Ken <ken.museth@gmail.com> --------- Signed-off-by: Ken <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

* Add timer for MGPU PointsToGrid Signed-off-by: Matthew Cong <mcong@nvidia.com> * Avoid blocking kernel launches due to cudaFree Signed-off-by: Matthew Cong <mcong@nvidia.com> * Fix overlapping of copies Signed-off-by: Matthew Cong <mcong@nvidia.com> * Pipeline pointsPerVoxelPrefix sum Signed-off-by: Matthew Cong <mcong@nvidia.com> * Cleanup and optimization Signed-off-by: Matthew Cong <mcong@nvidia.com> * Further improve pipelining Signed-off-by: Matthew Cong <mcong@nvidia.com> * Remove unnecessary sync point Signed-off-by: Matthew Cong <mcong@nvidia.com> * Revert "Remove unnecessary sync point" This reverts commit f01b36ad852f32ec65517b3baa08ea267d5edc2d. Signed-off-by: Matthew Cong <mcong@nvidia.com> * Switch to GPU sync Signed-off-by: Matthew Cong <mcong@nvidia.com> * Fix event destruction race condition and reduce host thread latency Signed-off-by: Matthew Cong <mcong@nvidia.com> * Revert timer add Signed-off-by: Matthew Cong <mcong@nvidia.com> * More fine-grained pipelining Signed-off-by: Matthew Cong <mcong@nvidia.com> * Fix event wait/destruction race condition Signed-off-by: Matthew Cong <mcong@nvidia.com> --------- Signed-off-by: Matthew Cong <mcong@nvidia.com>

* minor cleanup Signed-off-by: Ken <ken.museth@gmail.com> * minor improvements to nanovdb::tools::cuda::PointsToGrid Signed-off-by: Ken <ken.museth@gmail.com> --------- Signed-off-by: Ken <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

Signed-off-by: Matthew Cong <mcong@nvidia.com>

matthewdcong requested review from Idclip, apradhana, danrbailey, jmlait, kmuseth and richhones as code owners April 18, 2025 20:58

matthewdcong force-pushed the sync/aswf_master branch 7 times, most recently from 5bb0830 to 50b9ecb Compare April 21, 2025 15:46

kmuseth approved these changes Apr 21, 2025

View reviewed changes

matthewdcong and others added 16 commits April 21, 2025 11:47

Fix missing OpenVDB dependency for NanoVDB-only build

92d689c

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Separate PointsToGrid kernel lambdas into functors

a79ec33

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Add missing iomanip include for latest GTest

a96cb41

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Fix format strings and Windows conversion warning

97b2299

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Fix ODR violation due to anonymous namespace in SignedFloodFill

f96095f

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Add missing const variant of deviceData(device)

d21bec8

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Add missing inline for DeviceMesh functions

242b573

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Add missing NCCL ifdef

1d1899b

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Fix invalid free in UnifiedBuffer move constructor

4d500e1

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Change UnifiedBuffer prefetch and advise to be const

1d4cf67

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Fix missing relocatable device code flag

d130267

Signed-off-by: Matthew Cong <mcong@nvidia.com>

matthewdcong and others added 27 commits April 21, 2025 11:48

Fix namespace for AbsDiff and RelDiff ostream overloads

22e17e7

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Fix out-of-bounds device id

4adbcca

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Fix TBB template deduction in C++20

690104f

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Fix DeviceMesh.h header to be self-contained

74ae3a5

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Add missing constructor for C++20 used in AddBlindData.cuh

267da7d

Signed-off-by: Matthew Cong <mcong@nvidia.com> Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

removed dead code and updated the minor version number

c945c36

Signed-off-by: Ken <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

Fix class-memaccess warning due to non-trivial copy-assignment in Gri…

462aafd

…dBlindMetaData Signed-off-by: Matthew Cong <mcong@nvidia.com> Co-authored-by: Ken Museth <1495380+kmuseth@users.noreply.github.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

minor simplification in CreateNanoGrid

d0e37df

* minor simplification in CreateNanoGrid Signed-off-by: Ken <ken.museth@gmail.com> * fixed typo Signed-off-by: Ken <ken.museth@gmail.com> --------- Signed-off-by: Ken <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

fixed typos and improved documentation in NanoVDB

72406ff

Signed-off-by: Ken <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

Consolidate DeviceGuard into DeviceMesh header

dd7f12e

Signed-off-by: Matthew Cong <mcong@nvidia.com>

revert deprecation warning

88eb8c9

Signed-off-by: Ken <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

minor cleanup

9b728cb

Signed-off-by: Ken <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

Adding /bigobj to TestNanoVDB.cu compilation options to fix Windows b…

e48a8f8

…uild failures * Adding /bigobj to TestNanoVDB.cu compilation options to fix Windows build failure --------- Signed-off-by: Jonathan Swartz <jonathan@jswartz.info> Signed-off-by: Matthew Cong <mcong@nvidia.com>

Suppress stringop-overflow warning due to GCC 11 regression

5f1033d

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Improve points to grid

1f35dfe

* minor cleanup Signed-off-by: Ken <ken.museth@gmail.com> * minor improvements to nanovdb::tools::cuda::PointsToGrid Signed-off-by: Ken <ken.museth@gmail.com> --------- Signed-off-by: Ken <ken.museth@gmail.com> Signed-off-by: Matthew Cong <mcong@nvidia.com>

Fix Windows warnings

5a0c07d

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Force set _WIN32 flag

16fb132

Signed-off-by: Matthew Cong <mcong@nvidia.com>

Move away from deprecated 20.04 runners

2ba65fe

Signed-off-by: Matthew Cong <mcong@nvidia.com>

matthewdcong force-pushed the sync/aswf_master branch from 50b9ecb to 2ba65fe Compare April 21, 2025 18:50

swahtz merged commit 6e7ab0d into AcademySoftwareFoundation:master Apr 23, 2025
34 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update NanoVDB with the latest changes#2031

Update NanoVDB with the latest changes#2031
swahtz merged 43 commits into
AcademySoftwareFoundation:masterfrom
matthewdcong:sync/aswf_master

matthewdcong commented Apr 18, 2025

Uh oh!

kmuseth left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

matthewdcong commented Apr 18, 2025

Uh oh!

kmuseth left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants