I am using Open3D to visualize point clouds in Python. Essentially, what I want to do is add another point to the point cloud programmatically and then render it in real time.
This is what I have so far. I could not find any solution to this.
In the code below, I show one possible solution, but it is not effective. The points get added and a new window is opened as soon as the first one is closed. This is not what I want. I want it to dynamically show new points, without closing and opening again. As well as the fact that a new variable is created, which I think can be problematic when working with larger and larger data sets
import open3d as o3d import numpy as np #Create two random points randomPoints = np.random.rand(2, 3) pointSet = o3d.geometry.PointCloud() pointSet.points = o3d.utility.Vector3dVector(randomPoints) #Visualize the two random points o3d.visualization.draw_geometries([pointSet]) #Here I want to add more points to the pointSet #This solution does not work effective #Create another random set p1 = np.random.rand(3, 3) p2 = np.concatenate((pointSet.points, p1), axis=0) pointSet2 = o3d.geometry.PointCloud() pointSet2.points = o3d.utility.Vector3dVector(p2) o3d.visualization.draw_geometries([pointSet2])
Is there any possible solution to this?
If not, what other libraries can I look at that has the ability to render new incoming points in real time.
Advertisement
Answer
New points can be added and visualized interactively to a PointCloud
by extending PointCloud.points
with the new coordinates.
This is because when we use numpy arrays, we need to create a Vector3dVector
isntance which has the convenient method extend
implemented. From the docs:
extend(*args, **kwargs)
Overloaded function.
- extend(self: open3d.cpu.pybind.utility.Vector3dVector, L: open3d.cpu.pybind.utility.Vector3dVector) -> None
Extend the list by appending all the items in the given list
- extend(self: open3d.cpu.pybind.utility.Vector3dVector, L: Iterable) -> None
Extend the list by appending all the items in the given list
So we can use different object instances e.g. ndarrays
, Vector3dVector
, lists
etc.
A toy example and its result:
import open3d as o3d import numpy as np import time # create visualizer and window. vis = o3d.visualization.Visualizer() vis.create_window(height=480, width=640) # initialize pointcloud instance. pcd = o3d.geometry.PointCloud() # *optionally* add initial points points = np.random.rand(10, 3) pcd.points = o3d.utility.Vector3dVector(points) # include it in the visualizer before non-blocking visualization. vis.add_geometry(pcd) # to add new points each dt secs. dt = 0.01 # number of points that will be added n_new = 10 previous_t = time.time() # run non-blocking visualization. # To exit, press 'q' or click the 'x' of the window. keep_running = True while keep_running: if time.time() - previous_t > dt: # Options (uncomment each to try them out): # 1) extend with ndarrays. pcd.points.extend(np.random.rand(n_new, 3)) # 2) extend with Vector3dVector instances. # pcd.points.extend( # o3d.utility.Vector3dVector(np.random.rand(n_new, 3))) # 3) other iterables, e.g # pcd.points.extend(np.random.rand(n_new, 3).tolist()) vis.update_geometry(pcd) previous_t = time.time() keep_running = vis.poll_events() vis.update_renderer() vis.destroy_window()
Why not create an updated geometry and remove the old one?
For completeness, other (which I believe to be not better) alternative approach could consist on the following steps:
- Remove the current
PointCloud
- concatenate the new points as in the OP’s question
- Create new
Pointcloud
and add it to the visualizer.
This yields worse perfomance and barely allows interaction with the visualization.
To see this, let’s have a look to the following comparison, with the same settings (code below). Both versions run the same time (~10 secs).
Using extend |
Removing and creating PointCloud |
---|---|
✔️ Allows interaction | ❌ Difficult interaction |
Mean execution time (for adding points): 0.590 ms | Mean execution time (for adding points): 1.550 ms |
Code to reproduce:
import open3d as o3d import numpy as np import time # Global settings. dt = 3e-2 # to add new points each dt secs. t_total = 10 # total time to run this script. n_new = 10 # number of points that will be added each iteration. #--- # 1st, using extend. Run non-blocking visualization. # create visualizer and window. vis = o3d.visualization.Visualizer() vis.create_window(height=480, width=640) # initialize pointcloud instance. pcd = o3d.geometry.PointCloud() # *optionally* add initial points points = np.random.rand(10, 3) pcd.points = o3d.utility.Vector3dVector(points) # include it in the visualizer before non-blocking visualization. vis.add_geometry(pcd) exec_times = [] current_t = time.time() t0 = current_t while current_t - t0 < t_total: previous_t = time.time() while current_t - previous_t < dt: s = time.time() # Options (uncomment each to try it out): # 1) extend with ndarrays. pcd.points.extend(np.random.rand(n_new, 3)) # 2) extend with Vector3dVector instances. # pcd.points.extend( # o3d.utility.Vector3dVector(np.random.rand(n_new, 3))) # 3) other iterables, e.g # pcd.points.extend(np.random.rand(n_new, 3).tolist()) vis.update_geometry(pcd) current_t = time.time() exec_times.append(time.time() - s) vis.poll_events() vis.update_renderer() print(f"Using extendttt# Points: {len(pcd.points)},n" f"ttttttMean execution time:{np.mean(exec_times):.5f}") vis.destroy_window() # --- # 2nd, using remove + create + add PointCloud. Run non-blocking visualization. # create visualizer and window. vis = o3d.visualization.Visualizer() vis.create_window(height=480, width=640) # initialize pointcloud instance. pcd = o3d.geometry.PointCloud() points = np.random.rand(10, 3) pcd.points = o3d.utility.Vector3dVector(points) vis.add_geometry(pcd) exec_times = [] current_t = time.time() t0 = current_t previous_t = current_t while current_t - t0 < t_total: previous_t = time.time() while current_t - previous_t < dt: s = time.time() # remove, create and add new geometry. vis.remove_geometry(pcd) pcd = o3d.geometry.PointCloud() points = np.concatenate((points, np.random.rand(n_new, 3))) pcd.points = o3d.utility.Vector3dVector(points) vis.add_geometry(pcd) current_t = time.time() exec_times.append(time.time() - s) current_t = time.time() vis.poll_events() vis.update_renderer() print(f"Without using extendt# Points: {len(pcd.points)},n" f"ttttttMean execution time:{np.mean(exec_times):.5f}") vis.destroy_window()