Realistic Rendering of Transparent Objects: A Novel View Synthesis Approach

View synthesis is a technique used to create novel views of a scene from a set of input images. NeRF (Neural Radiance Fields) is a popular method that uses machine learning and volume rendering to generate 3D representations of a scene. However, NeRF has limitations when dealing with transparent objects as it only considers straight rays and does not account for refraction. Recent studies have proposed various approaches to address this challenge, including marking the area corresponding to transparent objects and optimizing the 3D spatially-varying IoR (inverse radiance) inside the box.
One approach is to use volume rendering to represent the scene, considering two types of rays: straight rays that do not interact with transparent objects and refracted rays that are affected by the object’s transparency. The network estimates color and density values at each sampling point along these rays and uses volume rendering to determine the final pixel color.
Another approach is to optimize the MLP (multi-layer perceptron) using a loss function that calculates the difference between the rendered and true pixel colors. This equation takes into account the distance between the sampling points and the density values estimated by the network.
In summary, NeRF has limitations when dealing with transparent objects in view synthesis, but recent studies have proposed ways to address this challenge using volume rendering and MLP optimization. By considering refracted rays and optimizing the 3D spatially-varying IoR, these methods can improve the accuracy of novel view synthesis for scenes containing transparent objects.