WetBrush is the first real-time full 3D oil painting system that physically simulates fluid dynamics as well as individual bristle interactions. The original research paper (can be in this page) was a collaboration between Adobe Research and The Ohio State University Graphics group. We have been in collaboration with Nvidia as well. This is an on-going research project and we aim to explore the front of physically based natural media art simulation.
- Try out WetBrush at Tech Museum of San Jose!
- Media coverage: NVIDIA Blog, The Verge, Engadget, PopSci, Gizmodo, DigitalArts, StudioDaily, Zimrick, WCCFTech, Digital Journal, IBT, etc.
- SIGGRAPH 2017 Featured Talk.
- GTC 2016 Share-Your-Science interview video.
- Research paper published at SIGGRAPH Asia 2016.
My research interest lies in physically based simulation, real-time physics and rendering, 3D reconstruction and virtual reality.
Modeling, Evaluation and Optimization of Interlocking Shell Pieces
Miaojun Yao, Zhili Chen, Weiwei Xu, Huamin Wang
Computer Graphics Forum (Pacific Graphics 2017).
While the 3D printing technology has become increasingly popular in recent years, it suffers from two critical limitations: expensive printing material and long printing time. An effective solution is to hollow the 3D model into a shell and print the shell by parts. Unfortunately, making shell pieces tightly assembled and easy to disassemble seem to be two contradictory conditions, and there exists no easy way to satisfy them at the same time yet. In this paper, we present a computational system to design an interlocking structure of a partitioned shell model, which uses only male and female connectors to lock shell pieces in the assembled configuration. Given a mesh segmentation input, our system automatically finds an optimal installation plan specifying both the installation order and the installation directions of the pieces, and then builds the models of the shell pieces using optimized shell thickness and connector sizes. To find the optimal installation plan, we develop simulation-based and data-driven metrics, and we incorporate them into an optimal plan search algorithm with fast pruning and local optimization strategies. The whole system is automatic, except for the shape design of the key piece. The interlocking structure does not introduce new gaps on the outer surface, which would become noticeable inevitably due to limited printer precision. Our experiment shows that the assembled object is strong against separation, yet still easy to disassemble.
6-DOF VR Videos with a Single 360-Camera
Jingwei Huang, Zhili Chen, Duygu Ceylan, Hailin Jin
IEEE Virtual Reality (VR) 2017 Proceedings.
Recent breakthroughs in consumer level virtual reality (VR) headsets are creating a growing user-base in demand for immersive, full 3D VR experiences. While monoscopic 360-videos are perhaps the most prevalent type of content for VR headsets, they lack 3D information and thus cannot be viewed with full 6 degree-of-freedom(DOF). We present an approach that addresses this limitation via a novel warping algorithm that can synthesize new views both with rotational and translational motion of the viewpoint. This enables the ability to perform VR playback of input monoscopic 360-videos files in full stereo with full 6-DOF of head motion. Our method synthesizes novel views for each eye in accordance with the 6-DOF motion of the headset. Our solution tailors standard structure-frommotion and dense reconstruction algorithms to work accurately for 360-videos and is optimized for GPUs to achieve VR frame rates (>120 fps). We demonstrate the effectiveness our approach on a variety of videos with interesting content.
Wetbrush: GPU-based 3D painting simulation at the bristle level
Zhili Chen, Byungmoon Kim, Daichi Ito and Huamin Wang
ACM Transactions on Graphics (SIGGRAPH Asia), vol. 34, no. 6. 2015.
We present a real-time painting system that simulates the interactions among brush, paint, and canvas at the bristle level. The key challenge is how to model and simulate sub-pixel paint details, given the limited computational resource in each time step. To achieve this goal, we propose to define paint liquid in a hybrid fashion: the liquid close to the brush is modeled by particles, and the liquid away from the brush is modeled by a density field. Based on this representation, we develop a variety of techniques to ensure the performance and robustness of our simulator under large time steps, including brush and particle simulations in non-inertial frames, a fixed-point method for accelerating Jacobi iterations, and a new Eulerian-Lagrangian approach for simulating detailed liquid effects. The resulting system can realistically simulate not only the motions of brush bristles and paint liquid, but also the liquid transfer processes among different representations. We implement the whole system on GPU by CUDA. Our experiment shows that artists can use the system to draw realistic and vivid digital paintings, by applying the painting techniques that they are familiar with but not offered by many existing systems.
Level-set-based partitioning and packing optimization of a printable model
Miaojun Yao, Zhili Chen, Linjie Luo, Rui Wang and Huamin Wang
ACM Transactions on Graphics (SIGGRAPH Asia), vol. 34, no. 6. 2015.
As the 3D printing technology starts to revolutionize our daily life and the manufacturing industries, a critical problem is about to e-merge: how can we find an automatic way to divide a 3D model into multiple printable pieces, so as to save the space, to reduce the printing time, or to make a large model printable by small printers. In this paper, we present a systematic study on the partitioning and packing of 3D models under the multi-phase level set framework. We first construct analysis tools to evaluate the qualities of a partitioning using six metrics: stress load, surface details, interface area, packed size, printability, and assembling. Based on this analysis, we then formulate level set methods to improve the qualities of the partitioning according to the metrics. These methods are integrated into an automatic system, which repetitively and locally optimizes the partitioning. Given the optimized partitioning result, we further provide a container structure modeling algorithm to facilitate the packing process of the printed pieces. Our experiment shows that the system can generate quality partitioning of various 3D models for space saving and fast production purposes.
Physics-Inspired Adaptive Fracture Refinement
Zhili Chen, Miaojun Yao, Renguo Feng and Huamin Wang
ACM Transactions on Graphics (SIGGRAPH), vol. 33, no. 4. 2014.
Physically based animation of detailed fracture effects is not only computationally expensive, but also difficult to implement due to numerical instability. In this paper, we propose a physics-inspired approach to enrich low-resolution fracture animation by realistic fracture details. Given a custom-designed material strength field, we adaptively refine a coarse fracture surface into a detailed one, based on a discrete gradient descent flow. Using the new fracture surface, we then generate a high-resolution fracture animation with details on both the fracture surface and the exterior surface. Our experiment shows that this approach is simple, fast, and friendly to user design and control. It can generate realistic fracture animations within a few seconds.
Modeling friction and air effects between cloth and deformable bodies
Zhili Chen, Renguo Feng and Huamin Wang
ACM Transactions on Graphics (SIGGRAPH), vol. 32, no. 4. 2013
Real-world cloth exhibits complex behaviors when it contacts deformable bodies. In this paper, we study how to improve the simulation of cloth-body interactions from three perspectives: collision, friction, and air pressure. We propose an efficient and robust algorithm to detect the collisions between cloth and deformable bodies, using the surface traversal technique. We develop a friction measurement device and we use it to capture frictional data from real-world experiments. The derived friction model can realistically handle complex friction properties of cloth, including anisotropy and nonlinearity. To produce pressure effects caused by the air between cloth and deformable bodies, we define an air mass field on the cloth layer and we use real-world air permeability data to animate it over time. Our results demonstrate the efficiency and accuracy of our system in simulating objects with a three-layer structure (i.e., a cloth layer, an air layer, and an inner body layer), such as pillows, comforters, down jackets, and stuffed toys.
Other smaller projects I worked on.
OpenGL-based oil painting simulation engine
Predecessor of WetBrush.
Water effects and animation in kinect-based game "Canyon Adventures"
2014/1 - 2014/5
This becomes a published game! Check it out here!