A Matter of Illusion: Interview With James O'Brien

The average person is unlikely to be familiar with the concept of Digital Molecular Matter, though any member of the game-playing or movie-going public has almost certainly encountered it. Repeatedly.

Digital Molecular Matter (DMM) is a physics engine based on an algorithm for fracture and deformation developed by University of California professor James O'Brien. DMM is as pervasive as it is unknown to the general public, though gamers may recall the release of "Star Wars: The Force Unleashed" in 2009, and the startling new level of realism the game brought. Players were impressed by the dynamic bending, breaking and splintering of virtual materials that, until that point, had not been modeled to such an accurate degree.

Six years later, the engine is still impressive. Realizing this, I decided to ask James some questions about the project — which, to date, has been used in more than 90 feature films.

How did you end up creating the project? Was the original intent for it to be used in other applications, like it is now?

"The initial work on Fracture Simulation was done while I was at Georgia Tech. I published a couple of papers on the basic method, and then I came to Berkeley. At Berkeley, we kept working on the algorithms.

I was approached by a company called Pixelux about commercializing [Fracture Simulation] for use in destruction modeling in games and film. We started working on a highly optimized implementation. Eventually we released a plugin for Maya, a software package used for CGI in films. We also worked with Lucas Arts to integrate the system into "The Force Unleashed" games. We then had a couple of companies, such as MPC, license the code for integration into their own effects pipeline.

The main engineer I worked with at Pixelux was Eric Parker, and the primary engineer at MPC was Ben Cole. In 2015, the three of us shared an Academy Award for our work."

---

Academy Award Citation: "To Ben Cole for the design of the Kali Destruction System, to Eric Parker for the development of the Digital Molecular Matter toolkit, and to James O’Brien for his influential research on the finite element methods that served as a foundation for these tools. The combined innovations in Kali and DMM provide artists with an intuitive, art-directable system for the creation of scalable and realistic fracture and deformation simulations. These tools established finite element methods as a new reference point for believable on-screen destruction."

---

I remember DMM being demonstrated before the release of "Star Wars: The Force Unleashed." Were you approached by them, or did you pursue companies that you thought might have been interested?

"Pixelux did a pitch to them with an early version of the software. It turned out that one of the Lucas people had been a huge fan of a video game that Eric [Parker] had written years ago, and was very excited to work with us. It’s a small world, sometimes!"

How different is the project now, compared to the 2009 version?

"I’m not actively working on DMM anymore. Pixelux still maintains it and adds features with occasional new releases. Most of my time now is spent working on a new startup activity at a company called Avametric, where we're using cloth simulation to model clothing so customers can virtually try it on."

Is there anything you wish you could add to DMM, or change?

"We’ve done some new work on fracture simulation that uses adaptive meshing. I think that method would work very well for VFX, but it’s fairly different from the tetrahedral method that DMM is built on. So I don’t think Pixelux will pursue it. Perhaps another company will be interested?"

Was there a point where you realized your work was becoming very significant?

"Well, winning the Academy Award was nice. I think that the release of "Star Wars: The Force Unleashed" was another significant event, as was the first movie to use DMM extensively, "Sucker Punch." I’ve been told that DMM was used in a very small way in "Avatar," but "Sucker Punch" was really the first. Since "Sucker Punch," there have been a steady stream of films using DMM. Last time I looked, it was more than 80 of them. The Wikipedia page has a partial list of some of the places DMM has been used."

Are there any similar projects in "the wild" that you particularly admire?

"The fluid simulation Naiad by Robert Bridson is pretty impressive. However, their company was bought by Autodesk, and I think they are still working to integrate Naiad’s features into an Autodesk product. I’m excited to see what that will be."

If you were to adopt a different career, what would it be?

"Now, why would I want to do that?"

You mentioned a new virtual clothing try-on startup — can you tell me about the company?

"The company is Avametric, and we're focused on developing a system to allow people to virtually try on clothing online. Our system makes it easy for shoppers to capture the 3D measurements of their bodies, for retailers to acquire digital models of their clothing products, and for us to combine the data using physics simulation and realistic rendering to accurately show people how they will look in clothing before they buy it. Personally, I’m very excited about this new project. We are making fast progress, and the results are exciting."

What inspired the new direction?

"In the past, I got a kick out of seeing my work end up in films where millions of people could enjoy it. But the thing with VFX is that if it’s done well, the viewer shouldn't even realize it’s an effect. With this clothing effort, people will be interacting directly with our simulated clothing. That’s going to be awesome."

James O’Brien is a Professor of Computer Science at UC Berkeley, where he teaches courses primarily in computer graphics and animation. He also currently serves as a Director-at-Large of ACM SIGGRAPH.