Kapnetix - AI for faster motion capture editing

Gauss algorithmic has been fortunate enough to work with one of the world's largest gaming publishers, and while we cannot speak much about this case right now, it has given us as AI practitioners an exclusive opportunity to see how our favorite video games are developed. During this process, we discovered that it's both amazing work, but also extremely time-consuming. A single game typically takes multiple years to complete and release. From what we have seen a lot is simply down to the lack of intelligent automation options. The automation tools that somewhat work have mostly been developed by game developers themselves and solve isolated problems.

At Gauss Algorithmic we saw that we could seize the opportunity and create AI-powered tools for game developers. When developing a product, a key to success is finding the right focus. You cannot do everything. We decided to focus on motion capture cleanup. This is a simple and understandable problem, yet it’s really hard to automate.

The challenges with optical motion tracking

Most top-level developers use an optical motion tracking system. In simplicity, high-speed cameras are used to track back light beams reflecting from coated plastic balls (markers) attached to the element we want to track motion. This is the most accurate technology at this point in time and has been heavily used in some of the best AAA Games and blockbuster movies. The challenge with this technology is that markers are not visible all the time and the system loses its understanding of that point in 3D space. This can lead to various deformations of the movement and the character movement can look unnatural.

A team of motion professionals is then deployed to fix these issues, but here’s where the catch is. It actually takes longer to clean up the data than to acquire it. We’ve had the possibility to talk to tens of motion capture professionals and came to an interesting metric.

1 hour of raw captured motion takes roughly 1 day to shoot and 5 days to clean up.

That absolutely shocked us, but as well emphasized how big this opportunity is for AI. In fact, we could be automating a very large and very expensive amount of work that creative professionals do want to do.

Building a new product category

With our experience, we understand that no one can create a successful business/product without market validation. This is an important first step before you get too deep in development. Thanks to our professional services work we knew that deep neural networks are capable of replicating some of the work motion professionals do, but this is just a single customer. The task here is simple: talk to as many potential customers as possible and try to understand if they have the problem and how they are dealing with it.

Through small prototypes showcased at trade shows, directly contacting professionals online, and even discussing with existing technology suppliers, we saw that professionals were dealing with the problem (on different levels) everywhere. We also received organic conversions on our website kapnetix.ai, where teams expressed their frustrations with the mocap cleanup process.

We are now in the phase of product development. Focusing first on what we’ve seen and been told are the most in-demand areas. Marker cloud reconstruction is ready for trials and we’ll soon release an automated retargeting solution. To find out more we suggest visiting kapnetix.ai and following its social media. If you’re interested in building a similar AI product then contact us directly.

‍