The associate professor is bridging science and art to redesign the interface between the real and virtual worlds. Lost treasures, hidden tombs, videos games – your imagination is the limit. 

How does a computer engineer come to study art and design?

KP: About midway through my degree I felt it was not the right fit. My parents said, “Look, it’s a great degree to have. Finish it off.” After that I went to UC-Irvine and into a program called Arts Computation Engineering with the idea of combining artists and computer scientists and engineers. That was a life-changing event for me. It has shaped the way I teach and think about things.

But you found your way back to computer science and took on some unexpected projects?

KP: A lot of my work revolves around mixed reality — having tangible actual objects in projected environments. I’ve worked on projects with art historians trying to find a lost da Vinci painting in Florence, Italy. I also worked on a National Geographic project trying to find the lost tomb of Genghis Khan in Mongolia.

What are some of the questions surrounding virtual/mixed reality that brought you to UW-Madison?

KP: When we try to take real-life environments and represent them in a computer or a video game, it loses a lot of detail. For example, because home environments are cluttered and chaotic and complex, they’re really hard to put into a computer. How can we render things fast enough for virtual reality in a way that we can actually experience them?

What’s your goal in simplest terms?

KP: Our goal has always been to try to visualize the data at its native resolution, which is really hard.

Can you give us an example of visualization in action?

KP: We have worked with crime scene investigators on using this technology to investigate crime scenes. You don’t want to lose information that might be pertinent to a crime scene. You also don’t want to open yourself up for the defense asking: “how do we know that you didn’t manipulate the data or throw data away that we needed?”

Your Accelerator project focuses on improved data rendering and visualization methods. What are some cool applications down the road?

KP: One clear application is the video game industry. Realistic graphics are a key feature. Right now, the way you make realistic graphics is you hire incredibly talented artists who try to make a 3-D model on a computer that looks realistic. Anything you can do to capture the real world and put that in a pipeline saves time, saves money, gives you more recognition.

You also work on historic preservation?

KP: We have partners with the National Park Service who do shipwreck reconstructions. One of their problems is they can either see the shipwreck as a whole, or as a small area of detail. Not both. We can completely solve that.

What about sports?

KP: Sports is definitely an application that often resonates with people. I can envision a sporting event in which you can choose your own camera position. Imagine watching a basketball game or a football game and being able to move the camera to any location, almost like a movie.

What was your first interaction with WARF?

KP: I co-teach a wearable tech course and have students developing new cool technology. They wanted to know about intellectual property and so we reached out to WARF. We’ve had [WARF IP Manager] Justin Anderson come out and talk to our class every single semester. When students are doing this type of cutting-edge creative work, IP issues come up and it’s really important to think through them.

Big picture question. How has the pandemic changed our relationship to space and to one another?

KP: In COVID times, the idea of being able to do more and more things remotely is really powerful. My hope is that we get the best of both worlds. It’s going to be very interesting to get to the other side of this.