Help Shape the Future of Space Exploration

Join The Planetary Society Now  arrow.png

Join our eNewsletter for updates & action alerts

    Please leave this field empty
Blogs
Facebook Twitter Email RSS AddThis

Headshot of Emily Lakdawalla

Mattias Malmer's amazing 3D views of Churyumov-Gerasimenko

Posted By Emily Lakdawalla

02-10-2014 13:14 CDT

Topics: Rosetta and Philae, pretty pictures, comets, amateur image processing, comet Churyumov-Gerasimenko, 3D, animation

I'm thrilled to be able to share with you all a spectacular set of images of Rosetta's comet, produced from NavCam data by a master space image processing enthusiast. Mattias Malmer has been working with space image data for many years, producing iconic images -- his version of the Mariner 10 global view of Venus is my go-to image of that planet. He dropped out of the space image processing scene for a few years, but the amazing Rosetta NavCam views of Churyumov-Gerasimenko have drawn him back. Behold:

This movie of comet Churyumov-Gerasimenko rotating was generated by draping the September 24, 2014 NavCam photo over a 3D model of the comet.

ESA / Rosetta / NavCam / Mattias Malmer

Here are two more videos, for which you'll need a pair of 3D glasses:

In fact, Mattias has processed 20 NavCam images of the comet into 3D views. I've embedded three of them below, but you can download the entire set here in a single Zip file (12.6 MB).

Here's the comet at its duckiest:

Synthetic 3D view of Churyumov-Gerasimenko from August 22, 2014 image

ESA / Rosetta / NavCam / Mattias Malmer

Synthetic 3D view of Churyumov-Gerasimenko from August 22, 2014 image
This 3D view of comet Churyumov-Gerasimenko was generated by draping the August 22, 2014 NavCam photo over a 3D model of the comet and rendering it from two different points of view.

Crossed-eye stereo

Parallel-eye stereo

Flicker gif

A view facing the Philae landing site:

Synthetic 3D view of Churyumov-Gerasimenko from August 7, 2014 NavCam image

ESA / Rosetta / NavCam / Mattias Malmer

Synthetic 3D view of Churyumov-Gerasimenko from August 7, 2014 NavCam image
This 3D view of comet Churyumov-Gerasimenko was generated by draping the August 7, 2014 NavCam photo over a 3D model of the comet and rendering it from two different points of view.

Crossed-eye stereo

Parallel-eye stereo

Flicker gif

And a gorgeous view generated from the latest NavCam image, released today:

Synthetic 3D view of Churyumov-Gerasimenko from September 26, 2014 image

ESA / Rosetta / NavCam / Mattias Malmer

Synthetic 3D view of Churyumov-Gerasimenko from September 26, 2014 image
This 3D view of comet Churyumov-Gerasimenko was generated by draping the September 26, 2014 NavCam mosaic over a 3D model of the comet and rendering it from two different points of view.

Crossed-eye stereo

Parallel-eye stereo

Flicker gif

How did he create these amazingly dimensional views? Mattias is a 3D digital imaging professional, and he brought professional tools and techniques to this data set. He explained to me:

These stereo views were made by projecting the original images over a digital model of the comet. This model was then rendered from two slightly different viewpoints to create the different perspectives for each eye. These two views were then combined into anaglyphs.

The digital shape model itself was slightly more challenging to create. To build it I used several techniques. The two most important are "stereo correlation" and "space carving". Both of these techniques are based on analyzing features in the source images in relation to the same feature in other images. They both hinge on the fact that you know the exact position and orientation of the cameras that shot the images. This information is possible to reconstruct using photometry. There are a few software packages out there that can help in doing that. I used "image modeler".

With the camera positions and pointings in hand, I could can then go on to reconstructing the comet itself. First I roughed it out using space carving. Imagine having a block of clay in front of you. Then you use an overhead projector placed where one of the cameras was in relation to the comet. You shine an image of the comet's silhouette shape onto the clay. Then carve away all the clay not being lit by the sillhouette. This will leave you with a cylinder shaped object that only looks like our comet if seen exactly from the projector's view. So then we go on and move the projector to a new camera position, light it with the corresponding silhouette, and carve our clay. And we do this for as many viewpoints as possible. What's left is a chiseled but fairly accurate representation of our comet. In the computer i did this not by carving in clay but by writing code that did billions of tests to see if points in a 1000 by 1000 by 1000 grid were inside or outside the silhouettes.

I further refined this chiseled shape using dense stereo correlation. We are blessed with one really great pair of OSIRIS images that were shot a short time apart and therefore depict the comet from two slightly different viewpoints. By having the computer compare those images, you can calculate how much each feature in the image moved between the two frames. And knowing that, you can use trigonometry to calculate those features' position in space. With the quality of the OSIRIS images we get very good data. I used this high-quality stereo data to refine the crude space carving model in the areas with coverage.

Here's a view of what the shape model looks like with no photograph draped over it. You can see the virtual chisel marks created by the process that Mattias describes above. In some places, the texture is finer. That's where he was able to refine the model with the single OSIRIS stereo pair.

Mattias Malmer's shape model of comet Churyumov-Gerasimenko as of September 30, 2014

Mattias Malmer

Mattias Malmer's shape model of comet Churyumov-Gerasimenko as of September 30, 2014
Malmer generated this shape model from many Rosetta NavCam photos (credit: ESA / Rosetta / NavCam) and a single OSIRIS stereo pair on the comet (credit: ESA / Rosetta / MPS for OSIRIS-Team MPS / UPD / LAM / IAA / SSO / INTA / UPM / DASP / IDA).

Below is the single 3D image that has been released by the OSIRIS camera team. This is the only 3D image in this entire post that's made from two actual photos taken from different perspectives. It's amazing how much Mattias has managed to accomplish with space carving of NavCam images and stereo correlation on this one OSIRIS stereo pair. Mattias is desperate for more 3D images that he can use to improve his shape model, especially in areas not visible in this OSIRIS image pair. In the grand scheme of things, we don't have long to wait; even if no more 3D OSIRIS images ever come out in press releases, the formal data release is scheduled for May 21, 2015 (or six months after the planned landing of Philae on November 21). Soon!

Churyumov-Gerasimenko in 3D, August 7, 2014

ESA / Rosetta / DLR / MPS for OSIRIS Team MPS / UPD / LAM / IAA / SSO / INTA / UPM / DASP / IDA

Churyumov-Gerasimenko in 3D, August 7, 2014
The two images used to make the anaglyph were taken on 7 August 2014 from a distance of 104 kilometers with Rosetta's OSIRIS narrow-angle camera. They are separated by 17 minutes.

Crossed-eye stereo

Parallel-eye stereo

Flicker gif

 
See other posts from October 2014

 

Read more blog entries about: Rosetta and Philae, pretty pictures, comets, amateur image processing, comet Churyumov-Gerasimenko, 3D, animation

Comments:

Jonathan Ursin: 10/03/2014 09:59 CDT

AWESOME!!!

Caleb Strom: 10/05/2014 02:37 CDT

Hi Emily, I know this is a little off topic but I was wondering if you had any advice for someone considering going into planetary geology. I am interested in all the research going on with Mars, the moon, and other bodies of the solar system but it seems most who study other planets are geophysicists not geologists though I could be wrong. I guess I am asking what path would be the best to take. I know you are not actually involved in the research but you do know a lot about the field so I thought you might have some advice. I apologize if this is not the proper place for this question.

Brian: 10/06/2014 05:04 CDT

Caleb, I studied geophysics in undergrad, and have been considering planetary geology... I had felt it seemed to involve a lot more geology than geophysics. I suppose it's really a mix of both. I'm also looking to figure out the next steps in my career that may hopefully involve planetary geo. So I'd also love to hear from Emily. Of course, I am not sure if this is the proper place either. P.S. That first video was amazing. Props to Mr. Malmer!

Emily Lakdawalla: 10/07/2014 01:41 CDT

Caleb and Brian: I went to Brown to do planetary geology after studying geology as an undergrad. The people in my program had a mix of backgrounds, mostly geology or astronomy with some physics. It sort of depends what you want to do. The people working landed Mars missions these days often have strong Earth field geology backgrounds. At another end of the spectrum are planetary astronomers who use big telescopes and do spectroscopy. Then there are modelers, people who do weather or planetary interiors or what have you -- that takes lots of physics. What do you enjoy? Studying any one of them can be a good major for job prospects even if you don't stay in research. But if you do stay in research, it's crucial to do what you enjoy, because you never make much money so you have to feel you're being paid in the joy you have for your work.

Bob Ware: 10/12/2014 04:34 CDT

Absolutely stunning work by Mattias Malmer! Thank you Mattias Malmer for the effort you put into this. Thank you Emily fo sharing it with us!

Leave a Comment:

You must be logged in to submit a comment. Log in now.

Space in Images

Pretty pictures and
awe-inspiring science.

See More

Join The Planetary Society

Let’s explore the cosmos together!

Become a Member

Connect With Us

Facebook, Twitter, YouTube and more…
Continue the conversation with our online community!