Help Shape the Future of Space Exploration

Join The Planetary Society Now  arrow.png

Join our eNewsletter for updates & action alerts

    Please leave this field empty
Blogs

Headshot of Emily Lakdawalla

Explaining the new black-and-white Mastcam and MARDI raw images

Posted by Emily Lakdawalla

24-08-2012 17:05 CDT

Topics: pretty pictures, amateur image processing, Mars, Curiosity (Mars Science Laboratory)

If you've been obsessively checking the Curiosity raw images websites for new pictures from Mars, you might have noticed something weird: a bunch of Mastcam images and a few from MARDI that are black-and-white instead of color, and which have a peculiar checkerboard pattern. Here's an example full-resolution image. Let's examine it up close:

Grayscale Mastcam image up close

NASA / JPL / MSSS

Grayscale Mastcam image up close
Zooming in closely to a Mastcam photo reveals a checkerboard pattern.

Weird, isn't it? Why are they checkerboardy, and why are they black-and-white when Mastcam is a color camera?

Before I talk about the checkerboard, I just want to note that there's a little block of especially dark pixels -- that's likely a blemish on the detector.

Okay. There are several things going on here. The first has to do with how Mastcam achieves its color vision, which is different from how most space cameras do it. Most space cameras are black-and-white cameras, and you get color pictures by placing three color filters in front of the camera, taking one picture at a time, and combining the three into a single color photo.

Mastcam is more like the camera you might have in your pocket.  Its detector has color filters painted on it. But you need three different color filters to reconstruct color as the human eye would see it, so each pixel in Mastcam's detector is painted over with a red, or green, or blue filter, in a design originally described by Bryce Bayer. There are two green pixels for each red and each blue pixel, because the human eye is most responsive to color in that wavelength. The green pixels are arranged just as in a checkerboard; one row goes red, green, red, green, red, green... and the next row goes green, blue, green, blue, green, blue...

(That picture's from Wikipedia.) Now, the pixels sitting behind these filters don't know anything about color; they only know how many photons hit them. So the resulting data file is still technically a grayscale image, but it's one where every fourth pixel shows the number of photons received through a blue filter. The grayscale Mastcam photo at the top of this post is a photo that has not been processed to decode the color information that it contains. It's sort of neat because you can easily see the Bayer pattern. Mars is generally orange, so it's brightest in red wavelengths, darkest in blue wavelengths. It's easy, then, to pick out the blue pixels -- they're the dark ones arranged in a neat pattern.

Seeing the Bayer pattern in a Mastcam image
Seeing the Bayer pattern in a Mastcam image

To turn this image into a color picture, it must be "de-mosaiced." You take only the green pixels, and interpolate values into the places where the red and blue pixels were; you take the blue pixels, and interpolate values into the red and green areas; and same for red. There are lots of different, often computationally intense ways of demosaicing Bayer-filtered images to turn them into color.

Some of the guys at unmannedspaceflight.com have had a go at demosaicing these raw images. Here's two example results, one of them with the color left alone, the other processed to bring out details. There's something I particularly like about the second one -- it looks like a vintage postcard! But they're also a little bit weird looking, because there's still a checkerboard pattern of weird green and purple squares on top of the image. Why is that? (No, they haven't been run through Instagram filters.)

Demosaiced version of a Mastcam-100 image of Mt Sharp

NASA / JPL / MSSS / Damia Bouic

Demosaiced version of a Mastcam-100 image of Mt Sharp
Telephoto view of the flank of Mount Sharp, Curiosity sol 17

NASA / JPL / MSSS / Fred B.

Telephoto view of the flank of Mount Sharp, Curiosity sol 17
Curiosity took this photo of the flank of Mount Sharp on Sol 17 (August 23, 2012). It was returned to Earth without compression, so it needed to be demosaiced to reveal its color.

It has to do with how the images were posted to the Web. They came down from the rover in a sort of camera raw format, with actual pixel values, with lossless compression. But before they were posted to the Web, they were saved as JPEGs. The JPEG compression algorithm is a "lossy" one -- it changes the values of pixels in ways that help it to reduce the file size. It works on 16-by-16 blocks of pixels. When there are sharp boundaries between colors in JPEG-compressed images, you often see artifacts that have a square or blocky shape. And since this entire photo has every fourth pixel considerably darker than the adjacent ones, the JPEG compression resulted in edge artifacts on those dark pixels, which means they leaked into adjacent red and green pixels, and caused wonky effects during demosaicing.

Unless JPL decides to change the way they share these particular kinds of images (by, say, saving them in PNG or GIF format, which would be lossless), any color images produced from processing these JPEGged versions of the uncompressed Mastcam pictures are going to have these weird blocky color artifacts.

Even with those artifacts, though, they are inexpressibly cool. We have never, ever seen a landscape like this on Mars, at least not from the ground. And we're going to be driving closer to it, much closer, and even in to some of this canyon country. Let's get rolling!

 
See other posts from August 2012

 

Or read more blog entries about: pretty pictures, amateur image processing, Mars, Curiosity (Mars Science Laboratory)

Comments:

z3n0n: 08/24/2012 08:40 CDT

The GRGB pixels could be sorted into four separate quarter fields before they are compressed with JPEG which would reduce the mixing of the images that results in the creation of a color image. If Curiosity could do this it would reduce the bandwidth needed to check out the CCD for bad spots and product a usable result at the same time.

Astroboy: 08/25/2012 09:53 CDT

Since the rover could potentially operate at night, it would be cool to see a wide field, 15-30 second exposure of some of the constellations framed by Mt. Sharp and the crater rim. Perhaps other shots of the planets or the Martian moons, too. Be creative JPL/NASA!

Leave a Comment:

You must be logged in to submit a comment. Log in now.
Facebook Twitter Email RSS AddThis

Blog Search

JOIN THE
PLANETARY SOCIETY

Our Curiosity Knows No Bounds!

Become a member of The Planetary Society and together we will create the future of space exploration.

Join Us

Featured Images

Caprock and sand ripples, Curiosity sol 679

Curiosity drives over a ripple, sol 683
Curiosity from HiRISE on June 27, 2014
Tracks in and out of a dune, Curiosity sol 674
More Images

Featured Video

View Larger »

Fly to an Asteroid!

Travel to Bennu on the OSIRIS-REx spacecraft!

Send your name

Join the New Millennium Committee

Let’s invent the future together!

Become a Member

Connect With Us

Facebook! Twitter! Google+ and more…
Continue the conversation with our online community!