Join Donate

Emily LakdawallaSeptember 1, 2010

Getting to the real science image data: It's not that hard!

If I have to, I will drag reluctant people one at a time to plunge into NASA's Planetary Data System. The PDS, as I've mentioned before, is the repository for space missions' science data; it's the public clearinghouse, the Library of Congress for data, into which every person on Earth is invited to explore the solar system. Few people on Earth do so, however, because the PDS is designed for scientists' use and so is intimidating and sometimes even incomprehensible to amateurs. But, every day, the folks at the PDS' various Nodes are working diligently to make my previous sentence less and less true, and I am trying to encourage more people to find the gems that are hidden within the archives using online search tools.

Over the last few days on I've been having a constructive argument with Cumbrian Sky blogger Stuart Atkinson about this topic, and I'm happy to say that I've finally won, dragging him -- kicking and screaming all the way -- into the PDS, as this blog entry shows. Hopefully, by recounting the tale, I can drag a few of you readers along too.

For six years Stu has done lots of work with the rovers' raw images, most recently using them to illustrate his chronicle of Opportunity's progress toward Endeavour crater, which is great; that's what the raw images are for, to allow us to ride along on the journey. But they have drawbacks.

To explain: raw images are posted to the Mars Exploration Rover mission website within hours of their acquisition on Earth. We call them "raw" because they are, as space images go, very undercooked; virtually nothing has been done to them to clean up the blemishes that result from dust on the camera optics and other sources of noise and distortion. To make them easier for the public to read, they are converted to JPEG format (which makes file sizes small, but which reduces the quality of the images, blurring sharp edges and fuzzing colors). They are also "stretched" -- the contrast of the images, which may start out very subdued, is jacked up to make the lightest pixels pure white and the darkest pixels pure black. These make the images unusable (or, if not unusable, very poor) for science, while maintaining their utility as snapshots from the mission -- a perfect compromise to allow the public to follow along with what's going on, while permitting the mission scientists their few months of proprietary use of their hard-won data.

But it's a compromise, and it does seriously affect the quality of the images. To illustrate this point, here's a blink animation comparing the same Opportunity photo in its raw and archived versions.

'Raw' and archived Mars Exploration Rover images: what's the difference?

NASA / JPL-Caltech / Cornell / color mosaic by Emily Lakdawalla

'Raw' and archived Mars Exploration Rover images: what's the difference?
This blink animation compares two versions of a photo captured by Opportunity of the meteorite named Mackinac Island on sol 2037. One is "raw" -- the version made available on the Internet within hours of its return to Earth -- and the other is "archived" -- the version made available through NASA's Planetary Data System more than six months later. Before they are posted to the Internet, two operations are performed on the raw images. First, the contrast is stretched, so that the brightest few percent of pixels are set to be totally white, and the darkest few percent of pixels are set to be totally black. As a result of the contrast stretch, the highlight glints off of the metallic surface of the meteorite are lost, since brightest areas of adjacent rock are set to similar pixel values; at the same time, detail visible in the shadows of the archived image are lost in the raw version. Second, the images are converted to JPEG format, which reduces their file size substantially but also reduces the quality of the images. The JPEG conversion tends to add noise near sharp edges, resulting in a loss of detail.

So when Stu posted some work he'd done stacking and processing some older images, taken in October of last year. I objected: why do all this processing work with the cruddy raw JPEGs? His reply, essentially, was that he didn't know how to get to the archived data, and didn't think he was capable of doing it anyway. I argued he was wrong, and set about determining the path to archived data in a friendly format using the fewest number of clicks and nothing more than a browser. Following that path, Stu immediately got results -- which did prove I was right, but more importantly, proved that someone who didn't think he was capable of dealing with the archived data could, in fact, not only browse and download it, but process it using the skills he already had:

Color images made from raw JPEGs (left) and archived images (right)

NASA / JPL-Caltech / Cornell / color composites by Stuart Atkinson

Color images made from raw JPEGs (left) and archived images (right)
Because of the automated contrast stretch applied to the rover "raw" JPEG images, color images produced from them can look odd. In particular, the brightest and darkest areas of any image often have a color cast to them that is not real; bright areas often look yellow, and dark areas often look blue. Also, unusually colored things covering small areas of the original image -- like the glints off the surface of the metallic meteorite -- are simply invisible in the raw versions.

Here's the path I led Stu down: For the rovers, it requires as few as four clicks of the mouse and, optionally, a typed number.

That's it! You've got archived data. The versions you can get through that browse-and-click interface do have an automatic contrast stretch applied, but it's not nearly as aggressive as the one that's used on the raw images website. If you want images without a contrast stretch, you'll have to use the little download links, download images in IMG format, and figure out how to process them, which is not very hard but which is not something I'll make you newbies do today. Because you're at the Imaging Node, a similar set of clicks will get you straight to lots of other spacecraft image data. Try it out and explore the solar system! I suggest checking the Viking Orbiter archive of Phobos or Deimos images, or try Cassini images of Calypso or Phoebe, or LCROSS images of its departing Centaur stage, etc....

A few tips and tricks you can do with the Planetary Image Atlas Search Tool for rover images:

Why do I care so much about this? Because the pictures speak for the missions, and the archived data just looks so much nicer than the raw JPEGs. Use the raw images to follow the everyday happenings on the mission -- but use the real data, please, if you want to put the rovers' best wheels forward.

Read more: pretty pictures, Opportunity, data art (was amateur image processing), Mars Exploration Rovers, Mars, explaining image processing

You are here:
Emily Lakdawalla 2017 headshot square serene
Emily Lakdawalla

Solar System Specialist for The Planetary Society
Read more articles by Emily Lakdawalla

Comments & Sharing
Bill Nye and people
Let's Change the World

Become a member of The Planetary Society and together we will create the future of space exploration.

Join Today

The Planetary Fund

Help advance robotic and human space exploration, defend our planet, and search for life.


"We're changing the world. Are you in?"
- CEO Bill Nye

Sign Up for Email Updates