Help Shape the Future of Space Exploration

Join The Planetary Society Now  arrow.png

Join our eNewsletter for updates & action alerts

    Please leave this field empty

Mark Lemmon

Who is the photographer behind Mars rover photos? Answer from Mark Lemmon

Posted by Mark Lemmon

21-12-2012 11:10 CST

Topics: pretty pictures, Phoenix, Mars, Curiosity (Mars Science Laboratory), explaining image processing

Editor's note: I received an email from Miguel Santesmases, a Spanish fine arts student, asking: "Who is operating [the rover] cameras, who shoots the trigger, who directs them to a precise object or mountain, how is all this done? I have been searching the web but it is as if the cameras shoot by themselves, the dream of the old days of photography, the picture made by itself, but I know it has to be someone there, or a team of photographers." I forwarded his question to two Martian camera operators, Mark Lemmon and Jim Bell. Here is Mark's response. --ESL

It would be nice if the pictures took themselves. But it takes a village, it seems, to get a picture taken on Mars.

Technically, you could say that the Payload Uplink Lead who writes the command sequence is the photographer. In that sense, Elaina McCartney at Malin Space Science Systems (formerly at Cornell) has taken more pictures on Mars than anyone else. This person (PUL, or on Curiosity, PUL-2--I'll explain the number below) is as close as you get to someone who "pulls the trigger". The PUL writes the commands, validates them, makes sure they are safe and will achieve the objective, and approves them for uplink and execution. The command sequence is what the rover sees and executes, after format conversion (essentially, compiling).

For some pictures, that adequately captures things. Drive direction mosaics [sets of images taken to plan the rover's next drive] and other routine sequences are executed the same way every time. The PUL takes the desire to have a product and turns it into the commands. There is no aesthetic consideration involved, just a repeatable operational consideration.

Other pictures require a bigger perspective. Someone on the rover science team looks at the routine Navcam survey and says, "Hey, that's cool, we need a color picture." They provide a specific intent for what is in the picture, whether it should be color or use other filters, what resolution and level of compression is acceptable. All of this is mostly done with internet-friendly planning tools. I can designate a target and have everyone see it, and create an activity plan that says what kind of picture I want. Usually, this is simple: there is a target, and the camera will be aimed right at it.

Some images require even more consideration. Aiming straight at a single target works, but some pictures are actually composed in detail. Mostly, this is to help capture multiple interesting things, or an extended target. Mosaics, especially, get this kind of attention. A science team member may draw a box on a Navcam image, or specify azimuth and elevation coordinates, or just specify a center and size.

Lighting is commonly treated very intentionally. Some images are taken whenever they can be. More often, illumination and shadowing is a concern, even to the point of requiring the rover to shut down during the normal operations day in order to be awake in the late evening or early morning.

Finally, some images do get a very detailed attention to composition, with aesthetic considerations and scientific or operational considerations all playing a role. These usually have illumination and shadowing specified, as well as very specific composition. A recent color mosaic across Endeavor crater comes to mind, as does a section of the Phoenix site panorama that showed the Robotic Arm poised above the lander ready for sample delivery. Any sunset or astronomical imaging gets at least this level of detail. The Phobos and Deimos transits were planned over weeks, with dozens of people involved (in that case there was a balance between getting the desired images and a requirement to avoid damaging ChemCam in the process, as it is very sensitive to direct Sun light).

Phoenix mission success panorama (the

NASA / JPL / UA / Texas A & M / color mosaic by James Canvin

Phoenix mission success panorama (the "Peter Pan")
This is James Canvin's version of Phoenix' Mission Success Panorama, which includes 150 separate camera pointings taken from sols 13 to 43. Canvin produces his mosaics using software he developed for Mars Exploration Rover panoramas. The DVD, encoded with the names of more than 250,000 Earthlings who signed up to ride along with Phoenix to the surface of Mars, is located on the lander deck just below the scoop on the end of the robotic arm. (The robotic arm appears disembodied because the lander did not take a photo of the arm higher in the air.)

At the other extreme, the Phoenix SSI is as close as I've been to pictures that take themselves. With the Phoenix operating system, subroutines were stored on board. A single command line from the ground could then take a large multi-filter mosaic; usually it is one command per picture (or stereo pair).

The process of getting Mastcam images is fascinating and fun. As an example, say we've just driven with MSL, and gotten an end of drive Navcam mosaic (courtesy of yestersol's Navcam PUL). Scientists come into the planning process with a dozen ideas of images to get, among other things. The day starts as chaos. Over hours, that chaos becomes a plan, with a down-selected set of images and other activities in a scripted plan that captures all of the intent of the picture-taking.

Over a few more hours, that plan becomes lines of code that will execute on the rover the next day. Several people may participate in designing a mosaic (over telecon, internet chat, email, etc.). At this time, for Curiosity's Mastcam (and MAHLI and MARDI), a first-shift PUL-1 from the science team will be deeply involved in the discussions, and have a good understanding of the product that is desired and the resources (bits, time) it will take to get there. A second-shift PUL-2 from the camera engineering team will come in and eventually write the sequence. The two PULs work together in the middle, and sometimes for a while later, to make sure that the intent is followed to the best of the rover's ability.

A late afternoon M-100 mosaic of Mt. Sharp on sol 45 really tested this--we had driven since the last targeting Navcam to include all of Mt. Sharp. So I worked closely with the PUL-2 throughout the sol (back on Mars time) to design the mosaic to actually get all of this big thing that refused to show up in our planning tools. Every frame was tweaked a few times until we were satisfied. Then, we all go home, and the next day it is chaos-to-code in hours all over again.

Mount Sharp / Aeolis Mons, Curiosity sol 45 Mastcam-100 panorama

NASA / JPL / MSSS / Damia Bouic

Mount Sharp / Aeolis Mons, Curiosity sol 45 Mastcam-100 panorama
Following a drive on sol 45, Curiosity captured a high-resolution panorama with her Mastcam-100 (a 100 mm telephoto lens), of the mountains located at the center of Gale Crater. The view shows a lot of details, and give us the opportunity to contemplate its quasi-infinite layers and the mesas of the right part.

By the way, for a single snap shot you might have the Geology Science Theme Group conceive and design it en masse; the PUL-1 plan it; the entire (on staff) Science Operations Working Group discuss it and include it in the daily plan, the PUL-2 actually write it, and the engineering uplink team review and approve it before the Ace hits the button to radiate it, with the sol's command, bundle to the rover. That's a group the size of a small village.

For my part, when a person puts in effort at composition, especially when it makes the result aesthetic as well as functional, that person is the photographer. But of course, that is what I mostly do, so it is a bit self-serving. I prefer to write command sequences relatively infrequently.

See other posts from December 2012


Or read more blog entries about: pretty pictures, Phoenix, Mars, Curiosity (Mars Science Laboratory), explaining image processing


Aurore de Blois: 12/22/2012 04:22 CST

Thanks for the great article- I am hoping to see a LOT more 360 degree panoramas from Curiosity, especially in Stereo. The panos from Spirit and Opportunity are just incredible but Curiosity is located in an area with more geological points of interest- ones that are going to make for some amazing panoramaic images- lets see those happen!

Bob Ware: 01/02/2013 07:25 CST

Thanks for the great explanation. I hope you all decide to eventually shoot a downhill pan before you come back down or go 'over the hill'. I'm not sure what your plans are.

Leave a Comment:

You must be logged in to submit a comment. Log in now.
Facebook Twitter Email RSS AddThis

Blog Search

Essential Advocacy

Our Advocacy Program 
provides each Society member 
a voice in the process.

Funding is critical. The more 
we have, the more effective 
we can be, translating into more 
missions, more science, 
and more exploration.


Featured Images

BEAM expansion progress
THAICOM 8 drone ship landing
BEAM on Canadarm
Mars near opposition from Pic du midi Observatory, 2016
More Images

Featured Video

The Planetary Post - A Visit To JPL

Watch Now

Space in Images

Pretty pictures and
awe-inspiring science.

See More

Join the New Millennium Committee

Let’s invent the future together!

Become a Member

Connect With Us

Facebook, Twitter, YouTube and more…
Continue the conversation with our online community!