Login or register:
Login:
Password:
Register!


Imageserver Favorite:


Portfolio Galleries:
Animals
Architecture
Flowers
Landscape
People
Wildlife

Recent Subjects:
Landout
5-inch sky robot
Vacuum printing
e-bike
Farnsworth Fusor


Search:


Resources:
ImageServer

Friends:
Jon Sullivan
Mark Blair


New Articles:

Southern California Glider Landout Database
Soaring
5-Inch Sky Robot
FDM in Vacuum
Gin
Books
Hacktastic e-bike
Farnsworth Fusor
Land Anchor
Turchickentato
Bowmaking
Custom Discovery Roof Rack
August, 2012 - Sierra Backpacking
July 22nd, 2012 - Hiking in Santa Barbara
July 15th, 2012 - Valley of Fire
May 5th, 2012 - Afton Canyon
April 28th, 2012 - Devils Playground
Alpine Tripod
Convict Creek Trail
January 2012 - Mustangs
January 3rd, 2012 - Heart Lake
August 7th, 2011 - Kelso Dunes
August 5th, 2011 - Lundy Canyon Hike
August 2011, Mammoth Archery
Birds in the garden
June 4th, 2011 - San Gorgonio
May 29th, 2011 - Sequoia National Forest
April 23rd, 2011 - Living Desert
April 2nd, 2011 - Death Valley
March 8th, 2011 - Mountain Palm Springs
February 13th, 2011 - Iron Smelt
Blacksmithing
November 13th, 2010 - Mojave
October, 2010 - Mammoth
Android
September, 2010 - Mammoth
September, 2010 - Duck Lake Trail Backpacking
Red
Iron bloom forging
August 28th, 2010 - Mt. San Jacinto
OSM Import: US Designated Wilderness
July 25th, 2010 - Mojave Mustangs
July 17th, 2010 - Mojave Exploration
Bloomery furnace iron smelting
Open Street Map: Mojave Project
June 13th, 2010 - Mojave
June 6th, 2010 - El Cajon Trails
Wolf Mountain Sanctuary
Carrizo Gorge
March 28th, 2010 - Salton Sea
March 21st, 2010 - South Main Divide
March 13th, 2010 - Anza Borrego Wildflowers
March 7th, 2010 - CSULB Japanese Garden
February 2010 - Mammoth
GeoRSS
AISlib
OpenStreetMap
Dakota and Asha Celebrate Christmas, 2009
November 21st, 2009 - Mojave Road
November 14th, 2009 - Anza Borrego
Exploring The East Mojave: The Afton Canyon Area
Broken flex plate
Remote Image Serving
Astro/night photography in Inyo National Forest
Wild Mustang Sightings
RSS
September 26th, 2009 - Night Photography In Frazier Park
Whiskey
Brandy
August 15th, 2009 - Catalina dive trip
Astrophotography
Sensornet
January 24th, 2009 - Mojave Exploration
July 2008 Mammoth Vacation
AIS
President Barack Obama!
Rachel Maddow
Big Geek
Barack Obama
April 12th, 2008 - Wildflowers and Landmarks
My Grandfather's Alfa Romeo Spider
March 8th, 2008 - Carrizo Plain
Bridge To Nowhere
High Availability
October 20th, 2007 - Big Bear Camping
October 22nd, 2007 - Fire
Scottish Highlands, Aug 7th, 2006
Scottish Highlands, Aug 6th, 2006
August 5th, 2007 - Duck Lake Trail
May 26th, 2007 - Kelso Dunes
Culloden Battlefield, Aug 5th, 2006
May 20th, 2006 - Irwindale Renaissance Faire
Edinburgh, Aug 4th, 2006
The Clifs of Moher, Aug 3rd, 2006
The Burren, Aug 2nd, 2006
Bunratty Castle, Aug 1st, 2006
May 5th, 2007 - Mojave
Truck Audio/Data Network
2007 - Master Bath Remodel
Centrum: Exit
The Ring of Kerry, Jul 31st, 2006
Nikon D200
Victory in 2006!
Blarney and Killarney, Jul 30th, 2006
Dublin and Cork, Jul 29th, 2006
Dublin, Jul 27th & 28th, 2006
Married!
What Can I Do?
April 30th, 2006 - Anza Borrego
New desktop: Intel 805 D
Macro Photography
Jan 7th and 14th, 2006 - Hiking
Whiting Ranch Hiking
Engaged!
Digital Photography with Linux
September 5th, 2005 - Living Desert
August 19th, 2005 - Mammoth
Hiking and Photography
July 30th, 2005 - Tucker Wildlife Sanctuary
Nikon D70
Death, Fright and Photography
Mmmmmm Eggs
MythTV
Inova T4
May 14th, 2005 - Red Rock
April 2nd, 2005 - Death Valley
Count Every Vote Act of 2005
Image Archiving
Linear Logic ScanGuage
Gentoo Linux
November 6, 2004 - Mojave
Super Tuesday, 2004
John Kerry
Kayaking
Irish Stew
ImageServer
Ireland, 2004
Canon A80 Camera
Jul 25, 2004 - Death Valley
Chronic Hiccups
May 4th, 2004
Landscaping - My Front Slope
Stump Pullin' Yeeee Haw!
Feb 22nd, 2004
Feb 16th, 2004
PostgreSQL Logfile Analysis
Spam
Mountains? Desert? Jan 30th, 2004
Jan 28th, 2004
Encryption
Ceiling Cargo Basket
Front Bumper Version 2
Asha
Exide Orbitals
Land Rover Valve Jobs
Spirits
The Matrix: Revolutions
Halloween 2003
Greg Davis CDL Linkage
Ouzo
Democracy
Mom's Turkey Gravy
Grandma's Guacamole
Top Nodes
Julian Pie Company
DeCSS
The KB1DIG 2-meter Halo Antenna
Incomming searches
Gardening is hard!
Aug 13th, 2003
SQL and Perl
Cancun 2003
Jul 9th, 2003
Aprs intelegence
Jun 17th, 2003
Some People's Comments
Dakota is a silly dog
The Matrix: Reloaded
Chris' Stage Bottle Harness
April 23rd, 2003
Cracked Radiator!
Black Wednesday
DVD Burning Under Linux
My Satellite Phone
Wind!
My Near-Death Experience
Laser Cannon Revival
Front Bumper, Version 2
SpamAssassin
The Critters
Dakota
Milton
KPC 3 Plus and HTX-252
My House
Moving, moving moving...
Mobile 1
Portola Hills
New new house
Suse Linux
Database images
In Truck Dr. Pepper
My Favorite Toilet
Kelso Dunes
Desert Trips
Ifulmuh
Late Thoughts: Dr. Pepper Cooler
265/75-R16 Tires on a DII
Linux
George W. Bush
Rants
Driving
Reservations
Horses Sep 14 2002
Obsession
August 17th Yukon Dives
Less notifications
My Custom Front Bumper
Bracketless, Renamable Links
Discreet Winch
Welding
Jul 28 2002 Day Trip
My Firewall
Jedi Group, my T1 and money
A Bumperless Discovery!
My Custom Rear Bumper
Vanessa's 24th Birthday
Jun 30th Dive to Long Beach Canyon
PHP/PostgreSQL String Quoting
Tonsillectomy, Uvulaectomy and Turbinite Reduction
Searching functionality
240 Watt CO2 Laser Cannon
My Workspace
Dr. Pepper
The Tulsa Rib Company
The quality of hard disks these days
Email notification of articles
Email notification of comments
erikburrows.com source code
User Bios
User Preferences
Login feature
Renisance Faire Jun 9, 2002
Computers hate me, and it is mutual.
Star Wars Sucks!
Horses, Jun 1, 2002
Land Rover Mileage
Insomnia, Robin goes evil.
100 Watt Diode Laser Test Firing 1
Amateur Radio
The Matrix
2001: A Space Odyssey
TDI Deco Class
Horses, Apr 30, 2002
APRS
Movies
Blackbird
My Truck Batteries
My Truck
Vasquez Rocks
The Zope Bible
PSK31
Mojave Apr-12-2002
100 Watt Diode Laser
The New www.erikburrows.com
Hunter the Kitty
Horses
Geeks
Yukon May-13-2001
Computers
Matts Desert Pics
Mojave Feb-10-2001
Mojave Apr-01-2001
Programming Languages
Ironage Jul-4-2001
Mojave Jan-27-2001
Ironage Feb-03-2001
Mojave Jun-09-2001
SCUBA
Jedi Group
Digital Photography with Linux   -   2005/09/15Viewed 196 times this month, last update: 2009/03/13


These days, effective digital photograph processing entails several steps, and usually several tools. This is especially true with more advanced or "fringe" equipment or formats, and given the propensity of camera and software manufacturers to support only the top few computer platforms, we Linux users tend to be left behind. Well, in recent years, Open/Free software has been catching up, and we've gotten to the point of being very effective and efficient. It may not be as easy to set up a good workflow on Linux as it is on Windows, but with a little work, and the right tools, we have the capability.



I've done a lot of research on the subject, and for the benefit of everybody's time, I've compiled a list of helpful tools, and I'll explain how I use them.

Monitor Color Calibration:

The mechanics of downloading/copying the files, processing and converting them, then archiving and printing them is often what people jump to first, but the first step really, is color calibration. After long last, I've finally found a way to calibrate my monitor properly, without installing Windows just to use the couple hundred dollar spider. Norman Koren wrote a great article on monitor calibration, using simple, easy to follow instructions, and without the need for expensive hardware. In just a few minutes, almost any monitor can be calibrated, which means what you see on your monitor will be much closer to what your lens saw.

This step should get your work flow reasonably well color-calibrated up to the printing step. I have not yet found a way to color calibrate a printer under Linux, though if you use a print lab for your printing, then it's less of an issue.

Image Downloading:

To get the image files from the camera into your computer, there are two options:
  1. Use F-Spot, DigiKam or Picasa to download the pictures directly from the camera via USB cable, or from a memory-card reader. This is by far the easiest method, and on modern Linux distributions (Ubuntu is what I use), when you plug in your camera, one of F-Spot or DigiKam will pop up, and ask if you want to download your images.
  2. Use command-line tools and/or scripts to download images. This is now probably the least popular method, but I like to have a tool-agnostic workflow, so this is how I do it.
I use a simple script, which creates a time-stamped directory to put the images in. Your script may vary, and have extras like automatic thumbnail generation or reformatting, but the basic script is:
#!/bin/sh

# The name of the new directory, eg 2005-09-15.11:30
dir=`date +'%F.%R'`

# Make the new directory
mkdir $HOME/images/$dir

# Go into the new directory
cd $HOME/images/$dir

# Mount the flash card/camera
mount /dev/sda1 /mnt/flash

# Copy the files
find /mnt/flash/dcim -type f -exec cp -v '{}' . ;

# Unmount the card
umount /mnt/flash

Processing, Option #1:

Generally, digital cameras produce two types of files, JPEG and RAW. JPEG is a highly compressed format, but is what we call "lossy". It actually throws out some information in order to get the file to compress more. Almost all cameras can produce JPEG files, but to use JPEG files effectively, you will need to experiment some. Your usually have several controls: Size, quality, sharpness, saturation, etc. JPEG files are the smallest (meaning you can fit more on your memory card), and easiest format to deal with, but you will need to experiment with the above settings to see how much quality or size you are willing to lose to get that size advantage. Remember, you can always make your images smaller in the computer, but you can not make them larger again, without losing quality. (Relative to size.) The same goes for sharpness and saturation. You can always sharpen and saturate later, but going backward costs quality.

Many cameras, mostly "prosumer" and "pro" cameras also produce RAW files in addition to JPEG (and sometimes other formats). RAW files are completely unprocessed by the camera, and don't compress very well, so the are large, filling up memory cards faster, and taking longer to download and process. However, they have not lost any information, and often give you a little extra latitude in processing before you start losing information. For instance, you can often adjust the exposure a little without washing out shadows or highlights. (Increasing exposure does add noise though, so there's no free lunch here either.)
To use RAW files on Linux, you will need a converter tool. There are a few, but the one I use is UFraw. It has a command-line interface, and a Gimp plugin, making it very convenient. UFraw uses the DCraw library, which supports several RAW formats, including Nikon and Canon.

After I download my pictures, I use an image viewer like XV to see what pictures are worth processing. XV does not read RAW files of course, so when I'm not shooting in JPEG mode, I use RAW+JPEG mode, so that the camera generates a good quality JPEG along side the RAW file, which saves a lot of time.
After I've selected a few pictures that are good enough to put the effort into processing, I fire up the Gimp. Gimp stands for Gnu Image Manipulation Program, and is an exellent program, often compared favorably to Photoshop. I've never used Photoshop, and I'm sure it's just great, but I love the Gimp, it's very full featured, and easy to use.
JPEG files can be opened directly, RAW files must be opened with the UFraw plugin. When opening a RAW file, the UFraw window pops up, which allows you to adjust some of the image parameters before conversion to 8-bit color for the gimp. My Nikon D70 produces 12-bit RAW files, but almost everybody uses 8 bits per color, as in JPEG files, so a conversion takes place. Obviously, some information gets lost, but by tweaking exposure, white-balance, contrast and saturation, this loss can be minimized.
Once you have your image opened in the Gimp, a world of tools are available for tweaking it. My goal is to take the picture correctly while I'm still looking at it, so I try to reprocess it as little as possible. Generally, this means cropping, noise reduction, color tweaking, and sharpening, in that order. Cropping is easy with the "crop" tool.
Noise reduction is not always needed, and should be used very sparingly. If your image has noise, in my opinion, you didn't take the picture right. You needed either a lower ISO speed, or a slower shutter speed. Noise means you didn't have enough light. Reducing image noise causes you to lose information, so the trick is to minimize that loss. There are many tools to reduce noise, but my favorite is the Dcam Noise plugin. I find that it is very effective at reducing noise, but maintains image sharpness very well. It can also be used to smooth out people's complexions in distant pictures. Color tweaking is much more challenging, and is something you can take a lifetime to learn, but there are many resources out there. Try GimpGuru or The Gimp Tutorials. I try to use the curves tool as little as possible, but just a touch will help a lot. For instance, the picture above needed just a little help in the high-blues.

Last is sharpening. I prefer to do all my sharpening in the post-process, rather than let the camera do it, because as with everything, less is more. Too much sharpening can make the image look weird, or bring out noise. It's best to do it on a big, bright monitor, rather than on the tiny LCD on your camera. The gimp "unsharp mask" filter is the standard. Unsharp mask may sound like it would blur the picture, but what it actually does is apply a "mask" over the parts that are un-sharp, making them sharper. Sharpening with the unsharp mask is kind of like a blunt hammer though, often you want to sharpen just the well-defined edges in your image, and for that you need the smart-sharpener. You can do "smart sharpening" by hand, as described by Eric R. Jeschke here, or you can make your life easier, and use the smart-sharpening plugin.
There is one more step that I sometimes employ. It is called "dynamic range extending". This works only for pictures taken in RAW format, and utilizes that wiggle room in exposure. The problem is that digital camera sensors are have a very small "dynamic range". This means that it's hard for them to expose both very bright, and very dark colors at the same time. Often you have to decide which to expose properly, causing you to loose the other. The dynamic range extender plugin, written by Olli Salonen, who also wrote the smart sharpening plugin, takes two images, layered on top of each other, and combines them, usin g the highlights from one, and the shadows from the other, bringing them both into proper exposure. To use it, switch to RAW mode, and take the picture. (Or several, at slightly different exposures. This is called "bracketing".) Open the best looking picture (best on average) in the gimp. Use UFraw's exposure slider to adjust the exposure until the highlights are exposed correctly, and click "open". Next, open the same picture again. Leave all the other parameters the same, and adjust the exposure until the shadows are exposed properly, then click "open". Now cut and paste one picture into a new layer of the other, and run the dynamic range extender tool. You should end up with a very nice looking, high-dynamic-range picture.

Processing, Option #2
The above set of tools for image processing, namely the Gimp, UFRaw, and Gimp plugins is an effective, complete system. It does have some drawbacks however. First and foremost, it's not very intuitive. This causes it to take up more time than it should, and feels more like programming than painting. For all of the middle-steps, there is another very capable linux alternative: LightZone.
LightZone is not an open-source product, but rather a closed-source, for-profit product produced by Light Crafts Inc. of Palo Alto California. It sells for $200 list, but when you download the 30-day trial version, you'll get emailed a coupon for 25% off. The product installs and runs perfectly on my machine, and is much, much more comfortable to use than the ufraw/gimp workflow. The LightZone forums have also proven to be a very useful resource for questions and techniques.
I strongly recommend trying out LightZone if you're a Linux using photographer, weather or not you're already comfortable in the Gimp environment. The free 30-day trial version is fully capable.
LightZone lacks some of the functionality of the UfRaw/Gimp workflow though. Gimp's smart sharpening and noise reduction plugins are smarter than LightZone's, the gimp can do multi-frame HDR, which LightZone can't, and UfRaw's upcoming integration with LensFun will give it lens correction functionality that LightZone doesn't yet have. Because of this, I'll continue to use both tools, and I'm very happy to have options!
Processing, Option #4
There is a brand-new option: Wine, the windows-API emulator for Linux just released version 1.0, after 15 years of work. Wine 1.0 now supports Photoshop.

Printing:

The last step is printing. This is the real benefit of digital photography. I often print only about one in every fifty pictures I take, which saves an awful lot of money. Printing at home with an ink-jet printer is fine, but for the best quality, I recommend either a print lab, on-line print lab, or, I've found that the "Sony Picture Station" at your local Kinkos/FedEx produces very high quality prints. For us Linux users, printing at home can be a challenge. Step one is to find the best quality print drivers. For my HP PSC-750, this means the drivers that come with CUPS. The easiest to use, and most powerful tool is gimp-print, but it doesn't support as many printers as CUPS. Gimp-print allows you to adjust almost every print parameter, including layout. If you can, use gimp-print. If you can't, like me, I use Xpp. It has as many adjustments possible as gimp-print, but isn't as easy to use, and isn't integrated into the Gimp.

Archiving:

Now you have hundreds of pictures, in nice, neatly time stamped directories, and you want to be able to search for pictures from your last birthday, or vacation. You might be able to guess the date and time, but that's kind of a pain, and only works as well as your memory does. You need some way to describe all your images, and search for them later. I use a program called MaPiVi, which I describe using in another article called Image Archiving.

Geo-Tagging:
90% of my photos are landscapes or travel pictures, so I often like to reference where the photos were taken. Storing positional information within image metadata is known as geo-tagging.
For this purpose, I almost always have a GPS receiver on in my pocket or backpack while I'm out shooting. I take care to synchronize the camera's clock to the GPS time (which is always accurate, as long as it has signal). Then, when I get home, I download my tracklog, and use that to geo-tag my pictures.
The tools I use for this are GPSMan to download the tracklog from my Garmin Rino GPS. I save my tracklogs in GPX format, which is a good open-standard XML-based format.
I then use Happy Camel to geo-tag my images from the GPX tracklog file. I use Happy Camel becuase it's command-line based, so easily scriptable. The photobook manager programs that come with Gnome and KDE also have geo-tagging functionality.
The script I use to invoke Happy Camel is very simple, selecting the most recent tracklog file to use:
#!/bin/sh
happycamel -v 3 -x -t `ls -1 --sort=time ~/tracklogs/*.gpx | head -n 1` $@
This script will take a list of files to geotag, and use the most recent tracklog file to find coordinates.

I hope all this helps, if you would like clarification on any of the steps, or have a suggestion or correction, please comment or email me.

Comments:
nEO (2005-10-16): But it's seem, you don't use any "Color space management"?
If you shoot with raw format, you'd better using color space management and ICC profile.
But gimp seem will not support color space until gimp-2.4.
only lcms can deal with ICC profile now?

Erik (2005-10-17): Color space management is a topic that I would like to explore, but you are correct in that Linux applications, and video and print drivers lag behind in this respect. This is something I will keep up on, and if you find useful information, please pass it on!

newmikey (2005-10-19): UFRAW can handle color profiles which are applied before gamma adjustment. I use the Bibble profiles in UFRAW and the results are very satisfactory.
BTW: Great resource, this site!

Erik (2005-10-20): newmikey, For Nikon owners, UFRAW reads the camera's built-in color profile right out of the NIF file. It would be best to have the monitor's color profile, but using the monitor calibration procedure above, you'll get most of the way there.

Todd M (2005-10-23): The new Photoshop handles RAW files now. I just got it and it's a great program! I use it to process the RAW images from my Canon EOS 20D. The look awesome!

newmikey (2006-06-30): Todd M: do you run Photoshop under Wine (Crossover or other) ?

Couzin2000 (2007-09-14): I just read your article, very cool. The existence of LightZone was unknown to me, and I'M sure to try it soon.
Maybe I can interest you in a VERY awesome program, which is very resemblant to Photoshop (the later versions, like Creative Suite 2): check out http://www.kanzelsberger.com/. You won't regret it!
Pixel is a full-fledged photo editor that handles color profiles, and even though it is a little more expensive than Gimp (35$ right now), it's still lightyears ahead of Photoshop (1000$+). Very much worth the viewing!

Erik (2007-09-18): Cousin2000, Pixel does look like a cool product. It's always good to have more tools in the toolbox.

My name (2008-01-15): Have you more ideas like that?

Erik (2008-01-17): I keep this page updated when I have new ideas.

Craig (2008-07-08): The Decam Noise link is outdated and I could not find this plugin available for download anywhere. What is option # 2 then?

Erik (2008-07-09): Craig, you're right, DCam seems to have totally gone away! I will work to evaluate the remaining tools, but in the mean time, here's what seem to be the best available:

Wavelet Denoise

GREYCstoration

ISO noise reduction




Also note: UFRaw's new AHD interpolator has wavelet noise reduction built-in.

Rax (2008-09-14): Bibble also makes tools that work on Linux. However I have little experience with it.

I have been trying the Lightzone trial recently, and am quite impressed with the quality of the jpegs it produces.
Too bad I can't afford the full version right now :(

Thanks for a good article.

Paul Finnigan (2010-03-02): Colour management is here! Gnome Color Manager is now available from the Gnome GIT. If you run Fedora it is available from the Rawhide repository.

It does the job and allows you to calibrate devices using argyllcms.

No more guesswork. Colour is the same on the screen and printer here!

Erik (2010-03-03): Thanks Paul! As soon as that makes it into a Ubuntu release, I'll check it out! (Really don't feel like compiling Gnome.)

See also: Nikon D70, May 20th, 2006 - Irwindale Renaissance Faire

permalink

Email me!


Erik Griffin Burrows and ErikBurrows.com are not responsible for any damage or loss caused by viewing this site, or actions taken as a result of reading any part of this site. Articles and comments on this site should not be taken as legal, medical, scientific, engineering, botanical, dietary or any other kind of professional direction. This site is not affiliated with any business institution.

You get the idea. This site is just another big pile of misinformed, uneducated, unsolicited opinion, and should be judged as no more truthful or accurate as anything else on the internet.

Don't Panic!