Saturday, August 23, 2014

Digitizing Photographs

This is step 8 of 10 in the tutorial Collecting 3D shape data using StereoMorph

The StereoMorph package provides a new, easy-to-use digitizing application for collecting landmark and Bézier curves from photographs. Even if you don’t use StereoMorph for collecting 3D landmarks and curves, you might find the digitizing app useful for collecting 2D data from photographs. The app runs in a user's default web browser. Safari, Chrome and Opera all provide full compatibility with the app (Firefox does not allow some features of the app and the app is not tested for Internet Explorer). Although the app runs in a browser you do not have to be connected to the internet to use it. The app runs on a local server, with the R package R package shiny handling communication between the browser and R console. In this step, I’ll provide an overview of how to get started using the app.

StereoMorph Digitizing Application.
1. Load the StereoMorph library into the current R session, if not already loaded and ensure that the StereoMorph Tutorial folder is your current working directory (see Getting Started).

> library(StereoMorph)

2. Call the function digitizeImage() with input parameters specifying the image(s) to be digitized, where the landmarks and curves should be saved and the names of the landmarks and curves to be collected. Using the files in the StereoMorph Tutorial folder, the function call would look like this:

> digitizeImage(
   image.file = 'Object Images',
   landmarks.file = 'Landmarks 2D',
   control.points.file = 'Control points 2D',
   curve.points.file = 'Curve points 2D',
   landmarks.ref = 'landmarks_ref.txt',
   curves.ref = 'curves_ref.txt')

The first argument, image.file, is the only required argument. It specifies the file path of the image or images to be digitized. Here it’s a folder that contains all the object images in the tutorial folder. You can also input a vector of file paths if you only want to digitize a particular set of images. If you input several photographs into digitizeImage() you can switch from one photo to the next within the app.

The argument landmark.file is where the landmarks will be saved to (or loaded from if some landmarks have already been digitized). If you input a folder, the filenames of the landmark files will be saved with the same as the image names, only with the extension “.txt”. If you want different names for the landmark filenames, you can input the file names as a vector.

There are two curve file inputs. The control.points.file specifies where to save the control points. These are the points (the red points in the image below) that you can drag around to define and adjust the curve shape. The curve.points.file specifies where to save the points that actually make up the curve. These are the several hundred points at single pixel spacing (the blue points in the image below) that describe the actual curve and can be used in subsequent analyses.


A 3-point Bézier curve with control points in red and curve points in blue.
The landmarks to be digitized are input via landmarks.ref. In the example above, we’ve input a file path ( landmarks_ref.txt) with the landmarks listed as a single column, each separated by a new line.

     cranium_occipital
     basioccipital_proc_ant
     basipterygoid_proc_ant_L
     basipterygoid_proc_post_L
     ...

landmarks.ref can also be a vector of landmark names, rather than a file path.

The curves to be digitized are also input as a file path (curves_ref.txt). In curves_ref.txt, the curves are in the form of a three-column matrix. The StereoMorph Digitizing App assumes that all curve start and end points are also landmarks. So the curves to be digitized are defined using a matrix where the first column is the name of the curve, the second column is the start point/landmark and the last column is the end point/landmark.

tomium_L     upperbeak_tip         upperbeak_tomium_prox_L
tomium_R     upperbeak_tip         upperbeak_tomium_prox_R
orbit_L      preorbital_proc_L     postorbital_proc_L


For instance, in the above curve reference matrix the first curve is tomium_L, which starts at the landmark upperbeak_tip and ends at the landmark upperbeak_tomium_prox_L. curve_ref can also be a matrix of reference curves, rather than a path to the file containing the matrix.

Once you call digitizeImage() the digitizing app will open in your default web browser, displaying the first image in the Object images folder.
StereoMorph Digitizing Application after loading first image.
The left two-thirds of the window are the image frame. This is where you can navigate around the image and add landmarks and curves using your mouse or trackpad. The right two-thirds is the control panel, with which you can view and save landmark/curve lists and navigate between different images.
StereoMorph Digitizing App control panel.
The ‘Info’ tab contains some basic information about the image, the ‘Landmarks’ tab contains the list of landmarks and the ‘Curves’ tab contains the list of curves.

3. You can navigate around the image in the image frame using the same basic mouse actions as in Google Maps. Position your cursor somewhere over the image and scroll just as you would scroll up and down a web page (using either the scroll wheel of a mouse or the scroll feature on a trackpad). This will zoom in and out of the image. To more quickly navigate around the image, the zoom tracks the position of your cursor and zooms in and out of particular region of the image based on the current position of your cursor. For instance if you wanted to zoom in to the bottom-left corner of the image, you would position your cursor in the bottom-left of the image and scroll in the appropriate direction. This will increase the size of the image while simultaneously positioning the bottom-left corner of the image into the center of the image frame.

Zoom into the object for precise landmark positioning.
4. Click and drag somewhere on the image. This will cause the image to move with your cursor. Just be sure that your cursor is not over a selected landmark or control point - this will cause the marker to move rather than the image.

5. Click on the ‘Landmarks’ tab in the control panel. This lists all of the landmarks input via landmarks.ref and their corresponding pixel coordinates.

6. To add, move or delete a landmark, you must first select that landmark. Click on the corresponding row in the landmarks table or, if the marker is already digitized, double-click on the landmark itself with the mouse (pressing the letter ‘x’ while the cursor is over the landmark in the image frame is a keyboard shortcut).

Selecting a landmark with the cursor. Once selected, the corresponding row will become bold.
7. Once the landmark is selected, set its location by double-clicking anywhere over the image (or positioning the cursor and pressing ‘x’). You can then move the landmark by clicking-and-dragging it or using the arrows on the keyboard.

Move a landmark by clicking-and-dragging with the mouse cursor.
8. To delete a landmark, just type ‘d’ once the landmark is selected. Landmarks with ‘-’ values in the landmarks table will be ignored when saving.

9. Click on the ‘Curves’ tab in the control panel to access the digitized Bézier curves. This lists all of the curves input via curves.ref and their corresponding pixel coordinates, including the starting and ending control points (both are also treated as landmarks) and all the control points in between that define the Bézier curve. The app treats curves with more than three control points as Bézier splines (a string of Bézier curves, joined at the ends).

The Curves tab lists the start, end and middle control points for every Bézier curve.
The control points can be added and moved in same way as the landmarks: by selecting the control point, double-clicking somewhere in the image frame and then clicking-and-dragging or using the arrow keys to reposition. Control points can also be deleted by pressing ‘d’ while the point is selected.

Bézier control points can be moved by clicking-and-dragging with the cursor or using the arrow keys.
10. Once you have collected landmarks or curves, press one of the three submit buttons in the control panel to save them. ‘Submit Landmarks’ only saves the landmarks and ‘Submit Curves’ only saves the curves (both control points and curve points). ‘Submit All’ saves both the landmarks and curves. This saves them to the location you specified in the input to digitizeImage().

Use the 'Submit' Buttons in the bottom of the control panel to save the landmarks and curves.
If a curve.points.file was included, R will also save a file with all of the curve points. They will look something like this,

pterygoid_crest_R0001 1140 1672
pterygoid_crest_R0002 1140 1671
pterygoid_crest_R0003 1140 1670
pterygoid_crest_R0004 1140 1669
pterygoid_crest_R0005 1140 1668
...

at single pixel spacing and with numbers added after the curve name to indicate the point order. These are the points we’ll use in the next step to reconstruct the curves into 3D.

11. Click ‘Next Image’. This will move to the next image if more than one image or a folder of images is specified in image.file.

Move to the previous or next image using the buttons at the bottom of the control panel.
Although, this tutorial uses the full features of the app, you can also use the app to digitize only landmarks or only curves. To only digitize landmarks, call digitizeImage() with only a landmark file and landmark reference list.

> digitizeImage(
   image.file = 'Object Images',
   landmarks.file = 'Landmarks 2D',
   landmarks.ref = 'landmarks_ref.txt')

You can also call digitizeImage() with just image.file if you’d simply like to check the pixel coordinates of a few features in a photograph.

> digitizeImage(image.file = 'Object Images')

For more detailed instructions on how to use the StereoMorph Digitizing App, check out the digitizeImage() function in the StereoMorph package manual or click on the ‘Help’ link at the top right corner of the application window. The help file will open in a new browser window and has information on all the features available in the digitizing app, including keyboard shortcuts.

Opening the digitizing app help file.
The next step will cover how to reconstruct the digitized landmarks and curve points in 3D.

Go back to the previous step: Photographing an object

No comments:

Post a Comment