Friday, April 4, 2014

Measuring checkerboard square size

This is step 3 of 10 in the tutorial Collecting 3D shape data using StereoMorph

In order to use the checkerboard pattern we made in Creating a Checkerboard Pattern to calibrate cameras in stereo and make accurate measurements of objects in 3D, we need an accurate measure of the printed checkerboard square size in real-world units (e.g. millimeters). This can be accomplished in two ways: (1) calculate the printed size from the printing resolution (dots per inch, or DPI) and scaling and (2) photograph the checkerboard with a ruler and manually digitize points along the ruler to scale the checkerboard to real-world units. This section will demonstrate how to measure square size using both methods.

Measuring square size from DPI and scaling

When a printer prints an image, it uses the size of the image in pixels, the DPI (dots per inch) setting for that particular image and the user-specified scaling to reproduce an image on paper. These three quantities can be used, with reasonable accuracy, to calculate the size of the printed image in real-world units.

Image size in pixels * (1 / DPI) * Scaling * (25.4 mm / 1 inch) = Image size in mm

We can use the same formula to calculate the size of the squares in a checkerboard pattern by replacing image size with the square size in pixels.

1. Get the square size in pixels. This is the square.size parameter input to drawCheckerboard() when creating the checkerboard image (for the tutorial calibration pattern this was 200px).

2. Find the image DPI. This can usually be found by looking at the properties of the image in the program you are using to print. For example, in Preview, the image DPI can be found in the Inspector (Tools > Show Inspector).

Looking up the image DPI in Preview.
Unless you have modified the DPI manually, the image should have a default DPI of 72 pixels/inch.

3. Get the scaling at which the checkerboard was printed. The scaling is set in the print dialog box for the program you are using to print the image. For the tutorial calibration pattern this was 9%.

Setting the print scaling in the Print dialog box.
4. Enter these values into the formula to calculate the square size in millimeters.

200 px * (1 / 72 px/in) * 0.09 * (25.4 mm/in) = 6.35 mm

A 200 pixels-per-square checkerboard, scaled to 6% was used in this tutorial to test the calibration accuracy. We can also calculate the size of these squares in real-world units.

I’ve found that, for some printers, these formulas provide an accurate measure of the printed square size. I used two different printers for this tutorial: an HP Color LaserJet 4700dn to print the 9% checkerboard and an HP Deskjet F4200 to print the 6% checkerboard. Measuring the square size of the 6% checkerboard using a ruler (see the following section) returned nearly the exact same value as the formula above (4.2327 mm). However, ruler measurements for the 9% checkerboard yield a square size of approximately 6.365 mm, 0.015 mm greater than is predicted from the formula above. The actual square size is likely 6.365 mm since this provides the lowest error in combination with a test calibration square size of 4.233 mm. My best guess is that the accuracy of the above formulas will vary from printer to printer, emphasizing the importance of confirming the square size using a ruler as detailed in the following section.

Measuring square size using a ruler

For users who want to verify that their printer is producing checkerboards at the expected size, this section will demonstrate how to measure the checkerboard square size using a ruler.

Materials needed for this section:
  • Ruler (required precision will depend on the application)
  • A printed checkerboard pattern

1. Locate a good ruler. For this tutorial and for my work, I use a 12" Single "A" - #46-IM precision rule from Schaedler (approximately $30, including tax and shipping).

12" Single "A" - #46-IM precision rule from Schaedler.
These rules have an accuracy tolerance of better than 0.00024”. I haven’t tried a calibration with a standard office ruler or the rulers that come with dissection kits. Either might work just as well, but if you’re concerned about accuracy a precision rule might be a worthwhile investment.

2. Take a photograph of the ruler and the checkerboard pattern so that they are both visible in the same image.

A photograph of a checkerboard pattern and a ruler.
Nikon D5000 with AF-S DX Nikkor 18-55mm lens at 55mm, f/36.
It is essential that your camera lens does not have any significant lens distortion. For the above image, I used a Nikon D5000, which came with an 18-55 mm zoom lens, a standard lens often sold with Nikons. If you use the 18-55 mm lens at 18 mm (completely zoomed out), you get barrel distortion.

Checkerboard and ruler photographed at 18 mm focal length. Notice the curvature of the ruler which should be perfectly straight.
Nikon D5000 with AF-S DX Nikkor 18-55mm lens at 18mm, f/36.
At 55 mm, distortion is insignificant; in this tutorial all stereo camera images were taken with an AF-S DX Nikkor 18-55 mm lens at 55 mm.

Also be sure that the entire checkerboard pattern falls within the image. The checkerboard should be positioned approximately coplanar with the image plane or the end of the camera lens (i.e. the shaft of the camera lens should be at a right angle to the checkerboard). If the checkerboard is at an angle relative to the image plane, some squares will be closer to the image plane than others, resulting in a difference in size on the imaging plane (this is the perspective effect). For the same reason, the ruler should be in the same plane as the checkerboard pattern. If the ruler has some depth to it, raise the checkerboard so that it is coplanar with the points you’ll be digitizing on the ruler.

3. Upload the photograph of the checkerboard and ruler to your computer. The StereoMorph Tutorial folder ‘Measure square size’ has photographs of the two checkerboards used in this tutorial with a ruler. For demonstration, we’ll use the checkerboard scaled to 6%.

4. Load the StereoMorph library if it isn’t already loaded and ensure that the StereoMorph Tutorial folder is your current working directory.

> library(StereoMorph)

5. Specify the file path(s) of the image(s) or a folder containing the image(s).

> image.file <- 'Measure square size/Checkerboard 6p.JPG'

6. Specify where to save the corners and verification image(s).

> corner.file <- 'Measure square size corners/Checkerboard 6p.txt'
> verify.file <- 'Measure square size verify/Checkerboard 6p.JPG'

7. Specify the number of internal corners in the checkerboard.

> nx <- 21
> ny <- 14

8. Call findCheckerboardCorners() to find the internal corners in the image.

> corners <- findCheckerboardCorners(image.file=image.file, nx=nx, ny=ny, corner.file=corner.file, verify.file=verify.file)

To measure the square size in real-world units, we need to manually digitize several points at equal intervals along the ruler.

50 points digitized along a precision rule at 1 mm spacing (zoomed in).
This can be done easily with StereoMorph’s digitizing application. We’ll use the basic features of the digitizing application to do this – a more thorough introduction to the digitizing app will come in a later section (Digitizing Photographs).

9. Specify where to save the manually digitized points (“landmarks”) along the ruler. We’ll start with an empty (and not currently existing) file.

> landmarks.file <- 'Measure square size ruler/Checkerboard 6p t.txt'

10. Specify a text file containing a list of the points to be collected. In this case, this is simply a list of names for an ordered point set. The file ‘Ruler points mm.txt’ in the StereoMorph Tutorial Folder contains a list from ‘mm_pt001’ to ‘mm_pt200’. You can collect less than 200 points or you can add additional names to collect more.

> landmarks.ref <- 'Ruler points mm.txt'

This input parameter can also be a vector.

> landmarks.ref <- paste0('mm_pt', formatC(1:200, NULL, 3, "d", "0"))

11. Call digitizeImage() to launch the StereoMorph digitizing app.

> digitizeImage(image.file=image.file, landmarks.file=landmarks.file, landmarks.ref=landmarks.ref)

The app should launch in your default web browser.

The StereoMorph digitizing app with the checkerboard and ruler.

The left portion of the app is the image frame. This is where you can navigate around the image and add the ruler points using your mouse or trackpad. The right two-thirds is the control panel, where you can view and save the points you’ve digitized.

12. Click on the ‘Landmarks’ tab in the control panel, if this is not already selected. You’ll see a list of the ruler point names with two columns (‘x’ and ‘y’) where the digitized points will be added. The first point should be selected (indicated by bold) by default.

13. Position your cursor somewhere over the image near the top-right corner and scroll either up or down. By scrolling either up or down (the direction will depend on your customized system settings) you can zoom in or out of the image (similar to Google Maps). To more quickly navigate around the image, the zoom tracks the position of your cursor and zooms in and out of particular region of the image based on the current position of your cursor. So, with the cursor near the top right, the size of the image increased while simultaneously positioning the top-right corner of the image closer to the center of the image frame.

14. Click anywhere on the image (single click) and drag. This causes the image to move with your cursor. By scrolling and clicking and dragging you can navigate around the image.

15. Point your cursor at the end of one of the ruler lines just outside the edge of the checkerboard, opposite the numbers (e.g. at the 5 cm line) and double-click or press the ‘x’ on the keyboard.

Double-click or press ‘x’ to digitize a point at place indicated by the cursor.

This will cause the selected point (here, ‘mm_pt001’) to be added at the current location of the cursor. By default the app automatically advances to the next point (here, ‘mm_pt002’). This can be turned on and off by pressing shift+a but we’ll keep it on for now because it makes collecting several sequential points faster. The first ruler point was probably not positioned exactly where we want it, so we'll re-position it.

17. Press ‘p’ on the keyboard to select the previous point (‘mm_pt001’) or select it by clicking anywhere in its row in the landmark tab. The point will change from blue to green.

16. Use the arrows on the keyboard to move the point up, down, left and right at single pixel increments. To be as precise as possible, it’s important to position the points consistently along the ruler and the bottom left or right edge of the line is easy to use as a consistent landmark.

17. Once the point is re-positioned to the bottom edge of the ruler line, press ‘n’ to advance to the next point.

18. Position the cursor at the next millimeter tick mark (those with numbers above them) and double-click or press ‘x’ to add the second point.

Positioning a second ruler point.
This might also need some repositioning.

19. Place the cursor over the second ruler point and either double click or press ‘x’. This is a third way in which to select a point.

20. Use the arrow keys to re-position the point more precisely.

If you were digitizing an image with your own checkerboard, you would repeat these steps across the image, collecting around 100 points. Several points will provide a large sample of measurements, essential for achieving high accuracy. Additionally, since it is only possible to manually digitize a point to pixel resolution, digitizing several points provides a means of measuring the distance to sub-pixel resolution.

Digitized points can be deleted by pressing the ‘d’ key. Any points in the landmark table with coordinates (–, –) will be ignored when saving.

21. Click the ‘Submit Landmarks’ button to save the current landmarks. If you open this file (e.g. ‘Checkerboard 6p.txt’ in the ‘Measure square size ruler’ folder), you’ll see each digitized point as a separate row in a matrix.

mm_pt001    4317    451
mm_pt002    4278    450

If you need to go back and edit these points or add new points, specify the same file as the file to save landmarks to and these points will be loaded into the app.

22. Click the ‘Exit App’ button to close the app and return to the R console.

23. We’ll load in a file of already digitized ruler points for this image. Change the save-as file path for the ruler points to the ‘Checkerboard 6p.txt’ file in the ‘Measure square size ruler’ folder of the tutorial folder.

> landmarks.file <- 'Measure square size ruler/Checkerboard 6p.txt'

24. Then, re-launch the app.

> digitizeImage(image.file=image.file, landmarks.file=landmarks.file, landmarks.ref=landmarks.ref)

Points digitized manually along the ruler used to measure the checkerboard square size in real-world units.

The 115 digitized ruler points at 1mm spacing in ‘Checkerboard 6p.txt’ will be loaded into the image frame. These are the points we’ll use to measure the square size.

25. Click ‘Exit App’ to close the app and return to the R console.

26. Before measuring the square size, specify the size of the interval we’ve digitized on the ruler in real-world coordinates.

> <- '1 mm'

27. Call measureCheckerboardSize().

> measure <- measureCheckerboardSize(corner.file=corner.file, nx=nx, ruler.file=landmarks.file,

28. Lastly, call summary() on the output.

> summary(measure)
measureCheckerboardSize Summary
   Checks for planarity of checkerboard
      Mean length of opposing sides: 3301.76 px x 2138.61 px
      Difference in length of opposing sides: 1.42 px x -3.1 px
   Simple checkerboard model fit
      Square size in pixels: 165.0367 px
      Mean distance of points from model: 0.64 px +/- 0.5
   Ruler points model fit
      Distance between points on ruler: 38.99043 px
      Mean distance of points from model: 0.73 px +/- 0.58
   Real-world units
      Square size in real-world units: 4.232748 mm
      Real-world units per pixel: 0.02564732 mm

This reports the output of several different operations performed by measureCheckerboardSize(). The function first tests the planarity of the checkerboard relative to the image plane. If the checkerboard is tilted in any direction, opposing sides will differ substantially in their length. The differences in lengths of opposing sides shouldn’t differ by more than a few pixels. measureCheckerboardSize() can be called without the ruler parameters to check the corners for planarity before digitizing any ruler points.

Rather than simply finding the distance between corner points, the function fits a transformed grid model to all of the points. This model accounts for the perspective effects of grid positioned in 3D space and imaged on a 2D plane. Fitting a model to the points ensures greater robustness in the side measurements, taking into account the position of all the internal corners rather than simply the four corners.

The function then fits a simple checkerboard model to the points, using the minimal number of parameters necessary to define a 2D square grid. The model goodness-of-fit is reported as the mean distance of the input points from the model-generated points. This should be less than a pixel. For the 6% percent checkerboard above the input points differ from the model-generated points by 0.64 pixels on average, indicating that the corners extracted from the photograph very closely resemble a perfectly planar grid. The checkerboard square size is then calculated from the model fit parameters. As opposed to simply averaging the distance between consecutive points (which leads to a biased estimate), the model fitting provides a robust, best-fit solution to the square size. Here, the best-fit estimate of the square size is approximately 165 pixels.

Lastly, the function measures the ruler interval in pixels, again using a model fitting procedure to find a best-fit solution. Only in this instance, the model is a line with points at equal intervals. Here, the best-fit distance between ruler points is 38.99 pixels. The goodness-of-fit is reported as the mean distance of the input ruler points from the model, as with the internal corners.

The square size in real-world units can then be calculated as the square size in pixels multiplied by the input parameter, divided by the ruler interval in pixels.

Square size in pixels * ( / Ruler interval in pixels) =
Square size in units

The square size in the same units as is reported on the second to last line of the summary (and returned as measure$square.size.rwu) along with the size of each pixel in the photograph in real-world units. For this image, each pixel in the plane of the checkerboard is about 26 x 26 microns.

By digitizing many points along the ruler and hundreds of corner points at subpixel resolution, we’re able to measure the square size with an accuracy exceeding even the pixel resolution of the image. There is a certain amount of error in measuring the square size using a ruler. I’ve found that measurements made with a ruler can differ among repeated tests by up to 10 microns. Note that for this printer, measurements using a ruler yield a square size, 4.232748 mm, nearly identical to that calculated from the printer resolution and scaling, 4.23333 mm; this may not always be the case. Repeating these steps for the 9% scaled checkerboard (printed using a different printer), I found the square size to be 6.365 mm, 0.015 mm greater than is predicted from calculations simply using DPI and scaling. In general, measuring the square size using a ruler provides the highest accuracy and these measurements can be confirmed by calibrating a set of cameras using a checkerboard of one size and testing the accuracy with a checkerboard of a different size.

Go to the next step: Arranging the cameras
Go back to the previous step: Auto-detecting checkerboard corners

No comments:

Post a Comment