Saturday, April 12, 2014

Reconstructing 2D points and curves into 3D

This is step 10 of 10 in the tutorial Collecting 3D shape data using StereoMorph

This section will demonstrate how to unify 3D landmarks and curve points from several aspects into a single point set, reflect landmarks missing on one side and align the whole set to the midline plane.

Unifying landmarks


If the same object has been photographed in more than one position (aspects), the landmarks and curve points collected from each aspect will be in different coordinate systems.

Aspect 1
Aspect 2
Aspect 3
In order to combine these three aspects, they must be unified based on shared points. This can be done in StereoMorph using the unifyLandmarks() function.

1. Load the StereoMorph package, if not already loaded, and the ‘rgl’ R package for viewing 3D points. Also ensure that the StereoMorph Tutorial folder is your current working directory.

> library(rgl)
> library(StereoMorph)

2. Specify the file paths of each aspect of 3D landmarks and curves. For demonstration, the landmarks and curve points reconstructed in the previous section will be used.

> landmarks_3d <- paste0("Landmarks and curves 3D/obj_a", 1:3, ".txt")

3. Read each of these into an array using the readLandmarksToArray() function.

> lm.array <- readLandmarksToArray(landmarks_3d, row.names=1)

4. Call unifyLandmarks() to unify all the aspects into a single point set.

> unify_lm <- unifyLandmarks(lm.array, min.common=5)

The unifyLandmarks() function begins by choosing two point sets and aligning them with each other based on three or more shared points. Then, any additional point sets are unified with this combined point set, one-by-one, saving each unified point set at each step as the new combined point set.

unifyLandmarks() finds an ideal sequence based on the error of unification (how well the common points line up with each other). By setting min.common to 5, unifyLandmarks() will only align two aspects if they share at least five points. This does not mean that all point sets have to share the same five points; once two point sets are unified, any points shared between those two point sets and the next point set will be used in the alignment. Thus, the number of shared points used for each unification will usually depend on the order in which the points are unified. While it’s possible to unify points based on only three points this is likely to cause poor alignments, especially if the shared points happen to be nearly collinear.

5. Use the summary() function to see the unification sequence and errors.

The first part shows the sequence in which the point sets were unified followed by the root-mean-square error (here, in millimeters) for each unification step.

> summary(unify_lm)

unifyLandmarks Summary
Unification sequence: 1, 2, 3
Unification RMSE:
   0.7412698
   0.8801677

The second part shows the unification errors by landmark for each sequence (indicated by the number in double brackets). Both indicate that the unification errors are fairly low. The major source of error here is the difficulty of finding the exact same points from different views of an object.

Unification landmark errors by sequence:
    [[1]]
        basipterygoid_proc_post_R 0.160657
        jugal_upperbeak_R 1.046571
        opisthotic_process_R 0.8580522
        ...

    [[2]]
        foramen_magnum_sup 0.7183007
        mand_condyle_quadrate_lat_L 0.6221617
        mand_condyle_quadrate_lat_R 0.2217136
        ...

6. Plot the points using plot3d() from the ‘rgl’ package.

> pts <- na.omit(unify_lm$lm.matrix)
> r <- apply(pts, 2, 'max') - apply(pts, 2, 'min')
> plot3d(pts, aspect=c(r/r[3]), size=3)

All landmarks and curve points unified into a single point set.
7. Save the unified landmarks to a text file.

> write.table(unify_lm$lm.matrix, file="Landmarks 3D unified/obj.txt", quote=F, sep="\t", col.names=F, row.names=T)

Reflecting missing landmarks


When collecting landmarks from objects with left/right (bilateral) symmetry, it’s often not possible to collect every landmark on both sides of the object. In this case, the plane of bilateral symmetry can be used to reflect landmarks only present on one side across the midline, creating a set of landmarks complete on both sides (Klingenberg, Barluenga & Meyer 2002). This can be done using the reflectMissingLandmarks() function in StereoMorph.

1. Load the StereoMorph package, if not already loaded, and the ‘rgl’ R package for viewing 3D points. Also ensure that the StereoMorph Tutorial folder is your current working directory.

> nx <- library(StereoMorph)
> ny <- library(rgl)


2. Specify the file paths of a 3D point set of landmarks and/or curve points.

> lm_unified <- paste0("Landmarks 3D unified/obj.txt")

3. Import the landmarks as a matrix.

> lm.matrix <- readLandmarksToMatrix(lm_unified, row.names=1)

4. Call reflectMissingLandmarks().

> reflect <- reflectMissingLandmarks(lm.matrix, average=TRUE)

Or, if you have just completed the unification step, you can use unify_lm$lm.matrix.

> reflect <- reflectMissingLandmarks(unify_lm$lm.matrix, average=TRUE)

Setting average to TRUE will reflect the missing landmarks across the midline and then average the left and right sides (so that they are mirror images).

5. Print a summary of the bilateral errors.

> summary(reflect)

The bilateral errors are calculated by measuring the distance between a point present on both sides already and its contralateral point after reflection (and before averaging).

6. Plot the points using plot3d() from the ‘rgl’ package.

> pts <- na.omit(reflect$lm.matrix)
> r <- apply(pts, 2, 'max') - apply(pts, 2, 'min')
> plot3d(pts, aspect=c(r/r[3]), size=3)


Unified landmarks after reflecting landmarks missing on one side across the midline.
7. Save the reflected landmarks to a text file.

> write.table(reflect$lm.matrix, file="Landmarks 3D reflected/obj.txt", quote=F, sep="\t", col.names=F, row.names=T)

Aligning landmarks to the midline


The 3D landmark and curve points resulting from the previous steps of reconstruction and unification will be arbitrarily positioned and oriented in 3D space. For visualization purposes it’s often desirable to align a set of landmarks to the midline plane so that multiple objects can be viewed in a consistent orientation.

This can be done with the StereoMorph function alignLandmarksToMidline(). This function translates and rotates the points so that the midline points are aligned with the xy-plane. If bilateral landmarks were averaged in the reflection step, the midline points will lie exactly in the midline plane (i.e. they will have z-values of zero). Aligning landmarks to the midline plane does not change the scaling of the landmarks or the position of any landmarks relative to one another.

1. Load the StereoMorph package, if not already loaded, and the ‘rgl’ R package for viewing 3D points. Also ensure that the StereoMorph Tutorial folder is your current working directory.

> nx <- library(StereoMorph)
> ny <- library(rgl)


2. Specify the file paths of a 3D point set of landmarks and/or curve points.

> lm_reflected <- paste0("Landmarks 3D reflected/obj.txt")

3. Import the landmarks as a matrix.

> lm.matrix <- readLandmarksToMatrix(lm_reflected, row.names=1)

4. Call reflectMissingLandmarks().

> align <- alignLandmarksToMidline(lm.matrix)

Or, if you have just completed the unification step, you can use unify_lm$lm.matrix.

> align <- alignLandmarksToMidline(reflect$lm.matrix)

Calling summary() will print the midline alignment errors (the distances between the midline points and the estimated midline plane). However, since the bilateral landmarks were averaged in the reflection step, all of the midline points were already aligned to the midline plane. Thus, all of the midline alignment errors are zero.

5. Plot the points using plot3d() from the ‘rgl’ package.

> pts <- na.omit(align$lm.matrix)
> r <- apply(pts, 2, 'max') - apply(pts, 2, 'min')
> plot3d(pts, aspect=c(r/r[3]), size=3)


The landmarks are now aligned along the midline and landmarks on opposite sides have the same coordinates except an equal and opposite value along the z-axis.

> align$lm.matrix['quadrate_jugal_L', ]
[1] -45.431501 -6.010435 16.372618
> align$lm.matrix['quadrate_jugal_R', ]
[1] -45.431501 -6.010435 -16.372618


6. Save the aligned landmarks to a text file.

> write.table(align$lm.matrix, file="Landmarks 3D aligned/obj.txt", quote=F, sep="\t", col.names=F, row.names=T)

Unified landmarks after aligning the landmarks to the midline.
In ten steps, we've constructed a stereo camera setup, photographed and digitized landmarks and curves on the specimen and created a 3D reconstruction of these landmarks and curves. I hope that you've found this tutorial helpful and easy-to-follow. Please feel free to leave questions, comments and suggestions.


Return to the main tutorial page: Collecting 3D shape data using StereoMorph
Go back to the previous step: Reconstructing 2D points and curves into 3D

No comments:

Post a Comment