Skip to end of metadata
Go to start of metadata

Project Structure:

augmented(main)

read parameters to get the camera matrix and distortion coefficient 

get/update extrinsic parameters

open camera and initiate window

detect mark

draw objects

ar_aruco

similar to augmented.cpp but using the Aruco module instead

ar_harris

detect harris corner, find iphone target and apply augmented reality

ar_harris_detect

detect harris corner

ar_img ( augmented reality from image)

read images from a given directory

draw/update shapes on detected targets in each image

shape

draw shapes with given intrinsic and extrinsic parameters

calib

camera calibration

write to params.txt on camera matrix and distortion coefficient


Compilation and Running

augmented:

 make augmented

../bin/augmented ../data/params.txt

ar_aruco: 

make ar_aruco

../bin/ar_aruco ../data/params.txt

ar_img:

make ar_img

../bin/ar_img <image directory> <params.txt>

ar_harris

make ar_harris

../bin/ar_harris <params.txt>

ar_harris_detect

make ar_harris_detect

../bin/ar_harris_detect <params.txt>

calib

make calib

../bin/calib

Tasks:

task 1 - 3: camera calibration

The program first finds chessboard corners. If patterns found, users are able to capture some frames and store the images in the data directory. The program meanwhile records the 3D position of each corner in world coordinates. To calibrate the camera, the program then computes camera Matrix and distortion coefficients from the captured image pool. The intrinsic parameters were then saved in params.txt for later use.

 

CameraMatrix
[741.6874443430698, 0, 321.5858494603505;
0, 741.6874443430698, 254.6163302347387;
0, 0, 1]
distCoeffs
[-0.2443894337493308, 2.380506690770593, -0.005574932810936765, 0.01429998283309843, -8.577150180995915]

Error is 0.559419


some captured frames:

 

task 4 - 6: get camera position and draw shapes

The program reads the extrinsic parameters from params.txt. We use cv::projectPoints to transform object points into real-world image points related to the targets. We drew some 3D & 2D shapes augmented from the chessboard.

To watch the demo video, click link or the image below: AR

 

task 7: Harris corners

Using the Harris corner function from OpenCV library, we are able to detect corners in the world and work with targets out of the corners.

AR through Harris Corner (click link or image to view the video on youtube)

Extensions:

1, Aruco marker detection

(Aruco installation needed)

(click image to view video or see aruco marker detection)

2, multiple targets

Here is an example of having two chessboards in the scene:

3, try several cameras

We tested 3 cameras:

logitech C270 Desktop desktop camera:

CameraMatrix:
[1520.20974118723, 0, 974.0809412753549;
0, 1520.20974118723, 547.9555212362962;
0, 0, 1]
distCoeffs:
[0.01562629126808835, -0.529654879408201, -0.00272563495302247, 0.007163170504457198, 1.254115900715106]

error:

1.74774

Macbook Pro webcam:

CameraMatrix:
[897.7464413203074, 0, 637.8893691161974;
0, 897.7464413203074, 375.3225807946969;
0, 0, 1]
distCoeffs:
[-0.1083433066960941, 0.5376106891614889, 0.0003838343864146616, 0.004929627244628014, -0.892936629856659]

error:

0.232864

Little Red two-eyes alien camera:

CameraMatrix
[741.6874443430698, 0, 321.5858494603505;
0, 741.6874443430698, 254.6163302347387;
0, 0, 1]
distCoeffs
[-0.2443894337493308, 2.380506690770593, -0.005574932810936765, 0.01429998283309843, -8.577150180995915]

error
0.559419

(this little red camera has the largest distortion coefficient, showing that it has the poorest quality) 

4, other targets

Demo Video (through youtube, no extra downloading):

(To watch the video, click image or the link here: AR on phone screen )

5, static image AR

We first pre-captured some images from the camera. Then using ar_img program, we are able to add virtual objects in the targets detected within the image.

Picture example 1:

Picture example 2:

 

6, "camouflage" target

We drew a colored rectangle over the target board so that it does not look like a target anymore.

Acknowledgment:

This is a group project by Mike Zheng and Heidi He.

Useful Links and References:

Aruco in OpenCV:

https://docs.opencv.org/4.0.0-beta/db/da9/tutorial_aruco_board_detection.html

Aruco documentation:

https://docs.google.com/document/d/1QU9KoBtjSM2kF6ITOjQ76xqL7H0TEtXriJX5kwi9Kgc/edit

 

Labels