Post on 16-Jan-2016
description
Visão estéreo- correspondência e reconstrução -
Cap. 7 Trucco & Verry
Reconstrução da forma
Captura de movimento
Basic principle to recover position from stereo images: Triangulation
• Requires correspondence and camera calibration
Correpondência por semelhança
Sum of Square Differences – SSD
ou
Correlação
Correspondênciapor vizinhança correlacionada
Semelhança de duas regiões WW(SSD – Sum of Squared Difference)
min),(),(),( 00 évusvuyxd
W
Wx
W
Wy
yvyxuxIyyxxIvus 2002001 ))(,)((),(),(
x0
y0
x0+u
y0+v
Semelhança de duas regiões WW(correlação)
W
Wx
W
Wy
yvyxuxIyyxxIvus 2002001 ))(,)((),(),(
W
Wx
W
Wy
yyxxIvus 2001 ),(),(
W
Wx
W
Wy
yvyxuxIyyxxI ))(,)((),(2 002001
W
Wx
W
Wy
yvyxuxI 2002 ))(,)((
constante
constante
W
Wx
W
Wyvu
yvyxuxIyyxxI ))(,)((),( 002001),(
max
Semelhança de duas regiões WW(Normalização)
W
Wx
W
Wyvu
yvyxuxIyyxxI ))(,)((),( 002001),(
max
222
2
11
2211
),()ˆ),((ˆ),(
)ˆ),()(ˆ),((max
IvyuxIIyxI
IvyuxIIyxIvu
Normalizando:
2
222
22
2
11
11
),( )ˆ),((
)ˆ),((
ˆ),(
)ˆ),((min
IvyuxI
IvyuxI
IyxI
IyxIvu
W
Wx
W
Wyvu
yvyxuxIyyxxI 2002001
),(
))(,)((),(min
Correspondence between points
• With characteristics
-
Correspondence problem: lack of characterists
-
Ostridge egg on a Chinese checker board
Correspondência com luz estruturada
Estéreo Ativo
Taxonomy of active range acquisition methods
Active shape acquisition
Contact
Nondestructive
Non-contact
Transmissive
Reflective
CT
Asla Sá et al, Coded Structure Light for 3D-Photograpy: an Overview, Revista de Informática Teórica e Aplicada, Volume IX, Número 2, Porto Alegre, 2002
Brian Curless. New Methods for Surface Reconstruction from Range Images. PhDDissertation. Stanford University. 1997
Non-optical
Sonar
Microwave radar
Optical Radar
Triangulation
Active stereo
Active depth from defocus
Passive
Shape from focus
Destructive
Slices
CMM
Shape from shading
Shape from silhouettes
Active
…
…
Active stereo solution
Use a light source to mark corresponding points
uncalibrated light source calibrated light source
One point at the time: long capture process.
Active stereo: capturing many points
Use of a digital projector as a structured light source
Pattern with several elements in a way where each element can be identified univocally
point coding: prone to errors stripes: more robust
Methods for light coding: temporal codification
Project, in sequence, a series of slides that code in the image a binary number.
n slides for 2n stripes. Two ilumination levels. Static scene. Code one axis.
Posdamer, J. L. Altschuler, M. D. Surface Measurement by Space-Encoded Projected Beam Systems. Comput. Graphics Image Process. 18, pp. 1-17, 1982.
slide1 slide2 codeslide3
problem:all transitions occur in the same place!
can be also111 or 001!
Código de Gray código binário
Código binário1 bit: 0 12 bits: 00 01 10 113 bits: 000 001 010 011 100 101 110 111 Código de Gray1 bit: 0 12 bits: 00 01 11 103 bits: 000 001 011 010 110 111 101 100
ordem invertida
Código de Gray código binário
Código binário Código de Gray
Robust temporal codification: Gray coding
Inokuchi, Seiji. Sato, Kosuki. Matsuda, Fumio. Range Imaging for 3D Object Recognition. Proc. Int. Conf. on Pattern Recognition, pp.806-808, 1984.
transitions occur in different places
Example of Gray coding
needs too many slides!
Color Gray coding
better yet…
reduces the number of slides by 3
(b,s)-BCSL Coding
Sá, Asla Medeiros. Medeiros, Esdras Soares. Carvalho, Paulo Cezar Pinto. Velho, Luiz. Coded Structured Light for 3D-Photography: an Overview. Revista de Informática Teórica e Aplicada, Vol. 9, No. 2, outubro 2002
20
A practical difficulty in the border detection
example with the monochrome Gray code
Edge detection Projecting positive and negative slides
is a robust way to recover edges.
51
60
2100
40410
01816
32rgb-BCSL coding
(+)
(-)
slide 1 slide 2
Recovering colored codes
BBBB
GGGG
RRRR
pruI
pruI
pruI
,
,
ii
ii ru
uI
if
if
1
0
i
i
p
p negative slide
positive slide
ambient lightreflection factorsprojected light
iii IIr
iii III
Implementação do BCSL//A função getBcslStripeCode retorna o código de transição de faixa conforme a seqüência de cores fornecida.//Observe a ordem em que as cores devem ser passadas:// Primeiro as cores da imagem 1 e depois da imagem 2// Primeiro a faixa da esquerda e depois a faixa da direita////O código das cores e das bases é conforme a tabela abaixo.
//Padrão 3_2 //base 3//1 - vermelho//2 - verde//3 - azul
//Padrão 4_2 //base 4//1 - vermelho//2 - verde//3 - azul//4 - magenta
//Padrão 6_2//base 6//1 - vermelho//2 - verde//3 - azul//4 - ciano//5 - magenta//6 - amarelo
int getBcslStripeCode(int base, int colorLeft1, int colorRight1,int colorLeft2, int colorRight2);
int getBcslStripeCode(int base, int colorLeft1, int colorRight1,int colorLeft2, int colorRight2){ int aux1, aux2,linha,coluna; colorLeft2--; colorRight2--; colorLeft1--; colorRight1--;
linha = (colorLeft1 * base) + colorLeft2; aux1 = (colorRight2 - colorLeft2); aux2 = (colorRight1 - colorLeft1); aux1 = (aux1>0)?(aux1-1):((base-1)+aux1); aux2 = (aux2>0)?(aux2-1):((base-1)+aux2); coluna = ((aux2) * (base-1)) + (aux1); switch(base){ case 3:
return matrix3_2[linha *4+coluna];break;
case 4:return matrix4_2[linha *9 +coluna];break;
case 6:return matrix6_2[linha *25 +coluna];break;
default:printf("Error: invalid BCSL base\n");return -1;
}}
int matrix3_2[4*9]={ 0, 3, 6, 9,14, 17, 19, 11,28, 34, 22, 24,26, 29, 18, 21, 1, 31, 33, 35,15, 4, 8, 13,16, 23, 32, 12,27, 5, 7, 25, 2, 10, 20, 30
};….
teoria pode ser complicadamas a implementação é muito simples!
Mapa de profundiade
Disparidade x Profundidade
Disparidade
Profundidade versus disparidade
zx ol x or
z
P
cl cr
xlxr
T
Z
f
fZ
xxT
Z
T rl
d
Tf
xx
TfZ
rl
)()( rl xxTZTZf
)( rl xxZTf
Geometria Epipolar
Correspondência pela Geometria das Câmeras
Guido Gerig
Epipolar Geometry ctd.
Geometria Epipolar: notação
l
Ol
P
r
Or
Pl
pl
xcl
ycl
zcl
xcr
ycr
zcr
pr
Pr
eler
Linhaepipolar
Linhaepipolar
Example: converging cameras
Example: motion parallel with image plane
Example: forward motion
e
e’
Geometria Epipolar: relações básicas
l
P
r
xcl
ycl
zcl
xcr
ycr
zcr
)( rll
lrr eyePRP
ll
ll Z
fPp r
r
rr Z
fPp
rleye
rPlP
leyereye
Produto vetorial (revistado)
2121
2121
2121
212121212121
222
11121 )()()(
xyyx
xzzx
yzzy
xyyxxzzxyzzy
zyx
zyx kji
kji
pp
2
2
2
11
11
11
2121
2121
2121
21
0
0
0
z
y
x
xy
xz
yz
xyyx
xzzx
yzzy
pp
0
0
0
0
det 1111
11
11
11
zz zyxzyx
xy
xz
yz
Matriz Essencial 0)( lrl
Trll PeyeeyeP
0 lrl
T
r
Tlr PeyePR
r
Tlr
rll
rll
lrr PReyePeyePRP )()(
llrl SPPeye
0
0
0
x
rly
rl
x
rlz
rl
y
rlz
rl
eyeeye
eyeeye
eyeeye
S
0llr
Tr PSRP 0l
Tr EPP
Matriz essencial
0lT
r Epprl
rl
ZZ
ff
l
eye l
P
r
eye r
Pl
pl
xcl
ycl
zcl
xcr
ycr
zcr
pr
Pr
eler
rleye
Parâmetros extrínsecosxc
yc
zc
yw
xw
zw
w
w
w
cwcwcw
cwcwcw
cwcwcw
c
c
c
cw
Z
Y
X
Z
Y
X
kkkjki
jkjjji
ikijii
P
Pw
Pc t
tPP wc
1w
w
w
z
y
x
cwcwcw
cwcwcw
cwcwcw
c
c
c
cZ
Y
X
t
t
t
Z
Y
X
kkkjki
jkjjji
ikijii
P
tPRPtRP wwcwc
z
y
x
cwcwcw
cwcwcw
cwcwcw
cw
eye
eye
eye
kkkjki
jkjjji
ikijii
eyet
Rotação de a para b (left to right)
lrlrlr
lrlrlr
lrlrlrlr
kkjkik
kjjjij
kijiii
R
Twl
wr
lw
wr
lr RRRRR
lwlwlw
lwlwlw
lwlwlw
wrwrwr
wrwrwr
wrwrwrlr
kkjkik
kjjjij
kijiii
kkjkik
kjjjij
kijiii
R
rwrwrw
rwrwrw
rwrwrw
wlwlwl
wlwlwl
wlwlwlrl
kkjkik
kjjjij
kijiii
kkjkik
kjjjij
kijiii
R
Vetor do eye de b em a
lrwl
rl eyeeyeReye
X w
Y w
Z w
eye lxcl
ycl
zcl rleye
xcr
ycr
zcr
eye r
Glu Look At
Dados: eye, center, up (definem o sistema de coordenadas do olho)
Determine a matriz que leva do sistema de Coordenadas dos Objetospara o sistema de Coordenadas do Olho
void gluLookAt(GLdouble eyex, GLdouble eyey, GLdouble eyez, GLdouble centerx, GLdouble centery, GLdouble centerz, GLdouble upx, GLdouble upy, GLdouble upz);
up eye
center
Coordenadas dosobjetos
Coordenadas doolho
eye
Calculo do sistema - xe ye ze
ye
center
eye
zo
yo
xo
zexe
up
centereyecentereye
z
1
e
ee
e zupzup
x
1
eee xzy
dados:eye, center, up
Translada o eye para origem
center
eye
zo
yo
xo
zexe
ye
zo
yo
xo
center
eye
zexe
ye
1000
100
010
001
z
y
x
eye
eye
eye
T
Roda xe ye ze para xw yw zw
xe , xo
ye , yo
ze , zo
1000
0
0
0
ezeyex
ezeyex
ezeyex
zzz
yyy
xxx
R
zo
yo
xo
center
eye
ze
xe
ye
Matriz LookAt do OpenGL
11 0
eyeI
0
0R
1000
100
010
001
1000
0
0
0
z
y
x
exexex
eyeyex
ezeyex
at eye
eye
eye
zzz
yyy
xxx
RTL
1000
100
010
001
z
y
x
eye
eye
eye
T
centereyecentereye
z
1
e
ee
e zupzup
x
1
eee xzy
1000
0
0
0
exexex
eyeyex
ezeyex
zzz
yyy
xxx
R
10
eyeRR
Matriz essencial (código C)
Matrix epiEssencialMatrix( Matrix Ra, Vector eye_a, Matrix Rb, Vector eye_b) { Matrix Rba = algMult(Rb,algTransp(Ra)); Vector eye = algMult(Ra,algSub(eye_b,eye_a); Matrix S = algVectorProductMatrix(eye); Matrix E = algMult(Rba,S);
return E;}
Matriz Essencial
T
l
Ol
P
r
Or
Pl
pl
xcl
ycl
zcl
xcr
ycr
zcr
pr
Pr
eler
0lT
r Epp
lr Epu
0rT
r up
f
y
x
rp
c
b
a
ru 0 cfbyaxrT
r up
Câmera para imagem
Z
Y
X
osf
osf
w
y
x
yy
xx
h
h
100
0
0
yy
im
xx
im
oZ
Y
s
fy
oZ
X
s
fx
wy
wx
y
x
h
h
im
im
)(
)(
yimy
ximx
oys
oxs
y
xp
ZY
ZX
f
Geometria Epipolar: Matriz Fundamental
lll PMp
0lT
r EPP
01 ll
Tr
Tr pEMMp
1 lT
r EMMF
Matriz fundamental0l
Tr pFp
lr pFu
Z
Y
X
osf
osf
w
v
u
yy
xx
100
0
0rrr PMp
lll pMP 1 Tr
Tr
Tr
MpP
lp lM lP
Matriz fundamental 1 lT
r EMMF
100
0
0
0
0
0
100
0
0
ylyl
l
xlxl
l
x
rly
rl
x
rlz
rl
y
rlz
rl
lrlrlr
lrlrlr
lrlrlr
T
yryr
r
xrxr
r
osf
osf
osf
osf
eyeeye
eyeeye
eyeeye
kkjkik
kjjjij
kijiii
F
333231
232221
131211
FFF
FFF
FFF
F Pode ser estimada diretamente se conhecermos pelo menos oito pares de pontos correspondentes
0lT
r pFp
Transformações do OpenGL
xe
ye
ze
center
eye
zo
yo
xo
up
xn
yn
zn
w
z
y
x
c
c
c
110
0
0
z
y
x
z
y
x
viewe
e
e
M
1e
e
e
c
c
c
z
y
x
w
z
y
x
P
w
z
y
x
wz
y
x
c
c
c
n
n
n1
Matriz de projeção
[ H ] = ?
xe
ye
ze1
2
34
5
6
7
8
xe
ye
ze1
2
34
56
78
0100
00
000
000
nffn
n
n
H
Transforma o prisma de visão cubo normalizado [-1,1]×[-1,1] ×[-1,1]
xe
ye
ze
lr
b
t
xe
ye
ze
-(r-l)/2
(r-l)/2
-(t-b)/2
(t-b)/2
(f-n)/2 -(f-n)/2
1000
2/)(100
2)(010
2)(001
nf
bt
lr
T
xe
ye
ze
111
-1-1-1
far
near
1000
0)(200
00)(20
000)(2
nf
bt
lr
S
xn
yn
zn
111
-1-1-1
near
far
1000
0100
0010
0001
M
Matriz Frustum do OpenGL
1000
2/)(100
2)(010
2)(001
nf
bt
lr
T
1000
0)(200
00)(20
000)(2
nf
bt
lr
S
1000
0100
0010
0001
M
0100
00
000
000
nffn
n
n
H
0100
2)(00
02
0
002
nf
fn
nf
nfbt
bt
bt
nlr
lr
lr
n
NHP
OpenGL Spec
1000
2/)(100
2)(010
2)(001
1000
0)(200
00)(20
000)(2
1000
0100
0010
0001
nf
bt
lr
nf
bt
lr
N
Transformação para o viewport
xn
yn
zn
111
-1-1-1
void glViewport(int x0, int y0, int w, int h );
zw[0.. zmax], zmax = 2n-1 geralmente 65535
2
10
nw
xwxx
2
10
nw
yhyy
2
1max
nw
zzz
xw
yw
w
h
0
y0
x0
Revendo as transformações
1000
100
010
001
1000
0
0
0
0100
2)(00
02
0
002
100022
00
20
20
200
2
maxmaxz
y
x
exexex
eyeyex
ezeyex
eye
eye
eye
zzz
yyy
xxx
nf
fn
nf
nfbt
bt
bt
nlr
lr
lr
n
zz
hh
ww
cM tPRPtRP wwcwc
100
0
0
yy
xx
osf
osf
1w
w
w
z
y
x
cwcwcw
cwcwcw
cwcwcw
Z
Y
X
t
t
t
kkkjki
jkjjji
ikijii
Sistemas de coordenadasyim
f xim
yc
vista lateral
oczc
f =n
fovy oy
xim
yim
h s y
ox
oc
eixo óptico
x0
y0
yc
xc
zc
Z
Y
X
P
y'
x'
w sx
x'
y'
pixel
sx
sy
Revendo as transformações
0100
2)(00
02
0
02
0
0100
2)(00
02
0
02
0
0100
2)(00
002
0
0002
100022
00
20
20
200
2
maxmax
nf
fn
nf
nf
h
s
f
w
s
f
nf
fn
nf
nf
h
s
f
w
s
f
nf
fn
nf
nfhs
fws
f
zz
hh
ww
y
x
y
x
y
x
100
0
0
yy
xx
osf
osf
Matriz Fundamental (código C)
Matrix epiFundamentalMatrix( Matrix Ma, Matrix Ra, Vector eye_a, Matrix Mb, Matrix Rb, Vector eye_b){ Matrix E = epiEssencialMatrix(Ra,eye_a,Rb,eye_b); Matrix invMa = algInv(Ma); Matrix invMbTransp = algTransp(algInv(Mb)); Matrix tmp = algMult(invMbTransp,E); Matrix F = algMult(tmp,invMa); return F;}
Estimativa direta da Matriz Fundamental
O algoritmo de 8 pontos
Estimating Fundamental Matrix
Each point correspondence can be expressed as a linear equation
0
1
1
333231
232221
131211
r
r
ll y
x
FFF
FFF
FFF
yx
01
33
32
31
23
22
21
13
12
11
F
F
F
F
F
F
F
F
F
yxyyyxxxyxxx rrrlrrrrlrl
The 8-point algorithm
0lT
r pFp
Estimating Fundamental Matrix
0
1
1
1
1
33
32
31
23
22
21
13
12
11
33
22
11111
F
F
F
F
F
F
F
F
F
xx
xx
xx
xyxxx
nr
nl
rl
rl
lllrl
The 8-point algorithm
0FA }]{[
TUDVA
F é a coluna de V correspondente ao menor valor singular
Estimating Fundamental MatrixThe 8-point algorithm
333231
232221
131211
FFF
FFF
FFF
F deveria ter posto 2!
TUDVF Seja D' = D com o menor valor singular = 0
TVUDF ''
FF 'min0'det F
The Normalized 8-point AlgorithmRichard Hartley
0lT
r Fpp
0'' llTr
T
r pFTTp
Tpp '
lTr FTTF '
:= T
1000 0 -500
0 1000 -500
0 0 1
:= Tt
1000 0 0
0 1000 0
-500 -500 1
1250000 250000 -500
250000 1250000 -500
-500 -500 1
The Normalized 8-point AlgorithmRichard Hartley
centróide
escale para a distância média ficar em 2
Retificação de Imagens
Retificação
UNC-CH
Rectification ctd.
before
after
Guido Gerig
Retificação de imagens
z'
xc
yc
zc
y'
x'O O'
fz
y
x
z
y
x
c
c
c
'
'
'
pontoprincipal
Trucco e Verri
Retificação de imagens
l
Ol
P
r
Or
pl
xcl
ycl
zcl
xcr
ycr
zcr
pr
el erT
T
Te 1
0
1222 x
y
yx
T
T
TTe
213 eee
T
T
T
rect
3
2
1
e
e
e
R
Trucco e Verri
Retificação de imagens
T
T
T
rect
3
2
1
e
e
e
R
Trucco e Verri
1. Construa:
2. Defina:rectrrectl RRRRR ,
3. Aplique:
Pr= R(Pl - T)
'
'
'
z
y
x
llpR
'
'
'
''
z
y
x
z
flp
3. Aplique:
'
'
'
z
y
x
ddpR
'
'
'
''
z
y
x
z
fdp
Stereo image rectification
Steve Seitz, University of Washington
Stereo image rectification
Image Reprojection• reproject image planes onto common
plane parallel to line between optical centers• a homography (3x3 transform)
applied to both input images• pixel motion is horizontal after this transformation• C. Loop and Z. Zhang. Computing Rectifying Homographies for Stereo Vision
. IEEE Conf. Computer Vision and Pattern Recognition, 1999.
Steve Seitz, University of Washington
ll
Image Rectification
pp
P
O O
l lp p
e e
•Common Image Plane•Parallel Epipolar Lines•Search Correspondenceson scan line d
fBZ
Steve Seitz, University of Washington
Reconstrução
Reconstrução por triangulação
r
reye
l
leye
lp rp
rP
rrll pRpn
lapnc
rTb pR
rlr
rll cba eyenpRp
)( rll
lrr eyePRP
rleye
Reconstrução por triangulação
z
lr
y
lr
x
lr
zzrrllz
yyrrlly
xxrrllx
c
b
a
np
np
np
eye
eye
eye
pR
pR
pR
)(
)(
)(
wpP2
ca ll l
Tlwl
lww PRPRP
rrll pRpn
rlr
rll cba eyenpRp
Outro processo de reconstrução
Z
Y
X
osf
osf
w
v
u
yy
xx
100
0
0
10100
00
00
Z
Y
X
osf
osf
w
v
u
yy
xx
Miguel Ribo, Axel Pinz, Anton L. Fuhrmann “A new Optical Tracking System for Virtual and Augmented Reality Applications”,
ll P0IKp 3
lr PTRKp
l
l
l
l
l P
p
p
p
p
3
2
1
l
r
r
r
r P
p
p
p
p
3
2
1
44
32
31
32
31
44
xrr
rr
ll
ll
x
pvppuppvppup
B
r
r
l
l
0BP l
ll
lllu
Pp
Pp3
1
013 lllll u PpPp
Reconstruction
O O’
pp’
PP
Z
Pfp
Z
Pfp
t
tPRtPRP
tPRPT
1
Steve Seitz, University of Washington
Reconstruction
Z
Pfp
tPRtPRP T
tPR
tPRfp T
3
T
T
T
R
R
R
R
3
2
1
tPR
tPRfx T
T
3
1
Z
Pfp
f
pZP
Equation 1
Equation 2
pRfRx
tRfRxfZ T
T
13
13
(From equations 1 and 2)
Steve Seitz, University of Washington
Reconstruction up to a Scale Factor
• Assume that intrinsic parameters of both cameras are known• Essential Matrix is known up to a scale factor (for example, estimated from the 8 point algorithm).
Steve Seitz, University of Washington
Reconstruction up to a Scale Factor
Rtk
T TT tRRtk 2 Tttk 2
22222
22222
22222
YXZYZX
ZYZXYX
ZXYXZY
TTkTTkTTk
TTkTTkTTk
TTkTTkTTk
TraceT 222222 22 tkTTTk ZYX
tk R
t
tkR
t
tk
sgnsgn
TT ttEE ˆˆˆˆ
2
2
2
ˆ1ˆˆˆˆ
ˆˆˆ1ˆˆ
ˆˆˆˆˆ1
ZZYZX
ZYYYX
ZXYXX
TTTTT
TTTTT
TTTTT
Rtk ˆsgn E
Steve Seitz, University of Washington
Reconstruction up to a Scale Factor
T
T
T
E
E
E
E
3
2
1
ˆ
ˆ
ˆ
ˆ
T
T
T
R
R
R
R
3
2
1
Let 3,2,1 ,ˆˆ itEw ii
It can be proved that
2133
1322
3211
wwwR
wwwR
wwwR
Steve Seitz, University of Washington
Reconstruction up to a Scale Factor
We have two choices of t, (t+ and t-) because of sign ambiguityand two choices of E, (E+ and E-).
This gives us four pairs of translation vectors and rotation matrices.
Steve Seitz, University of Washington
Reconstruction up to a Scale Factor
Given and E t
1. Construct the vectors w, and compute R2. Reconstruct the Z and Z’ for each point3. If the signs of Z and Z’ of the reconstructed points are
a) both negative for some point, change the sign ofand go to step 2.
b) different for some point, change the sign of each entryof and go to step 1.
c) both positive for all points, exit.
t
E
pRfRx
tRfRxfZ T
T
13
13
pfRxR
tfRxRfZ T
T
13
13
Steve Seitz, University of Washington
Proposed system: equipament
2 cameras and 1 projector(fast)
1 moving camera and 1 projector(slow)
Proposed system: 32rgb-BCSL coding
left right
positiveslide
positiveslide
negativeslide
Where is a point in the other image?
u u
One solution: (u,v) coordinates
double the number of photos!
Epipolar geometry
l
eyel
P
r
eyer
Pl
pl
xcl
ycl
zcl
xcr
ycr
zcr
pr
Pr
eler
EpipolarLine
EpipolarLine
Epipolar correspondence
100
0
0
0
0
0
100
0
0
ylyl
l
xlxl
l
x
rly
rl
x
rlz
rl
y
rlz
rl
lrlrlr
lrlrlr
lrlrlr
T
yryr
r
xrxr
r
osf
osf
osf
osf
eyeeye
eyeeye
eyeeye
kkjkik
kjjjij
kijiii
F
0lT
r pFp
Reconstruction by triangulation: ideia
r
reye
l
leye
lp rp
rP
rrll pRpn
lapnc
rTb pR
rlr
rll cba eyenpRp
)( rll
lrr eyePRP
rleye
Reconstruction by triangulation: algebra
z
lr
y
lr
x
lr
zzrrllz
yyrrlly
xxrrllx
c
b
a
np
np
np
eye
eye
eye
pR
pR
pR
)(
)(
)(
wpP2
ca ll l
Tlwl
lww PRPRP
rlr
rll cba eyenpRp
Captured data
Cylinder model
n
jpj
n
j
Tpjpj tensor
nn 00
)(1
))((1
cpcpcpM
zzzyzx
yzyyyx
xzxyxx
zyx
z
y
x
z
y
x
tensorM
1e
2e
3e
321321 ),(ˆˆˆ Meee rseigenvecto
n
jjp n 0
1pc
axis of the points pi:
covariance matrix:
centroid:
Initial cylinder adjustment
pc 2e
3e
cc
1ez c
3ˆ)( ecc aRpc
)( . ˆmin 3 pjja cpe a
tangent plane perpendicular to ê3:
first guess for cc:
first guess for zc:
Results of the initial cylinder adjustment
Giving a set of points P and a model Q, find the rigid body motion (R, t) that minimizes:
where q(pi) is the point in Q correspondent to pi.
Model fitting problem
2||)()(||min tRppq ii
i
ICP (Iteractive Closest Point) Algorithm
begins with a initial guess for the model pose( R and t )
at each iteration, q(pi) is the point in Q closest to Rpi + t
R e t are computed to minimize the error
P. J. Besl and N. D. McKay, A Method for Registration of 3-D Shapes, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 14, No. 12, February 1992
2||)()(||min tRppq ii
i
Projection of a point on a cylinder
0p
p
cz
ppip
Plane : 0 czpp
Axis : ctt zpp 0)(
cc
cit zz
zpp
0
ciii tt zppp 0)(
i
ip
R
pp
ppp
Given : Rc ,,, 0 zpp
Compute : pp
ICP step
• find centroids: p0 e q0
• Define pi’= pi – p0 , qi’= qi – q0
• Define
where
• Rotation: R = M(MT M) –1/2
• Translation: t = q0– Rp0
zzzyzx
yzyyyx
xzxyxx
SSS
SSS
SSS
M
etc,, ,,,,iy
iixxyix
iixxx qpSqpS
Results
Direct measure