Next: 7.5 Probabilistic Context-Free Grammars Up: Chapter7Ambiguity Resoltion: Statistical Methods Previous: Box 7.1 Getting Reliable

7.4 Obtaining Lexical Probabilities

$BBg5,LO$J(B corpus $B$,EE;R2=$5$l$k$H(B,$B$=$N>pJs$H(B parser $B$r7k$SIU$1$F@:(B $BEY$N$h$$(B parsing $B$r9T$&$3$H$,$G$-$k$H4|BT$5$l$k(B
$B$9$Y$F$N(B courpus $B$,@5$7$$(B data $B$J$i$P(B viterbi algorithm $B$N7k2L$r(B $B$=$N$^$^(B parser $B$KEO$;$P$h$$(B
$B$7$+$7(B,$B$9$Y$F$N(B courpus $B$,@5$7$$(B data $B$O8B$i$J$$(B, $B$h$j:GE,$J2r$r(B $B5a$a$kI,MW@-$,$"$k(B
$B$H$J$k$H(B, viterbi algorithm $B$N$h$&$J:GBg(B category $B$N3d$jEv$F$r(B $B$?$@$R$H$D$@$15a$a$k$h$&$J(B algorithm $B$G$O$J$/(B, context $B$r9MN8$7$F(B $B3FC18l$KM=A[$5$l$k(B category $B$N3NN($r$9$Y$F5a$a$F(B, parser $B$KEO$7$?(B $B$[$&$,$$$$$N$G$O$J$$$+(B?
$BF~NO(B context $B$N>r7o2<$G$NG$0U$NIJ;l(B,$BG$0U$NC18l0LCV$N=P8=3NN((B, P(w_t/L_i|Input)
context-independ $B$JC18l$N=P8=3NN((B Figure 7.13
context $B$r9MN8$7$F(B ``The flies like flowers'' $B$NCf$G(B, ``The flies'' $B$N(B flies $B$,L>;l$G$"$k3NN($r5a$a$k(B.

$\begin{eqnarray*}PROB(flies/N\vert The\ files) &=& \frac{PROB(flies/N\vert The\ ... ... \\ &=& 9.58 \cdot 10^{-3}/9.59 \cdot 10^{-3} \\ &=& 0.9988 \end{eqnarray*}$
- $BJ,;R(B
  The/? flies/N $B$H$J$k9M$($&$k$9$Y$F$N3NN($NOB(B
  The/ART flies/N ( $9.58 \cdot 10^{-3}$ )
  The/N flies/N ( $1.13 \cdot 10^{-6}$ )
  The/P flies/N ( $4.55 \cdot 10^{-9}$ )
- $BJ,Jl(B
  The/? flies/? $B$H$J$k9M$($&$k$9$Y$F3NN($NOB(B
  The/? flies/V $B$H$$$&CM$b9MN8(B ( $9.959 \cdot 10^{-3}$ )
$B$3$N(B context $B$G(B flies $B$,(B N $B$K$J$k3NN($O(B 0.9998
$B$3$N$h$&$J(B context $B$r9MN8$7$?3NN(CM$r8zN($h$/5a$a$k$K$O(B? viterbi algorithm $B$K;w$?
forward probability
- $w_1,\cdots,w_t$ $B$NC18lNs$,=P8=$7(B,$B$+$D(Bt $BHVL\$NC18l$,(B category i $B$KB0$9$k3NN((B
  
  $\begin{eqnarray*}\alpha_i(t) &=& PROB(w_t/L_i,w_1,\cdots, w_t) \\ &=& PROB(w_1/?,w_2/?,\cdots,w_t/L_i) \end{eqnarray*}$
- $B4pK\E*$K(B vitebi algorithm $B$HF1$8(B
  $\max \rightarrow \sum$ $B$KJQ99(B ( $SEQSCORE \rightarrow SEQSUM = \alpha_i(t)$ )
  $B2DG=$JAH9g$o$;$N3NN($NOB(B
- SEQSUM$B$NCM$G@55,2=(B
  
  $\begin{eqnarray*}PROB(w_t/L_i\vert w_1,\cdots, w_t) &\simeq& \frac{SEQSUM(i,t)}{... ...SUM(j,t)} \\ &=& \frac{\alpha_i(t)}{\sum_{j=1}^N \alpha_j(t)} \end{eqnarray*}$
  
  Figure 7.14 $B;2>H(B
- Figure 7.15 viterbi algorithm $B$G(B $\max \rightarrow \sum$ $B$KJQ99$7(B $B$?>l9g$N(B score ( $=\alpha_i(t))$
- Figure 7.16 $\alpha_i(t)$ $B$+$i5a$a$?3NN(CM(B
  ($BNc(B)
  
  $\begin{eqnarray*}PROB(flowers/N\vert the\ flies\ like\ flowers) &\simeq& \frac{1... ... 10^{-5}}{1.1 \cdot 10^{-5} + 3.6 \cdot 10^{-7}} \\ &=& 0.967 \end{eqnarray*}$
backward probability
- $w_t,\cdots,w_T$ $B$NC18lNs$,=P8=$7(B,$B$+$D(Bt $BHVL\$NC18l$,(B category i $B$KB0$9$k3NN((B
- forward probability $B$HF1$8(B, $B5a$a$k8~$-$,5U(B, $BJ8Kv$+$i;O$^$k(B
lexical probabilites for word w_t
- forward brobability, backword probability $B$rAH$_9g$o$;$k(B
  
  $\begin{eqnarray*}PROB(w_t\vert L_i) = \frac{\alpha_i(t) \cdot \beta_i(t)}{\sum_{j=1}^{N}(\alpha_j(t) \cdot \beta_j(t))} \end{eqnarray*}$

1999-08-03