7.4 Obtaining Lexical Probabilities

• $BBg5,LO$J(B corpus $B$,EE;R2=$5$l$k$H(B,$B$=$N>pJs$H(B parser $B$r7k$SIU$1$F@:(B $BEY$N$h$$(B parsing $B$r9T$&$3$H$,$G$-$k$H4|BT$5$l$k(B
• $B$9$Y$F$N(B courpus $B$,@5$7$$(B data $B$J$i$P(B viterbi algorithm $B$N7k2L$r(B $B$=$N$^$^(B parser $B$KEO$;$P$h$$(B
• $B$7$+$7(B,$B$9$Y$F$N(B courpus $B$,@5$7$$(B data $B$O8B$i$J$$(B, $B$h$j:GE,$J2r$r(B $B5a$a$kI,MW@-$,$"$k(B
• $B$H$J$k$H(B, viterbi algorithm $B$N$h$&$J:GBg(B category $B$N3d$jEv$F$r(B $B$?$@$R$H$D$@$15a$a$k$h$&$J(B algorithm $B$G$O$J$/(B, context $B$r9MN8$7$F(B $B3FC18l$KM=A[$5$l$k(B category $B$N3NN($r$9$Y$F5a$a$F(B, parser $B$KEO$7$?(B $B$[$&$,$$$$$N$G$O$J$$$+(B?
  $BF~NO(B context $B$N>r7o2<$G$NG$0U$NIJ;l(B,$BG$0U$NC18l0LCV$N=P8=3NN((B, P(w_t/L_i|Input)
• context-independ $B$JC18l$N=P8=3NN((B Figure 7.13
• context $B$r9MN8$7$F(B ``The flies like flowers'' $B$NCf$G(B, ``The flies'' $B$N(B flies $B$,L>;l$G$"$k3NN($r5a$a$k(B.
  
  $$\begin{eqnarray*}PROB(flies/N\vert The\ files) &=& \frac{PROB(flies/N\vert The\ ... ... \\ &=& 9.58 \cdot 10^{-3}/9.59 \cdot 10^{-3} \\ &=& 0.9988 \end{eqnarray*}$$
  
  
  • $BJ,;R(B
    The/? flies/N $B$H$J$k9M$($&$k$9$Y$F$N3NN($NOB(B
    The/ART flies/N ( $9.58 \cdot 10^{-3}$)
    The/N flies/N ( $1.13 \cdot 10^{-6}$)
    The/P flies/N ( $4.55 \cdot 10^{-9}$)
  • $BJ,Jl(B
    The/? flies/? $B$H$J$k9M$($&$k$9$Y$F3NN($NOB(B
    The/? flies/V $B$H$$$&CM$b9MN8(B ( $9.959 \cdot 10^{-3}$)
  $B$3$N(B context $B$G(B flies $B$,(B N $B$K$J$k3NN($O(B 0.9998
  $B$3$N$h$&$J(B context $B$r9MN8$7$?3NN(CM$r8zN($h$/5a$a$k$K$O(B? viterbi algorithm $B$K;w$?forward probability $\alpha_i(t)$
  • $w_1,\cdots,w_t$ $B$NC18lNs$,=P8=$7(B,$B$+$D(Bt $BHVL\$NC18l$,(B category i $B$KB0$9$k3NN((B
    
    $$\begin{eqnarray*}\alpha_i(t) &=& PROB(w_t/L_i,w_1,\cdots, w_t) \\ &=& PROB(w_1/?,w_2/?,\cdots,w_t/L_i) \end{eqnarray*}$$
    
    
  • $B4pK\E*$K(B vitebi algorithm $B$HF1$8(B
    $\max \rightarrow \sum$ $B$KJQ99(B ( $SEQSCORE \rightarrow SEQSUM = \alpha_i(t)$)
    $B2DG=$JAH9g$o$;$N3NN($NOB(B
  • SEQSUM$B$NCM$G@55,2=(B
    
    
    $$\begin{eqnarray*}PROB(w_t/L_i\vert w_1,\cdots, w_t) &\simeq& \frac{SEQSUM(i,t)}{... ...SUM(j,t)} \\ &=& \frac{\alpha_i(t)}{\sum_{j=1}^N \alpha_j(t)} \end{eqnarray*}$$
    
    Figure 7.14 $B;2>H(B
  • Figure 7.15 viterbi algorithm $B$G(B $\max \rightarrow \sum$ $B$KJQ99$7(B $B$?>l9g$N(B score ( $=\alpha_i(t))$
  • Figure 7.16 $\alpha_i(t)$ $B$+$i5a$a$?3NN(CM(B
    ($BNc(B)
    
    $$\begin{eqnarray*}PROB(flowers/N\vert the\ flies\ like\ flowers) &\simeq& \frac{1... ... 10^{-5}}{1.1 \cdot 10^{-5} + 3.6 \cdot 10^{-7}} \\ &=& 0.967 \end{eqnarray*}$$
    
    
• backward probability $\beta_i(t)$
  • $w_t,\cdots,w_T$ $B$NC18lNs$,=P8=$7(B,$B$+$D(Bt $BHVL\$NC18l$,(B category i $B$KB0$9$k3NN((B
  • forward probability $B$HF1$8(B, $B5a$a$k8~$-$,5U(B, $BJ8Kv$+$i;O$^$k(B
• lexical probabilites for word w_t
  
  • forward brobability, backword probability $B$rAH$_9g$o$;$k(B
    
    $$\begin{eqnarray*}PROB(w_t\vert L_i) = \frac{\alpha_i(t) \cdot \beta_i(t)}{\sum_{j=1}^{N}(\alpha_j(t) \cdot \beta_j(t))} \end{eqnarray*}$$