生成式AI原理与实战-cp1

理解语言模型

词袋模型,将一句话中出现的单词,每个单词出现的次数记录下来,如此就可以用数字化的方式表达出来。

比如, That is a cute dog, my cat is cute

分词: that is a cute dog my cat

那么my cat is cute就可以这样表示 [0 1 0 1 0 1 1]

缺点很明显, 词序,词义都丢失了

  • Copyright: Copyright is owned by the author. For commercial reprints, please contact the author for authorization. For non-commercial reprints, please indicate the source.
  • Copyrights © 2015-2025 Xudong0722
  • Visitors: | Views:

请我喝杯咖啡吧~

支付宝
微信