插件GitHub Copilot 一份NLP快速入门的roadmap,也分享给你,如果感兴趣可以学习学习
- https://www.deeplearningbook.org/contents/applications.html NLP基础(只看12.4,其中四级标题(如12.4.3.1等)下的细节内容可略过)
- https://huggingface.co/docs/transformers/preprocessing 数据预处理(主要看Natural Language Processing部分)
- https://huggingface.co/docs/transformers/training 训练(主要看Train in native PyTorch和Train with PyTorch Trainer部分)
- https://zhuanlan.zhihu.com/p/338817680 复习Transformers
- https://arxiv.org/pdf/1910.10683.pdf
- https://zhuanlan.zhihu.com/p/444438914 学习Encoder-decoder预训练语言模型(以T5为例,论文看前13页)
- https://huggingface.co/docs/transformers/tasks/summarization 应用Encoder-decoder预训练语言模型进行文本摘要生成
- https://arxiv.org/pdf/1810.04805.pdf +https://arxiv.org/pdf/1907.11692.pdf
- https://zhuanlan.zhihu.com/p/363014957
- https://huggingface.co/docs/transformers/tasks/masked_language_modeling 学习Encoder-only预训练语言模型(以BERT为例,第一篇论文看前5页,第二篇论文看第一页)
- https://huggingface.co/docs/transformers/tasks/token_classification 应用Encoder-only预训练语言模型进行文本分类和序列标注式实体识别
- https://cdn.openai.com/better-language-models [========] language_models_are_unsupervised_multitask_learners.pdf
- https://zhuanlan.zhihu.com/p/630970209
- https://huggingface.co/docs/transformers/tasks/language_modeling 学习和应用Decoder-only预训练语言模型(以GPT-2为例,论文看前4页)
- https://huggingface.co/THUDM/chatglm3-6b/blob/main/modeling_chatglm.py
- https://github.com/THUDM/ChatGLM3/tree/main/finetune_demo 学习和应用General Language Models(论文看前4页,以代码为准)