Top Python Natural language processing repositories on GitHub

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

• nlp
• natural-language-processing
• pytorch
• pytorch-transformers
• transformer
• model-hub

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

• fine-tuning
• llama
• llm
• peft
• transformers
• rlhf

AiLearning：数据分析+机器学习实战+线性代数+PyTorch+NLTK+TF2

• fp-growth
• apriori
• mahchine-leaning
• naivebayes
• svm
• adaboost

微舆：人人可用的多Agent舆情分析助手，打破信息茧房，还原舆情原貌，预测未来走向，辅助决策！从0实现，不依赖任何框架。

• agent-framework
• data-analysis
• multi-agent-system
• nlp
• public-opinion-analysis
• python3

TensorFlow code and pre-trained models for BERT

• nlp
• google
• natural-language-processing
• natural-language-understanding
• tensorflow

A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.

• llm
• nlp
• python
• gemini-ai
• information-extration
• large-language-models

中文分词 词性标注 命名实体识别 依存句法分析 成分句法分析 语义依存分析 语义角色标注 指代消解 风格转换 语义相似度 新词发现 关键词短语提取 自动摘要 文本分类聚类 拼音简繁转换 自然语言处理

• nlp
• natural-language-processing
• hanlp
• pos-tagging
• dependency-parser
• text-classification

💫 Industrial-strength Natural Language Processing (NLP) in Python

• natural-language-processing
• data-science
• machine-learning
• python
• cython
• nlp

An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.

• large-language-models
• nlp
• knowledge-curation
• naacl
• report-generation
• retrieval-augmented-generation

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

• nlp
• pre-trained-model
• unilm
• minilm
• layoutlm
• layoutxlm

🤗 The largest hub of ready-to-use datasets for AI models with fast, easy-to-use and efficient data manipulation tools

• nlp
• datasets
• pytorch
• tensorflow
• pandas
• numpy

💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants

• nlp
• machine-learning
• machine-learning-library
• bot
• bots
• botkit

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

• llm
• plm
• pre-trained-language-models
• alpaca
• llama
• nlp

Topic Modelling for Humans

• gensim
• topic-modeling
• information-retrieval
• machine-learning
• natural-language-processing
• nlp

NLTK Source

• nltk
• python
• nlp
• natural-language-processing
• machine-learning

A very simple framework for state-of-the-art Natural Language Processing (NLP)

• pytorch
• nlp
• named-entity-recognition
• sequence-labeling
• semantic-role-labeling
• word-embeddings

Nano vLLM

• inference
• llm
• pytorch
• transformer
• deep-learning
• nlp

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

• nlp
• embedding
• bert
• ernie
• paddlenlp
• pretrained-models

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

• python
• search
• nlp
• semantic-search
• vector-search
• txtai

An open-source NLP research library, built on PyTorch.

• pytorch
• nlp
• natural-language-processing
• deep-learning
• data-science
• python

Large Language Model Text Generation Inference

• bloom
• nlp
• pytorch
• inference
• gpt
• deep-learning

This repository contains code examples for the Stanford's course: TensorFlow for Deep Learning Research.

• tensorflow
• deep-learning
• tutorial
• nlp
• natural-language-processing
• chatbot

Pre-Training with Whole Word Masking for Chinese BERT（中文BERT-wwm系列模型）

• chinese-bert
• tensorflow
• pytorch
• bert
• nlp
• roberta

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

• bloom
• deep-learning
• distributed-systems
• language-models
• large-language-models
• machine-learning

A PyTorch implementation of the Transformer model in "Attention is All You Need".

• attention
• deep-learning
• attention-is-all-you-need
• pytorch
• nlp
• natural-language-processing