Top Python Natural language processing repositories on GitHub

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

• nlp
• natural-language-processing
• pytorch
• pytorch-transformers
• transformer
• model-hub

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

• fine-tuning
• llama
• llm
• peft
• transformers
• rlhf

AiLearning：数据分析+机器学习实战+线性代数+PyTorch+NLTK+TF2

• fp-growth
• apriori
• mahchine-leaning
• naivebayes
• svm
• adaboost

微舆：人人可用的多Agent舆情分析助手，打破信息茧房，还原舆情原貌，预测未来走向，辅助决策！从0实现，不依赖任何框架。

• agent-framework
• data-analysis
• multi-agent-system
• nlp
• public-opinion-analysis
• python3

TensorFlow code and pre-trained models for BERT

• nlp
• google
• natural-language-processing
• natural-language-understanding
• tensorflow

A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.

• llm
• nlp
• python
• gemini-ai
• information-extration
• large-language-models

Natural Language Processing for the next decade. Tokenization, Part-of-Speech Tagging, Named Entity Recognition, Syntactic & Semantic Dependency Parsing, Document Classification

• nlp
• natural-language-processing
• hanlp
• pos-tagging
• dependency-parser
• text-classification

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

• calibration
• classification
• decision-model
• huggingface
• jev
• modernbert

💫 Industrial-strength Natural Language Processing (NLP) in Python

• natural-language-processing
• data-science
• machine-learning
• python
• cython
• nlp

An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.

• large-language-models
• nlp
• knowledge-curation
• naacl
• report-generation
• retrieval-augmented-generation

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

• nlp
• pre-trained-model
• unilm
• minilm
• layoutlm
• layoutxlm

🤗 The largest hub of ready-to-use datasets for AI models with fast, easy-to-use and efficient data manipulation tools

• nlp
• datasets
• pytorch
• tensorflow
• pandas
• numpy

💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants

• nlp
• machine-learning
• machine-learning-library
• bot
• bots
• botkit

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

• llm
• plm
• pre-trained-language-models
• alpaca
• llama
• nlp

Topic Modelling for Humans

• gensim
• topic-modeling
• information-retrieval
• machine-learning
• natural-language-processing
• nlp

Nano vLLM

• inference
• llm
• pytorch
• transformer
• deep-learning
• nlp

NLTK Source

• nltk
• python
• nlp
• natural-language-processing
• machine-learning

A very simple framework for state-of-the-art Natural Language Processing (NLP)

• pytorch
• nlp
• named-entity-recognition
• sequence-labeling
• semantic-role-labeling
• word-embeddings

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

• python
• search
• nlp
• semantic-search
• vector-search
• txtai