Top Python Transformers repositories on GitHub

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

• fine-tuning
• llama
• llm
• peft
• transformers
• rlhf

🧑‍🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠

• deep-learning
• deep-learning-tutorial
• pytorch
• gan
• transformers
• reinforcement-learning

Learn it. Build it. Ship it for others.

• agents
• ai
• ai-agents
• ai-engineering
• computer-vision
• course

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

• artificial-intelligence
• attention-mechanism
• transformers
• computer-vision
• image-classification

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

• adapter
• diffusion
• llm
• parameter-efficient-learning
• python
• pytorch

Machine Learning Engineering Open Book

• pytorch
• slurm
• large-language-models
• llm
• machine-learning
• scalability

Private AI platform for agents, assistants and enterprise search. Built-in Agent Builder, Deep research, Document analysis, Multi-model support, and API connectivity for agents.

• ai
• python
• natural-language-processing
• react
• chatgpt
• docsgpt

Ongoing research training transformer models at scale

• large-language-models
• model-para
• transformers

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

• attention-mechanism
• deep-learning
• gpt
• gpt-2
• gpt-3
• language-model

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

• nlp
• embedding
• bert
• ernie
• paddlenlp
• pretrained-models

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

• python
• search
• nlp
• semantic-search
• vector-search
• txtai

A PyTorch-based Speech Toolkit

• speech-recognition
• speech-toolkit
• speaker-recognition
• speech-to-text
• speech-enhancement
• speech-separation

Semantic segmentation models with 500+ pretrained convolutional and transformer-based backbones.

• segmentation
• image-processing
• pspnet
• unet
• unet-pytorch
• pytorch

An open-source framework for detecting, redacting, masking, and anonymizing sensitive data (PII) across text, images, and structured data. Supports NLP, pattern matching, and customizable pipelines.

• python
• pii
• privacy
• data-anonymization
• de-identification
• data-masking

An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)

• transformers
• vllm
• large-language-models
• raylib
• reinforcement-learning-from-human-feedback
• reinforcement-learning

A straightforward method for training your LLM, from downloading data to generating text.

• gemini
• large-language-models
• llm
• openai
• training
• transformers

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

• pytorch
• deployment
• llm
• artificial-intelligence
• machine-learning
• whisper

SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer

• diffusion
• dit
• pytorch
• sana
• text-to-image-generation
• transformers

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.

• pytorch
• llm
• transformers
• gpu