Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models

A curated list for Efficient Large Language Models

[NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support Llama-3/3.1, Llama-2, LLaMA, BLOOM, Vicuna, Baichuan, TinyLlama, etc.

Automated Identification of Redundant Layer Blocks for Pruning in Large Language Models

A research library for pytorch-based neural network pruning, compression, and more.

[AAAI 2024] Fluctuation-based Adaptive Structured Pruning for Large Language Models

Model optimizer used in Adlik.

Caffe/Neon prototxt training file for our Neurocomputing2017 work: Fuzzy Quantitative Deep Compression Network

This project provides tools to load and prune large language models using a structured pruning method.

KEN: Unleash the power of large language models with the easiest and universal non-parametric pruning algorithm

Official code for "EC-SNN: Splitting Deep Spiking Neural Networks on Edge Devices" (IJCAI2024)

Hierarchical Ensemble Pruning

[EMNLP 2024] Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning

This repository has the porpouse of give a solution to the travelling sales man problem

[PRL 2024] This is the code repo for our label-free pruning and retraining technique for autoregressive Text-VQA Transformers (TAP, TAP†).

This repository contains scripts to prune Wav2vec2 using a neuroevolution-based method. More details about this method can be found in the paper Compressing Wav2vec2 for Embedded Applications.

[JCST 2023] "Inductive Lottery Ticket Learning for Graph Neural Networks" by Yongduo Sui, Xiang Wang, Tianlong Chen, Meng Wang, Xiangnan He, Tat-Seng Chua.

Experiments for channel-based Structured Pruning Adapters

a C++ chess engine with support for full chess rules and an AI opponent. It implements move generation, position evaluation, and a minimax search with alpha-beta pruning to choose the best moves. This project is designed to be modular and extensible.

This project implements Genetic Programming (GP) to evolve and optimize mathematical expressions that best fit given data. It leverages tree-based evolutionary algorithms to generate, evaluate, and refine expressions using selection, crossover, and mutation.