跳转到主要内容

标签(标签)

资源精选(342) Go开发(108) Go语言(103) Go(99) angular(82) LLM(75) 大语言模型(63) 人工智能(53) 前端开发(50) LangChain(43) golang(43) 机器学习(39) Go工程师(38) Go程序员(38) Go开发者(36) React(33) Go基础(29) Python(24) Vue(22) Web开发(20) Web技术(19) 精选资源(19) 深度学习(19) Java(18) ChatGTP(17) Cookie(16) android(16) 前端框架(13) JavaScript(13) Next.js(12) 安卓(11) 聊天机器人(10) typescript(10) 资料精选(10) NLP(10) 第三方Cookie(9) Redwoodjs(9) LLMOps(9) Go语言中级开发(9) 自然语言处理(9) PostgreSQL(9) 区块链(9) mlops(9) 安全(9) 全栈开发(8) ChatGPT(8) OpenAI(8) Linux(8) AI(8) GraphQL(8) iOS(8) 软件架构(7) Go语言高级开发(7) AWS(7) C++(7) 数据科学(7) whisper(6) Prisma(6) 隐私保护(6) RAG(6) JSON(6) DevOps(6) 数据可视化(6) wasm(6) 计算机视觉(6) 算法(6) Rust(6) 微服务(6) 隐私沙盒(5) FedCM(5) 语音识别(5) Angular开发(5) 快速应用开发(5) 提示工程(5) Agent(5) LLaMA(5) 低代码开发(5) Go测试(5) gorm(5) REST API(5) 推荐系统(5) WebAssembly(5) GameDev(5) CMS(5) CSS(5) machine-learning(5) 机器人(5) 游戏开发(5) Blockchain(5) Web安全(5) Kotlin(5) 低代码平台(5) 机器学习资源(5) Go资源(5) Nodejs(5) PHP(5) Swift(5) 智能体(4) devin(4) Blitz(4) javascript框架(4) Redwood(4) GDPR(4) 生成式人工智能(4) Angular16(4) Alpaca(4) 编程语言(4) SAML(4) JWT(4) JSON处理(4) Go并发(4) kafka(4) 移动开发(4) 移动应用(4) security(4) 隐私(4) spring-boot(4) 物联网(4) nextjs(4) 网络安全(4) API(4) Ruby(4) 信息安全(4) flutter(4) 专家智能体(3) Chrome(3) CHIPS(3) 3PC(3) SSE(3) 人工智能软件工程师(3) LLM Agent(3) Remix(3) Ubuntu(3) GPT4All(3) 软件开发(3) 问答系统(3) 开发工具(3) 最佳实践(3) RxJS(3) SSR(3) Node.js(3) Dolly(3) 移动应用开发(3) 低代码(3) IAM(3) Web框架(3) CORS(3) 基准测试(3) Go语言数据库开发(3) Oauth2(3) 并发(3) 主题(3) Theme(3) earth(3) nginx(3) 软件工程(3) azure(3) keycloak(3) 生产力工具(3) gpt3(3) 工作流(3) C(3) jupyter(3) 认证(3) prometheus(3) GAN(3) Spring(3) 逆向工程(3) 应用安全(3) Docker(3) Django(3) R(3) .NET(3) 大数据(3) Hacking(3) 渗透测试(3) C++资源(3) Mac(3) 微信小程序(3) Python资源(3) JHipster(3) 大型语言模型(2) 语言模型(2) 可穿戴设备(2) JDK(2) SQL(2) Apache(2) Hashicorp Vault(2) Spring Cloud Vault(2) Go语言Web开发(2) Go测试工程师(2) WebSocket(2) 容器化(2) AES(2) 加密(2) 输入验证(2) ORM(2) Fiber(2) Postgres(2) Gorilla Mux(2) Go数据库开发(2) 模块(2) 泛型(2) 指针(2) HTTP(2) PostgreSQL开发(2) Vault(2) K8s(2) Spring boot(2) R语言(2) 深度学习资源(2) 半监督学习(2) semi-supervised-learning(2) architecture(2) 普罗米修斯(2) 嵌入模型(2) productivity(2) 编码(2) Qt(2) 前端(2) Rust语言(2) NeRF(2) 神经辐射场(2) 元宇宙(2) CPP(2) 数据分析(2) spark(2) 流处理(2) Ionic(2) 人体姿势估计(2) human-pose-estimation(2) 视频处理(2) deep-learning(2) kotlin语言(2) kotlin开发(2) burp(2) Chatbot(2) npm(2) quantum(2) OCR(2) 游戏(2) game(2) 内容管理系统(2) MySQL(2) python-books(2) pentest(2) opengl(2) IDE(2) 漏洞赏金(2) Web(2) 知识图谱(2) PyTorch(2) 数据库(2) reverse-engineering(2) 数据工程(2) swift开发(2) rest(2) robotics(2) ios-animation(2) 知识蒸馏(2) 安卓开发(2) nestjs(2) solidity(2) 爬虫(2) 面试(2) 容器(2) C++精选(2) 人工智能资源(2) Machine Learning(2) 备忘单(2) 编程书籍(2) angular资源(2) 速查表(2) cheatsheets(2) SecOps(2) mlops资源(2) R资源(2) DDD(2) 架构设计模式(2) 量化(2) Hacking资源(2) 强化学习(2) flask(2) 设计(2) 性能(2) Sysadmin(2) 系统管理员(2) Java资源(2) 机器学习精选(2) android资源(2) android-UI(2) Mac资源(2) iOS资源(2) Vue资源(2) flutter资源(2) JavaScript精选(2) JavaScript资源(2) Rust开发(2) deeplearning(2) RAD(2)

Awesome-AutoML-Papers 是自动化机器学习论文、文章、教程、幻灯片和项目的精选列表。 为这个知识库加注星标,然后您就可以及时了解这个蓬勃发展的研究领域的最新进展。 感谢所有为这个项目做出贡献的人。 

What is AutoML?

Automated Machine Learning (AutoML) provides methods and processes to make Machine Learning available for non-Machine Learning experts, to improve efficiency of Machine Learning and to accelerate research on Machine Learning.

Machine Learning (ML) has achieved considerable successes in recent years and an ever-growing number of disciplines rely on it. However, this success crucially relies on human machine learning experts to perform the following tasks:

  • Preprocess the data,
  • Select appropriate features,
  • Select an appropriate model family,
  • Optimize model hyperparameters,
  • Postprocess machine learning models,
  • Critically analyze the results obtained.

As the complexity of these tasks is often beyond non-ML-experts, the rapid growth of machine learning applications has created a demand for off-the-shelf machine learning methods that can be used easily and without expert knowledge. We call the resulting research area that targets progressive automation of machine learning AutoML. As a new sub-area in machine learning, AutoML has got more attention not only in machine learning but also in computer vision, natural language processing and graph computing.

There are no formal definition of AutoML. From the descriptions of most papers,the basic procedure of AutoML can be shown as the following.

AutoML approaches are already mature enough to rival and sometimes even outperform human machine learning experts. Put simply, AutoML can lead to improved performance while saving substantial amounts of time and money, as machine learning experts are both hard to find and expensive. As a result, commercial interest in AutoML has grown dramatically in recent years, and several major tech companies and start-up companies are now developing their own AutoML systems. An overview comparison of some of them can be summarized to the following table.

Company AutoFE HPO NAS
4paradigm ×
Alibaba × ×
Baidu × ×
Determined AI ×
Google
DataCanvas
H2O.ai ×
Microsoft ×
MLJAR
RapidMiner ×
Tencent × ×

Awesome-AutoML-Papers includes very up-to-date overviews of the bread-and-butter techniques we need in AutoML:

  • Automated Data Clean (Auto Clean)
  • Automated Feature Engineering (Auto FE)
  • Hyperparameter Optimization (HPO)
  • Meta-Learning
  • Neural Architecture Search (NAS)

Table of Contents

Papers

Surveys

  • 2019 | AutoML: A Survey of the State-of-the-Art | Xin He, et al. | arXiv | PDF
  • 2019 | Survey on Automated Machine Learning | Marc Zoeller, Marco F. Huber | arXiv | PDF
  • 2019 | Automated Machine Learning: State-of-The-Art and Open Challenges | Radwa Elshawi, et al. | arXiv | PDF
  • 2018 | Taking Human out of Learning Applications: A Survey on Automated Machine Learning | Quanming Yao, et al. | arXiv | PDF
  • 2020 | On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice | Li Yang, et al. | Neurocomputing | PDF
  • 2020 | Automated Machine Learning--a brief review at the end of the early years | Escalante, H. J. | arXiv | PDF

Automated Feature Engineering

  • Expand Reduce

    • 2017 | AutoLearn — Automated Feature Generation and Selection | Ambika Kaul, et al. | ICDM | PDF
    • 2017 | One button machine for automating feature engineering in relational databases | Hoang Thanh Lam, et al. | arXiv | PDF
    • 2016 | Automating Feature Engineering | Udayan Khurana, et al. | NIPS | PDF
    • 2016 | ExploreKit: Automatic Feature Generation and Selection | Gilad Katz, et al. | ICDM | PDF
    • 2015 | Deep Feature Synthesis: Towards Automating Data Science Endeavors | James Max Kanter, Kalyan Veeramachaneni | DSAA | PDF
  • Hierarchical Organization of Transformations

    • 2016 | Cognito: Automated Feature Engineering for Supervised Learning | Udayan Khurana, et al. | ICDMW | PDF
  • Meta Learning

    • 2020 | AutoML Pipeline Selection: Efficiently Navigating the Combinatorial Space | Chengrun Yang, et al. | KDD | PDF
    • 2017 | Learning Feature Engineering for Classification | Fatemeh Nargesian, et al. | IJCAI | PDF
  • Reinforcement Learning

    • 2017 | Feature Engineering for Predictive Modeling using Reinforcement Learning | Udayan Khurana, et al. | arXiv | PDF
    • 2010 | Feature Selection as a One-Player Game | Romaric Gaudel, Michele Sebag | ICML | PDF

Architecture Search

  • Evolutionary Algorithms

    • 2019 | Evolutionary Neural AutoML for Deep Learning | Jason Liang, et al. | GECCO | PDF
    • 2017 | Large-Scale Evolution of Image Classifiers | Esteban Real, et al. | PMLR | PDF
    • 2002 | Evolving Neural Networks through Augmenting Topologies | Kenneth O.Stanley, Risto Miikkulainen | Evolutionary Computation | PDF
  • Local Search

    • 2017 | Simple and Efficient Architecture Search for Convolutional Neural Networks | Thomoas Elsken, et al. | ICLR | PDF
  • Meta Learning

    • 2016 | Learning to Optimize | Ke Li, Jitendra Malik | arXiv | PDF
  • Reinforcement Learning

    • 2018 | AMC: AutoML for Model Compression and Acceleration on Mobile Devices | Yihui He, et al. | ECCV | PDF
    • 2018 | Efficient Neural Architecture Search via Parameter Sharing | Hieu Pham, et al. | arXiv | PDF
    • 2017 | Neural Architecture Search with Reinforcement Learning | Barret Zoph, Quoc V. Le | ICLR | PDF
  • Transfer Learning

    • 2017 | Learning Transferable Architectures for Scalable Image Recognition | Barret Zoph, et al. | arXiv | PDF
  • Network Morphism

    • 2019 | Auto-Keras: An Efficient Neural Architecture Search System | Haifeng Jin, et al. | KDD | PDF
  • Continuous Optimization

    • 2018 | Neural Architecture Optimization | Renqian Luo, et al. | arXiv | PDF
    • 2019 | DARTS: Differentiable Architecture Search | Hanxiao Liu, et al. | ICLR | PDF

Frameworks

  • 2019 | Auptimizer -- an Extensible, Open-Source Framework for Hyperparameter Tuning | Jiayi Liu, et al. | IEEE Big Data | PDF
  • 2019 | Towards modular and programmable architecture search | Renato Negrinho, et al. | NeurIPS | PDF
  • 2019 | Evolutionary Neural AutoML for Deep Learning | Jason Liang, et al. | arXiv | PDF
  • 2017 | ATM: A Distributed, Collaborative, Scalable System for Automated Machine Learning | T. Swearingen, et al. | IEEE | PDF
  • 2017 | Google Vizier: A Service for Black-Box Optimization | Daniel Golovin, et al. | KDD |PDF
  • 2015 | AutoCompete: A Framework for Machine Learning Competitions | Abhishek Thakur, et al. | ICML | PDF

Hyperparameter Optimization

  • Bayesian Optimization

    • 2020 | Bayesian Optimization of Risk Measures | NeurIPS | PDF
    • 2020 | BOTORCH: A Framework for Efficient Monte-Carlo Bayesian Optimization | NeurIPS | PDF
    • 2020 | Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly | JMLR | PDF
    • 2019 | Bayesian Optimization with Unknown Search Space | NeurIPS | PDF
    • 2019 | Constrained Bayesian optimization with noisy experiments | PDF
    • 2019 | Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning | NeurIPS | PDF
    • 2019 | Practical Two-Step Lookahead Bayesian Optimization | NeurIPS | PDF
    • 2019 | Predictive entropy search for multi-objective bayesian optimization with constraints | PDF
    • 2018 | BOCK: Bayesian optimization with cylindrical kernels | ICML | PDF
    • 2018 | Efficient High Dimensional Bayesian Optimization with Additivity and Quadrature Fourier Features | Mojmír Mutný, et al. | NeurIPS | PDF
    • 2018 | High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups. | PMLR | PDF
    • 2018 | Maximizing acquisition functions for Bayesian optimization | NeurIPS | PDF
    • 2018 | Scalable hyperparameter transfer learning | NeurIPS | PDF
    • 2016 | Bayesian Optimization with Robust Bayesian Neural Networks | Jost Tobias Springenberg, et al. | NIPS | PDF
    • 2016 | Scalable Hyperparameter Optimization with Products of Gaussian Process Experts | Nicolas Schilling, et al. | PKDD | PDF
    • 2016 | Taking the Human Out of the Loop: A Review of Bayesian Optimization | Bobak Shahriari, et al. | IEEE | PDF
    • 2016 | Towards Automatically-Tuned Neural Networks | Hector Mendoza, et al. | JMLR | PDF
    • 2016 | Two-Stage Transfer Surrogate Model for Automatic Hyperparameter Optimization | Martin Wistuba, et al. | PKDD | PDF
    • 2015 | Efficient and Robust Automated Machine Learning | PDF
    • 2015 | Hyperparameter Optimization with Factorized Multilayer Perceptrons | Nicolas Schilling, et al. | PKDD | PDF
    • 2015 | Hyperparameter Search Space Pruning - A New Component for Sequential Model-Based Hyperparameter Optimization | Martin Wistua, et al. | PDF
    • 2015 | Joint Model Choice and Hyperparameter Optimization with Factorized Multilayer Perceptrons | Nicolas Schilling, et al. | ICTAI | PDF
    • 2015 | Learning Hyperparameter Optimization Initializations | Martin Wistuba, et al. | DSAA | PDF
    • 2015 | Scalable Bayesian optimization using deep neural networks | Jasper Snoek, et al. | ACM | PDF
    • 2015 | Sequential Model-free Hyperparameter Tuning | Martin Wistuba, et al. | ICDM | PDF
    • 2013 | Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms | PDF
    • 2013 | Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures | J. Bergstra | JMLR | PDF
    • 2012 | Practical Bayesian Optimization of Machine Learning Algorithms | PDF
    • 2011 | Sequential Model-Based Optimization for General Algorithm Configuration(extended version) | PDF
  • Evolutionary Algorithms

    • 2020 | Delta-STN: Efficient Bilevel Optimization for Neural Networks using Structured Response Jacobians | Juhan Bae, Roger Grosse | Neurips | PDF
    • 2018 | Autostacker: A Compositional Evolutionary Learning System | Boyuan Chen, et al. | arXiv | PDF
    • 2017 | Large-Scale Evolution of Image Classifiers | Esteban Real, et al. | PMLR | PDF
    • 2016 | Automating biomedical data science through tree-based pipeline optimization | Randal S. Olson, et al. | ECAL | PDF
    • 2016 | Evaluation of a tree-based pipeline optimization tool for automating data science | Randal S. Olson, et al. | GECCO | PDF
  • Lipschitz Functions

    • 2017 | Global Optimization of Lipschitz functions | C´edric Malherbe, Nicolas Vayatis | arXiv | PDF
  • Local Search

    • 2009 | ParamILS: An Automatic Algorithm Configuration Framework | Frank Hutter, et al. | JAIR | PDF
  • Meta Learning

    • 2019 | OBOE: Collaborative Filtering for AutoML Model Selection | Chengrun Yang, et al. | KDD | PDF
    • 2019 | SMARTML: A Meta Learning-Based Framework for Automated Selection and Hyperparameter Tuning for Machine Learning Algorithms | PDF
    • 2008 | Cross-Disciplinary Perspectives on Meta-Learning for Algorithm Selection | PDF
  • Particle Swarm Optimization

    • 2017 | Particle Swarm Optimization for Hyper-parameter Selection in Deep Neural Networks | Pablo Ribalta Lorenzo, et al. | GECCO | PDF
    • 2008 | Particle Swarm Optimization for Parameter Determination and Feature Selection of Support Vector Machines | Shih-Wei Lin, et al. | Expert Systems with Applications | PDF
  • Random Search

    • 2016 | Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization | Lisha Li, et al. | arXiv | PDF
    • 2012 | Random Search for Hyper-Parameter Optimization | James Bergstra, Yoshua Bengio | JMLR | PDF
    • 2011 | Algorithms for Hyper-parameter Optimization | James Bergstra, et al. | NIPS | PDF
  • Transfer Learning

    • 2016 | Efficient Transfer Learning Method for Automatic Hyperparameter Tuning | Dani Yogatama, Gideon Mann | JMLR | PDF
    • 2016 | Flexible Transfer Learning Framework for Bayesian Optimisation | Tinu Theckel Joy, et al. | PAKDD | PDF
    • 2016 | Hyperparameter Optimization Machines | Martin Wistuba, et al. | DSAA | PDF
    • 2013 | Collaborative Hyperparameter Tuning | R´emi Bardenet, et al. | ICML | PDF

Miscellaneous

  • 2020 | Automated Machine Learning Techniques for Data Streams | Alexandru-Ionut Imbrea | PDF
  • 2018 | Accelerating Neural Architecture Search using Performance Prediction | Bowen Baker, et al. | ICLR | PDF
  • 2017 | Automatic Frankensteining: Creating Complex Ensembles Autonomously | Martin Wistuba, et al. | SIAM | PDF
  • 2018 | Characterizing classification datasets: A study of meta-features for meta-learning | Rivolli, Adriano, et al. | arXiv | PDF
  • 2020 | Putting the Human Back in the AutoML Loop | Xanthopoulos, Iordanis, et al. | EDBT/ICDT | PDF

Tutorials

Bayesian Optimization

  • 2018 | A Tutorial on Bayesian Optimization. | PDF
  • 2010 | A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning | PDF

Meta Learning

  • 2008 | Metalearning - A Tutorial | PDF

Blog

Type Blog Title Link
HPO Bayesian Optimization for Hyperparameter Tuning Link
Meta-Learning Learning to learn Link
Meta-Learning Why Meta-learning is Crucial for Further Advances of Artificial Intelligence? Link

Books

Year of Publication Type Book Title Authors Publisher Link
2009 Meta-Learning Metalearning - Applications to Data Mining Brazdil, P., Giraud Carrier, C., Soares, C., Vilalta, R. Springer Download
2019 HPO, Meta-Learning, NAS AutoML: Methods, Systems, Challenges Frank Hutter, Lars Kotthoff, Joaquin Vanschoren   Download
2021 Learning Automated Machine Learning in Action Qinquan Song, Haifeng Jin, Xia Hu Manning Publications Download

Videos

| Title | Author | Link | | AutoML Basics: Automated Machine Learning in Action | Qinquan Song, Haifeng Jin, Xia Hu | (https://www.youtube.com/watch?v=9KpieG0B7VM) |

Projects

Project Type Language License Link
AdaNet NAS Python Apache-2.0 Github
Advisor HPO Python Apache-2.0 Github
AMLA HPO, NAS Python Apache-2.0 Github
ATM HPO Python MIT Github
Auger HPO Python Commercial Homepage
auptimizer HPO, NAS Python (support R script) GPL-3.0 Github
Auto-Keras NAS Python License Github
AutoML Vision NAS Python Commercial Homepage
AutoML Video Intelligence NAS Python Commercial Homepage
AutoML Natural Language NAS Python Commercial Homepage
AutoML Translation NAS Python Commercial Homepage
AutoML Tables AutoFE, HPO Python Commercial Homepage
AutoPyTorch HPO, NAS Python Apache-2.0 Github
HyperGBM HPO Python Python Github
HyperKeras NAS Python Python Github
Hypernets HPO, NAS Python Python Github
auto-sklearn HPO Python License Github
auto_ml HPO Python MIT Github
BayesianOptimization HPO Python MIT Github
BayesOpt HPO C++ AGPL-3.0 Github
comet HPO Python Commercial Homepage
DataRobot HPO Python Commercial Homepage
DEvol NAS Python MIT Github
DeepArchitect NAS Python MIT Github
Determined HPO, NAS Python Apache-2.0 Github
Driverless AI AutoFE Python Commercial Homepage
FAR-HO HPO Python MIT Github
H2O AutoML HPO Python, R, Java, Scala Apache-2.0 Github
HpBandSter HPO Python BSD-3-Clause Github
HyperBand HPO Python License Github
Hyperopt HPO Python License Github
Hyperopt-sklearn HPO Python License Github
Hyperparameter Hunter HPO Python MIT Github
Katib HPO Python Apache-2.0 Github
MateLabs HPO Python Commercial Github
Milano HPO Python Apache-2.0 Github
MLJAR AutoFE, HPO, NAS Python MIT Github
mlr3automl HPO R LGPL-3.0 GitHub
nasbot NAS Python MIT Github
neptune HPO Python Commercial Homepage
NNI HPO, NAS Python MIT Github
Oboe HPO Python BSD-3-Clause Github
Optunity HPO Python License Github
R2.ai HPO   Commercial Homepage
RBFOpt HPO Python License Github
RoBO HPO Python BSD-3-Clause Github
Scikit-Optimize HPO Python License Github
SigOpt HPO Python Commercial Homepage
SMAC3 HPO Python License Github
TPOT AutoFE, HPO Python LGPL-3.0 Github
TransmogrifAI HPO Scala BSD-3-Clause Github
Tune HPO Python Apache-2.0 Github
Xcessiv HPO Python Apache-2.0 Github
SmartML HPO R GPL-3.0 Github
MLBox AutoFE, HPO Python BSD-3 License Github
AutoAI Watson AutoFE, HPO   Commercial Homepage

Slides

Type Slide Title Authors Link
AutoFE Automated Feature Engineering for Predictive Modeling Udyan Khurana, etc al. Download
HPO A Tutorial on Bayesian Optimization for Machine Learning Ryan P. Adams Download
HPO Bayesian Optimisation Gilles Louppe Download

Acknowledgement

Special thanks to everyone who contributed to this project.

Name Bio
Alexander Robles PhD Student @UNICAMP-Brazil
derekflint  
endymecy Senior Researcher @Tencent
Eric  
Erin LeDell Chief Machine Learning Scientist @H2O.ai
fwcore  
Gaurav Mittal  
Hernan Ceferino Vazquez PhD, Data Science Expert @MercadoLibre
Kaustubh Damania  
Lilian Besson PhD Student @CentraleSupélec
罗磊  
Marc  
Mohamed Maher  
Neil Conway CTO @Determined AI
Richard Liaw PhD Student @UC Berkeley
Randy Olson Lead Data Scientist @LifeEGX
Slava Kurilyak Founder, CEO @Produvia
Saket Maheshwary AI Researcher
shaido987  
sophia-wright-blue  
tengben0905  
xuehui @Microsoft
Yihui He Grad Student @CMU

原文:https://github.com/hibayesian/awesome-automl-papers