Book about NNI

[Ivan Gridin (survexman)]

Hi everyone!

For me, NNI is a very promising framework. I think it can be the best and very popular framework for AutoML and NAS tasks.

There is an idea to write a book about NNI: "Hands-on Neural Network Intelligence".

I'd like to give explanations of the main concepts and ideas. Give a lot of practical examples of hyper-parameter optimization and especially cover the NAS topic.

I already have experience in writing, this is one of my books:
https://www.amazon.com/Learning-Genetic-Algorithms-Python-capabilities-ebook/dp/B08WKBFHGV

I think that can help popularize the framework.

What do you think?

Any ideas and contributions appreciated 👍

[Scarlett Li (scarlett2018)]

Ivan Gridin (@survexman) - Appreciate for the comment and believes in NNI. I'm the PM for the project. We highly encourage our community contributors to share experience through examples, tutorials, books.

Would you like to list out an outline for your idea? Feel free to use this thread to gather people joining as author(s). We can also help to distribute the recruit in other NNI fans channels.

[Ivan Gridin (survexman)]

Scarlett Li (@scarlett2018) , thank you for your reply!

At first, I want to say that I'll keep this book updated. It will not be too hard for me because I'm a permanent user of NNI. It means there'll be different editions that contain all actual updates.

This book is for practical data scientists and ML experts. This book does not cover the essentials of the machine and deep learning.

I want to choose PyTorch as a primary deep learning framework for practical examples in the book.

Here is the outline.

Chapter 1. Introduction to NN

• Installation

• Trial, Search Space, Experiment
  Explaining and visualizing basic concepts of optimization in NNI context.

• Finding maxima of a multivariate function
  NNI can be used as an optimization engine for any problem. I want to start explaining NNI principles from the purely mathematical task of finding the maxima of a function using a simple hyper-parameter tuning engine.

• Interacting with NNI
  About NNI environment, its GUI, how to save results and etc.

Chapter 2. Hyper-parameter tuning

The Real-world time-series prediction LSTM model (weather: rain prediction) will be used as a practical example for this chapter.

• Preparing a model for hyper-parameter tuning

• Running Experiment

• Interpreting Results

• Debugging

Chapter 3. Hyper-parameter tuners

Description of main concepts of most popular tuners. The main idea is to give a reader an intuitive explanation of the most popular tuners. Because often developers do not understand which tuner to chose and how it works.

Chapter 4. Neural Architecture Search: Multi-trial

This is the most exciting part for me. I think NAS techniques sometimes are treated as very complicated methods. And usually, developers and data scientists use hyper-parameter tuning only. I'd like to help people go a little further here because it is not so hard as it seems.

We will take the text classification problem as an example for this chapter.

• Constructing a search space for architecture search

• Running architecture search
  (with random strategy)

• Exploration strategies (short overview of Random Strategy, RL Strategy, etc.)

• Comparing exploration strategies

Chapter 5. Neural Architecture Search: One-shot

Here I will describe the main idea of one-shot NAS and its practical implementations.

• What is one-shot NAS?

• ENAS

• DARTS
  �

Chapter 6. Model Compression

• What is model compression?
  Here I'll explain why model compression is important and why NAS algorithms usually do not support model compression.

• Trying to compress your model
  Simple practical example here.

• Pruning
  Introduction to pruners.

• Quantization
  Explanation and visualization of quantization.

Chapter 7. Advanced NNI techniques

Here we will cover advanced techniques of NNI usage. Distributed experiments, different platforms, speed up methods, etc.

Let me know what do you think?

[Ivan Gridin (survexman)]

Scarlett Li (@scarlett2018) - I can send a request to some publishers by myself, or maybe you can help me with this somehow?

[Scarlett Li (scarlett2018)]

Scarlett Li (@scarlett2018) - I can send a request to some publishers by myself, or maybe you can help me with this somehow?

Hi Ivan Gridin (@survexman), there do have publishers contact me, let me connect some of them to you.

[Ivan Gridin (survexman)]

Happy to announce that the book about NNI has been published recently!

https://www.amazon.com/Automated-Learning-Neural-Network-Intelligence/dp/1484281489/

[Shabnam Ataee (shabnam-ataee)]

Thanks! It is very useful, but I am not able to run codes in nni 3.0

I would like to use TensorFlow implementation of DARTs (which is not in nni 2.7). Could you please advise me how I can use it?