Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify slides #43

Merged
merged 3 commits into from
Jul 1, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions slides.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,18 +44,18 @@ def sort_paths(plist):
number1 = int(''.join(filter(str.isdigit, nxt_element.name)))
number2 = int(''.join(filter(str.isdigit, plist[j].name)))

if number1 >= 10:
if number1 > 10 and number1 <= 90:
number1 /= 10

if number2 >= 10:
if number2 > 10 and number2 <= 90:
number2 /= 10

while number2 > number1 and j >= 0:
plist[j+1] = plist[j]
j = j - 1
number2 = int(''.join(filter(str.isdigit, plist[j].name)))

if number2 >= 10:
if number2 > 10 and number2 <= 90:
number2 /= 10
plist[j+1] = nxt_element

Expand Down
Binary file added slides/autoML.pdf
Binary file not shown.
Binary file modified slides/w01_big_picture.pdf
Binary file not shown.
Binary file modified slides/w02_evaluation.pdf
Binary file not shown.
Binary file modified slides/w03_algorithm_selection.pdf
Binary file not shown.
Binary file modified slides/w04_hpo_basics.pdf
Binary file not shown.
Binary file modified slides/w05_gps.pdf
Binary file not shown.
Binary file modified slides/w06_hpo_bo.pdf
Binary file not shown.
Binary file modified slides/w07_hpo_speedup.pdf
Binary file not shown.
Binary file modified slides/w08_hpo_multicrit.pdf
Binary file not shown.
Binary file added slides/w09_nas.pdf
Binary file not shown.
Binary file modified slides/w11_l2l_hpo_control.pdf
Binary file not shown.
Binary file modified slides/w12_analysis.pdf
Binary file not shown.
Binary file modified slides/w13_beyond_automl.pdf
Binary file not shown.
Binary file modified slides/w15_wrap_up.pdf
Binary file not shown.
Binary file added w09_nas/images/AutoPytorch-ConfigSpace.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added w09_nas/images/AutoPytorch-ShapedResNet.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added w09_nas/images/AutoPytorch-performance.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added w09_nas/images/conv_fabric_1.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added w09_nas/images/conv_fabric_2.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added w09_nas/images/conv_fabric_3.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified w09_nas/t07_oneshot_nas.pdf
Binary file not shown.
19 changes: 11 additions & 8 deletions w09_nas/t07_oneshot_nas.tex
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@
\item The first type of one-shot models: \alert{convolutional neural fabrics} %/ \alert{lattice}

\centering
\only<2-5>{\includegraphics[width=0.7\textwidth]{images/conv_fabric_1.png}\\}
\only<6>{\hspace*{-0.91cm}\vspace*{0.3cm}\includegraphics[width=1.02\textwidth]{images/conv_fabric_3.png}\vspace*{-0.5cm}}
\only<2-5>{\includegraphics[width=0.7\textwidth]{images/conv_fabric_1.jpg}\\}
\only<6>{\hspace*{-0.91cm}\vspace*{0.3cm}\includegraphics[width=1.02\textwidth]{images/conv_fabric_3.jpg}\vspace*{-0.5cm}}

\only<2>{\bigskip\bigskip\includegraphics[width=0.7\textwidth]{images/conv_fabric_2.png}}
\only<2>{\bigskip\bigskip\includegraphics[width=0.7\textwidth]{images/conv_fabric_2.jpg}}
\pause
\medskip
\myit{
Expand Down Expand Up @@ -321,20 +321,23 @@
\end{itemize}

\onslide<5>{
\begin{minipage}{0.4\textwidth}
\begin{columns}
\column{0.01\textwidth}
\column{0.4\textwidth}
\begin{itemize}
\footnotesize
\item \textbf{Pitfall:} the correlation between architectures evaluated with the one-shot weights and retrained from scratch (stand-alone models) should be high
\item If not, \textbf{selecting the best architecture based on the one-shot weights} is sub-optimal.
\end{itemize}
\end{minipage}
\hspace{1cm}
\begin{minipage}{0.5\textwidth}
\column{0.5\textwidth}
\begin{center}
\vspace*{-0.3cm}
\includegraphics[width=.7\textwidth]{images/bender_correlation_1.png}\\
\scriptsize{From \smaller{\lit{\href{http://proceedings.mlr.press/v80/bender18a/bender18a.pdf}{Bender et al. 2018}}}}
\end{center}
\end{minipage}
\column{0.09\textwidth}

\end{columns}
}
}
%---------------------------------------------------------------
Expand Down
Binary file modified w09_nas/t09_NASlib.pdf
Binary file not shown.
10 changes: 5 additions & 5 deletions w09_nas/t09_NASlib.tex
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
\begin{document}
\maketitle
%----------------------------------------------------------------------
\myframe{Motivation for NASLib \litw{Zela et al., 2020}}{
\myframe{Motivation for NASLib \litw{Zela et al. 2020}}{

\alert{NASLib} is a \alert{framework} for easily implementing different NAS methods, aiming to:
\medskip
Expand Down Expand Up @@ -132,13 +132,13 @@
\myframe{Tabular benchmarks for one-shot NAS}{
\centering
\myit{
\item NAS-Bench-101 \lit{\href{https://arxiv.org/abs/1902.09635}{Ying et al. 2019}} is not directly compatible with one-shot NAS methods
\item NAS-Bench-101 \lit{\href{https://arxiv.org/pdf/1902.09635.pdf}{Ying et al. 2019}} is not directly compatible with one-shot NAS methods
\myit{
\item Mainly due to the constraint of at most 9 edges in the cell
}
\bigskip
\pause
\item \alert{NAS-Bench-1Shot1} \lit{\href{https://openreview.net/forum?id=SJx9ngStPH}{Zela et al. 2020}}
\item \alert{NAS-Bench-1Shot1} \lit{\href{https://openreview.net/pdf?id=SJx9ngStPH}{Zela et al. 2020}}
\myit{
\item[-] \alert{3 sub-spaces of NAS-Bench-101} that are compatible with one-shot methods
\myit{
Expand All @@ -148,7 +148,7 @@
}
\bigskip
\pause
\item \alert{NAS-Bench-201} \lit{\href{https://openreview.net/forum?id=HJxyZkBKDr}{Dong and Yang. 2020}}
\item \alert{NAS-Bench-201} \lit{\href{https://openreview.net/pdf?id=HJxyZkBKDr}{Dong and Yang. 2020}}
\myit{
% \item[-] Same graph representation as in DARTS
\item[-] Much smaller than NAS-Bench-101 and largest NAS-Bench-1Shot1 subspace
Expand All @@ -167,7 +167,7 @@
\myit{
\item<1-> NAS-Bench-201 is already integrated in NASLib and we can run any one-shot optimizer on it
\medskip
\item<2-> We can also combine random perturbations \lit{\href{https://arxiv.org/pdf/2002.05283.pdf}{Chen and Hsieh, 2020}} with any one-shot optimizer
\item<2-> We can also combine random perturbations \lit{\href{https://arxiv.org/pdf/2002.05283.pdf}{Chen and Hsieh. 2020}} with any one-shot optimizer
\medskip
\item<3-> We can also evaluate black-box optimizers cheaply with a tabular benchmark
}
Expand Down
Binary file added w09_nas/t10_practical_recommendations.pdf
Binary file not shown.
12 changes: 6 additions & 6 deletions w09_nas/t10_practical_recommendations.tex
Original file line number Diff line number Diff line change
Expand Up @@ -62,14 +62,14 @@
\item[-] If the compute budget suffices, \alert{optimize them jointly}, e.g., using BOHB
\myit{
\item[+] Auto-PyTorch Tabular \lit{\href{https://github.com/automl/Auto-PyTorch}{Zimmer,\,Lindauer\,\& Hutter, 2020}}
\item[+] Auto-RL \lit{\href{https://arxiv.org/abs/1812.11951}{Runge et al, 2019}}
\item[+] Auto-RL \lit{\href{https://arxiv.org/pdf/1812.11951.pdf}{Runge et al. 2019}}
}
\medskip
\pause
\item[-] Else
\myit{
\item[+] If you have decent hyperparameters:\\
\alert{run NAS, followed by HPO for fine-tuning} \lit{\href{https://arxiv.org/abs/1905.07443}{Saikat et al, 2019}}
\alert{run NAS, followed by HPO for fine-tuning} \lit{\href{https://arxiv.org/pdf/1905.07443.pdf}{Saikat et al. 2019}}
\smallskip
\pause
\item[+] If you don't have decent hyperparameters: \alert{first run HPO} to get competitive
Expand Down Expand Up @@ -110,18 +110,18 @@

\column{0.29\textwidth}
\onslide<1->{
\includegraphics[width=\linewidth]{images/AutoPytorch-ShapedResNet.png}\\
\includegraphics[width=\linewidth]{images/AutoPytorch-ShapedResNet.jpg}\\
}
\column{0.42\textwidth}
\vspace*{-0.7cm}
\onslide<1->{
\begin{center}
\includegraphics[width=0.9\linewidth]{images/AutoPytorch-ConfigSpace.png}
\includegraphics[width=0.9\linewidth]{images/AutoPytorch-ConfigSpace.jpg}
\end{center}
}
\onslide<5->{
\vspace*{-0.3cm}\hspace*{-1.1cm}
\includegraphics[width=1.1\linewidth]{images/AutoPytorch-performance.png}
\includegraphics[width=1.1\linewidth]{images/AutoPytorch-performance.jpg}
% \end{center}
}

Expand Down Expand Up @@ -158,7 +158,7 @@
}
\onslide<4->{
\myit{
\item Both NAS and HPO improved the state of the art \lit{\href{https://arxiv.org/abs/1905.07443}{Saikat et al, 2019}}:
\item Both NAS and HPO improved the state of the art \lit{\href{https://arxiv.org/pdf/1905.07443.pdf}{Saikat et al. 2019}}:
\myit{
\item End-point-error (EPE) on Sintel dataset: \alert{2.36 $\rightarrow$ 2.14 (by DARTS)}
\item Subsequent HPO: \alert{2.14 $\rightarrow$ 1.94 (by BOHB)}
Expand Down