A General-Purpose Transferable Predictor for Neural Architecture Search
SIAM International Conference on Data Mining, 2023
Neural architecture search spends its time scoring candidates. A predictor can guess that score from the architecture, but most predictors of that period only understood one search space, because they expected a fixed recipe of blocks.
This paper represents each network as a computation graph of primitive operations, then learns the graph with contrastive learning on unlabeled architectures. A small head, fine-tuned on a handful of scored networks, can rank a family the predictor was not built for.
Affiliations on the paper include Huawei's Canadian lab, the University of Alberta, and UBC.
Authors, in the arXiv order: Fred X. Han, Keith G. Mills, Fabian Chudak, Parsa Riahi, Mohammad Salameh, Jialin Zhang, Wei Lu, Shangling Jui, Di Niu.
- NAS-Bench-201
- Spearman 0.917
- NAS-Bench-301
- Spearman 0.892
- NAS-Bench-101 search
- 94.23 percent, 700 queries
- ImageNet
- 79.2 percent top-1
The Spearman figures are means over five fine-tunes. The paper prints a spread of about 0.01. The search and ImageNet numbers are the single reported results. I am not adding figures that are not in the paper.
Other manuscripts from that year are not listed. I have not confirmed they are public, and I will not call a manuscript a paper until it is.