microsoft/nni

Does ProxylessNas implementation really support optimizing inference latency?

Open

#3,113 opened on Nov 22, 2020

View on GitHub
 (1 comment) (0 reactions) (1 assignee)Python (13,504 stars) (1,830 forks)batch import
NAShelp wantednew featureuser raised

Description

Environment:

  • NNI version: v1.9 (including current master)
  • NNI mode (local|remote|pai):
  • Client OS:
  • Server OS (for remote mode only):
  • Python version:
  • PyTorch/TensorFlow version:
  • Is conda/virtualenv/venv used?:
  • Is running in Docker?:

Log message:

  • nnimanager.log:
  • dispatcher.log:
  • nnictl stdout and stderr:

What issue meet, what's expected?:

The most important feature of ProxylessNas is that it can balance deployment latency and accuracy with simple regularization parameters. But this feature is clearly missing in nni. I only found loss = criterion(outputs, labels) and loss.backward() where the criterion is just a cross-entropy loss and only applicable for image classification.

Would nni team consider adding this feature? If not, would you mind I writing this feature and pulling a request?

How to reproduce it?:

Additional information:

Contributor guide

Tech stack
pythonpytorch
Domain
machine learningperformance
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
ProxylessNAS understandingNNI NAS codebasePyTorch
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
35
Research direction
The issue requests adding latency optimization to ProxylessNAS. Currently, only cross entropy loss is used. To implement, study the original ProxylessNAS paper for latency regularization, then modify the loss function in the NNI ProxylessNAS trainer to include a latency penalty. The relevant code is likely under `nni/nas/pytorch/proxylessnas`. Maintainer response is needed to clarify scope.