microsoft/nni

Does ProxylessNas implementation really support optimizing inference latency?

Open

#3,113 opened on Nov 22, 2020

View on GitHub
 (1 comment) (0 reactions) (1 assignee)Python (1,830 forks)batch import
NAShelp wantednew featureuser raised

Repository metrics

Stars
 (13,504 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

Environment:

  • NNI version: v1.9 (including current master)
  • NNI mode (local|remote|pai):
  • Client OS:
  • Server OS (for remote mode only):
  • Python version:
  • PyTorch/TensorFlow version:
  • Is conda/virtualenv/venv used?:
  • Is running in Docker?:

Log message:

  • nnimanager.log:
  • dispatcher.log:
  • nnictl stdout and stderr:

What issue meet, what's expected?:

The most important feature of ProxylessNas is that it can balance deployment latency and accuracy with simple regularization parameters. But this feature is clearly missing in nni. I only found loss = criterion(outputs, labels) and loss.backward() where the criterion is just a cross-entropy loss and only applicable for image classification.

Would nni team consider adding this feature? If not, would you mind I writing this feature and pulling a request?

How to reproduce it?:

Additional information:

Contributor guide

Research direction
Investigate how to incorporate latency regularization into ProxylessNAS loss function. This involves defining a latency prediction model or lookup table, and adding a latency term to the loss. The current implementation only uses cross entropy loss. You need to modify the training loop to compute latency penalty based on the architecture weights and backpropagate through it.
Tech stack
pythonpytorch
Domain
machine learning
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Mostly clear
Prerequisites
PythonPyTorchNeural Architecture Search basics
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40