OpenNMT/OpenNMT-tf

Adding benchmarks to Readme

Open

#464 opened on Jul 1, 2019

View on GitHub
 (1 comment) (0 reactions) (0 assignees)Python (1,428 stars) (394 forks)batch import
documentationhelp wanted

Description

Hi, Is it possible to add benchmarks of some models into documentation for comparison purposes ? Also run time would be helpful. For example 1M iteration takes a weekend with GTX 1080.

Contributor guide

Tech stack
pythontensorflow
Domain
documentationmachine learning
Issue type
documentation
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Basic knowledge of OpenNMT tfAccess to GPU for benchmarking
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
70
Research direction
Start by exploring the repository for existing benchmark scripts or model configurations. Check if there are any precomputed benchmark results elsewhere. Run the training scripts on a GPU (e.g., GTX 1080) for a small number of iterations and record time per iteration and final metrics. Update the README.md with a new 'Benchmarks' section including a table of models, hardware used, and runtime. Reference any relevant files like 'scripts/benchmark.py' or 'configs/*.yml'.