clab/dynet

Implement binary io for faster/lighter serialization

Open

#648 opened on Jun 30, 2017

View on GitHub
 (8 comments) (0 reactions) (0 assignees)C++ (701 forks)github user discovery
enhancementgood first issue

Repository metrics

Stars
 (3,433 stars)
PR merge metrics
 (PR metrics pending)

Description

Saving a medium sized seq2seq model takes between 2x and 4x longer as before, is anyone else experiencing something similar?

EDIT: some more information:

  • the model weighs 340MB once saved (which is less than before 2.0)
  • There are 3 subcollections

Contributor guide

Research direction
Examine the current serialization code in DyNet, identify performance bottlenecks, and propose a binary serialization format. Consider using existing libraries like Protocol Buffers or custom binary encoding.
Tech stack
cpp
Domain
machine learning
Issue type
Performance
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Needs investigation
Prerequisites
C++Git
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
45