getzola/zola

Architecture for less iterations over the library

Open

#1112 opened on Jul 31, 2020

View on GitHub
 (12 comments) (0 reactions) (0 assignees)Rust (15,349 stars) (1,046 forks)batch import
enhancementhelp wanted

Description

This is not exactly a bug report, but when i was investigating site::load() i realized the pages/sections collections get iterated over many times. For a simple build, it iterates over:

  • sections (loop l. 174)
  • pages (loop l. 182)
  • sections & pages (library.check_for_path_collisions() l.197)
  • pages (library.populate_taxonomies() l. 205)
  • sections and pages (self.populate_sections() l. 207, 5 times for sections, 2 for pages because of other calls inside this function)
  • sections and pages (self.render_markdown l. 208, 2 times for pages)
  • sections and pages (check_internal_links_with_anchors() l. 212)
  • sections and pages (check_external_links() l.215, only in check mode)

Without check mode, that's a total of 9 iterations over sections and 8 iterations over pages. This is only for iterations after the library is built. There may be some compiler magic behind the scenes to "reunite" some iterations in there, but my intuition is there's a lot of potential performance improvements to be investigated here.

I think some of the useless iterations are due to the absence of library structure due to the glob pattern (reuniting children pages with related sections, and building content hierarchy). Maybe a recursive descent into the content folder could help here?

Contributor guide

Tech stack
rust
Domain
performancetooling
Issue type
performance
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
RustZola codebase knowledge
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
35
Research direction
Investigate the iterations listed in the issue, particularly in the library component. Files involved are likely in the library module, with sections and pages collections. Consider recursing into the content folder to reduce multiple passes. Look at comments on the issue for any prior discussion on architecture changes. Profiling the current iterations would help identify bottlenecks.