GitHub Issues as a documentation source in `living-doc-collector-gh`

Open
  • Is something wrong?
#114 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
8/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Active
Tech stack
github, python
Domain
api, documentation

Research direction

This is a multi-part epic, not a single change, so pick the first unchecked child: the core-issues testbed project. Read docs/reference/authoring-formats.md (GitHub issue-body layout) and docs/examples/gh-issues/ in living-doc, then the existing doc_issues/ code in living-doc-collector-gh. Done means that child is merged and its issues parse with no warning.

Written by the indexing model from the issue text.

Description

type:epic
Background

GitHub Issues is planned; in v0.1.0 it is not a documentation source. In living-doc-collector-gh v0.2.0 the
doc-issues mode is kept aside: doc_issues/ is out of the build, and doc-issues: true fails at start. Its
doc_issues/body_parser.py reads an older layout, a table of acceptance criteria. Its HTTP layer has no documented
retry policy or timeouts. The github-token input is declared, but only this mode would read it. utilities already
ships the canonical issue-body parser (authoring.issue_body) and the doc-entities contract, unused, and its
github/rate_limiter.py and github/decorators.py are imported only by the kept-aside mode. living-doc has the
canonical layout (docs/reference/authoring-formats.md § GitHub issue-body layout) and a corpus,
docs/examples/gh-issues/. docs/tutorials/gh-issues-to-markdown.md and docs/guides/github-collection.md mark the
source planned. The testbed has no project documented in issues.

#19 planned this before the shared contracts existed. Its one idea that still holds, a local-directory source so that
CI runs offline, is a subtask below.

Goals
  • doc-issues mines the issues labelled DocumentedUserStory, DocumentedFeature or DocumentedFunctionality. It
    reads them with the canonical parser and writes doc-entities.json through write_artifact, with a required
    project-id and the partial-collection rules. A closed issue whose ## Status says active stays active, with
    tracker_state: closed.
  • A robust HTTP layer: REST rate-limit reset, secondary 403 / 429 with Retry-After, 5xx, GraphQL RATE_LIMITED, a
    timeout on every call, and an immediate failure on GraphQL INSUFFICIENT_SCOPES that names the scope. GitHub
    Enterprise Server and GHE.com get their API URL, and every GitHub URL is built in one module.
  • The issue-body format is settled before real bodies flow: Markdown is turned into the source-code format before it
    is parsed, and list nesting has one meaning.
  • The testbed gets an issues project; living-doc CI runs the gh-issues chain over the corpus and that project,
    with the multi-source negative runs.
  • The tutorial carries a workflow, and a release lists the mode.
Subtasks

In landing order. Each child is drafted when the previous one has merged.

  • living-doc-testbed: a core-issues project. Its entities exist only as issues, each with exactly one
    Documented* label, and its folder holds only a README.md (the issue list) and .feature scenarios. Coverage
    matches core, with one closed issue whose ## Status is active. Every body parses with no warning.
  • living-doc-utilities: an issue body is turned from Markdown into the source-code format before it is
    normalised and parsed (tech-debt issue, a child).
  • living-doc-utilities: decide what list nesting means in an issue body (tech-debt issue, a child; an owner
    decision).
  • living-doc-collector-gh: doc-issues on authoring.issue_body. body_parser.py is deleted, after a
    characterisation test against 2a27a0c, the last commit where the mode ran. utilities' golden entities come
    through unchanged except source_ref. An issue with no parseable id is skipped with MISSING_ENTITY_ID and
    counted. doc_issues/README.md links the canonical layout instead of restating it.
  • living-doc-collector-gh: the HTTP layer (the retries, timeouts, scope failure and URLs above, configured on the
    one PyGithub factory). Its PR states whether utilities' github.rate_limiter and github.decorators are
    still imported, and links the utilities follow-up.
  • living-doc-collector-gh: a local-directory source for doc-issues, one body file per issue with its label and
    title, so a CI run needs no network and no token.
  • living-doc-collector-gh: a configurable label set (tech-debt issue, a child; optional for the first release).
  • living-doc-utilities: the github and html install extras and the two github/ modules, decided once the
    HTTP layer has landed (tech-debt issue, related).
  • living-doc-toolkit: doc-entities input for normalize-docs and coverage-matrix. Shared with the Azure
    DevOps epic: whichever epic gets there first builds it.
  • living-doc: the gh-issues snapshot chain over docs/examples/gh-issues/ and core-issues. Every artifact
    validates with no warning, and the MIXED_DOCUMENTATION_SOURCES and MULTIPLE_DOCUMENTATION_SOURCES negative
    runs fail with their codes.
  • living-doc agentic layer: a skill writes an entity into an issue body (tech-debt issue, a child).
  • living-doc: the tutorial and guide lose "planned" and get a workflow, then a milestone run over core-issues,
    then the release.
Related / References
  • living-doc docs/reference/authoring-formats.md § GitHub issue-body layout, docs/examples/gh-issues/,
    docs/guides/github-collection.md, docs/tutorials/gh-issues-to-markdown.md.
  • living-doc-utilities docs/authoring/parsers.md, docs/contracts/pipeline-rules.md.
  • living-doc-collector-gh doc_issues/README.md, action.yml (github-token).
  • #19 (superseded by this epic).
Dominant language
Python
Stars
0
Forks
0
Avg merge
6h 14m
Merged PRs (30d)
27

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from AbsaOSS/living-doc

All issues in AbsaOSS/living-doc

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.