bfcneedsupdate() compares 'expires' by date, so short max-age resources are stale immediately after download

Ouverte
#75 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

Évaluation

Difficulté
3/5
Temps estimé
1-2 jours
Accessibilité débutants
67/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
Active
Stack technique
r
Domaine
backend

Piste de recherche

Start in R/BiocFileCache-class.R at the bfcneedsupdate() expiry check near line 1024, then inspect its tests for expiry and ETag/Last-Modified behavior. Reproduce with a resource whose max-age is shorter than the time until midnight; the fix is done when it is not marked stale immediately after download and the existing freshness checks still behave correctly.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Problem

bfcneedsupdate() checks expiry by calendar date:

https://github.com/Bioconductor/BiocFileCache/blob/devel/R/BiocFileCache-class.R#L1024

expired <- as.Date(file_expires, optional=TRUE) <= Sys.Date()

expires is set to response time + max-age (since #55). When max-age is shorter than the time left until midnight, the expiry date is today. The check is then TRUE immediately, and because expired short-circuits the ETag/Last-Modified comparison, the resource reports as stale right after download.

bioconductor.org now serves files through Cloudflare with short max-age values, so this affects most Bioconductor-hosted resources:

URL cache-control
packages/stats/bioc/bioc_pkg_stats.tab max-age=300, s-maxage=3600
checkResults/devel/bioc-LATEST/BUILD_STATUS_DB.txt max-age=300, s-maxage=31536000
packages/3.23/bioc/VIEWS max-age=30, s-maxage=31536000

Reprex

BiocFileCache 3.2.0 (release); the same line is in devel 3.3.3.

library(BiocFileCache)
url <- "https://bioconductor.org/packages/stats/bioc/bioc_pkg_stats.tab"
bfc <- BiocFileCache(tempfile(), ask = FALSE)
bfcadd(bfc, rname = url, fpath = url, rtype = "web", action = "copy", download = TRUE)
rid <- bfcquery(bfc, url, "rname", exact = TRUE)$rid
bfcinfo(bfc, rid)$expires   # a few minutes from now, i.e. today
bfcneedsupdate(bfc, rid)    # TRUE, seconds after the download

Impact

Any package that does if (bfcneedsupdate(...)) bfcdownload(...) on these URLs re-downloads them on every call, so the cache does nothing. BiocPkgTools hit this as a test failure on r-universe (seandavi/BiocPkgTools#92) and worked around it in seandavi/BiocPkgTools#93 by doing its own size/ETag/Last-Modified check.

Suggested fix

Compare full timestamps instead of dates, e.g.

expired <- as.POSIXct(file_expires, tz = "GMT", optional = TRUE) <= Sys.time()

It would also be worth considering whether an expired entry should fall through to the ETag / Last-Modified check rather than returning TRUE outright. With max-age=30, a cache that has merely passed max-age is usually still current, and a matching ETag is stronger evidence than expiry.

Langage dominant
R
Étoiles
14
Forks
9
Métriques de merge des PR
Aucune PR mergée en 30 j

Préparer son environnement

Ce projet ne fournit ni conteneur de développement, ni Dockerfile, ni guide de contribution : l'installation est à votre charge. Commencez par son README, et consultez notre guide de la première contribution pour les étapes générales.

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de Bioconductor/BiocFileCache

Toutes les issues de Bioconductor/BiocFileCache

Issues similaires

Plus d'issues R

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.