Is GitHub raw storage reliable?

Open
#213 16 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
20/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
github, node.js
Domain
cloud

Research direction

Start with the linked Node app and its test that writes misc/now.txt, then compare the non-raw and raw GitHub URLs described in the report. Determine whether the observed delay is reproducible and whether this repository has a specific change or documentation update that could address it; completion is not defined by the issue.

Written by the indexing model from the issue text.

Description

I've been building apps on the assumption that if you store something on GitHub, when you get the raw version of that thing, it'll be current. I was getting inconsistent results, so I decided to test that assumption, and it seems that it is not reliable storage.

If you read an object immediately after writing it, you will not always get back what you wrote. It's not just a matter of waiting a second or two, sometimes it's wrong for up to 10 seconds (the delay I programmed into the test).

Here's the Node app I'm running to test this. It wakes up every minute and saves a file to GitHub with the current local time string. If I reload the non-raw version of the page, it usually seems to have the current value. But if I reload the raw version of the page, it often does not have the correct value.

If you see a problem in this app, please let me know. I had big plans for using GitHub for user storage. It'll make a huge difference in interop, common ground in the fast growing Tools for Thought space.

Here are the results of the app for the first 30 minutes.

everyMinute: 1:58:13 PM, status == 200, bad
everyMinute: 1:59:00 PM, status == 200, bad
everyMinute: 2:00:00 PM, status == 200, bad
everyMinute: 2:01:00 PM, status == 200, good
everyMinute: 2:02:00 PM, status == 200, bad
everyMinute: 2:03:00 PM, status == 200, bad
everyMinute: 2:04:00 PM, status == 200, bad
everyMinute: 2:05:00 PM, status == 200, bad
everyMinute: 2:06:00 PM, status == 200, bad
everyMinute: 2:07:00 PM, status == 200, good
everyMinute: 2:08:00 PM, status == 200, bad
everyMinute: 2:09:00 PM, status == 200, bad
everyMinute: 2:10:00 PM, status == 200, bad
everyMinute: 2:11:00 PM, status == 200, bad
everyMinute: 2:12:00 PM, status == 200, bad
everyMinute: 2:13:00 PM, status == 200, good
everyMinute: 2:14:00 PM, status == 200, bad
everyMinute: 2:15:00 PM, status == 200, bad
everyMinute: 2:16:00 PM, status == 200, bad
everyMinute: 2:17:00 PM, status == 200, bad
everyMinute: 2:18:00 PM, status == 200, bad
everyMinute: 2:19:00 PM, status == 200, good
everyMinute: 2:20:00 PM, status == 200, bad
everyMinute: 2:21:00 PM, status == 200, bad
everyMinute: 2:22:00 PM, status == 200, bad
everyMinute: 2:23:00 PM, status == 200, bad
everyMinute: 2:24:00 PM, status == 200, bad
everyMinute: 2:25:00 PM, status == 200, good
everyMinute: 2:26:00 PM, status == 200, bad
everyMinute: 2:27:00 PM, status == 200, bad
everyMinute: 2:28:00 PM, status == 200, bad

You can see that more often than not the raw value is incorrect after ten seconds.

I'm leaving the app running for a while, so if you want to verify this, you can.

I'll post a note in the GitHub thread when I turn the app off.

Dominant language
HTML
Stars
134
Forks
10
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from scripting/Scripting-News

All issues in scripting/Scripting-News

Similar issues

More Cloud issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.