rclone/rclone

Add support for GOIndex/GDIndex?

オープン

#5,225 opened on 2021/04/09

 (6 件のコメント) (1 件のリアクション) (0 人の担当者)Go (5,090 件のフォーク)batch import
Remote: HTTPenhancementhelp wanted

Repository metrics

Stars
 (57,185 個のスター)
PR merge metrics
 (平均マージ 32d 20h) (30d で 23 merged PRs)

説明

The associated forum post URL from https://forum.rclone.org

N/A, but I think the rclone docs told me to directly make an issue for such a thing on the http backend (sorry if I should have made a post on the forums first):

This has been tested with common webservers such as Apache/Nginx/Caddy and will likely work with file listings from most web servers. (If it doesn't then please file an issue, or send a pull request!)

What is your current rclone version (output from rclone version)?

rclone v1.55.0
- os/type: linux
- os/arch: amd64
- go/version: go1.16.2
- go/linking: static
- go/tags: cmount

What problem are you are trying to solve?

rclone does not support GOIndex/GDIndex, but it can!

Similar to #3913, however it is not entirely JS based!

JS is only used to hide google app credentials from the public, all files/folders are fetchable via http request (POST to get a list of files, you get a JSON response listing the mime-type of file (/something custom if it's a folder / google doc), the google ID (irrelevant), the name of the file, and the modTime, GET with url params to either get a file / get a page that can view the file in browser / get a page that lists the relevant folder in-browser by sending a post request to the same URL and then parsing the response.

For context, a JS listing response is as follows:

{
    "files": [
        {
            "id":"[redacted]",
            "name":"Music",
            "mimeType":"application/vnd.google-apps.folder",
            "modifiedTime":"2021-03-09T13:03:52.009Z"
        },
        {
            "id":"[redacted]",
            "name":"Video",
            "mimeType":"application/vnd.google-apps.folder",
            "modifiedTime":"2021-03-09T13:03:59.865Z"
        },
        {
            "id":"[redacted]",
            "name":"0x.goindex.gif",
            "mimeType":"image/gif",
            "modifiedTime":"2021-03-09T14:01:58.788Z",
            "size":"458131"
        },
        ...
    ]
}

No idea if I had to redact that stuff or not, but just in case.

People use this + some server / serverless setup (usually cheap/free cloudflare workers) to get around Google Drive sharing limits (because this way it's technically not necessarily sharing, it's accessing Google Drive in some way and then piping the file request from a custom server to the user being shared with). People also use this because no email sharing necessary, multiple drive support in some cases, easier / better preview support, custom statistics, etc.

How do you think rclone should be changed to solve that?

I'd love to make a pull request, but I wanted to make the issue first because

I wanted to make sure such a PR would be welcome, and if yes:

  • warn that it would be relatively large
    • and I don't know the policy on commit size
  • warn that it would take till at least the weekend, as I'm not that familiar with Go and I'd have to set up decent tests
    • and as a result should I start a PR and add to it, requesting a review when finished, or just work and then eventually submit a PR
  • didn't know if this should be in the http backend or some other, custom backend (because there are some custom subtleties regarding user-defined initial http parameters).
    • If the latter, if GOIndex and GDIndex should be separated, since while they are mostly the same they have some minute differences in terms of parameters to be sent (GDIndex is a complete rewrite by someone else, don't know why this was done though)
    • How to handle the case of multiple drives / root folders, because that's a thing with these
    • Some of these (depending on the user) even allow upload support
  • don't know if I'd just have to add a backend, or if I would possibly have to touch code elsewhere (either's fine, I just can't figure out which)
  • If I should add special error cases for Cloudflare workers or not (since this is the common use case)
    • if there should be custom ratelimit parameters for cloudflare workers because of such limits, or if this would just be handled via something like --poll-interval
  • How to handle tests for such a thing
  • any other suggestions / recommendations

コントリビューターガイド