Draft: Chunk up large orders in planet orders request

Aperta
  • Qualcosa non va?
#945 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
38/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Ferma
Stack tecnologico
python
Ambito
api, cli

Direzione di ricerca

Start with the planet data search, planet orders request, and planet orders create entry points described in the issue, then review the demonstrated ml.py chunking flow. Done means large order input is split into chunks and planet orders create accepts the resulting multiline output, with the command-line behavior and UI worked out.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

proposal

Chunk up large orders in planet orders request directly from a search.

Note: planet orders create would also have to be able to take in multiline input, as planet orders request would now output one line for each order chunk. How to do it is already demonstrated, we just have to work out the UI.

The script below is a slight alteration of the example script in #944. It demonstrates how to do a version of this:

$> planet data search --limit 500 PSScene | python ml.py --items - --chunksize=300
500
chunk: 300
chunk: 200

ml.py

import json

import click


class MultiJSON(click.ParamType):
    name = 'MultiJSON'

    def convert(self, value, param, ctx) -> list:
        if isinstance(value, list):
            conv = value
        else:
            # read from raw json
            # skip this if value is a Path object or something else
            if isinstance(value, str) and (value.startswith('{')
                                           or value.startswith('[')):
                print('hereh')
                try:
                    conv = json.loads(value)
                    if isinstance(conv, dict):
                        conv = [conv]

                except json.decoder.JSONDecodeError:
                    self.fail(
                        'JSON string is not valid JSON. Make sure the entire '
                        'JSON string is single-quoted and string entries are '
                        'double-quoted.')

            # read from stdin or file
            else:
                try:
                    with click.open_file(value) as f:
                        conv = [json.loads(line) for line in f if line]
                except FileNotFoundError:
                    self.fail('File not found.')
                except json.decoder.JSONDecodeError:
                    stream = 'stdin' if value == '-' else 'JSON file'
                    self.fail(f'{stream} does not contain valid JSON.')

        if not len(conv):
            self.fail('JSON cannot be empty.')

        return conv


@click.command()
@click.option('--items', type=MultiJSON(), required=True)
@click.option('--chunksize', type=int, default=5)
def hello(items, chunksize):
    click.echo(len(items))

    # https://stackoverflow.com/a/312464
    def chunks(lst, n):
        """Yield successive n-sized chunks from lst."""
        for i in range(0, len(lst), n):
            yield lst[i:i + n]
    
    for chunk in chunks(items, chunksize):
        click.echo(f'chunk: {len(chunk)}')


if __name__ == '__main__':
    hello()
Lingua principale
Python
Stelle
300
Fork
100
Merge medio
6g 18h
PR unite (30g)
1

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di planetlabs/planet-client-python

Tutte le issue di planetlabs/planet-client-python

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.