Draft: Chunk up large orders in planet orders request

Aberta
  • Algo errado?
#945 0 comentários 0 reações 0 responsáveis Ver no GitHub

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
38/100
Tipo de issue
Funcionalidade
Clareza
Razoavelmente clara
Status de atividade
Estagnada
Stack de tecnologia
python
Domínio
api, cli

Direção de pesquisa

Start with the planet data search, planet orders request, and planet orders create entry points described in the issue, then review the demonstrated ml.py chunking flow. Done means large order input is split into chunks and planet orders create accepts the resulting multiline output, with the command-line behavior and UI worked out.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

proposal

Chunk up large orders in planet orders request directly from a search.

Note: planet orders create would also have to be able to take in multiline input, as planet orders request would now output one line for each order chunk. How to do it is already demonstrated, we just have to work out the UI.

The script below is a slight alteration of the example script in #944. It demonstrates how to do a version of this:

$> planet data search --limit 500 PSScene | python ml.py --items - --chunksize=300
500
chunk: 300
chunk: 200

ml.py

import json

import click


class MultiJSON(click.ParamType):
    name = 'MultiJSON'

    def convert(self, value, param, ctx) -> list:
        if isinstance(value, list):
            conv = value
        else:
            # read from raw json
            # skip this if value is a Path object or something else
            if isinstance(value, str) and (value.startswith('{')
                                           or value.startswith('[')):
                print('hereh')
                try:
                    conv = json.loads(value)
                    if isinstance(conv, dict):
                        conv = [conv]

                except json.decoder.JSONDecodeError:
                    self.fail(
                        'JSON string is not valid JSON. Make sure the entire '
                        'JSON string is single-quoted and string entries are '
                        'double-quoted.')

            # read from stdin or file
            else:
                try:
                    with click.open_file(value) as f:
                        conv = [json.loads(line) for line in f if line]
                except FileNotFoundError:
                    self.fail('File not found.')
                except json.decoder.JSONDecodeError:
                    stream = 'stdin' if value == '-' else 'JSON file'
                    self.fail(f'{stream} does not contain valid JSON.')

        if not len(conv):
            self.fail('JSON cannot be empty.')

        return conv


@click.command()
@click.option('--items', type=MultiJSON(), required=True)
@click.option('--chunksize', type=int, default=5)
def hello(items, chunksize):
    click.echo(len(items))

    # https://stackoverflow.com/a/312464
    def chunks(lst, n):
        """Yield successive n-sized chunks from lst."""
        for i in range(0, len(lst), n):
            yield lst[i:i + n]
    
    for chunk in chunks(items, chunksize):
        click.echo(f'chunk: {len(chunk)}')


if __name__ == '__main__':
    hello()
Linguagem predominante
Python
Estrelas
300
Forks
100
Merge médio
6d 18h
PRs com merge (30d)
1

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de planetlabs/planet-client-python

Todas as issues de planetlabs/planet-client-python

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.