Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Optional omitted Complex Input with default format is generated errorneously (?)

Abierto
  • ¿Algo no está bien?
#633 1 comentario 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
42/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Estancado
Stack tecnológico
python
Área
api

Línea de trabajo

Empieza reproduciendo la solicitud Execute con hummingbird/processes/wps_ncdump.py y la definición de Weaver en weaver/processes/wps_package.py#L758; después, sigue cómo PyWPS analiza las entradas opcionales omitidas en request.inputs. Done debería distinguir un conjunto de datos omitido de los datos enviados sin depender de la ruta del archivo ni de los campos de formato predeterminados, con cobertura de regresión para esta solicitud.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Description

@cehbrecht @tomkralidis @jachym
I would like to better understand the procedure of handling inputs (how they get generated) for the following specific use case.

Given a process that has the following inputs definition :

[...]
<DataInputs>
  <Input minOccurs="0" maxOccurs="100">
	<ows:Identifier>dataset</ows:Identifier>
	<ows:Title>Dataset</ows:Title>
	<ows:Abstract>Enter a URL pointing to a NetCDF file (optional)</ows:Abstract>
	<ComplexData>
	  <Default>
		<Format>
		  <MimeType>application/x-netcdf</MimeType>
		</Format>
	  </Default>
	  <Supported>
		<Format>
		  <MimeType>application/x-netcdf</MimeType>
		</Format>
	  </Supported>
	</ComplexData>
  </Input>
  <Input minOccurs="0" maxOccurs="100">
	<ows:Identifier>dataset_opendap</ows:Identifier>
	<ows:Title>Remote OpenDAP Data URL</ows:Title>
	<ows:Abstract>Or provide a remote OpenDAP data URL, for example: http://my.opendap/thredds/dodsC/path/to/file.nc</ows:Abstract>
	<ows:Metadata xlink:href="https://www.iana.org/assignments/media-types/media-types.xhtml" xlink:title="application/x-ogc-dods" xlink:type="simple"/>
	<LiteralData>
	  <ows:DataType ows:reference="urn:ogc:def:dataType:OGC:1.1:string">string</ows:DataType>
	  <ows:AnyValue/>
	</LiteralData>
  </Input>
</DataInputs>
[...]

When I submit an execution with only input dataset_opendap provided with some URL string, the _handler(self, request, response) method of the process that ends up being called contains the following request.inputs:

request.inputs = {
  'dataset': [<pywps.inout.inputs.ComplexInput object at 0x7f140d941a10>], 
  'dataset_opendap': deque([<pywps.inout.inputs.LiteralInput object at 0x7f140d956c90>], maxlen=100)
}

My execution XML does not contain dataset, so it gets generated somehow by default following parsing.

<wps100:Execute xmlns:wps100="http://www.opengis.net/wps/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" service="WPS" version="1.0.0" xsi:schemaLocation="http://www.opengis.net/wps/1.0.0 http://schemas.opengis.net/wps/1.0.0/wpsExecute_request.xsd">
	<ows110:Identifier xmlns:ows110="http://www.opengis.net/ows/1.1">ncdump</ows110:Identifier>
	<wps100:DataInputs>
		<wps100:Input>
			<ows110:Identifier xmlns:ows110="http://www.opengis.net/ows/1.1">dataset_opendap</ows110:Identifier>
			<wps100:Data>
				<wps100:LiteralData>http://localhost8001/ows/proxy/thredds/dodsC/birdhouse/testdata/ta_Amon_MRI-CGCM3_decadal1980_r1i1p1_199101-200012.nc</wps100:LiteralData>
			</wps100:Data>
		</wps100:Input>
	</wps100:DataInputs>
	<wps100:ResponseForm>
		<wps100:ResponseDocument storeExecuteResponse="true" status="true" lineage="true">
			<wps100:Output asReference="true">
				<ows110:Identifier xmlns:ows110="http://www.opengis.net/ows/1.1">output</ows110:Identifier>
			</wps100:Output>
		</wps100:ResponseDocument>
	</wps100:ResponseForm>
</wps100:Execute>

I'm trying to understand why the dataset input even gets generated in request.inputs following parsing since it is omitted completely from the request.
This input is causing me problems, because I need to do some post-processing to convert PyWPS inputs into my package definitions.

Is there some way that I need to employ to detect omitted inputs to discard them explicitly vs real inputs with submitted data?
Is there some flag that I would guarantee me that this input is only the default definition and does not contain any actual data?

I cannot rely on data field to detect omitted inputs because it gets filled by the "default format" application/x-netcdf, which could be submitted real data contents:

{"mimeType": "application/x-netcdf", "encoding": null, "schema": null, "maximumMegabytes": null, "default": true}

The only (very hackish/unreliable) field I could use to detect inputs to drop is file which contains a reference to {workdir}/input instead of {workdir}/input_{uuid}.
Any better guidance would be greatly appreciated.

Expand this to see full details contents of requests.inputs["dataset"] 

{ComplexInput}  
    _data_format = {Format} 
        _encoding = {NoneType} None
        _extension = {NoneType} None
        _mime_type = {str} 'application/x-netcdf'
        _schema = {NoneType} None
        encoding = {str} ''
        extension = {str} ''
        json = {dict} {'mime_type': 'application/x-netcdf', 'encoding': '', 'schema': '', 'extension': ''}
        mime_type = {str} 'application/x-netcdf'
        schema = {str} ''
    _default = {dict} {'mimeType': 'application/x-netcdf', 'encoding': None, 'schema': None, 'maximumMegabytes': None, 'default': True}
    _default_type = {int} 3
    _iohandler = {DataHandler} 
        _data = {dict}  
        _file = {str} '/tmp/weaver-hybrid/pywps_process_pw40isee/input'
        _ref = {weakref} 
        _stream = {NoneType} None
        base64 = {str} 'Traceback (most recent call last):\n  File "/opt/pycharm-pro/plugins/python/helpers/pydev/_pydevd_bundle/pydevd_resolver.py", line 178, in _getPyDictionary\n    attr = getattr(var, n)\n  File "/home/francis/dev/miniconda/envs/weaver-py3/lib/python3.7/site-pa
        data = {dict} {'mimeType': 'application/x-netcdf', 'encoding': None, 'schema': None, 'maximumMegabytes': None, 'default': True}
        file = {str} '/tmp/weaver-hybrid/pywps_process_pw40isee/input'
        mem = {NoneType} None
        post_data = {str} 'Traceback (most recent call last):\n  File "/opt/pycharm-pro/plugins/python/helpers/pydev/_pydevd_bundle/pydevd_resolver.py", line 178, in _getPyDictionary\n    attr = getattr(var, n)\n  File "/home/francis/dev/miniconda/envs/weaver-py3/lib/python3.7/site-pa
        prop = {str} 'data'
        size = {int} 0
        stream = {StringIO} <_io.StringIO object at 0x7f140d8aa690>
        url = {str} 'file:///tmp/weaver-hybrid/pywps_process_pw40isee/input'
    _supported_formats = {tuple} 
    _workdir = {str} '/tmp/weaver-hybrid/pywps_process_pw40isee'
    abstract = {str} 'Enter a URL pointing to a NetCDF file (optional)'
    as_reference = {bool} False
    base64 = {str} 'Traceback (most recent call last):\n  File "/opt/pycharm-pro/plugins/python/helpers/pydev/_pydevd_bundle/pydevd_resolver.py", line 178, in _getPyDictionary\n    attr = getattr(var, n)\n  File "/home/francis/dev/miniconda/envs/weaver-py3/lib/python3.7/site-pa
    data = {dict} {'mimeType': 'application/x-netcdf', 'encoding': None, 'schema': None, 'maximumMegabytes': None, 'default': True}
    data_format = {Format} 
    data_set = {bool} True
    extension = {str} ''
    file = {str} '/tmp/weaver-hybrid/pywps_process_pw40isee/input'
    identifier = {str} 'dataset'
    inpt = {dict} {}
    json = {dict} {'identifier': 'dataset', 'title': 'Dataset', 'abstract': 'Enter a URL pointing to a NetCDF file (optional)', 'keywords': [], 'metadata': [], 'type': 'complex', 'data_format': {'mime_type': 'application/x-netcdf', 'encoding': '', 'schema': '', 'extension': ''}, 'asreference': False, 'supported_formats': [{'mime_type': 'application/x-netcdf', 'encoding': '', 'schema': '', 'extension': ''}], 'workdir': '/tmp/weaver-hybrid/pywps_process_pw40isee', 'mode': 0, 'min_occurs': 0, 'max_occurs': 100, 'translations': None, 'data': "", 'mimetype': 'application/x-netcdf'}
    keywords = {list} []
    max_occurs = {int} 100
    metadata = {list} []
    method = {str} ''
    min_occurs = {int} 0
    post_data = {str} 'Traceback (most recent call last):\n  File "/opt/pycharm-pro/plugins/python/helpers/pydev/_pydevd_bundle/pydevd_resolver.py", line 178, in _getPyDictionary\n    attr = getattr(var, n)\n  File "/home/francis/dev/miniconda/envs/weaver-py3/lib/python3.7/site-pa
    prop = {str} 'data'
    size = {int} 0
    source_type = {int} 3
    stream = {StringIO}  
    supported_formats = {tuple} 
    title = {str} 'Dataset'
    translations = {NoneType} None
    url = {str} 'file:///tmp/weaver-hybrid/pywps_process_pw40isee/input'
    uuid = {NoneType} None
    valid_mode = {int} 0
    workdir = {str} '/tmp/weaver-hybrid/pywps_process_pw40isee'

Environment

  • operating system:
  • Python version: 3.7
  • PyWPS version: tried with both with 4.4.5 and 4.5.0, same result
  • source/distribution
  • git clone
  • Debian
  • PyPI
  • zip/tar.gz
  • other (please specify):
  • web server
  • Apache/mod_wsgi
  • CGI
  • other (please specify): gunicorn with Pyramid WebApp that nests PyWPS's service

Steps to Reproduce

Using this process:
https://github.com/bird-house/hummingbird/blob/master/hummingbird/processes/wps_ncdump.py

It is executed indirectly by Weaver using this definition:
https://github.com/crim-ca/weaver/blob/4.1.0/weaver/processes/wps_package.py#L758

Lenguaje dominante
Python
Estrellas
186
Forks
116
Merge medio
11 d 20 h
PR fusionados (30 d)
1

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de geopython/pywps

Todos los issues de geopython/pywps

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.