o19s/elasticsearch-learning-to-rank

`_msearch` support when logging features

Open

#305 opened on May 9, 2020

View on GitHub
 (5 comments) (0 reactions) (0 assignees)Java (1,456 stars) (366 forks)batch import
help wanted

Description

Howdy! When logging features it sure would be faster to get to use the msearch api, though what's documented in the examples is just search. When trying to use that, hit:

~/.local/lib/python3.7/site-packages/elasticsearch/connection/base.py in _raise_error(self, status_code, raw_data)
    176
    177         raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
--> 178             status_code, error_message, additional_info
    179         )

RequestError: RequestError(400, 'illegal_argument_exception', 'key [ext] is not supported in the metadata section')

Contributor guide

Tech stack
python
Domain
machine learningsearch
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Understanding of Python elasticsearch clientKnowledge of Elasticsearch's msearch APIFamiliarity with the feature logging codebase
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
45
Research direction
Examine the file `demo/collect features.py` (line 62) to understand how feature logging currently uses the search API. The error occurs because the `ext` key is not supported in multi search metadata. To implement ` msearch`, you need to modify the code to batch multiple queries using the multi search endpoint while ensuring the `ext` object is placed in the query body rather than the metadata. Review Elasticsearch's ` msearch` API documentation and the Python elasticsearch client's `msearch` method.