vespa-engine/vespa

Support feeding from Spark

開放

#9,158 建立於 2019年4月23日

 (6 則留言) (0 個反應) (0 位負責人)Java (561 個分叉)batch import
enhancementgood first issue

倉庫指標

星標
 (4,948 顆星)
PR 合併指標
 (PR 指標待抓取)

描述

Today, the Hadoop integration tools for Vespa support Hadoop and Pig for feeding and querying Vespa. The Pig feeder is a thin wrapper around the Vespa HTTP client.

We should support feeding directly from Spark as well, to avoid Spark pipelines having to write to HDFS and run another Pig job for the actual feeding. Similarly to the Pig feeder, this could be implemented as a thin wrapper around the HTTP client.

貢獻者指南