salesforce/TransmogrifAI

Factor out all common spark/hadoop properties

开放

#87 创建于 2018年8月24日

 (8 条评论) (0 个反应) (0 位负责人)Scala (399 个派生)batch import
good first issue

仓库指标

星标
 (2,274 个星标)
PR 合并指标
 (30 天内没有已合并 PR)

描述

Problem We currently largely overlapping spark.gradle files especially in terms of spark properties.

$ git ls-files | grep spark.gradle
gradle/spark.gradle
helloworld/gradle/spark.gradle
templates/simple/spark.gradle

Solution Provide a way to have a single spark.gradle or at least a single spark-transmogrifai.conf file with common properties that is passed via --properties-file to Spark.

Alternatives

  • common properties file
  • refactored spark.gradle

Additional context DRY

贡献者指南