salesforce/TransmogrifAI

Factor out all common spark/hadoop properties

開放

#87 建立於 2018年8月24日

 (8 則留言) (0 個反應) (0 位負責人)Scala (399 個分叉)batch import
good first issue

倉庫指標

星標
 (2,274 顆星)
PR 合併指標
 (30 天內沒有已合併 PR)

描述

Problem We currently largely overlapping spark.gradle files especially in terms of spark properties.

$ git ls-files | grep spark.gradle
gradle/spark.gradle
helloworld/gradle/spark.gradle
templates/simple/spark.gradle

Solution Provide a way to have a single spark.gradle or at least a single spark-transmogrifai.conf file with common properties that is passed via --properties-file to Spark.

Alternatives

  • common properties file
  • refactored spark.gradle

Additional context DRY

貢獻者指南