apache/beam

DoFn.process should raise exception if something other than a List is returned

已關閉

#18,712 建立於 2022年6月3日

 (4 則留言) (0 個反應) (1 位負責人)Java (4,097 個分叉)batch import
P3coregood first issueimprovementpython

倉庫指標

星標
 (7,313 顆星)
PR 合併指標
 (PR 指標待抓取)

描述

The process method of DoFns can either return values or yield values. In the case of returning values, it expects a List of elements to be returned. When returning a single value, it is easy to forget this, and return the value instead.

Correct way:

class SomeDoFn(beam.DoFn)   def process(self, elem):     return ['a']

Incorrect way:

class SomeDoFn(beam.DoFn)   def process(self, elem):     return 'a'

A pipeline with the incorrect DoFn will fail will a cryptic error message without a direct indication that the actual error is due to SomeDoFn returning an element instead of a List containing that element. This issue is very time-consuming to track down.

It would be good if the pipeline could raise an exception or otherwise indicate that the DoFn is incorrectly returning an element instead of a List to make it easier to identify the error.

Imported from Jira BEAM-3530. Original Jira may contain additional context. Reported by: chuanyu.

貢獻者指南