good first issuehelp wantedingestion
倉庫指標
- Star
- (4,937 star)
- PR 合併指標
- (平均合併 6天 7小時) (30 天內合併 186 個 PR)
描述
- search for ingestion file is hard-coded to recursive, no option to do non-recursive scan
- glob setting is matching absolute path, which should obviously exclude part from the root directory
- ingestion config is sometimes required to be escape but sometimes it is not - depending on the plugin it is using
- ingestion job returns empty list of segments being pushed when segment generation fails - it should error out if # of files is different from # of segment generated.
- some of the 2-stage segment generation job doesn't populate exception/throwable back to the main thread and thus executions like SEGMENT_CREATE --> SEGMENT_UPLOAD will end up in a success state with an empty TAR file upload result;
TODO:
- provide recursive flag in commandline utils
- remove glob matcher prefix for inputFileDir
- unifies plugin config parsing
- pipe-back segment job runner / task runner exception back to main thread.