Why managing previous and predecessors per separate?
维护者通常 2 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 20/100
- Issue 类型
- 重构
- 描述清晰度
- 需要澄清
- 活跃度
- 停滞
- 技术栈
- python
调研方向
从 python/old_code/pywayang/src/pywy/orchestrator/operator.py 开始,重点查看 Operator 构造函数以及 getIterator、set_successor 和 set_predecessor 方法。在提出关系模型之前,追踪 previous、predecessor 和 successor 在旧版 PyWayang 代码的其他位置是如何使用的;完成这项工作需要一个达成一致的设计以及相应的 issue 范围。
由索引模型根据 Issue 内容生成。
描述
Why managing previous and predecessors per separate?
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import pickle
import cloudpickle
from pywy.config.config_reader import get_source_types
from pywy.config.config_reader import get_sink_types
from pywy.config.config_reader import get_boundary_types
import logging
pickle_protocol = pickle.HIGHEST_PROTOCOL
# Describes an Operation over an intermediate result
# Each operation could be processed by Python or Java platforms
class Operator:
def __init__(
self, operator_type=None, udf=None, previous=None,
iterator=None, python_exec=False
):
# Operator ID
self.id = id(self)
# Operator Type
self.operator_type = operator_type
# Set Boundaries
if self.operator_type in get_boundary_types():
self.boundary = True
else:
self.boundary = False
# UDF Function
self.udf = udf
# Source types must come with an Iterator
self.iterator = iterator
if operator_type in get_source_types():
if iterator is None:
print("Source Operator Type without an Iterator")
raise
else:
self.source = True
else:
self.source = False
# Sink Operators
if operator_type in get_sink_types():
self.sink = True
else:
self.sink = False
# TODO Why managing previous and predecessors per separate?
self.previous = previous
self.successor = []
self.predecessor = []
self.parameters = {}
# Set predecessors and successors from previous
if self.previous:
for prev in self.previous:
if prev is not None:
prev.set_successor(self)
self.set_predecessor(prev)
self.python_exec = python_exec
logging.info("Operator:" + str(self.getID()) + ", type:" + self.operator_type + ", PythonExecutable: " +
str(self.python_exec) +
", is boundary: " + str(self.is_boundary()) + ", is source: " +
str(self.source) + ", is sink: " + str(self.sink))
def getID(self):
return self.id
def is_source(self):
return self.source
def is_sink(self):
return self.sink
def is_boundary(self):
return self.boundary
def serialize_udf(self):
self.udf = cloudpickle.dumps(self.udf)
def getIterator(self):
if self.is_source():
return self.iterator
# TODO this should iterate through previous REDESIGN
return self.udf(self.previous[0].getIterator())
def set_parameter(self, key, value):
self.parameters[key] = value
def set_successor(self, suc):
if (not self.is_sink()) and self.successor.count(suc) == 0:
self.successor.append(suc)
def set_predecessor(self, suc):
if self.predecessor.count(suc) == 0:
self.predecessor.append(suc)
ea17eb1209cadc4bb2dbe95f752590de9723a51d
- 主要语言
- Java
- 星标
- 284
- 派生
- 147
- 平均合并
- 2 天 13 小时
- 30 天内合并 PR
- 13
环境准备
- 提供 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
apache/wayang 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 82/100
维护者通常 2 天内回复
-
JavaReservoirSampleOperator returns the whole input when the dataset size is unknown可能已有人在做 @sasmith-14 于 7 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 72/100
维护者通常 2 天内回复
-
Record.compareTo() throws ClassCastException when comparing records可能已有人在做 @harshitsingh070 于 70 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 82/100
维护者通常 2 天内回复
-
Unused parameter in function ProjectionDescriptor.createPojoJavaImplementation()可能已有人在做 @AliRana30 于 77 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 76/100
维护者通常 2 天内回复
-
DISCLAIMER and some "(incubating)" references are still around after graduation可能已有人在做 @sasmith-14 于 1 天前认领。 未关闭
难度 3/5 1-2 天 新手友好度 66/100
apache/wayang#895 · 1 条评论 · 已指派 1 人 ·
维护者通常 2 天内回复
相似的 Issue
-
good first issue
难度 2/5 1-3 小时 新手友好度 78/100
seleniumboot/selenium-boot#275 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 68/100
jenkins-infra/plugin-modernizer-tool#2070 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 1/5 1 小时以内 新手友好度 84/100
orientechnologies/orientdb#11020 ·
维护者通常 1 天内回复
-
Event search answers 500 when calendars, attendees or organizers holds a null element可能重新可做 关联的 PR 已关闭且未合并。 未关闭ai-driven-qa bug Minor
难度 2/5 1-3 小时 新手友好度 70/100
linagora/twake-calendar-side-service#1615 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 62/100
eclipse-tradista/tradista#371 ·
维护者通常 1 天内回复