wala/ML

Adding duplicate tensor data sources during dataset enumeration

Aberta

#141 aberto em 2 de fev. de 2024

 (1 comentário) (0 reação) (0 responsável)Java (17 forks)auto 404
good first issuequestion

Métricas do repositório

Stars
 (27 estrelas)
Métricas de merge de PR
 (Métricas PR pendentes)

Description

I am seeing in the logs multiple SSA variables corresponding to a single Python variable being added as tensor dataset sources. This might be okay; dataset sources aren't really the same as tensor sources not stemming from datasets. In other words, the dataset holds the tensors, whereas in the non-dataset case, the tensors are generated from some API (e.g., tf.ones()).

But, I'm unsure. It makes the test values look weird (where are there so many tensor variables in a function?). It may also cause confused when we start tracking shape/d-types for TF2 APIs. I don't think it would hurt just to have the final SSA variable be marked as a tensor dataset source (though one could question how we are representing dataset sources, particularly when it comes tracking shapes; should they be separate?).

One could also ask why, in the SSA, are multiple variables representing the same Python variable.

Example

This is what I am seeing:

# Test enumerate. The first element of the tuple returned isn't a tensor.

import tensorflow as tf


def f(a):
    pass


def g(a):
    pass


dataset = tf.data.Dataset.from_tensor_slices([1, 2, 3])

for step, element in enumerate(dataset, 1):
    f(step)
    g(element)

That's the input. In the logs, there are two SSA variables representing element, namely, v278 and v282:

106   v265 = invokeFunction < PythonLoader, LCodeBody, do()LRoot; > v3,v249,v258:#1 @106 exception:v266tf2_test_dataset11.py [16:21] -> [16:42] [265=[temp 3]3=[enumerate]249=[dataset]]
BB3
107   v269 = new <PythonLoader,Ltuple>@107   tf2_test_dataset11.py [3:0] -> [18:14]
108   v271 = global:global step              tf2_test_dataset11.py [16:4] -> [16:8]
109   fieldref v269.v257:#0 = v271 = v271    tf2_test_dataset11.py [3:0] -> [18:14]
110   v273 = global:global element           tf2_test_dataset11.py [16:10] -> [16:17]
111   fieldref v269.v259:#1 = v273 = v273    tf2_test_dataset11.py [3:0] -> [18:14]
112   v274 = a property name of v265         <no information> [265=[temp 3]]
113   v276 = fieldref v274.v257:#0           tf2_test_dataset11.py [16:4] -> [16:8] [276=[step]]
115   v278 = fieldref v274.v259:#1           tf2_test_dataset11.py [16:10] -> [16:17] [278=[element]]
117   v267 = binaryop(ne) v268:#null , v274  tf2_test_dataset11.py [3:0] -> [18:14]
118   conditional branch(eq, to iindex=-1) v267,v257:#0tf2_test_dataset11.py [3:0] -> [18:14]
BB4
119   v280 = new <PythonLoader,Ltuple>@119   tf2_test_dataset11.py [3:0] -> [18:14]
120   fieldref v280.v257:#0 = v276 = v276    tf2_test_dataset11.py [3:0] -> [18:14] [276=[step]]
121   fieldref v280.v259:#1 = v278 = v278    tf2_test_dataset11.py [3:0] -> [18:14] [278=[element]]
122   v279 = fieldref v265.v280              tf2_test_dataset11.py [3:0] -> [18:14] [265=[temp 3]]
123   v281 = fieldref v279.v257:#0           tf2_test_dataset11.py [16:4] -> [16:8] [281=[step]]
125   v282 = fieldref v279.v259:#1           tf2_test_dataset11.py [16:10] -> [16:17] [282=[element]]

Producing the following corresponding logs:

[INFO] Added dataflow source from tensor dataset: [Node: <Code body of function Lscript tf2_test_dataset11.py> Context: CallStringContext: [ com.ibm.wala.FakeRootClass.fakeRootMethod()V@2 ], v278]:[Empty].
[INFO] Added dataflow source from tensor dataset: [Node: <Code body of function Lscript tf2_test_dataset11.py> Context: CallStringContext: [ com.ibm.wala.FakeRootClass.fakeRootMethod()V@2 ], v282]:[Empty].

We really only need the second one, I think, unless there is some particular reason there are multiple SSA variables representing the same Python variable. But, since v278 is never referenced again, so I would say not.

Guia do colaborador