[Documentation] Explain serial and parallel submission

Không có tệp tài liệu hoặc entry point nào được nêu. Hãy bắt đầu bằng cách xác định tài liệu về cách sử dụng executor, sau đó dùng các ví dụ SingleNodeExecutor được cung cấp để giải thích việc gửi tuần tự và song song, thứ tự phụ thuộc, thời gian chờ khác nhau và max_workers; công việc được xem là hoàn tất khi thứ tự gửi được khuyến nghị và hành vi thực thi tương ứng đã rõ ràng.

This can be illustrated with the following examples:

from time import sleep
from executorlib import SingleNodeExecutor

def wait_function(i, j, dependency):
    sleep(i)
    return [i, j]

def get_results(future_lst):
    """sort results based on time of completion"""
    result_lst, tmp_lst = [], []
    while len(future_lst) > 0:
        for f in future_lst:
            if f.done():
                result_lst.append(f.result())
            else:
                tmp_lst.append(f)
        future_lst = tmp_lst 
        tmp_lst = []
    return result_lst

with SingleNodeExecutor(max_workers=2) as exe:
    future_lst = []
    for i in range(5):
        f = None
        for j in range(4):
            f = exe.submit(wait_function, i=i, j=j, dependency=f)
            future_lst.append(f)

    print(get_results(future_lst=future_lst))

>>> [[0, 0], [1, 0], [2, 0], [3, 0], [0, 1], [1, 1], [4, 0], [2, 1], [0, 2], [3, 1], [1, 2], [2, 2], [0, 3], [4, 1], [1, 3], [3, 2], [2, 3], [4, 2], [3, 3], [4, 3]]

The wait time of the tasks depends only on the i parameter. In addition, functions with increasing j parameter depend on each other, so [1, 1] can only be executed once [1, 0] is completed. Given the structure of the loops the functions are submitted as follows:

[
[0, 0], [1, 0], [2, 0], [3, 0], [4, 0],
[0, 1], [1, 1], [2, 1], [3, 1], [4, 1],
[0, 2], [1, 2], [2, 2], [3, 2], [4, 2],
[0, 3], [1, 3], [2, 3], [3, 3], [4, 3],
]

But the execution order differs, starting with [0, 0], [1, 0], [2, 0], [3, 0] followed by [0, 1], [1, 1] and [4, 0]. The reason is the varying wait time. [0, 0], [1, 0], [2, 0], [3, 0], [4, 0] wait 0 seconds while [0, 1], [1, 1], [2, 1], [3, 1], [4, 1] wait 1 second and so on. Finally, only two tasks can be executed at the same time as the number of workers is restricted to max_workers=2.

In analogy:

with SingleNodeExecutor(max_workers=2) as exe:
    future_lst = []
    for j in range(4):
        f = None
        for i in range(5):
            f = exe.submit(wait_function, i=i, j=j, dependency=f)
            future_lst.append(f)

    print(get_results(future_lst=future_lst))

>>> [[0, 0], [0, 1], [0, 2], [0, 3], [1, 0], [1, 1], [1, 2], [1, 3], [2, 1], [2, 0], [2, 2], [2, 3], [3, 0], [3, 1], [3, 2], [3, 3], [4, 0], [4, 1], [4, 2], [4, 3]]

Here the execution is transposed:

[
[0, 0], [0, 1], [0, 2], [0, 3],
[1, 0], [1, 1], [1, 2], [1, 3],
[2, 0], [2, 1], [2, 2], [2, 3],
[3, 0], [3, 1], [3, 2], [3, 3],
[4, 0], [4, 1], [4, 2], [4, 3],
]

As the executed tasks have the same run time the execution order equals the submission order, which is the recommended case.