Extreme slow down when using Map#lock/unlock from multiple threads [API-1968]

未关闭
#616 3 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
38/100
Issue 类型
缺陷
描述清晰度
基本清楚
活跃度
停滞
技术栈
python

调研方向

首先,使用一个 Hazelcast 客户端、多个线程以及 Map#lock/unlock 运行提供的 Python 复现程序,然后检查阻塞式 map lock 以及 lock、get、put 和 unlock 操作所使用的客户端路径。完成标准是确定并发执行为何比顺序执行慢得多,并验证报告中的工作负载不再出现该性能下降。

由索引模型根据 Issue 内容生成。

描述

Python 3 to-jira Type: Defect

Running the Pessimistic Locking example from multiple threads using the same client takes vastly different amounts of time.

The sequential runs of the same code can give:

It took 68.364421721 second(s) to finish.
It took 2666.208992902 second(s) to finish.

Using one Hazelcast instance. The behaviour is reproducible on MacOS and Linux. Using python3

The code sample:

import hazelcast
from time import perf_counter
from concurrent.futures import ThreadPoolExecutor
from threading import Thread

if __name__ == "__main__":
    client = hazelcast.client.HazelcastClient(
    cluster_name="dev", 
    lifecycle_listeners=[
        lambda state: print("Lifecycle event >>>", state),
    ]
    ) 

# Defining send_query func and pooled_query func, to send requests using ThreadPoolExecutor
desired_range = range(10000)
desired_threads = 10

# Create a Distributed Map in the cluster
counter_map = client.get_map("counter-map").blocking()

counter_map.put("key", 0)

# Pesimistic Locking
def send_query(range_parameter):
    for i in range_parameter:
    	counter_map.lock("key")
    	try:
    		counter = counter_map.get("key") + 1
    		counter_map.put("key", counter)
    	finally:
    		counter_map.unlock("key")

def pooled_query():
    # create the thread pool
    n_threads = desired_threads
    with ThreadPoolExecutor(n_threads) as executor:
        # push query for each thread
        _ = [executor.submit(send_query, desired_range) for i in range(0, desired_threads)]

def pooled_query_thread():
    threads = []
    for i in range(desired_threads):
        thread = Thread(target=send_query, args=(desired_range,))
        threads.append(thread)

    for thr in threads:
        thr.start()

    for thr in threads:
        thr.join()

## Timer start
start = perf_counter()

## Invoke ThreadPoolExecutor to send requests
if __name__ == "__main__":
    # pooled_query()
    pooled_query_thread()

## Timer end
finish = perf_counter()    

print(f'It took {finish-start} second(s) to finish.')

for key, value in counter_map.entry_set():
    print(key, value)

client.shutdown()
主要语言
Python
星标
115
派生
78
平均合并
2 天 23 小时
30 天内合并 PR
2

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

hazelcast/hazelcast-python-client 的其他 Issue

查看 hazelcast/hazelcast-python-client 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。