Streamz with websocket not steaming any data
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- pandas, python
- Domain
- stream-processing
Research direction
Start at the WebSocket callbacks, update_dataframe, source.emit, and source.map(print_result) shown in the issue. Trace whether messages reach source.emit and whether the mapped consumer runs; done means identifying the missing connection or usage detail that prevents received data from being printed.
Written by the indexing model from the issue text.
Description
I am trying to figure out a right way of streaming data using streamz with websocket. My streaming data is loaded using websocket but not streaming anything.
import config
import websocket, json
import pandas as pd
from streamz.dataframe import DataFrame
from streamz import Stream
import time
ticker_list = ["SCHG"]
# Create DataFrame with specified columns and ticker_list as index
df = pd.DataFrame(columns=['Bid', 'Ask', 'Time'], index=ticker_list)
source = Stream()
def update_dataframe(json_message, df):
ticker_data = json_message[0]
ticker_symbol = ticker_data['S']
df.loc[ticker_symbol, 'Bid'] = ticker_data['bp']
df.loc[ticker_symbol, 'Ask'] = ticker_data['ap']
df.loc[ticker_symbol, 'Time'] = pd.to_datetime(ticker_data['t'], unit='s').tz_localize('UTC').tz_convert('America/New_York').strftime('%Y-%m-%d %H:%M:%S%z')
source.emit(df.copy()) # Emit the updated DataFrame to the stream
def on_open(ws):
print("opened connection")
# Authentication
auth_data = {"action":"auth","key": config.API_KEY,"secret": config.SECRET_KEY}
# Subscribe
ws.send(json.dumps(auth_data))
listen_message = {"action":"subscribe","quotes":ticker_list}
ws.send(json.dumps(listen_message))
print('subscribed')
def on_message(ws, message):
json_message = json.loads(message)
# Update the DataFrame and emit it to the stream
update_dataframe(json_message, df)
def on_close(ws):
print("closed connection")
def print_result(x):
print(x)
sink = source.map(print_result) # Set up a sink to consume and process the emitted data
socket = "wss://stream.data.alpaca.markets/v2/iex"
ws = websocket.WebSocketApp(socket, on_open=on_open, on_message=on_message, on_close=on_close)
ws.run_forever()
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 149
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from python-streamz/streamz
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
python-streamz/streamz#481 · 4 comments ·
-
Combining the streamz.Stream.filenames() and streamz.Stream.from_textfile() using dask scatter? Open
Difficulty 5/5 Over a week Newbie friendliness 25/100
python-streamz/streamz#480 · 2 comments ·
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
python-streamz/streamz#479 · 1 comment ·
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
python-streamz/streamz#478 · 6 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 10/100
python-streamz/streamz#476 · 17 comments · 2 reactions ·
All issues in python-streamz/streamz
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
use-agent-os/agent-os#3314 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
BasedHardware/omi#15662 · 1 comment ·
-
documentation help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
AiursoftWeb/AnduinOS-2#19 ·