Using PyJulia from Streamlit App

Bắt đầu bằng cách tái hiện vấn đề với streamlit_julia_test.py, julia_test.jl và các phiên bản Python, Julia, PyJulia và Streamlit được liệt kê trên macOS. So sánh các đường dẫn main_include, api_compiled_false và python-jl, sau đó xác minh rằng ứng dụng có thể chạy và trả về ma trận Julia mà không gặp lỗi trình thông dịch không được hỗ trợ, lỗi thiếu thư viện hoặc segmentation fault.

Summary

I'm trying to use a julia function from a streamlit app using PyJulia. Created a toy example to test the interaction, simply returning a matrix from a julia functions based on a single parameter to specify the value of the diagonal elements.

Will also note at the outset that both julia_import_method = "api_compiled_false" and julia_import_method = "main_include" works when importing the function in Spyder IDE (rather than at the command line to launch the streamlit app via streamlit run streamlit_julia_test.py).

My project directory looks like:

─ my_project_directory

• julia_test.jl
• streamlit_julia_test.py

The julia function is in julia_test.jl and just simply returns a matrix with diagonals specified by the v parameter:

function get_matrix_from_julia(v::Int)
   m=[v 0
      0 v]
 return m
end

The streamlit app is streamlit_julia_test.py and is defined as:

import os
from io import BytesIO

import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
import streamlit as st

# options:
# main_include
# api_compiled_false
# dont_import_julia
julia_import_method = "api_compiled_false"

if julia_import_method == "main_include":

    import julia
    from julia import Main
    Main.include("julia_test.jl")

elif julia_import_method == "api_compiled_false":

    # works in Spyder IDE
    from julia.api import Julia
    jl = Julia(compiled_modules=False)

    this_dir = os.getcwd()
    julia_test_path = """include(\""""+ this_dir + """/julia_test.jl\"""" +")"""
    print(julia_test_path)
    jl.eval(julia_test_path)
    get_matrix_from_julia = jl.eval("get_matrix_from_julia")

elif julia_import_method == "dont_import_julia":
    print("Not importing ")
else:
    ValueError("Not handling this case:" + julia_import_method)


st.header('Using Julia in Streamlit App Example')

st.text("Using Method:" + julia_import_method)

matrix_element = st.selectbox('Set Matrix Diagonal to:', [1,2,3])

matrix_numpy = np.array([[matrix_element,0],[0,matrix_element]])

col1, col2 = st.columns([4,4])

with col1:

    fig, ax = plt.subplots(figsize=(5,5))
    sns.heatmap(matrix_numpy, ax = ax, cmap="Blues",annot=True)
    ax.set_title('Matrix Using Python Numpy')
    buf = BytesIO()
    fig.savefig(buf, format="png")
    st.image(buf)

with col2:

    if julia_import_method == "dont_import_julia":
        matrix_julia = matrix_numpy
    else:
        matrix_julia = get_matrix_from_julia(matrix_element)

    fig, ax = plt.subplots(figsize=(5,5))
    sns.heatmap(matrix_julia, ax = ax, cmap="Blues",annot=True)
    ax.set_title('Matrix from External Julia Script')
    buf = BytesIO()
    fig.savefig(buf, format="png")
    st.image(buf)

If the app were working correctly, it would look like this (which can be reproduced by setting the julia_import_method = "dont_import_julia" on line 13):

Testing

When I try julia_import_method = "main_include", I get the well known error:

julia.core.UnsupportedPythonError: It seems your Julia and PyJulia setup are not supported.

Julia executable:
    julia
Python interpreter and libpython used by PyCall.jl:
    /Users/myusername/opt/anaconda3/bin/python3
    /Users/myusername/opt/anaconda3/lib/libpython3.9.dylib
Python interpreter used to import PyJulia and its libpython.
    /Users/myusername/opt/anaconda3/bin/python
    /Users/myusername/opt/anaconda3/lib/libpython3.9.dylib

Your Python interpreter "/Users/myusername/opt/anaconda3/bin/python"
is statically linked to libpython.  Currently, PyJulia does not fully
support such Python interpreter.

The easiest workaround is to pass `compiled_modules=False` to `Julia`
constructor.  To do so, first *reboot* your Python REPL (if this happened
inside an interactive session) and then evaluate:

    >>> from julia.api import Julia
    >>> jl = Julia(compiled_modules=False)

Another workaround is to run your Python script with `python-jl`
command bundled in PyJulia.  You can simply do:

    $ python-jl PATH/TO/YOUR/SCRIPT.py

See `python-jl --help` for more information.

For more information, see:

    https://pyjulia.readthedocs.io/en/latest/troubleshooting.html

As suggested, when I set julia_import_method = "api_compiled_false", I get a seg fault:

include("my_project_directory/julia_test.jl")
2022-04-03 10:23:13.406 Traceback (most recent call last):
  File "/Users/myusername/opt/anaconda3/lib/python3.9/site-packages/streamlit/script_runner.py", line 430, in _run_script
    exec(code, module.__dict__)
  File "my_project_directory/streamlit_julia_test.py", line 25, in <module>
    jl = Julia(compiled_modules=False)
  File "/Users/myusername/.local/lib/python3.9/site-packages/julia/core.py", line 502, in __init__
    if not self.api.was_initialized:  # = jl_is_initialized()
  File "/Users/myusername/.local/lib/python3.9/site-packages/julia/libjulia.py", line 114, in __getattr__
    return getattr(self.libjulia, name)
  File "/Users/myusername/opt/anaconda3/lib/python3.9/ctypes/__init__.py", line 395, in __getattr__
    func = self.__getitem__(name)
  File "/Users/myuserame/opt/anaconda3/lib/python3.9/ctypes/__init__.py", line 400, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: dlsym(0x21b8e5840, was_initialized): symbol not found


signal (11): Segmentation fault: 11
in expression starting at none:0
Allocations: 35278345 (Pool: 35267101; Big: 11244); GC: 36
zsh: segmentation fault  streamlit run streamlit_julia_test.py

I've also tried the alternative recommendation provided in the PyJulia response message regarding the use of:

python-jl my_project_directory/streamlit_julia_test.py

But I get this error when running the python-jl command:

INTEL MKL ERROR: dlopen(/Users/myusername/opt/anaconda3/lib/libmkl_intel_thread.1.dylib, 0x0009): Library not loaded: @rpath/libiomp5.dylib
  Referenced from: /Users/myusername/opt/anaconda3/lib/libmkl_intel_thread.1.dylib
  Reason: tried: '/Applications/Julia-1.7.app/Contents/Resources/julia/bin/../lib/libiomp5.dylib' (no such file), '/usr/local/lib/libiomp5.dylib' (no such file), '/usr/lib/libiomp5.dylib' (no such file).
Intel MKL FATAL ERROR: Cannot load libmkl_intel_thread.1.dylib.

So I'm stuck, thanks in advance for a modified reproducible example or instructions for the following system specs!

System specs:
Mac OS Monterey 12.2.1 (Chip - Mac M1 Pro)
Python 3.9.7
Julia 1.7.2
PyJulia 0.5.8.dev
Streamlit 1.7.0