FlareSolverr/FlareSolverr

Logical action for when the partition is full

Aberta

#924 aberto em 17 de out. de 2023

Ver no GitHub
 (2 comentários) (0 reação) (0 responsável)Python (438 forks)batch import
help wanted

Métricas do repositório

Stars
 (4.368 estrelas)
Métricas de merge de PR
 (Nenhuma PRs mesclada em 30d)

Description

Have you checked our README?

  • I have checked the README

Have you followed our Troubleshooting?

  • I have followed your Troubleshooting

Is there already an issue for your problem?

  • I have checked older issues, open and closed

Have you checked the discussions?

  • I have read the Discussions

Environment

CentOS Linux 7 (Core)
Kernel 4.11.8-1.el7.elrepo.x86_64
Python 3.11.6
FlareSolverr 3.3.6
Google Chrome 118.0.5993.70
Headless true

Description

While everything was working fine on the last version. All of a sudden I got the following error:

(env311) root@server2 [/nabi/FlareSolverr]# python src/flaresolverr.py
2023-10-17 04:50:47 INFO     FlareSolverr 3.3.6
2023-10-17 04:50:47 INFO     Testing web browser installation...
2023-10-17 04:50:47 INFO     Platform: Linux-4.11.8-1.el7.elrepo.x86_64-x86_64-with-glibc2.17
2023-10-17 04:50:47 INFO     Chrome / Chromium path: /usr/bin/chromium
2023-10-17 04:50:47 INFO     Chrome / Chromium major version: 118
2023-10-17 04:50:47 INFO     Launching web browser...
Traceback (most recent call last):
  File "/nabi/FlareSolverr/src/utils.py", line 296, in get_user_agent
    driver = get_webdriver()
             ^^^^^^^^^^^^^^^
  File "/nabi/FlareSolverr/src/utils.py", line 177, in get_webdriver
    driver = uc.Chrome(options=options, browser_executable_path=browser_executable_path,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nabi/FlareSolverr/src/undetected_chromedriver/__init__.py", line 474, in __init__
    super(Chrome, self).__init__(
  File "/nabi/FlareSolverr/env311/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
    super().__init__(
  File "/nabi/FlareSolverr/env311/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 56, in __init__
    super().__init__(
  File "/nabi/FlareSolverr/env311/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 206, in __init__
    self.start_session(capabilities)
  File "/nabi/FlareSolverr/src/undetected_chromedriver/__init__.py", line 732, in start_session
    super(selenium.webdriver.chrome.webdriver.WebDriver, self).start_session(
  File "/nabi/FlareSolverr/env311/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 290, in start_session
    response = self.execute(Command.NEW_SESSION, caps)["value"]
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nabi/FlareSolverr/env311/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 345, in execute
    self.error_handler.check_response(response)
  File "/nabi/FlareSolverr/env311/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: disconnected: Unable to receive message from renderer
  (failed to check if window was closed: disconnected: not connected to DevTools)
  (Session info: chrome=118.0.5993.70)
Stacktrace:
#0 0x55d62a04d4e3 <unknown>
#1 0x55d629d7cc76 <unknown>
#2 0x55d629d66284 <unknown>
#3 0x55d629d65fa0 <unknown>
#4 0x55d629d649bf <unknown>
#5 0x55d629d64fed <unknown>
#6 0x55d629d639e1 <unknown>
#7 0x55d629d6c416 <unknown>
#8 0x55d629d63794 <unknown>
#9 0x55d629d65c29 <unknown>
#10 0x55d629d649bf <unknown>
#11 0x55d629d64fed <unknown>
#12 0x55d629d639e1 <unknown>
#13 0x55d629d5ba72 <unknown>
#14 0x55d629d63794 <unknown>
#15 0x55d629d62db4 <unknown>
#16 0x55d629d62a96 <unknown>
#17 0x55d629d7e402 <unknown>
#18 0x55d629d5614c <unknown>
#19 0x55d629d552b5 <unknown>
#20 0x55d629de0e2c <unknown>
#21 0x55d629de047f <unknown>
#22 0x55d629dd7de3 <unknown>
#23 0x55d629dad2dd <unknown>
#24 0x55d629dae34e <unknown>
#25 0x55d62a00d3e4 <unknown>
#26 0x55d62a0113d7 <unknown>
#27 0x55d62a01bb20 <unknown>
#28 0x55d62a012023 <unknown>
#29 0x55d629fe01aa <unknown>
#30 0x55d62a0366b8 <unknown>
#31 0x55d62a036847 <unknown>
#32 0x55d62a046243 <unknown>
#33 0x7f159c290ea5 start_thread


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nabi/FlareSolverr/src/flaresolverr.py", line 105, in <module>
    flaresolverr_service.test_browser_installation()
  File "/nabi/FlareSolverr/src/flaresolverr_service.py", line 72, in test_browser_installation
    user_agent = utils.get_user_agent()
                 ^^^^^^^^^^^^^^^^^^^^^^
  File "/nabi/FlareSolverr/src/utils.py", line 300, in get_user_agent
    raise Exception("Error getting browser User-Agent. " + str(e))
Exception: Error getting browser User-Agent. Message: disconnected: Unable to receive message from renderer
  (failed to check if window was closed: disconnected: not connected to DevTools)
  (Session info: chrome=118.0.5993.70)
Stacktrace:
#0 0x55d62a04d4e3 <unknown>
#1 0x55d629d7cc76 <unknown>
#2 0x55d629d66284 <unknown>
#3 0x55d629d65fa0 <unknown>
#4 0x55d629d649bf <unknown>
#5 0x55d629d64fed <unknown>
#6 0x55d629d639e1 <unknown>
#7 0x55d629d6c416 <unknown>
#8 0x55d629d63794 <unknown>
#9 0x55d629d65c29 <unknown>
#10 0x55d629d649bf <unknown>
#11 0x55d629d64fed <unknown>
#12 0x55d629d639e1 <unknown>
#13 0x55d629d5ba72 <unknown>
#14 0x55d629d63794 <unknown>
#15 0x55d629d62db4 <unknown>
#16 0x55d629d62a96 <unknown>
#17 0x55d629d7e402 <unknown>
#18 0x55d629d5614c <unknown>
#19 0x55d629d552b5 <unknown>
#20 0x55d629de0e2c <unknown>
#21 0x55d629de047f <unknown>
#22 0x55d629dd7de3 <unknown>
#23 0x55d629dad2dd <unknown>
#24 0x55d629dae34e <unknown>
#25 0x55d62a00d3e4 <unknown>
#26 0x55d62a0113d7 <unknown>
#27 0x55d62a01bb20 <unknown>
#28 0x55d62a012023 <unknown>
#29 0x55d629fe01aa <unknown>
#30 0x55d62a0366b8 <unknown>
#31 0x55d62a036847 <unknown>
#32 0x55d62a046243 <unknown>
#33 0x7f159c290ea5 start_thread

After 2 days of confusion Check the source code for bugs Test on multiple servers Testing various Chromium versions Testing and checking inside Docker server Test my own proxies Remote access tests Many searches on the Internet

And after 2 whole days my time was wasted and I was completely disappointed and confused. I happened to see the problem in a place I never thought:

(env311) root@server2 [/nabi/FlareSolverr]# df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        3.9G     0  3.9G   0% /dev
tmpfs           3.9G   12K  3.9G   1% /dev/shm
tmpfs           3.9G  345M  3.6G   9% /run
tmpfs           3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/sda2       191G  147G   35G  81% /
**/dev/sda5       2.0G  2.0G     0 100% /tmp**
/dev/sda1       480M   94M  357M  21% /boot
tmpfs           799M  4.0K  799M   1% /run/user/0


(env311) root@server2 [/nabi/FlareSolverr]# ls /tmp
lost+found                            tmp2yt92bdq  tmpb11gr_37  tmpjn_tvvyt            tmpr257tvnv
mc-root                               tmp2yz04mu1  tmpb531_a5n  tmpjp8jfxex            tmpr3jjpmer
puppeteer_dev_chrome_profile-0SzjUq   tmp30d7g5xu  tmpb60ohdlf  tmpjto_kr02            tmprg8ukogj
puppeteer_dev_chrome_profile-3Apt8K   tmp33z4igqs  tmpb62tlgwj  tmpjuvlf_0q            tmprpr2p8pr
puppeteer_dev_chrome_profile-4QeEZP   tmp362ptpfv  tmpb7azyw1z  tmpjyso8ty_            tmprt3b8gp2
puppeteer_dev_chrome_profile-4ud6nn   tmp3jjo5ffz  tmpb8veevlt  tmpk8znbczw            tmpryt2zect
puppeteer_dev_chrome_profile-66je1M   tmp3ptcr7_f  tmpbih28a47  tmpkapqeqrh            tmps0z2rgj4
puppeteer_dev_chrome_profile-9bvJRL   tmp3q0p5tvn  tmpbk17g2as  tmpkcq1v0il            tmps9ck1hof
puppeteer_dev_chrome_profile-aiRADV   tmp41hv_r18  tmpbodfwaje  tmp_k_eso0x            tmps9s35ylr
puppeteer_dev_chrome_profile-cD4Gtj   tmp41lpsmkn  tmpbplqcw92  tmpknf0819n            tmpsanjn928
puppeteer_dev_chrome_profile-CHgzlj   tmp43cndxd_  tmpbtmrawlp  tmpknwfsjc2            tmp.sDyesBBrOt
puppeteer_dev_chrome_profile-ctHcE3   tmp45994cbn  tmpbuc_2bpe  tmp_ku7pms4            tmpsj_qgfyh
puppeteer_dev_chrome_profile-CxmwBZ   tmp4_63ws9s  tmpbvohofjo  tmpkvv7gr1_            tmpspqq1iyj
puppeteer_dev_chrome_profile-DIMYkG   tmp46rmqhyb  tmpc0agx5bf  tmpkxwovalk            tmpsqkmji27
puppeteer_dev_chrome_profile-dJxScN   tmp4a1v81wd  tmpc1gnqdyt  tmpl01qvfmt            tmp.SRfhaJMKRj
puppeteer_dev_chrome_profile-DMFbWl   tmp4bak079j  tmpc_3w32u0  tmpl6bltdow            tmpsyfwgoa7
puppeteer_dev_chrome_profile-fULBhM   tmp4bru59fc  tmpc71myr9r  tmpl_6ip6ej            tmpt2x8559k
puppeteer_dev_chrome_profile-fwoKYb   tmp4gw58y3t  tmpc7hhxny4  tmpl8sqrlcd            tmpt3liygls
puppeteer_dev_chrome_profile-hwqVVk   tmp4i2nf1g4  tmpca9pbvny  tmplatw4c4l            tmptavxqwki
puppeteer_dev_chrome_profile-IkFddX   tmp4lcs0zap  tmpcch0rzu4  tmplf4v1kn6            tmptq8jz26z
puppeteer_dev_chrome_profile-j6xqD0   tmp4ncqhcp9  tmpce6pk_3i  tmplivsv7z0            tmptqiucvjb
puppeteer_dev_chrome_profile-j6zk7P   tmp4o33r53x  tmpcfsfpodr  tmpln7lbt88            tmp_tt3bl_z
puppeteer_dev_chrome_profile-Jtxq1r   tmp4ssdoik4  tmpcht3mrne  tmplqkrthz0            tmpu2_nkwbf
puppeteer_dev_chrome_profile-K6arvJ   tmp5026jy27  tmpcoiz36wg  tmplry2m7d_            tmpu35m8hje
puppeteer_dev_chrome_profile-kT2lhW   tmp54kv54od  tmpcpilci1p  tmplzhhf11q            tmpu4pofipi
puppeteer_dev_chrome_profile-lShjEk   tmp_55_s2zj  tmpctgiyoh_  tmpm2hut_ra            tmpufxyl_5r
puppeteer_dev_chrome_profile-MW6yqL   tmp57h54539  tmpd1ixnu2h  tmpm35rhru9            tmpuiqliq0m
puppeteer_dev_chrome_profile-NDvMfA   tmp5a_0qvzz  tmpd519b1sw  tmpm4ltrc3o            tmpuj4hiitq
puppeteer_dev_chrome_profile-OUxdQ0   tmp5hii5v22  tmpd61e4dwj  tmpm5bbpfm_cacert.pem  tmpukc24tnm
puppeteer_dev_chrome_profile-PElFIB   tmp5pcmhg7c  tmpd86sfbkm  tmpma7j2p4q            tmpuq1nek01
puppeteer_dev_chrome_profile-PGLt1z   tmp5u_vc86s  tmpd8u0tyhs  tmpmavi51qb            tmpusc5k6hx
puppeteer_dev_chrome_profile-Qw1jmP   tmp5xmkehwb  tmpdovfppnc  tmpmjnuo4rt            tmpv4rp45lz
puppeteer_dev_chrome_profile-R7ty3e   tmp5zbury0u  tmpdwgalisu  tmpmvqwz4in            tmpv7p8d756
puppeteer_dev_chrome_profile-rB2VzQ   tmp643mysye  tmpe3vpxb4v  tmpn0jn8h09            tmpva_u10wd
puppeteer_dev_chrome_profile-RJg2aa   tmp64n8v4ha  tmpe8y6c6nf  tmpn_8bb2jw            tmpvcfbocbi
puppeteer_dev_chrome_profile-Ryg7af   tmp66foayuj  tmpeew1mzvf  tmpncelm3x5            tmpvcmx2gob
puppeteer_dev_chrome_profile-SkLwfx   tmp67pkyu2c  tmpehewmk4m  tmpnd3_xutf            tmpve7rhdfo
puppeteer_dev_chrome_profile-sq1Tzc   tmp_68ydxfq  tmpeint0p2e  tmpnh_i2dbz            tmpviaqh_xd
puppeteer_dev_chrome_profile-ssK2K8   tmp6drheatn  tmpekvv0rb7  tmpnms5jb59            tmpvsab08su
puppeteer_dev_chrome_profile-stvhQC   tmp6dzs2fym  tmpem1h7qt1  tmpnnxl3f_f            tmpw8mw1se5
puppeteer_dev_chrome_profile-ttN9oN   tmp6fympdd5  tmpeypgqhhf  tmpno1yzevc            tmpwc67d_sz
puppeteer_dev_chrome_profile-WGovaR   tmp6fynwgiv  tmpf1vj4xi3  tmpnr_twrxe            tmpwdyfxjco
puppeteer_dev_profile-0ydPaB          tmp6hda4572  tmpf5a94w1j  tmpnsf12e1d            tmpwe6mhwzk
puppeteer_dev_profile-BpCg0h          tmp6qizukeu  tmpf_8z_txv  tmpnzscyyb_            tmpwhevqmd7
puppeteer_dev_profile-Dr1efw          tmp6tcg19v2  tmpf_aguhqb  tmpo3c0prdn            tmpwrcfnac8
puppeteer_dev_profile-e07GgP          tmp6ub8fyw0  tmpfhrn9dyg  tmpo58yp6qw            tmpwsr6vw5r
puppeteer_dev_profile-PDl4d3          tmp6xgws51a  tmpfwfeqboa  tmpo64pavxc            tmpwtn3obd9
puppeteer_dev_profile-rORDLI          tmp6xx3s8e_  tmpg7h7n9vo  tmpo7nsy4f6            tmpwwn8e4vh
puppeteer_dev_profile-VMFiOq          tmp7917f30_  tmpg828le66  tmpocdxhc7k            tmpwwx9j94z
puppeteer_dev_profile-XcDoYh          tmp79il713d  tmpgfg3urgw  tmpoj3toohz            tmpwy9dr97l
puppeteer_dev_profile-xviTMQ          tmp7is5k054  tmpgfydb54e  tmpon8ujak3            tmpx14vdegv
puppeteer_dev_profile-FviTMD          tmp7k08uh2l  tmpgm3qaeaq  tmpoqg5_94s            tmpx1jhg2ni
tmp06nqkm8t                           tmp7u65245r  tmpgmljq48b  tmpotd4d_vn            tmpx43zimjq
tmp0gnnm8br                           tmp7wlv0a8u  tmpgmsorqyk  tmpotz3xvb2            tmpx45uw_8g
tmp0je9pzxp                           tmp802et1vy  tmpgpecku2f  tmpox1ylym2            tmpx5fxhusf
tmp_0o9updu                           tmp80z8zedd  tmpgqay12uh  tmpp2ak4x_p            tmpxciftnqc
tmp0rufnanu                           tmp8f3ms4x0  tmpg_snxj4k  tmpp47456to            tmpxd5l6pwi
tmp0udaycff                           tmp8gakl9sc  tmph15l8u5y  tmpp7vtp2tp            tmpxe3b119d
tmp0vtu65sj                           tmp8ltbafv_  tmph7a0nfa2  tmpp8ngbuf6            tmpxnz34q4m
tmp0wvtbzvh                           tmp8qmehqxu  tmph7p_zmc4  tmpp8wyznd1            tmpxotelrjr
tmp0yh1a7cd                           tmp8vc49gq6  tmphadwh90l  tmppbp7rsik            tmpxp2lntx1
tmp11oifb1g                           tmp8x_p_b61  tmphb8m0b4d  tmpps1yhb3q            tmpy2ijw0dz
tmp1g7l7_a9                           tmp8yfyr_gd  tmphkbmk7yi  tmppvruabbs            tmpy5x6fz9e
tmp1h56by0l                           tmp91uz4f25  tmphmvs9y2w  tmppx2wn817            tmpy5yq2mj4
tmp1i_9n_lb                           tmp9a94dtyz  tmphnhkqizk  tmppzn04xan            tmpyilfafnh
tmp1vdek48h                           tmp9b6byfjz  tmphywe4yc_  tmpq1ciswa_            tmp.yN1qSgp9SE
tmp1xa3flh9                           tmp9eks9ihg  tmpi5pc0x4e  tmpq2wx9ty4            tmpy_nwe9fv
tmp1ypdcsi6                           tmp9f_5u_sx  tmpi9tmn3my  tmpq33_tvdm            tmpyx_skuvt
tmp1ytcbv3b                           tmp9fyok4z1  tmpidqeorbq  tmpq_3g6ne_            tmpz0tya1h5
tmp21tqb0bj                           tmp9hcmsggc  tmpihx86lra  tmpq62006dz            tmpz2arsh59
tmp25nw7g5h                           tmp9xipkv0n  tmpikjg_q50  tmpqde8thax            tmpz4ftl4vb
tmp26lrd1c8                           tmp9zq5tz22  tmpilogapo9  tmpqiir8cxt            tmpzbuvm_ac
tmp29vfkego                           tmpa31yefgk  tmpiwv9b5mf  tmpqklzgzik            tmpzi1txfxh
tmp2kw9fg3q                           tmpa9iwqb4m  tmpj5n8_gly  tmpqkwrjopu            tmpzjig9q_q
tmp2q2tikrd                           tmpady3pte5  tmpj8pmiadr  tmpqpbtytz1            tmpzjltxmzs
tmp2ulfbx_o                           tmpaem007ub  tmpjbqahzqd  tmpqpj_3l2u            tmpzn6rdyug
tmp2veu1dyb                           tmpajray7il  tmpjfousy0k  tmpqtpaw3v_            tmpzsasixwe
tmp2xxt4oum                           tmpausb12dg  tmpjkhslcfz  tmpqz358yzi            tmpzwbhyvsz
tmp2y89ail6                           tmpaxrwzkxt  tmpjmqim467  tmpr_1dhnmp

Yes! My /tmp partition was full!

I just wanted to inform about this so that if you encounter such errors, you can solve the problem quickly.

Meanwhile, I deleted the tmp, but if it fills up again quickly, I can't easily increase the size of this partition. Maybe a solution for me is to set the tmp directory path myself. And of course, I'm still not sure if the tmp filling is caused by FlareSolverr. But in any case, maybe a logical action is if the partition has no space, the program will issue a logical error.

Anyway, my main goal was just to inform.

Logged Error Messages

Above.

Screenshots

No response

Guia do colaborador

Direção de pesquisa
O problema ocorre em utils.py (get webdriver, get user agent) quando /tmp está cheio. Um contribuidor deve adicionar uma verificação de espaço em disco antes de iniciar o Chrome, ou tratar a exceção 'disconnected' para sugerir a limpeza de /tmp. Alternativamente, modificar flaresolverr service.py para usar um diretório temporário configurável. A solução alternativa do usuário fornece um ponto de partida claro.
Pilha de tecnologia
python
Domain
backend
Tipo Issue
Bug
DifficultyDificuldade de implementação estimada para um novo contribuidor, de 1 para alterações muito pequenas a 5 para trabalho de nível especializado.
2
Tempo estimadoUm intervalo de tempo aproximado para um colaborador experiente investigar, implementar, testar e preparar um pull request.
1-3 horas
Status da atividadeQuão disponível o issue aparece agora: novo, ativo, obsoleto, bloqueado ou aguardando entrada do mantenedor.
Ativo
ClarityCom que clareza o issue explica a mudança esperada, os critérios de aceitação e a próxima etapa.
Claro
Prerequisites
Linuxdisk space check
Simpatia para novatosUma pontuação de 1 a 100 que estima o quão acessível este issue é para colaboradores iniciantes.
70