[enhancement]: Requesting addition of flash-attn and sage/sage2-attn optimizations in InvokeAI

Is there an existing issue for this?

• I have searched the existing issues

Contact Details

No response

What should this feature add?

I ran a trial of Wan2.2-A14B using i2v lighting workflow, it took 11.5 minutes for graph execution. Details of trial - Model (downloaded from CivitAI) DaSiWa-i2v-Q5 GGUF (high and low), 720p resolution, 5s, 81 frames @16fps, 4-steps with lighting LoRAs (high and low) at strength 1.0, single image start frame. The graph execution took long time, perhaps running on SDPA attention. It might be possible to improve the graph execution time with Flash Attention or Sage/Sage2 Attention implementation in InvokeAI. I request implementation of the same if feasible.

Alternatives

N/A

Additional Content

Console log:
Started Invoke process with PID 27144
[2026-08-23 06:08:20,064]::[InvokeAI]::INFO --> PyTorch CUDA memory allocator: cudaMallocAsync
[2026-08-23 06:08:26,423]::[InvokeAI]::INFO --> Using torch device: NVIDIA GeForce RTX 5090 Laptop GPU
[2026-08-23 06:08:27,459]::[InvokeAI]::INFO --> cuDNN version: 90701
Siglip2ImageProcessorFast is deprecated. The Fast suffix for image processors has been removed; use
Siglip2ImageProcessor instead.
[2026-08-23 06:08:35,952]::[InvokeAI]::INFO --> Patchmatch initialized
[2026-08-23 06:08:37,157]::[InvokeAI]::INFO --> InvokeAI version 6.14.0-rc2
[2026-08-23 06:08:37,157]::[InvokeAI]::INFO --> Root directory = C:\Users\vidz3\Invoke
[2026-08-23 06:08:37,158]::[InvokeAI]::INFO --> Initializing database at C:\Users\vidz3\Invoke\database
s\invokeai.db
[2026-08-23 06:08:37,180]::[InvokeAI]::INFO --> JWT secret loaded from database
[2026-08-23 06:08:37,184]::[ModelManagerService]::INFO --> [MODEL CACHE] Calculated model RAM cache siz
e: 21390.56 MB. Heuristics applied: [1, 2].
[2026-08-23 06:08:37,184]::[ModelManagerService]::INFO --> Model cache global RAM budget: 20.89 GB acro
ss 1 device cache(s).
[2026-08-23 06:08:37,198]::[ModelInstallService]::INFO --> Restoring incomplete installs
[2026-08-23 06:08:37,198]::[ModelInstallService]::INFO --> Finished restoring incomplete installs
[2026-08-23 06:08:37,838]::[InvokeAI]::INFO --> Cleaned database (freed 0.09MB)
[2026-08-23 06:08:37,838]::[InvokeAI]::INFO --> Invoke running on http://127.0.0.1:9090 (Press CTRL+C t
o quit)
[2026-08-23 06:09:59,428]::[InvokeAI]::INFO --> Executing queue item 2010, session 796aa17e-e97c-4612-8
50d-54d84f113a8f on cuda:0
[2026-08-23 06:09:59,832]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '70f2ccdf-91eb-41
fc-b36e-1e350e7c5966:vae' (AutoencoderKLWan) onto cuda device #0 in 0.15s. Total model size: 242.03MB,
VRAM: 242.03MB (100.0%)
[2026-08-23 06:10:13,312]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '4f01ee84-34c9-4e
7c-8f24-c746308fa918:tokenizer' (T5Tokenizer) onto cuda device #0 in 0.00s. Total model size: 0.24MB, V
RAM: 0.00MB (0.0%)
Loading weights: 100%|███████████████████████████████████████████| 242/242 [00:00<00:00, 14263.93it/s]
[2026-08-23 06:10:19,770]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '4f01ee84-34c9-4e
7c-8f24-c746308fa918:text_encoder' (UMT5EncoderModel) onto cuda device #0 in 6.09s. Total model size: 1
2838.48MB, VRAM: 12838.48MB (100.0%)
[2026-08-23 06:10:20,018]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '4f01ee84-34c9-4e
7c-8f24-c746308fa918:tokenizer' (T5Tokenizer) onto cuda device #0 in 0.00s. Total model size: 0.24MB, V
RAM: 0.00MB (0.0%)
[2026-08-23 06:10:20,026]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '4f01ee84-34c9-4e
7c-8f24-c746308fa918:text_encoder' (UMT5EncoderModel) onto cuda device #0 in 0.01s. Total model size: 1
2838.48MB, VRAM: 12838.48MB (100.0%)
Denoising Wan 2.2 video (81 frames): 0%| | 0/4 [00:00<?, ?it/s][
2026-08-23 06:10:30,514]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model 'c87d7a59-aea7-493
9-8f9e-7f7be3c05bd2:transformer' (WanTransformer3DModel) onto cuda device #0 in 1.41s. Total model size
: 10290.63MB, VRAM: 10290.63MB (100.0%)
Denoising Wan 2.2 video (81 frames): 50%|██████████████▌ | 2/4 [05:09<05:08, 154.24s/it][
2026-08-23 06:15:42,365]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '7c63bf92-c992-4f0
7-8292-35787fb8d95c:transformer' (WanTransformer3DModel) onto cuda device #0 in 1.66s. Total model size
: 10290.63MB, VRAM: 10290.63MB (100.0%)
Denoising Wan 2.2 video (81 frames): 100%|█████████████████████████████| 4/4 [10:21<00:00, 155.27s/it]
[2026-08-23 06:20:41,703]::[ModelManagerService]::INFO --> [MODEL CACHE] Loaded model '70f2ccdf-91eb-41
fc-b36e-1e350e7c5966:vae' (AutoencoderKLWan) onto cuda device #0 in 0.05s. Total model size: 242.03MB,
VRAM: 242.03MB (100.0%)
[2026-08-23 06:20:41,704]::[InvokeAI]::INFO --> Running Wan VAE decode: 21 latent frames -> 81 pixel fr
ames at 720x1088
[2026-08-23 06:21:09,711]::[InvokeAI]::INFO --> Encoding MP4: 81 frames @ 16 fps (5.06s) at 720x1088 vi
a libx264
[2026-08-23 06:21:11,320]::[InvokeAI]::INFO --> MP4 encode complete: 3548.3 KB
[2026-08-23 06:21:11,326]::[InvokeAI]::INFO --> Video file written: C:\Users\vidz3\Invoke\outputs\video
s\277164cf-9df7-48d6-bfda-1ef76e897c0f.mp4
[2026-08-23 06:21:12,156]::[InvokeAI]::INFO --> Thumbnail written: C:\Users\vidz3\Invoke\outputs\videos
\thumbnails\277164cf-9df7-48d6-bfda-1ef76e897c0f.webp
[2026-08-23 06:21:12,158]::[InvokeAI]::INFO --> Sidecar written: C:\Users\vidz3\Invoke\outputs\videos\s
idecars\277164cf-9df7-48d6-bfda-1ef76e897c0f.json
[2026-08-23 06:21:12,159]::[InvokeAI]::INFO --> Saved video: 277164cf-9df7-48d6-bfda-1ef76e897c0f.mp4
[2026-08-23 06:21:12,355]::[InvokeAI]::INFO --> Graph stats: 796aa17e-e97c-4612-850d-54d84f113a8f
Node Calls Seconds VRAM Change
rand_int 1 0.004s +0.000G
image 1 0.026s +0.000G
integer 1 0.001s +0.000G
wan_i2v_ideal_dimensions 1 0.001s +0.000G
wan_model_loader 1 0.002s +0.000G
wan_lora_loader 2 0.001s +0.000G
wan_ref_image_encoder 1 12.850s +0.236G
wan_text_encoder 2 7.827s +12.546G
wan_video_denoise 1 621.122s -1.581G
wan_l2v 1 30.889s +0.236G
TOTAL GRAPH EXECUTION TIME: 672.722s
TOTAL GRAPH WALL TIME: 672.725s
RAM used by InvokeAI process: 13.27G (+12.359G)
RAM used to load models: 34.02G
VRAM in use: 11.437G
RAM cache statistics:
Model cache hits: 10
Model cache misses: 16
Models cached: 4
Models cleared from cache: 1
Cache high water mark: 20.67/20.89G

Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None)>
Traceback (most recent call last):
File "C:\Users\vidz3\AppData\Roaming\uv\python\cpython-3.12.12-windows-x86_64-none\Lib\asyncio\events
.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\vidz3\AppData\Roaming\uv\python\cpython-3.12.12-windows-x86_64-none\Lib\asyncio\proact
or_events.py", line 165, in _call_connection_lost
self._sock.shutdown(socket.SHUT_RDWR)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None)>
Traceback (most recent call last):
File "C:\Users\vidz3\AppData\Roaming\uv\python\cpython-3.12.12-windows-x86_64-none\Lib\asyncio\events
.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\vidz3\AppData\Roaming\uv\python\cpython-3.12.12-windows-x86_64-none\Lib\asyncio\proact
or_events.py", line 165, in _call_connection_lost
self._sock.shutdown(socket.SHUT_RDWR)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None)>
Traceback (most recent call last):
File "C:\Users\vidz3\AppData\Roaming\uv\python\cpython-3.12.12-windows-x86_64-none\Lib\asyncio\events
.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\vidz3\AppData\Roaming\uv\python\cpython-3.12.12-windows-x86_64-none\Lib\asyncio\proact
or_events.py", line 165, in _call_connection_lost
self._sock.shutdown(socket.SHUT_RDWR)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None)>
Traceback (most recent call last):
File "C:\Users\vidz3\AppData\Roaming\uv\python\cpython-3.12.12-windows-x86_64-none\Lib\asyncio\events
.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\vidz3\AppData\Roaming\uv\python\cpython-3.12.12-windows-x86_64-none\Lib\asyncio\proact
or_events.py", line 165, in _call_connection_lost
self._sock.shutdown(socket.SHUT_RDWR)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None)>
Traceback (most recent call last):
File "C:\Users\vidz3\AppData\Roaming\uv\python\cpython-3.12.12-windows-x86_64-none\Lib\asyncio\events
.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\vidz3\AppData\Roaming\uv\python\cpython-3.12.12-windows-x86_64-none\Lib\asyncio\proact
or_events.py", line 165, in _call_connection_lost
self._sock.shutdown(socket.SHUT_RDWR)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None)>
Traceback (most recent call last):
File "C:\Users\vidz3\AppData\Roaming\uv\python\cpython-3.12.12-windows-x86_64-none\Lib\asyncio\events
.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\vidz3\AppData\Roaming\uv\python\cpython-3.12.12-windows-x86_64-none\Lib\asyncio\proact
or_events.py", line 165, in _call_connection_lost
self._sock.shutdown(socket.SHUT_RDWR)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None)>
Traceback (most recent call last):
File "C:\Users\vidz3\AppData\Roaming\uv\python\cpython-3.12.12-windows-x86_64-none\Lib\asyncio\events
.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\vidz3\AppData\Roaming\uv\python\cpython-3.12.12-windows-x86_64-none\Lib\asyncio\proact
or_events.py", line 165, in _call_connection_lost
self._sock.shutdown(socket.SHUT_RDWR)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Exception in callback _ProactorBasePipeTransport._call_connection_lost(None)
handle: <Handle _ProactorBasePipeTransport._call_connection_lost(None)>
Traceback (most recent call last):
File "C:\Users\vidz3\AppData\Roaming\uv\python\cpython-3.12.12-windows-x86_64-none\Lib\asyncio\events
.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\vidz3\AppData\Roaming\uv\python\cpython-3.12.12-windows-x86_64-none\Lib\asyncio\proact
or_events.py", line 165, in _call_connection_lost
self._sock.shutdown(socket.SHUT_RDWR)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Shutting down...
Invoke shut down normally