Jeffrey Morgan
|
cb534e6ac2
use 10% vram overhead for cuda
|
1 anno fa |
Jeffrey Morgan
|
08f1e18965
Offload layers to GPU based on new model size estimates (#1850)
|
1 anno fa |
Daniel Hiltgen
|
d74ce6bd4f
Detect very old CUDA GPUs and fall back to CPU
|
1 anno fa |
Daniel Hiltgen
|
a2ad952440
Fix windows system memory lookup
|
1 anno fa |
Daniel Hiltgen
|
d966b730ac
Switch windows build to fully dynamic
|
1 anno fa |
Daniel Hiltgen
|
7555ea44f8
Revamp the dynamic library shim
|
1 anno fa |
Daniel Hiltgen
|
1b991d0ba9
Refine build to support CPU only
|
1 anno fa |
Daniel Hiltgen
|
35934b2e05
Adapted rocm support to cgo based llama.cpp
|
1 anno fa |