 Jeffrey Morgan
|
be721ca0df
add more search paths for cuda libs
|
1 年之前 |
|
Jeffrey Morgan
|
34344d801c
clean up cmake `build` directory when cross compiling macOS builds
|
1 年之前 |
 Robin Glauser
|
e868c8a5c7
Update api.md (#1878)
|
1 年之前 |
|
Jeffrey Morgan
|
c336693f07
calculate overhead based number of gpu devices (#1875)
|
1 年之前 |
 Daniel Hiltgen
|
e89dc1d54b
Merge pull request #1874 from dhiltgen/correct_cuda_min
|
1 年之前 |
 Daniel Hiltgen
|
1961a81f03
Set corret CUDA minimum compute capability version
|
1 年之前 |
|
Jeffrey Morgan
|
8a8c7e7f8d
only build for metal on `arm64`
|
1 年之前 |
|
Jeffrey Morgan
|
6df83e6daa
update rough cuda overhead estimate to 15% + 384MiB
|
1 年之前 |
 Michael Yang
|
62023177f6
Merge pull request #1614 from jmorganca/mxyng/fix-set-template
|
1 年之前 |
|
Jeffrey Morgan
|
6164f378f2
revert cuda overhead to 20%
|
1 年之前 |
|
Jeffrey Morgan
|
f387e9631b
use runner if cuda alloc won't fit
|
1 年之前 |
|
Jeffrey Morgan
|
6566387ae3
add `TODO` for cuda overhead
|
1 年之前 |
|
Jeffrey Morgan
|
37708931fb
update cuda overhead to 20% to fix crashes when switching between models and large context sizes
|
1 年之前 |
|
Jeffrey Morgan
|
f6cb0a553c
update cuda overhead to 15% or 400MiB
|
1 年之前 |
|
Jeffrey Morgan
|
2680078c13
fix build on linux
|
1 年之前 |
|
Jeffrey Morgan
|
f1b7e5f560
update overhead to 15%
|
1 年之前 |
|
Jeffrey Morgan
|
cb534e6ac2
use 10% vram overhead for cuda
|
1 年之前 |
|
Jeffrey Morgan
|
58ce2d8273
better estimate scratch buffer size
|
1 年之前 |
|
Jeffrey Morgan
|
18ddf6d57d
fix windows build
|
1 年之前 |
|
Michael Yang
|
61e6502449
Merge pull request #1818 from jmorganca/mxyng/fix-alt-prompt
|
1 年之前 |
|
Jeffrey Morgan
|
08f1e18965
Offload layers to GPU based on new model size estimates (#1850)
|
1 年之前 |
 Bruce MacDonald
|
7e8f7c8358
remove ggml automatic re-pull (#1856)
|
1 年之前 |
|
Bruce MacDonald
|
3f3eb19a3b
document response in modelfile template variables (#1428)
|
1 年之前 |
|
Daniel Hiltgen
|
059ae4585e
Merge pull request #1834 from dhiltgen/old_cuda
|
1 年之前 |
|
Daniel Hiltgen
|
6347f501ca
Merge pull request #1828 from dhiltgen/fix_llava
|
1 年之前 |
|
Jeffrey Morgan
|
5feec959ad
dont use `-Wall` in static build (#1833)
|
1 年之前 |
|
Jeffrey Morgan
|
dbdd50b283
add `-DCMAKE_SYSTEM_NAME=Darwin` cmake flag (#1832)
|
1 年之前 |
 Daniel Hiltgen
|
d74ce6bd4f
Detect very old CUDA GPUs and fall back to CPU
|
1 年之前 |
 Guilherme Baptista
|
57942b4676
Update README.md - Community Integrations - Ollama for Ruby (#1830)
|
1 年之前 |
|
Daniel Hiltgen
|
e0d05b0f1e
Accept windows paths for image processing
|
1 年之前 |
