 Jeffrey Morgan
|
c4cf8ad559
llm: avoid loading model if system memory is too small (#5637)
|
vor 9 Monaten |
 Daniel Hiltgen
|
f6f759fc5f
Detect CUDA OS Overhead
|
vor 9 Monaten |
|
Daniel Hiltgen
|
ef757da2c9
Better nvidia GPU discovery logging
|
vor 10 Monaten |
|
Daniel Hiltgen
|
b2799f111b
Move libraries out of users path
|
vor 10 Monaten |
|
Jeffrey Morgan
|
163cd3e77c
gpu: add env var for detecting Intel oneapi gpus (#5076)
|
vor 10 Monaten |
|
Daniel Hiltgen
|
6f351bf586
review comments and coverage
|
vor 11 Monaten |
|
Daniel Hiltgen
|
fc37c192ae
Refine CPU load behavior with system memory visibility
|
vor 11 Monaten |
|
Daniel Hiltgen
|
434dfe30c5
Reintroduce nvidia nvml library for windows
|
vor 11 Monaten |
|
Daniel Hiltgen
|
4e2b7e181d
Refactor intel gpu discovery
|
vor 11 Monaten |
|
Daniel Hiltgen
|
6fd04ca922
Improve multi-gpu handling at the limit
|
vor 11 Monaten |
|
Daniel Hiltgen
|
43ed358f9a
Refine GPU discovery to bootstrap once
|
vor 11 Monaten |
|
Daniel Hiltgen
|
efac488675
Revert "Limit GPU lib search for now (#4777)"
|
vor 11 Monaten |
|
Daniel Hiltgen
|
aac367636d
Actually skip PhysX on windows
|
vor 10 Monaten |
 Michael Yang
|
bf7edb0d5d
lint linux
|
vor 11 Monaten |
|
Jeffrey Morgan
|
476fb8e892
Limit GPU lib search for now (#4777)
|
vor 11 Monaten |
 Daniel Hiltgen
|
646371f56d
Merge pull request #3278 from zhewang1-intc/rebase_ollama_main
|
vor 11 Monaten |
 Patrick Devine
|
4cc3be3035
Move envconfig and consolidate env vars (#4608)
|
vor 11 Monaten |
 Wang,Zhe
|
fd5971be0b
support ollama run on Intel GPUs
|
vor 11 Monaten |
|
Daniel Hiltgen
|
30a7d7096c
Bump VRAM buffer back up
|
vor 11 Monaten |
|
Daniel Hiltgen
|
8727a9c140
Record more GPU information
|
vor 1 Jahr |
|
Michael Yang
|
4736391bfb
llm: add minimum based on layer size
|
vor 1 Jahr |
|
Daniel Hiltgen
|
380378cc80
Use our libraries first
|
vor 1 Jahr |
|
Daniel Hiltgen
|
af9eb36f9f
Merge pull request #4135 from dhiltgen/no_physx
|
vor 1 Jahr |
|
Daniel Hiltgen
|
06093fd396
Merge pull request #4067 from dhiltgen/cudart
|
vor 1 Jahr |
|
Daniel Hiltgen
|
f56aa20014
Centralize server config handling
|
vor 1 Jahr |
|
Daniel Hiltgen
|
b1ad3a43cb
Skip PhysX cudart library
|
vor 1 Jahr |
|
Daniel Hiltgen
|
089daaeabc
Add CUDA Driver API for GPU discovery
|
vor 1 Jahr |
|
Daniel Hiltgen
|
34b9db5afc
Request and model concurrency
|
vor 1 Jahr |
|
Michael Yang
|
7e33a017c0
partial offloading
|
vor 1 Jahr |
|
Daniel Hiltgen
|
1f11b52511
Refined min memory from testing
|
vor 1 Jahr |
