 Patrick Devine
|
4cc3be3035
Move envconfig and consolidate env vars (#4608)
|
11 月之前 |
 Sang Park
|
4434d7f447
Correct typo in error message (#4535)
|
11 月之前 |
 Daniel Hiltgen
|
ec231a7923
Remove VRAM convergence check for windows
|
11 月之前 |
|
Patrick Devine
|
6845988807
Ollama `ps` command for showing currently loaded models (#4327)
|
11 月之前 |
|
Daniel Hiltgen
|
4142c3ef7c
Always use the sorted list of GPUs
|
11 月之前 |
 Jeffrey Morgan
|
bb6fd02298
Don't clamp ctx size in `PredictServerFit` (#4317)
|
11 月之前 |
|
Daniel Hiltgen
|
354ad9254e
Wait for GPU free memory reporting to converge
|
11 月之前 |
|
Jeffrey Morgan
|
c9f98622b1
Skip scheduling cancelled requests, always reload unloaded runners (#4189)
|
1 年之前 |
|
Jeffrey Morgan
|
dfa2f32ca0
unload in critical section (#4187)
|
1 年之前 |
|
Daniel Hiltgen
|
f56aa20014
Centralize server config handling
|
1 年之前 |
|
Jeffrey Morgan
|
942c979232
allocate a large enough kv cache for all parallel requests (#4162)
|
1 年之前 |
|
Daniel Hiltgen
|
20f6c06569
Make maximum pending request configurable
|
1 年之前 |
 Mark Ward
|
63c763685f
log when the waiting for the process to stop to help debug when other tasks execute during this wait.
|
1 年之前 |
|
Mark Ward
|
f4a73d57a4
fix runner expire during active use. Clearing the expire timer as it is used. Allowing the finish to assign an expire timer so that the runner will expire after no use.
|
1 年之前 |
|
Daniel Hiltgen
|
d6e3b64582
Fix concurrency for CPU mode
|
1 年之前 |
 Daniel Hiltgen
|
9b5a3c5991
Merge pull request #3914 from dhiltgen/mac_perf
|
1 年之前 |
|
Jeffrey Morgan
|
00b0699c75
Reload model if `num_gpu` changes (#3920)
|
1 年之前 |
|
Daniel Hiltgen
|
b123be5b71
Adjust context size for parallelism
|
1 年之前 |
 Bryce Reitano
|
36a6daccab
Restructure loading conditional chain
|
1 年之前 |
|
Bryce Reitano
|
284e02bed0
Move ggml loading to when we attempt fitting
|
1 年之前 |
|
Daniel Hiltgen
|
34b9db5afc
Request and model concurrency
|
1 年之前 |
