 jmorganca
|
ad7e641815
add batch embeddings
|
hai 1 ano |
|
Jeffrey Morgan
|
bb31def011
return code `499` when user cancels request while a model is loading (#3955)
|
hai 1 ano |
|
Jeffrey Morgan
|
993cf8bf55
llm: limit generation to 10x context size to avoid run on generations (#3918)
|
hai 1 ano |
 Daniel Hiltgen
|
6e76348df7
Merge pull request #3834 from dhiltgen/not_found_in_path
|
hai 1 ano |
 Daniel Hiltgen
|
58888a74bc
Detect and recover if runner removed
|
hai 1 ano |
|
Daniel Hiltgen
|
34b9db5afc
Request and model concurrency
|
hai 1 ano |
|
Daniel Hiltgen
|
8711d03df7
Report errors on server lookup instead of path lookup failure
|
hai 1 ano |
|
Daniel Hiltgen
|
aa72281eae
Trim spaces and quotes from llm lib override
|
hai 1 ano |
 Michael Yang
|
3cf483fe48
add stablelm graph calculation
|
hai 1 ano |
|
Michael Yang
|
a8b9b930b4
account for all non-repeating layers
|
hai 1 ano |
|
Michael Yang
|
26df674785
scale graph based on gpu count
|
hai 1 ano |
|
Michael Yang
|
41a272de9f
darwin: no partial offloading if required memory greater than system
|
hai 1 ano |
|
Jeffrey Morgan
|
a0b8a32eb4
Terminate subprocess if receiving `SIGINT` or `SIGTERM` signals while model is loading (#3653)
|
hai 1 ano |
|
Michael Yang
|
7e33a017c0
partial offloading
|
hai 1 ano |
|
Michael Yang
|
8b2c10061c
refactor tensor query
|
hai 1 ano |
|
Daniel Hiltgen
|
c5ff443b9f
Handle very slow model loads
|
hai 1 ano |
|
Michael Yang
|
be517e491c
no rope parameters
|
hai 1 ano |
|
Michael Yang
|
12e923e158
update graph size estimate
|
hai 1 ano |
|
Daniel Hiltgen
|
464d817824
Merge pull request #3464 from dhiltgen/subprocess
|
hai 1 ano |
|
Daniel Hiltgen
|
6589eb8a8c
Revert options as a ref in the server
|
hai 1 ano |
|
Michael Yang
|
80163ebcb5
fix metal gpu
|
hai 1 ano |
|
Daniel Hiltgen
|
58d95cc9bd
Switch back to subprocessing for llama.cpp
|
hai 1 ano |
