 Michael Yang
|
75a07dd8f7
integrate mllama.cpp to server.cpp
|
vor 7 Monaten |
 jmorganca
|
71e76f8c90
server.cpp: cleanup cross attention state
|
vor 7 Monaten |
|
jmorganca
|
7d5e0ff80e
add server.cpp and patches
|
vor 7 Monaten |
 Jesse Gross
|
93ac3760cb
runner: Flush pending responses before returning
|
vor 7 Monaten |
|
Jeffrey Morgan
|
5e2653f9fe
llm: update llama.cpp commit to 8962422 (#6618)
|
vor 8 Monaten |
 FellowTraveler
|
94fff5805f
Fix sprintf to snprintf (#5664)
|
vor 8 Monaten |
 Daniel Hiltgen
|
90ca84172c
Fix embeddings memory corruption (#6467)
|
vor 8 Monaten |
|
Jeffrey Morgan
|
15c2d8fe14
server: parallelize embeddings in API web handler instead of in subprocess runner (#6220)
|
vor 9 Monaten |
|
Jeffrey Morgan
|
e04c7012c2
update llama.cpp submodule to `1e6f6554` (#6208)
|
vor 9 Monaten |
 royjhan
|
86b907f82a
sort batch results (#6189)
|
vor 9 Monaten |
|
royjhan
|
1b44d873e7
Add Metrics to `api\embed` response (#5709)
|
vor 9 Monaten |
|
Jeffrey Morgan
|
68ee42f995
update llama.cpp submodule to `6eeaeba1` (#6039)
|
vor 9 Monaten |
 Daniel Hiltgen
|
e12fff8810
Enable windows error dialog for subprocess startup
|
vor 9 Monaten |
|
royjhan
|
b9f5e16c80
Introduce `/api/embed` endpoint supporting batch embedding (#5127)
|
vor 9 Monaten |
|
Jeffrey Morgan
|
d8def1ff94
llm: allow gemma 2 to context shift (#5534)
|
vor 10 Monaten |
|
Jeffrey Morgan
|
0e09c380fc
llm: print caching notices in debug only (#5533)
|
vor 10 Monaten |
|
Jeffrey Morgan
|
d89454de80
Use slot with cached prompt instead of least recently used (#5492)
|
vor 10 Monaten |
|
royjhan
|
3b5a4a77f3
Return Correct Prompt Eval Count Regardless of Cache Prompt (#5371)
|
vor 10 Monaten |
|
Jeffrey Morgan
|
717f7229eb
Do not shift context for sliding window models (#5368)
|
vor 10 Monaten |
|
Michael Yang
|
9d91e5e587
remove confusing log message
|
vor 10 Monaten |
|
Daniel Hiltgen
|
fb9cdfa723
Fix server.cpp for the new cuda build macros
|
vor 11 Monaten |
|
Jeffrey Morgan
|
ead259d877
llm: fix seed value not being applied to requests (#4986)
|
vor 11 Monaten |
|
Jeffrey Morgan
|
34f142797a
llm: always add bos token to prompt (#4941)
|
vor 11 Monaten |
|
Michael Yang
|
829ff87bd1
revert tokenize ffi (#4761)
|
vor 11 Monaten |
|
Michael Yang
|
de781b37c8
rm unused infill
|
vor 1 Jahr |
|
Michael Yang
|
3e21799377
rm unused system prompt
|
vor 1 Jahr |
|
Michael Yang
|
26a00a0410
use ffi for tokenizing/detokenizing
|
vor 1 Jahr |
|
Michael Yang
|
714adb8bd1
bump (#4597)
|
vor 11 Monaten |
|
Daniel Hiltgen
|
b37b496a12
Wire up load progress
|
vor 11 Monaten |
 Sam
|
e15307fdf4
feat: add support for flash_attn (#4120)
|
vor 11 Monaten |
