 Roy Han
|
8f6d0242b6
refactoring
|
9 months ago |
|
Roy Han
|
c697eb2a9b
fix hanging on single string
|
9 months ago |
|
Roy Han
|
bcb63e6e0e
touches
|
9 months ago |
 royjhan
|
b7c622dd32
Merge branch 'main' into royh-batchembed
|
10 months ago |
 Jeffrey Morgan
|
d8def1ff94
llm: allow gemma 2 to context shift (#5534)
|
10 months ago |
|
Jeffrey Morgan
|
0e09c380fc
llm: print caching notices in debug only (#5533)
|
10 months ago |
|
Jeffrey Morgan
|
d89454de80
Use slot with cached prompt instead of least recently used (#5492)
|
10 months ago |
|
Roy Han
|
17de2b4405
Refactoring of legacy and new
|
10 months ago |
|
royjhan
|
3b5a4a77f3
Return Correct Prompt Eval Count Regardless of Cache Prompt (#5371)
|
10 months ago |
|
royjhan
|
a5f23d766e
Merge branch 'main' into royh-batchembed
|
10 months ago |
|
Roy Han
|
512e0a7bde
Clean up
|
10 months ago |
|
Roy Han
|
aee25acb5b
move normalization to go
|
10 months ago |
|
Jeffrey Morgan
|
717f7229eb
Do not shift context for sliding window models (#5368)
|
10 months ago |
|
Roy Han
|
80c1a3f812
playing around with truncate stuff
|
10 months ago |
|
Roy Han
|
c111d8bb51
normalization
|
10 months ago |
|
Roy Han
|
5213c12354
clean up
|
10 months ago |
|
Roy Han
|
49e341147d
add server function
|
10 months ago |
|
Roy Han
|
c406fa7a4c
api/embed draft
|
10 months ago |
|
Roy Han
|
ff191d7cba
Initial Draft
|
10 months ago |
 Michael Yang
|
9d91e5e587
remove confusing log message
|
10 months ago |
 Daniel Hiltgen
|
fb9cdfa723
Fix server.cpp for the new cuda build macros
|
11 months ago |
|
Jeffrey Morgan
|
ead259d877
llm: fix seed value not being applied to requests (#4986)
|
10 months ago |
|
Jeffrey Morgan
|
34f142797a
llm: always add bos token to prompt (#4941)
|
11 months ago |
|
Michael Yang
|
829ff87bd1
revert tokenize ffi (#4761)
|
11 months ago |
|
Michael Yang
|
de781b37c8
rm unused infill
|
11 months ago |
|
Michael Yang
|
3e21799377
rm unused system prompt
|
11 months ago |
|
Michael Yang
|
26a00a0410
use ffi for tokenizing/detokenizing
|
11 months ago |
|
Michael Yang
|
714adb8bd1
bump (#4597)
|
11 months ago |
|
Daniel Hiltgen
|
b37b496a12
Wire up load progress
|
11 months ago |
 Sam
|
e15307fdf4
feat: add support for flash_attn (#4120)
|
11 months ago |
