 royjhan
|
a5f23d766e
Merge branch 'main' into royh-batchembed
|
10 months ago |
 Roy Han
|
00a4cb26ca
use float32
|
10 months ago |
 Josh Yan
|
33a65e3ba3
error
|
10 months ago |
|
Roy Han
|
aee25acb5b
move normalization to go
|
10 months ago |
 Daniel Hiltgen
|
3518aaef33
Merge pull request #4218 from dhiltgen/auto_parallel
|
10 months ago |
|
Roy Han
|
c111d8bb51
normalization
|
10 months ago |
|
Roy Han
|
49e341147d
add server function
|
10 months ago |
|
Roy Han
|
c406fa7a4c
api/embed draft
|
10 months ago |
|
Roy Han
|
ff191d7cba
Initial Draft
|
10 months ago |
 Blake Mizerany
|
cb42e607c5
llm: speed up gguf decoding by a lot (#5246)
|
10 months ago |
|
Roy Han
|
0f87628b6d
Revert "Initial Batch Embedding"
|
10 months ago |
 Daniel Hiltgen
|
17b7186cd7
Enable concurrency by default
|
1 year ago |
|
Daniel Hiltgen
|
5bf5aeec01
Refine mmap default logic on linux
|
10 months ago |
|
Daniel Hiltgen
|
96624aa412
Merge pull request #5072 from dhiltgen/windows_path
|
10 months ago |
|
Roy Han
|
c22d54895a
Initial Batch Embedding
|
10 months ago |
|
Daniel Hiltgen
|
7784ca33ce
Tighten up memory prediction logging
|
10 months ago |
|
Daniel Hiltgen
|
171796791f
Adjust mmap logic for cuda windows for faster model load
|
10 months ago |
|
Daniel Hiltgen
|
b2799f111b
Move libraries out of users path
|
10 months ago |
|
Daniel Hiltgen
|
da3bf23354
Workaround gfx900 SDMA bugs
|
11 months ago |
|
Daniel Hiltgen
|
6f351bf586
review comments and coverage
|
11 months ago |
|
Daniel Hiltgen
|
fc37c192ae
Refine CPU load behavior with system memory visibility
|
11 months ago |
|
Daniel Hiltgen
|
6fd04ca922
Improve multi-gpu handling at the limit
|
11 months ago |
 Craig Hughes
|
b84aea1685
Critical fix from llama.cpp JSON grammar to forbid un-escaped escape characters inside strings, which breaks parsing. (#3782)
|
11 months ago |
 Michael Yang
|
e40145a39d
lint
|
11 months ago |
|
Michael Yang
|
c895a7d13f
some gocritic
|
11 months ago |
|
Michael Yang
|
829ff87bd1
revert tokenize ffi (#4761)
|
11 months ago |
 Jeffrey Morgan
|
a50a87a7b8
partial offloading: allow flash attention and disable mmap (#4734)
|
11 months ago |
|
Michael Yang
|
26a00a0410
use ffi for tokenizing/detokenizing
|
1 year ago |
|
Daniel Hiltgen
|
92c81e8117
Give the final model loading more time
|
11 months ago |
 Lei Jitang
|
7487229c34
llm/server.go: Fix 2 minor typos (#4661)
|
11 months ago |
