 Daniel Hiltgen
|
ccd7785859
Merge pull request #5243 from dhiltgen/modelfile_use_mmap
|
10 месяцев назад |
 Daniel Hiltgen
|
0e982bc1f4
Fix corner cases on tmp cleaner on mac
|
10 месяцев назад |
 Josh Yan
|
33a65e3ba3
error
|
10 месяцев назад |
|
Daniel Hiltgen
|
97c9e11768
Switch use_mmap to a pointer type
|
10 месяцев назад |
|
Daniel Hiltgen
|
3518aaef33
Merge pull request #4218 from dhiltgen/auto_parallel
|
10 месяцев назад |
 Blake Mizerany
|
cb42e607c5
llm: speed up gguf decoding by a lot (#5246)
|
11 месяцев назад |
|
Daniel Hiltgen
|
17b7186cd7
Enable concurrency by default
|
1 год назад |
|
Daniel Hiltgen
|
5bf5aeec01
Refine mmap default logic on linux
|
11 месяцев назад |
|
Daniel Hiltgen
|
96624aa412
Merge pull request #5072 from dhiltgen/windows_path
|
11 месяцев назад |
|
Daniel Hiltgen
|
7784ca33ce
Tighten up memory prediction logging
|
11 месяцев назад |
|
Daniel Hiltgen
|
171796791f
Adjust mmap logic for cuda windows for faster model load
|
11 месяцев назад |
|
Daniel Hiltgen
|
b2799f111b
Move libraries out of users path
|
11 месяцев назад |
|
Daniel Hiltgen
|
da3bf23354
Workaround gfx900 SDMA bugs
|
11 месяцев назад |
|
Daniel Hiltgen
|
6f351bf586
review comments and coverage
|
11 месяцев назад |
|
Daniel Hiltgen
|
fc37c192ae
Refine CPU load behavior with system memory visibility
|
11 месяцев назад |
|
Daniel Hiltgen
|
6fd04ca922
Improve multi-gpu handling at the limit
|
1 год назад |
 Craig Hughes
|
b84aea1685
Critical fix from llama.cpp JSON grammar to forbid un-escaped escape characters inside strings, which breaks parsing. (#3782)
|
11 месяцев назад |
 Michael Yang
|
e40145a39d
lint
|
1 год назад |
|
Michael Yang
|
c895a7d13f
some gocritic
|
1 год назад |
|
Michael Yang
|
829ff87bd1
revert tokenize ffi (#4761)
|
11 месяцев назад |
 Jeffrey Morgan
|
a50a87a7b8
partial offloading: allow flash attention and disable mmap (#4734)
|
11 месяцев назад |
|
Michael Yang
|
26a00a0410
use ffi for tokenizing/detokenizing
|
1 год назад |
|
Daniel Hiltgen
|
92c81e8117
Give the final model loading more time
|
11 месяцев назад |
 Lei Jitang
|
7487229c34
llm/server.go: Fix 2 minor typos (#4661)
|
1 год назад |
|
Daniel Hiltgen
|
0165ba1651
Merge pull request #4638 from dhiltgen/better_error
|
1 год назад |
|
Daniel Hiltgen
|
c4209d6d21
Report better warning on client closed abort of load
|
1 год назад |
 Patrick Devine
|
4cc3be3035
Move envconfig and consolidate env vars (#4608)
|
1 год назад |
|
Daniel Hiltgen
|
b37b496a12
Wire up load progress
|
1 год назад |
|
Jeffrey Morgan
|
38255d2af1
Use flash attention flag for now (#4580)
|
1 год назад |
 Sam
|
e15307fdf4
feat: add support for flash_attn (#4120)
|
1 год назад |
