 Michael Yang
|
094df37563
remove unused struct
|
1 year ago |
 Bruce MacDonald
|
bd93a94abd
fix MB VRAM log output (#824)
|
1 year ago |
|
Michael Yang
|
f55bdb6f10
Merge pull request #799 from deichbewohner/jsonmarshaling
|
1 year ago |
|
Michael Yang
|
2870a9bfc8
Merge pull request #812 from jmorganca/mxyng/fix-format-string
|
1 year ago |
 Arne Müller
|
8fa3f366ad
Removed newline trimming and used buffer directly in POST request.
|
1 year ago |
|
Michael Yang
|
fddb303f23
fix: format string wrong type
|
1 year ago |
|
Michael Yang
|
cb4a80b693
fix: regression unsupported metal types
|
1 year ago |
 Arne Müller
|
ee94693b1a
handling unescaped json marshaling
|
1 year ago |
|
Michael Yang
|
11d82d7b9b
update checkvram
|
1 year ago |
|
Michael Yang
|
92189a5855
fix memory check
|
1 year ago |
|
Michael Yang
|
d790bf9916
Merge pull request #783 from jmorganca/mxyng/fix-gpu-offloading
|
1 year ago |
|
Michael Yang
|
35afac099a
do not use gpu binary when num_gpu == 0
|
1 year ago |
|
Michael Yang
|
811c3d1900
no gpu if vram < 2GB
|
1 year ago |
|
Bruce MacDonald
|
6fe178134d
improve api error handling (#781)
|
1 year ago |
|
Bruce MacDonald
|
56497663c8
relay model runner error message to client (#720)
|
1 year ago |
|
Michael Yang
|
b599946b74
add format bytes
|
1 year ago |
|
Bruce MacDonald
|
77295f716e
prevent waiting on exited command (#752)
|
1 year ago |
|
Bruce MacDonald
|
f2ba1311aa
improve vram safety with 5% vram memory buffer (#724)
|
1 year ago |
|
Bruce MacDonald
|
5d22319a2c
rename server subprocess (#700)
|
1 year ago |
|
Bruce MacDonald
|
9e2de1bd2c
increase streaming buffer size (#692)
|
1 year ago |
|
Michael Yang
|
c02c0cd483
starcoder
|
1 year ago |
|
Bruce MacDonald
|
b1f7123301
clean up num_gpu calculation code (#673)
|
1 year ago |
|
Bruce MacDonald
|
1fbf3585d6
Relay default values to llama runner (#672)
|
1 year ago |
|
Bruce MacDonald
|
9771b1ec51
windows runner fixes (#637)
|
1 year ago |
|
Michael Yang
|
f40b3de758
use int64 consistently
|
1 year ago |
|
Bruce MacDonald
|
86279f4ae3
unbound max num gpu layers (#591)
|
1 year ago |
|
Bruce MacDonald
|
4cba75efc5
remove tmp directories created by previous servers (#559)
|
1 year ago |
|
Bruce MacDonald
|
1255bc9b45
only package 11.8 runner
|
1 year ago |
|
Bruce MacDonald
|
4e8be787c7
pack in cuda libs
|
1 year ago |
|
Bruce MacDonald
|
66003e1d05
subprocess improvements (#524)
|
1 year ago |
