 Michael Yang
|
eaed6f8c45
add max context length check
|
hai 1 ano |
|
Michael Yang
|
2bb2bdd5d4
fix lint
|
hai 1 ano |
 Jeffrey Morgan
|
08f1e18965
Offload layers to GPU based on new model size estimates (#1850)
|
hai 1 ano |
 Bruce MacDonald
|
811b1f03c8
deprecate ggml
|
hai 1 ano |
|
Jeffrey Morgan
|
d9a250e9b5
seek to end of file when decoding older model formats
|
hai 1 ano |
|
Jeffrey Morgan
|
944519ed16
seek to eof for older model binaries
|
hai 1 ano |
|
Michael Yang
|
72e7a49aa9
seek instead of copyn
|
hai 1 ano |
|
Michael Yang
|
2cb0fa7d40
split from into one or more models
|
hai 1 ano |
|
Michael Yang
|
b2816bca67
unnecessary ReadSeeker for DecodeGGML
|
hai 1 ano |
|
Michael Yang
|
125d0a013a
ggufv3
|
hai 1 ano |
|
Michael Yang
|
c02c0cd483
starcoder
|
hai 1 ano |
|
Bruce MacDonald
|
86279f4ae3
unbound max num gpu layers (#591)
|
hai 1 ano |
|
Bruce MacDonald
|
4cba75efc5
remove tmp directories created by previous servers (#559)
|
hai 1 ano |
|
Bruce MacDonald
|
66003e1d05
subprocess improvements (#524)
|
hai 1 ano |
|
Bruce MacDonald
|
2540c9181c
support for packaging in multiple cuda runners (#509)
|
hai 1 ano |
|
Michael Yang
|
7dee25a07f
fix falcon decode
|
hai 1 ano |
|
Bruce MacDonald
|
09dd2aeff9
GGUF support (#441)
|
hai 1 ano |
|
Michael Yang
|
b1cececb8e
add 34b model type
|
hai 1 ano |
|
Michael Yang
|
a894cc792d
model and file type as strings
|
hai 1 ano |
|
Michael Yang
|
6ed991c8e2
ggml: fix off by one error
|
hai 1 ano |
|
Michael Yang
|
fccf8d179f
partial decode ggml bin for more info
|
hai 1 ano |
