Commit History

Autor SHA1 Mensaxe Data
  Jeffrey Morgan 37708931fb update cuda overhead to 20% to fix crashes when switching between models and large context sizes hai 1 ano
  Jeffrey Morgan f6cb0a553c update cuda overhead to 15% or 400MiB hai 1 ano
  Jeffrey Morgan 2680078c13 fix build on linux hai 1 ano
  Jeffrey Morgan f1b7e5f560 update overhead to 15% hai 1 ano
  Jeffrey Morgan cb534e6ac2 use 10% vram overhead for cuda hai 1 ano
  Jeffrey Morgan 08f1e18965 Offload layers to GPU based on new model size estimates (#1850) hai 1 ano
  Daniel Hiltgen d74ce6bd4f Detect very old CUDA GPUs and fall back to CPU hai 1 ano
  Daniel Hiltgen a2ad952440 Fix windows system memory lookup hai 1 ano
  Daniel Hiltgen d966b730ac Switch windows build to fully dynamic hai 1 ano
  Daniel Hiltgen 7555ea44f8 Revamp the dynamic library shim hai 1 ano
  Daniel Hiltgen 1b991d0ba9 Refine build to support CPU only hai 1 ano
  Daniel Hiltgen 35934b2e05 Adapted rocm support to cgo based llama.cpp hai 1 ano