Commit History

Author SHA1 Message Date
  Michael Yang 1d359e737e typo 11 months ago
  Michael Yang 50b9056e09 count memory up to NumGPU 11 months ago
  Jeffrey Morgan bb6fd02298 Don't clamp ctx size in `PredictServerFit` (#4317) 11 months ago
  Daniel Hiltgen bee2f4a3b0 Record GPU usage information 1 year ago
  Michael Yang 4736391bfb llm: add minimum based on layer size 1 year ago
  Daniel Hiltgen f56aa20014 Centralize server config handling 1 year ago
  Jeffrey Morgan f0c454ab57 gpu: add 512MiB to darwin minimum, metal doesn't have partial offloading overhead (#4068) 1 year ago
  Michael Yang f81f308118 fix gemma, command-r layer weights 1 year ago
  Michael Yang 7bb7cb8a60 only count output tensors 1 year ago
  Daniel Hiltgen 5445aaa94e Add back memory escape valve 1 year ago
  Daniel Hiltgen 34b9db5afc Request and model concurrency 1 year ago