فهرست منبع

runner.go: Fix off by one in batch size check

When adding tokens to a batch, the index is zero based but is
checked against being greater than the max batch size. This results
in an out-of-bounds access when the final token is added.
Jesse Gross 8 ماه پیش
والد
کامیت
5d34320b7c
1فایلهای تغییر یافته به همراه1 افزوده شده و 1 حذف شده
  1. 1 1
      llama/runner/runner.go

+ 1 - 1
llama/runner/runner.go

@@ -178,7 +178,7 @@ func (s *Server) run(ctx context.Context) {
 
 
 				for j, t := range seq.tokens {
 				for j, t := range seq.tokens {
 					// todo: make this n_batch
 					// todo: make this n_batch
-					if j > s.batchSize {
+					if j >= s.batchSize {
 						break
 						break
 					}
 					}
 					batch.Add(t, seq.nPast, []int{i}, !seq.prompt())
 					batch.Add(t, seq.nPast, []int{i}, !seq.prompt())