Bladeren bron

runner.go: Don't decode if nothing has been added to the batch

If nothing has been added to a batch then decoding will fail if
attempted. This can happen, for example, if the run loop is woken
up but we realize that we have the generation limit.
Jesse Gross 8 maanden geleden
bovenliggende
commit
5a441d227a
1 gewijzigde bestanden met toevoegingen van 4 en 0 verwijderingen
  1. 4 0
      llama/runner/runner.go

+ 4 - 0
llama/runner/runner.go

@@ -184,6 +184,10 @@ func (s *Server) run(ctx context.Context) {
 				seq.iBatch = batch.NumTokens() - 1
 				seq.iBatch = batch.NumTokens() - 1
 			}
 			}
 
 
+			if batch.NumTokens() == 0 {
+				continue
+			}
+
 			err := s.lc.Decode(batch)
 			err := s.lc.Decode(batch)
 			if err != nil {
 			if err != nil {
 				slog.Error("failed to decode batch", "error", err)
 				slog.Error("failed to decode batch", "error", err)