فهرست منبع

runner: Initialize numPredict

numPredict is used to enforce a limit on the number of tokens to
generate. Is it passed in from Ollama but it is never stored to
be checked.
Jesse Gross 8 ماه پیش
والد
کامیت
0c2f95f3de
1فایلهای تغییر یافته به همراه1 افزوده شده و 0 حذف شده
  1. 1 0
      llama/runner/runner.go

+ 1 - 0
llama/runner/runner.go

@@ -91,6 +91,7 @@ func (s *Server) NewSequence(prompt string, numPredict int, stop []string, param
 	return &Sequence{
 	return &Sequence{
 		tokens:          tokens,
 		tokens:          tokens,
 		n_prompt_tokens: len(tokens),
 		n_prompt_tokens: len(tokens),
+		numPredict:      numPredict,
 		responses:       make(chan string, 1),
 		responses:       make(chan string, 1),
 		embedding:       make(chan []float32, 1),
 		embedding:       make(chan []float32, 1),
 		samplingCtx:     sc,
 		samplingCtx:     sc,