浏览代码

Return Correct Prompt Eval Count Regardless of Cache Prompt (#5371)

* openai compatibility

* Revert "openai compatibility"

This reverts commit d3f98a811e00fc497d889c8c45b0cfec5b64690c.

* remove erroneous subtraction of prompt cache
royjhan 10 月之前
父节点
当前提交
3b5a4a77f3
共有 1 个文件被更改,包括 1 次插入1 次删除
  1. 1 1
      llm/ext_server/server.cpp

+ 1 - 1
llm/ext_server/server.cpp

@@ -1732,7 +1732,7 @@ struct llama_server_context
                             slot.n_past -= 1;
                         }
 
-                        slot.n_prompt_tokens_processed = slot.n_prompt_tokens - slot.n_past;
+                        slot.n_prompt_tokens_processed = slot.n_prompt_tokens;
 
                         if (slot.ga_n != 1)
                         {