Prechádzať zdrojové kódy

Return Correct Prompt Eval Count Regardless of Cache Prompt (#5371)

* openai compatibility

* Revert "openai compatibility"

This reverts commit d3f98a811e00fc497d889c8c45b0cfec5b64690c.

* remove erroneous subtraction of prompt cache
royjhan 10 mesiacov pred
rodič
commit
3b5a4a77f3
1 zmenil súbory, kde vykonal 1 pridanie a 1 odobranie
  1. 1 1
      llm/ext_server/server.cpp

+ 1 - 1
llm/ext_server/server.cpp

@@ -1732,7 +1732,7 @@ struct llama_server_context
                             slot.n_past -= 1;
                             slot.n_past -= 1;
                         }
                         }
 
 
-                        slot.n_prompt_tokens_processed = slot.n_prompt_tokens - slot.n_past;
+                        slot.n_prompt_tokens_processed = slot.n_prompt_tokens;
 
 
                         if (slot.ga_n != 1)
                         if (slot.ga_n != 1)
                         {
                         {