فهرست منبع

Core fix for num_predict not working.

1) max_tokens was being looked for in openai_payload, but is present in openai_payload['options'], so is never found.
2) After copying the value for max_tokens to num_predict, delete max_tokens from the dictionary. This is to prevent Ollama throwing a warning about invalid option (max_tokens)
ferret99gt 2 ماه پیش
والد
کامیت
fea169a9c0
1فایلهای تغییر یافته به همراه5 افزوده شده و 4 حذف شده
  1. 5 4
      backend/open_webui/utils/payload.py

+ 5 - 4
backend/open_webui/utils/payload.py

@@ -178,10 +178,11 @@ def convert_payload_openai_to_ollama(openai_payload: dict) -> dict:
     if openai_payload.get("options"):
     if openai_payload.get("options"):
         ollama_payload["options"] = openai_payload["options"]
         ollama_payload["options"] = openai_payload["options"]
         ollama_options = openai_payload["options"]
         ollama_options = openai_payload["options"]
-
-    # Mapping OpenAI's `max_tokens` -> Ollama's `num_predict`
-    if "max_tokens" in openai_payload:
-        ollama_options["num_predict"] = openai_payload["max_tokens"]
+        
+        # Re-Mapping OpenAI's `max_tokens` -> Ollama's `num_predict`
+        if "max_tokens" in ollama_options:
+            ollama_options["num_predict"] = ollama_options["max_tokens"] 
+            del ollama_options["max_tokens"] # To prevent Ollama warning of invalid option provided        
 
 
     # Add options to payload if any have been set
     # Add options to payload if any have been set
     if ollama_options:
     if ollama_options: