소스 검색

Core fix for num_predict not working.

1) max_tokens was being looked for in openai_payload, but is present in openai_payload['options'], so is never found.
2) After copying the value for max_tokens to num_predict, delete max_tokens from the dictionary. This is to prevent Ollama throwing a warning about invalid option (max_tokens)
ferret99gt 2 달 전
부모
커밋
fea169a9c0
1개의 변경된 파일5개의 추가작업 그리고 4개의 파일을 삭제
  1. 5 4
      backend/open_webui/utils/payload.py

+ 5 - 4
backend/open_webui/utils/payload.py

@@ -178,10 +178,11 @@ def convert_payload_openai_to_ollama(openai_payload: dict) -> dict:
     if openai_payload.get("options"):
         ollama_payload["options"] = openai_payload["options"]
         ollama_options = openai_payload["options"]
-
-    # Mapping OpenAI's `max_tokens` -> Ollama's `num_predict`
-    if "max_tokens" in openai_payload:
-        ollama_options["num_predict"] = openai_payload["max_tokens"]
+        
+        # Re-Mapping OpenAI's `max_tokens` -> Ollama's `num_predict`
+        if "max_tokens" in ollama_options:
+            ollama_options["num_predict"] = ollama_options["max_tokens"] 
+            del ollama_options["max_tokens"] # To prevent Ollama warning of invalid option provided        
 
     # Add options to payload if any have been set
     if ollama_options: