Merge remote-tracking branch 'upstream/dev' into feat/oauth

CHANGELOG.md

@@ -5,6 +5,81 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.3.4] - 2024-06-12
+
+### Fixed
+
+- **🔒 Mixed Content with HTTPS Issue**: Resolved a problem where mixed content (HTTP and HTTPS) was causing security warnings and blocking resources on HTTPS sites.
+- **🔍 Web Search Issue**: Addressed the problem where web search functionality was not working correctly. The 'ENABLE_RAG_LOCAL_WEB_FETCH' option has been reintroduced to restore proper web searching capabilities.
+- **💾 RAG Template Not Being Saved**: Fixed an issue where the RAG template was not being saved correctly, ensuring your custom templates are now preserved as expected.
+
+## [0.3.3] - 2024-06-12
+
+### Added
+
+- **🛠️ Native Python Function Calling**: Introducing native Python function calling within Open WebUI. We’ve also included a built-in code editor to seamlessly develop and integrate function code within the 'Tools' workspace. With this, you can significantly enhance your LLM’s capabilities by creating custom RAG pipelines, web search tools, and even agent-like features such as sending Discord messages.
+- **🌐 DuckDuckGo Integration**: Added DuckDuckGo as a web search provider, giving you more search options.
+- **🌏 Enhanced Translations**: Improved translations for Vietnamese and Chinese languages, making the interface more accessible.
+
+### Fixed
+
+- **🔗 Web Search URL Error Handling**: Fixed the issue where a single URL error would disrupt the data loading process in Web Search mode. Now, such errors will be handled gracefully to ensure uninterrupted data loading.
+- **🖥️ Frontend Responsiveness**: Resolved the problem where the frontend would stop responding if the backend encounters an error while downloading a model. Improved error handling to maintain frontend stability.
+- **🔧 Dependency Issues in pip**: Fixed issues related to pip installations, ensuring all dependencies are correctly managed to prevent installation errors.
+
+## [0.3.2] - 2024-06-10
+
+### Added
+
+- **🔍 Web Search Query Status**: The web search query will now persist in the results section to aid in easier debugging and tracking of search queries.
+- **🌐 New Web Search Provider**: We have added Serply as a new option for web search providers, giving you more choices for your search needs.
+- **🌏 Improved Translations**: We've enhanced translations for Chinese and Portuguese.
+
+### Fixed
+
+- **🎤 Audio File Upload Issue**: The bug that prevented audio files from being uploaded in chat input has been fixed, ensuring smooth communication.
+- **💬 Message Input Handling**: Improved the handling of message inputs by instantly clearing images and text after sending, along with immediate visual indications when a response message is loading, enhancing user feedback.
+- **⚙️ Parameter Registration and Validation**: Fixed the issue where parameters were not registering in certain cases and addressed the problem where users were unable to save due to invalid input errors.
+
+## [0.3.1] - 2024-06-09
+
+### Fixed
+
+- **💬 Chat Functionality**: Resolved the issue where chat functionality was not working for specific models.
+
+## [0.3.0] - 2024-06-09
+
+### Added
+
+- **📚 Knowledge Support for Models**: Attach documents directly to models from the models workspace, enhancing the information available to each model.
+- **🎙️ Hands-Free Voice Call Feature**: Initiate voice calls without needing to use your hands, making interactions more seamless.
+- **📹 Video Call Feature**: Enable video calls with supported vision models like Llava and GPT-4o, adding a visual dimension to your communications.
+- **🎛️ Enhanced UI for Voice Recording**: Improved user interface for the voice recording feature, making it more intuitive and user-friendly.
+- **🌐 External STT Support**: Now support for external Speech-To-Text services, providing more flexibility in choosing your STT provider.
+- **⚙️ Unified Settings**: Consolidated settings including document settings under a new admin settings section for easier management.
+- **🌑 Dark Mode Splash Screen**: A new splash screen for dark mode, ensuring a consistent and visually appealing experience for dark mode users.
+- **📥 Upload Pipeline**: Directly upload pipelines from the admin settings > pipelines section, streamlining the pipeline management process.
+- **🌍 Improved Language Support**: Enhanced support for Chinese and Ukrainian languages, better catering to a global user base.
+
+### Fixed
+
+- **🛠️ Playground Issue**: Fixed the playground not functioning properly, ensuring a smoother user experience.
+- **🔥 Temperature Parameter Issue**: Corrected the issue where the temperature value '0' was not being passed correctly.
+- **📝 Prompt Input Clearing**: Resolved prompt input textarea not being cleared right away, ensuring a clean slate for new inputs.
+- **✨ Various UI Styling Issues**: Fixed numerous user interface styling problems for a more cohesive look.
+- **👥 Active Users Display**: Fixed active users showing active sessions instead of actual users, now reflecting accurate user activity.
+- **🌐 Community Platform Compatibility**: The Community Platform is back online and fully compatible with Open WebUI.
+
+### Changed
+
+- **📝 RAG Implementation**: Updated the RAG (Retrieval-Augmented Generation) implementation to use a system prompt for context, instead of overriding the user's prompt.
+- **🔄 Settings Relocation**: Moved Models, Connections, Audio, and Images settings to the admin settings for better organization.
+- **✍️ Improved Title Generation**: Enhanced the default prompt for title generation, yielding better results.
+- **🔧 Backend Task Management**: Tasks like title generation and search query generation are now managed on the backend side and controlled only by the admin.
+- **🔍 Editable Search Query Prompt**: You can now edit the search query generation prompt, offering more control over how queries are generated.
+- **📏 Prompt Length Threshold**: Set the prompt length threshold for search query generation from the admin settings, giving more customization options.
+- **📣 Settings Consolidation**: Merged the Banners admin setting with the Interface admin setting for a more streamlined settings area.
+
 ## [0.2.5] - 2024-06-05
 
 ### Added

README.md

@@ -29,11 +29,15 @@ Open WebUI is an [extensible](https://github.com/open-webui/pipelines), feature-
 
 - ✒️🔢 **Full Markdown and LaTeX Support**: Elevate your LLM experience with comprehensive Markdown and LaTeX capabilities for enriched interaction.
 
+- 🎤📹 **Hands-Free Voice/Video Call**: Experience seamless communication with integrated hands-free voice and video call features, allowing for a more dynamic and interactive chat environment.
+
 - 🛠️ **Model Builder**: Easily create Ollama models via the Web UI. Create and add custom characters/agents, customize chat elements, and import models effortlessly through [Open WebUI Community](https://openwebui.com/) integration.
 
+- 🐍 **Native Python Function Calling Tool**: Enhance your LLMs with built-in code editor support in the tools workspace. Bring Your Own Function (BYOF) by simply adding your pure Python functions, enabling seamless integration with LLMs.
+
 - 📚 **Local RAG Integration**: Dive into the future of chat interactions with groundbreaking Retrieval Augmented Generation (RAG) support. This feature seamlessly integrates document interactions into your chat experience. You can load documents directly into the chat or add files to your document library, effortlessly accessing them using the `#` command before a query.
 
-- 🔍 **Web Search for RAG**: Perform web searches using providers like `SearXNG`, `Google PSE`, `Brave Search`, `serpstack`, and `serper`, and inject the results directly into your chat experience.
+- 🔍 **Web Search for RAG**: Perform web searches using providers like `SearXNG`, `Google PSE`, `Brave Search`, `serpstack`, `serper`, `Serply`, `DuckDuckGo` and `TavilySearch` and inject the results directly into your chat experience.
 
 - 🌐 **Web Browsing Capability**: Seamlessly integrate websites into your chat experience using the `#` command followed by a URL. This feature allows you to incorporate web content directly into your conversations, enhancing the richness and depth of your interactions.
 
@@ -146,10 +150,19 @@ docker run --rm --volume /var/run/docker.sock:/var/run/docker.sock containrrr/wa
 
 In the last part of the command, replace `open-webui` with your container name if it is different.
 
-### Moving from Ollama WebUI to Open WebUI
-
 Check our Migration Guide available in our [Open WebUI Documentation](https://docs.openwebui.com/migration/).
 
+### Using the Dev Branch 🌙
+
+> [!WARNING]
+> The `:dev` branch contains the latest unstable features and changes. Use it at your own risk as it may have bugs or incomplete features.
+
+If you want to try out the latest bleeding-edge features and are okay with occasional instability, you can use the `:dev` tag like this:
+
+```bash
+docker run -d -p 3000:8080 -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:dev
+```
+
 ## What's Next? 🌟
 
 Discover upcoming features on our roadmap in the [Open WebUI Documentation](https://docs.openwebui.com/roadmap/).

TROUBLESHOOTING.md

@@ -18,6 +18,10 @@ If you're experiencing connection issues, it’s often due to the WebUI docker c
 docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main
 ```
 
+### Error on Slow Reponses for Ollama
+
+Open WebUI has a default timeout of 5 minutes for Ollama to finish generating the response. If needed, this can be adjusted via the environment variable AIOHTTP_CLIENT_TIMEOUT, which sets the timeout in seconds.
+
 ### General Connection Errors
 
 **Ensure Ollama Version is Up-to-Date**: Always start by checking that you have the latest version of Ollama. Visit [Ollama's official site](https://ollama.com/) for the latest updates.

backend/apps/audio/main.py

@@ -17,13 +17,12 @@ from fastapi.middleware.cors import CORSMiddleware
 from faster_whisper import WhisperModel
 from pydantic import BaseModel
 
-
+import uuid
 import requests
 import hashlib
 from pathlib import Path
 import json
 
-
 from constants import ERROR_MESSAGES
 from utils.utils import (
     decode_token,
@@ -41,10 +40,15 @@ from config import (
     WHISPER_MODEL_DIR,
     WHISPER_MODEL_AUTO_UPDATE,
     DEVICE_TYPE,
-    AUDIO_OPENAI_API_BASE_URL,
-    AUDIO_OPENAI_API_KEY,
-    AUDIO_OPENAI_API_MODEL,
-    AUDIO_OPENAI_API_VOICE,
+    AUDIO_STT_OPENAI_API_BASE_URL,
+    AUDIO_STT_OPENAI_API_KEY,
+    AUDIO_TTS_OPENAI_API_BASE_URL,
+    AUDIO_TTS_OPENAI_API_KEY,
+    AUDIO_STT_ENGINE,
+    AUDIO_STT_MODEL,
+    AUDIO_TTS_ENGINE,
+    AUDIO_TTS_MODEL,
+    AUDIO_TTS_VOICE,
     AppConfig,
 )
 
@@ -61,10 +65,17 @@ app.add_middleware(
 )
 
 app.state.config = AppConfig()
-app.state.config.OPENAI_API_BASE_URL = AUDIO_OPENAI_API_BASE_URL
-app.state.config.OPENAI_API_KEY = AUDIO_OPENAI_API_KEY
-app.state.config.OPENAI_API_MODEL = AUDIO_OPENAI_API_MODEL
-app.state.config.OPENAI_API_VOICE = AUDIO_OPENAI_API_VOICE
+
+app.state.config.STT_OPENAI_API_BASE_URL = AUDIO_STT_OPENAI_API_BASE_URL
+app.state.config.STT_OPENAI_API_KEY = AUDIO_STT_OPENAI_API_KEY
+app.state.config.STT_ENGINE = AUDIO_STT_ENGINE
+app.state.config.STT_MODEL = AUDIO_STT_MODEL
+
+app.state.config.TTS_OPENAI_API_BASE_URL = AUDIO_TTS_OPENAI_API_BASE_URL
+app.state.config.TTS_OPENAI_API_KEY = AUDIO_TTS_OPENAI_API_KEY
+app.state.config.TTS_ENGINE = AUDIO_TTS_ENGINE
+app.state.config.TTS_MODEL = AUDIO_TTS_MODEL
+app.state.config.TTS_VOICE = AUDIO_TTS_VOICE
 
 # setting device type for whisper model
 whisper_device_type = DEVICE_TYPE if DEVICE_TYPE and DEVICE_TYPE == "cuda" else "cpu"
@@ -74,41 +85,101 @@ SPEECH_CACHE_DIR = Path(CACHE_DIR).joinpath("./audio/speech/")
 SPEECH_CACHE_DIR.mkdir(parents=True, exist_ok=True)
 
 
-class OpenAIConfigUpdateForm(BaseModel):
-    url: str
-    key: str
-    model: str
-    speaker: str
+class TTSConfigForm(BaseModel):
+    OPENAI_API_BASE_URL: str
+    OPENAI_API_KEY: str
+    ENGINE: str
+    MODEL: str
+    VOICE: str
+
+
+class STTConfigForm(BaseModel):
+    OPENAI_API_BASE_URL: str
+    OPENAI_API_KEY: str
+    ENGINE: str
+    MODEL: str
+
+
+class AudioConfigUpdateForm(BaseModel):
+    tts: TTSConfigForm
+    stt: STTConfigForm
+
+
+from pydub import AudioSegment
+from pydub.utils import mediainfo
+
+
+def is_mp4_audio(file_path):
+    """Check if the given file is an MP4 audio file."""
+    if not os.path.isfile(file_path):
+        print(f"File not found: {file_path}")
+        return False
+
+    info = mediainfo(file_path)
+    if (
+        info.get("codec_name") == "aac"
+        and info.get("codec_type") == "audio"
+        and info.get("codec_tag_string") == "mp4a"
+    ):
+        return True
+    return False
+
+
+def convert_mp4_to_wav(file_path, output_path):
+    """Convert MP4 audio file to WAV format."""
+    audio = AudioSegment.from_file(file_path, format="mp4")
+    audio.export(output_path, format="wav")
+    print(f"Converted {file_path} to {output_path}")
 
 
 @app.get("/config")
-async def get_openai_config(user=Depends(get_admin_user)):
+async def get_audio_config(user=Depends(get_admin_user)):
     return {
-        "OPENAI_API_BASE_URL": app.state.config.OPENAI_API_BASE_URL,
-        "OPENAI_API_KEY": app.state.config.OPENAI_API_KEY,
-        "OPENAI_API_MODEL": app.state.config.OPENAI_API_MODEL,
-        "OPENAI_API_VOICE": app.state.config.OPENAI_API_VOICE,
+        "tts": {
+            "OPENAI_API_BASE_URL": app.state.config.TTS_OPENAI_API_BASE_URL,
+            "OPENAI_API_KEY": app.state.config.TTS_OPENAI_API_KEY,
+            "ENGINE": app.state.config.TTS_ENGINE,
+            "MODEL": app.state.config.TTS_MODEL,
+            "VOICE": app.state.config.TTS_VOICE,
+        },
+        "stt": {
+            "OPENAI_API_BASE_URL": app.state.config.STT_OPENAI_API_BASE_URL,
+            "OPENAI_API_KEY": app.state.config.STT_OPENAI_API_KEY,
+            "ENGINE": app.state.config.STT_ENGINE,
+            "MODEL": app.state.config.STT_MODEL,
+        },
     }
 
 
 @app.post("/config/update")
-async def update_openai_config(
-    form_data: OpenAIConfigUpdateForm, user=Depends(get_admin_user)
+async def update_audio_config(
+    form_data: AudioConfigUpdateForm, user=Depends(get_admin_user)
 ):
-    if form_data.key == "":
-        raise HTTPException(status_code=400, detail=ERROR_MESSAGES.API_KEY_NOT_FOUND)
+    app.state.config.TTS_OPENAI_API_BASE_URL = form_data.tts.OPENAI_API_BASE_URL
+    app.state.config.TTS_OPENAI_API_KEY = form_data.tts.OPENAI_API_KEY
+    app.state.config.TTS_ENGINE = form_data.tts.ENGINE
+    app.state.config.TTS_MODEL = form_data.tts.MODEL
+    app.state.config.TTS_VOICE = form_data.tts.VOICE
 
-    app.state.config.OPENAI_API_BASE_URL = form_data.url
-    app.state.config.OPENAI_API_KEY = form_data.key
-    app.state.config.OPENAI_API_MODEL = form_data.model
-    app.state.config.OPENAI_API_VOICE = form_data.speaker
+    app.state.config.STT_OPENAI_API_BASE_URL = form_data.stt.OPENAI_API_BASE_URL
+    app.state.config.STT_OPENAI_API_KEY = form_data.stt.OPENAI_API_KEY
+    app.state.config.STT_ENGINE = form_data.stt.ENGINE
+    app.state.config.STT_MODEL = form_data.stt.MODEL
 
     return {
-        "status": True,
-        "OPENAI_API_BASE_URL": app.state.config.OPENAI_API_BASE_URL,
-        "OPENAI_API_KEY": app.state.config.OPENAI_API_KEY,
-        "OPENAI_API_MODEL": app.state.config.OPENAI_API_MODEL,
-        "OPENAI_API_VOICE": app.state.config.OPENAI_API_VOICE,
+        "tts": {
+            "OPENAI_API_BASE_URL": app.state.config.TTS_OPENAI_API_BASE_URL,
+            "OPENAI_API_KEY": app.state.config.TTS_OPENAI_API_KEY,
+            "ENGINE": app.state.config.TTS_ENGINE,
+            "MODEL": app.state.config.TTS_MODEL,
+            "VOICE": app.state.config.TTS_VOICE,
+        },
+        "stt": {
+            "OPENAI_API_BASE_URL": app.state.config.STT_OPENAI_API_BASE_URL,
+            "OPENAI_API_KEY": app.state.config.STT_OPENAI_API_KEY,
+            "ENGINE": app.state.config.STT_ENGINE,
+            "MODEL": app.state.config.STT_MODEL,
+        },
     }
 
 
@@ -125,13 +196,21 @@ async def speech(request: Request, user=Depends(get_verified_user)):
         return FileResponse(file_path)
 
     headers = {}
-    headers["Authorization"] = f"Bearer {app.state.config.OPENAI_API_KEY}"
+    headers["Authorization"] = f"Bearer {app.state.config.TTS_OPENAI_API_KEY}"
     headers["Content-Type"] = "application/json"
 
+    try:
+        body = body.decode("utf-8")
+        body = json.loads(body)
+        body["model"] = app.state.config.TTS_MODEL
+        body = json.dumps(body).encode("utf-8")
+    except Exception as e:
+        pass
+
     r = None
     try:
         r = requests.post(
-            url=f"{app.state.config.OPENAI_API_BASE_URL}/audio/speech",
+            url=f"{app.state.config.TTS_OPENAI_API_BASE_URL}/audio/speech",
             data=body,
             headers=headers,
             stream=True,
@@ -181,41 +260,110 @@ def transcribe(
         )
 
     try:
-        filename = file.filename
-        file_path = f"{UPLOAD_DIR}/{filename}"
+        ext = file.filename.split(".")[-1]
+
+        id = uuid.uuid4()
+        filename = f"{id}.{ext}"
+
+        file_dir = f"{CACHE_DIR}/audio/transcriptions"
+        os.makedirs(file_dir, exist_ok=True)
+        file_path = f"{file_dir}/{filename}"
+
+        print(filename)
+
         contents = file.file.read()
         with open(file_path, "wb") as f:
             f.write(contents)
             f.close()
 
-        whisper_kwargs = {
-            "model_size_or_path": WHISPER_MODEL,
-            "device": whisper_device_type,
-            "compute_type": "int8",
-            "download_root": WHISPER_MODEL_DIR,
-            "local_files_only": not WHISPER_MODEL_AUTO_UPDATE,
-        }
-
-        log.debug(f"whisper_kwargs: {whisper_kwargs}")
-
-        try:
-            model = WhisperModel(**whisper_kwargs)
-        except:
-            log.warning(
-                "WhisperModel initialization failed, attempting download with local_files_only=False"
+        if app.state.config.STT_ENGINE == "":
+            whisper_kwargs = {
+                "model_size_or_path": WHISPER_MODEL,
+                "device": whisper_device_type,
+                "compute_type": "int8",
+                "download_root": WHISPER_MODEL_DIR,
+                "local_files_only": not WHISPER_MODEL_AUTO_UPDATE,
+            }
+
+            log.debug(f"whisper_kwargs: {whisper_kwargs}")
+
+            try:
+                model = WhisperModel(**whisper_kwargs)
+            except:
+                log.warning(
+                    "WhisperModel initialization failed, attempting download with local_files_only=False"
+                )
+                whisper_kwargs["local_files_only"] = False
+                model = WhisperModel(**whisper_kwargs)
+
+            segments, info = model.transcribe(file_path, beam_size=5)
+            log.info(
+                "Detected language '%s' with probability %f"
+                % (info.language, info.language_probability)
             )
-            whisper_kwargs["local_files_only"] = False
-            model = WhisperModel(**whisper_kwargs)
 
-        segments, info = model.transcribe(file_path, beam_size=5)
-        log.info(
-            "Detected language '%s' with probability %f"
-            % (info.language, info.language_probability)
-        )
+            transcript = "".join([segment.text for segment in list(segments)])
 
-        transcript = "".join([segment.text for segment in list(segments)])
+            data = {"text": transcript.strip()}
 
-        return {"text": transcript.strip()}
+            # save the transcript to a json file
+            transcript_file = f"{file_dir}/{id}.json"
+            with open(transcript_file, "w") as f:
+                json.dump(data, f)
+
+            print(data)
+
+            return data
+
+        elif app.state.config.STT_ENGINE == "openai":
+            if is_mp4_audio(file_path):
+                print("is_mp4_audio")
+                os.rename(file_path, file_path.replace(".wav", ".mp4"))
+                # Convert MP4 audio file to WAV format
+                convert_mp4_to_wav(file_path.replace(".wav", ".mp4"), file_path)
+
+            headers = {"Authorization": f"Bearer {app.state.config.STT_OPENAI_API_KEY}"}
+
+            files = {"file": (filename, open(file_path, "rb"))}
+            data = {"model": "whisper-1"}
+
+            print(files, data)
+
+            r = None
+            try:
+                r = requests.post(
+                    url=f"{app.state.config.STT_OPENAI_API_BASE_URL}/audio/transcriptions",
+                    headers=headers,
+                    files=files,
+                    data=data,
+                )
+
+                r.raise_for_status()
+
+                data = r.json()
+
+                # save the transcript to a json file
+                transcript_file = f"{file_dir}/{id}.json"
+                with open(transcript_file, "w") as f:
+                    json.dump(data, f)
+
+                print(data)
+                return data
+            except Exception as e:
+                log.exception(e)
+                error_detail = "Open WebUI: Server Connection Error"
+                if r is not None:
+                    try:
+                        res = r.json()
+                        if "error" in res:
+                            error_detail = f"External: {res['error']['message']}"
+                    except:
+                        error_detail = f"External: {e}"
+
+                raise HTTPException(
+                    status_code=r.status_code if r != None else 500,
+                    detail=error_detail,
+                )
 
     except Exception as e:
         log.exception(e)

backend/apps/ollama/main.py

@@ -41,13 +41,12 @@ from utils.utils import (
     get_admin_user,
 )
 
-from utils.models import get_model_id_from_custom_model_id
-
 
 from config import (
     SRC_LOG_LEVELS,
     OLLAMA_BASE_URLS,
     ENABLE_OLLAMA_API,
+    AIOHTTP_CLIENT_TIMEOUT,
     ENABLE_MODEL_FILTER,
     MODEL_FILTER_LIST,
     UPLOAD_DIR,
@@ -156,7 +155,9 @@ async def cleanup_response(
 async def post_streaming_url(url: str, payload: str):
     r = None
     try:
-        session = aiohttp.ClientSession(trust_env=True)
+        session = aiohttp.ClientSession(
+            trust_env=True, timeout=aiohttp.ClientTimeout(total=AIOHTTP_CLIENT_TIMEOUT)
+        )
         r = await session.post(url, data=payload)
         r.raise_for_status()
 
@@ -728,7 +729,6 @@ async def generate_chat_completion(
     model_info = Models.get_model_by_id(model_id)
 
     if model_info:
-        print(model_info)
         if model_info.base_model_id:
             payload["model"] = model_info.base_model_id
 
@@ -754,6 +754,14 @@ async def generate_chat_completion(
             if model_info.params.get("num_ctx", None):
                 payload["options"]["num_ctx"] = model_info.params.get("num_ctx", None)
 
+            if model_info.params.get("num_batch", None):
+                payload["options"]["num_batch"] = model_info.params.get(
+                    "num_batch", None
+                )
+
+            if model_info.params.get("num_keep", None):
+                payload["options"]["num_keep"] = model_info.params.get("num_keep", None)
+
             if model_info.params.get("repeat_last_n", None):
                 payload["options"]["repeat_last_n"] = model_info.params.get(
                     "repeat_last_n", None
@@ -764,7 +772,7 @@ async def generate_chat_completion(
                     "frequency_penalty", None
                 )
 
-            if model_info.params.get("temperature", None):
+            if model_info.params.get("temperature", None) is not None:
                 payload["options"]["temperature"] = model_info.params.get(
                     "temperature", None
                 )
@@ -849,9 +857,14 @@ async def generate_chat_completion(
 
 
 # TODO: we should update this part once Ollama supports other types
+class OpenAIChatMessageContent(BaseModel):
+    type: str
+    model_config = ConfigDict(extra="allow")
+
+
 class OpenAIChatMessage(BaseModel):
     role: str
-    content: str
+    content: Union[str, OpenAIChatMessageContent]
 
     model_config = ConfigDict(extra="allow")
 
@@ -879,7 +892,6 @@ async def generate_openai_chat_completion(
     model_info = Models.get_model_by_id(model_id)
 
     if model_info:
-        print(model_info)
         if model_info.base_model_id:
             payload["model"] = model_info.base_model_id
 

backend/apps/openai/main.py

@@ -345,108 +345,155 @@ async def get_models(url_idx: Optional[int] = None, user=Depends(get_current_use
             )
 
 
-@app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE"])
-async def proxy(path: str, request: Request, user=Depends(get_verified_user)):
+@app.post("/chat/completions")
+@app.post("/chat/completions/{url_idx}")
+async def generate_chat_completion(
+    form_data: dict,
+    url_idx: Optional[int] = None,
+    user=Depends(get_verified_user),
+):
     idx = 0
+    payload = {**form_data}
 
-    body = await request.body()
-    # TODO: Remove below after gpt-4-vision fix from Open AI
-    # Try to decode the body of the request from bytes to a UTF-8 string (Require add max_token to fix gpt-4-vision)
+    model_id = form_data.get("model")
+    model_info = Models.get_model_by_id(model_id)
 
-    payload = None
+    if model_info:
+        if model_info.base_model_id:
+            payload["model"] = model_info.base_model_id
 
-    try:
-        if "chat/completions" in path:
-            body = body.decode("utf-8")
-            body = json.loads(body)
+        model_info.params = model_info.params.model_dump()
 
-            payload = {**body}
+        if model_info.params:
+            if model_info.params.get("temperature", None) is not None:
+                payload["temperature"] = float(model_info.params.get("temperature"))
 
-            model_id = body.get("model")
-            model_info = Models.get_model_by_id(model_id)
+            if model_info.params.get("top_p", None):
+                payload["top_p"] = int(model_info.params.get("top_p", None))
 
-            if model_info:
-                print(model_info)
-                if model_info.base_model_id:
-                    payload["model"] = model_info.base_model_id
+            if model_info.params.get("max_tokens", None):
+                payload["max_tokens"] = int(model_info.params.get("max_tokens", None))
 
-                model_info.params = model_info.params.model_dump()
+            if model_info.params.get("frequency_penalty", None):
+                payload["frequency_penalty"] = int(
+                    model_info.params.get("frequency_penalty", None)
+                )
+
+            if model_info.params.get("seed", None):
+                payload["seed"] = model_info.params.get("seed", None)
+
+            if model_info.params.get("stop", None):
+                payload["stop"] = (
+                    [
+                        bytes(stop, "utf-8").decode("unicode_escape")
+                        for stop in model_info.params["stop"]
+                    ]
+                    if model_info.params.get("stop", None)
+                    else None
+                )
 
-                if model_info.params:
-                    if model_info.params.get("temperature", None):
-                        payload["temperature"] = int(
-                            model_info.params.get("temperature")
+        if model_info.params.get("system", None):
+            # Check if the payload already has a system message
+            # If not, add a system message to the payload
+            if payload.get("messages"):
+                for message in payload["messages"]:
+                    if message.get("role") == "system":
+                        message["content"] = (
+                            model_info.params.get("system", None) + message["content"]
                         )
+                        break
+                else:
+                    payload["messages"].insert(
+                        0,
+                        {
+                            "role": "system",
+                            "content": model_info.params.get("system", None),
+                        },
+                    )
 
-                    if model_info.params.get("top_p", None):
-                        payload["top_p"] = int(model_info.params.get("top_p", None))
+    else:
+        pass
 
-                    if model_info.params.get("max_tokens", None):
-                        payload["max_tokens"] = int(
-                            model_info.params.get("max_tokens", None)
-                        )
+    model = app.state.MODELS[payload.get("model")]
+    idx = model["urlIdx"]
 
-                    if model_info.params.get("frequency_penalty", None):
-                        payload["frequency_penalty"] = int(
-                            model_info.params.get("frequency_penalty", None)
-                        )
+    if "pipeline" in model and model.get("pipeline"):
+        payload["user"] = {"name": user.name, "id": user.id}
 
-                    if model_info.params.get("seed", None):
-                        payload["seed"] = model_info.params.get("seed", None)
-
-                    if model_info.params.get("stop", None):
-                        payload["stop"] = (
-                            [
-                                bytes(stop, "utf-8").decode("unicode_escape")
-                                for stop in model_info.params["stop"]
-                            ]
-                            if model_info.params.get("stop", None)
-                            else None
-                        )
+    # Check if the model is "gpt-4-vision-preview" and set "max_tokens" to 4000
+    # This is a workaround until OpenAI fixes the issue with this model
+    if payload.get("model") == "gpt-4-vision-preview":
+        if "max_tokens" not in payload:
+            payload["max_tokens"] = 4000
+        log.debug("Modified payload:", payload)
 
-                if model_info.params.get("system", None):
-                    # Check if the payload already has a system message
-                    # If not, add a system message to the payload
-                    if payload.get("messages"):
-                        for message in payload["messages"]:
-                            if message.get("role") == "system":
-                                message["content"] = (
-                                    model_info.params.get("system", None)
-                                    + message["content"]
-                                )
-                                break
-                        else:
-                            payload["messages"].insert(
-                                0,
-                                {
-                                    "role": "system",
-                                    "content": model_info.params.get("system", None),
-                                },
-                            )
-            else:
-                pass
+    # Convert the modified body back to JSON
+    payload = json.dumps(payload)
+
+    print(payload)
 
-            model = app.state.MODELS[payload.get("model")]
+    url = app.state.config.OPENAI_API_BASE_URLS[idx]
+    key = app.state.config.OPENAI_API_KEYS[idx]
 
-            idx = model["urlIdx"]
+    print(payload)
 
-            if "pipeline" in model and model.get("pipeline"):
-                payload["user"] = {"name": user.name, "id": user.id}
+    headers = {}
+    headers["Authorization"] = f"Bearer {key}"
+    headers["Content-Type"] = "application/json"
 
-            # Check if the model is "gpt-4-vision-preview" and set "max_tokens" to 4000
-            # This is a workaround until OpenAI fixes the issue with this model
-            if payload.get("model") == "gpt-4-vision-preview":
-                if "max_tokens" not in payload:
-                    payload["max_tokens"] = 4000
-                log.debug("Modified payload:", payload)
+    r = None
+    session = None
+    streaming = False
 
-            # Convert the modified body back to JSON
-            payload = json.dumps(payload)
+    try:
+        session = aiohttp.ClientSession(trust_env=True)
+        r = await session.request(
+            method="POST",
+            url=f"{url}/chat/completions",
+            data=payload,
+            headers=headers,
+        )
 
-    except json.JSONDecodeError as e:
-        log.error("Error loading request body into a dictionary:", e)
+        r.raise_for_status()
 
-    print(payload)
+        # Check if response is SSE
+        if "text/event-stream" in r.headers.get("Content-Type", ""):
+            streaming = True
+            return StreamingResponse(
+                r.content,
+                status_code=r.status,
+                headers=dict(r.headers),
+                background=BackgroundTask(
+                    cleanup_response, response=r, session=session
+                ),
+            )
+        else:
+            response_data = await r.json()
+            return response_data
+    except Exception as e:
+        log.exception(e)
+        error_detail = "Open WebUI: Server Connection Error"
+        if r is not None:
+            try:
+                res = await r.json()
+                print(res)
+                if "error" in res:
+                    error_detail = f"External: {res['error']['message'] if 'message' in res['error'] else res['error']}"
+            except:
+                error_detail = f"External: {e}"
+        raise HTTPException(status_code=r.status if r else 500, detail=error_detail)
+    finally:
+        if not streaming and session:
+            if r:
+                r.close()
+            await session.close()
+
+
+@app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE"])
+async def proxy(path: str, request: Request, user=Depends(get_verified_user)):
+    idx = 0
+
+    body = await request.body()
 
     url = app.state.config.OPENAI_API_BASE_URLS[idx]
     key = app.state.config.OPENAI_API_KEYS[idx]
@@ -466,7 +513,7 @@ async def proxy(path: str, request: Request, user=Depends(get_verified_user)):
         r = await session.request(
             method=request.method,
             url=target_url,
-            data=payload if payload else body,
+            data=body,
             headers=headers,
         )
 

backend/apps/rag/main.py

@@ -8,12 +8,15 @@ from fastapi import (
     Form,
 )
 from fastapi.middleware.cors import CORSMiddleware
+import requests
 import os, shutil, logging, re
+from datetime import datetime
 
 from pathlib import Path
-from typing import List, Union, Sequence
+from typing import List, Union, Sequence, Iterator, Any
 
 from chromadb.utils.batch_utils import create_batches
+from langchain_core.documents import Document
 
 from langchain_community.document_loaders import (
     WebBaseLoader,
@@ -30,6 +33,7 @@ from langchain_community.document_loaders import (
     UnstructuredExcelLoader,
     UnstructuredPowerPointLoader,
     YoutubeLoader,
+    OutlookMessageLoader,
 )
 from langchain.text_splitter import RecursiveCharacterTextSplitter
 
@@ -67,7 +71,9 @@ from apps.rag.search.main import SearchResult
 from apps.rag.search.searxng import search_searxng
 from apps.rag.search.serper import search_serper
 from apps.rag.search.serpstack import search_serpstack
-
+from apps.rag.search.serply import search_serply
+from apps.rag.search.duckduckgo import search_duckduckgo
+from apps.rag.search.tavily import search_tavily
 
 from utils.misc import (
     calculate_sha256,
@@ -113,6 +119,8 @@ from config import (
     SERPSTACK_API_KEY,
     SERPSTACK_HTTPS,
     SERPER_API_KEY,
+    SERPLY_API_KEY,
+    TAVILY_API_KEY,
     RAG_WEB_SEARCH_RESULT_COUNT,
     RAG_WEB_SEARCH_CONCURRENT_REQUESTS,
     RAG_EMBEDDING_OPENAI_BATCH_SIZE,
@@ -165,6 +173,8 @@ app.state.config.BRAVE_SEARCH_API_KEY = BRAVE_SEARCH_API_KEY
 app.state.config.SERPSTACK_API_KEY = SERPSTACK_API_KEY
 app.state.config.SERPSTACK_HTTPS = SERPSTACK_HTTPS
 app.state.config.SERPER_API_KEY = SERPER_API_KEY
+app.state.config.SERPLY_API_KEY = SERPLY_API_KEY
+app.state.config.TAVILY_API_KEY = TAVILY_API_KEY
 app.state.config.RAG_WEB_SEARCH_RESULT_COUNT = RAG_WEB_SEARCH_RESULT_COUNT
 app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS = RAG_WEB_SEARCH_CONCURRENT_REQUESTS
 
@@ -392,6 +402,8 @@ async def get_rag_config(user=Depends(get_admin_user)):
                 "serpstack_api_key": app.state.config.SERPSTACK_API_KEY,
                 "serpstack_https": app.state.config.SERPSTACK_HTTPS,
                 "serper_api_key": app.state.config.SERPER_API_KEY,
+                "serply_api_key": app.state.config.SERPLY_API_KEY,
+                "tavily_api_key": app.state.config.TAVILY_API_KEY,
                 "result_count": app.state.config.RAG_WEB_SEARCH_RESULT_COUNT,
                 "concurrent_requests": app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS,
             },
@@ -419,6 +431,8 @@ class WebSearchConfig(BaseModel):
     serpstack_api_key: Optional[str] = None
     serpstack_https: Optional[bool] = None
     serper_api_key: Optional[str] = None
+    serply_api_key: Optional[str] = None
+    tavily_api_key: Optional[str] = None
     result_count: Optional[int] = None
     concurrent_requests: Optional[int] = None
 
@@ -469,6 +483,8 @@ async def update_rag_config(form_data: ConfigUpdateForm, user=Depends(get_admin_
         app.state.config.SERPSTACK_API_KEY = form_data.web.search.serpstack_api_key
         app.state.config.SERPSTACK_HTTPS = form_data.web.search.serpstack_https
         app.state.config.SERPER_API_KEY = form_data.web.search.serper_api_key
+        app.state.config.SERPLY_API_KEY = form_data.web.search.serply_api_key
+        app.state.config.TAVILY_API_KEY = form_data.web.search.tavily_api_key
         app.state.config.RAG_WEB_SEARCH_RESULT_COUNT = form_data.web.search.result_count
         app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS = (
             form_data.web.search.concurrent_requests
@@ -497,6 +513,8 @@ async def update_rag_config(form_data: ConfigUpdateForm, user=Depends(get_admin_
                 "serpstack_api_key": app.state.config.SERPSTACK_API_KEY,
                 "serpstack_https": app.state.config.SERPSTACK_HTTPS,
                 "serper_api_key": app.state.config.SERPER_API_KEY,
+                "serply_api_key": app.state.config.SERPLY_API_KEY,
+                "tavily_api_key": app.state.config.TAVILY_API_KEY,
                 "result_count": app.state.config.RAG_WEB_SEARCH_RESULT_COUNT,
                 "concurrent_requests": app.state.config.RAG_WEB_SEARCH_CONCURRENT_REQUESTS,
             },
@@ -693,7 +711,7 @@ def get_web_loader(url: Union[str, Sequence[str]], verify_ssl: bool = True):
     # Check if the URL is valid
     if not validate_url(url):
         raise ValueError(ERROR_MESSAGES.INVALID_URL)
-    return WebBaseLoader(
+    return SafeWebBaseLoader(
         url,
         verify_ssl=verify_ssl,
         requests_per_second=RAG_WEB_SEARCH_CONCURRENT_REQUESTS,
@@ -744,7 +762,8 @@ def search_web(engine: str, query: str) -> list[SearchResult]:
     - BRAVE_SEARCH_API_KEY
     - SERPSTACK_API_KEY
     - SERPER_API_KEY
-
+    - SERPLY_API_KEY
+    - TAVILY_API_KEY
     Args:
         query (str): The query to search for
     """
@@ -802,6 +821,26 @@ def search_web(engine: str, query: str) -> list[SearchResult]:
             )
         else:
             raise Exception("No SERPER_API_KEY found in environment variables")
+    elif engine == "serply":
+        if app.state.config.SERPLY_API_KEY:
+            return search_serply(
+                app.state.config.SERPLY_API_KEY,
+                query,
+                app.state.config.RAG_WEB_SEARCH_RESULT_COUNT,
+            )
+        else:
+            raise Exception("No SERPLY_API_KEY found in environment variables")
+    elif engine == "duckduckgo":
+        return search_duckduckgo(query, app.state.config.RAG_WEB_SEARCH_RESULT_COUNT)
+    elif engine == "tavily":
+        if app.state.config.TAVILY_API_KEY:
+            return search_tavily(
+                app.state.config.TAVILY_API_KEY,
+                query,
+                app.state.config.RAG_WEB_SEARCH_RESULT_COUNT,
+            )
+        else:
+            raise Exception("No TAVILY_API_KEY found in environment variables")
     else:
         raise Exception("No search engine API key found in environment variables")
 
@@ -809,6 +848,9 @@ def search_web(engine: str, query: str) -> list[SearchResult]:
 @app.post("/web/search")
 def store_web_search(form_data: SearchForm, user=Depends(get_current_user)):
     try:
+        logging.info(
+            f"trying to web search with {app.state.config.RAG_WEB_SEARCH_ENGINE, form_data.query}"
+        )
         web_results = search_web(
             app.state.config.RAG_WEB_SEARCH_ENGINE, form_data.query
         )
@@ -879,6 +921,13 @@ def store_docs_in_vector_db(docs, collection_name, overwrite: bool = False) -> b
     texts = [doc.page_content for doc in docs]
     metadatas = [doc.metadata for doc in docs]
 
+    # ChromaDB does not like datetime formats
+    # for meta-data so convert them to string.
+    for metadata in metadatas:
+        for key, value in metadata.items():
+            if isinstance(value, datetime):
+                metadata[key] = str(value)
+
     try:
         if overwrite:
             for collection in CHROMA_CLIENT.list_collections():
@@ -965,6 +1014,7 @@ def get_loader(filename: str, file_content_type: str, file_path: str):
         "swift",
         "vue",
         "svelte",
+        "msg",
     ]
 
     if file_ext == "pdf":
@@ -999,6 +1049,8 @@ def get_loader(filename: str, file_content_type: str, file_path: str):
         "application/vnd.openxmlformats-officedocument.presentationml.presentation",
     ] or file_ext in ["ppt", "pptx"]:
         loader = UnstructuredPowerPointLoader(file_path)
+    elif file_ext == "msg":
+        loader = OutlookMessageLoader(file_path)
     elif file_ext in known_source_ext or (
         file_content_type and file_content_type.find("text/") >= 0
     ):
@@ -1209,6 +1261,33 @@ def reset(user=Depends(get_admin_user)) -> bool:
     return True
 
 
+class SafeWebBaseLoader(WebBaseLoader):
+    """WebBaseLoader with enhanced error handling for URLs."""
+
+    def lazy_load(self) -> Iterator[Document]:
+        """Lazy load text from the url(s) in web_path with error handling."""
+        for path in self.web_paths:
+            try:
+                soup = self._scrape(path, bs_kwargs=self.bs_kwargs)
+                text = soup.get_text(**self.bs_get_text_kwargs)
+
+                # Build metadata
+                metadata = {"source": path}
+                if title := soup.find("title"):
+                    metadata["title"] = title.get_text()
+                if description := soup.find("meta", attrs={"name": "description"}):
+                    metadata["description"] = description.get(
+                        "content", "No description found."
+                    )
+                if html := soup.find("html"):
+                    metadata["language"] = html.get("lang", "No language found.")
+
+                yield Document(page_content=text, metadata=metadata)
+            except Exception as e:
+                # Log the error and continue with the next URL
+                log.error(f"Error loading {path}: {e}")
+
+
 if ENV == "dev":
 
     @app.get("/ef")

backend/apps/rag/search/duckduckgo.py

@@ -0,0 +1,46 @@
+import logging
+
+from apps.rag.search.main import SearchResult
+from duckduckgo_search import DDGS
+from config import SRC_LOG_LEVELS
+
+log = logging.getLogger(__name__)
+log.setLevel(SRC_LOG_LEVELS["RAG"])
+
+
+def search_duckduckgo(query: str, count: int) -> list[SearchResult]:
+    """
+    Search using DuckDuckGo's Search API and return the results as a list of SearchResult objects.
+    Args:
+        query (str): The query to search for
+        count (int): The number of results to return
+
+    Returns:
+        List[SearchResult]: A list of search results
+    """
+    # Use the DDGS context manager to create a DDGS object
+    with DDGS() as ddgs:
+        # Use the ddgs.text() method to perform the search
+        ddgs_gen = ddgs.text(
+            query, safesearch="moderate", max_results=count, backend="api"
+        )
+        # Check if there are search results
+        if ddgs_gen:
+            # Convert the search results into a list
+            search_results = [r for r in ddgs_gen]
+
+    # Create an empty list to store the SearchResult objects
+    results = []
+    # Iterate over each search result
+    for result in search_results:
+        # Create a SearchResult object and append it to the results list
+        results.append(
+            SearchResult(
+                link=result["href"],
+                title=result.get("title"),
+                snippet=result.get("body"),
+            )
+        )
+    print(results)
+    # Return the list of search results
+    return results

backend/apps/rag/search/searxng.py

@@ -25,6 +25,7 @@ def search_searxng(
 
     Keyword Args:
         language (str): Language filter for the search results; e.g., "en-US". Defaults to an empty string.
+        safesearch (int): Safe search filter for safer web results; 0 = off, 1 = moderate, 2 = strict. Defaults to 1 (moderate).
         time_range (str): Time range for filtering results by date; e.g., "2023-04-05..today" or "all-time". Defaults to ''.
         categories: (Optional[List[str]]): Specific categories within which the search should be performed, defaulting to an empty string if not provided.
 
@@ -37,6 +38,7 @@ def search_searxng(
 
     # Default values for optional parameters are provided as empty strings or None when not specified.
     language = kwargs.get("language", "en-US")
+    safesearch = kwargs.get("safesearch", "1")
     time_range = kwargs.get("time_range", "")
     categories = "".join(kwargs.get("categories", []))
 
@@ -44,6 +46,7 @@ def search_searxng(
         "q": query,
         "format": "json",
         "pageno": 1,
+        "safesearch": safesearch,
         "language": language,
         "time_range": time_range,
         "categories": categories,

backend/apps/rag/search/serply.py

@@ -0,0 +1,68 @@
+import json
+import logging
+
+import requests
+from urllib.parse import urlencode
+
+from apps.rag.search.main import SearchResult
+from config import SRC_LOG_LEVELS
+
+log = logging.getLogger(__name__)
+log.setLevel(SRC_LOG_LEVELS["RAG"])
+
+
+def search_serply(
+    api_key: str,
+    query: str,
+    count: int,
+    hl: str = "us",
+    limit: int = 10,
+    device_type: str = "desktop",
+    proxy_location: str = "US",
+) -> list[SearchResult]:
+    """Search using serper.dev's API and return the results as a list of SearchResult objects.
+
+    Args:
+        api_key (str): A serply.io API key
+        query (str): The query to search for
+        hl (str): Host Language code to display results in (reference https://developers.google.com/custom-search/docs/xml_results?hl=en#wsInterfaceLanguages)
+        limit (int): The maximum number of results to return [10-100, defaults to 10]
+    """
+    log.info("Searching with Serply")
+
+    url = "https://api.serply.io/v1/search/"
+
+    query_payload = {
+        "q": query,
+        "language": "en",
+        "num": limit,
+        "gl": proxy_location.upper(),
+        "hl": hl.lower(),
+    }
+
+    url = f"{url}{urlencode(query_payload)}"
+    headers = {
+        "X-API-KEY": api_key,
+        "X-User-Agent": device_type,
+        "User-Agent": "open-webui",
+        "X-Proxy-Location": proxy_location,
+    }
+
+    response = requests.request("GET", url, headers=headers)
+    response.raise_for_status()
+
+    json_response = response.json()
+    log.info(f"results from serply search: {json_response}")
+
+    results = sorted(
+        json_response.get("results", []), key=lambda x: x.get("realPosition", 0)
+    )
+
+    return [
+        SearchResult(
+            link=result["link"],
+            title=result.get("title"),
+            snippet=result.get("description"),
+        )
+        for result in results[:count]
+    ]

backend/apps/rag/search/tavily.py

@@ -0,0 +1,39 @@
+import logging
+
+import requests
+
+from apps.rag.search.main import SearchResult
+from config import SRC_LOG_LEVELS
+
+log = logging.getLogger(__name__)
+log.setLevel(SRC_LOG_LEVELS["RAG"])
+
+
+def search_tavily(api_key: str, query: str, count: int) -> list[SearchResult]:
+    """Search using Tavily's Search API and return the results as a list of SearchResult objects.
+
+    Args:
+        api_key (str): A Tavily Search API key
+        query (str): The query to search for
+
+    Returns:
+        List[SearchResult]: A list of search results
+    """
+    url = "https://api.tavily.com/search"
+    data = {"query": query, "api_key": api_key}
+
+    response = requests.post(url, json=data)
+    response.raise_for_status()
+
+    json_response = response.json()
+
+    raw_search_results = json_response.get("results", [])
+
+    return [
+        SearchResult(
+            link=result["url"],
+            title=result.get("title", ""),
+            snippet=result.get("content"),
+        )
+        for result in raw_search_results[:count]
+    ]

backend/apps/rag/search/testdata/serply.json

@@ -0,0 +1,206 @@
+{
+	"ads": [],
+	"ads_count": 0,
+	"answers": [],
+	"results": [
+		{
+			"title": "Apple",
+			"link": "https://www.apple.com/",
+			"description": "Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, ...",
+			"additional_links": [
+				{
+					"text": "AppleApplehttps://www.apple.com",
+					"href": "https://www.apple.com/"
+				}
+			],
+			"cite": {},
+			"subdomains": [
+				{
+					"title": "Support",
+					"link": "https://support.apple.com/",
+					"description": "SupportContact - iPhone Support - Billing and Subscriptions - Apple Repair"
+				},
+				{
+					"title": "Store",
+					"link": "https://www.apple.com/store",
+					"description": "StoreShop iPhone - Shop iPad - App Store - Shop Mac - ..."
+				},
+				{
+					"title": "Mac",
+					"link": "https://www.apple.com/mac/",
+					"description": "MacMacBook Air - MacBook Pro - iMac - Compare Mac models - Mac mini"
+				},
+				{
+					"title": "iPad",
+					"link": "https://www.apple.com/ipad/",
+					"description": "iPadShop iPad - iPad Pro - iPad Air - Compare iPad models - ..."
+				},
+				{
+					"title": "Watch",
+					"link": "https://www.apple.com/watch/",
+					"description": "WatchShop Apple Watch - Series 9 - SE - Ultra 2 - Nike - Hermès - ..."
+				}
+			],
+			"realPosition": 1
+		},
+		{
+			"title": "Apple",
+			"link": "https://www.apple.com/",
+			"description": "Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, ...",
+			"additional_links": [
+				{
+					"text": "AppleApplehttps://www.apple.com",
+					"href": "https://www.apple.com/"
+				}
+			],
+			"cite": {},
+			"realPosition": 2
+		},
+		{
+			"title": "Apple Inc.",
+			"link": "https://en.wikipedia.org/wiki/Apple_Inc.",
+			"description": "Apple Inc. (formerly Apple Computer, Inc.) is an American multinational corporation and technology company headquartered in Cupertino, California, ...",
+			"additional_links": [
+				{
+					"text": "Apple Inc.Wikipediahttps://en.wikipedia.org › wiki › Apple_Inc",
+					"href": "https://en.wikipedia.org/wiki/Apple_Inc."
+				},
+				{
+					"text": "",
+					"href": "https://en.wikipedia.org/wiki/Apple_Inc."
+				},
+				{
+					"text": "History",
+					"href": "https://en.wikipedia.org/wiki/History_of_Apple_Inc."
+				},
+				{
+					"text": "List of Apple products",
+					"href": "https://en.wikipedia.org/wiki/List_of_Apple_products"
+				},
+				{
+					"text": "Litigation involving Apple Inc.",
+					"href": "https://en.wikipedia.org/wiki/Litigation_involving_Apple_Inc."
+				},
+				{
+					"text": "Apple Park",
+					"href": "https://en.wikipedia.org/wiki/Apple_Park"
+				}
+			],
+			"cite": {
+				"domain": "https://en.wikipedia.org › wiki › Apple_Inc",
+				"span": " › wiki › Apple_Inc"
+			},
+			"realPosition": 3
+		},
+		{
+			"title": "Apple Inc. (AAPL) Company Profile & Facts",
+			"link": "https://finance.yahoo.com/quote/AAPL/profile/",
+			"description": "Apple Inc. designs, manufactures, and markets smartphones, personal computers, tablets, wearables, and accessories worldwide. The company offers iPhone, a line ...",
+			"additional_links": [
+				{
+					"text": "Apple Inc. (AAPL) Company Profile & FactsYahoo Financehttps://finance.yahoo.com › quote › AAPL › profile",
+					"href": "https://finance.yahoo.com/quote/AAPL/profile/"
+				}
+			],
+			"cite": {
+				"domain": "https://finance.yahoo.com › quote › AAPL › profile",
+				"span": " › quote › AAPL › profile"
+			},
+			"realPosition": 4
+		},
+		{
+			"title": "Apple Inc - Company Profile and News",
+			"link": "https://www.bloomberg.com/profile/company/AAPL:US",
+			"description": "Apple Inc. Apple Inc. designs, manufactures, and markets smartphones, personal computers, tablets, wearables and accessories, and sells a variety of related ...",
+			"additional_links": [
+				{
+					"text": "Apple Inc - Company Profile and NewsBloomberghttps://www.bloomberg.com › company › AAPL:US",
+					"href": "https://www.bloomberg.com/profile/company/AAPL:US"
+				},
+				{
+					"text": "",
+					"href": "https://www.bloomberg.com/profile/company/AAPL:US"
+				}
+			],
+			"cite": {
+				"domain": "https://www.bloomberg.com › company › AAPL:US",
+				"span": " › company › AAPL:US"
+			},
+			"realPosition": 5
+		},
+		{
+			"title": "Apple Inc. | History, Products, Headquarters, & Facts",
+			"link": "https://www.britannica.com/money/Apple-Inc",
+			"description": "May 22, 2024 — Apple Inc. is an American multinational technology company that revolutionized the technology sector through its innovation of computer ...",
+			"additional_links": [
+				{
+					"text": "Apple Inc. | History, Products, Headquarters, & FactsBritannicahttps://www.britannica.com › money › Apple-Inc",
+					"href": "https://www.britannica.com/money/Apple-Inc"
+				},
+				{
+					"text": "",
+					"href": "https://www.britannica.com/money/Apple-Inc"
+				}
+			],
+			"cite": {
+				"domain": "https://www.britannica.com › money › Apple-Inc",
+				"span": " › money › Apple-Inc"
+			},
+			"realPosition": 6
+		}
+	],
+	"shopping_ads": [],
+	"places": [
+		{
+			"title": "Apple Inc."
+		},
+		{
+			"title": "Apple Inc"
+		},
+		{
+			"title": "Apple Inc"
+		}
+	],
+	"related_searches": {
+		"images": [],
+		"text": [
+			{
+				"title": "apple inc full form",
+				"link": "https://www.google.com/search?sca_esv=6b6df170a5c9891b&sca_upv=1&q=Apple+Inc+full+form&sa=X&ved=2ahUKEwjLxuSJwM-GAxUHODQIHYuJBhgQ1QJ6BAhPEAE"
+			},
+			{
+				"title": "apple company history",
+				"link": "https://www.google.com/search?sca_esv=6b6df170a5c9891b&sca_upv=1&q=Apple+company+history&sa=X&ved=2ahUKEwjLxuSJwM-GAxUHODQIHYuJBhgQ1QJ6BAhOEAE"
+			},
+			{
+				"title": "apple store",
+				"link": "https://www.google.com/search?sca_esv=6b6df170a5c9891b&sca_upv=1&q=Apple+Store&sa=X&ved=2ahUKEwjLxuSJwM-GAxUHODQIHYuJBhgQ1QJ6BAhQEAE"
+			},
+			{
+				"title": "apple id",
+				"link": "https://www.google.com/search?sca_esv=6b6df170a5c9891b&sca_upv=1&q=Apple+id&sa=X&ved=2ahUKEwjLxuSJwM-GAxUHODQIHYuJBhgQ1QJ6BAhSEAE"
+			},
+			{
+				"title": "apple inc industry",
+				"link": "https://www.google.com/search?sca_esv=6b6df170a5c9891b&sca_upv=1&q=Apple+Inc+industry&sa=X&ved=2ahUKEwjLxuSJwM-GAxUHODQIHYuJBhgQ1QJ6BAhREAE"
+			},
+			{
+				"title": "apple login",
+				"link": "https://www.google.com/search?sca_esv=6b6df170a5c9891b&sca_upv=1&q=Apple+login&sa=X&ved=2ahUKEwjLxuSJwM-GAxUHODQIHYuJBhgQ1QJ6BAhTEAE"
+			}
+		]
+	},
+	"image_results": [],
+	"carousel": [],
+	"total": 2450000000,
+	"knowledge_graph": "",
+	"related_questions": [
+		"What does the Apple Inc do?",
+		"Why did Apple change to Apple Inc?",
+		"Who owns Apple Inc.?",
+		"What is Apple Inc best known for?"
+	],
+	"carousel_count": 0,
+	"ts": 2.491065263748169,
+	"device_type": null
+}

backend/apps/rag/utils.py

@@ -20,7 +20,7 @@ from langchain.retrievers import (
 
 from typing import Optional
 
-
+from utils.misc import get_last_user_message, add_or_update_system_message
 from config import SRC_LOG_LEVELS, CHROMA_CLIENT
 
 log = logging.getLogger(__name__)
@@ -236,10 +236,9 @@ def get_embedding_function(
         return lambda query: generate_multiple(query, func)
 
 
-def rag_messages(
+def get_rag_context(
     docs,
     messages,
-    template,
     embedding_function,
     k,
     reranking_function,
@@ -247,31 +246,7 @@ def rag_messages(
     hybrid_search,
 ):
     log.debug(f"docs: {docs} {messages} {embedding_function} {reranking_function}")
-
-    last_user_message_idx = None
-    for i in range(len(messages) - 1, -1, -1):
-        if messages[i]["role"] == "user":
-            last_user_message_idx = i
-            break
-
-    user_message = messages[last_user_message_idx]
-
-    if isinstance(user_message["content"], list):
-        # Handle list content input
-        content_type = "list"
-        query = ""
-        for content_item in user_message["content"]:
-            if content_item["type"] == "text":
-                query = content_item["text"]
-                break
-    elif isinstance(user_message["content"], str):
-        # Handle text content input
-        content_type = "text"
-        query = user_message["content"]
-    else:
-        # Fallback in case the input does not match expected types
-        content_type = None
-        query = ""
+    query = get_last_user_message(messages)
 
     extracted_collections = []
     relevant_contexts = []
@@ -342,33 +317,7 @@ def rag_messages(
 
     context_string = context_string.strip()
 
-    ra_content = rag_template(
-        template=template,
-        context=context_string,
-        query=query,
-    )
-
-    log.debug(f"ra_content: {ra_content}")
-
-    if content_type == "list":
-        new_content = []
-        for content_item in user_message["content"]:
-            if content_item["type"] == "text":
-                # Update the text item's content with ra_content
-                new_content.append({"type": "text", "text": ra_content})
-            else:
-                # Keep other types of content as they are
-                new_content.append(content_item)
-        new_user_message = {**user_message, "content": new_content}
-    else:
-        new_user_message = {
-            **user_message,
-            "content": ra_content,
-        }
-
-    messages[last_user_message_idx] = new_user_message
-
-    return messages, citations
+    return context_string, citations
 
 
 def get_model_path(model: str, update_model: bool = False):

backend/apps/socket/main.py

@@ -10,7 +10,7 @@ app = socketio.ASGIApp(sio, socketio_path="/ws/socket.io")
 
 # Dictionary to maintain the user pool
 
-
+SESSION_POOL = {}
 USER_POOL = {}
 USAGE_POOL = {}
 # Timeout duration in seconds
@@ -19,8 +19,6 @@ TIMEOUT_DURATION = 3
 
 @sio.event
 async def connect(sid, environ, auth):
-    print("connect ", sid)
-
     user = None
     if auth and "token" in auth:
         data = decode_token(auth["token"])
@@ -29,10 +27,14 @@ async def connect(sid, environ, auth):
             user = Users.get_user_by_id(data["id"])
 
         if user:
-            USER_POOL[sid] = user.id
+            SESSION_POOL[sid] = user.id
+            if user.id in USER_POOL:
+                USER_POOL[user.id].append(sid)
+            else:
+                USER_POOL[user.id] = [sid]
+
             print(f"user {user.name}({user.id}) connected with session ID {sid}")
 
-            print(len(set(USER_POOL)))
             await sio.emit("user-count", {"count": len(set(USER_POOL))})
             await sio.emit("usage", {"models": get_models_in_use()})
 
@@ -50,16 +52,20 @@ async def user_join(sid, data):
             user = Users.get_user_by_id(data["id"])
 
         if user:
-            USER_POOL[sid] = user.id
+
+            SESSION_POOL[sid] = user.id
+            if user.id in USER_POOL:
+                USER_POOL[user.id].append(sid)
+            else:
+                USER_POOL[user.id] = [sid]
+
             print(f"user {user.name}({user.id}) connected with session ID {sid}")
 
-            print(len(set(USER_POOL)))
             await sio.emit("user-count", {"count": len(set(USER_POOL))})
 
 
 @sio.on("user-count")
 async def user_count(sid):
-    print("user-count", sid)
     await sio.emit("user-count", {"count": len(set(USER_POOL))})
 
 
@@ -68,14 +74,12 @@ def get_models_in_use():
     models_in_use = []
     for model_id, data in USAGE_POOL.items():
         models_in_use.append(model_id)
-    print(f"Models in use: {models_in_use}")
 
     return models_in_use
 
 
 @sio.on("usage")
 async def usage(sid, data):
-    print(f'Received "usage" event from {sid}: {data}')
 
     model_id = data["model"]
 
@@ -103,7 +107,6 @@ async def usage(sid, data):
 
 async def remove_after_timeout(sid, model_id):
     try:
-        print("remove_after_timeout", sid, model_id)
         await asyncio.sleep(TIMEOUT_DURATION)
         if model_id in USAGE_POOL:
             print(USAGE_POOL[model_id]["sids"])
@@ -113,7 +116,6 @@ async def remove_after_timeout(sid, model_id):
             if len(USAGE_POOL[model_id]["sids"]) == 0:
                 del USAGE_POOL[model_id]
 
-            print(f"Removed usage data for {model_id} due to timeout")
             # Broadcast the usage data to all clients
             await sio.emit("usage", {"models": get_models_in_use()})
     except asyncio.CancelledError:
@@ -123,9 +125,14 @@ async def remove_after_timeout(sid, model_id):
 
 @sio.event
 async def disconnect(sid):
-    if sid in USER_POOL:
-        disconnected_user = USER_POOL.pop(sid)
-        print(f"user {disconnected_user} disconnected with session ID {sid}")
+    if sid in SESSION_POOL:
+        user_id = SESSION_POOL[sid]
+        del SESSION_POOL[sid]
+
+        USER_POOL[user_id].remove(sid)
+
+        if len(USER_POOL[user_id]) == 0:
+            del USER_POOL[user_id]
 
         await sio.emit("user-count", {"count": len(USER_POOL)})
     else:

backend/apps/webui/internal/migrations/012_add_tools.py

@@ -0,0 +1,61 @@
+"""Peewee migrations -- 009_add_models.py.
+
+Some examples (model - class or model name)::
+
+    > Model = migrator.orm['table_name']            # Return model in current state by name
+    > Model = migrator.ModelClass                   # Return model in current state by name
+
+    > migrator.sql(sql)                             # Run custom SQL
+    > migrator.run(func, *args, **kwargs)           # Run python function with the given args
+    > migrator.create_model(Model)                  # Create a model (could be used as decorator)
+    > migrator.remove_model(model, cascade=True)    # Remove a model
+    > migrator.add_fields(model, **fields)          # Add fields to a model
+    > migrator.change_fields(model, **fields)       # Change fields
+    > migrator.remove_fields(model, *field_names, cascade=True)
+    > migrator.rename_field(model, old_field_name, new_field_name)
+    > migrator.rename_table(model, new_table_name)
+    > migrator.add_index(model, *col_names, unique=False)
+    > migrator.add_not_null(model, *field_names)
+    > migrator.add_default(model, field_name, default)
+    > migrator.add_constraint(model, name, sql)
+    > migrator.drop_index(model, *col_names)
+    > migrator.drop_not_null(model, *field_names)
+    > migrator.drop_constraints(model, *constraints)
+
+"""
+
+from contextlib import suppress
+
+import peewee as pw
+from peewee_migrate import Migrator
+
+
+with suppress(ImportError):
+    import playhouse.postgres_ext as pw_pext
+
+
+def migrate(migrator: Migrator, database: pw.Database, *, fake=False):
+    """Write your migrations here."""
+
+    @migrator.create_model
+    class Tool(pw.Model):
+        id = pw.TextField(unique=True)
+        user_id = pw.TextField()
+
+        name = pw.TextField()
+        content = pw.TextField()
+        specs = pw.TextField()
+
+        meta = pw.TextField()
+
+        created_at = pw.BigIntegerField(null=False)
+        updated_at = pw.BigIntegerField(null=False)
+
+        class Meta:
+            table_name = "tool"
+
+
+def rollback(migrator: Migrator, database: pw.Database, *, fake=False):
+    """Write your rollback migrations here."""
+
+    migrator.remove_model("tool")

backend/apps/webui/internal/migrations/011_add_user_oauth_sub.py → backend/apps/webui/internal/migrations/013_add_user_oauth_sub.py

@@ -1,4 +1,4 @@
-"""Peewee migrations -- 011_add_user_oauth_sub.py.
+"""Peewee migrations -- 013_add_user_oauth_sub.py.
 
 Some examples (model - class or model name)::
 

backend/apps/webui/main.py

@@ -8,6 +8,7 @@ from apps.webui.routers import (
     users,
     chats,
     documents,
+    tools,
     models,
     prompts,
     configs,
@@ -27,9 +28,9 @@ from config import (
     WEBHOOK_URL,
     WEBUI_AUTH_TRUSTED_EMAIL_HEADER,
     JWT_EXPIRES_IN,
-    AppConfig,
-    ENABLE_COMMUNITY_SHARING,
     WEBUI_BANNERS,
+    ENABLE_COMMUNITY_SHARING,
+    AppConfig,
 )
 
 app = FastAPI()
@@ -40,6 +41,7 @@ app.state.config = AppConfig()
 
 app.state.config.ENABLE_SIGNUP = ENABLE_SIGNUP
 app.state.config.JWT_EXPIRES_IN = JWT_EXPIRES_IN
+app.state.AUTH_TRUSTED_EMAIL_HEADER = WEBUI_AUTH_TRUSTED_EMAIL_HEADER
 
 
 app.state.config.SHOW_ADMIN_DETAILS = SHOW_ADMIN_DETAILS
@@ -56,7 +58,7 @@ app.state.config.BANNERS = WEBUI_BANNERS
 app.state.config.ENABLE_COMMUNITY_SHARING = ENABLE_COMMUNITY_SHARING
 
 app.state.MODELS = {}
-app.state.AUTH_TRUSTED_EMAIL_HEADER = WEBUI_AUTH_TRUSTED_EMAIL_HEADER
+app.state.TOOLS = {}
 
 
 app.add_middleware(
@@ -72,6 +74,7 @@ app.include_router(users.router, prefix="/users", tags=["users"])
 app.include_router(chats.router, prefix="/chats", tags=["chats"])
 
 app.include_router(documents.router, prefix="/documents", tags=["documents"])
+app.include_router(tools.router, prefix="/tools", tags=["tools"])
 app.include_router(models.router, prefix="/models", tags=["models"])
 app.include_router(prompts.router, prefix="/prompts", tags=["prompts"])
 app.include_router(memories.router, prefix="/memories", tags=["memories"])

backend/apps/webui/models/memories.py

@@ -65,6 +65,20 @@ class MemoriesTable:
         else:
             return None
 
+    def update_memory_by_id(
+        self,
+        id: str,
+        content: str,
+    ) -> Optional[MemoryModel]:
+        try:
+            memory = Memory.get(Memory.id == id)
+            memory.content = content
+            memory.updated_at = int(time.time())
+            memory.save()
+            return MemoryModel(**model_to_dict(memory))
+        except:
+            return None
+
     def get_memories(self) -> List[MemoryModel]:
         try:
             memories = Memory.select()

backend/apps/webui/models/tools.py

@@ -0,0 +1,132 @@
+from pydantic import BaseModel
+from peewee import *
+from playhouse.shortcuts import model_to_dict
+from typing import List, Union, Optional
+import time
+import logging
+from apps.webui.internal.db import DB, JSONField
+
+import json
+
+from config import SRC_LOG_LEVELS
+
+log = logging.getLogger(__name__)
+log.setLevel(SRC_LOG_LEVELS["MODELS"])
+
+####################
+# Tools DB Schema
+####################
+
+
+class Tool(Model):
+    id = CharField(unique=True)
+    user_id = CharField()
+    name = TextField()
+    content = TextField()
+    specs = JSONField()
+    meta = JSONField()
+    updated_at = BigIntegerField()
+    created_at = BigIntegerField()
+
+    class Meta:
+        database = DB
+
+
+class ToolMeta(BaseModel):
+    description: Optional[str] = None
+
+
+class ToolModel(BaseModel):
+    id: str
+    user_id: str
+    name: str
+    content: str
+    specs: List[dict]
+    meta: ToolMeta
+    updated_at: int  # timestamp in epoch
+    created_at: int  # timestamp in epoch
+
+
+####################
+# Forms
+####################
+
+
+class ToolResponse(BaseModel):
+    id: str
+    user_id: str
+    name: str
+    meta: ToolMeta
+    updated_at: int  # timestamp in epoch
+    created_at: int  # timestamp in epoch
+
+
+class ToolForm(BaseModel):
+    id: str
+    name: str
+    content: str
+    meta: ToolMeta
+
+
+class ToolsTable:
+    def __init__(self, db):
+        self.db = db
+        self.db.create_tables([Tool])
+
+    def insert_new_tool(
+        self, user_id: str, form_data: ToolForm, specs: List[dict]
+    ) -> Optional[ToolModel]:
+        tool = ToolModel(
+            **{
+                **form_data.model_dump(),
+                "specs": specs,
+                "user_id": user_id,
+                "updated_at": int(time.time()),
+                "created_at": int(time.time()),
+            }
+        )
+
+        try:
+            result = Tool.create(**tool.model_dump())
+            if result:
+                return tool
+            else:
+                return None
+        except Exception as e:
+            print(f"Error creating tool: {e}")
+            return None
+
+    def get_tool_by_id(self, id: str) -> Optional[ToolModel]:
+        try:
+            tool = Tool.get(Tool.id == id)
+            return ToolModel(**model_to_dict(tool))
+        except:
+            return None
+
+    def get_tools(self) -> List[ToolModel]:
+        return [ToolModel(**model_to_dict(tool)) for tool in Tool.select()]
+
+    def update_tool_by_id(self, id: str, updated: dict) -> Optional[ToolModel]:
+        try:
+            query = Tool.update(
+                **updated,
+                updated_at=int(time.time()),
+            ).where(Tool.id == id)
+            query.execute()
+
+            tool = Tool.get(Tool.id == id)
+            return ToolModel(**model_to_dict(tool))
+        except:
+            return None
+
+    def delete_tool_by_id(self, id: str) -> bool:
+        try:
+            query = Tool.delete().where((Tool.id == id))
+            query.execute()  # Remove the rows, return number of rows removed.
+
+            return True
+        except:
+            return False
+
+
+Tools = ToolsTable(DB)

backend/apps/webui/routers/chats.py

@@ -161,7 +161,7 @@ async def get_archived_session_user_chat_list(
 ############################
 
 
-@router.post("/archive/all", response_model=List[ChatTitleIdResponse])
+@router.post("/archive/all", response_model=bool)
 async def archive_all_chats(user=Depends(get_current_user)):
     return Chats.archive_all_chats_by_user_id(user.id)
 

backend/apps/webui/routers/documents.py

@@ -73,7 +73,7 @@ async def create_new_doc(form_data: DocumentForm, user=Depends(get_admin_user)):
 ############################
 
 
-@router.get("/name/{name}", response_model=Optional[DocumentResponse])
+@router.get("/doc", response_model=Optional[DocumentResponse])
 async def get_doc_by_name(name: str, user=Depends(get_current_user)):
     doc = Documents.get_doc_by_name(name)
 
@@ -105,7 +105,7 @@ class TagDocumentForm(BaseModel):
     tags: List[dict]
 
 
-@router.post("/name/{name}/tags", response_model=Optional[DocumentResponse])
+@router.post("/doc/tags", response_model=Optional[DocumentResponse])
 async def tag_doc_by_name(form_data: TagDocumentForm, user=Depends(get_current_user)):
     doc = Documents.update_doc_content_by_name(form_data.name, {"tags": form_data.tags})
 
@@ -128,7 +128,7 @@ async def tag_doc_by_name(form_data: TagDocumentForm, user=Depends(get_current_u
 ############################
 
 
-@router.post("/name/{name}/update", response_model=Optional[DocumentResponse])
+@router.post("/doc/update", response_model=Optional[DocumentResponse])
 async def update_doc_by_name(
     name: str, form_data: DocumentUpdateForm, user=Depends(get_admin_user)
 ):
@@ -152,7 +152,7 @@ async def update_doc_by_name(
 ############################
 
 
-@router.delete("/name/{name}/delete", response_model=bool)
+@router.delete("/doc/delete", response_model=bool)
 async def delete_doc_by_name(name: str, user=Depends(get_admin_user)):
     result = Documents.delete_doc_by_name(name)
     return result

backend/apps/webui/routers/memories.py

@@ -44,6 +44,10 @@ class AddMemoryForm(BaseModel):
     content: str
 
 
+class MemoryUpdateModel(BaseModel):
+    content: Optional[str] = None
+
+
 @router.post("/add", response_model=Optional[MemoryModel])
 async def add_memory(
     request: Request, form_data: AddMemoryForm, user=Depends(get_verified_user)
@@ -62,6 +66,34 @@ async def add_memory(
     return memory
 
 
+@router.post("/{memory_id}/update", response_model=Optional[MemoryModel])
+async def update_memory_by_id(
+    memory_id: str,
+    request: Request,
+    form_data: MemoryUpdateModel,
+    user=Depends(get_verified_user),
+):
+    memory = Memories.update_memory_by_id(memory_id, form_data.content)
+    if memory is None:
+        raise HTTPException(status_code=404, detail="Memory not found")
+
+    if form_data.content is not None:
+        memory_embedding = request.app.state.EMBEDDING_FUNCTION(form_data.content)
+        collection = CHROMA_CLIENT.get_or_create_collection(
+            name=f"user-memory-{user.id}"
+        )
+        collection.upsert(
+            documents=[form_data.content],
+            ids=[memory.id],
+            embeddings=[memory_embedding],
+            metadatas=[
+                {"created_at": memory.created_at, "updated_at": memory.updated_at}
+            ],
+        )
+
+    return memory
+
+
 ############################
 # QueryMemory
 ############################

backend/apps/webui/routers/tools.py

@@ -0,0 +1,183 @@
+from fastapi import Depends, FastAPI, HTTPException, status, Request
+from datetime import datetime, timedelta
+from typing import List, Union, Optional
+
+from fastapi import APIRouter
+from pydantic import BaseModel
+import json
+
+from apps.webui.models.tools import Tools, ToolForm, ToolModel, ToolResponse
+from apps.webui.utils import load_toolkit_module_by_id
+
+from utils.utils import get_current_user, get_admin_user
+from utils.tools import get_tools_specs
+from constants import ERROR_MESSAGES
+
+from importlib import util
+import os
+
+from config import DATA_DIR
+
+
+TOOLS_DIR = f"{DATA_DIR}/tools"
+os.makedirs(TOOLS_DIR, exist_ok=True)
+
+
+router = APIRouter()
+
+############################
+# GetToolkits
+############################
+
+
+@router.get("/", response_model=List[ToolResponse])
+async def get_toolkits(user=Depends(get_current_user)):
+    toolkits = [toolkit for toolkit in Tools.get_tools()]
+    return toolkits
+
+
+############################
+# ExportToolKits
+############################
+
+
+@router.get("/export", response_model=List[ToolModel])
+async def get_toolkits(user=Depends(get_admin_user)):
+    toolkits = [toolkit for toolkit in Tools.get_tools()]
+    return toolkits
+
+
+############################
+# CreateNewToolKit
+############################
+
+
+@router.post("/create", response_model=Optional[ToolResponse])
+async def create_new_toolkit(
+    request: Request, form_data: ToolForm, user=Depends(get_admin_user)
+):
+    if not form_data.id.isidentifier():
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail="Only alphanumeric characters and underscores are allowed in the id",
+        )
+
+    form_data.id = form_data.id.lower()
+
+    toolkit = Tools.get_tool_by_id(form_data.id)
+    if toolkit == None:
+        toolkit_path = os.path.join(TOOLS_DIR, f"{form_data.id}.py")
+        try:
+            with open(toolkit_path, "w") as tool_file:
+                tool_file.write(form_data.content)
+
+            toolkit_module = load_toolkit_module_by_id(form_data.id)
+
+            TOOLS = request.app.state.TOOLS
+            TOOLS[form_data.id] = toolkit_module
+
+            specs = get_tools_specs(TOOLS[form_data.id])
+            toolkit = Tools.insert_new_tool(user.id, form_data, specs)
+
+            if toolkit:
+                return toolkit
+            else:
+                raise HTTPException(
+                    status_code=status.HTTP_400_BAD_REQUEST,
+                    detail=ERROR_MESSAGES.DEFAULT("Error creating toolkit"),
+                )
+        except Exception as e:
+            print(e)
+            raise HTTPException(
+                status_code=status.HTTP_400_BAD_REQUEST,
+                detail=ERROR_MESSAGES.DEFAULT(e),
+            )
+    else:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail=ERROR_MESSAGES.ID_TAKEN,
+        )
+
+
+############################
+# GetToolkitById
+############################
+
+
+@router.get("/id/{id}", response_model=Optional[ToolModel])
+async def get_toolkit_by_id(id: str, user=Depends(get_admin_user)):
+    toolkit = Tools.get_tool_by_id(id)
+
+    if toolkit:
+        return toolkit
+    else:
+        raise HTTPException(
+            status_code=status.HTTP_401_UNAUTHORIZED,
+            detail=ERROR_MESSAGES.NOT_FOUND,
+        )
+
+
+############################
+# UpdateToolkitById
+############################
+
+
+@router.post("/id/{id}/update", response_model=Optional[ToolModel])
+async def update_toolkit_by_id(
+    request: Request, id: str, form_data: ToolForm, user=Depends(get_admin_user)
+):
+    toolkit_path = os.path.join(TOOLS_DIR, f"{id}.py")
+
+    try:
+        with open(toolkit_path, "w") as tool_file:
+            tool_file.write(form_data.content)
+
+        toolkit_module = load_toolkit_module_by_id(id)
+
+        TOOLS = request.app.state.TOOLS
+        TOOLS[id] = toolkit_module
+
+        specs = get_tools_specs(TOOLS[id])
+
+        updated = {
+            **form_data.model_dump(exclude={"id"}),
+            "specs": specs,
+        }
+
+        print(updated)
+        toolkit = Tools.update_tool_by_id(id, updated)
+
+        if toolkit:
+            return toolkit
+        else:
+            raise HTTPException(
+                status_code=status.HTTP_400_BAD_REQUEST,
+                detail=ERROR_MESSAGES.DEFAULT("Error updating toolkit"),
+            )
+
+    except Exception as e:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail=ERROR_MESSAGES.DEFAULT(e),
+        )
+
+
+############################
+# DeleteToolkitById
+############################
+
+
+@router.delete("/id/{id}/delete", response_model=bool)
+async def delete_toolkit_by_id(request: Request, id: str, user=Depends(get_admin_user)):
+    result = Tools.delete_tool_by_id(id)
+
+    if result:
+        TOOLS = request.app.state.TOOLS
+        if id in TOOLS:
+            del TOOLS[id]
+
+        # delete the toolkit file
+        toolkit_path = os.path.join(TOOLS_DIR, f"{id}.py")
+        os.remove(toolkit_path)
+
+    return result

backend/apps/webui/routers/utils.py

@@ -7,6 +7,8 @@ from pydantic import BaseModel
 
 from fpdf import FPDF
 import markdown
+import black
+
 
 from apps.webui.internal.db import DB
 from utils.utils import get_admin_user
@@ -26,6 +28,21 @@ async def get_gravatar(
     return get_gravatar_url(email)
 
 
+class CodeFormatRequest(BaseModel):
+    code: str
+
+
+@router.post("/code/format")
+async def format_code(request: CodeFormatRequest):
+    try:
+        formatted_code = black.format_str(request.code, mode=black.Mode())
+        return {"code": formatted_code}
+    except black.NothingChanged:
+        return {"code": request.code}
+    except Exception as e:
+        raise HTTPException(status_code=400, detail=str(e))
+
+
 class MarkdownForm(BaseModel):
     md: str
 

backend/apps/webui/utils.py

@@ -0,0 +1,23 @@
+from importlib import util
+import os
+
+from config import TOOLS_DIR
+
+
+def load_toolkit_module_by_id(toolkit_id):
+    toolkit_path = os.path.join(TOOLS_DIR, f"{toolkit_id}.py")
+    spec = util.spec_from_file_location(toolkit_id, toolkit_path)
+    module = util.module_from_spec(spec)
+
+    try:
+        spec.loader.exec_module(module)
+        print(f"Loaded module: {module.__name__}")
+        if hasattr(module, "Tools"):
+            return module.Tools()
+        else:
+            raise Exception("No Tools class found")
+    except Exception as e:
+        print(f"Error loading module: {toolkit_id}")
+        # Move the file to the error folder
+        os.rename(toolkit_path, f"{toolkit_path}.error")
+        raise e

backend/config.py

@@ -435,7 +435,11 @@ STATIC_DIR = Path(os.getenv("STATIC_DIR", BACKEND_DIR / "static")).resolve()
 
 frontend_favicon = FRONTEND_BUILD_DIR / "favicon.png"
 if frontend_favicon.exists():
-    shutil.copyfile(frontend_favicon, STATIC_DIR / "favicon.png")
+    try:
+        shutil.copyfile(frontend_favicon, STATIC_DIR / "favicon.png")
+    except Exception as e:
+        logging.error(f"An error occurred: {e}")
+
 else:
     logging.warning(f"Frontend favicon not found at {frontend_favicon}")
 
@@ -493,6 +497,14 @@ DOCS_DIR = os.getenv("DOCS_DIR", f"{DATA_DIR}/docs")
 Path(DOCS_DIR).mkdir(parents=True, exist_ok=True)
 
 
+####################################
+# Tools DIR
+####################################
+
+TOOLS_DIR = os.getenv("TOOLS_DIR", f"{DATA_DIR}/tools")
+Path(TOOLS_DIR).mkdir(parents=True, exist_ok=True)
+
+
 ####################################
 # LITELLM_CONFIG
 ####################################
@@ -542,6 +554,7 @@ OLLAMA_API_BASE_URL = os.environ.get(
 )
 
 OLLAMA_BASE_URL = os.environ.get("OLLAMA_BASE_URL", "")
+AIOHTTP_CLIENT_TIMEOUT = int(os.environ.get("AIOHTTP_CLIENT_TIMEOUT", "300"))
 K8S_FLAG = os.environ.get("K8S_FLAG", "")
 USE_OLLAMA_DOCKER = os.environ.get("USE_OLLAMA_DOCKER", "false")
 
@@ -744,6 +757,78 @@ ADMIN_EMAIL = PersistentConfig(
 )
 
 
+####################################
+# TASKS
+####################################
+
+
+TASK_MODEL = PersistentConfig(
+    "TASK_MODEL",
+    "task.model.default",
+    os.environ.get("TASK_MODEL", ""),
+)
+
+TASK_MODEL_EXTERNAL = PersistentConfig(
+    "TASK_MODEL_EXTERNAL",
+    "task.model.external",
+    os.environ.get("TASK_MODEL_EXTERNAL", ""),
+)
+
+TITLE_GENERATION_PROMPT_TEMPLATE = PersistentConfig(
+    "TITLE_GENERATION_PROMPT_TEMPLATE",
+    "task.title.prompt_template",
+    os.environ.get(
+        "TITLE_GENERATION_PROMPT_TEMPLATE",
+        """Here is the query:
+{{prompt:middletruncate:8000}}
+
+Create a concise, 3-5 word phrase with an emoji as a title for the previous query. Suitable Emojis for the summary can be used to enhance understanding but avoid quotation marks or special formatting. RESPOND ONLY WITH THE TITLE TEXT.
+
+Examples of titles:
+📉 Stock Market Trends
+🍪 Perfect Chocolate Chip Recipe
+Evolution of Music Streaming
+Remote Work Productivity Tips
+Artificial Intelligence in Healthcare
+🎮 Video Game Development Insights""",
+    ),
+)
+
+
+SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE = PersistentConfig(
+    "SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE",
+    "task.search.prompt_template",
+    os.environ.get(
+        "SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE",
+        """You are tasked with generating web search queries. Give me an appropriate query to answer my question for google search. Answer with only the query. Today is {{CURRENT_DATE}}.
+        
+Question:
+{{prompt:end:4000}}""",
+    ),
+)
+
+SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD = PersistentConfig(
+    "SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD",
+    "task.search.prompt_length_threshold",
+    int(
+        os.environ.get(
+            "SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD",
+            100,
+        )
+    ),
+)
+
+TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE = PersistentConfig(
+    "TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE",
+    "task.tools.prompt_template",
+    os.environ.get(
+        "TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE",
+        """Tools: {{TOOLS}}
+If a function tool doesn't match the query, return an empty string. Else, pick a function tool, fill in the parameters from the function tool's schema, and return it in the format { "name": \"functionName\", "parameters": { "key": "value" } }. Only pick a function if the user asks.  Only return the object. Do not return any other text.""",
+    ),
+)
+
+
 ####################################
 # WEBUI_SECRET_KEY
 ####################################
@@ -991,6 +1076,17 @@ SERPER_API_KEY = PersistentConfig(
     os.getenv("SERPER_API_KEY", ""),
 )
 
+SERPLY_API_KEY = PersistentConfig(
+    "SERPLY_API_KEY",
+    "rag.web.search.serply_api_key",
+    os.getenv("SERPLY_API_KEY", ""),
+)
+
+TAVILY_API_KEY = PersistentConfig(
+    "TAVILY_API_KEY",
+    "rag.web.search.tavily_api_key",
+    os.getenv("TAVILY_API_KEY", ""),
+)
 
 RAG_WEB_SEARCH_RESULT_COUNT = PersistentConfig(
     "RAG_WEB_SEARCH_RESULT_COUNT",
@@ -1072,25 +1168,59 @@ IMAGE_GENERATION_MODEL = PersistentConfig(
 # Audio
 ####################################
 
-AUDIO_OPENAI_API_BASE_URL = PersistentConfig(
-    "AUDIO_OPENAI_API_BASE_URL",
-    "audio.openai.api_base_url",
-    os.getenv("AUDIO_OPENAI_API_BASE_URL", OPENAI_API_BASE_URL),
-)
-AUDIO_OPENAI_API_KEY = PersistentConfig(
-    "AUDIO_OPENAI_API_KEY",
-    "audio.openai.api_key",
-    os.getenv("AUDIO_OPENAI_API_KEY", OPENAI_API_KEY),
-)
-AUDIO_OPENAI_API_MODEL = PersistentConfig(
-    "AUDIO_OPENAI_API_MODEL",
-    "audio.openai.api_model",
-    os.getenv("AUDIO_OPENAI_API_MODEL", "tts-1"),
-)
-AUDIO_OPENAI_API_VOICE = PersistentConfig(
-    "AUDIO_OPENAI_API_VOICE",
-    "audio.openai.api_voice",
-    os.getenv("AUDIO_OPENAI_API_VOICE", "alloy"),
+AUDIO_STT_OPENAI_API_BASE_URL = PersistentConfig(
+    "AUDIO_STT_OPENAI_API_BASE_URL",
+    "audio.stt.openai.api_base_url",
+    os.getenv("AUDIO_STT_OPENAI_API_BASE_URL", OPENAI_API_BASE_URL),
+)
+
+AUDIO_STT_OPENAI_API_KEY = PersistentConfig(
+    "AUDIO_STT_OPENAI_API_KEY",
+    "audio.stt.openai.api_key",
+    os.getenv("AUDIO_STT_OPENAI_API_KEY", OPENAI_API_KEY),
+)
+
+AUDIO_STT_ENGINE = PersistentConfig(
+    "AUDIO_STT_ENGINE",
+    "audio.stt.engine",
+    os.getenv("AUDIO_STT_ENGINE", ""),
+)
+
+AUDIO_STT_MODEL = PersistentConfig(
+    "AUDIO_STT_MODEL",
+    "audio.stt.model",
+    os.getenv("AUDIO_STT_MODEL", "whisper-1"),
+)
+
+AUDIO_TTS_OPENAI_API_BASE_URL = PersistentConfig(
+    "AUDIO_TTS_OPENAI_API_BASE_URL",
+    "audio.tts.openai.api_base_url",
+    os.getenv("AUDIO_TTS_OPENAI_API_BASE_URL", OPENAI_API_BASE_URL),
+)
+AUDIO_TTS_OPENAI_API_KEY = PersistentConfig(
+    "AUDIO_TTS_OPENAI_API_KEY",
+    "audio.tts.openai.api_key",
+    os.getenv("AUDIO_TTS_OPENAI_API_KEY", OPENAI_API_KEY),
+)
+
+
+AUDIO_TTS_ENGINE = PersistentConfig(
+    "AUDIO_TTS_ENGINE",
+    "audio.tts.engine",
+    os.getenv("AUDIO_TTS_ENGINE", ""),
+)
+
+
+AUDIO_TTS_MODEL = PersistentConfig(
+    "AUDIO_TTS_MODEL",
+    "audio.tts.model",
+    os.getenv("AUDIO_TTS_MODEL", "tts-1"),
+)
+
+AUDIO_TTS_VOICE = PersistentConfig(
+    "AUDIO_TTS_VOICE",
+    "audio.tts.voice",
+    os.getenv("AUDIO_TTS_VOICE", "alloy"),
 )
 
 

backend/constants.py

@@ -32,6 +32,7 @@ class ERROR_MESSAGES(str, Enum):
     COMMAND_TAKEN = "Uh-oh! This command is already registered. Please choose another command string."
     FILE_EXISTS = "Uh-oh! This file is already registered. Please choose another file."
 
+    ID_TAKEN = "Uh-oh! This id is already registered. Please choose another id string."
     MODEL_ID_TAKEN = "Uh-oh! This model id is already registered. Please choose another model id string."
 
     NAME_TAG_TAKEN = "Uh-oh! This name tag is already registered. Please choose another name tag string."

backend/main.py

@@ -13,8 +13,12 @@ import logging
 import aiohttp
 import requests
 import mimetypes
+import shutil
+import os
+import inspect
+import asyncio
 
-from fastapi import FastAPI, Request, Depends, status
+from fastapi import FastAPI, Request, Depends, status, UploadFile, File, Form
 from fastapi.staticfiles import StaticFiles
 from fastapi.responses import JSONResponse
 from fastapi import HTTPException
@@ -27,21 +31,33 @@ from starlette.responses import StreamingResponse, Response, RedirectResponse
 
 
 from apps.socket.main import app as socket_app
-from apps.ollama.main import app as ollama_app, get_all_models as get_ollama_models
-from apps.openai.main import app as openai_app, get_all_models as get_openai_models
+from apps.ollama.main import (
+    app as ollama_app,
+    OpenAIChatCompletionForm,
+    get_all_models as get_ollama_models,
+    generate_openai_chat_completion as generate_ollama_chat_completion,
+)
+from apps.openai.main import (
+    app as openai_app,
+    get_all_models as get_openai_models,
+    generate_chat_completion as generate_openai_chat_completion,
+)
 
 from apps.audio.main import app as audio_app
 from apps.images.main import app as images_app
 from apps.rag.main import app as rag_app
 from apps.webui.main import app as webui_app
 
-import asyncio
+
 from pydantic import BaseModel
 from typing import List, Optional
 
 from apps.webui.models.auths import Auths
-from apps.webui.models.models import Models
+from apps.webui.models.models import Models, ModelModel
+from apps.webui.models.tools import Tools
 from apps.webui.models.users import Users
+from apps.webui.utils import load_toolkit_module_by_id
+
 from utils.misc import parse_duration
 from utils.utils import (
     get_admin_user,
@@ -51,7 +67,14 @@ from utils.utils import (
     get_password_hash,
     create_token,
 )
-from apps.rag.utils import rag_messages
+from utils.task import (
+    title_generation_template,
+    search_query_generation_template,
+    tools_function_calling_generation_template,
+)
+from utils.misc import get_last_user_message, add_or_update_system_message
+
+from apps.rag.utils import get_rag_context, rag_template
 
 from config import (
     CONFIG_DATA,
@@ -72,14 +95,20 @@ from config import (
     SRC_LOG_LEVELS,
     WEBHOOK_URL,
     ENABLE_ADMIN_EXPORT,
-    AppConfig,
     WEBUI_BUILD_HASH,
+    TASK_MODEL,
+    TASK_MODEL_EXTERNAL,
+    TITLE_GENERATION_PROMPT_TEMPLATE,
+    SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE,
+    SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD,
+    TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE,
     OAUTH_PROVIDERS,
     ENABLE_OAUTH_SIGNUP,
     OAUTH_MERGE_ACCOUNTS_BY_EMAIL,
     WEBUI_SECRET_KEY,
     WEBUI_SESSION_COOKIE_SAME_SITE,
     WEBUI_SESSION_COOKIE_SECURE,
+    AppConfig,
 )
 from constants import ERROR_MESSAGES, WEBHOOK_MESSAGES
 from utils.webhook import post_webhook
@@ -134,27 +163,133 @@ app.state.config.ENABLE_OLLAMA_API = ENABLE_OLLAMA_API
 app.state.config.ENABLE_MODEL_FILTER = ENABLE_MODEL_FILTER
 app.state.config.MODEL_FILTER_LIST = MODEL_FILTER_LIST
 
-
 app.state.config.WEBHOOK_URL = WEBHOOK_URL
 
 
+app.state.config.TASK_MODEL = TASK_MODEL
+app.state.config.TASK_MODEL_EXTERNAL = TASK_MODEL_EXTERNAL
+app.state.config.TITLE_GENERATION_PROMPT_TEMPLATE = TITLE_GENERATION_PROMPT_TEMPLATE
+app.state.config.SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE = (
+    SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE
+)
+app.state.config.SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD = (
+    SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD
+)
+app.state.config.TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE = (
+    TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE
+)
+
 app.state.MODELS = {}
 
 origins = ["*"]
 
-# Custom middleware to add security headers
-# class SecurityHeadersMiddleware(BaseHTTPMiddleware):
-#     async def dispatch(self, request: Request, call_next):
-#         response: Response = await call_next(request)
-#         response.headers["Cross-Origin-Opener-Policy"] = "same-origin"
-#         response.headers["Cross-Origin-Embedder-Policy"] = "require-corp"
-#         return response
 
+async def get_function_call_response(messages, tool_id, template, task_model_id, user):
+    tool = Tools.get_tool_by_id(tool_id)
+    tools_specs = json.dumps(tool.specs, indent=2)
+    content = tools_function_calling_generation_template(template, tools_specs)
+
+    user_message = get_last_user_message(messages)
+    prompt = (
+        "History:\n"
+        + "\n".join(
+            [
+                f"{message['role'].upper()}: \"\"\"{message['content']}\"\"\""
+                for message in messages[::-1][:4]
+            ]
+        )
+        + f"\nQuery: {user_message}"
+    )
+
+    print(prompt)
+
+    payload = {
+        "model": task_model_id,
+        "messages": [
+            {"role": "system", "content": content},
+            {"role": "user", "content": f"Query: {prompt}"},
+        ],
+        "stream": False,
+    }
+
+    try:
+        payload = filter_pipeline(payload, user)
+    except Exception as e:
+        raise e
+
+    model = app.state.MODELS[task_model_id]
 
-# app.add_middleware(SecurityHeadersMiddleware)
+    response = None
+    try:
+        if model["owned_by"] == "ollama":
+            response = await generate_ollama_chat_completion(
+                OpenAIChatCompletionForm(**payload), user=user
+            )
+        else:
+            response = await generate_openai_chat_completion(payload, user=user)
 
+        content = None
 
-class RAGMiddleware(BaseHTTPMiddleware):
+        if hasattr(response, "body_iterator"):
+            async for chunk in response.body_iterator:
+                data = json.loads(chunk.decode("utf-8"))
+                content = data["choices"][0]["message"]["content"]
+
+            # Cleanup any remaining background tasks if necessary
+            if response.background is not None:
+                await response.background()
+        else:
+            content = response["choices"][0]["message"]["content"]
+
+        # Parse the function response
+        if content is not None:
+            print(f"content: {content}")
+            result = json.loads(content)
+            print(result)
+
+            # Call the function
+            if "name" in result:
+                if tool_id in webui_app.state.TOOLS:
+                    toolkit_module = webui_app.state.TOOLS[tool_id]
+                else:
+                    toolkit_module = load_toolkit_module_by_id(tool_id)
+                    webui_app.state.TOOLS[tool_id] = toolkit_module
+
+                function = getattr(toolkit_module, result["name"])
+                function_result = None
+                try:
+                    # Get the signature of the function
+                    sig = inspect.signature(function)
+                    # Check if '__user__' is a parameter of the function
+                    if "__user__" in sig.parameters:
+                        # Call the function with the '__user__' parameter included
+                        function_result = function(
+                            **{
+                                **result["parameters"],
+                                "__user__": {
+                                    "id": user.id,
+                                    "email": user.email,
+                                    "name": user.name,
+                                    "role": user.role,
+                                },
+                            }
+                        )
+                    else:
+                        # Call the function without modifying the parameters
+                        function_result = function(**result["parameters"])
+                except Exception as e:
+                    print(e)
+
+                # Add the function result to the system prompt
+                if function_result:
+                    return function_result
+    except Exception as e:
+        print(f"Error: {e}")
+
+    return None
+
+
+class ChatCompletionMiddleware(BaseHTTPMiddleware):
     async def dispatch(self, request: Request, call_next):
         return_citations = False
 
@@ -171,35 +306,98 @@ class RAGMiddleware(BaseHTTPMiddleware):
             # Parse string to JSON
             data = json.loads(body_str) if body_str else {}
 
+            user = get_current_user(
+                get_http_authorization_cred(request.headers.get("Authorization"))
+            )
+
+            # Remove the citations from the body
             return_citations = data.get("citations", False)
             if "citations" in data:
                 del data["citations"]
 
-            # Example: Add a new key-value pair or modify existing ones
-            # data["modified"] = True  # Example modification
+            # Set the task model
+            task_model_id = data["model"]
+            if task_model_id not in app.state.MODELS:
+                raise HTTPException(
+                    status_code=status.HTTP_404_NOT_FOUND,
+                    detail="Model not found",
+                )
+
+            # Check if the user has a custom task model
+            # If the user has a custom task model, use that model
+            if app.state.MODELS[task_model_id]["owned_by"] == "ollama":
+                if (
+                    app.state.config.TASK_MODEL
+                    and app.state.config.TASK_MODEL in app.state.MODELS
+                ):
+                    task_model_id = app.state.config.TASK_MODEL
+            else:
+                if (
+                    app.state.config.TASK_MODEL_EXTERNAL
+                    and app.state.config.TASK_MODEL_EXTERNAL in app.state.MODELS
+                ):
+                    task_model_id = app.state.config.TASK_MODEL_EXTERNAL
+
+            prompt = get_last_user_message(data["messages"])
+            context = ""
+
+            # If tool_ids field is present, call the functions
+            if "tool_ids" in data:
+                print(data["tool_ids"])
+                for tool_id in data["tool_ids"]:
+                    print(tool_id)
+                    try:
+                        response = await get_function_call_response(
+                            messages=data["messages"],
+                            tool_id=tool_id,
+                            template=app.state.config.TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE,
+                            task_model_id=task_model_id,
+                            user=user,
+                        )
+
+                        if response:
+                            context += ("\n" if context != "" else "") + response
+                    except Exception as e:
+                        print(f"Error: {e}")
+                del data["tool_ids"]
+
+                print(f"tool_context: {context}")
+
+            # If docs field is present, generate RAG completions
             if "docs" in data:
                 data = {**data}
-                data["messages"], citations = rag_messages(
+                rag_context, citations = get_rag_context(
                     docs=data["docs"],
                     messages=data["messages"],
-                    template=rag_app.state.config.RAG_TEMPLATE,
                     embedding_function=rag_app.state.EMBEDDING_FUNCTION,
                     k=rag_app.state.config.TOP_K,
                     reranking_function=rag_app.state.sentence_transformer_rf,
                     r=rag_app.state.config.RELEVANCE_THRESHOLD,
                     hybrid_search=rag_app.state.config.ENABLE_RAG_HYBRID_SEARCH,
                 )
+
+                if rag_context:
+                    context += ("\n" if context != "" else "") + rag_context
+
                 del data["docs"]
 
-                log.debug(
-                    f"data['messages']: {data['messages']}, citations: {citations}"
+                log.debug(f"rag_context: {rag_context}, citations: {citations}")
+
+            if context != "":
+                system_prompt = rag_template(
+                    rag_app.state.config.RAG_TEMPLATE, context, prompt
+                )
+
+                print(system_prompt)
+
+                data["messages"] = add_or_update_system_message(
+                    f"\n{system_prompt}", data["messages"]
                 )
 
             modified_body_bytes = json.dumps(data).encode("utf-8")
 
             # Replace the request body with the modified one
             request._body = modified_body_bytes
-
             # Set custom header to ensure content-length matches new body length
             request.headers.__dict__["_list"] = [
                 (b"content-length", str(len(modified_body_bytes)).encode("utf-8")),
@@ -242,7 +440,80 @@ class RAGMiddleware(BaseHTTPMiddleware):
             yield data
 
 
-app.add_middleware(RAGMiddleware)
+app.add_middleware(ChatCompletionMiddleware)
+
+
+def filter_pipeline(payload, user):
+    user = {"id": user.id, "name": user.name, "role": user.role}
+    model_id = payload["model"]
+    filters = [
+        model
+        for model in app.state.MODELS.values()
+        if "pipeline" in model
+        and "type" in model["pipeline"]
+        and model["pipeline"]["type"] == "filter"
+        and (
+            model["pipeline"]["pipelines"] == ["*"]
+            or any(
+                model_id == target_model_id
+                for target_model_id in model["pipeline"]["pipelines"]
+            )
+        )
+    ]
+    sorted_filters = sorted(filters, key=lambda x: x["pipeline"]["priority"])
+
+    model = app.state.MODELS[model_id]
+
+    if "pipeline" in model:
+        sorted_filters.append(model)
+
+    for filter in sorted_filters:
+        r = None
+        try:
+            urlIdx = filter["urlIdx"]
+
+            url = openai_app.state.config.OPENAI_API_BASE_URLS[urlIdx]
+            key = openai_app.state.config.OPENAI_API_KEYS[urlIdx]
+
+            if key != "":
+                headers = {"Authorization": f"Bearer {key}"}
+                r = requests.post(
+                    f"{url}/{filter['id']}/filter/inlet",
+                    headers=headers,
+                    json={
+                        "user": user,
+                        "body": payload,
+                    },
+                )
+
+                r.raise_for_status()
+                payload = r.json()
+        except Exception as e:
+            # Handle connection error here
+            print(f"Connection error: {e}")
+
+            if r is not None:
+                try:
+                    res = r.json()
+                except:
+                    pass
+                if "detail" in res:
+                    raise Exception(r.status_code, res["detail"])
+
+            else:
+                pass
+
+    if "pipeline" not in app.state.MODELS[model_id]:
+        if "chat_id" in payload:
+            del payload["chat_id"]
+
+        if "title" in payload:
+            del payload["title"]
+
+        if "task" in payload:
+            del payload["task"]
+
+    return payload
 
 
 class PipelineMiddleware(BaseHTTPMiddleware):
@@ -260,85 +531,17 @@ class PipelineMiddleware(BaseHTTPMiddleware):
             # Parse string to JSON
             data = json.loads(body_str) if body_str else {}
 
-            model_id = data["model"]
-            filters = [
-                model
-                for model in app.state.MODELS.values()
-                if "pipeline" in model
-                and "type" in model["pipeline"]
-                and model["pipeline"]["type"] == "filter"
-                and (
-                    model["pipeline"]["pipelines"] == ["*"]
-                    or any(
-                        model_id == target_model_id
-                        for target_model_id in model["pipeline"]["pipelines"]
-                    )
-                )
-            ]
-            sorted_filters = sorted(filters, key=lambda x: x["pipeline"]["priority"])
-
-            user = None
-            if len(sorted_filters) > 0:
-                try:
-                    user = get_current_user(
-                        get_http_authorization_cred(
-                            request.headers.get("Authorization")
-                        )
-                    )
-                    user = {"id": user.id, "name": user.name, "role": user.role}
-                except:
-                    pass
-
-            model = app.state.MODELS[model_id]
-
-            if "pipeline" in model:
-                sorted_filters.append(model)
-
-            for filter in sorted_filters:
-                r = None
-                try:
-                    urlIdx = filter["urlIdx"]
-
-                    url = openai_app.state.config.OPENAI_API_BASE_URLS[urlIdx]
-                    key = openai_app.state.config.OPENAI_API_KEYS[urlIdx]
-
-                    if key != "":
-                        headers = {"Authorization": f"Bearer {key}"}
-                        r = requests.post(
-                            f"{url}/{filter['id']}/filter/inlet",
-                            headers=headers,
-                            json={
-                                "user": user,
-                                "body": data,
-                            },
-                        )
-
-                        r.raise_for_status()
-                        data = r.json()
-                except Exception as e:
-                    # Handle connection error here
-                    print(f"Connection error: {e}")
-
-                    if r is not None:
-                        try:
-                            res = r.json()
-                            if "detail" in res:
-                                return JSONResponse(
-                                    status_code=r.status_code,
-                                    content=res,
-                                )
-                        except:
-                            pass
-
-                    else:
-                        pass
-
-            if "pipeline" not in app.state.MODELS[model_id]:
-                if "chat_id" in data:
-                    del data["chat_id"]
+            user = get_current_user(
+                get_http_authorization_cred(request.headers.get("Authorization"))
+            )
 
-                if "title" in data:
-                    del data["title"]
+            try:
+                data = filter_pipeline(data, user)
+            except Exception as e:
+                return JSONResponse(
+                    status_code=e.args[0],
+                    content={"detail": e.args[1]},
+                )
 
             modified_body_bytes = json.dumps(data).encode("utf-8")
             # Replace the request body with the modified one
@@ -499,6 +702,302 @@ async def get_models(user=Depends(get_verified_user)):
     return {"data": models}
 
 
+@app.get("/api/task/config")
+async def get_task_config(user=Depends(get_verified_user)):
+    return {
+        "TASK_MODEL": app.state.config.TASK_MODEL,
+        "TASK_MODEL_EXTERNAL": app.state.config.TASK_MODEL_EXTERNAL,
+        "TITLE_GENERATION_PROMPT_TEMPLATE": app.state.config.TITLE_GENERATION_PROMPT_TEMPLATE,
+        "SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE": app.state.config.SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE,
+        "SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD": app.state.config.SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD,
+        "TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE": app.state.config.TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE,
+    }
+
+
+class TaskConfigForm(BaseModel):
+    TASK_MODEL: Optional[str]
+    TASK_MODEL_EXTERNAL: Optional[str]
+    TITLE_GENERATION_PROMPT_TEMPLATE: str
+    SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE: str
+    SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD: int
+    TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE: str
+
+
+@app.post("/api/task/config/update")
+async def update_task_config(form_data: TaskConfigForm, user=Depends(get_admin_user)):
+    app.state.config.TASK_MODEL = form_data.TASK_MODEL
+    app.state.config.TASK_MODEL_EXTERNAL = form_data.TASK_MODEL_EXTERNAL
+    app.state.config.TITLE_GENERATION_PROMPT_TEMPLATE = (
+        form_data.TITLE_GENERATION_PROMPT_TEMPLATE
+    )
+    app.state.config.SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE = (
+        form_data.SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE
+    )
+    app.state.config.SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD = (
+        form_data.SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD
+    )
+    app.state.config.TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE = (
+        form_data.TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE
+    )
+
+    return {
+        "TASK_MODEL": app.state.config.TASK_MODEL,
+        "TASK_MODEL_EXTERNAL": app.state.config.TASK_MODEL_EXTERNAL,
+        "TITLE_GENERATION_PROMPT_TEMPLATE": app.state.config.TITLE_GENERATION_PROMPT_TEMPLATE,
+        "SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE": app.state.config.SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE,
+        "SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD": app.state.config.SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD,
+        "TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE": app.state.config.TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE,
+    }
+
+
+@app.post("/api/task/title/completions")
+async def generate_title(form_data: dict, user=Depends(get_verified_user)):
+    print("generate_title")
+
+    model_id = form_data["model"]
+    if model_id not in app.state.MODELS:
+        raise HTTPException(
+            status_code=status.HTTP_404_NOT_FOUND,
+            detail="Model not found",
+        )
+
+    # Check if the user has a custom task model
+    # If the user has a custom task model, use that model
+    if app.state.MODELS[model_id]["owned_by"] == "ollama":
+        if app.state.config.TASK_MODEL:
+            task_model_id = app.state.config.TASK_MODEL
+            if task_model_id in app.state.MODELS:
+                model_id = task_model_id
+    else:
+        if app.state.config.TASK_MODEL_EXTERNAL:
+            task_model_id = app.state.config.TASK_MODEL_EXTERNAL
+            if task_model_id in app.state.MODELS:
+                model_id = task_model_id
+
+    print(model_id)
+    model = app.state.MODELS[model_id]
+
+    template = app.state.config.TITLE_GENERATION_PROMPT_TEMPLATE
+
+    content = title_generation_template(
+        template, form_data["prompt"], user.model_dump()
+    )
+
+    payload = {
+        "model": model_id,
+        "messages": [{"role": "user", "content": content}],
+        "stream": False,
+        "max_tokens": 50,
+        "chat_id": form_data.get("chat_id", None),
+        "title": True,
+    }
+
+    print(payload)
+
+    try:
+        payload = filter_pipeline(payload, user)
+    except Exception as e:
+        return JSONResponse(
+            status_code=e.args[0],
+            content={"detail": e.args[1]},
+        )
+
+    if model["owned_by"] == "ollama":
+        return await generate_ollama_chat_completion(
+            OpenAIChatCompletionForm(**payload), user=user
+        )
+    else:
+        return await generate_openai_chat_completion(payload, user=user)
+
+
+@app.post("/api/task/query/completions")
+async def generate_search_query(form_data: dict, user=Depends(get_verified_user)):
+    print("generate_search_query")
+
+    if len(form_data["prompt"]) < app.state.config.SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail=f"Skip search query generation for short prompts (< {app.state.config.SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD} characters)",
+        )
+
+    model_id = form_data["model"]
+    if model_id not in app.state.MODELS:
+        raise HTTPException(
+            status_code=status.HTTP_404_NOT_FOUND,
+            detail="Model not found",
+        )
+
+    # Check if the user has a custom task model
+    # If the user has a custom task model, use that model
+    if app.state.MODELS[model_id]["owned_by"] == "ollama":
+        if app.state.config.TASK_MODEL:
+            task_model_id = app.state.config.TASK_MODEL
+            if task_model_id in app.state.MODELS:
+                model_id = task_model_id
+    else:
+        if app.state.config.TASK_MODEL_EXTERNAL:
+            task_model_id = app.state.config.TASK_MODEL_EXTERNAL
+            if task_model_id in app.state.MODELS:
+                model_id = task_model_id
+
+    print(model_id)
+    model = app.state.MODELS[model_id]
+
+    template = app.state.config.SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE
+
+    content = search_query_generation_template(
+        template, form_data["prompt"], user.model_dump()
+    )
+
+    payload = {
+        "model": model_id,
+        "messages": [{"role": "user", "content": content}],
+        "stream": False,
+        "max_tokens": 30,
+        "task": True,
+    }
+
+    print(payload)
+
+    try:
+        payload = filter_pipeline(payload, user)
+    except Exception as e:
+        return JSONResponse(
+            status_code=e.args[0],
+            content={"detail": e.args[1]},
+        )
+
+    if model["owned_by"] == "ollama":
+        return await generate_ollama_chat_completion(
+            OpenAIChatCompletionForm(**payload), user=user
+        )
+    else:
+        return await generate_openai_chat_completion(payload, user=user)
+
+
+@app.post("/api/task/emoji/completions")
+async def generate_emoji(form_data: dict, user=Depends(get_verified_user)):
+    print("generate_emoji")
+
+    model_id = form_data["model"]
+    if model_id not in app.state.MODELS:
+        raise HTTPException(
+            status_code=status.HTTP_404_NOT_FOUND,
+            detail="Model not found",
+        )
+
+    # Check if the user has a custom task model
+    # If the user has a custom task model, use that model
+    if app.state.MODELS[model_id]["owned_by"] == "ollama":
+        if app.state.config.TASK_MODEL:
+            task_model_id = app.state.config.TASK_MODEL
+            if task_model_id in app.state.MODELS:
+                model_id = task_model_id
+    else:
+        if app.state.config.TASK_MODEL_EXTERNAL:
+            task_model_id = app.state.config.TASK_MODEL_EXTERNAL
+            if task_model_id in app.state.MODELS:
+                model_id = task_model_id
+
+    print(model_id)
+    model = app.state.MODELS[model_id]
+
+    template = '''
+Your task is to reflect the speaker's likely facial expression through a fitting emoji. Interpret emotions from the message and reflect their facial expression using fitting, diverse emojis (e.g., 😊, 😢, 😡, 😱).
+
+Message: """{{prompt}}"""
+'''
+
+    content = title_generation_template(
+        template, form_data["prompt"], user.model_dump()
+    )
+
+    payload = {
+        "model": model_id,
+        "messages": [{"role": "user", "content": content}],
+        "stream": False,
+        "max_tokens": 4,
+        "chat_id": form_data.get("chat_id", None),
+        "task": True,
+    }
+
+    print(payload)
+
+    try:
+        payload = filter_pipeline(payload, user)
+    except Exception as e:
+        return JSONResponse(
+            status_code=e.args[0],
+            content={"detail": e.args[1]},
+        )
+
+    if model["owned_by"] == "ollama":
+        return await generate_ollama_chat_completion(
+            OpenAIChatCompletionForm(**payload), user=user
+        )
+    else:
+        return await generate_openai_chat_completion(payload, user=user)
+
+
+@app.post("/api/task/tools/completions")
+async def get_tools_function_calling(form_data: dict, user=Depends(get_verified_user)):
+    print("get_tools_function_calling")
+
+    model_id = form_data["model"]
+    if model_id not in app.state.MODELS:
+        raise HTTPException(
+            status_code=status.HTTP_404_NOT_FOUND,
+            detail="Model not found",
+        )
+
+    # Check if the user has a custom task model
+    # If the user has a custom task model, use that model
+    if app.state.MODELS[model_id]["owned_by"] == "ollama":
+        if app.state.config.TASK_MODEL:
+            task_model_id = app.state.config.TASK_MODEL
+            if task_model_id in app.state.MODELS:
+                model_id = task_model_id
+    else:
+        if app.state.config.TASK_MODEL_EXTERNAL:
+            task_model_id = app.state.config.TASK_MODEL_EXTERNAL
+            if task_model_id in app.state.MODELS:
+                model_id = task_model_id
+
+    print(model_id)
+    template = app.state.config.TOOLS_FUNCTION_CALLING_PROMPT_TEMPLATE
+
+    try:
+        context = await get_function_call_response(
+            form_data["messages"], form_data["tool_id"], template, model_id, user
+        )
+        return context
+    except Exception as e:
+        return JSONResponse(
+            status_code=e.args[0],
+            content={"detail": e.args[1]},
+        )
+
+
+@app.post("/api/chat/completions")
+async def generate_chat_completions(form_data: dict, user=Depends(get_verified_user)):
+    model_id = form_data["model"]
+    if model_id not in app.state.MODELS:
+        raise HTTPException(
+            status_code=status.HTTP_404_NOT_FOUND,
+            detail="Model not found",
+        )
+
+    model = app.state.MODELS[model_id]
+    print(model)
+
+    if model["owned_by"] == "ollama":
+        return await generate_ollama_chat_completion(
+            OpenAIChatCompletionForm(**form_data), user=user
+        )
+    else:
+        return await generate_openai_chat_completion(form_data, user=user)
+
+
 @app.post("/api/chat/completed")
 async def chat_completed(form_data: dict, user=Depends(get_verified_user)):
     data = form_data
@@ -591,6 +1090,63 @@ async def get_pipelines_list(user=Depends(get_admin_user)):
     }
 
 
+@app.post("/api/pipelines/upload")
+async def upload_pipeline(
+    urlIdx: int = Form(...), file: UploadFile = File(...), user=Depends(get_admin_user)
+):
+    print("upload_pipeline", urlIdx, file.filename)
+    # Check if the uploaded file is a python file
+    if not file.filename.endswith(".py"):
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail="Only Python (.py) files are allowed.",
+        )
+
+    upload_folder = f"{CACHE_DIR}/pipelines"
+    os.makedirs(upload_folder, exist_ok=True)
+    file_path = os.path.join(upload_folder, file.filename)
+
+    try:
+        # Save the uploaded file
+        with open(file_path, "wb") as buffer:
+            shutil.copyfileobj(file.file, buffer)
+
+        url = openai_app.state.config.OPENAI_API_BASE_URLS[urlIdx]
+        key = openai_app.state.config.OPENAI_API_KEYS[urlIdx]
+
+        headers = {"Authorization": f"Bearer {key}"}
+
+        with open(file_path, "rb") as f:
+            files = {"file": f}
+            r = requests.post(f"{url}/pipelines/upload", headers=headers, files=files)
+
+        r.raise_for_status()
+        data = r.json()
+
+        return {**data}
+    except Exception as e:
+        # Handle connection error here
+        print(f"Connection error: {e}")
+
+        detail = "Pipeline not found"
+        if r is not None:
+            try:
+                res = r.json()
+                if "detail" in res:
+                    detail = res["detail"]
+            except:
+                pass
+
+        raise HTTPException(
+            status_code=(r.status_code if r is not None else status.HTTP_404_NOT_FOUND),
+            detail=detail,
+        )
+    finally:
+        # Ensure the file is deleted after the upload is completed or on failure
+        if os.path.exists(file_path):
+            os.remove(file_path)
+
+
 class AddPipelineForm(BaseModel):
     url: str
     urlIdx: int
@@ -857,6 +1413,15 @@ async def get_app_config():
             "enable_community_sharing": webui_app.state.config.ENABLE_COMMUNITY_SHARING,
             "enable_admin_export": ENABLE_ADMIN_EXPORT,
         },
+        "audio": {
+            "tts": {
+                "engine": audio_app.state.config.TTS_ENGINE,
+                "voice": audio_app.state.config.TTS_VOICE,
+            },
+            "stt": {
+                "engine": audio_app.state.config.STT_ENGINE,
+            },
+        },
         "oauth": {
             "providers": {
                 name: config.get("name", name)
@@ -925,7 +1490,7 @@ async def get_app_changelog():
 @app.get("/api/version/updates")
 async def get_app_latest_release_version():
     try:
-        async with aiohttp.ClientSession() as session:
+        async with aiohttp.ClientSession(trust_env=True) as session:
             async with session.get(
                 "https://api.github.com/repos/open-webui/open-webui/releases/latest"
             ) as response:

backend/requirements.txt

@@ -57,4 +57,8 @@ authlib==1.3.0
 black==24.4.2
 langfuse==2.33.0
 youtube-transcript-api==0.6.2
-pytube==15.0.0
+pytube==15.0.0
+
+extract_msg
+pydub
+duckduckgo-search~=6.1.5

backend/start.sh

@@ -20,12 +20,12 @@ if test "$WEBUI_SECRET_KEY $WEBUI_JWT_SECRET_KEY" = " "; then
   WEBUI_SECRET_KEY=$(cat "$KEY_FILE")
 fi
 
-if [ "$USE_OLLAMA_DOCKER" = "true" ]; then
+if [[ "${USE_OLLAMA_DOCKER,,}" == "true" ]]; then
     echo "USE_OLLAMA is set to true, starting ollama serve."
     ollama serve &
 fi
 
-if [ "$USE_CUDA_DOCKER" = "true" ]; then
+if [[ "${USE_CUDA_DOCKER,,}" == "true" ]]; then
   echo "CUDA is enabled, appending LD_LIBRARY_PATH to include torch/cudnn & cublas libraries."
   export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib/python3.11/site-packages/torch/lib:/usr/local/lib/python3.11/site-packages/nvidia/cudnn/lib"
 fi

backend/start_windows.bat

@@ -8,6 +8,7 @@ cd /d "%SCRIPT_DIR%" || exit /b
 
 SET "KEY_FILE=.webui_secret_key"
 IF "%PORT%"=="" SET PORT=8080
+IF "%HOST%"=="" SET HOST=0.0.0.0
 SET "WEBUI_SECRET_KEY=%WEBUI_SECRET_KEY%"
 SET "WEBUI_JWT_SECRET_KEY=%WEBUI_JWT_SECRET_KEY%"
 
@@ -29,4 +30,4 @@ IF "%WEBUI_SECRET_KEY%%WEBUI_JWT_SECRET_KEY%" == " " (
 
 :: Execute uvicorn
 SET "WEBUI_SECRET_KEY=%WEBUI_SECRET_KEY%"
-uvicorn main:app --host 0.0.0.0 --port "%PORT%" --forwarded-allow-ips '*'
+uvicorn main:app --host "%HOST%" --port "%PORT%" --forwarded-allow-ips '*'

backend/utils/misc.py

@@ -3,7 +3,48 @@ import hashlib
 import json
 import re
 from datetime import timedelta
-from typing import Optional
+from typing import Optional, List
+
+
+def get_last_user_message(messages: List[dict]) -> str:
+    for message in reversed(messages):
+        if message["role"] == "user":
+            if isinstance(message["content"], list):
+                for item in message["content"]:
+                    if item["type"] == "text":
+                        return item["text"]
+            return message["content"]
+    return None
+
+
+def get_last_assistant_message(messages: List[dict]) -> str:
+    for message in reversed(messages):
+        if message["role"] == "assistant":
+            if isinstance(message["content"], list):
+                for item in message["content"]:
+                    if item["type"] == "text":
+                        return item["text"]
+            return message["content"]
+    return None
+
+
+def add_or_update_system_message(content: str, messages: List[dict]):
+    """
+    Adds a new system message at the beginning of the messages list
+    or updates the existing system message at the beginning.
+
+    :param msg: The message to be added or appended.
+    :param messages: The list of message dictionaries.
+    :return: The updated list of message dictionaries.
+    """
+
+    if messages and messages[0].get("role") == "system":
+        messages[0]["content"] += f"{content}\n{messages[0]['content']}"
+    else:
+        # Insert at the beginning
+        messages.insert(0, {"role": "system", "content": content})
+
+    return messages
 
 
 def get_gravatar_url(email):
@@ -193,8 +234,14 @@ def parse_ollama_modelfile(model_text):
     system_desc_match = re.search(
         r'SYSTEM\s+"""(.+?)"""', model_text, re.DOTALL | re.IGNORECASE
     )
+    system_desc_match_single = re.search(
+        r"SYSTEM\s+([^\n]+)", model_text, re.IGNORECASE
+    )
+
     if system_desc_match:
         data["params"]["system"] = system_desc_match.group(1).strip()
+    elif system_desc_match_single:
+        data["params"]["system"] = system_desc_match_single.group(1).strip()
 
     # Parse messages
     messages = []

backend/utils/models.py

@@ -1,10 +0,0 @@
-from apps.webui.models.models import Models, ModelModel, ModelForm, ModelResponse
-
-
-def get_model_id_from_custom_model_id(id: str):
-    model = Models.get_model_by_id(id)
-
-    if model:
-        return model.id
-    else:
-        return id

backend/utils/task.py

@@ -0,0 +1,117 @@
+import re
+import math
+
+from datetime import datetime
+from typing import Optional
+
+
+def prompt_template(
+    template: str, user_name: str = None, current_location: str = None
+) -> str:
+    # Get the current date
+    current_date = datetime.now()
+
+    # Format the date to YYYY-MM-DD
+    formatted_date = current_date.strftime("%Y-%m-%d")
+
+    # Replace {{CURRENT_DATE}} in the template with the formatted date
+    template = template.replace("{{CURRENT_DATE}}", formatted_date)
+
+    if user_name:
+        # Replace {{USER_NAME}} in the template with the user's name
+        template = template.replace("{{USER_NAME}}", user_name)
+
+    if current_location:
+        # Replace {{CURRENT_LOCATION}} in the template with the current location
+        template = template.replace("{{CURRENT_LOCATION}}", current_location)
+
+    return template
+
+
+def title_generation_template(
+    template: str, prompt: str, user: Optional[dict] = None
+) -> str:
+    def replacement_function(match):
+        full_match = match.group(0)
+        start_length = match.group(1)
+        end_length = match.group(2)
+        middle_length = match.group(3)
+
+        if full_match == "{{prompt}}":
+            return prompt
+        elif start_length is not None:
+            return prompt[: int(start_length)]
+        elif end_length is not None:
+            return prompt[-int(end_length) :]
+        elif middle_length is not None:
+            middle_length = int(middle_length)
+            if len(prompt) <= middle_length:
+                return prompt
+            start = prompt[: math.ceil(middle_length / 2)]
+            end = prompt[-math.floor(middle_length / 2) :]
+            return f"{start}...{end}"
+        return ""
+
+    template = re.sub(
+        r"{{prompt}}|{{prompt:start:(\d+)}}|{{prompt:end:(\d+)}}|{{prompt:middletruncate:(\d+)}}",
+        replacement_function,
+        template,
+    )
+
+    template = prompt_template(
+        template,
+        **(
+            {"user_name": user.get("name"), "current_location": user.get("location")}
+            if user
+            else {}
+        ),
+    )
+
+    return template
+
+
+def search_query_generation_template(
+    template: str, prompt: str, user: Optional[dict] = None
+) -> str:
+
+    def replacement_function(match):
+        full_match = match.group(0)
+        start_length = match.group(1)
+        end_length = match.group(2)
+        middle_length = match.group(3)
+
+        if full_match == "{{prompt}}":
+            return prompt
+        elif start_length is not None:
+            return prompt[: int(start_length)]
+        elif end_length is not None:
+            return prompt[-int(end_length) :]
+        elif middle_length is not None:
+            middle_length = int(middle_length)
+            if len(prompt) <= middle_length:
+                return prompt
+            start = prompt[: math.ceil(middle_length / 2)]
+            end = prompt[-math.floor(middle_length / 2) :]
+            return f"{start}...{end}"
+        return ""
+
+    template = re.sub(
+        r"{{prompt}}|{{prompt:start:(\d+)}}|{{prompt:end:(\d+)}}|{{prompt:middletruncate:(\d+)}}",
+        replacement_function,
+        template,
+    )
+
+    template = prompt_template(
+        template,
+        **(
+            {"user_name": user.get("name"), "current_location": user.get("location")}
+            if user
+            else {}
+        ),
+    )
+    return template
+
+
+def tools_function_calling_generation_template(template: str, tools_specs: str) -> str:
+    template = template.replace("{{TOOLS}}", tools_specs)
+    return template

backend/utils/tools.py

@@ -0,0 +1,73 @@
+import inspect
+from typing import get_type_hints, List, Dict, Any
+
+
+def doc_to_dict(docstring):
+    lines = docstring.split("\n")
+    description = lines[1].strip()
+    param_dict = {}
+
+    for line in lines:
+        if ":param" in line:
+            line = line.replace(":param", "").strip()
+            param, desc = line.split(":", 1)
+            param_dict[param.strip()] = desc.strip()
+    ret_dict = {"description": description, "params": param_dict}
+    return ret_dict
+
+
+def get_tools_specs(tools) -> List[dict]:
+    function_list = [
+        {"name": func, "function": getattr(tools, func)}
+        for func in dir(tools)
+        if callable(getattr(tools, func)) and not func.startswith("__")
+    ]
+
+    specs = []
+    for function_item in function_list:
+        function_name = function_item["name"]
+        function = function_item["function"]
+
+        function_doc = doc_to_dict(function.__doc__ or function_name)
+        specs.append(
+            {
+                "name": function_name,
+                # TODO: multi-line desc?
+                "description": function_doc.get("description", function_name),
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        param_name: {
+                            "type": param_annotation.__name__.lower(),
+                            **(
+                                {
+                                    "enum": (
+                                        str(param_annotation.__args__)
+                                        if hasattr(param_annotation, "__args__")
+                                        else None
+                                    )
+                                }
+                                if hasattr(param_annotation, "__args__")
+                                else {}
+                            ),
+                            "description": function_doc.get("params", {}).get(
+                                param_name, param_name
+                            ),
+                        }
+                        for param_name, param_annotation in get_type_hints(
+                            function
+                        ).items()
+                        if param_name != "return" and param_name != "__user__"
+                    },
+                    "required": [
+                        name
+                        for name, param in inspect.signature(
+                            function
+                        ).parameters.items()
+                        if param.default is param.empty
+                    ],
+                },
+            }
+        )
+
+    return specs

cypress/e2e/settings.cy.ts

@@ -28,19 +28,6 @@ describe('Settings', () => {
 		});
 	});
 
-	context('Connections', () => {
-		it('user can open the Connections modal and hit save', () => {
-			cy.get('button').contains('Connections').click();
-			cy.get('button').contains('Save').click();
-		});
-	});
-
-	context('Models', () => {
-		it('user can open the Models modal', () => {
-			cy.get('button').contains('Models').click();
-		});
-	});
-
 	context('Interface', () => {
 		it('user can open the Interface modal and hit save', () => {
 			cy.get('button').contains('Interface').click();
@@ -55,14 +42,6 @@ describe('Settings', () => {
 		});
 	});
 
-	context('Images', () => {
-		it('user can open the Images modal and hit save', () => {
-			cy.get('button').contains('Images').click();
-			// Currently fails because the backend requires a valid URL
-			// cy.get('button').contains('Save').click();
-		});
-	});
-
 	context('Chats', () => {
 		it('user can open the Chats modal', () => {
 			cy.get('button').contains('Chats').click();

docs/CONTRIBUTING.md

@@ -41,7 +41,7 @@ Looking to contribute? Great! Here's how you can help:
 
 We welcome pull requests. Before submitting one, please:
 
-1. Discuss your idea or issue in the [issues section](https://github.com/open-webui/open-webui/issues).
+1. Open a discussion regarding your ideas [here](https://github.com/open-webui/open-webui/discussions/new/choose).
 2. Follow the project's coding standards and include tests for new features.
 3. Update documentation as necessary.
 4. Write clear, descriptive commit messages.

package-lock.json

@@ -1,17 +1,21 @@
 {
 	"name": "open-webui",
-	"version": "0.2.5",
+	"version": "0.3.4",
 	"lockfileVersion": 3,
 	"requires": true,
 	"packages": {
 		"": {
 			"name": "open-webui",
-			"version": "0.2.5",
+			"version": "0.3.4",
 			"dependencies": {
+				"@codemirror/lang-javascript": "^6.2.2",
+				"@codemirror/lang-python": "^6.1.6",
+				"@codemirror/theme-one-dark": "^6.1.2",
 				"@pyscript/core": "^0.4.32",
 				"@sveltejs/adapter-node": "^1.3.1",
 				"async": "^3.2.5",
 				"bits-ui": "^0.19.7",
+				"codemirror": "^6.0.1",
 				"dayjs": "^1.11.10",
 				"eventsource-parser": "^1.1.2",
 				"file-saver": "^2.0.5",
@@ -108,6 +112,119 @@
 			"resolved": "https://registry.npmjs.org/@braintree/sanitize-url/-/sanitize-url-6.0.4.tgz",
 			"integrity": "sha512-s3jaWicZd0pkP0jf5ysyHUI/RE7MHos6qlToFcGWXVp+ykHOy77OUMrfbgJ9it2C5bow7OIQwYYaHjk9XlBQ2A=="
 		},
+		"node_modules/@codemirror/autocomplete": {
+			"version": "6.16.2",
+			"resolved": "https://registry.npmjs.org/@codemirror/autocomplete/-/autocomplete-6.16.2.tgz",
+			"integrity": "sha512-MjfDrHy0gHKlPWsvSsikhO1+BOh+eBHNgfH1OXs1+DAf30IonQldgMM3kxLDTG9ktE7kDLaA1j/l7KMPA4KNfw==",
+			"dependencies": {
+				"@codemirror/language": "^6.0.0",
+				"@codemirror/state": "^6.0.0",
+				"@codemirror/view": "^6.17.0",
+				"@lezer/common": "^1.0.0"
+			},
+			"peerDependencies": {
+				"@codemirror/language": "^6.0.0",
+				"@codemirror/state": "^6.0.0",
+				"@codemirror/view": "^6.0.0",
+				"@lezer/common": "^1.0.0"
+			}
+		},
+		"node_modules/@codemirror/commands": {
+			"version": "6.6.0",
+			"resolved": "https://registry.npmjs.org/@codemirror/commands/-/commands-6.6.0.tgz",
+			"integrity": "sha512-qnY+b7j1UNcTS31Eenuc/5YJB6gQOzkUoNmJQc0rznwqSRpeaWWpjkWy2C/MPTcePpsKJEM26hXrOXl1+nceXg==",
+			"dependencies": {
+				"@codemirror/language": "^6.0.0",
+				"@codemirror/state": "^6.4.0",
+				"@codemirror/view": "^6.27.0",
+				"@lezer/common": "^1.1.0"
+			}
+		},
+		"node_modules/@codemirror/lang-javascript": {
+			"version": "6.2.2",
+			"resolved": "https://registry.npmjs.org/@codemirror/lang-javascript/-/lang-javascript-6.2.2.tgz",
+			"integrity": "sha512-VGQfY+FCc285AhWuwjYxQyUQcYurWlxdKYT4bqwr3Twnd5wP5WSeu52t4tvvuWmljT4EmgEgZCqSieokhtY8hg==",
+			"dependencies": {
+				"@codemirror/autocomplete": "^6.0.0",
+				"@codemirror/language": "^6.6.0",
+				"@codemirror/lint": "^6.0.0",
+				"@codemirror/state": "^6.0.0",
+				"@codemirror/view": "^6.17.0",
+				"@lezer/common": "^1.0.0",
+				"@lezer/javascript": "^1.0.0"
+			}
+		},
+		"node_modules/@codemirror/lang-python": {
+			"version": "6.1.6",
+			"resolved": "https://registry.npmjs.org/@codemirror/lang-python/-/lang-python-6.1.6.tgz",
+			"integrity": "sha512-ai+01WfZhWqM92UqjnvorkxosZ2aq2u28kHvr+N3gu012XqY2CThD67JPMHnGceRfXPDBmn1HnyqowdpF57bNg==",
+			"dependencies": {
+				"@codemirror/autocomplete": "^6.3.2",
+				"@codemirror/language": "^6.8.0",
+				"@codemirror/state": "^6.0.0",
+				"@lezer/common": "^1.2.1",
+				"@lezer/python": "^1.1.4"
+			}
+		},
+		"node_modules/@codemirror/language": {
+			"version": "6.10.2",
+			"resolved": "https://registry.npmjs.org/@codemirror/language/-/language-6.10.2.tgz",
+			"integrity": "sha512-kgbTYTo0Au6dCSc/TFy7fK3fpJmgHDv1sG1KNQKJXVi+xBTEeBPY/M30YXiU6mMXeH+YIDLsbrT4ZwNRdtF+SA==",
+			"dependencies": {
+				"@codemirror/state": "^6.0.0",
+				"@codemirror/view": "^6.23.0",
+				"@lezer/common": "^1.1.0",
+				"@lezer/highlight": "^1.0.0",
+				"@lezer/lr": "^1.0.0",
+				"style-mod": "^4.0.0"
+			}
+		},
+		"node_modules/@codemirror/lint": {
+			"version": "6.8.0",
+			"resolved": "https://registry.npmjs.org/@codemirror/lint/-/lint-6.8.0.tgz",
+			"integrity": "sha512-lsFofvaw0lnPRJlQylNsC4IRt/1lI4OD/yYslrSGVndOJfStc58v+8p9dgGiD90ktOfL7OhBWns1ZETYgz0EJA==",
+			"dependencies": {
+				"@codemirror/state": "^6.0.0",
+				"@codemirror/view": "^6.0.0",
+				"crelt": "^1.0.5"
+			}
+		},
+		"node_modules/@codemirror/search": {
+			"version": "6.5.6",
+			"resolved": "https://registry.npmjs.org/@codemirror/search/-/search-6.5.6.tgz",
+			"integrity": "sha512-rpMgcsh7o0GuCDUXKPvww+muLA1pDJaFrpq/CCHtpQJYz8xopu4D1hPcKRoDD0YlF8gZaqTNIRa4VRBWyhyy7Q==",
+			"dependencies": {
+				"@codemirror/state": "^6.0.0",
+				"@codemirror/view": "^6.0.0",
+				"crelt": "^1.0.5"
+			}
+		},
+		"node_modules/@codemirror/state": {
+			"version": "6.4.1",
+			"resolved": "https://registry.npmjs.org/@codemirror/state/-/state-6.4.1.tgz",
+			"integrity": "sha512-QkEyUiLhsJoZkbumGZlswmAhA7CBU02Wrz7zvH4SrcifbsqwlXShVXg65f3v/ts57W3dqyamEriMhij1Z3Zz4A=="
+		},
+		"node_modules/@codemirror/theme-one-dark": {
+			"version": "6.1.2",
+			"resolved": "https://registry.npmjs.org/@codemirror/theme-one-dark/-/theme-one-dark-6.1.2.tgz",
+			"integrity": "sha512-F+sH0X16j/qFLMAfbciKTxVOwkdAS336b7AXTKOZhy8BR3eH/RelsnLgLFINrpST63mmN2OuwUt0W2ndUgYwUA==",
+			"dependencies": {
+				"@codemirror/language": "^6.0.0",
+				"@codemirror/state": "^6.0.0",
+				"@codemirror/view": "^6.0.0",
+				"@lezer/highlight": "^1.0.0"
+			}
+		},
+		"node_modules/@codemirror/view": {
+			"version": "6.28.0",
+			"resolved": "https://registry.npmjs.org/@codemirror/view/-/view-6.28.0.tgz",
+			"integrity": "sha512-fo7CelaUDKWIyemw4b+J57cWuRkOu4SWCCPfNDkPvfWkGjM9D5racHQXr4EQeYCD6zEBIBxGCeaKkQo+ysl0gA==",
+			"dependencies": {
+				"@codemirror/state": "^6.4.0",
+				"style-mod": "^4.1.0",
+				"w3c-keyname": "^2.2.4"
+			}
+		},
 		"node_modules/@colors/colors": {
 			"version": "1.5.0",
 			"resolved": "https://registry.npmjs.org/@colors/colors/-/colors-1.5.0.tgz",
@@ -825,6 +942,47 @@
 				"@jridgewell/sourcemap-codec": "^1.4.14"
 			}
 		},
+		"node_modules/@lezer/common": {
+			"version": "1.2.1",
+			"resolved": "https://registry.npmjs.org/@lezer/common/-/common-1.2.1.tgz",
+			"integrity": "sha512-yemX0ZD2xS/73llMZIK6KplkjIjf2EvAHcinDi/TfJ9hS25G0388+ClHt6/3but0oOxinTcQHJLDXh6w1crzFQ=="
+		},
+		"node_modules/@lezer/highlight": {
+			"version": "1.2.0",
+			"resolved": "https://registry.npmjs.org/@lezer/highlight/-/highlight-1.2.0.tgz",
+			"integrity": "sha512-WrS5Mw51sGrpqjlh3d4/fOwpEV2Hd3YOkp9DBt4k8XZQcoTHZFB7sx030A6OcahF4J1nDQAa3jXlTVVYH50IFA==",
+			"dependencies": {
+				"@lezer/common": "^1.0.0"
+			}
+		},
+		"node_modules/@lezer/javascript": {
+			"version": "1.4.16",
+			"resolved": "https://registry.npmjs.org/@lezer/javascript/-/javascript-1.4.16.tgz",
+			"integrity": "sha512-84UXR3N7s11MPQHWgMnjb9571fr19MmXnr5zTv2XX0gHXXUvW3uPJ8GCjKrfTXmSdfktjRK0ayKklw+A13rk4g==",
+			"dependencies": {
+				"@lezer/common": "^1.2.0",
+				"@lezer/highlight": "^1.1.3",
+				"@lezer/lr": "^1.3.0"
+			}
+		},
+		"node_modules/@lezer/lr": {
+			"version": "1.4.1",
+			"resolved": "https://registry.npmjs.org/@lezer/lr/-/lr-1.4.1.tgz",
+			"integrity": "sha512-CHsKq8DMKBf9b3yXPDIU4DbH+ZJd/sJdYOW2llbW/HudP5u0VS6Bfq1hLYfgU7uAYGFIyGGQIsSOXGPEErZiJw==",
+			"dependencies": {
+				"@lezer/common": "^1.0.0"
+			}
+		},
+		"node_modules/@lezer/python": {
+			"version": "1.1.14",
+			"resolved": "https://registry.npmjs.org/@lezer/python/-/python-1.1.14.tgz",
+			"integrity": "sha512-ykDOb2Ti24n76PJsSa4ZoDF0zH12BSw1LGfQXCYJhJyOGiFTfGaX0Du66Ze72R+u/P35U+O6I9m8TFXov1JzsA==",
+			"dependencies": {
+				"@lezer/common": "^1.2.0",
+				"@lezer/highlight": "^1.0.0",
+				"@lezer/lr": "^1.0.0"
+			}
+		},
 		"node_modules/@melt-ui/svelte": {
 			"version": "0.76.0",
 			"resolved": "https://registry.npmjs.org/@melt-ui/svelte/-/svelte-0.76.0.tgz",
@@ -2224,12 +2382,12 @@
 			}
 		},
 		"node_modules/braces": {
-			"version": "3.0.2",
-			"resolved": "https://registry.npmjs.org/braces/-/braces-3.0.2.tgz",
-			"integrity": "sha512-b8um+L1RzM3WDSzvhm6gIz1yfTbBt6YTlcEKAvsmqCZZFw46z626lVj9j1yEPW33H5H+lBQpZMP1k8l+78Ha0A==",
+			"version": "3.0.3",
+			"resolved": "https://registry.npmjs.org/braces/-/braces-3.0.3.tgz",
+			"integrity": "sha512-yQbXgO/OSZVD2IsiLlro+7Hf6Q18EJrKSEsdoMzKePKXct3gvD8oLcOQdIzGupr5Fj+EDe8gO/lxc1BzfMpxvA==",
 			"dev": true,
 			"dependencies": {
-				"fill-range": "^7.0.1"
+				"fill-range": "^7.1.1"
 			},
 			"engines": {
 				"node": ">=8"
@@ -2769,6 +2927,20 @@
 				"plain-tag": "^0.1.3"
 			}
 		},
+		"node_modules/codemirror": {
+			"version": "6.0.1",
+			"resolved": "https://registry.npmjs.org/codemirror/-/codemirror-6.0.1.tgz",
+			"integrity": "sha512-J8j+nZ+CdWmIeFIGXEFbFPtpiYacFMDR8GlHK3IyHQJMCaVRfGx9NT+Hxivv1ckLWPvNdZqndbr/7lVhrf/Svg==",
+			"dependencies": {
+				"@codemirror/autocomplete": "^6.0.0",
+				"@codemirror/commands": "^6.0.0",
+				"@codemirror/language": "^6.0.0",
+				"@codemirror/lint": "^6.0.0",
+				"@codemirror/search": "^6.0.0",
+				"@codemirror/state": "^6.0.0",
+				"@codemirror/view": "^6.0.0"
+			}
+		},
 		"node_modules/coincident": {
 			"version": "1.2.3",
 			"resolved": "https://registry.npmjs.org/coincident/-/coincident-1.2.3.tgz",
@@ -2891,6 +3063,11 @@
 				"layout-base": "^1.0.0"
 			}
 		},
+		"node_modules/crelt": {
+			"version": "1.0.6",
+			"resolved": "https://registry.npmjs.org/crelt/-/crelt-1.0.6.tgz",
+			"integrity": "sha512-VQ2MBenTq1fWZUH9DJNGti7kKv6EeAuYr3cLwxUWhIu1baTaXh4Ib5W2CqHVqib4/MqbYGJqiL3Zb8GJZr3l4g=="
+		},
 		"node_modules/cross-spawn": {
 			"version": "7.0.3",
 			"resolved": "https://registry.npmjs.org/cross-spawn/-/cross-spawn-7.0.3.tgz",
@@ -4429,9 +4606,9 @@
 			"integrity": "sha512-P9bmyZ3h/PRG+Nzga+rbdI4OEpNDzAVyy74uVO9ATgzLK6VtAsYybF/+TOCvrc0MO793d6+42lLyZTw7/ArVzA=="
 		},
 		"node_modules/fill-range": {
-			"version": "7.0.1",
-			"resolved": "https://registry.npmjs.org/fill-range/-/fill-range-7.0.1.tgz",
-			"integrity": "sha512-qOo9F+dMUmC2Lcb4BbVvnKJxTPjCm+RRpe4gDuGrzkL7mEVl/djYSu2OdQ2Pa302N4oqkSg9ir6jaLWJ2USVpQ==",
+			"version": "7.1.1",
+			"resolved": "https://registry.npmjs.org/fill-range/-/fill-range-7.1.1.tgz",
+			"integrity": "sha512-YsGpe3WHLK8ZYi4tWDg2Jy3ebRz2rXowDxnld4bkQB00cc/1Zw9AWnC0i9ztDJitivtQvaI9KaLyKrc+hBW0yg==",
 			"dev": true,
 			"dependencies": {
 				"to-regex-range": "^5.0.1"
@@ -8278,6 +8455,11 @@
 				"url": "https://github.com/sponsors/antfu"
 			}
 		},
+		"node_modules/style-mod": {
+			"version": "4.1.2",
+			"resolved": "https://registry.npmjs.org/style-mod/-/style-mod-4.1.2.tgz",
+			"integrity": "sha512-wnD1HyVqpJUI2+eKZ+eo1UwghftP6yuFheBqqe+bWCotBjC2K1YnteJILRMs3SM4V/0dLEW1SC27MWP5y+mwmw=="
+		},
 		"node_modules/stylis": {
 			"version": "4.3.2",
 			"resolved": "https://registry.npmjs.org/stylis/-/stylis-4.3.2.tgz",
@@ -10022,6 +10204,11 @@
 				"he": "^1.2.0"
 			}
 		},
+		"node_modules/w3c-keyname": {
+			"version": "2.2.8",
+			"resolved": "https://registry.npmjs.org/w3c-keyname/-/w3c-keyname-2.2.8.tgz",
+			"integrity": "sha512-dpojBhNsCNN7T82Tm7k26A6G9ML3NkhDsnw9n/eoxSRlVBB4CEtIQ/KTCLI2Fwf3ataSXRhYFkQi3SlnFwPvPQ=="
+		},
 		"node_modules/walk-sync": {
 			"version": "2.2.0",
 			"resolved": "https://registry.npmjs.org/walk-sync/-/walk-sync-2.2.0.tgz",

package.json

@@ -1,6 +1,6 @@
 {
 	"name": "open-webui",
-	"version": "0.2.5",
+	"version": "0.3.4",
 	"private": true,
 	"scripts": {
 		"dev": "npm run pyodide:fetch && vite dev --host",
@@ -16,7 +16,7 @@
 		"format:backend": "black . --exclude \".venv/|/venv/\"",
 		"i18n:parse": "i18next --config i18next-parser.config.ts && prettier --write \"src/lib/i18n/**/*.{js,json}\"",
 		"cy:open": "cypress open",
-		"test:frontend": "vitest",
+		"test:frontend": "vitest --passWithNoTests",
 		"pyodide:fetch": "node scripts/prepare-pyodide.js"
 	},
 	"devDependencies": {
@@ -48,10 +48,14 @@
 	},
 	"type": "module",
 	"dependencies": {
+		"@codemirror/lang-javascript": "^6.2.2",
+		"@codemirror/lang-python": "^6.1.6",
+		"@codemirror/theme-one-dark": "^6.1.2",
 		"@pyscript/core": "^0.4.32",
 		"@sveltejs/adapter-node": "^1.3.1",
 		"async": "^3.2.5",
 		"bits-ui": "^0.19.7",
+		"codemirror": "^6.0.1",
 		"dayjs": "^1.11.10",
 		"eventsource-parser": "^1.1.2",
 		"file-saver": "^2.0.5",

pyproject.toml

@@ -26,8 +26,6 @@ dependencies = [
     "PyMySQL==1.1.0",
     "bcrypt==4.1.3",
 
-    "litellm[proxy]==1.37.20",
-
     "boto3==1.34.110",
 
     "argon2-cffi==23.1.0",
@@ -67,6 +65,10 @@ dependencies = [
     "langfuse==2.33.0",
     "youtube-transcript-api==0.6.2",
     "pytube==15.0.0",
+    "extract_msg",
+    "pydub",
+    "duckduckgo-search~=6.1.5"
+
 ]
 readme = "README.md"
 requires-python = ">= 3.11, < 3.12.0a1"

requirements-dev.lock

@@ -12,7 +12,6 @@
 aiohttp==3.9.5
     # via langchain
     # via langchain-community
-    # via litellm
     # via open-webui
 aiosignal==1.3.1
     # via aiohttp
@@ -20,11 +19,9 @@ annotated-types==0.6.0
     # via pydantic
 anyio==4.3.0
     # via httpx
-    # via openai
     # via starlette
     # via watchfiles
 apscheduler==3.10.4
-    # via litellm
     # via open-webui
 argon2-cffi==23.1.0
     # via open-webui
@@ -40,7 +37,6 @@ av==11.0.0
     # via faster-whisper
 backoff==2.2.1
     # via langfuse
-    # via litellm
     # via posthog
     # via unstructured
 bcrypt==4.1.3
@@ -48,6 +44,7 @@ bcrypt==4.1.3
     # via open-webui
     # via passlib
 beautifulsoup4==4.12.3
+    # via extract-msg
     # via unstructured
 bidict==0.23.1
     # via python-socketio
@@ -85,18 +82,21 @@ chromadb==0.5.0
     # via open-webui
 click==8.1.7
     # via black
+    # via duckduckgo-search
     # via flask
-    # via litellm
     # via nltk
     # via peewee-migrate
-    # via rq
     # via typer
     # via uvicorn
+colorclass==2.2.2
+    # via oletools
 coloredlogs==15.0.1
     # via onnxruntime
+compressed-rtf==1.0.6
+    # via extract-msg
 cryptography==42.0.7
     # via authlib
-    # via litellm
+    # via msoffcrypto-tool
     # via pyjwt
 ctranslate2==4.2.1
     # via faster-whisper
@@ -112,33 +112,34 @@ defusedxml==0.7.1
 deprecated==1.2.14
     # via opentelemetry-api
     # via opentelemetry-exporter-otlp-proto-grpc
-distro==1.9.0
-    # via openai
 dnspython==2.6.1
     # via email-validator
 docx2txt==0.8
     # via open-webui
+duckduckgo-search==6.1.5
+    # via open-webui
+easygui==0.98.3
+    # via oletools
+ebcdic==1.1.1
+    # via extract-msg
 ecdsa==0.19.0
     # via python-jose
 email-validator==2.1.1
     # via fastapi
-    # via pydantic
 emoji==2.11.1
     # via unstructured
 et-xmlfile==1.1.0
     # via openpyxl
+extract-msg==0.48.5
+    # via open-webui
 fake-useragent==1.5.1
     # via open-webui
 fastapi==0.111.0
     # via chromadb
-    # via fastapi-sso
     # via langchain-chroma
-    # via litellm
     # via open-webui
 fastapi-cli==0.0.4
     # via fastapi
-fastapi-sso==0.10.0
-    # via litellm
 faster-whisper==1.0.2
     # via open-webui
 filelock==3.14.0
@@ -194,8 +195,6 @@ grpcio==1.63.0
     # via opentelemetry-exporter-otlp-proto-grpc
 grpcio-status==1.62.2
     # via google-api-core
-gunicorn==22.0.0
-    # via litellm
 h11==0.14.0
     # via httpcore
     # via uvicorn
@@ -209,9 +208,7 @@ httptools==0.6.1
     # via uvicorn
 httpx==0.27.0
     # via fastapi
-    # via fastapi-sso
     # via langfuse
-    # via openai
 huggingface-hub==0.23.0
     # via faster-whisper
     # via sentence-transformers
@@ -228,7 +225,6 @@ idna==3.7
     # via unstructured-client
     # via yarl
 importlib-metadata==7.0.0
-    # via litellm
     # via opentelemetry-api
 importlib-resources==6.4.0
     # via chromadb
@@ -237,7 +233,6 @@ itsdangerous==2.2.0
 jinja2==3.1.4
     # via fastapi
     # via flask
-    # via litellm
     # via torch
 jmespath==1.0.1
     # via boto3
@@ -275,8 +270,8 @@ langsmith==0.1.57
     # via langchain
     # via langchain-community
     # via langchain-core
-litellm==1.37.20
-    # via open-webui
+lark==1.1.8
+    # via rtfde
 lxml==5.2.2
     # via unstructured
 markdown==3.6
@@ -297,6 +292,8 @@ monotonic==1.6
     # via posthog
 mpmath==1.3.0
     # via sympy
+msoffcrypto-tool==5.4.1
+    # via oletools
 multidict==6.0.5
     # via aiohttp
     # via yarl
@@ -328,15 +325,19 @@ numpy==1.26.4
     # via transformers
     # via unstructured
 oauthlib==3.2.2
-    # via fastapi-sso
     # via kubernetes
     # via requests-oauthlib
+olefile==0.47
+    # via extract-msg
+    # via msoffcrypto-tool
+    # via oletools
+oletools==0.60.1
+    # via pcodedmp
+    # via rtfde
 onnxruntime==1.17.3
     # via chromadb
     # via faster-whisper
     # via rapidocr-onnxruntime
-openai==1.28.1
-    # via litellm
 opencv-python==4.9.0.80
     # via rapidocr-onnxruntime
 opencv-python-headless==4.9.0.80
@@ -378,15 +379,14 @@ ordered-set==4.1.0
     # via deepdiff
 orjson==3.10.3
     # via chromadb
+    # via duckduckgo-search
     # via fastapi
     # via langsmith
-    # via litellm
 overrides==7.7.0
     # via chromadb
 packaging==23.2
     # via black
     # via build
-    # via gunicorn
     # via huggingface-hub
     # via langchain-core
     # via langfuse
@@ -398,8 +398,11 @@ pandas==2.2.2
     # via open-webui
 passlib==1.7.4
     # via open-webui
+    # via passlib
 pathspec==0.12.1
     # via black
+pcodedmp==1.2.6
+    # via oletools
 peewee==3.17.5
     # via open-webui
     # via peewee-migrate
@@ -440,27 +443,28 @@ pycparser==2.22
 pydantic==2.7.1
     # via chromadb
     # via fastapi
-    # via fastapi-sso
     # via google-generativeai
     # via langchain
     # via langchain-core
     # via langfuse
     # via langsmith
     # via open-webui
-    # via openai
 pydantic-core==2.18.2
     # via pydantic
+pydub==0.25.1
+    # via open-webui
 pygments==2.18.0
     # via rich
 pyjwt==2.8.0
-    # via litellm
     # via open-webui
+    # via pyjwt
 pymysql==1.1.0
     # via open-webui
 pypandoc==1.13
     # via open-webui
-pyparsing==3.1.2
+pyparsing==2.4.7
     # via httplib2
+    # via oletools
 pypdf==4.2.0
     # via open-webui
     # via unstructured-client
@@ -468,6 +472,8 @@ pypika==0.48.9
     # via chromadb
 pyproject-hooks==1.1.0
     # via build
+pyreqwest-impersonate==0.4.7
+    # via duckduckgo-search
 python-dateutil==2.9.0.post0
     # via botocore
     # via kubernetes
@@ -475,7 +481,6 @@ python-dateutil==2.9.0.post0
     # via posthog
     # via unstructured-client
 python-dotenv==1.0.1
-    # via litellm
     # via uvicorn
 python-engineio==4.9.0
     # via python-socketio
@@ -487,7 +492,6 @@ python-magic==0.4.27
     # via unstructured
 python-multipart==0.0.9
     # via fastapi
-    # via litellm
     # via open-webui
 python-socketio==5.11.2
     # via open-webui
@@ -506,7 +510,6 @@ pyyaml==6.0.1
     # via langchain
     # via langchain-community
     # via langchain-core
-    # via litellm
     # via rapidocr-onnxruntime
     # via transformers
     # via uvicorn
@@ -516,11 +519,10 @@ rapidfuzz==3.9.0
     # via unstructured
 rapidocr-onnxruntime==1.3.22
     # via open-webui
-redis==5.0.4
-    # via rq
+red-black-tree-mod==1.20
+    # via extract-msg
 regex==2024.5.10
     # via nltk
-    # via tiktoken
     # via transformers
 requests==2.32.2
     # via chromadb
@@ -530,11 +532,9 @@ requests==2.32.2
     # via langchain
     # via langchain-community
     # via langsmith
-    # via litellm
     # via open-webui
     # via posthog
     # via requests-oauthlib
-    # via tiktoken
     # via transformers
     # via unstructured
     # via unstructured-client
@@ -543,11 +543,11 @@ requests-oauthlib==2.0.0
     # via kubernetes
 rich==13.7.1
     # via typer
-rq==1.16.2
-    # via litellm
 rsa==4.9
     # via google-auth
     # via python-jose
+rtfde==0.1.1
+    # via extract-msg
 s3transfer==0.10.1
     # via boto3
 safetensors==0.4.3
@@ -559,9 +559,6 @@ scipy==1.13.0
     # via sentence-transformers
 sentence-transformers==2.7.0
     # via open-webui
-setuptools==69.5.1
-    # via ctranslate2
-    # via opentelemetry-instrumentation
 shapely==2.0.4
     # via rapidocr-onnxruntime
 shellingham==1.5.4
@@ -580,7 +577,6 @@ six==1.16.0
 sniffio==1.3.1
     # via anyio
     # via httpx
-    # via openai
 soupsieve==2.5
     # via beautifulsoup4
 sqlalchemy==2.0.30
@@ -600,12 +596,9 @@ tenacity==8.3.0
     # via langchain-core
 threadpoolctl==3.5.0
     # via scikit-learn
-tiktoken==0.6.0
-    # via litellm
 tokenizers==0.15.2
     # via chromadb
     # via faster-whisper
-    # via litellm
     # via transformers
 torch==2.3.0
     # via sentence-transformers
@@ -614,7 +607,6 @@ tqdm==4.66.4
     # via google-generativeai
     # via huggingface-hub
     # via nltk
-    # via openai
     # via sentence-transformers
     # via transformers
 transformers==4.39.3
@@ -627,7 +619,6 @@ typing-extensions==4.11.0
     # via fastapi
     # via google-generativeai
     # via huggingface-hub
-    # via openai
     # via opentelemetry-sdk
     # via pydantic
     # via pydantic-core
@@ -644,6 +635,7 @@ tzdata==2024.1
     # via pandas
 tzlocal==5.2
     # via apscheduler
+    # via extract-msg
 ujson==5.10.0
     # via fastapi
 unstructured==0.14.0
@@ -660,8 +652,8 @@ urllib3==2.2.1
 uvicorn==0.22.0
     # via chromadb
     # via fastapi
-    # via litellm
     # via open-webui
+    # via uvicorn
 uvloop==0.19.0
     # via uvicorn
 validators==0.28.1
@@ -689,3 +681,6 @@ youtube-transcript-api==0.6.2
     # via open-webui
 zipp==3.18.1
     # via importlib-metadata
+setuptools==69.5.1
+    # via ctranslate2
+    # via opentelemetry-instrumentation

requirements.lock

@@ -12,7 +12,6 @@
 aiohttp==3.9.5
     # via langchain
     # via langchain-community
-    # via litellm
     # via open-webui
 aiosignal==1.3.1
     # via aiohttp
@@ -20,11 +19,9 @@ annotated-types==0.6.0
     # via pydantic
 anyio==4.3.0
     # via httpx
-    # via openai
     # via starlette
     # via watchfiles
 apscheduler==3.10.4
-    # via litellm
     # via open-webui
 argon2-cffi==23.1.0
     # via open-webui
@@ -40,7 +37,6 @@ av==11.0.0
     # via faster-whisper
 backoff==2.2.1
     # via langfuse
-    # via litellm
     # via posthog
     # via unstructured
 bcrypt==4.1.3
@@ -48,6 +44,7 @@ bcrypt==4.1.3
     # via open-webui
     # via passlib
 beautifulsoup4==4.12.3
+    # via extract-msg
     # via unstructured
 bidict==0.23.1
     # via python-socketio
@@ -85,18 +82,21 @@ chromadb==0.5.0
     # via open-webui
 click==8.1.7
     # via black
+    # via duckduckgo-search
     # via flask
-    # via litellm
     # via nltk
     # via peewee-migrate
-    # via rq
     # via typer
     # via uvicorn
+colorclass==2.2.2
+    # via oletools
 coloredlogs==15.0.1
     # via onnxruntime
+compressed-rtf==1.0.6
+    # via extract-msg
 cryptography==42.0.7
     # via authlib
-    # via litellm
+    # via msoffcrypto-tool
     # via pyjwt
 ctranslate2==4.2.1
     # via faster-whisper
@@ -112,33 +112,34 @@ defusedxml==0.7.1
 deprecated==1.2.14
     # via opentelemetry-api
     # via opentelemetry-exporter-otlp-proto-grpc
-distro==1.9.0
-    # via openai
 dnspython==2.6.1
     # via email-validator
 docx2txt==0.8
     # via open-webui
+duckduckgo-search==6.1.5
+    # via open-webui
+easygui==0.98.3
+    # via oletools
+ebcdic==1.1.1
+    # via extract-msg
 ecdsa==0.19.0
     # via python-jose
 email-validator==2.1.1
     # via fastapi
-    # via pydantic
 emoji==2.11.1
     # via unstructured
 et-xmlfile==1.1.0
     # via openpyxl
+extract-msg==0.48.5
+    # via open-webui
 fake-useragent==1.5.1
     # via open-webui
 fastapi==0.111.0
     # via chromadb
-    # via fastapi-sso
     # via langchain-chroma
-    # via litellm
     # via open-webui
 fastapi-cli==0.0.4
     # via fastapi
-fastapi-sso==0.10.0
-    # via litellm
 faster-whisper==1.0.2
     # via open-webui
 filelock==3.14.0
@@ -194,8 +195,6 @@ grpcio==1.63.0
     # via opentelemetry-exporter-otlp-proto-grpc
 grpcio-status==1.62.2
     # via google-api-core
-gunicorn==22.0.0
-    # via litellm
 h11==0.14.0
     # via httpcore
     # via uvicorn
@@ -209,9 +208,7 @@ httptools==0.6.1
     # via uvicorn
 httpx==0.27.0
     # via fastapi
-    # via fastapi-sso
     # via langfuse
-    # via openai
 huggingface-hub==0.23.0
     # via faster-whisper
     # via sentence-transformers
@@ -228,7 +225,6 @@ idna==3.7
     # via unstructured-client
     # via yarl
 importlib-metadata==7.0.0
-    # via litellm
     # via opentelemetry-api
 importlib-resources==6.4.0
     # via chromadb
@@ -237,7 +233,6 @@ itsdangerous==2.2.0
 jinja2==3.1.4
     # via fastapi
     # via flask
-    # via litellm
     # via torch
 jmespath==1.0.1
     # via boto3
@@ -275,8 +270,8 @@ langsmith==0.1.57
     # via langchain
     # via langchain-community
     # via langchain-core
-litellm==1.37.20
-    # via open-webui
+lark==1.1.8
+    # via rtfde
 lxml==5.2.2
     # via unstructured
 markdown==3.6
@@ -297,6 +292,8 @@ monotonic==1.6
     # via posthog
 mpmath==1.3.0
     # via sympy
+msoffcrypto-tool==5.4.1
+    # via oletools
 multidict==6.0.5
     # via aiohttp
     # via yarl
@@ -328,15 +325,19 @@ numpy==1.26.4
     # via transformers
     # via unstructured
 oauthlib==3.2.2
-    # via fastapi-sso
     # via kubernetes
     # via requests-oauthlib
+olefile==0.47
+    # via extract-msg
+    # via msoffcrypto-tool
+    # via oletools
+oletools==0.60.1
+    # via pcodedmp
+    # via rtfde
 onnxruntime==1.17.3
     # via chromadb
     # via faster-whisper
     # via rapidocr-onnxruntime
-openai==1.28.1
-    # via litellm
 opencv-python==4.9.0.80
     # via rapidocr-onnxruntime
 opencv-python-headless==4.9.0.80
@@ -378,15 +379,14 @@ ordered-set==4.1.0
     # via deepdiff
 orjson==3.10.3
     # via chromadb
+    # via duckduckgo-search
     # via fastapi
     # via langsmith
-    # via litellm
 overrides==7.7.0
     # via chromadb
 packaging==23.2
     # via black
     # via build
-    # via gunicorn
     # via huggingface-hub
     # via langchain-core
     # via langfuse
@@ -398,8 +398,11 @@ pandas==2.2.2
     # via open-webui
 passlib==1.7.4
     # via open-webui
+    # via passlib
 pathspec==0.12.1
     # via black
+pcodedmp==1.2.6
+    # via oletools
 peewee==3.17.5
     # via open-webui
     # via peewee-migrate
@@ -440,27 +443,28 @@ pycparser==2.22
 pydantic==2.7.1
     # via chromadb
     # via fastapi
-    # via fastapi-sso
     # via google-generativeai
     # via langchain
     # via langchain-core
     # via langfuse
     # via langsmith
     # via open-webui
-    # via openai
 pydantic-core==2.18.2
     # via pydantic
+pydub==0.25.1
+    # via open-webui
 pygments==2.18.0
     # via rich
 pyjwt==2.8.0
-    # via litellm
     # via open-webui
+    # via pyjwt
 pymysql==1.1.0
     # via open-webui
 pypandoc==1.13
     # via open-webui
-pyparsing==3.1.2
+pyparsing==2.4.7
     # via httplib2
+    # via oletools
 pypdf==4.2.0
     # via open-webui
     # via unstructured-client
@@ -468,6 +472,8 @@ pypika==0.48.9
     # via chromadb
 pyproject-hooks==1.1.0
     # via build
+pyreqwest-impersonate==0.4.7
+    # via duckduckgo-search
 python-dateutil==2.9.0.post0
     # via botocore
     # via kubernetes
@@ -475,7 +481,6 @@ python-dateutil==2.9.0.post0
     # via posthog
     # via unstructured-client
 python-dotenv==1.0.1
-    # via litellm
     # via uvicorn
 python-engineio==4.9.0
     # via python-socketio
@@ -487,7 +492,6 @@ python-magic==0.4.27
     # via unstructured
 python-multipart==0.0.9
     # via fastapi
-    # via litellm
     # via open-webui
 python-socketio==5.11.2
     # via open-webui
@@ -506,7 +510,6 @@ pyyaml==6.0.1
     # via langchain
     # via langchain-community
     # via langchain-core
-    # via litellm
     # via rapidocr-onnxruntime
     # via transformers
     # via uvicorn
@@ -516,11 +519,10 @@ rapidfuzz==3.9.0
     # via unstructured
 rapidocr-onnxruntime==1.3.22
     # via open-webui
-redis==5.0.4
-    # via rq
+red-black-tree-mod==1.20
+    # via extract-msg
 regex==2024.5.10
     # via nltk
-    # via tiktoken
     # via transformers
 requests==2.32.2
     # via chromadb
@@ -530,11 +532,9 @@ requests==2.32.2
     # via langchain
     # via langchain-community
     # via langsmith
-    # via litellm
     # via open-webui
     # via posthog
     # via requests-oauthlib
-    # via tiktoken
     # via transformers
     # via unstructured
     # via unstructured-client
@@ -543,11 +543,11 @@ requests-oauthlib==2.0.0
     # via kubernetes
 rich==13.7.1
     # via typer
-rq==1.16.2
-    # via litellm
 rsa==4.9
     # via google-auth
     # via python-jose
+rtfde==0.1.1
+    # via extract-msg
 s3transfer==0.10.1
     # via boto3
 safetensors==0.4.3
@@ -559,9 +559,6 @@ scipy==1.13.0
     # via sentence-transformers
 sentence-transformers==2.7.0
     # via open-webui
-setuptools==69.5.1
-    # via ctranslate2
-    # via opentelemetry-instrumentation
 shapely==2.0.4
     # via rapidocr-onnxruntime
 shellingham==1.5.4
@@ -580,7 +577,6 @@ six==1.16.0
 sniffio==1.3.1
     # via anyio
     # via httpx
-    # via openai
 soupsieve==2.5
     # via beautifulsoup4
 sqlalchemy==2.0.30
@@ -600,12 +596,9 @@ tenacity==8.3.0
     # via langchain-core
 threadpoolctl==3.5.0
     # via scikit-learn
-tiktoken==0.6.0
-    # via litellm
 tokenizers==0.15.2
     # via chromadb
     # via faster-whisper
-    # via litellm
     # via transformers
 torch==2.3.0
     # via sentence-transformers
@@ -614,7 +607,6 @@ tqdm==4.66.4
     # via google-generativeai
     # via huggingface-hub
     # via nltk
-    # via openai
     # via sentence-transformers
     # via transformers
 transformers==4.39.3
@@ -627,7 +619,6 @@ typing-extensions==4.11.0
     # via fastapi
     # via google-generativeai
     # via huggingface-hub
-    # via openai
     # via opentelemetry-sdk
     # via pydantic
     # via pydantic-core
@@ -644,6 +635,7 @@ tzdata==2024.1
     # via pandas
 tzlocal==5.2
     # via apscheduler
+    # via extract-msg
 ujson==5.10.0
     # via fastapi
 unstructured==0.14.0
@@ -660,8 +652,8 @@ urllib3==2.2.1
 uvicorn==0.22.0
     # via chromadb
     # via fastapi
-    # via litellm
     # via open-webui
+    # via uvicorn
 uvloop==0.19.0
     # via uvicorn
 validators==0.28.1
@@ -689,3 +681,6 @@ youtube-transcript-api==0.6.2
     # via open-webui
 zipp==3.18.1
     # via importlib-metadata
+setuptools==69.5.1
+    # via ctranslate2
+    # via opentelemetry-instrumentation

src/app.css

@@ -28,6 +28,10 @@ math {
 	@apply rounded-lg;
 }
 
+.markdown a {
+	@apply underline;
+}
+
 ol > li {
 	counter-increment: list-number;
 	display: block;
@@ -92,10 +96,18 @@ select {
 	visibility: hidden;
 }
 
+.scrollbar-hidden::-webkit-scrollbar-corner {
+	display: none;
+}
+
 .scrollbar-none::-webkit-scrollbar {
 	display: none; /* for Chrome, Safari and Opera */
 }
 
+.scrollbar-none::-webkit-scrollbar-corner {
+	display: none;
+}
+
 .scrollbar-none {
 	-ms-overflow-style: none; /* IE and Edge */
 	scrollbar-width: none; /* Firefox */
@@ -111,3 +123,16 @@ input::-webkit-inner-spin-button {
 input[type='number'] {
 	-moz-appearance: textfield; /* Firefox */
 }
+
+.cm-editor {
+	height: 100%;
+	width: 100%;
+}
+
+.cm-scroller {
+	@apply scrollbar-hidden;
+}
+
+.cm-editor.cm-focused {
+	outline: none;
+}

src/app.html

@@ -32,6 +32,9 @@
 				} else if (localStorage.theme && localStorage.theme === 'system') {
 					systemTheme = window.matchMedia('(prefers-color-scheme: dark)').matches;
 					document.documentElement.classList.add(systemTheme ? 'dark' : 'light');
+				} else if (localStorage.theme && localStorage.theme === 'her') {
+					document.documentElement.classList.add('dark');
+					document.documentElement.classList.add('her');
 				} else {
 					document.documentElement.classList.add('dark');
 				}
@@ -59,15 +62,7 @@
 
 		<div
 			id="splash-screen"
-			style="
-				position: fixed;
-				z-index: 100;
-				background: #fff;
-				top: 0;
-				left: 0;
-				width: 100%;
-				height: 100%;
-			"
+			style="position: fixed; z-index: 100; top: 0; left: 0; width: 100%; height: 100%"
 		>
 			<style type="text/css" nonce="">
 				html {
@@ -76,20 +71,138 @@
 			</style>
 
 			<img
+				id="logo"
 				style="
 					position: absolute;
 					width: 6rem;
 					height: 6rem;
-					top: 46%;
+					top: 41%;
 					left: 50%;
-					margin: -40px 0 0 -40px;
+					margin-left: -3rem;
 				"
 				src="/logo.svg"
 			/>
 
+			<div
+				style="
+					position: absolute;
+					top: 33%;
+					left: 50%;
+
+					width: 24rem;
+					margin-left: -12rem;
+
+					display: flex;
+					flex-direction: column;
+					align-items: center;
+				"
+			>
+				<img
+					id="logo-her"
+					style="width: 13rem; height: 13rem"
+					src="/logo.svg"
+					class="animate-pulse-fast"
+				/>
+
+				<div style="position: relative; width: 24rem; margin-top: 0.5rem">
+					<div
+						id="progress-background"
+						style="
+							position: absolute;
+							width: 100%;
+							height: 0.75rem;
+
+							border-radius: 9999px;
+							background-color: #fafafa9a;
+						"
+					></div>
+
+					<div
+						id="progress-bar"
+						style="
+							position: absolute;
+							width: 0%;
+							height: 0.75rem;
+							border-radius: 9999px;
+							background-color: #fff;
+						"
+						class="bg-white"
+					></div>
+				</div>
+			</div>
+
 			<!-- <span style="position: absolute; bottom: 32px; left: 50%; margin: -36px 0 0 -36px">
 				Footer content
 			</span> -->
 		</div>
 	</body>
 </html>
+
+<style type="text/css" nonce="">
+	html {
+		overflow-y: hidden !important;
+	}
+
+	#splash-screen {
+		background: #fff;
+	}
+	html.dark #splash-screen {
+		background: #000;
+	}
+
+	html.dark #splash-screen img {
+		filter: invert(1);
+	}
+
+	html.her #splash-screen {
+		background: #983724;
+	}
+
+	#logo-her {
+		display: none;
+	}
+
+	#progress-background {
+		display: none;
+	}
+
+	#progress-bar {
+		display: none;
+	}
+
+	html.her #logo {
+		display: none;
+	}
+
+	html.her #logo-her {
+		display: block;
+		filter: invert(1);
+	}
+
+	html.her #progress-background {
+		display: block;
+	}
+
+	html.her #progress-bar {
+		display: block;
+	}
+
+	@media (max-width: 24rem) {
+		html.her #progress-background {
+			display: none;
+		}
+
+		html.her #progress-bar {
+			display: none;
+		}
+	}
+
+	@keyframes pulse {
+		50% {
+			opacity: 0.65;
+		}
+	}
+	.animate-pulse-fast {
+		animation: pulse 1.5s cubic-bezier(0.4, 0, 0.6, 1) infinite;
+	}
+</style>

src/lib/apis/audio/index.ts

@@ -98,7 +98,7 @@ export const synthesizeOpenAISpeech = async (
 	token: string = '',
 	speaker: string = 'alloy',
 	text: string = '',
-	model: string = 'tts-1'
+	model?: string
 ) => {
 	let error = null;
 
@@ -109,9 +109,9 @@ export const synthesizeOpenAISpeech = async (
 			'Content-Type': 'application/json'
 		},
 		body: JSON.stringify({
-			model: model,
 			input: text,
-			voice: speaker
+			voice: speaker,
+			...(model && { model })
 		})
 	})
 		.then(async (res) => {

src/lib/apis/documents/index.ts

@@ -76,7 +76,10 @@ export const getDocs = async (token: string = '') => {
 export const getDocByName = async (token: string, name: string) => {
 	let error = null;
 
-	const res = await fetch(`${WEBUI_API_BASE_URL}/documents/name/${name}`, {
+	const searchParams = new URLSearchParams();
+	searchParams.append('name', name);
+
+	const res = await fetch(`${WEBUI_API_BASE_URL}/documents/docs?${searchParams.toString()}`, {
 		method: 'GET',
 		headers: {
 			Accept: 'application/json',
@@ -113,7 +116,10 @@ type DocUpdateForm = {
 export const updateDocByName = async (token: string, name: string, form: DocUpdateForm) => {
 	let error = null;
 
-	const res = await fetch(`${WEBUI_API_BASE_URL}/documents/name/${name}/update`, {
+	const searchParams = new URLSearchParams();
+	searchParams.append('name', name);
+
+	const res = await fetch(`${WEBUI_API_BASE_URL}/documents/doc/update?${searchParams.toString()}`, {
 		method: 'POST',
 		headers: {
 			Accept: 'application/json',
@@ -154,7 +160,10 @@ type TagDocForm = {
 export const tagDocByName = async (token: string, name: string, form: TagDocForm) => {
 	let error = null;
 
-	const res = await fetch(`${WEBUI_API_BASE_URL}/documents/name/${name}/tags`, {
+	const searchParams = new URLSearchParams();
+	searchParams.append('name', name);
+
+	const res = await fetch(`${WEBUI_API_BASE_URL}/documents/doc/tags?${searchParams.toString()}`, {
 		method: 'POST',
 		headers: {
 			Accept: 'application/json',
@@ -190,7 +199,10 @@ export const tagDocByName = async (token: string, name: string, form: TagDocForm
 export const deleteDocByName = async (token: string, name: string) => {
 	let error = null;
 
-	const res = await fetch(`${WEBUI_API_BASE_URL}/documents/name/${name}/delete`, {
+	const searchParams = new URLSearchParams();
+	searchParams.append('name', name);
+
+	const res = await fetch(`${WEBUI_API_BASE_URL}/documents/doc/delete?${searchParams.toString()}`, {
 		method: 'DELETE',
 		headers: {
 			Accept: 'application/json',

src/lib/apis/index.ts

@@ -104,6 +104,195 @@ export const chatCompleted = async (token: string, body: ChatCompletedForm) => {
 	return res;
 };
 
+export const getTaskConfig = async (token: string = '') => {
+	let error = null;
+
+	const res = await fetch(`${WEBUI_BASE_URL}/api/task/config`, {
+		method: 'GET',
+		headers: {
+			Accept: 'application/json',
+			'Content-Type': 'application/json',
+			...(token && { authorization: `Bearer ${token}` })
+		}
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.catch((err) => {
+			console.log(err);
+			error = err;
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res;
+};
+
+export const updateTaskConfig = async (token: string, config: object) => {
+	let error = null;
+
+	const res = await fetch(`${WEBUI_BASE_URL}/api/task/config/update`, {
+		method: 'POST',
+		headers: {
+			Accept: 'application/json',
+			'Content-Type': 'application/json',
+			...(token && { authorization: `Bearer ${token}` })
+		},
+		body: JSON.stringify(config)
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.catch((err) => {
+			console.log(err);
+			if ('detail' in err) {
+				error = err.detail;
+			} else {
+				error = err;
+			}
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res;
+};
+
+export const generateTitle = async (
+	token: string = '',
+	model: string,
+	prompt: string,
+	chat_id?: string
+) => {
+	let error = null;
+
+	const res = await fetch(`${WEBUI_BASE_URL}/api/task/title/completions`, {
+		method: 'POST',
+		headers: {
+			Accept: 'application/json',
+			'Content-Type': 'application/json',
+			Authorization: `Bearer ${token}`
+		},
+		body: JSON.stringify({
+			model: model,
+			prompt: prompt,
+			...(chat_id && { chat_id: chat_id })
+		})
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.catch((err) => {
+			console.log(err);
+			if ('detail' in err) {
+				error = err.detail;
+			}
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res?.choices[0]?.message?.content.replace(/["']/g, '') ?? 'New Chat';
+};
+
+export const generateEmoji = async (
+	token: string = '',
+	model: string,
+	prompt: string,
+	chat_id?: string
+) => {
+	let error = null;
+
+	const res = await fetch(`${WEBUI_BASE_URL}/api/task/emoji/completions`, {
+		method: 'POST',
+		headers: {
+			Accept: 'application/json',
+			'Content-Type': 'application/json',
+			Authorization: `Bearer ${token}`
+		},
+		body: JSON.stringify({
+			model: model,
+			prompt: prompt,
+			...(chat_id && { chat_id: chat_id })
+		})
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.catch((err) => {
+			console.log(err);
+			if ('detail' in err) {
+				error = err.detail;
+			}
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	const response = res?.choices[0]?.message?.content.replace(/["']/g, '') ?? null;
+
+	if (response) {
+		if (/\p{Extended_Pictographic}/u.test(response)) {
+			return response.match(/\p{Extended_Pictographic}/gu)[0];
+		}
+	}
+
+	return null;
+};
+
+export const generateSearchQuery = async (
+	token: string = '',
+	model: string,
+	messages: object[],
+	prompt: string
+) => {
+	let error = null;
+
+	const res = await fetch(`${WEBUI_BASE_URL}/api/task/query/completions`, {
+		method: 'POST',
+		headers: {
+			Accept: 'application/json',
+			'Content-Type': 'application/json',
+			Authorization: `Bearer ${token}`
+		},
+		body: JSON.stringify({
+			model: model,
+			messages: messages,
+			prompt: prompt
+		})
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.catch((err) => {
+			console.log(err);
+			if ('detail' in err) {
+				error = err.detail;
+			}
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res?.choices[0]?.message?.content.replace(/["']/g, '') ?? prompt;
+};
+
 export const getPipelinesList = async (token: string = '') => {
 	let error = null;
 
@@ -133,6 +322,43 @@ export const getPipelinesList = async (token: string = '') => {
 	return pipelines;
 };
 
+export const uploadPipeline = async (token: string, file: File, urlIdx: string) => {
+	let error = null;
+
+	// Create a new FormData object to handle the file upload
+	const formData = new FormData();
+	formData.append('file', file);
+	formData.append('urlIdx', urlIdx);
+
+	const res = await fetch(`${WEBUI_BASE_URL}/api/pipelines/upload`, {
+		method: 'POST',
+		headers: {
+			...(token && { authorization: `Bearer ${token}` })
+			// 'Content-Type': 'multipart/form-data' is not needed as Fetch API will set it automatically
+		},
+		body: formData
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.catch((err) => {
+			console.log(err);
+			if ('detail' in err) {
+				error = err.detail;
+			} else {
+				error = err;
+			}
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res;
+};
+
 export const downloadPipeline = async (token: string, url: string, urlIdx: string) => {
 	let error = null;
 

src/lib/apis/memories/index.ts

@@ -3,7 +3,7 @@ import { WEBUI_API_BASE_URL } from '$lib/constants';
 export const getMemories = async (token: string) => {
 	let error = null;
 
-	const res = await fetch(`${WEBUI_API_BASE_URL}/memories`, {
+	const res = await fetch(`${WEBUI_API_BASE_URL}/memories/`, {
 		method: 'GET',
 		headers: {
 			Accept: 'application/json',
@@ -59,6 +59,37 @@ export const addNewMemory = async (token: string, content: string) => {
 	return res;
 };
 
+export const updateMemoryById = async (token: string, id: string, content: string) => {
+	let error = null;
+
+	const res = await fetch(`${WEBUI_API_BASE_URL}/memories/${id}/update`, {
+		method: 'POST',
+		headers: {
+			Accept: 'application/json',
+			'Content-Type': 'application/json',
+			authorization: `Bearer ${token}`
+		},
+		body: JSON.stringify({
+			content: content
+		})
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.catch((err) => {
+			error = err.detail;
+			console.log(err);
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res;
+};
+
 export const queryMemory = async (token: string, content: string) => {
 	let error = null;
 

src/lib/apis/tools/index.ts

@@ -0,0 +1,193 @@
+import { WEBUI_API_BASE_URL } from '$lib/constants';
+
+export const createNewTool = async (token: string, tool: object) => {
+	let error = null;
+
+	const res = await fetch(`${WEBUI_API_BASE_URL}/tools/create`, {
+		method: 'POST',
+		headers: {
+			Accept: 'application/json',
+			'Content-Type': 'application/json',
+			authorization: `Bearer ${token}`
+		},
+		body: JSON.stringify({
+			...tool
+		})
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.catch((err) => {
+			error = err.detail;
+			console.log(err);
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res;
+};
+
+export const getTools = async (token: string = '') => {
+	let error = null;
+
+	const res = await fetch(`${WEBUI_API_BASE_URL}/tools/`, {
+		method: 'GET',
+		headers: {
+			Accept: 'application/json',
+			'Content-Type': 'application/json',
+			authorization: `Bearer ${token}`
+		}
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.then((json) => {
+			return json;
+		})
+		.catch((err) => {
+			error = err.detail;
+			console.log(err);
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res;
+};
+
+export const exportTools = async (token: string = '') => {
+	let error = null;
+
+	const res = await fetch(`${WEBUI_API_BASE_URL}/tools/export`, {
+		method: 'GET',
+		headers: {
+			Accept: 'application/json',
+			'Content-Type': 'application/json',
+			authorization: `Bearer ${token}`
+		}
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.then((json) => {
+			return json;
+		})
+		.catch((err) => {
+			error = err.detail;
+			console.log(err);
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res;
+};
+
+export const getToolById = async (token: string, id: string) => {
+	let error = null;
+
+	const res = await fetch(`${WEBUI_API_BASE_URL}/tools/id/${id}`, {
+		method: 'GET',
+		headers: {
+			Accept: 'application/json',
+			'Content-Type': 'application/json',
+			authorization: `Bearer ${token}`
+		}
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.then((json) => {
+			return json;
+		})
+		.catch((err) => {
+			error = err.detail;
+
+			console.log(err);
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res;
+};
+
+export const updateToolById = async (token: string, id: string, tool: object) => {
+	let error = null;
+
+	const res = await fetch(`${WEBUI_API_BASE_URL}/tools/id/${id}/update`, {
+		method: 'POST',
+		headers: {
+			Accept: 'application/json',
+			'Content-Type': 'application/json',
+			authorization: `Bearer ${token}`
+		},
+		body: JSON.stringify({
+			...tool
+		})
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.then((json) => {
+			return json;
+		})
+		.catch((err) => {
+			error = err.detail;
+
+			console.log(err);
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res;
+};
+
+export const deleteToolById = async (token: string, id: string) => {
+	let error = null;
+
+	const res = await fetch(`${WEBUI_API_BASE_URL}/tools/id/${id}/delete`, {
+		method: 'DELETE',
+		headers: {
+			Accept: 'application/json',
+			'Content-Type': 'application/json',
+			authorization: `Bearer ${token}`
+		}
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.then((json) => {
+			return json;
+		})
+		.catch((err) => {
+			error = err.detail;
+
+			console.log(err);
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res;
+};

src/lib/apis/utils/index.ts

@@ -22,6 +22,39 @@ export const getGravatarUrl = async (email: string) => {
 	return res;
 };
 
+export const formatPythonCode = async (code: string) => {
+	let error = null;
+
+	const res = await fetch(`${WEBUI_API_BASE_URL}/utils/code/format`, {
+		method: 'POST',
+		headers: {
+			'Content-Type': 'application/json'
+		},
+		body: JSON.stringify({
+			code: code
+		})
+	})
+		.then(async (res) => {
+			if (!res.ok) throw await res.json();
+			return res.json();
+		})
+		.catch((err) => {
+			console.log(err);
+
+			error = err;
+			if (err.detail) {
+				error = err.detail;
+			}
+			return null;
+		});
+
+	if (error) {
+		throw error;
+	}
+
+	return res;
+};
+
 export const downloadChatAsPDF = async (chat: object) => {
 	let error = null;
 

src/lib/components/admin/Settings.svelte

@@ -0,0 +1,390 @@
+<script>
+	import { getContext, tick } from 'svelte';
+	import { toast } from 'svelte-sonner';
+
+	import Database from './Settings/Database.svelte';
+
+	import General from './Settings/General.svelte';
+	import Users from './Settings/Users.svelte';
+
+	import Pipelines from './Settings/Pipelines.svelte';
+	import Audio from './Settings/Audio.svelte';
+	import Images from './Settings/Images.svelte';
+	import Interface from './Settings/Interface.svelte';
+	import Models from './Settings/Models.svelte';
+	import Connections from './Settings/Connections.svelte';
+	import Documents from './Settings/Documents.svelte';
+	import WebSearch from './Settings/WebSearch.svelte';
+	import { config } from '$lib/stores';
+	import { getBackendConfig } from '$lib/apis';
+
+	const i18n = getContext('i18n');
+
+	let selectedTab = 'general';
+</script>
+
+<div class="flex flex-col lg:flex-row w-full h-full py-2 lg:space-x-4">
+	<div
+		class="tabs flex flex-row overflow-x-auto space-x-1 max-w-full lg:space-x-0 lg:space-y-1 lg:flex-col lg:flex-none lg:w-44 dark:text-gray-200 text-xs text-left scrollbar-none"
+	>
+		<button
+			class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 lg:flex-none flex text-right transition {selectedTab ===
+			'general'
+				? 'bg-gray-200 dark:bg-gray-800'
+				: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
+			on:click={() => {
+				selectedTab = 'general';
+			}}
+		>
+			<div class=" self-center mr-2">
+				<svg
+					xmlns="http://www.w3.org/2000/svg"
+					viewBox="0 0 16 16"
+					fill="currentColor"
+					class="w-4 h-4"
+				>
+					<path
+						fill-rule="evenodd"
+						d="M6.955 1.45A.5.5 0 0 1 7.452 1h1.096a.5.5 0 0 1 .497.45l.17 1.699c.484.12.94.312 1.356.562l1.321-1.081a.5.5 0 0 1 .67.033l.774.775a.5.5 0 0 1 .034.67l-1.08 1.32c.25.417.44.873.561 1.357l1.699.17a.5.5 0 0 1 .45.497v1.096a.5.5 0 0 1-.45.497l-1.699.17c-.12.484-.312.94-.562 1.356l1.082 1.322a.5.5 0 0 1-.034.67l-.774.774a.5.5 0 0 1-.67.033l-1.322-1.08c-.416.25-.872.44-1.356.561l-.17 1.699a.5.5 0 0 1-.497.45H7.452a.5.5 0 0 1-.497-.45l-.17-1.699a4.973 4.973 0 0 1-1.356-.562L4.108 13.37a.5.5 0 0 1-.67-.033l-.774-.775a.5.5 0 0 1-.034-.67l1.08-1.32a4.971 4.971 0 0 1-.561-1.357l-1.699-.17A.5.5 0 0 1 1 8.548V7.452a.5.5 0 0 1 .45-.497l1.699-.17c.12-.484.312-.94.562-1.356L2.629 4.107a.5.5 0 0 1 .034-.67l.774-.774a.5.5 0 0 1 .67-.033L5.43 3.71a4.97 4.97 0 0 1 1.356-.561l.17-1.699ZM6 8c0 .538.212 1.026.558 1.385l.057.057a2 2 0 0 0 2.828-2.828l-.058-.056A2 2 0 0 0 6 8Z"
+						clip-rule="evenodd"
+					/>
+				</svg>
+			</div>
+			<div class=" self-center">{$i18n.t('General')}</div>
+		</button>
+
+		<button
+			class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
+			'users'
+				? 'bg-gray-200 dark:bg-gray-800'
+				: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
+			on:click={() => {
+				selectedTab = 'users';
+			}}
+		>
+			<div class=" self-center mr-2">
+				<svg
+					xmlns="http://www.w3.org/2000/svg"
+					viewBox="0 0 16 16"
+					fill="currentColor"
+					class="w-4 h-4"
+				>
+					<path
+						d="M8 8a2.5 2.5 0 1 0 0-5 2.5 2.5 0 0 0 0 5ZM3.156 11.763c.16-.629.44-1.21.813-1.72a2.5 2.5 0 0 0-2.725 1.377c-.136.287.102.58.418.58h1.449c.01-.077.025-.156.045-.237ZM12.847 11.763c.02.08.036.16.046.237h1.446c.316 0 .554-.293.417-.579a2.5 2.5 0 0 0-2.722-1.378c.374.51.653 1.09.813 1.72ZM14 7.5a1.5 1.5 0 1 1-3 0 1.5 1.5 0 0 1 3 0ZM3.5 9a1.5 1.5 0 1 0 0-3 1.5 1.5 0 0 0 0 3ZM5 13c-.552 0-1.013-.455-.876-.99a4.002 4.002 0 0 1 7.753 0c.136.535-.324.99-.877.99H5Z"
+					/>
+				</svg>
+			</div>
+			<div class=" self-center">{$i18n.t('Users')}</div>
+		</button>
+
+		<button
+			class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
+			'connections'
+				? 'bg-gray-200 dark:bg-gray-800'
+				: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
+			on:click={() => {
+				selectedTab = 'connections';
+			}}
+		>
+			<div class=" self-center mr-2">
+				<svg
+					xmlns="http://www.w3.org/2000/svg"
+					viewBox="0 0 16 16"
+					fill="currentColor"
+					class="w-4 h-4"
+				>
+					<path
+						d="M1 9.5A3.5 3.5 0 0 0 4.5 13H12a3 3 0 0 0 .917-5.857 2.503 2.503 0 0 0-3.198-3.019 3.5 3.5 0 0 0-6.628 2.171A3.5 3.5 0 0 0 1 9.5Z"
+					/>
+				</svg>
+			</div>
+			<div class=" self-center">{$i18n.t('Connections')}</div>
+		</button>
+
+		<button
+			class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
+			'models'
+				? 'bg-gray-200 dark:bg-gray-800'
+				: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
+			on:click={() => {
+				selectedTab = 'models';
+			}}
+		>
+			<div class=" self-center mr-2">
+				<svg
+					xmlns="http://www.w3.org/2000/svg"
+					viewBox="0 0 20 20"
+					fill="currentColor"
+					class="w-4 h-4"
+				>
+					<path
+						fill-rule="evenodd"
+						d="M10 1c3.866 0 7 1.79 7 4s-3.134 4-7 4-7-1.79-7-4 3.134-4 7-4zm5.694 8.13c.464-.264.91-.583 1.306-.952V10c0 2.21-3.134 4-7 4s-7-1.79-7-4V8.178c.396.37.842.688 1.306.953C5.838 10.006 7.854 10.5 10 10.5s4.162-.494 5.694-1.37zM3 13.179V15c0 2.21 3.134 4 7 4s7-1.79 7-4v-1.822c-.396.37-.842.688-1.306.953-1.532.875-3.548 1.369-5.694 1.369s-4.162-.494-5.694-1.37A7.009 7.009 0 013 13.179z"
+						clip-rule="evenodd"
+					/>
+				</svg>
+			</div>
+			<div class=" self-center">{$i18n.t('Models')}</div>
+		</button>
+
+		<button
+			class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
+			'documents'
+				? 'bg-gray-200 dark:bg-gray-800'
+				: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
+			on:click={() => {
+				selectedTab = 'documents';
+			}}
+		>
+			<div class=" self-center mr-2">
+				<svg
+					xmlns="http://www.w3.org/2000/svg"
+					viewBox="0 0 24 24"
+					fill="currentColor"
+					class="w-4 h-4"
+				>
+					<path d="M11.625 16.5a1.875 1.875 0 1 0 0-3.75 1.875 1.875 0 0 0 0 3.75Z" />
+					<path
+						fill-rule="evenodd"
+						d="M5.625 1.5H9a3.75 3.75 0 0 1 3.75 3.75v1.875c0 1.036.84 1.875 1.875 1.875H16.5a3.75 3.75 0 0 1 3.75 3.75v7.875c0 1.035-.84 1.875-1.875 1.875H5.625a1.875 1.875 0 0 1-1.875-1.875V3.375c0-1.036.84-1.875 1.875-1.875Zm6 16.5c.66 0 1.277-.19 1.797-.518l1.048 1.048a.75.75 0 0 0 1.06-1.06l-1.047-1.048A3.375 3.375 0 1 0 11.625 18Z"
+						clip-rule="evenodd"
+					/>
+					<path
+						d="M14.25 5.25a5.23 5.23 0 0 0-1.279-3.434 9.768 9.768 0 0 1 6.963 6.963A5.23 5.23 0 0 0 16.5 7.5h-1.875a.375.375 0 0 1-.375-.375V5.25Z"
+					/>
+				</svg>
+			</div>
+			<div class=" self-center">{$i18n.t('Documents')}</div>
+		</button>
+
+		<button
+			class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
+			'web'
+				? 'bg-gray-200 dark:bg-gray-800'
+				: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
+			on:click={() => {
+				selectedTab = 'web';
+			}}
+		>
+			<div class=" self-center mr-2">
+				<svg
+					xmlns="http://www.w3.org/2000/svg"
+					viewBox="0 0 24 24"
+					fill="currentColor"
+					class="w-4 h-4"
+				>
+					<path
+						d="M21.721 12.752a9.711 9.711 0 0 0-.945-5.003 12.754 12.754 0 0 1-4.339 2.708 18.991 18.991 0 0 1-.214 4.772 17.165 17.165 0 0 0 5.498-2.477ZM14.634 15.55a17.324 17.324 0 0 0 .332-4.647c-.952.227-1.945.347-2.966.347-1.021 0-2.014-.12-2.966-.347a17.515 17.515 0 0 0 .332 4.647 17.385 17.385 0 0 0 5.268 0ZM9.772 17.119a18.963 18.963 0 0 0 4.456 0A17.182 17.182 0 0 1 12 21.724a17.18 17.18 0 0 1-2.228-4.605ZM7.777 15.23a18.87 18.87 0 0 1-.214-4.774 12.753 12.753 0 0 1-4.34-2.708 9.711 9.711 0 0 0-.944 5.004 17.165 17.165 0 0 0 5.498 2.477ZM21.356 14.752a9.765 9.765 0 0 1-7.478 6.817 18.64 18.64 0 0 0 1.988-4.718 18.627 18.627 0 0 0 5.49-2.098ZM2.644 14.752c1.682.971 3.53 1.688 5.49 2.099a18.64 18.64 0 0 0 1.988 4.718 9.765 9.765 0 0 1-7.478-6.816ZM13.878 2.43a9.755 9.755 0 0 1 6.116 3.986 11.267 11.267 0 0 1-3.746 2.504 18.63 18.63 0 0 0-2.37-6.49ZM12 2.276a17.152 17.152 0 0 1 2.805 7.121c-.897.23-1.837.353-2.805.353-.968 0-1.908-.122-2.805-.353A17.151 17.151 0 0 1 12 2.276ZM10.122 2.43a18.629 18.629 0 0 0-2.37 6.49 11.266 11.266 0 0 1-3.746-2.504 9.754 9.754 0 0 1 6.116-3.985Z"
+					/>
+				</svg>
+			</div>
+			<div class=" self-center">{$i18n.t('Web Search')}</div>
+		</button>
+
+		<button
+			class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
+			'interface'
+				? 'bg-gray-200 dark:bg-gray-800'
+				: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
+			on:click={() => {
+				selectedTab = 'interface';
+			}}
+		>
+			<div class=" self-center mr-2">
+				<svg
+					xmlns="http://www.w3.org/2000/svg"
+					viewBox="0 0 16 16"
+					fill="currentColor"
+					class="w-4 h-4"
+				>
+					<path
+						fill-rule="evenodd"
+						d="M2 4.25A2.25 2.25 0 0 1 4.25 2h7.5A2.25 2.25 0 0 1 14 4.25v5.5A2.25 2.25 0 0 1 11.75 12h-1.312c.1.128.21.248.328.36a.75.75 0 0 1 .234.545v.345a.75.75 0 0 1-.75.75h-4.5a.75.75 0 0 1-.75-.75v-.345a.75.75 0 0 1 .234-.545c.118-.111.228-.232.328-.36H4.25A2.25 2.25 0 0 1 2 9.75v-5.5Zm2.25-.75a.75.75 0 0 0-.75.75v4.5c0 .414.336.75.75.75h7.5a.75.75 0 0 0 .75-.75v-4.5a.75.75 0 0 0-.75-.75h-7.5Z"
+						clip-rule="evenodd"
+					/>
+				</svg>
+			</div>
+			<div class=" self-center">{$i18n.t('Interface')}</div>
+		</button>
+
+		<button
+			class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
+			'audio'
+				? 'bg-gray-200 dark:bg-gray-800'
+				: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
+			on:click={() => {
+				selectedTab = 'audio';
+			}}
+		>
+			<div class=" self-center mr-2">
+				<svg
+					xmlns="http://www.w3.org/2000/svg"
+					viewBox="0 0 16 16"
+					fill="currentColor"
+					class="w-4 h-4"
+				>
+					<path
+						d="M7.557 2.066A.75.75 0 0 1 8 2.75v10.5a.75.75 0 0 1-1.248.56L3.59 11H2a1 1 0 0 1-1-1V6a1 1 0 0 1 1-1h1.59l3.162-2.81a.75.75 0 0 1 .805-.124ZM12.95 3.05a.75.75 0 1 0-1.06 1.06 5.5 5.5 0 0 1 0 7.78.75.75 0 1 0 1.06 1.06 7 7 0 0 0 0-9.9Z"
+					/>
+					<path
+						d="M10.828 5.172a.75.75 0 1 0-1.06 1.06 2.5 2.5 0 0 1 0 3.536.75.75 0 1 0 1.06 1.06 4 4 0 0 0 0-5.656Z"
+					/>
+				</svg>
+			</div>
+			<div class=" self-center">{$i18n.t('Audio')}</div>
+		</button>
+
+		<button
+			class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
+			'images'
+				? 'bg-gray-200 dark:bg-gray-800'
+				: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
+			on:click={() => {
+				selectedTab = 'images';
+			}}
+		>
+			<div class=" self-center mr-2">
+				<svg
+					xmlns="http://www.w3.org/2000/svg"
+					viewBox="0 0 16 16"
+					fill="currentColor"
+					class="w-4 h-4"
+				>
+					<path
+						fill-rule="evenodd"
+						d="M2 4a2 2 0 0 1 2-2h8a2 2 0 0 1 2 2v8a2 2 0 0 1-2 2H4a2 2 0 0 1-2-2V4Zm10.5 5.707a.5.5 0 0 0-.146-.353l-1-1a.5.5 0 0 0-.708 0L9.354 9.646a.5.5 0 0 1-.708 0L6.354 7.354a.5.5 0 0 0-.708 0l-2 2a.5.5 0 0 0-.146.353V12a.5.5 0 0 0 .5.5h8a.5.5 0 0 0 .5-.5V9.707ZM12 5a1 1 0 1 1-2 0 1 1 0 0 1 2 0Z"
+						clip-rule="evenodd"
+					/>
+				</svg>
+			</div>
+			<div class=" self-center">{$i18n.t('Images')}</div>
+		</button>
+
+		<button
+			class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
+			'pipelines'
+				? 'bg-gray-200 dark:bg-gray-800'
+				: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
+			on:click={() => {
+				selectedTab = 'pipelines';
+			}}
+		>
+			<div class=" self-center mr-2">
+				<svg
+					xmlns="http://www.w3.org/2000/svg"
+					viewBox="0 0 24 24"
+					fill="currentColor"
+					class="size-4"
+				>
+					<path
+						d="M11.644 1.59a.75.75 0 0 1 .712 0l9.75 5.25a.75.75 0 0 1 0 1.32l-9.75 5.25a.75.75 0 0 1-.712 0l-9.75-5.25a.75.75 0 0 1 0-1.32l9.75-5.25Z"
+					/>
+					<path
+						d="m3.265 10.602 7.668 4.129a2.25 2.25 0 0 0 2.134 0l7.668-4.13 1.37.739a.75.75 0 0 1 0 1.32l-9.75 5.25a.75.75 0 0 1-.71 0l-9.75-5.25a.75.75 0 0 1 0-1.32l1.37-.738Z"
+					/>
+					<path
+						d="m10.933 19.231-7.668-4.13-1.37.739a.75.75 0 0 0 0 1.32l9.75 5.25c.221.12.489.12.71 0l9.75-5.25a.75.75 0 0 0 0-1.32l-1.37-.738-7.668 4.13a2.25 2.25 0 0 1-2.134-.001Z"
+					/>
+				</svg>
+			</div>
+			<div class=" self-center">{$i18n.t('Pipelines')}</div>
+		</button>
+
+		<button
+			class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
+			'db'
+				? 'bg-gray-200 dark:bg-gray-800'
+				: ' hover:bg-gray-300 dark:hover:bg-gray-850'}"
+			on:click={() => {
+				selectedTab = 'db';
+			}}
+		>
+			<div class=" self-center mr-2">
+				<svg
+					xmlns="http://www.w3.org/2000/svg"
+					viewBox="0 0 16 16"
+					fill="currentColor"
+					class="w-4 h-4"
+				>
+					<path d="M8 7c3.314 0 6-1.343 6-3s-2.686-3-6-3-6 1.343-6 3 2.686 3 6 3Z" />
+					<path
+						d="M8 8.5c1.84 0 3.579-.37 4.914-1.037A6.33 6.33 0 0 0 14 6.78V8c0 1.657-2.686 3-6 3S2 9.657 2 8V6.78c.346.273.72.5 1.087.683C4.42 8.131 6.16 8.5 8 8.5Z"
+					/>
+					<path
+						d="M8 12.5c1.84 0 3.579-.37 4.914-1.037.366-.183.74-.41 1.086-.684V12c0 1.657-2.686 3-6 3s-6-1.343-6-3v-1.22c.346.273.72.5 1.087.683C4.42 12.131 6.16 12.5 8 12.5Z"
+					/>
+				</svg>
+			</div>
+			<div class=" self-center">{$i18n.t('Database')}</div>
+		</button>
+	</div>
+
+	<div class="flex-1 mt-3 lg:mt-0 overflow-y-scroll">
+		{#if selectedTab === 'general'}
+			<General
+				saveHandler={() => {
+					toast.success($i18n.t('Settings saved successfully!'));
+				}}
+			/>
+		{:else if selectedTab === 'users'}
+			<Users
+				saveHandler={() => {
+					toast.success($i18n.t('Settings saved successfully!'));
+				}}
+			/>
+		{:else if selectedTab === 'connections'}
+			<Connections
+				on:save={() => {
+					toast.success($i18n.t('Settings saved successfully!'));
+				}}
+			/>
+		{:else if selectedTab === 'models'}
+			<Models />
+		{:else if selectedTab === 'documents'}
+			<Documents
+				saveHandler={() => {
+					toast.success($i18n.t('Settings saved successfully!'));
+				}}
+			/>
+		{:else if selectedTab === 'web'}
+			<WebSearch
+				saveHandler={async () => {
+					toast.success($i18n.t('Settings saved successfully!'));
+
+					await tick();
+					await config.set(await getBackendConfig());
+				}}
+			/>
+		{:else if selectedTab === 'interface'}
+			<Interface
+				on:save={() => {
+					toast.success($i18n.t('Settings saved successfully!'));
+				}}
+			/>
+		{:else if selectedTab === 'audio'}
+			<Audio
+				saveHandler={() => {
+					toast.success($i18n.t('Settings saved successfully!'));
+				}}
+			/>
+		{:else if selectedTab === 'images'}
+			<Images
+				on:save={() => {
+					toast.success($i18n.t('Settings saved successfully!'));
+				}}
+			/>
+		{:else if selectedTab === 'db'}
+			<Database
+				saveHandler={() => {
+					toast.success($i18n.t('Settings saved successfully!'));
+				}}
+			/>
+		{:else if selectedTab === 'pipelines'}
+			<Pipelines
+				saveHandler={() => {
+					toast.success($i18n.t('Settings saved successfully!'));
+				}}
+			/>
+		{/if}
+	</div>
+</div>

src/lib/components/admin/Settings/Audio.svelte

@@ -0,0 +1,302 @@
+<script lang="ts">
+	import { getAudioConfig, updateAudioConfig } from '$lib/apis/audio';
+	import { user, settings, config } from '$lib/stores';
+	import { createEventDispatcher, onMount, getContext } from 'svelte';
+	import { toast } from 'svelte-sonner';
+	import Switch from '$lib/components/common/Switch.svelte';
+	import { getBackendConfig } from '$lib/apis';
+	const dispatch = createEventDispatcher();
+
+	const i18n = getContext('i18n');
+
+	export let saveHandler: Function;
+
+	// Audio
+
+	let TTS_OPENAI_API_BASE_URL = '';
+	let TTS_OPENAI_API_KEY = '';
+	let TTS_ENGINE = '';
+	let TTS_MODEL = '';
+	let TTS_VOICE = '';
+
+	let STT_OPENAI_API_BASE_URL = '';
+	let STT_OPENAI_API_KEY = '';
+	let STT_ENGINE = '';
+	let STT_MODEL = '';
+
+	let voices = [];
+	let models = [];
+	let nonLocalVoices = false;
+
+	const getOpenAIVoices = () => {
+		voices = [
+			{ name: 'alloy' },
+			{ name: 'echo' },
+			{ name: 'fable' },
+			{ name: 'onyx' },
+			{ name: 'nova' },
+			{ name: 'shimmer' }
+		];
+	};
+
+	const getOpenAIModels = () => {
+		models = [{ name: 'tts-1' }, { name: 'tts-1-hd' }];
+	};
+
+	const getWebAPIVoices = () => {
+		const getVoicesLoop = setInterval(async () => {
+			voices = await speechSynthesis.getVoices();
+
+			// do your loop
+			if (voices.length > 0) {
+				clearInterval(getVoicesLoop);
+			}
+		}, 100);
+	};
+
+	const updateConfigHandler = async () => {
+		const res = await updateAudioConfig(localStorage.token, {
+			tts: {
+				OPENAI_API_BASE_URL: TTS_OPENAI_API_BASE_URL,
+				OPENAI_API_KEY: TTS_OPENAI_API_KEY,
+				ENGINE: TTS_ENGINE,
+				MODEL: TTS_MODEL,
+				VOICE: TTS_VOICE
+			},
+			stt: {
+				OPENAI_API_BASE_URL: STT_OPENAI_API_BASE_URL,
+				OPENAI_API_KEY: STT_OPENAI_API_KEY,
+				ENGINE: STT_ENGINE,
+				MODEL: STT_MODEL
+			}
+		});
+
+		if (res) {
+			toast.success('Audio settings updated successfully');
+
+			config.set(await getBackendConfig());
+		}
+	};
+
+	onMount(async () => {
+		const res = await getAudioConfig(localStorage.token);
+
+		if (res) {
+			console.log(res);
+			TTS_OPENAI_API_BASE_URL = res.tts.OPENAI_API_BASE_URL;
+			TTS_OPENAI_API_KEY = res.tts.OPENAI_API_KEY;
+
+			TTS_ENGINE = res.tts.ENGINE;
+			TTS_MODEL = res.tts.MODEL;
+			TTS_VOICE = res.tts.VOICE;
+
+			STT_OPENAI_API_BASE_URL = res.stt.OPENAI_API_BASE_URL;
+			STT_OPENAI_API_KEY = res.stt.OPENAI_API_KEY;
+
+			STT_ENGINE = res.stt.ENGINE;
+			STT_MODEL = res.stt.MODEL;
+		}
+
+		if (TTS_ENGINE === 'openai') {
+			getOpenAIVoices();
+			getOpenAIModels();
+		} else {
+			getWebAPIVoices();
+		}
+	});
+</script>
+
+<form
+	class="flex flex-col h-full justify-between space-y-3 text-sm"
+	on:submit|preventDefault={async () => {
+		await updateConfigHandler();
+		dispatch('save');
+	}}
+>
+	<div class=" space-y-3 overflow-y-scroll scrollbar-hidden h-full">
+		<div class="flex flex-col gap-3">
+			<div>
+				<div class=" mb-1 text-sm font-medium">{$i18n.t('STT Settings')}</div>
+
+				<div class=" py-0.5 flex w-full justify-between">
+					<div class=" self-center text-xs font-medium">{$i18n.t('Speech-to-Text Engine')}</div>
+					<div class="flex items-center relative">
+						<select
+							class="dark:bg-gray-900 w-fit pr-8 rounded px-2 p-1 text-xs bg-transparent outline-none text-right"
+							bind:value={STT_ENGINE}
+							placeholder="Select an engine"
+						>
+							<option value="">{$i18n.t('Whisper (Local)')}</option>
+							<option value="openai">OpenAI</option>
+							<option value="web">{$i18n.t('Web API')}</option>
+						</select>
+					</div>
+				</div>
+
+				{#if STT_ENGINE === 'openai'}
+					<div>
+						<div class="mt-1 flex gap-2 mb-1">
+							<input
+								class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+								placeholder={$i18n.t('API Base URL')}
+								bind:value={STT_OPENAI_API_BASE_URL}
+								required
+							/>
+
+							<input
+								class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+								placeholder={$i18n.t('API Key')}
+								bind:value={STT_OPENAI_API_KEY}
+								required
+							/>
+						</div>
+					</div>
+
+					<hr class=" dark:border-gray-850 my-2" />
+
+					<div>
+						<div class=" mb-1.5 text-sm font-medium">{$i18n.t('STT Model')}</div>
+						<div class="flex w-full">
+							<div class="flex-1">
+								<input
+									list="model-list"
+									class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+									bind:value={STT_MODEL}
+									placeholder="Select a model"
+								/>
+
+								<datalist id="model-list">
+									<option value="whisper-1" />
+								</datalist>
+							</div>
+						</div>
+					</div>
+				{/if}
+			</div>
+
+			<hr class=" dark:border-gray-800" />
+
+			<div>
+				<div class=" mb-1 text-sm font-medium">{$i18n.t('TTS Settings')}</div>
+
+				<div class=" py-0.5 flex w-full justify-between">
+					<div class=" self-center text-xs font-medium">{$i18n.t('Text-to-Speech Engine')}</div>
+					<div class="flex items-center relative">
+						<select
+							class=" dark:bg-gray-900 w-fit pr-8 rounded px-2 p-1 text-xs bg-transparent outline-none text-right"
+							bind:value={TTS_ENGINE}
+							placeholder="Select a mode"
+							on:change={(e) => {
+								if (e.target.value === 'openai') {
+									getOpenAIVoices();
+									TTS_VOICE = 'alloy';
+									TTS_MODEL = 'tts-1';
+								} else {
+									getWebAPIVoices();
+									TTS_VOICE = '';
+								}
+							}}
+						>
+							<option value="">{$i18n.t('Web API')}</option>
+							<option value="openai">{$i18n.t('Open AI')}</option>
+						</select>
+					</div>
+				</div>
+
+				{#if TTS_ENGINE === 'openai'}
+					<div>
+						<div class="mt-1 flex gap-2 mb-1">
+							<input
+								class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+								placeholder={$i18n.t('API Base URL')}
+								bind:value={TTS_OPENAI_API_BASE_URL}
+								required
+							/>
+
+							<input
+								class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+								placeholder={$i18n.t('API Key')}
+								bind:value={TTS_OPENAI_API_KEY}
+								required
+							/>
+						</div>
+					</div>
+				{/if}
+
+				<hr class=" dark:border-gray-850 my-2" />
+
+				{#if TTS_ENGINE === ''}
+					<div>
+						<div class=" mb-1.5 text-sm font-medium">{$i18n.t('TTS Voice')}</div>
+						<div class="flex w-full">
+							<div class="flex-1">
+								<select
+									class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+									bind:value={TTS_VOICE}
+								>
+									<option value="" selected={TTS_VOICE !== ''}>{$i18n.t('Default')}</option>
+									{#each voices as voice}
+										<option
+											value={voice.voiceURI}
+											class="bg-gray-100 dark:bg-gray-700"
+											selected={TTS_VOICE === voice.voiceURI}>{voice.name}</option
+										>
+									{/each}
+								</select>
+							</div>
+						</div>
+					</div>
+				{:else if TTS_ENGINE === 'openai'}
+					<div class=" flex gap-2">
+						<div class="w-full">
+							<div class=" mb-1.5 text-sm font-medium">{$i18n.t('TTS Voice')}</div>
+							<div class="flex w-full">
+								<div class="flex-1">
+									<input
+										list="voice-list"
+										class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+										bind:value={TTS_VOICE}
+										placeholder="Select a voice"
+									/>
+
+									<datalist id="voice-list">
+										{#each voices as voice}
+											<option value={voice.name} />
+										{/each}
+									</datalist>
+								</div>
+							</div>
+						</div>
+						<div class="w-full">
+							<div class=" mb-1.5 text-sm font-medium">{$i18n.t('TTS Model')}</div>
+							<div class="flex w-full">
+								<div class="flex-1">
+									<input
+										list="model-list"
+										class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+										bind:value={TTS_MODEL}
+										placeholder="Select a model"
+									/>
+
+									<datalist id="model-list">
+										{#each models as model}
+											<option value={model.name} />
+										{/each}
+									</datalist>
+								</div>
+							</div>
+						</div>
+					</div>
+				{/if}
+			</div>
+		</div>
+	</div>
+	<div class="flex justify-end text-sm font-medium">
+		<button
+			class=" px-4 py-2 bg-emerald-700 hover:bg-emerald-800 text-gray-100 transition rounded-lg"
+			type="submit"
+		>
+			{$i18n.t('Save')}
+		</button>
+	</div>
+</form>

src/lib/components/admin/Settings/Banners.svelte

@@ -1,137 +0,0 @@
-<script lang="ts">
-	import { v4 as uuidv4 } from 'uuid';
-
-	import { getContext, onMount } from 'svelte';
-	import { banners as _banners } from '$lib/stores';
-	import type { Banner } from '$lib/types';
-
-	import { getBanners, setBanners } from '$lib/apis/configs';
-
-	import type { Writable } from 'svelte/store';
-	import type { i18n as i18nType } from 'i18next';
-	import Tooltip from '$lib/components/common/Tooltip.svelte';
-	import Switch from '$lib/components/common/Switch.svelte';
-	const i18n: Writable<i18nType> = getContext('i18n');
-
-	export let saveHandler: Function;
-
-	let banners: Banner[] = [];
-
-	onMount(async () => {
-		banners = await getBanners(localStorage.token);
-	});
-
-	const updateBanners = async () => {
-		_banners.set(await setBanners(localStorage.token, banners));
-	};
-</script>
-
-<form
-	class="flex flex-col h-full justify-between space-y-3 text-sm"
-	on:submit|preventDefault={async () => {
-		updateBanners();
-		saveHandler();
-	}}
->
-	<div class=" space-y-3 pr-1.5 overflow-y-scroll max-h-80 h-full">
-		<div class=" space-y-3 pr-1.5">
-			<div class="flex w-full justify-between mb-2">
-				<div class=" self-center text-sm font-semibold">
-					{$i18n.t('Banners')}
-				</div>
-
-				<button
-					class="p-1 px-3 text-xs flex rounded transition"
-					type="button"
-					on:click={() => {
-						if (banners.length === 0 || banners.at(-1).content !== '') {
-							banners = [
-								...banners,
-								{
-									id: uuidv4(),
-									type: '',
-									title: '',
-									content: '',
-									dismissible: true,
-									timestamp: Math.floor(Date.now() / 1000)
-								}
-							];
-						}
-					}}
-				>
-					<svg
-						xmlns="http://www.w3.org/2000/svg"
-						viewBox="0 0 20 20"
-						fill="currentColor"
-						class="w-4 h-4"
-					>
-						<path
-							d="M10.75 4.75a.75.75 0 00-1.5 0v4.5h-4.5a.75.75 0 000 1.5h4.5v4.5a.75.75 0 001.5 0v-4.5h4.5a.75.75 0 000-1.5h-4.5v-4.5z"
-						/>
-					</svg>
-				</button>
-			</div>
-			<div class="flex flex-col space-y-1">
-				{#each banners as banner, bannerIdx}
-					<div class=" flex justify-between">
-						<div class="flex flex-row flex-1 border rounded-xl dark:border-gray-800">
-							<select
-								class="w-fit capitalize rounded-xl py-2 px-4 text-xs bg-transparent outline-none"
-								bind:value={banner.type}
-							>
-								{#if banner.type == ''}
-									<option value="" selected disabled class="text-gray-900">{$i18n.t('Type')}</option
-									>
-								{/if}
-								<option value="info" class="text-gray-900">{$i18n.t('Info')}</option>
-								<option value="warning" class="text-gray-900">{$i18n.t('Warning')}</option>
-								<option value="error" class="text-gray-900">{$i18n.t('Error')}</option>
-								<option value="success" class="text-gray-900">{$i18n.t('Success')}</option>
-							</select>
-
-							<input
-								class="pr-5 py-1.5 text-xs w-full bg-transparent outline-none"
-								placeholder={$i18n.t('Content')}
-								bind:value={banner.content}
-							/>
-
-							<div class="relative top-1.5 -left-2">
-								<Tooltip content="Dismissible" className="flex h-fit items-center">
-									<Switch bind:state={banner.dismissible} />
-								</Tooltip>
-							</div>
-						</div>
-
-						<button
-							class="px-2"
-							type="button"
-							on:click={() => {
-								banners.splice(bannerIdx, 1);
-								banners = banners;
-							}}
-						>
-							<svg
-								xmlns="http://www.w3.org/2000/svg"
-								viewBox="0 0 20 20"
-								fill="currentColor"
-								class="w-4 h-4"
-							>
-								<path
-									d="M6.28 5.22a.75.75 0 00-1.06 1.06L8.94 10l-3.72 3.72a.75.75 0 101.06 1.06L10 11.06l3.72 3.72a.75.75 0 101.06-1.06L11.06 10l3.72-3.72a.75.75 0 00-1.06-1.06L10 8.94 6.28 5.22z"
-								/>
-							</svg>
-						</button>
-					</div>
-				{/each}
-			</div>
-		</div>
-	</div>
-	<div class="flex justify-end pt-3 text-sm font-medium">
-		<button
-			class=" px-4 py-2 bg-emerald-700 hover:bg-emerald-800 text-gray-100 transition rounded-lg"
-			type="submit"
-		>
-			Save
-		</button>
-	</div>
-</form>

src/lib/components/chat/Settings/Connections.svelte → src/lib/components/admin/Settings/Connections.svelte

@@ -23,10 +23,14 @@
 	import Switch from '$lib/components/common/Switch.svelte';
 	import Spinner from '$lib/components/common/Spinner.svelte';
 	import Tooltip from '$lib/components/common/Tooltip.svelte';
+	import { getModels as _getModels } from '$lib/apis';
 
 	const i18n = getContext('i18n');
 
-	export let getModels: Function;
+	const getModels = async () => {
+		const models = await _getModels(localStorage.token);
+		return models;
+	};
 
 	// External
 	let OLLAMA_BASE_URLS = [''];
@@ -40,6 +44,8 @@
 	let ENABLE_OLLAMA_API = null;
 
 	const verifyOpenAIHandler = async (idx) => {
+		OPENAI_API_BASE_URLS = OPENAI_API_BASE_URLS.map((url) => url.replace(/\/$/, ''));
+
 		OPENAI_API_BASE_URLS = await updateOpenAIUrls(localStorage.token, OPENAI_API_BASE_URLS);
 		OPENAI_API_KEYS = await updateOpenAIKeys(localStorage.token, OPENAI_API_KEYS);
 
@@ -59,6 +65,10 @@
 	};
 
 	const verifyOllamaHandler = async (idx) => {
+		OLLAMA_BASE_URLS = OLLAMA_BASE_URLS.filter((url) => url !== '').map((url) =>
+			url.replace(/\/$/, '')
+		);
+
 		OLLAMA_BASE_URLS = await updateOllamaUrls(localStorage.token, OLLAMA_BASE_URLS);
 
 		const res = await getOllamaVersion(localStorage.token, idx).catch((error) => {
@@ -74,6 +84,8 @@
 	};
 
 	const updateOpenAIHandler = async () => {
+		OPENAI_API_BASE_URLS = OPENAI_API_BASE_URLS.map((url) => url.replace(/\/$/, ''));
+
 		// Check if API KEYS length is same than API URLS length
 		if (OPENAI_API_KEYS.length !== OPENAI_API_BASE_URLS.length) {
 			// if there are more keys than urls, remove the extra keys
@@ -96,7 +108,10 @@
 	};
 
 	const updateOllamaUrlsHandler = async () => {
-		OLLAMA_BASE_URLS = OLLAMA_BASE_URLS.filter((url) => url !== '');
+		OLLAMA_BASE_URLS = OLLAMA_BASE_URLS.filter((url) => url !== '').map((url) =>
+			url.replace(/\/$/, '')
+		);
+
 		console.log(OLLAMA_BASE_URLS);
 
 		if (OLLAMA_BASE_URLS.length === 0) {
@@ -158,7 +173,7 @@
 		dispatch('save');
 	}}
 >
-	<div class="space-y-3 pr-1.5 overflow-y-scroll h-[24rem] max-h-[25rem]">
+	<div class="space-y-3 overflow-y-scroll scrollbar-hidden h-full">
 		{#if ENABLE_OPENAI_API !== null && ENABLE_OLLAMA_API !== null}
 			<div class=" space-y-3">
 				<div class="mt-2 space-y-2 pr-1.5">
@@ -300,7 +315,7 @@
 				</div>
 			</div>
 
-			<hr class=" dark:border-gray-700" />
+			<hr class=" dark:border-gray-850" />
 
 			<div class="pr-1.5 space-y-2">
 				<div class="flex justify-between items-center text-sm">

src/lib/components/admin/Settings/Database.svelte

@@ -30,7 +30,7 @@
 		saveHandler();
 	}}
 >
-	<div class=" space-y-3 pr-1.5 overflow-y-scroll max-h-80">
+	<div class=" space-y-3 overflow-y-scroll scrollbar-hidden h-full">
 		<div>
 			<div class=" mb-2 text-sm font-medium">{$i18n.t('Database')}</div>
 

src/lib/components/documents/Settings/General.svelte → src/lib/components/admin/Settings/Documents.svelte

@@ -9,8 +9,12 @@
 		updateEmbeddingConfig,
 		getRerankingConfig,
 		updateRerankingConfig,
-		resetUploadDir
+		resetUploadDir,
+		getRAGConfig,
+		updateRAGConfig
 	} from '$lib/apis/rag';
+	import ResetUploadDirConfirmDialog from '$lib/components/common/ConfirmDialog.svelte';
+	import ResetVectorDBConfirmDialog from '$lib/components/common/ConfirmDialog.svelte';
 
 	import { documents, models } from '$lib/stores';
 	import { onMount, getContext } from 'svelte';
@@ -31,6 +35,10 @@
 	let embeddingModel = '';
 	let rerankingModel = '';
 
+	let chunkSize = 0;
+	let chunkOverlap = 0;
+	let pdfExtractImages = true;
+
 	let OpenAIKey = '';
 	let OpenAIUrl = '';
 	let OpenAIBatchSize = 1;
@@ -152,6 +160,16 @@
 		if (querySettings.hybrid) {
 			rerankingModelUpdateHandler();
 		}
+
+		const res = await updateRAGConfig(localStorage.token, {
+			pdf_extract_images: pdfExtractImages,
+			chunk: {
+				chunk_overlap: chunkOverlap,
+				chunk_size: chunkSize
+			}
+		});
+
+		await updateQuerySettings(localStorage.token, querySettings);
 	};
 
 	const setEmbeddingConfig = async () => {
@@ -185,9 +203,46 @@
 		await setRerankingConfig();
 
 		querySettings = await getQuerySettings(localStorage.token);
+
+		const res = await getRAGConfig(localStorage.token);
+
+		if (res) {
+			pdfExtractImages = res.pdf_extract_images;
+
+			chunkSize = res.chunk.chunk_size;
+			chunkOverlap = res.chunk.chunk_overlap;
+		}
 	});
 </script>
 
+<ResetUploadDirConfirmDialog
+	bind:show={showResetUploadDirConfirm}
+	on:confirm={() => {
+		const res = resetUploadDir(localStorage.token).catch((error) => {
+			toast.error(error);
+			return null;
+		});
+
+		if (res) {
+			toast.success($i18n.t('Success'));
+		}
+	}}
+/>
+
+<ResetVectorDBConfirmDialog
+	bind:show={showResetConfirm}
+	on:confirm={() => {
+		const res = resetVectorDB(localStorage.token).catch((error) => {
+			toast.error(error);
+			return null;
+		});
+
+		if (res) {
+			toast.success($i18n.t('Success'));
+		}
+	}}
+/>
+
 <form
 	class="flex flex-col h-full justify-between space-y-3 text-sm"
 	on:submit|preventDefault={() => {
@@ -195,7 +250,7 @@
 		saveHandler();
 	}}
 >
-	<div class=" space-y-2.5 pr-1.5 overflow-y-scroll max-h-[28rem]">
+	<div class=" space-y-2.5 overflow-y-scroll scrollbar-hidden h-full">
 		<div class="flex flex-col gap-0.5">
 			<div class=" mb-0.5 text-sm font-medium">{$i18n.t('General Settings')}</div>
 
@@ -332,7 +387,7 @@
 			</div>
 		</div>
 
-		<hr class=" dark:border-gray-700 my-1" />
+		<hr class=" dark:border-gray-850 my-1" />
 
 		<div class="space-y-2" />
 		<div>
@@ -350,10 +405,8 @@
 							{#if !embeddingModel}
 								<option value="" disabled selected>{$i18n.t('Select a model')}</option>
 							{/if}
-							{#each $models.filter((m) => m.id && !m.external) as model}
-								<option value={model.name} class="bg-gray-100 dark:bg-gray-700"
-									>{model.name + ' (' + (model.size / 1024 ** 3).toFixed(1) + ' GB)'}</option
-								>
+							{#each $models.filter((m) => m.id && m.ollama && !(m?.preset ?? false)) as model}
+								<option value={model.id} class="bg-gray-100 dark:bg-gray-700">{model.name}</option>
 							{/each}
 						</select>
 					</div>
@@ -500,200 +553,173 @@
 
 		<hr class=" dark:border-gray-850" />
 
-		<div>
-			{#if showResetUploadDirConfirm}
-				<div class="flex justify-between rounded-md items-center py-2 px-3.5 w-full transition">
-					<div class="flex items-center space-x-3">
-						<svg
-							xmlns="http://www.w3.org/2000/svg"
-							viewBox="0 0 24 24"
-							fill="currentColor"
-							class="size-4"
-						>
-							<path
-								fill-rule="evenodd"
-								d="M5.625 1.5H9a3.75 3.75 0 0 1 3.75 3.75v1.875c0 1.036.84 1.875 1.875 1.875H16.5a3.75 3.75 0 0 1 3.75 3.75v7.875c0 1.035-.84 1.875-1.875 1.875H5.625a1.875 1.875 0 0 1-1.875-1.875V3.375c0-1.036.84-1.875 1.875-1.875ZM9.75 14.25a.75.75 0 0 0 0 1.5H15a.75.75 0 0 0 0-1.5H9.75Z"
-								clip-rule="evenodd"
-							/>
-							<path
-								d="M14.25 5.25a5.23 5.23 0 0 0-1.279-3.434 9.768 9.768 0 0 1 6.963 6.963A5.23 5.23 0 0 0 16.5 7.5h-1.875a.375.375 0 0 1-.375-.375V5.25Z"
-							/>
-						</svg>
-						<span>{$i18n.t('Are you sure?')}</span>
-					</div>
-
-					<div class="flex space-x-1.5 items-center">
-						<button
-							class="hover:text-white transition"
-							on:click={() => {
-								const res = resetUploadDir(localStorage.token).catch((error) => {
-									toast.error(error);
-									return null;
-								});
+		<div class=" ">
+			<div class=" text-sm font-medium">{$i18n.t('Query Params')}</div>
 
-								if (res) {
-									toast.success($i18n.t('Success'));
-								}
+			<div class=" flex">
+				<div class="  flex w-full justify-between">
+					<div class="self-center text-xs font-medium min-w-fit">{$i18n.t('Top K')}</div>
 
-								showResetUploadDirConfirm = false;
-							}}
-							type="button"
-						>
-							<svg
-								xmlns="http://www.w3.org/2000/svg"
-								viewBox="0 0 20 20"
-								fill="currentColor"
-								class="w-4 h-4"
-							>
-								<path
-									fill-rule="evenodd"
-									d="M16.704 4.153a.75.75 0 01.143 1.052l-8 10.5a.75.75 0 01-1.127.075l-4.5-4.5a.75.75 0 011.06-1.06l3.894 3.893 7.48-9.817a.75.75 0 011.05-.143z"
-									clip-rule="evenodd"
-								/>
-							</svg>
-						</button>
-						<button
-							class="hover:text-white transition"
-							type="button"
-							on:click={() => {
-								showResetUploadDirConfirm = false;
-							}}
-						>
-							<svg
-								xmlns="http://www.w3.org/2000/svg"
-								viewBox="0 0 20 20"
-								fill="currentColor"
-								class="w-4 h-4"
-							>
-								<path
-									d="M6.28 5.22a.75.75 0 00-1.06 1.06L8.94 10l-3.72 3.72a.75.75 0 101.06 1.06L10 11.06l3.72 3.72a.75.75 0 101.06-1.06L11.06 10l3.72-3.72a.75.75 0 00-1.06-1.06L10 8.94 6.28 5.22z"
-								/>
-							</svg>
-						</button>
+					<div class="self-center p-3">
+						<input
+							class=" w-full rounded-lg py-1.5 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+							type="number"
+							placeholder={$i18n.t('Enter Top K')}
+							bind:value={querySettings.k}
+							autocomplete="off"
+							min="0"
+						/>
 					</div>
 				</div>
-			{:else}
-				<button
-					class=" flex rounded-xl py-2 px-3.5 w-full hover:bg-gray-200 dark:hover:bg-gray-800 transition"
-					on:click={() => {
-						showResetUploadDirConfirm = true;
-					}}
-					type="button"
-				>
-					<div class=" self-center mr-3">
-						<svg
-							xmlns="http://www.w3.org/2000/svg"
-							viewBox="0 0 24 24"
-							fill="currentColor"
-							class="size-4"
-						>
-							<path
-								fill-rule="evenodd"
-								d="M5.625 1.5H9a3.75 3.75 0 0 1 3.75 3.75v1.875c0 1.036.84 1.875 1.875 1.875H16.5a3.75 3.75 0 0 1 3.75 3.75v7.875c0 1.035-.84 1.875-1.875 1.875H5.625a1.875 1.875 0 0 1-1.875-1.875V3.375c0-1.036.84-1.875 1.875-1.875ZM9.75 14.25a.75.75 0 0 0 0 1.5H15a.75.75 0 0 0 0-1.5H9.75Z"
-								clip-rule="evenodd"
-							/>
-							<path
-								d="M14.25 5.25a5.23 5.23 0 0 0-1.279-3.434 9.768 9.768 0 0 1 6.963 6.963A5.23 5.23 0 0 0 16.5 7.5h-1.875a.375.375 0 0 1-.375-.375V5.25Z"
+
+				{#if querySettings.hybrid === true}
+					<div class="flex w-full">
+						<div class=" self-center text-xs font-medium min-w-fit">
+							{$i18n.t('Minimum Score')}
+						</div>
+
+						<div class="self-center p-3">
+							<input
+								class=" w-full rounded-lg py-1.5 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+								type="number"
+								step="0.01"
+								placeholder={$i18n.t('Enter Score')}
+								bind:value={querySettings.r}
+								autocomplete="off"
+								min="0.0"
+								title={$i18n.t('The score should be a value between 0.0 (0%) and 1.0 (100%).')}
 							/>
-						</svg>
+						</div>
 					</div>
-					<div class=" self-center text-sm font-medium">{$i18n.t('Reset Upload Directory')}</div>
-				</button>
+				{/if}
+			</div>
+
+			{#if querySettings.hybrid === true}
+				<div class="mt-2 mb-1 text-xs text-gray-400 dark:text-gray-500">
+					{$i18n.t(
+						'Note: If you set a minimum score, the search will only return documents with a score greater than or equal to the minimum score.'
+					)}
+				</div>
+
+				<hr class=" dark:border-gray-850 my-3" />
 			{/if}
 
-			{#if showResetConfirm}
-				<div class="flex justify-between rounded-md items-center py-2 px-3.5 w-full transition">
-					<div class="flex items-center space-x-3">
-						<svg
-							xmlns="http://www.w3.org/2000/svg"
-							viewBox="0 0 16 16"
-							fill="currentColor"
-							class="w-4 h-4"
-						>
-							<path
-								fill-rule="evenodd"
-								d="M3.5 2A1.5 1.5 0 0 0 2 3.5v9A1.5 1.5 0 0 0 3.5 14h9a1.5 1.5 0 0 0 1.5-1.5v-7A1.5 1.5 0 0 0 12.5 4H9.621a1.5 1.5 0 0 1-1.06-.44L7.439 2.44A1.5 1.5 0 0 0 6.38 2H3.5Zm6.75 7.75a.75.75 0 0 0 0-1.5h-4.5a.75.75 0 0 0 0 1.5h4.5Z"
-								clip-rule="evenodd"
-							/>
-						</svg>
-						<span>{$i18n.t('Are you sure?')}</span>
-					</div>
+			<div>
+				<div class=" mb-2.5 text-sm font-medium">{$i18n.t('RAG Template')}</div>
+				<textarea
+					bind:value={querySettings.template}
+					class="w-full rounded-lg px-4 py-3 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none resize-none"
+					rows="4"
+				/>
+			</div>
+		</div>
 
-					<div class="flex space-x-1.5 items-center">
-						<button
-							class="hover:text-white transition"
-							on:click={() => {
-								const res = resetVectorDB(localStorage.token).catch((error) => {
-									toast.error(error);
-									return null;
-								});
+		<hr class=" dark:border-gray-850" />
 
-								if (res) {
-									toast.success($i18n.t('Success'));
-								}
+		<div class=" ">
+			<div class=" text-sm font-medium">{$i18n.t('Chunk Params')}</div>
 
-								showResetConfirm = false;
-							}}
-							type="button"
-						>
-							<svg
-								xmlns="http://www.w3.org/2000/svg"
-								viewBox="0 0 20 20"
-								fill="currentColor"
-								class="w-4 h-4"
-							>
-								<path
-									fill-rule="evenodd"
-									d="M16.704 4.153a.75.75 0 01.143 1.052l-8 10.5a.75.75 0 01-1.127.075l-4.5-4.5a.75.75 0 011.06-1.06l3.894 3.893 7.48-9.817a.75.75 0 011.05-.143z"
-									clip-rule="evenodd"
-								/>
-							</svg>
-						</button>
-						<button
-							class="hover:text-white transition"
-							on:click={() => {
-								showResetConfirm = false;
-							}}
-							type="button"
-						>
-							<svg
-								xmlns="http://www.w3.org/2000/svg"
-								viewBox="0 0 20 20"
-								fill="currentColor"
-								class="w-4 h-4"
-							>
-								<path
-									d="M6.28 5.22a.75.75 0 00-1.06 1.06L8.94 10l-3.72 3.72a.75.75 0 101.06 1.06L10 11.06l3.72 3.72a.75.75 0 101.06-1.06L11.06 10l3.72-3.72a.75.75 0 00-1.06-1.06L10 8.94 6.28 5.22z"
-								/>
-							</svg>
-						</button>
+			<div class=" my-2 flex gap-1.5">
+				<div class="  w-full justify-between">
+					<div class="self-center text-xs font-medium min-w-fit mb-1">{$i18n.t('Chunk Size')}</div>
+					<div class="self-center">
+						<input
+							class=" w-full rounded-lg py-1.5 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+							type="number"
+							placeholder={$i18n.t('Enter Chunk Size')}
+							bind:value={chunkSize}
+							autocomplete="off"
+							min="0"
+						/>
 					</div>
 				</div>
-			{:else}
-				<button
-					class=" flex rounded-xl py-2 px-3.5 w-full hover:bg-gray-200 dark:hover:bg-gray-800 transition"
-					on:click={() => {
-						showResetConfirm = true;
-					}}
-					type="button"
-				>
-					<div class=" self-center mr-3">
-						<svg
-							xmlns="http://www.w3.org/2000/svg"
-							viewBox="0 0 16 16"
-							fill="currentColor"
-							class="w-4 h-4"
-						>
-							<path
-								fill-rule="evenodd"
-								d="M3.5 2A1.5 1.5 0 0 0 2 3.5v9A1.5 1.5 0 0 0 3.5 14h9a1.5 1.5 0 0 0 1.5-1.5v-7A1.5 1.5 0 0 0 12.5 4H9.621a1.5 1.5 0 0 1-1.06-.44L7.439 2.44A1.5 1.5 0 0 0 6.38 2H3.5Zm6.75 7.75a.75.75 0 0 0 0-1.5h-4.5a.75.75 0 0 0 0 1.5h4.5Z"
-								clip-rule="evenodd"
-							/>
-						</svg>
+
+				<div class="w-full">
+					<div class=" self-center text-xs font-medium min-w-fit mb-1">
+						{$i18n.t('Chunk Overlap')}
 					</div>
-					<div class=" self-center text-sm font-medium">{$i18n.t('Reset Vector Storage')}</div>
-				</button>
-			{/if}
+
+					<div class="self-center">
+						<input
+							class="w-full rounded-lg py-1.5 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+							type="number"
+							placeholder={$i18n.t('Enter Chunk Overlap')}
+							bind:value={chunkOverlap}
+							autocomplete="off"
+							min="0"
+						/>
+					</div>
+				</div>
+			</div>
+
+			<div class="my-3">
+				<div class="flex justify-between items-center text-xs">
+					<div class=" text-xs font-medium">{$i18n.t('PDF Extract Images (OCR)')}</div>
+
+					<button
+						class=" text-xs font-medium text-gray-500"
+						type="button"
+						on:click={() => {
+							pdfExtractImages = !pdfExtractImages;
+						}}>{pdfExtractImages ? $i18n.t('On') : $i18n.t('Off')}</button
+					>
+				</div>
+			</div>
+		</div>
+
+		<hr class=" dark:border-gray-850" />
+
+		<div>
+			<button
+				class=" flex rounded-xl py-2 px-3.5 w-full hover:bg-gray-200 dark:hover:bg-gray-800 transition"
+				on:click={() => {
+					showResetUploadDirConfirm = true;
+				}}
+				type="button"
+			>
+				<div class=" self-center mr-3">
+					<svg
+						xmlns="http://www.w3.org/2000/svg"
+						viewBox="0 0 24 24"
+						fill="currentColor"
+						class="size-4"
+					>
+						<path
+							fill-rule="evenodd"
+							d="M5.625 1.5H9a3.75 3.75 0 0 1 3.75 3.75v1.875c0 1.036.84 1.875 1.875 1.875H16.5a3.75 3.75 0 0 1 3.75 3.75v7.875c0 1.035-.84 1.875-1.875 1.875H5.625a1.875 1.875 0 0 1-1.875-1.875V3.375c0-1.036.84-1.875 1.875-1.875ZM9.75 14.25a.75.75 0 0 0 0 1.5H15a.75.75 0 0 0 0-1.5H9.75Z"
+							clip-rule="evenodd"
+						/>
+						<path
+							d="M14.25 5.25a5.23 5.23 0 0 0-1.279-3.434 9.768 9.768 0 0 1 6.963 6.963A5.23 5.23 0 0 0 16.5 7.5h-1.875a.375.375 0 0 1-.375-.375V5.25Z"
+						/>
+					</svg>
+				</div>
+				<div class=" self-center text-sm font-medium">{$i18n.t('Reset Upload Directory')}</div>
+			</button>
+
+			<button
+				class=" flex rounded-xl py-2 px-3.5 w-full hover:bg-gray-200 dark:hover:bg-gray-800 transition"
+				on:click={() => {
+					showResetConfirm = true;
+				}}
+				type="button"
+			>
+				<div class=" self-center mr-3">
+					<svg
+						xmlns="http://www.w3.org/2000/svg"
+						viewBox="0 0 16 16"
+						fill="currentColor"
+						class="w-4 h-4"
+					>
+						<path
+							fill-rule="evenodd"
+							d="M3.5 2A1.5 1.5 0 0 0 2 3.5v9A1.5 1.5 0 0 0 3.5 14h9a1.5 1.5 0 0 0 1.5-1.5v-7A1.5 1.5 0 0 0 12.5 4H9.621a1.5 1.5 0 0 1-1.06-.44L7.439 2.44A1.5 1.5 0 0 0 6.38 2H3.5Zm6.75 7.75a.75.75 0 0 0 0-1.5h-4.5a.75.75 0 0 0 0 1.5h4.5Z"
+							clip-rule="evenodd"
+						/>
+					</svg>
+				</div>
+				<div class=" self-center text-sm font-medium">{$i18n.t('Reset Vector Storage')}</div>
+			</button>
 		</div>
 	</div>
 	<div class="flex justify-end pt-3 text-sm font-medium">

src/lib/components/admin/Settings/General.svelte

@@ -56,7 +56,7 @@
 		saveHandler();
 	}}
 >
-	<div class=" space-y-3 pr-1.5 overflow-y-scroll max-h-[22rem]">
+	<div class=" space-y-3 overflow-y-scroll scrollbar-hidden h-full">
 		{#if adminConfig !== null}
 			<div>
 				<div class=" mb-3 text-sm font-medium">{$i18n.t('General Settings')}</div>

src/lib/components/chat/Settings/Images.svelte → src/lib/components/admin/Settings/Images.svelte

@@ -23,8 +23,6 @@
 
 	const i18n = getContext('i18n');
 
-	export let saveSettings: Function;
-
 	let loading = false;
 
 	let imageGenerationEngine = '';
@@ -171,7 +169,7 @@
 		loading = false;
 	}}
 >
-	<div class=" space-y-3 pr-1.5 overflow-y-scroll max-h-[24rem]">
+	<div class=" space-y-3 overflow-y-scroll scrollbar-hidden">
 		<div>
 			<div class=" mb-1 text-sm font-medium">{$i18n.t('Image Settings')}</div>
 
@@ -228,7 +226,7 @@
 				</div>
 			</div>
 		</div>
-		<hr class=" dark:border-gray-700" />
+		<hr class=" dark:border-gray-850" />
 
 		{#if imageGenerationEngine === ''}
 			<div class=" mb-2.5 text-sm font-medium">{$i18n.t('AUTOMATIC1111 Base URL')}</div>
@@ -326,7 +324,7 @@
 		{/if}
 
 		{#if enableImageGeneration}
-			<hr class=" dark:border-gray-700" />
+			<hr class=" dark:border-gray-850" />
 
 			<div>
 				<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Set Default Model')}</div>

src/lib/components/admin/Settings/Interface.svelte

@@ -0,0 +1,339 @@
+<script lang="ts">
+	import { v4 as uuidv4 } from 'uuid';
+	import { toast } from 'svelte-sonner';
+
+	import { getBackendConfig, getTaskConfig, updateTaskConfig } from '$lib/apis';
+	import { setDefaultPromptSuggestions } from '$lib/apis/configs';
+	import { config, models, settings, user } from '$lib/stores';
+	import { createEventDispatcher, onMount, getContext } from 'svelte';
+
+	import { banners as _banners } from '$lib/stores';
+	import type { Banner } from '$lib/types';
+
+	import { getBanners, setBanners } from '$lib/apis/configs';
+
+	import Tooltip from '$lib/components/common/Tooltip.svelte';
+	import Switch from '$lib/components/common/Switch.svelte';
+
+	const dispatch = createEventDispatcher();
+
+	const i18n = getContext('i18n');
+
+	let taskConfig = {
+		TASK_MODEL: '',
+		TASK_MODEL_EXTERNAL: '',
+		TITLE_GENERATION_PROMPT_TEMPLATE: '',
+		SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE: '',
+		SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD: 0
+	};
+
+	let promptSuggestions = [];
+	let banners: Banner[] = [];
+
+	const updateInterfaceHandler = async () => {
+		taskConfig = await updateTaskConfig(localStorage.token, taskConfig);
+
+		promptSuggestions = await setDefaultPromptSuggestions(localStorage.token, promptSuggestions);
+		await updateBanners();
+
+		await config.set(await getBackendConfig());
+	};
+
+	onMount(async () => {
+		taskConfig = await getTaskConfig(localStorage.token);
+
+		promptSuggestions = $config?.default_prompt_suggestions;
+
+		banners = await getBanners(localStorage.token);
+	});
+
+	const updateBanners = async () => {
+		_banners.set(await setBanners(localStorage.token, banners));
+	};
+</script>
+
+<form
+	class="flex flex-col h-full justify-between space-y-3 text-sm"
+	on:submit|preventDefault={() => {
+		updateInterfaceHandler();
+		dispatch('save');
+	}}
+>
+	<div class="  overflow-y-scroll scrollbar-hidden h-full pr-1.5">
+		<div>
+			<div class=" mb-2.5 text-sm font-medium flex">
+				<div class=" mr-1">{$i18n.t('Set Task Model')}</div>
+				<Tooltip
+					content={$i18n.t(
+						'A task model is used when performing tasks such as generating titles for chats and web search queries'
+					)}
+				>
+					<svg
+						xmlns="http://www.w3.org/2000/svg"
+						fill="none"
+						viewBox="0 0 24 24"
+						stroke-width="1.5"
+						stroke="currentColor"
+						class="w-5 h-5"
+					>
+						<path
+							stroke-linecap="round"
+							stroke-linejoin="round"
+							d="m11.25 11.25.041-.02a.75.75 0 0 1 1.063.852l-.708 2.836a.75.75 0 0 0 1.063.853l.041-.021M21 12a9 9 0 1 1-18 0 9 9 0 0 1 18 0Zm-9-3.75h.008v.008H12V8.25Z"
+						/>
+					</svg>
+				</Tooltip>
+			</div>
+			<div class="flex w-full gap-2">
+				<div class="flex-1">
+					<div class=" text-xs mb-1">{$i18n.t('Local Models')}</div>
+					<select
+						class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+						bind:value={taskConfig.TASK_MODEL}
+						placeholder={$i18n.t('Select a model')}
+					>
+						<option value="" selected>{$i18n.t('Current Model')}</option>
+						{#each $models.filter((m) => m.owned_by === 'ollama') as model}
+							<option value={model.id} class="bg-gray-100 dark:bg-gray-700">
+								{model.name}
+							</option>
+						{/each}
+					</select>
+				</div>
+
+				<div class="flex-1">
+					<div class=" text-xs mb-1">{$i18n.t('External Models')}</div>
+					<select
+						class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+						bind:value={taskConfig.TASK_MODEL_EXTERNAL}
+						placeholder={$i18n.t('Select a model')}
+					>
+						<option value="" selected>{$i18n.t('Current Model')}</option>
+						{#each $models as model}
+							<option value={model.id} class="bg-gray-100 dark:bg-gray-700">
+								{model.name}
+							</option>
+						{/each}
+					</select>
+				</div>
+			</div>
+
+			<div class="mt-3">
+				<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Title Generation Prompt')}</div>
+				<textarea
+					bind:value={taskConfig.TITLE_GENERATION_PROMPT_TEMPLATE}
+					class="w-full rounded-lg py-3 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none resize-none"
+					rows="6"
+				/>
+			</div>
+
+			<div class="mt-3">
+				<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Search Query Generation Prompt')}</div>
+				<textarea
+					bind:value={taskConfig.SEARCH_QUERY_GENERATION_PROMPT_TEMPLATE}
+					class="w-full rounded-lg py-3 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none resize-none"
+					rows="6"
+				/>
+			</div>
+
+			<div class="mt-3">
+				<div class=" mb-2.5 text-sm font-medium">
+					{$i18n.t('Search Query Generation Prompt Length Threshold')}
+				</div>
+				<input
+					bind:value={taskConfig.SEARCH_QUERY_PROMPT_LENGTH_THRESHOLD}
+					class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none resize-none"
+					type="number"
+				/>
+			</div>
+		</div>
+
+		<hr class=" dark:border-gray-850 my-3" />
+
+		<div class=" space-y-3 {banners.length > 0 ? ' mb-3' : ''}">
+			<div class="flex w-full justify-between">
+				<div class=" self-center text-sm font-semibold">
+					{$i18n.t('Banners')}
+				</div>
+
+				<button
+					class="p-1 px-3 text-xs flex rounded transition"
+					type="button"
+					on:click={() => {
+						if (banners.length === 0 || banners.at(-1).content !== '') {
+							banners = [
+								...banners,
+								{
+									id: uuidv4(),
+									type: '',
+									title: '',
+									content: '',
+									dismissible: true,
+									timestamp: Math.floor(Date.now() / 1000)
+								}
+							];
+						}
+					}}
+				>
+					<svg
+						xmlns="http://www.w3.org/2000/svg"
+						viewBox="0 0 20 20"
+						fill="currentColor"
+						class="w-4 h-4"
+					>
+						<path
+							d="M10.75 4.75a.75.75 0 00-1.5 0v4.5h-4.5a.75.75 0 000 1.5h4.5v4.5a.75.75 0 001.5 0v-4.5h4.5a.75.75 0 000-1.5h-4.5v-4.5z"
+						/>
+					</svg>
+				</button>
+			</div>
+			<div class="flex flex-col space-y-1">
+				{#each banners as banner, bannerIdx}
+					<div class=" flex justify-between">
+						<div class="flex flex-row flex-1 border rounded-xl dark:border-gray-800">
+							<select
+								class="w-fit capitalize rounded-xl py-2 px-4 text-xs bg-transparent outline-none"
+								bind:value={banner.type}
+								required
+							>
+								{#if banner.type == ''}
+									<option value="" selected disabled class="text-gray-900">{$i18n.t('Type')}</option
+									>
+								{/if}
+								<option value="info" class="text-gray-900">{$i18n.t('Info')}</option>
+								<option value="warning" class="text-gray-900">{$i18n.t('Warning')}</option>
+								<option value="error" class="text-gray-900">{$i18n.t('Error')}</option>
+								<option value="success" class="text-gray-900">{$i18n.t('Success')}</option>
+							</select>
+
+							<input
+								class="pr-5 py-1.5 text-xs w-full bg-transparent outline-none"
+								placeholder={$i18n.t('Content')}
+								bind:value={banner.content}
+							/>
+
+							<div class="relative top-1.5 -left-2">
+								<Tooltip content={$i18n.t('Dismissible')} className="flex h-fit items-center">
+									<Switch bind:state={banner.dismissible} />
+								</Tooltip>
+							</div>
+						</div>
+
+						<button
+							class="px-2"
+							type="button"
+							on:click={() => {
+								banners.splice(bannerIdx, 1);
+								banners = banners;
+							}}
+						>
+							<svg
+								xmlns="http://www.w3.org/2000/svg"
+								viewBox="0 0 20 20"
+								fill="currentColor"
+								class="w-4 h-4"
+							>
+								<path
+									d="M6.28 5.22a.75.75 0 00-1.06 1.06L8.94 10l-3.72 3.72a.75.75 0 101.06 1.06L10 11.06l3.72 3.72a.75.75 0 101.06-1.06L11.06 10l3.72-3.72a.75.75 0 00-1.06-1.06L10 8.94 6.28 5.22z"
+								/>
+							</svg>
+						</button>
+					</div>
+				{/each}
+			</div>
+		</div>
+
+		{#if $user.role === 'admin'}
+			<div class=" space-y-3">
+				<div class="flex w-full justify-between mb-2">
+					<div class=" self-center text-sm font-semibold">
+						{$i18n.t('Default Prompt Suggestions')}
+					</div>
+
+					<button
+						class="p-1 px-3 text-xs flex rounded transition"
+						type="button"
+						on:click={() => {
+							if (promptSuggestions.length === 0 || promptSuggestions.at(-1).content !== '') {
+								promptSuggestions = [...promptSuggestions, { content: '', title: ['', ''] }];
+							}
+						}}
+					>
+						<svg
+							xmlns="http://www.w3.org/2000/svg"
+							viewBox="0 0 20 20"
+							fill="currentColor"
+							class="w-4 h-4"
+						>
+							<path
+								d="M10.75 4.75a.75.75 0 00-1.5 0v4.5h-4.5a.75.75 0 000 1.5h4.5v4.5a.75.75 0 001.5 0v-4.5h4.5a.75.75 0 000-1.5h-4.5v-4.5z"
+							/>
+						</svg>
+					</button>
+				</div>
+				<div class="grid lg:grid-cols-2 flex-col gap-1.5">
+					{#each promptSuggestions as prompt, promptIdx}
+						<div class=" flex dark:bg-gray-850 rounded-xl py-1.5">
+							<div class="flex flex-col flex-1 pl-1">
+								<div class="flex border-b dark:border-gray-800 w-full">
+									<input
+										class="px-3 py-1.5 text-xs w-full bg-transparent outline-none border-r dark:border-gray-800"
+										placeholder={$i18n.t('Title (e.g. Tell me a fun fact)')}
+										bind:value={prompt.title[0]}
+									/>
+
+									<input
+										class="px-3 py-1.5 text-xs w-full bg-transparent outline-none border-r dark:border-gray-800"
+										placeholder={$i18n.t('Subtitle (e.g. about the Roman Empire)')}
+										bind:value={prompt.title[1]}
+									/>
+								</div>
+
+								<input
+									class="px-3 py-1.5 text-xs w-full bg-transparent outline-none border-r dark:border-gray-800"
+									placeholder={$i18n.t('Prompt (e.g. Tell me a fun fact about the Roman Empire)')}
+									bind:value={prompt.content}
+								/>
+							</div>
+
+							<button
+								class="px-3"
+								type="button"
+								on:click={() => {
+									promptSuggestions.splice(promptIdx, 1);
+									promptSuggestions = promptSuggestions;
+								}}
+							>
+								<svg
+									xmlns="http://www.w3.org/2000/svg"
+									viewBox="0 0 20 20"
+									fill="currentColor"
+									class="w-4 h-4"
+								>
+									<path
+										d="M6.28 5.22a.75.75 0 00-1.06 1.06L8.94 10l-3.72 3.72a.75.75 0 101.06 1.06L10 11.06l3.72 3.72a.75.75 0 101.06-1.06L11.06 10l3.72-3.72a.75.75 0 00-1.06-1.06L10 8.94 6.28 5.22z"
+									/>
+								</svg>
+							</button>
+						</div>
+					{/each}
+				</div>
+
+				{#if promptSuggestions.length > 0}
+					<div class="text-xs text-left w-full mt-2">
+						{$i18n.t('Adjusting these settings will apply changes universally to all users.')}
+					</div>
+				{/if}
+			</div>
+		{/if}
+	</div>
+
+	<div class="flex justify-end text-sm font-medium">
+		<button
+			class=" px-4 py-2 bg-emerald-700 hover:bg-emerald-800 text-gray-100 transition rounded-lg"
+			type="submit"
+		>
+			{$i18n.t('Save')}
+		</button>
+	</div>
+</form>

src/lib/components/admin/Settings/Models.svelte

@@ -0,0 +1,1088 @@
+<script lang="ts">
+	import { toast } from 'svelte-sonner';
+	import { onMount, getContext } from 'svelte';
+
+	import { WEBUI_API_BASE_URL, WEBUI_BASE_URL } from '$lib/constants';
+	import { WEBUI_NAME, models, MODEL_DOWNLOAD_POOL, user, config } from '$lib/stores';
+	import { splitStream } from '$lib/utils';
+
+	import {
+		createModel,
+		deleteModel,
+		downloadModel,
+		getOllamaUrls,
+		getOllamaVersion,
+		pullModel,
+		uploadModel,
+		getOllamaConfig
+	} from '$lib/apis/ollama';
+	import { getModels as _getModels } from '$lib/apis';
+
+	import Tooltip from '$lib/components/common/Tooltip.svelte';
+	import Spinner from '$lib/components/common/Spinner.svelte';
+	import ModelDeleteConfirmDialog from '$lib/components/common/ConfirmDialog.svelte';
+
+	const i18n = getContext('i18n');
+
+	const getModels = async () => {
+		return await _getModels(localStorage.token);
+	};
+
+	let modelUploadInputElement: HTMLInputElement;
+
+	let showModelDeleteConfirm = false;
+
+	// Models
+
+	let ollamaEnabled = null;
+
+	let OLLAMA_URLS = [];
+	let selectedOllamaUrlIdx: string | null = null;
+
+	let updateModelId = null;
+	let updateProgress = null;
+
+	let showExperimentalOllama = false;
+
+	let ollamaVersion = null;
+	const MAX_PARALLEL_DOWNLOADS = 3;
+
+	let modelTransferring = false;
+	let modelTag = '';
+
+	let createModelLoading = false;
+	let createModelTag = '';
+	let createModelContent = '';
+	let createModelDigest = '';
+	let createModelPullProgress = null;
+
+	let digest = '';
+	let pullProgress = null;
+
+	let modelUploadMode = 'file';
+	let modelInputFile: File[] | null = null;
+	let modelFileUrl = '';
+	let modelFileContent = `TEMPLATE """{{ .System }}\nUSER: {{ .Prompt }}\nASSISTANT: """\nPARAMETER num_ctx 4096\nPARAMETER stop "</s>"\nPARAMETER stop "USER:"\nPARAMETER stop "ASSISTANT:"`;
+	let modelFileDigest = '';
+
+	let uploadProgress = null;
+	let uploadMessage = '';
+
+	let deleteModelTag = '';
+
+	const updateModelsHandler = async () => {
+		for (const model of $models.filter(
+			(m) =>
+				!(m?.preset ?? false) &&
+				m.owned_by === 'ollama' &&
+				(selectedOllamaUrlIdx === null
+					? true
+					: (m?.ollama?.urls ?? []).includes(selectedOllamaUrlIdx))
+		)) {
+			console.log(model);
+
+			updateModelId = model.id;
+			const [res, controller] = await pullModel(
+				localStorage.token,
+				model.id,
+				selectedOllamaUrlIdx
+			).catch((error) => {
+				toast.error(error);
+				return null;
+			});
+
+			if (res) {
+				const reader = res.body
+					.pipeThrough(new TextDecoderStream())
+					.pipeThrough(splitStream('\n'))
+					.getReader();
+
+				while (true) {
+					try {
+						const { value, done } = await reader.read();
+						if (done) break;
+
+						let lines = value.split('\n');
+
+						for (const line of lines) {
+							if (line !== '') {
+								let data = JSON.parse(line);
+
+								console.log(data);
+								if (data.error) {
+									throw data.error;
+								}
+								if (data.detail) {
+									throw data.detail;
+								}
+								if (data.status) {
+									if (data.digest) {
+										updateProgress = 0;
+										if (data.completed) {
+											updateProgress = Math.round((data.completed / data.total) * 1000) / 10;
+										} else {
+											updateProgress = 100;
+										}
+									} else {
+										toast.success(data.status);
+									}
+								}
+							}
+						}
+					} catch (error) {
+						console.log(error);
+					}
+				}
+			}
+		}
+
+		updateModelId = null;
+		updateProgress = null;
+	};
+
+	const pullModelHandler = async () => {
+		const sanitizedModelTag = modelTag.trim().replace(/^ollama\s+(run|pull)\s+/, '');
+		console.log($MODEL_DOWNLOAD_POOL);
+		if ($MODEL_DOWNLOAD_POOL[sanitizedModelTag]) {
+			toast.error(
+				$i18n.t(`Model '{{modelTag}}' is already in queue for downloading.`, {
+					modelTag: sanitizedModelTag
+				})
+			);
+			return;
+		}
+		if (Object.keys($MODEL_DOWNLOAD_POOL).length === MAX_PARALLEL_DOWNLOADS) {
+			toast.error(
+				$i18n.t('Maximum of 3 models can be downloaded simultaneously. Please try again later.')
+			);
+			return;
+		}
+
+		const [res, controller] = await pullModel(localStorage.token, sanitizedModelTag, '0').catch(
+			(error) => {
+				toast.error(error);
+				return null;
+			}
+		);
+
+		if (res) {
+			const reader = res.body
+				.pipeThrough(new TextDecoderStream())
+				.pipeThrough(splitStream('\n'))
+				.getReader();
+
+			MODEL_DOWNLOAD_POOL.set({
+				...$MODEL_DOWNLOAD_POOL,
+				[sanitizedModelTag]: {
+					...$MODEL_DOWNLOAD_POOL[sanitizedModelTag],
+					abortController: controller,
+					reader,
+					done: false
+				}
+			});
+
+			while (true) {
+				try {
+					const { value, done } = await reader.read();
+					if (done) break;
+
+					let lines = value.split('\n');
+
+					for (const line of lines) {
+						if (line !== '') {
+							let data = JSON.parse(line);
+							console.log(data);
+							if (data.error) {
+								throw data.error;
+							}
+							if (data.detail) {
+								throw data.detail;
+							}
+
+							if (data.status) {
+								if (data.digest) {
+									let downloadProgress = 0;
+									if (data.completed) {
+										downloadProgress = Math.round((data.completed / data.total) * 1000) / 10;
+									} else {
+										downloadProgress = 100;
+									}
+
+									MODEL_DOWNLOAD_POOL.set({
+										...$MODEL_DOWNLOAD_POOL,
+										[sanitizedModelTag]: {
+											...$MODEL_DOWNLOAD_POOL[sanitizedModelTag],
+											pullProgress: downloadProgress,
+											digest: data.digest
+										}
+									});
+								} else {
+									toast.success(data.status);
+
+									MODEL_DOWNLOAD_POOL.set({
+										...$MODEL_DOWNLOAD_POOL,
+										[sanitizedModelTag]: {
+											...$MODEL_DOWNLOAD_POOL[sanitizedModelTag],
+											done: data.status === 'success'
+										}
+									});
+								}
+							}
+						}
+					}
+				} catch (error) {
+					console.log(error);
+					if (typeof error !== 'string') {
+						error = error.message;
+					}
+
+					toast.error(error);
+					// opts.callback({ success: false, error, modelName: opts.modelName });
+				}
+			}
+
+			console.log($MODEL_DOWNLOAD_POOL[sanitizedModelTag]);
+
+			if ($MODEL_DOWNLOAD_POOL[sanitizedModelTag].done) {
+				toast.success(
+					$i18n.t(`Model '{{modelName}}' has been successfully downloaded.`, {
+						modelName: sanitizedModelTag
+					})
+				);
+
+				models.set(await getModels());
+			} else {
+				toast.error($i18n.t('Download canceled'));
+			}
+
+			delete $MODEL_DOWNLOAD_POOL[sanitizedModelTag];
+
+			MODEL_DOWNLOAD_POOL.set({
+				...$MODEL_DOWNLOAD_POOL
+			});
+		}
+
+		modelTag = '';
+		modelTransferring = false;
+	};
+
+	const uploadModelHandler = async () => {
+		modelTransferring = true;
+
+		let uploaded = false;
+		let fileResponse = null;
+		let name = '';
+
+		if (modelUploadMode === 'file') {
+			const file = modelInputFile ? modelInputFile[0] : null;
+
+			if (file) {
+				uploadMessage = 'Uploading...';
+
+				fileResponse = await uploadModel(localStorage.token, file, selectedOllamaUrlIdx).catch(
+					(error) => {
+						toast.error(error);
+						return null;
+					}
+				);
+			}
+		} else {
+			uploadProgress = 0;
+			fileResponse = await downloadModel(
+				localStorage.token,
+				modelFileUrl,
+				selectedOllamaUrlIdx
+			).catch((error) => {
+				toast.error(error);
+				return null;
+			});
+		}
+
+		if (fileResponse && fileResponse.ok) {
+			const reader = fileResponse.body
+				.pipeThrough(new TextDecoderStream())
+				.pipeThrough(splitStream('\n'))
+				.getReader();
+
+			while (true) {
+				const { value, done } = await reader.read();
+				if (done) break;
+
+				try {
+					let lines = value.split('\n');
+
+					for (const line of lines) {
+						if (line !== '') {
+							let data = JSON.parse(line.replace(/^data: /, ''));
+
+							if (data.progress) {
+								if (uploadMessage) {
+									uploadMessage = '';
+								}
+								uploadProgress = data.progress;
+							}
+
+							if (data.error) {
+								throw data.error;
+							}
+
+							if (data.done) {
+								modelFileDigest = data.blob;
+								name = data.name;
+								uploaded = true;
+							}
+						}
+					}
+				} catch (error) {
+					console.log(error);
+				}
+			}
+		} else {
+			const error = await fileResponse?.json();
+			toast.error(error?.detail ?? error);
+		}
+
+		if (uploaded) {
+			const res = await createModel(
+				localStorage.token,
+				`${name}:latest`,
+				`FROM @${modelFileDigest}\n${modelFileContent}`
+			);
+
+			if (res && res.ok) {
+				const reader = res.body
+					.pipeThrough(new TextDecoderStream())
+					.pipeThrough(splitStream('\n'))
+					.getReader();
+
+				while (true) {
+					const { value, done } = await reader.read();
+					if (done) break;
+
+					try {
+						let lines = value.split('\n');
+
+						for (const line of lines) {
+							if (line !== '') {
+								console.log(line);
+								let data = JSON.parse(line);
+								console.log(data);
+
+								if (data.error) {
+									throw data.error;
+								}
+								if (data.detail) {
+									throw data.detail;
+								}
+
+								if (data.status) {
+									if (
+										!data.digest &&
+										!data.status.includes('writing') &&
+										!data.status.includes('sha256')
+									) {
+										toast.success(data.status);
+									} else {
+										if (data.digest) {
+											digest = data.digest;
+
+											if (data.completed) {
+												pullProgress = Math.round((data.completed / data.total) * 1000) / 10;
+											} else {
+												pullProgress = 100;
+											}
+										}
+									}
+								}
+							}
+						}
+					} catch (error) {
+						console.log(error);
+						toast.error(error);
+					}
+				}
+			}
+		}
+
+		modelFileUrl = '';
+
+		if (modelUploadInputElement) {
+			modelUploadInputElement.value = '';
+		}
+		modelInputFile = null;
+		modelTransferring = false;
+		uploadProgress = null;
+
+		models.set(await getModels());
+	};
+
+	const deleteModelHandler = async () => {
+		const res = await deleteModel(localStorage.token, deleteModelTag, selectedOllamaUrlIdx).catch(
+			(error) => {
+				toast.error(error);
+			}
+		);
+
+		if (res) {
+			toast.success($i18n.t(`Deleted {{deleteModelTag}}`, { deleteModelTag }));
+		}
+
+		deleteModelTag = '';
+		models.set(await getModels());
+	};
+
+	const cancelModelPullHandler = async (model: string) => {
+		const { reader, abortController } = $MODEL_DOWNLOAD_POOL[model];
+		if (abortController) {
+			abortController.abort();
+		}
+		if (reader) {
+			await reader.cancel();
+			delete $MODEL_DOWNLOAD_POOL[model];
+			MODEL_DOWNLOAD_POOL.set({
+				...$MODEL_DOWNLOAD_POOL
+			});
+			await deleteModel(localStorage.token, model);
+			toast.success(`${model} download has been canceled`);
+		}
+	};
+
+	const createModelHandler = async () => {
+		createModelLoading = true;
+		const res = await createModel(
+			localStorage.token,
+			createModelTag,
+			createModelContent,
+			selectedOllamaUrlIdx
+		).catch((error) => {
+			toast.error(error);
+			return null;
+		});
+
+		if (res && res.ok) {
+			const reader = res.body
+				.pipeThrough(new TextDecoderStream())
+				.pipeThrough(splitStream('\n'))
+				.getReader();
+
+			while (true) {
+				const { value, done } = await reader.read();
+				if (done) break;
+
+				try {
+					let lines = value.split('\n');
+
+					for (const line of lines) {
+						if (line !== '') {
+							console.log(line);
+							let data = JSON.parse(line);
+							console.log(data);
+
+							if (data.error) {
+								throw data.error;
+							}
+							if (data.detail) {
+								throw data.detail;
+							}
+
+							if (data.status) {
+								if (
+									!data.digest &&
+									!data.status.includes('writing') &&
+									!data.status.includes('sha256')
+								) {
+									toast.success(data.status);
+								} else {
+									if (data.digest) {
+										createModelDigest = data.digest;
+
+										if (data.completed) {
+											createModelPullProgress =
+												Math.round((data.completed / data.total) * 1000) / 10;
+										} else {
+											createModelPullProgress = 100;
+										}
+									}
+								}
+							}
+						}
+					}
+				} catch (error) {
+					console.log(error);
+					toast.error(error);
+				}
+			}
+		}
+
+		models.set(await getModels());
+
+		createModelLoading = false;
+
+		createModelTag = '';
+		createModelContent = '';
+		createModelDigest = '';
+		createModelPullProgress = null;
+	};
+
+	onMount(async () => {
+		const ollamaConfig = await getOllamaConfig(localStorage.token);
+
+		if (ollamaConfig.ENABLE_OLLAMA_API) {
+			ollamaEnabled = true;
+
+			await Promise.all([
+				(async () => {
+					OLLAMA_URLS = await getOllamaUrls(localStorage.token).catch((error) => {
+						toast.error(error);
+						return [];
+					});
+
+					if (OLLAMA_URLS.length > 0) {
+						selectedOllamaUrlIdx = 0;
+					}
+				})(),
+				(async () => {
+					ollamaVersion = await getOllamaVersion(localStorage.token).catch((error) => false);
+				})()
+			]);
+		} else {
+			ollamaEnabled = false;
+			toast.error($i18n.t('Ollama API is disabled'));
+		}
+	});
+</script>
+
+<ModelDeleteConfirmDialog
+	bind:show={showModelDeleteConfirm}
+	on:confirm={() => {
+		deleteModelHandler();
+	}}
+/>
+
+<div class="flex flex-col h-full justify-between text-sm">
+	<div class=" space-y-3 overflow-y-scroll scrollbar-hidden h-full">
+		{#if ollamaEnabled}
+			{#if ollamaVersion !== null}
+				<div class="space-y-2 pr-1.5">
+					<div class="text-sm font-medium">{$i18n.t('Manage Ollama Models')}</div>
+
+					{#if OLLAMA_URLS.length > 0}
+						<div class="flex gap-2">
+							<div class="flex-1 pb-1">
+								<select
+									class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+									bind:value={selectedOllamaUrlIdx}
+									placeholder={$i18n.t('Select an Ollama instance')}
+								>
+									{#each OLLAMA_URLS as url, idx}
+										<option value={idx} class="bg-gray-100 dark:bg-gray-700">{url}</option>
+									{/each}
+								</select>
+							</div>
+
+							<div>
+								<div class="flex w-full justify-end">
+									<Tooltip content="Update All Models" placement="top">
+										<button
+											class="p-2.5 flex gap-2 items-center bg-gray-100 hover:bg-gray-200 text-gray-800 dark:bg-gray-850 dark:hover:bg-gray-800 dark:text-gray-100 rounded-lg transition"
+											on:click={() => {
+												updateModelsHandler();
+											}}
+										>
+											<svg
+												xmlns="http://www.w3.org/2000/svg"
+												viewBox="0 0 16 16"
+												fill="currentColor"
+												class="w-4 h-4"
+											>
+												<path
+													d="M7 1a.75.75 0 0 1 .75.75V6h-1.5V1.75A.75.75 0 0 1 7 1ZM6.25 6v2.94L5.03 7.72a.75.75 0 0 0-1.06 1.06l2.5 2.5a.75.75 0 0 0 1.06 0l2.5-2.5a.75.75 0 1 0-1.06-1.06L7.75 8.94V6H10a2 2 0 0 1 2 2v3a2 2 0 0 1-2 2H4a2 2 0 0 1-2-2V8a2 2 0 0 1 2-2h2.25Z"
+												/>
+												<path
+													d="M4.268 14A2 2 0 0 0 6 15h6a2 2 0 0 0 2-2v-3a2 2 0 0 0-1-1.732V11a3 3 0 0 1-3 3H4.268Z"
+												/>
+											</svg>
+										</button>
+									</Tooltip>
+								</div>
+							</div>
+						</div>
+
+						{#if updateModelId}
+							Updating "{updateModelId}" {updateProgress ? `(${updateProgress}%)` : ''}
+						{/if}
+					{/if}
+
+					<div class="space-y-2">
+						<div>
+							<div class=" mb-2 text-sm font-medium">{$i18n.t('Pull a model from Ollama.com')}</div>
+							<div class="flex w-full">
+								<div class="flex-1 mr-2">
+									<input
+										class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+										placeholder={$i18n.t('Enter model tag (e.g. {{modelTag}})', {
+											modelTag: 'mistral:7b'
+										})}
+										bind:value={modelTag}
+									/>
+								</div>
+								<button
+									class="px-2.5 bg-gray-100 hover:bg-gray-200 text-gray-800 dark:bg-gray-850 dark:hover:bg-gray-800 dark:text-gray-100 rounded-lg transition"
+									on:click={() => {
+										pullModelHandler();
+									}}
+									disabled={modelTransferring}
+								>
+									{#if modelTransferring}
+										<div class="self-center">
+											<svg
+												class=" w-4 h-4"
+												viewBox="0 0 24 24"
+												fill="currentColor"
+												xmlns="http://www.w3.org/2000/svg"
+											>
+												<style>
+													.spinner_ajPY {
+														transform-origin: center;
+														animation: spinner_AtaB 0.75s infinite linear;
+													}
+
+													@keyframes spinner_AtaB {
+														100% {
+															transform: rotate(360deg);
+														}
+													}
+												</style>
+												<path
+													d="M12,1A11,11,0,1,0,23,12,11,11,0,0,0,12,1Zm0,19a8,8,0,1,1,8-8A8,8,0,0,1,12,20Z"
+													opacity=".25"
+												/>
+												<path
+													d="M10.14,1.16a11,11,0,0,0-9,8.92A1.59,1.59,0,0,0,2.46,12,1.52,1.52,0,0,0,4.11,10.7a8,8,0,0,1,6.66-6.61A1.42,1.42,0,0,0,12,2.69h0A1.57,1.57,0,0,0,10.14,1.16Z"
+													class="spinner_ajPY"
+												/>
+											</svg>
+										</div>
+									{:else}
+										<svg
+											xmlns="http://www.w3.org/2000/svg"
+											viewBox="0 0 16 16"
+											fill="currentColor"
+											class="w-4 h-4"
+										>
+											<path
+												d="M8.75 2.75a.75.75 0 0 0-1.5 0v5.69L5.03 6.22a.75.75 0 0 0-1.06 1.06l3.5 3.5a.75.75 0 0 0 1.06 0l3.5-3.5a.75.75 0 0 0-1.06-1.06L8.75 8.44V2.75Z"
+											/>
+											<path
+												d="M3.5 9.75a.75.75 0 0 0-1.5 0v1.5A2.75 2.75 0 0 0 4.75 14h6.5A2.75 2.75 0 0 0 14 11.25v-1.5a.75.75 0 0 0-1.5 0v1.5c0 .69-.56 1.25-1.25 1.25h-6.5c-.69 0-1.25-.56-1.25-1.25v-1.5Z"
+											/>
+										</svg>
+									{/if}
+								</button>
+							</div>
+
+							<div class="mt-2 mb-1 text-xs text-gray-400 dark:text-gray-500">
+								{$i18n.t('To access the available model names for downloading,')}
+								<a
+									class=" text-gray-500 dark:text-gray-300 font-medium underline"
+									href="https://ollama.com/library"
+									target="_blank">{$i18n.t('click here.')}</a
+								>
+							</div>
+
+							{#if Object.keys($MODEL_DOWNLOAD_POOL).length > 0}
+								{#each Object.keys($MODEL_DOWNLOAD_POOL) as model}
+									{#if 'pullProgress' in $MODEL_DOWNLOAD_POOL[model]}
+										<div class="flex flex-col">
+											<div class="font-medium mb-1">{model}</div>
+											<div class="">
+												<div class="flex flex-row justify-between space-x-4 pr-2">
+													<div class=" flex-1">
+														<div
+															class="dark:bg-gray-600 bg-gray-500 text-xs font-medium text-gray-100 text-center p-0.5 leading-none rounded-full"
+															style="width: {Math.max(
+																15,
+																$MODEL_DOWNLOAD_POOL[model].pullProgress ?? 0
+															)}%"
+														>
+															{$MODEL_DOWNLOAD_POOL[model].pullProgress ?? 0}%
+														</div>
+													</div>
+
+													<Tooltip content={$i18n.t('Cancel')}>
+														<button
+															class="text-gray-800 dark:text-gray-100"
+															on:click={() => {
+																cancelModelPullHandler(model);
+															}}
+														>
+															<svg
+																class="w-4 h-4 text-gray-800 dark:text-white"
+																aria-hidden="true"
+																xmlns="http://www.w3.org/2000/svg"
+																width="24"
+																height="24"
+																fill="currentColor"
+																viewBox="0 0 24 24"
+															>
+																<path
+																	stroke="currentColor"
+																	stroke-linecap="round"
+																	stroke-linejoin="round"
+																	stroke-width="2"
+																	d="M6 18 17.94 6M18 18 6.06 6"
+																/>
+															</svg>
+														</button>
+													</Tooltip>
+												</div>
+												{#if 'digest' in $MODEL_DOWNLOAD_POOL[model]}
+													<div class="mt-1 text-xs dark:text-gray-500" style="font-size: 0.5rem;">
+														{$MODEL_DOWNLOAD_POOL[model].digest}
+													</div>
+												{/if}
+											</div>
+										</div>
+									{/if}
+								{/each}
+							{/if}
+						</div>
+
+						<div>
+							<div class=" mb-2 text-sm font-medium">{$i18n.t('Delete a model')}</div>
+							<div class="flex w-full">
+								<div class="flex-1 mr-2">
+									<select
+										class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+										bind:value={deleteModelTag}
+										placeholder={$i18n.t('Select a model')}
+									>
+										{#if !deleteModelTag}
+											<option value="" disabled selected>{$i18n.t('Select a model')}</option>
+										{/if}
+										{#each $models.filter((m) => !(m?.preset ?? false) && m.owned_by === 'ollama' && (selectedOllamaUrlIdx === null ? true : (m?.ollama?.urls ?? []).includes(selectedOllamaUrlIdx))) as model}
+											<option value={model.name} class="bg-gray-100 dark:bg-gray-700"
+												>{model.name +
+													' (' +
+													(model.ollama.size / 1024 ** 3).toFixed(1) +
+													' GB)'}</option
+											>
+										{/each}
+									</select>
+								</div>
+								<button
+									class="px-2.5 bg-gray-100 hover:bg-gray-200 text-gray-800 dark:bg-gray-850 dark:hover:bg-gray-800 dark:text-gray-100 rounded-lg transition"
+									on:click={() => {
+										showModelDeleteConfirm = true;
+									}}
+								>
+									<svg
+										xmlns="http://www.w3.org/2000/svg"
+										viewBox="0 0 16 16"
+										fill="currentColor"
+										class="w-4 h-4"
+									>
+										<path
+											fill-rule="evenodd"
+											d="M5 3.25V4H2.75a.75.75 0 0 0 0 1.5h.3l.815 8.15A1.5 1.5 0 0 0 5.357 15h5.285a1.5 1.5 0 0 0 1.493-1.35l.815-8.15h.3a.75.75 0 0 0 0-1.5H11v-.75A2.25 2.25 0 0 0 8.75 1h-1.5A2.25 2.25 0 0 0 5 3.25Zm2.25-.75a.75.75 0 0 0-.75.75V4h3v-.75a.75.75 0 0 0-.75-.75h-1.5ZM6.05 6a.75.75 0 0 1 .787.713l.275 5.5a.75.75 0 0 1-1.498.075l-.275-5.5A.75.75 0 0 1 6.05 6Zm3.9 0a.75.75 0 0 1 .712.787l-.275 5.5a.75.75 0 0 1-1.498-.075l.275-5.5a.75.75 0 0 1 .786-.711Z"
+											clip-rule="evenodd"
+										/>
+									</svg>
+								</button>
+							</div>
+						</div>
+
+						<div>
+							<div class=" mb-2 text-sm font-medium">{$i18n.t('Create a model')}</div>
+							<div class="flex w-full">
+								<div class="flex-1 mr-2 flex flex-col gap-2">
+									<input
+										class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+										placeholder={$i18n.t('Enter model tag (e.g. {{modelTag}})', {
+											modelTag: 'my-modelfile'
+										})}
+										bind:value={createModelTag}
+										disabled={createModelLoading}
+									/>
+
+									<textarea
+										bind:value={createModelContent}
+										class="w-full rounded-lg py-2 px-4 text-sm bg-gray-100 dark:text-gray-100 dark:bg-gray-850 outline-none resize-none scrollbar-hidden"
+										rows="6"
+										placeholder={`TEMPLATE """{{ .System }}\nUSER: {{ .Prompt }}\nASSISTANT: """\nPARAMETER num_ctx 4096\nPARAMETER stop "</s>"\nPARAMETER stop "USER:"\nPARAMETER stop "ASSISTANT:"`}
+										disabled={createModelLoading}
+									/>
+								</div>
+
+								<div class="flex self-start">
+									<button
+										class="px-2.5 py-2.5 bg-gray-100 hover:bg-gray-200 text-gray-800 dark:bg-gray-850 dark:hover:bg-gray-800 dark:text-gray-100 rounded-lg transition disabled:cursor-not-allowed"
+										on:click={() => {
+											createModelHandler();
+										}}
+										disabled={createModelLoading}
+									>
+										<svg
+											xmlns="http://www.w3.org/2000/svg"
+											viewBox="0 0 16 16"
+											fill="currentColor"
+											class="size-4"
+										>
+											<path
+												d="M7.25 10.25a.75.75 0 0 0 1.5 0V4.56l2.22 2.22a.75.75 0 1 0 1.06-1.06l-3.5-3.5a.75.75 0 0 0-1.06 0l-3.5 3.5a.75.75 0 0 0 1.06 1.06l2.22-2.22v5.69Z"
+											/>
+											<path
+												d="M3.5 9.75a.75.75 0 0 0-1.5 0v1.5A2.75 2.75 0 0 0 4.75 14h6.5A2.75 2.75 0 0 0 14 11.25v-1.5a.75.75 0 0 0-1.5 0v1.5c0 .69-.56 1.25-1.25 1.25h-6.5c-.69 0-1.25-.56-1.25-1.25v-1.5Z"
+											/>
+										</svg>
+									</button>
+								</div>
+							</div>
+
+							{#if createModelDigest !== ''}
+								<div class="flex flex-col mt-1">
+									<div class="font-medium mb-1">{createModelTag}</div>
+									<div class="">
+										<div class="flex flex-row justify-between space-x-4 pr-2">
+											<div class=" flex-1">
+												<div
+													class="dark:bg-gray-600 bg-gray-500 text-xs font-medium text-gray-100 text-center p-0.5 leading-none rounded-full"
+													style="width: {Math.max(15, createModelPullProgress ?? 0)}%"
+												>
+													{createModelPullProgress ?? 0}%
+												</div>
+											</div>
+										</div>
+										{#if createModelDigest}
+											<div class="mt-1 text-xs dark:text-gray-500" style="font-size: 0.5rem;">
+												{createModelDigest}
+											</div>
+										{/if}
+									</div>
+								</div>
+							{/if}
+						</div>
+
+						<div class="pt-1">
+							<div class="flex justify-between items-center text-xs">
+								<div class=" text-sm font-medium">{$i18n.t('Experimental')}</div>
+								<button
+									class=" text-xs font-medium text-gray-500"
+									type="button"
+									on:click={() => {
+										showExperimentalOllama = !showExperimentalOllama;
+									}}>{showExperimentalOllama ? $i18n.t('Hide') : $i18n.t('Show')}</button
+								>
+							</div>
+						</div>
+
+						{#if showExperimentalOllama}
+							<form
+								on:submit|preventDefault={() => {
+									uploadModelHandler();
+								}}
+							>
+								<div class=" mb-2 flex w-full justify-between">
+									<div class="  text-sm font-medium">{$i18n.t('Upload a GGUF model')}</div>
+
+									<button
+										class="p-1 px-3 text-xs flex rounded transition"
+										on:click={() => {
+											if (modelUploadMode === 'file') {
+												modelUploadMode = 'url';
+											} else {
+												modelUploadMode = 'file';
+											}
+										}}
+										type="button"
+									>
+										{#if modelUploadMode === 'file'}
+											<span class="ml-2 self-center">{$i18n.t('File Mode')}</span>
+										{:else}
+											<span class="ml-2 self-center">{$i18n.t('URL Mode')}</span>
+										{/if}
+									</button>
+								</div>
+
+								<div class="flex w-full mb-1.5">
+									<div class="flex flex-col w-full">
+										{#if modelUploadMode === 'file'}
+											<div
+												class="flex-1 {modelInputFile && modelInputFile.length > 0 ? 'mr-2' : ''}"
+											>
+												<input
+													id="model-upload-input"
+													bind:this={modelUploadInputElement}
+													type="file"
+													bind:files={modelInputFile}
+													on:change={() => {
+														console.log(modelInputFile);
+													}}
+													accept=".gguf,.safetensors"
+													required
+													hidden
+												/>
+
+												<button
+													type="button"
+													class="w-full rounded-lg text-left py-2 px-4 bg-white dark:text-gray-300 dark:bg-gray-850"
+													on:click={() => {
+														modelUploadInputElement.click();
+													}}
+												>
+													{#if modelInputFile && modelInputFile.length > 0}
+														{modelInputFile[0].name}
+													{:else}
+														{$i18n.t('Click here to select')}
+													{/if}
+												</button>
+											</div>
+										{:else}
+											<div class="flex-1 {modelFileUrl !== '' ? 'mr-2' : ''}">
+												<input
+													class="w-full rounded-lg text-left py-2 px-4 bg-white dark:text-gray-300 dark:bg-gray-850 outline-none {modelFileUrl !==
+													''
+														? 'mr-2'
+														: ''}"
+													type="url"
+													required
+													bind:value={modelFileUrl}
+													placeholder={$i18n.t('Type Hugging Face Resolve (Download) URL')}
+												/>
+											</div>
+										{/if}
+									</div>
+
+									{#if (modelUploadMode === 'file' && modelInputFile && modelInputFile.length > 0) || (modelUploadMode === 'url' && modelFileUrl !== '')}
+										<button
+											class="px-2.5 bg-gray-100 hover:bg-gray-200 text-gray-800 dark:bg-gray-850 dark:hover:bg-gray-800 dark:text-gray-100 rounded-lg disabled:cursor-not-allowed transition"
+											type="submit"
+											disabled={modelTransferring}
+										>
+											{#if modelTransferring}
+												<div class="self-center">
+													<svg
+														class=" w-4 h-4"
+														viewBox="0 0 24 24"
+														fill="currentColor"
+														xmlns="http://www.w3.org/2000/svg"
+													>
+														<style>
+															.spinner_ajPY {
+																transform-origin: center;
+																animation: spinner_AtaB 0.75s infinite linear;
+															}
+
+															@keyframes spinner_AtaB {
+																100% {
+																	transform: rotate(360deg);
+																}
+															}
+														</style>
+														<path
+															d="M12,1A11,11,0,1,0,23,12,11,11,0,0,0,12,1Zm0,19a8,8,0,1,1,8-8A8,8,0,0,1,12,20Z"
+															opacity=".25"
+														/>
+														<path
+															d="M10.14,1.16a11,11,0,0,0-9,8.92A1.59,1.59,0,0,0,2.46,12,1.52,1.52,0,0,0,4.11,10.7a8,8,0,0,1,6.66-6.61A1.42,1.42,0,0,0,12,2.69h0A1.57,1.57,0,0,0,10.14,1.16Z"
+															class="spinner_ajPY"
+														/>
+													</svg>
+												</div>
+											{:else}
+												<svg
+													xmlns="http://www.w3.org/2000/svg"
+													viewBox="0 0 16 16"
+													fill="currentColor"
+													class="w-4 h-4"
+												>
+													<path
+														d="M7.25 10.25a.75.75 0 0 0 1.5 0V4.56l2.22 2.22a.75.75 0 1 0 1.06-1.06l-3.5-3.5a.75.75 0 0 0-1.06 0l-3.5 3.5a.75.75 0 0 0 1.06 1.06l2.22-2.22v5.69Z"
+													/>
+													<path
+														d="M3.5 9.75a.75.75 0 0 0-1.5 0v1.5A2.75 2.75 0 0 0 4.75 14h6.5A2.75 2.75 0 0 0 14 11.25v-1.5a.75.75 0 0 0-1.5 0v1.5c0 .69-.56 1.25-1.25 1.25h-6.5c-.69 0-1.25-.56-1.25-1.25v-1.5Z"
+													/>
+												</svg>
+											{/if}
+										</button>
+									{/if}
+								</div>
+
+								{#if (modelUploadMode === 'file' && modelInputFile && modelInputFile.length > 0) || (modelUploadMode === 'url' && modelFileUrl !== '')}
+									<div>
+										<div>
+											<div class=" my-2.5 text-sm font-medium">{$i18n.t('Modelfile Content')}</div>
+											<textarea
+												bind:value={modelFileContent}
+												class="w-full rounded-lg py-2 px-4 text-sm bg-gray-100 dark:text-gray-100 dark:bg-gray-850 outline-none resize-none"
+												rows="6"
+											/>
+										</div>
+									</div>
+								{/if}
+								<div class=" mt-1 text-xs text-gray-400 dark:text-gray-500">
+									{$i18n.t('To access the GGUF models available for downloading,')}
+									<a
+										class=" text-gray-500 dark:text-gray-300 font-medium underline"
+										href="https://huggingface.co/models?search=gguf"
+										target="_blank">{$i18n.t('click here.')}</a
+									>
+								</div>
+
+								{#if uploadMessage}
+									<div class="mt-2">
+										<div class=" mb-2 text-xs">{$i18n.t('Upload Progress')}</div>
+
+										<div class="w-full rounded-full dark:bg-gray-800">
+											<div
+												class="dark:bg-gray-600 bg-gray-500 text-xs font-medium text-gray-100 text-center p-0.5 leading-none rounded-full"
+												style="width: 100%"
+											>
+												{uploadMessage}
+											</div>
+										</div>
+										<div class="mt-1 text-xs dark:text-gray-500" style="font-size: 0.5rem;">
+											{modelFileDigest}
+										</div>
+									</div>
+								{:else if uploadProgress !== null}
+									<div class="mt-2">
+										<div class=" mb-2 text-xs">{$i18n.t('Upload Progress')}</div>
+
+										<div class="w-full rounded-full dark:bg-gray-800">
+											<div
+												class="dark:bg-gray-600 bg-gray-500 text-xs font-medium text-gray-100 text-center p-0.5 leading-none rounded-full"
+												style="width: {Math.max(15, uploadProgress ?? 0)}%"
+											>
+												{uploadProgress ?? 0}%
+											</div>
+										</div>
+										<div class="mt-1 text-xs dark:text-gray-500" style="font-size: 0.5rem;">
+											{modelFileDigest}
+										</div>
+									</div>
+								{/if}
+							</form>
+						{/if}
+					</div>
+				</div>
+			{:else if ollamaVersion === false}
+				<div>Ollama Not Detected</div>
+			{:else}
+				<div class="flex h-full justify-center">
+					<div class="my-auto">
+						<Spinner className="size-6" />
+					</div>
+				</div>
+			{/if}
+		{:else if ollamaEnabled === false}
+			<div>{$i18n.t('Ollama API is disabled')}</div>
+		{:else}
+			<div class="flex h-full justify-center">
+				<div class="my-auto">
+					<Spinner className="size-6" />
+				</div>
+			</div>
+		{/if}
+	</div>
+</div>

src/lib/components/admin/Settings/Pipelines.svelte

@@ -14,7 +14,8 @@
 		getModels,
 		getPipelinesList,
 		downloadPipeline,
-		deletePipeline
+		deletePipeline,
+		uploadPipeline
 	} from '$lib/apis';
 
 	import Spinner from '$lib/components/common/Spinner.svelte';
@@ -24,6 +25,9 @@
 	export let saveHandler: Function;
 
 	let downloading = false;
+	let uploading = false;
+
+	let pipelineFiles;
 
 	let PIPELINES_LIST = null;
 	let selectedPipelinesUrlIdx = '';
@@ -126,6 +130,41 @@
 		downloading = false;
 	};
 
+	const uploadPipelineHandler = async () => {
+		uploading = true;
+
+		if (pipelineFiles && pipelineFiles.length !== 0) {
+			const file = pipelineFiles[0];
+
+			console.log(file);
+
+			const res = await uploadPipeline(localStorage.token, file, selectedPipelinesUrlIdx).catch(
+				(error) => {
+					console.log(error);
+					toast.error('Something went wrong :/');
+					return null;
+				}
+			);
+
+			if (res) {
+				toast.success('Pipeline downloaded successfully');
+				setPipelines();
+				models.set(await getModels(localStorage.token));
+			}
+		} else {
+			toast.error('No file selected');
+		}
+
+		pipelineFiles = null;
+		const pipelineUploadInputElement = document.getElementById('pipeline-upload-input');
+
+		if (pipelineUploadInputElement) {
+			pipelineUploadInputElement.value = null;
+		}
+
+		uploading = false;
+	};
+
 	const deletePipelineHandler = async () => {
 		const res = await deletePipeline(
 			localStorage.token,
@@ -161,7 +200,7 @@
 		updateHandler();
 	}}
 >
-	<div class="  pr-1.5 overflow-y-scroll max-h-80 h-full">
+	<div class="overflow-y-scroll scrollbar-hidden h-full">
 		{#if PIPELINES_LIST !== null}
 			<div class="flex w-full justify-between mb-2">
 				<div class=" self-center text-sm font-semibold">
@@ -196,6 +235,91 @@
 					</div>
 				</div>
 
+				<div class=" my-2">
+					<div class=" mb-2 text-sm font-medium">
+						{$i18n.t('Upload Pipeline')}
+					</div>
+					<div class="flex w-full">
+						<div class="flex-1 mr-2">
+							<input
+								id="pipelines-upload-input"
+								bind:files={pipelineFiles}
+								type="file"
+								accept=".py"
+								hidden
+							/>
+
+							<button
+								class="w-full text-sm font-medium py-2 bg-transparent hover:bg-gray-100 border border-dashed dark:border-gray-800 dark:hover:bg-gray-850 text-center rounded-xl"
+								type="button"
+								on:click={() => {
+									document.getElementById('pipelines-upload-input')?.click();
+								}}
+							>
+								{#if pipelineFiles}
+									{pipelineFiles.length > 0 ? `${pipelineFiles.length}` : ''} pipeline(s) selected.
+								{:else}
+									{$i18n.t('Click here to select a py file.')}
+								{/if}
+							</button>
+						</div>
+						<button
+							class="px-2.5 bg-gray-100 hover:bg-gray-200 text-gray-800 dark:bg-gray-850 dark:hover:bg-gray-800 dark:text-gray-100 rounded-lg transition"
+							on:click={() => {
+								uploadPipelineHandler();
+							}}
+							disabled={uploading}
+							type="button"
+						>
+							{#if uploading}
+								<div class="self-center">
+									<svg
+										class=" w-4 h-4"
+										viewBox="0 0 24 24"
+										fill="currentColor"
+										xmlns="http://www.w3.org/2000/svg"
+									>
+										<style>
+											.spinner_ajPY {
+												transform-origin: center;
+												animation: spinner_AtaB 0.75s infinite linear;
+											}
+
+											@keyframes spinner_AtaB {
+												100% {
+													transform: rotate(360deg);
+												}
+											}
+										</style>
+										<path
+											d="M12,1A11,11,0,1,0,23,12,11,11,0,0,0,12,1Zm0,19a8,8,0,1,1,8-8A8,8,0,0,1,12,20Z"
+											opacity=".25"
+										/>
+										<path
+											d="M10.14,1.16a11,11,0,0,0-9,8.92A1.59,1.59,0,0,0,2.46,12,1.52,1.52,0,0,0,4.11,10.7a8,8,0,0,1,6.66-6.61A1.42,1.42,0,0,0,12,2.69h0A1.57,1.57,0,0,0,10.14,1.16Z"
+											class="spinner_ajPY"
+										/>
+									</svg>
+								</div>
+							{:else}
+								<svg
+									xmlns="http://www.w3.org/2000/svg"
+									viewBox="0 0 16 16"
+									fill="currentColor"
+									class="size-4"
+								>
+									<path
+										d="M7.25 10.25a.75.75 0 0 0 1.5 0V4.56l2.22 2.22a.75.75 0 1 0 1.06-1.06l-3.5-3.5a.75.75 0 0 0-1.06 0l-3.5 3.5a.75.75 0 0 0 1.06 1.06l2.22-2.22v5.69Z"
+									/>
+									<path
+										d="M3.5 9.75a.75.75 0 0 0-1.5 0v1.5A2.75 2.75 0 0 0 4.75 14h6.5A2.75 2.75 0 0 0 14 11.25v-1.5a.75.75 0 0 0-1.5 0v1.5c0 .69-.56 1.25-1.25 1.25h-6.5c-.69 0-1.25-.56-1.25-1.25v-1.5Z"
+									/>
+								</svg>
+							{/if}
+						</button>
+					</div>
+				</div>
+
 				<div class=" my-2">
 					<div class=" mb-2 text-sm font-medium">
 						{$i18n.t('Install from Github URL')}
@@ -384,6 +508,8 @@
 						</div>
 					</div>
 				{/if}
+			{:else}
+				<div>Pipelines Not Detected</div>
 			{/if}
 		{:else}
 			<div class="flex justify-center h-full">

src/lib/components/admin/Settings/Users.svelte

@@ -48,7 +48,7 @@
 		await config.set(await getBackendConfig());
 	}}
 >
-	<div class=" space-y-3 pr-1.5 overflow-y-scroll max-h-80">
+	<div class=" space-y-3 overflow-y-scroll max-h-full">
 		<div>
 			<div class=" mb-2 text-sm font-medium">{$i18n.t('User Permissions')}</div>
 
@@ -58,11 +58,11 @@
 				<button
 					class="p-1 px-3 text-xs flex rounded transition"
 					on:click={() => {
-						permissions.chat.deletion = !permissions.chat.deletion;
+						permissions.chat.deletion = !(permissions?.chat?.deletion ?? true);
 					}}
 					type="button"
 				>
-					{#if permissions.chat.deletion}
+					{#if permissions?.chat?.deletion ?? true}
 						<svg
 							xmlns="http://www.w3.org/2000/svg"
 							viewBox="0 0 16 16"
@@ -94,7 +94,7 @@
 			</div>
 		</div>
 
-		<hr class=" dark:border-gray-700 my-2" />
+		<hr class=" dark:border-gray-850 my-2" />
 
 		<div class="mt-2 space-y-3">
 			<div>

src/lib/components/admin/Settings/WebSearch.svelte

@@ -0,0 +1,330 @@
+<script lang="ts">
+	import { getRAGConfig, updateRAGConfig } from '$lib/apis/rag';
+	import Switch from '$lib/components/common/Switch.svelte';
+
+	import { documents, models } from '$lib/stores';
+	import { onMount, getContext } from 'svelte';
+	import { toast } from 'svelte-sonner';
+
+	const i18n = getContext('i18n');
+
+	export let saveHandler: Function;
+
+	let webConfig = null;
+	let webSearchEngines = [
+		'searxng',
+		'google_pse',
+		'brave',
+		'serpstack',
+		'serper',
+		'serply',
+		'duckduckgo',
+		'tavily'
+	];
+
+	let youtubeLanguage = 'en';
+	let youtubeTranslation = null;
+
+	const submitHandler = async () => {
+		const res = await updateRAGConfig(localStorage.token, {
+			web: webConfig,
+			youtube: {
+				language: youtubeLanguage.split(',').map((lang) => lang.trim()),
+				translation: youtubeTranslation
+			}
+		});
+	};
+
+	onMount(async () => {
+		const res = await getRAGConfig(localStorage.token);
+
+		if (res) {
+			webConfig = res.web;
+
+			youtubeLanguage = res.youtube.language.join(',');
+			youtubeTranslation = res.youtube.translation;
+		}
+	});
+</script>
+
+<form
+	class="flex flex-col h-full justify-between space-y-3 text-sm"
+	on:submit|preventDefault={async () => {
+		await submitHandler();
+		saveHandler();
+	}}
+>
+	<div class=" space-y-3 overflow-y-scroll scrollbar-hidden h-full">
+		{#if webConfig}
+			<div>
+				<div class=" mb-1 text-sm font-medium">
+					{$i18n.t('Web Search')}
+				</div>
+
+				<div>
+					<div class=" py-0.5 flex w-full justify-between">
+						<div class=" self-center text-xs font-medium">
+							{$i18n.t('Enable Web Search')}
+						</div>
+
+						<Switch bind:state={webConfig.search.enabled} />
+					</div>
+				</div>
+
+				<div class=" py-0.5 flex w-full justify-between">
+					<div class=" self-center text-xs font-medium">{$i18n.t('Web Search Engine')}</div>
+					<div class="flex items-center relative">
+						<select
+							class="dark:bg-gray-900 w-fit pr-8 rounded px-2 p-1 text-xs bg-transparent outline-none text-right"
+							bind:value={webConfig.search.engine}
+							placeholder={$i18n.t('Select a engine')}
+							required
+						>
+							<option disabled selected value="">{$i18n.t('Select a engine')}</option>
+							{#each webSearchEngines as engine}
+								<option value={engine}>{engine}</option>
+							{/each}
+						</select>
+					</div>
+				</div>
+
+				{#if webConfig.search.engine !== ''}
+					<div class="mt-1.5">
+						{#if webConfig.search.engine === 'searxng'}
+							<div>
+								<div class=" self-center text-xs font-medium mb-1">
+									{$i18n.t('Searxng Query URL')}
+								</div>
+
+								<div class="flex w-full">
+									<div class="flex-1">
+										<input
+											class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+											type="text"
+											placeholder={$i18n.t('Enter Searxng Query URL')}
+											bind:value={webConfig.search.searxng_query_url}
+											autocomplete="off"
+										/>
+									</div>
+								</div>
+							</div>
+						{:else if webConfig.search.engine === 'google_pse'}
+							<div>
+								<div class=" self-center text-xs font-medium mb-1">
+									{$i18n.t('Google PSE API Key')}
+								</div>
+
+								<div class="flex w-full">
+									<div class="flex-1">
+										<input
+											class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+											type="text"
+											placeholder={$i18n.t('Enter Google PSE API Key')}
+											bind:value={webConfig.search.google_pse_api_key}
+											autocomplete="off"
+										/>
+									</div>
+								</div>
+							</div>
+							<div class="mt-1.5">
+								<div class=" self-center text-xs font-medium mb-1">
+									{$i18n.t('Google PSE Engine Id')}
+								</div>
+
+								<div class="flex w-full">
+									<div class="flex-1">
+										<input
+											class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+											type="text"
+											placeholder={$i18n.t('Enter Google PSE Engine Id')}
+											bind:value={webConfig.search.google_pse_engine_id}
+											autocomplete="off"
+										/>
+									</div>
+								</div>
+							</div>
+						{:else if webConfig.search.engine === 'brave'}
+							<div>
+								<div class=" self-center text-xs font-medium mb-1">
+									{$i18n.t('Brave Search API Key')}
+								</div>
+
+								<div class="flex w-full">
+									<div class="flex-1">
+										<input
+											class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+											type="text"
+											placeholder={$i18n.t('Enter Brave Search API Key')}
+											bind:value={webConfig.search.brave_search_api_key}
+											autocomplete="off"
+										/>
+									</div>
+								</div>
+							</div>
+						{:else if webConfig.search.engine === 'serpstack'}
+							<div>
+								<div class=" self-center text-xs font-medium mb-1">
+									{$i18n.t('Serpstack API Key')}
+								</div>
+
+								<div class="flex w-full">
+									<div class="flex-1">
+										<input
+											class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+											type="text"
+											placeholder={$i18n.t('Enter Serpstack API Key')}
+											bind:value={webConfig.search.serpstack_api_key}
+											autocomplete="off"
+										/>
+									</div>
+								</div>
+							</div>
+						{:else if webConfig.search.engine === 'serper'}
+							<div>
+								<div class=" self-center text-xs font-medium mb-1">
+									{$i18n.t('Serper API Key')}
+								</div>
+
+								<div class="flex w-full">
+									<div class="flex-1">
+										<input
+											class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+											type="text"
+											placeholder={$i18n.t('Enter Serper API Key')}
+											bind:value={webConfig.search.serper_api_key}
+											autocomplete="off"
+										/>
+									</div>
+								</div>
+							</div>
+						{:else if webConfig.search.engine === 'serply'}
+							<div>
+								<div class=" self-center text-xs font-medium mb-1">
+									{$i18n.t('Serply API Key')}
+								</div>
+
+								<div class="flex w-full">
+									<div class="flex-1">
+										<input
+											class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+											type="text"
+											placeholder={$i18n.t('Enter Serply API Key')}
+											bind:value={webConfig.search.serply_api_key}
+											autocomplete="off"
+										/>
+									</div>
+								</div>
+							</div>
+						{:else if webConfig.search.engine === 'tavily'}
+							<div>
+								<div class=" self-center text-xs font-medium mb-1">
+									{$i18n.t('Tavily API Key')}
+								</div>
+
+								<div class="flex w-full">
+									<div class="flex-1">
+										<input
+											class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+											type="text"
+											placeholder={$i18n.t('Enter Tavily API Key')}
+											bind:value={webConfig.search.tavily_api_key}
+											autocomplete="off"
+										/>
+									</div>
+								</div>
+							</div>
+						{/if}
+					</div>
+				{/if}
+
+				{#if webConfig.search.enabled}
+					<div class="mt-2 flex gap-2 mb-1">
+						<div class="w-full">
+							<div class=" self-center text-xs font-medium mb-1">
+								{$i18n.t('Search Result Count')}
+							</div>
+
+							<input
+								class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+								placeholder={$i18n.t('Search Result Count')}
+								bind:value={webConfig.search.result_count}
+								required
+							/>
+						</div>
+
+						<div class="w-full">
+							<div class=" self-center text-xs font-medium mb-1">
+								{$i18n.t('Concurrent Requests')}
+							</div>
+
+							<input
+								class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+								placeholder={$i18n.t('Concurrent Requests')}
+								bind:value={webConfig.search.concurrent_requests}
+								required
+							/>
+						</div>
+					</div>
+				{/if}
+			</div>
+
+			<hr class=" dark:border-gray-850 my-2" />
+
+			<div>
+				<div class=" mb-1 text-sm font-medium">
+					{$i18n.t('Web Loader Settings')}
+				</div>
+
+				<div>
+					<div class=" py-0.5 flex w-full justify-between">
+						<div class=" self-center text-xs font-medium">
+							{$i18n.t('Bypass SSL verification for Websites')}
+						</div>
+
+						<button
+							class="p-1 px-3 text-xs flex rounded transition"
+							on:click={() => {
+								webConfig.ssl_verification = !webConfig.ssl_verification;
+								submitHandler();
+							}}
+							type="button"
+						>
+							{#if webConfig.ssl_verification === true}
+								<span class="ml-2 self-center">{$i18n.t('On')}</span>
+							{:else}
+								<span class="ml-2 self-center">{$i18n.t('Off')}</span>
+							{/if}
+						</button>
+					</div>
+				</div>
+
+				<div class=" mt-2 mb-1 text-sm font-medium">
+					{$i18n.t('Youtube Loader Settings')}
+				</div>
+
+				<div>
+					<div class=" py-0.5 flex w-full justify-between">
+						<div class=" w-20 text-xs font-medium self-center">{$i18n.t('Language')}</div>
+						<div class=" flex-1 self-center">
+							<input
+								class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
+								type="text"
+								placeholder={$i18n.t('Enter language codes')}
+								bind:value={youtubeLanguage}
+								autocomplete="off"
+							/>
+						</div>
+					</div>
+				</div>
+			</div>
+		{/if}
+	</div>
+	<div class="flex justify-end pt-3 text-sm font-medium">
+		<button
+			class=" px-4 py-2 bg-emerald-700 hover:bg-emerald-800 text-gray-100 transition rounded-lg"
+			type="submit"
+		>
+			{$i18n.t('Save')}
+		</button>
+	</div>
+</form>

src/lib/components/admin/SettingsModal.svelte

@@ -39,181 +39,5 @@
 				</svg>
 			</button>
 		</div>
-
-		<div class="flex flex-col md:flex-row w-full p-4 md:space-x-4">
-			<div
-				class="tabs flex flex-row overflow-x-auto space-x-1 md:space-x-0 md:space-y-1 md:flex-col flex-1 md:flex-none md:w-40 dark:text-gray-200 text-xs text-left mb-3 md:mb-0"
-			>
-				<button
-					class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
-					'general'
-						? 'bg-gray-200 dark:bg-gray-700'
-						: ' hover:bg-gray-300 dark:hover:bg-gray-800'}"
-					on:click={() => {
-						selectedTab = 'general';
-					}}
-				>
-					<div class=" self-center mr-2">
-						<svg
-							xmlns="http://www.w3.org/2000/svg"
-							viewBox="0 0 16 16"
-							fill="currentColor"
-							class="w-4 h-4"
-						>
-							<path
-								fill-rule="evenodd"
-								d="M6.955 1.45A.5.5 0 0 1 7.452 1h1.096a.5.5 0 0 1 .497.45l.17 1.699c.484.12.94.312 1.356.562l1.321-1.081a.5.5 0 0 1 .67.033l.774.775a.5.5 0 0 1 .034.67l-1.08 1.32c.25.417.44.873.561 1.357l1.699.17a.5.5 0 0 1 .45.497v1.096a.5.5 0 0 1-.45.497l-1.699.17c-.12.484-.312.94-.562 1.356l1.082 1.322a.5.5 0 0 1-.034.67l-.774.774a.5.5 0 0 1-.67.033l-1.322-1.08c-.416.25-.872.44-1.356.561l-.17 1.699a.5.5 0 0 1-.497.45H7.452a.5.5 0 0 1-.497-.45l-.17-1.699a4.973 4.973 0 0 1-1.356-.562L4.108 13.37a.5.5 0 0 1-.67-.033l-.774-.775a.5.5 0 0 1-.034-.67l1.08-1.32a4.971 4.971 0 0 1-.561-1.357l-1.699-.17A.5.5 0 0 1 1 8.548V7.452a.5.5 0 0 1 .45-.497l1.699-.17c.12-.484.312-.94.562-1.356L2.629 4.107a.5.5 0 0 1 .034-.67l.774-.774a.5.5 0 0 1 .67-.033L5.43 3.71a4.97 4.97 0 0 1 1.356-.561l.17-1.699ZM6 8c0 .538.212 1.026.558 1.385l.057.057a2 2 0 0 0 2.828-2.828l-.058-.056A2 2 0 0 0 6 8Z"
-								clip-rule="evenodd"
-							/>
-						</svg>
-					</div>
-					<div class=" self-center">{$i18n.t('General')}</div>
-				</button>
-
-				<button
-					class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
-					'users'
-						? 'bg-gray-200 dark:bg-gray-700'
-						: ' hover:bg-gray-300 dark:hover:bg-gray-800'}"
-					on:click={() => {
-						selectedTab = 'users';
-					}}
-				>
-					<div class=" self-center mr-2">
-						<svg
-							xmlns="http://www.w3.org/2000/svg"
-							viewBox="0 0 16 16"
-							fill="currentColor"
-							class="w-4 h-4"
-						>
-							<path
-								d="M8 8a2.5 2.5 0 1 0 0-5 2.5 2.5 0 0 0 0 5ZM3.156 11.763c.16-.629.44-1.21.813-1.72a2.5 2.5 0 0 0-2.725 1.377c-.136.287.102.58.418.58h1.449c.01-.077.025-.156.045-.237ZM12.847 11.763c.02.08.036.16.046.237h1.446c.316 0 .554-.293.417-.579a2.5 2.5 0 0 0-2.722-1.378c.374.51.653 1.09.813 1.72ZM14 7.5a1.5 1.5 0 1 1-3 0 1.5 1.5 0 0 1 3 0ZM3.5 9a1.5 1.5 0 1 0 0-3 1.5 1.5 0 0 0 0 3ZM5 13c-.552 0-1.013-.455-.876-.99a4.002 4.002 0 0 1 7.753 0c.136.535-.324.99-.877.99H5Z"
-							/>
-						</svg>
-					</div>
-					<div class=" self-center">{$i18n.t('Users')}</div>
-				</button>
-
-				<button
-					class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
-					'db'
-						? 'bg-gray-200 dark:bg-gray-700'
-						: ' hover:bg-gray-300 dark:hover:bg-gray-800'}"
-					on:click={() => {
-						selectedTab = 'db';
-					}}
-				>
-					<div class=" self-center mr-2">
-						<svg
-							xmlns="http://www.w3.org/2000/svg"
-							viewBox="0 0 16 16"
-							fill="currentColor"
-							class="w-4 h-4"
-						>
-							<path d="M8 7c3.314 0 6-1.343 6-3s-2.686-3-6-3-6 1.343-6 3 2.686 3 6 3Z" />
-							<path
-								d="M8 8.5c1.84 0 3.579-.37 4.914-1.037A6.33 6.33 0 0 0 14 6.78V8c0 1.657-2.686 3-6 3S2 9.657 2 8V6.78c.346.273.72.5 1.087.683C4.42 8.131 6.16 8.5 8 8.5Z"
-							/>
-							<path
-								d="M8 12.5c1.84 0 3.579-.37 4.914-1.037.366-.183.74-.41 1.086-.684V12c0 1.657-2.686 3-6 3s-6-1.343-6-3v-1.22c.346.273.72.5 1.087.683C4.42 12.131 6.16 12.5 8 12.5Z"
-							/>
-						</svg>
-					</div>
-					<div class=" self-center">{$i18n.t('Database')}</div>
-				</button>
-
-				<button
-					class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
-					'banners'
-						? 'bg-gray-200 dark:bg-gray-700'
-						: ' hover:bg-gray-300 dark:hover:bg-gray-800'}"
-					on:click={() => {
-						selectedTab = 'banners';
-					}}
-				>
-					<div class=" self-center mr-2">
-						<svg
-							xmlns="http://www.w3.org/2000/svg"
-							viewBox="0 0 24 24"
-							fill="currentColor"
-							class="size-4"
-						>
-							<path
-								d="M5.85 3.5a.75.75 0 0 0-1.117-1 9.719 9.719 0 0 0-2.348 4.876.75.75 0 0 0 1.479.248A8.219 8.219 0 0 1 5.85 3.5ZM19.267 2.5a.75.75 0 1 0-1.118 1 8.22 8.22 0 0 1 1.987 4.124.75.75 0 0 0 1.48-.248A9.72 9.72 0 0 0 19.266 2.5Z"
-							/>
-							<path
-								fill-rule="evenodd"
-								d="M12 2.25A6.75 6.75 0 0 0 5.25 9v.75a8.217 8.217 0 0 1-2.119 5.52.75.75 0 0 0 .298 1.206c1.544.57 3.16.99 4.831 1.243a3.75 3.75 0 1 0 7.48 0 24.583 24.583 0 0 0 4.83-1.244.75.75 0 0 0 .298-1.205 8.217 8.217 0 0 1-2.118-5.52V9A6.75 6.75 0 0 0 12 2.25ZM9.75 18c0-.034 0-.067.002-.1a25.05 25.05 0 0 0 4.496 0l.002.1a2.25 2.25 0 1 1-4.5 0Z"
-								clip-rule="evenodd"
-							/>
-						</svg>
-					</div>
-					<div class=" self-center">{$i18n.t('Banners')}</div>
-				</button>
-
-				<button
-					class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
-					'pipelines'
-						? 'bg-gray-200 dark:bg-gray-700'
-						: ' hover:bg-gray-300 dark:hover:bg-gray-800'}"
-					on:click={() => {
-						selectedTab = 'pipelines';
-					}}
-				>
-					<div class=" self-center mr-2">
-						<svg
-							xmlns="http://www.w3.org/2000/svg"
-							viewBox="0 0 24 24"
-							fill="currentColor"
-							class="size-4"
-						>
-							<path
-								d="M11.644 1.59a.75.75 0 0 1 .712 0l9.75 5.25a.75.75 0 0 1 0 1.32l-9.75 5.25a.75.75 0 0 1-.712 0l-9.75-5.25a.75.75 0 0 1 0-1.32l9.75-5.25Z"
-							/>
-							<path
-								d="m3.265 10.602 7.668 4.129a2.25 2.25 0 0 0 2.134 0l7.668-4.13 1.37.739a.75.75 0 0 1 0 1.32l-9.75 5.25a.75.75 0 0 1-.71 0l-9.75-5.25a.75.75 0 0 1 0-1.32l1.37-.738Z"
-							/>
-							<path
-								d="m10.933 19.231-7.668-4.13-1.37.739a.75.75 0 0 0 0 1.32l9.75 5.25c.221.12.489.12.71 0l9.75-5.25a.75.75 0 0 0 0-1.32l-1.37-.738-7.668 4.13a2.25 2.25 0 0 1-2.134-.001Z"
-							/>
-						</svg>
-					</div>
-					<div class=" self-center">{$i18n.t('Pipelines')}</div>
-				</button>
-			</div>
-			<div class="flex-1 md:min-h-[380px]">
-				{#if selectedTab === 'general'}
-					<General
-						saveHandler={() => {
-							toast.success($i18n.t('Settings saved successfully!'));
-						}}
-					/>
-				{:else if selectedTab === 'users'}
-					<Users
-						saveHandler={() => {
-							toast.success($i18n.t('Settings saved successfully!'));
-						}}
-					/>
-				{:else if selectedTab === 'db'}
-					<Database
-						saveHandler={() => {
-							toast.success($i18n.t('Settings saved successfully!'));
-						}}
-					/>
-				{:else if selectedTab === 'banners'}
-					<Banners
-						saveHandler={() => {
-							toast.success($i18n.t('Settings saved successfully!'));
-						}}
-					/>
-				{:else if selectedTab === 'pipelines'}
-					<Pipelines
-						saveHandler={() => {
-							toast.success($i18n.t('Settings saved successfully!'));
-						}}
-					/>
-				{/if}
-			</div>
-		</div>
 	</div>
 </Modal>

src/lib/components/chat/Chat.svelte

@@ -7,6 +7,10 @@
 	import { goto } from '$app/navigation';
 	import { page } from '$app/stores';
 
+	import type { Writable } from 'svelte/store';
+	import type { i18n as i18nType } from 'i18next';
+	import { OLLAMA_API_BASE_URL, OPENAI_API_BASE_URL, WEBUI_BASE_URL } from '$lib/constants';
+
 	import {
 		chatId,
 		chats,
@@ -19,11 +23,14 @@
 		WEBUI_NAME,
 		banners,
 		user,
-		socket
+		socket,
+		showCallOverlay,
+		tools
 	} from '$lib/stores';
 	import {
 		convertMessagesToHistory,
 		copyToClipboard,
+		extractSentencesForAudio,
 		promptTemplate,
 		splitStream
 	} from '$lib/utils';
@@ -39,30 +46,27 @@
 		getTagsById,
 		updateChatById
 	} from '$lib/apis/chats';
-	import {
-		generateOpenAIChatCompletion,
-		generateSearchQuery,
-		generateTitle
-	} from '$lib/apis/openai';
+	import { generateOpenAIChatCompletion } from '$lib/apis/openai';
+	import { runWebSearch } from '$lib/apis/rag';
+	import { createOpenAITextStream } from '$lib/apis/streaming';
+	import { queryMemory } from '$lib/apis/memories';
+	import { getUserSettings } from '$lib/apis/users';
+	import { chatCompleted, generateTitle, generateSearchQuery } from '$lib/apis';
 
+	import Banner from '../common/Banner.svelte';
 	import MessageInput from '$lib/components/chat/MessageInput.svelte';
 	import Messages from '$lib/components/chat/Messages.svelte';
 	import Navbar from '$lib/components/layout/Navbar.svelte';
-	import { OLLAMA_API_BASE_URL, OPENAI_API_BASE_URL, WEBUI_BASE_URL } from '$lib/constants';
-	import { createOpenAITextStream } from '$lib/apis/streaming';
-	import { queryMemory } from '$lib/apis/memories';
-	import type { Writable } from 'svelte/store';
-	import type { i18n as i18nType } from 'i18next';
-	import { runWebSearch } from '$lib/apis/rag';
-	import Banner from '../common/Banner.svelte';
-	import { getUserSettings } from '$lib/apis/users';
-	import { chatCompleted } from '$lib/apis';
+	import CallOverlay from './MessageInput/CallOverlay.svelte';
+	import { error } from '@sveltejs/kit';
 
 	const i18n: Writable<i18nType> = getContext('i18n');
 
 	export let chatIdProp = '';
 	let loaded = false;
 
+	const eventTarget = new EventTarget();
+
 	let stopResponseFlag = false;
 	let autoScroll = true;
 	let processing = '';
@@ -73,6 +77,10 @@
 	let selectedModels = [''];
 	let atSelectedModel: Model | undefined;
 
+	let selectedModelIds = [];
+	$: selectedModelIds = atSelectedModel !== undefined ? [atSelectedModel.id] : selectedModels;
+
+	let selectedToolIds = [];
 	let webSearchEnabled = false;
 
 	let chat = null;
@@ -103,7 +111,8 @@
 
 	$: if (chatIdProp) {
 		(async () => {
-			if (await loadChat()) {
+			console.log(chatIdProp);
+			if (chatIdProp && (await loadChat())) {
 				await tick();
 				loaded = true;
 
@@ -118,7 +127,11 @@
 
 	onMount(async () => {
 		if (!$chatId) {
-			await initNewChat();
+			chatId.subscribe(async (value) => {
+				if (!value) {
+					await initNewChat();
+				}
+			});
 		} else {
 			if (!($settings.saveChatHistory ?? true)) {
 				await goto('/');
@@ -292,10 +305,11 @@
 	};
 
 	//////////////////////////
-	// Ollama functions
+	// Chat functions
 	//////////////////////////
 
-	const submitPrompt = async (userPrompt, _user = null) => {
+	const submitPrompt = async (userPrompt, { _raw = false } = {}) => {
+		let _responses = [];
 		console.log('submitPrompt', $chatId);
 
 		selectedModels = selectedModels.map((modelId) =>
@@ -318,8 +332,18 @@
 				)
 			);
 		} else {
-			// Reset chat message textarea height
-			document.getElementById('chat-textarea').style.height = '';
+			// Reset chat input textarea
+			const chatTextAreaElement = document.getElementById('chat-textarea');
+
+			if (chatTextAreaElement) {
+				chatTextAreaElement.value = '';
+				chatTextAreaElement.style.height = '';
+			}
+
+			const _files = JSON.parse(JSON.stringify(files));
+			files = [];
+
+			prompt = '';
 
 			// Create user message
 			let userMessageId = uuidv4();
@@ -328,9 +352,8 @@
 				parentId: messages.length !== 0 ? messages.at(-1).id : null,
 				childrenIds: [],
 				role: 'user',
-				user: _user ?? undefined,
 				content: userPrompt,
-				files: files.length > 0 ? files : undefined,
+				files: _files.length > 0 ? _files : undefined,
 				timestamp: Math.floor(Date.now() / 1000), // Unix epoch
 				models: selectedModels.filter((m, mIdx) => selectedModels.indexOf(m) === mIdx)
 			};
@@ -346,51 +369,86 @@
 
 			// Wait until history/message have been updated
 			await tick();
+			_responses = await sendPrompt(userPrompt, userMessageId, { newChat: true });
+		}
 
-			// Create new chat if only one message in messages
-			if (messages.length == 1) {
-				if ($settings.saveChatHistory ?? true) {
-					chat = await createNewChat(localStorage.token, {
-						id: $chatId,
-						title: $i18n.t('New Chat'),
-						models: selectedModels,
-						system: $settings.system ?? undefined,
-						options: {
-							...($settings.params ?? {})
-						},
-						messages: messages,
-						history: history,
-						tags: [],
-						timestamp: Date.now()
-					});
-					await chats.set(await getChatList(localStorage.token));
-					await chatId.set(chat.id);
-				} else {
-					await chatId.set('local');
+		return _responses;
+	};
+
+	const sendPrompt = async (prompt, parentId, { modelId = null, newChat = false } = {}) => {
+		let _responses = [];
+
+		// If modelId is provided, use it, else use selected model
+		let selectedModelIds = modelId
+			? [modelId]
+			: atSelectedModel !== undefined
+			? [atSelectedModel.id]
+			: selectedModels;
+
+		// Create response messages for each selected model
+		const responseMessageIds = {};
+		for (const modelId of selectedModelIds) {
+			const model = $models.filter((m) => m.id === modelId).at(0);
+
+			if (model) {
+				let responseMessageId = uuidv4();
+				let responseMessage = {
+					parentId: parentId,
+					id: responseMessageId,
+					childrenIds: [],
+					role: 'assistant',
+					content: '',
+					model: model.id,
+					modelName: model.name ?? model.id,
+					userContext: null,
+					timestamp: Math.floor(Date.now() / 1000) // Unix epoch
+				};
+
+				// Add message to history and Set currentId to messageId
+				history.messages[responseMessageId] = responseMessage;
+				history.currentId = responseMessageId;
+
+				// Append messageId to childrenIds of parent message
+				if (parentId !== null) {
+					history.messages[parentId].childrenIds = [
+						...history.messages[parentId].childrenIds,
+						responseMessageId
+					];
 				}
-				await tick();
-			}
 
-			// Reset chat input textarea
-			prompt = '';
-			document.getElementById('chat-textarea').style.height = '';
-			files = [];
+				responseMessageIds[modelId] = responseMessageId;
+			}
+		}
+		await tick();
 
-			// Send prompt
-			await sendPrompt(userPrompt, userMessageId);
+		// Create new chat if only one message in messages
+		if (newChat && messages.length == 2) {
+			if ($settings.saveChatHistory ?? true) {
+				chat = await createNewChat(localStorage.token, {
+					id: $chatId,
+					title: $i18n.t('New Chat'),
+					models: selectedModels,
+					system: $settings.system ?? undefined,
+					options: {
+						...($settings.params ?? {})
+					},
+					messages: messages,
+					history: history,
+					tags: [],
+					timestamp: Date.now()
+				});
+				await chats.set(await getChatList(localStorage.token));
+				await chatId.set(chat.id);
+			} else {
+				await chatId.set('local');
+			}
+			await tick();
 		}
-	};
 
-	const sendPrompt = async (prompt, parentId, modelId = null) => {
 		const _chatId = JSON.parse(JSON.stringify($chatId));
 
 		await Promise.all(
-			(modelId
-				? [modelId]
-				: atSelectedModel !== undefined
-				? [atSelectedModel.id]
-				: selectedModels
-			).map(async (modelId) => {
+			selectedModelIds.map(async (modelId) => {
 				console.log('modelId', modelId);
 				const model = $models.filter((m) => m.id === modelId).at(0);
 
@@ -408,33 +466,8 @@
 						);
 					}
 
-					// Create response message
-					let responseMessageId = uuidv4();
-					let responseMessage = {
-						parentId: parentId,
-						id: responseMessageId,
-						childrenIds: [],
-						role: 'assistant',
-						content: '',
-						model: model.id,
-						modelName: model.name ?? model.id,
-						userContext: null,
-						timestamp: Math.floor(Date.now() / 1000) // Unix epoch
-					};
-
-					// Add message to history and Set currentId to messageId
-					history.messages[responseMessageId] = responseMessage;
-					history.currentId = responseMessageId;
-
-					// Append messageId to childrenIds of parent message
-					if (parentId !== null) {
-						history.messages[parentId].childrenIds = [
-							...history.messages[parentId].childrenIds,
-							responseMessageId
-						];
-					}
-
-					await tick();
+					let responseMessageId = responseMessageIds[modelId];
+					let responseMessage = history.messages[responseMessageId];
 
 					let userContext = null;
 					if ($settings?.memory ?? false) {
@@ -443,7 +476,6 @@
 								toast.error(error);
 								return null;
 							});
-
 							if (res) {
 								if (res.documents[0].length > 0) {
 									userContext = res.documents.reduce((acc, doc, index) => {
@@ -463,18 +495,17 @@
 					responseMessage.userContext = userContext;
 
 					const chatEventEmitter = await getChatEventEmitter(model.id, _chatId);
-
 					if (webSearchEnabled) {
 						await getWebSearchResults(model.id, parentId, responseMessageId);
 					}
 
+					let _response = null;
 					if (model?.owned_by === 'openai') {
-						await sendPromptOpenAI(model, prompt, responseMessageId, _chatId);
+						_response = await sendPromptOpenAI(model, prompt, responseMessageId, _chatId);
 					} else if (model) {
-						await sendPromptOllama(model, prompt, responseMessageId, _chatId);
+						_response = await sendPromptOllama(model, prompt, responseMessageId, _chatId);
 					}
-
-					console.log('chatEventEmitter', chatEventEmitter);
+					_responses.push(_response);
 
 					if (chatEventEmitter) clearInterval(chatEventEmitter);
 				} else {
@@ -484,81 +515,11 @@
 		);
 
 		await chats.set(await getChatList(localStorage.token));
-	};
-
-	const getWebSearchResults = async (model: string, parentId: string, responseId: string) => {
-		const responseMessage = history.messages[responseId];
-
-		responseMessage.status = {
-			done: false,
-			action: 'web_search',
-			description: $i18n.t('Generating search query')
-		};
-		messages = messages;
-
-		const prompt = history.messages[parentId].content;
-		let searchQuery = prompt;
-		if (prompt.length > 100) {
-			searchQuery = await generateChatSearchQuery(model, prompt);
-			if (!searchQuery) {
-				toast.warning($i18n.t('No search query generated'));
-				responseMessage.status = {
-					...responseMessage.status,
-					done: true,
-					error: true,
-					description: 'No search query generated'
-				};
-				messages = messages;
-				return;
-			}
-		}
-
-		responseMessage.status = {
-			...responseMessage.status,
-			description: $i18n.t("Searching the web for '{{searchQuery}}'", { searchQuery })
-		};
-		messages = messages;
-
-		const results = await runWebSearch(localStorage.token, searchQuery).catch((error) => {
-			console.log(error);
-			toast.error(error);
-
-			return null;
-		});
-
-		if (results) {
-			responseMessage.status = {
-				...responseMessage.status,
-				done: true,
-				description: $i18n.t('Searched {{count}} sites', { count: results.filenames.length }),
-				urls: results.filenames
-			};
-
-			if (responseMessage?.files ?? undefined === undefined) {
-				responseMessage.files = [];
-			}
-
-			responseMessage.files.push({
-				collection_name: results.collection_name,
-				name: searchQuery,
-				type: 'web_search_results',
-				urls: results.filenames
-			});
-
-			messages = messages;
-		} else {
-			responseMessage.status = {
-				...responseMessage.status,
-				done: true,
-				error: true,
-				description: 'No search results found'
-			};
-			messages = messages;
-		}
+		return _responses;
 	};
 
 	const sendPromptOllama = async (model, userPrompt, responseMessageId, _chatId) => {
-		model = model.id;
+		let _response = null;
 
 		const responseMessage = history.messages[responseMessageId];
 
@@ -617,17 +578,39 @@
 			}
 		});
 
-		const docs = messages
-			.filter((message) => message?.files ?? null)
-			.map((message) =>
-				message.files.filter((item) =>
-					['doc', 'collection', 'web_search_results'].includes(item.type)
+		let docs = [];
+
+		if (model?.info?.meta?.knowledge ?? false) {
+			docs = model.info.meta.knowledge;
+		}
+
+		docs = [
+			...docs,
+			...messages
+				.filter((message) => message?.files ?? null)
+				.map((message) =>
+					message.files.filter((item) =>
+						['doc', 'collection', 'web_search_results'].includes(item.type)
+					)
 				)
-			)
-			.flat(1);
+				.flat(1)
+		].filter(
+			(item, index, array) =>
+				array.findIndex((i) => JSON.stringify(i) === JSON.stringify(item)) === index
+		);
+
+		eventTarget.dispatchEvent(
+			new CustomEvent('chat:start', {
+				detail: {
+					id: responseMessageId
+				}
+			})
+		);
+
+		await tick();
 
 		const [res, controller] = await generateChatCompletion(localStorage.token, {
-			model: model,
+			model: model.id,
 			messages: messagesBody,
 			options: {
 				...($settings.params ?? {}),
@@ -642,6 +625,7 @@
 			},
 			format: $settings.requestFormat ?? undefined,
 			keep_alive: $settings.keepAlive ?? undefined,
+			tool_ids: selectedToolIds.length > 0 ? selectedToolIds : undefined,
 			docs: docs.length > 0 ? docs : undefined,
 			citations: docs.length > 0,
 			chat_id: $chatId
@@ -665,9 +649,10 @@
 						controller.abort('User: Stop Response');
 					} else {
 						const messages = createMessagesList(responseMessageId);
-						await chatCompletedHandler(model, messages);
+						await chatCompletedHandler(model.id, messages);
 					}
 
+					_response = responseMessage.content;
 					break;
 				}
 
@@ -693,6 +678,23 @@
 									continue;
 								} else {
 									responseMessage.content += data.message.content;
+
+									const sentences = extractSentencesForAudio(responseMessage.content);
+									sentences.pop();
+
+									// dispatch only last sentence and make sure it hasn't been dispatched before
+									if (
+										sentences.length > 0 &&
+										sentences[sentences.length - 1] !== responseMessage.lastSentence
+									) {
+										responseMessage.lastSentence = sentences[sentences.length - 1];
+										eventTarget.dispatchEvent(
+											new CustomEvent('chat', {
+												detail: { id: responseMessageId, content: sentences[sentences.length - 1] }
+											})
+										);
+									}
+
 									messages = messages;
 								}
 							} else {
@@ -719,25 +721,17 @@
 								messages = messages;
 
 								if ($settings.notificationEnabled && !document.hasFocus()) {
-									const notification = new Notification(
-										selectedModelfile
-											? `${
-													selectedModelfile.title.charAt(0).toUpperCase() +
-													selectedModelfile.title.slice(1)
-											  }`
-											: `${model}`,
-										{
-											body: responseMessage.content,
-											icon: selectedModelfile?.imageUrl ?? `${WEBUI_BASE_URL}/static/favicon.png`
-										}
-									);
+									const notification = new Notification(`${model.id}`, {
+										body: responseMessage.content,
+										icon: `${WEBUI_BASE_URL}/static/favicon.png`
+									});
 								}
 
-								if ($settings.responseAutoCopy) {
+								if ($settings?.responseAutoCopy ?? false) {
 									copyToClipboard(responseMessage.content);
 								}
 
-								if ($settings.responseAutoPlayback) {
+								if ($settings.responseAutoPlayback && !$showCallOverlay) {
 									await tick();
 									document.getElementById(`speak-button-${responseMessage.id}`)?.click();
 								}
@@ -795,6 +789,23 @@
 		stopResponseFlag = false;
 		await tick();
 
+		let lastSentence = extractSentencesForAudio(responseMessage.content)?.at(-1) ?? '';
+		if (lastSentence) {
+			eventTarget.dispatchEvent(
+				new CustomEvent('chat', {
+					detail: { id: responseMessageId, content: lastSentence }
+				})
+			);
+		}
+		eventTarget.dispatchEvent(
+			new CustomEvent('chat:finish', {
+				detail: {
+					id: responseMessageId,
+					content: responseMessage.content
+				}
+			})
+		);
+
 		if (autoScroll) {
 			scrollToBottom();
 		}
@@ -804,24 +815,46 @@
 			const _title = await generateChatTitle(userPrompt);
 			await setChatTitle(_chatId, _title);
 		}
+
+		return _response;
 	};
 
 	const sendPromptOpenAI = async (model, userPrompt, responseMessageId, _chatId) => {
+		let _response = null;
 		const responseMessage = history.messages[responseMessageId];
 
-		const docs = messages
-			.filter((message) => message?.files ?? null)
-			.map((message) =>
-				message.files.filter((item) =>
-					['doc', 'collection', 'web_search_results'].includes(item.type)
-				)
-			)
-			.flat(1);
+		let docs = [];
 
-		console.log(docs);
+		if (model?.info?.meta?.knowledge ?? false) {
+			docs = model.info.meta.knowledge;
+		}
+
+		docs = [
+			...docs,
+			...messages
+				.filter((message) => message?.files ?? null)
+				.map((message) =>
+					message.files.filter((item) =>
+						['doc', 'collection', 'web_search_results'].includes(item.type)
+					)
+				)
+				.flat(1)
+		].filter(
+			(item, index, array) =>
+				array.findIndex((i) => JSON.stringify(i) === JSON.stringify(item)) === index
+		);
 
 		scrollToBottom();
 
+		eventTarget.dispatchEvent(
+			new CustomEvent('chat:start', {
+				detail: {
+					id: responseMessageId
+				}
+			})
+		);
+		await tick();
+
 		try {
 			const [res, controller] = await generateOpenAIChatCompletion(
 				localStorage.token,
@@ -889,6 +922,7 @@
 					top_p: $settings?.params?.top_p ?? undefined,
 					frequency_penalty: $settings?.params?.frequency_penalty ?? undefined,
 					max_tokens: $settings?.params?.max_tokens ?? undefined,
+					tool_ids: selectedToolIds.length > 0 ? selectedToolIds : undefined,
 					docs: docs.length > 0 ? docs : undefined,
 					citations: docs.length > 0,
 					chat_id: $chatId
@@ -923,6 +957,8 @@
 							await chatCompletedHandler(model.id, messages);
 						}
 
+						_response = responseMessage.content;
+
 						break;
 					}
 
@@ -939,6 +975,23 @@
 						continue;
 					} else {
 						responseMessage.content += value;
+
+						const sentences = extractSentencesForAudio(responseMessage.content);
+						sentences.pop();
+
+						// dispatch only last sentence and make sure it hasn't been dispatched before
+						if (
+							sentences.length > 0 &&
+							sentences[sentences.length - 1] !== responseMessage.lastSentence
+						) {
+							responseMessage.lastSentence = sentences[sentences.length - 1];
+							eventTarget.dispatchEvent(
+								new CustomEvent('chat', {
+									detail: { id: responseMessageId, content: sentences[sentences.length - 1] }
+								})
+							);
+						}
+
 						messages = messages;
 					}
 
@@ -948,7 +1001,7 @@
 				}
 
 				if ($settings.notificationEnabled && !document.hasFocus()) {
-					const notification = new Notification(`OpenAI ${model}`, {
+					const notification = new Notification(`${model.id}`, {
 						body: responseMessage.content,
 						icon: `${WEBUI_BASE_URL}/static/favicon.png`
 					});
@@ -958,8 +1011,9 @@
 					copyToClipboard(responseMessage.content);
 				}
 
-				if ($settings.responseAutoPlayback) {
+				if ($settings.responseAutoPlayback && !$showCallOverlay) {
 					await tick();
+
 					document.getElementById(`speak-button-${responseMessage.id}`)?.click();
 				}
 
@@ -988,6 +1042,24 @@
 		stopResponseFlag = false;
 		await tick();
 
+		let lastSentence = extractSentencesForAudio(responseMessage.content)?.at(-1) ?? '';
+		if (lastSentence) {
+			eventTarget.dispatchEvent(
+				new CustomEvent('chat', {
+					detail: { id: responseMessageId, content: lastSentence }
+				})
+			);
+		}
+
+		eventTarget.dispatchEvent(
+			new CustomEvent('chat:finish', {
+				detail: {
+					id: responseMessageId,
+					content: responseMessage.content
+				}
+			})
+		);
+
 		if (autoScroll) {
 			scrollToBottom();
 		}
@@ -998,6 +1070,8 @@
 			const _title = await generateChatTitle(userPrompt);
 			await setChatTitle(_chatId, _title);
 		}
+
+		return _response;
 	};
 
 	const handleOpenAIError = async (error, res: Response | null, model, responseMessage) => {
@@ -1052,9 +1126,12 @@
 			let userPrompt = userMessage.content;
 
 			if ((userMessage?.models ?? [...selectedModels]).length == 1) {
+				// If user message has only one model selected, sendPrompt automatically selects it for regeneration
 				await sendPrompt(userPrompt, userMessage.id);
 			} else {
-				await sendPrompt(userPrompt, userMessage.id, message.model);
+				// If there are multiple models selected, use the model of the response message for regeneration
+				// e.g. many model chat
+				await sendPrompt(userPrompt, userMessage.id, { modelId: message.model });
 			}
 		}
 	};
@@ -1093,28 +1170,15 @@
 
 	const generateChatTitle = async (userPrompt) => {
 		if ($settings?.title?.auto ?? true) {
-			const model = $models.find((model) => model.id === selectedModels[0]);
-
-			const titleModelId =
-				model?.owned_by === 'openai' ?? false
-					? $settings?.title?.modelExternal ?? selectedModels[0]
-					: $settings?.title?.model ?? selectedModels[0];
-			const titleModel = $models.find((model) => model.id === titleModelId);
-
-			console.log(titleModel);
 			const title = await generateTitle(
 				localStorage.token,
-				$settings?.title?.prompt ??
-					$i18n.t(
-						"Create a concise, 3-5 word phrase as a header for the following query, strictly adhering to the 3-5 word limit and avoiding the use of the word 'title':"
-					) + ' {{prompt}}',
-				titleModelId,
+				selectedModels[0],
 				userPrompt,
-				$chatId,
-				titleModel?.owned_by === 'openai' ?? false
-					? `${OPENAI_API_BASE_URL}`
-					: `${OLLAMA_API_BASE_URL}/v1`
-			);
+				$chatId
+			).catch((error) => {
+				console.error(error);
+				return 'New Chat';
+			});
 
 			return title;
 		} else {
@@ -1122,29 +1186,6 @@
 		}
 	};
 
-	const generateChatSearchQuery = async (modelId: string, prompt: string) => {
-		const model = $models.find((model) => model.id === modelId);
-		const taskModelId =
-			model?.owned_by === 'openai' ?? false
-				? $settings?.title?.modelExternal ?? modelId
-				: $settings?.title?.model ?? modelId;
-		const taskModel = $models.find((model) => model.id === taskModelId);
-
-		const previousMessages = messages
-			.filter((message) => message.role === 'user')
-			.map((message) => message.content);
-
-		return await generateSearchQuery(
-			localStorage.token,
-			taskModelId,
-			previousMessages,
-			prompt,
-			taskModel?.owned_by === 'openai' ?? false
-				? `${OPENAI_API_BASE_URL}`
-				: `${OLLAMA_API_BASE_URL}/v1`
-		);
-	};
-
 	const setChatTitle = async (_chatId, _title) => {
 		if (_chatId === $chatId) {
 			title = _title;
@@ -1156,32 +1197,88 @@
 		}
 	};
 
-	const getTags = async () => {
-		return await getTagsById(localStorage.token, $chatId).catch(async (error) => {
-			return [];
+	const getWebSearchResults = async (model: string, parentId: string, responseId: string) => {
+		const responseMessage = history.messages[responseId];
+
+		responseMessage.statusHistory = [
+			{
+				done: false,
+				action: 'web_search',
+				description: $i18n.t('Generating search query')
+			}
+		];
+		messages = messages;
+
+		const prompt = history.messages[parentId].content;
+		let searchQuery = await generateSearchQuery(localStorage.token, model, messages, prompt).catch(
+			(error) => {
+				console.log(error);
+				return prompt;
+			}
+		);
+
+		if (!searchQuery) {
+			toast.warning($i18n.t('No search query generated'));
+			responseMessage.statusHistory.push({
+				done: true,
+				error: true,
+				action: 'web_search',
+				description: 'No search query generated'
+			});
+
+			messages = messages;
+		}
+
+		responseMessage.statusHistory.push({
+			done: false,
+			action: 'web_search',
+			description: $i18n.t(`Searching "{{searchQuery}}"`, { searchQuery })
 		});
-	};
+		messages = messages;
 
-	const addTag = async (tagName) => {
-		const res = await addTagById(localStorage.token, $chatId, tagName);
-		tags = await getTags();
+		const results = await runWebSearch(localStorage.token, searchQuery).catch((error) => {
+			console.log(error);
+			toast.error(error);
 
-		chat = await updateChatById(localStorage.token, $chatId, {
-			tags: tags
+			return null;
 		});
 
-		_tags.set(await getAllChatTags(localStorage.token));
-	};
+		if (results) {
+			responseMessage.statusHistory.push({
+				done: true,
+				action: 'web_search',
+				description: $i18n.t('Searched {{count}} sites', { count: results.filenames.length }),
+				query: searchQuery,
+				urls: results.filenames
+			});
 
-	const deleteTag = async (tagName) => {
-		const res = await deleteTagById(localStorage.token, $chatId, tagName);
-		tags = await getTags();
+			if (responseMessage?.files ?? undefined === undefined) {
+				responseMessage.files = [];
+			}
 
-		chat = await updateChatById(localStorage.token, $chatId, {
-			tags: tags
-		});
+			responseMessage.files.push({
+				collection_name: results.collection_name,
+				name: searchQuery,
+				type: 'web_search_results',
+				urls: results.filenames
+			});
+
+			messages = messages;
+		} else {
+			responseMessage.statusHistory.push({
+				done: true,
+				error: true,
+				action: 'web_search',
+				description: 'No search results found'
+			});
+			messages = messages;
+		}
+	};
 
-		_tags.set(await getAllChatTags(localStorage.token));
+	const getTags = async () => {
+		return await getTagsById(localStorage.token, $chatId).catch(async (error) => {
+			return [];
+		});
 	};
 </script>
 
@@ -1193,6 +1290,19 @@
 	</title>
 </svelte:head>
 
+<audio id="audioElement" src="" style="display: none;" />
+
+{#if $showCallOverlay}
+	<CallOverlay
+		{submitPrompt}
+		{stopResponse}
+		bind:files
+		modelId={selectedModelIds?.at(0) ?? null}
+		chatId={$chatId}
+		{eventTarget}
+	/>
+{/if}
+
 {#if !chatIdProp || (loaded && chatIdProp)}
 	<div
 		class="h-screen max-h-[100dvh] {$showSidebar
@@ -1266,8 +1376,16 @@
 				bind:files
 				bind:prompt
 				bind:autoScroll
+				bind:selectedToolIds
 				bind:webSearchEnabled
 				bind:atSelectedModel
+				availableToolIds={selectedModelIds.reduce((a, e, i, arr) => {
+					const model = $models.find((m) => m.id === e);
+					if (model?.info?.meta?.toolIds ?? false) {
+						return [...new Set([...a, ...model.info.meta.toolIds])];
+					}
+					return a;
+				}, [])}
 				{selectedModels}
 				{messages}
 				{submitPrompt}

src/lib/components/chat/MessageInput/CallOverlay.svelte

@@ -0,0 +1,843 @@
+<script lang="ts">
+	import { config, settings, showCallOverlay } from '$lib/stores';
+	import { onMount, tick, getContext } from 'svelte';
+
+	import {
+		blobToFile,
+		calculateSHA256,
+		extractSentencesForAudio,
+		findWordIndices
+	} from '$lib/utils';
+	import { generateEmoji } from '$lib/apis';
+	import { synthesizeOpenAISpeech, transcribeAudio } from '$lib/apis/audio';
+
+	import { toast } from 'svelte-sonner';
+
+	import Tooltip from '$lib/components/common/Tooltip.svelte';
+	import VideoInputMenu from './CallOverlay/VideoInputMenu.svelte';
+
+	const i18n = getContext('i18n');
+
+	export let eventTarget: EventTarget;
+
+	export let submitPrompt: Function;
+	export let stopResponse: Function;
+
+	export let files;
+
+	export let chatId;
+	export let modelId;
+
+	let loading = false;
+	let confirmed = false;
+	let interrupted = false;
+
+	let emoji = null;
+
+	let camera = false;
+	let cameraStream = null;
+
+	let chatStreaming = false;
+
+	let rmsLevel = 0;
+	let hasStartedSpeaking = false;
+	let mediaRecorder;
+	let audioChunks = [];
+
+	let videoInputDevices = [];
+	let selectedVideoInputDeviceId = null;
+
+	const getVideoInputDevices = async () => {
+		const devices = await navigator.mediaDevices.enumerateDevices();
+		videoInputDevices = devices.filter((device) => device.kind === 'videoinput');
+
+		if (!!navigator.mediaDevices.getDisplayMedia) {
+			videoInputDevices = [
+				...videoInputDevices,
+				{
+					deviceId: 'screen',
+					label: 'Screen Share'
+				}
+			];
+		}
+
+		console.log(videoInputDevices);
+		if (selectedVideoInputDeviceId === null && videoInputDevices.length > 0) {
+			selectedVideoInputDeviceId = videoInputDevices[0].deviceId;
+		}
+	};
+
+	const startCamera = async () => {
+		await getVideoInputDevices();
+
+		if (cameraStream === null) {
+			camera = true;
+			await tick();
+			try {
+				await startVideoStream();
+			} catch (err) {
+				console.error('Error accessing webcam: ', err);
+			}
+		}
+	};
+
+	const startVideoStream = async () => {
+		const video = document.getElementById('camera-feed');
+		if (video) {
+			if (selectedVideoInputDeviceId === 'screen') {
+				cameraStream = await navigator.mediaDevices.getDisplayMedia({
+					video: {
+						cursor: 'always'
+					},
+					audio: false
+				});
+			} else {
+				cameraStream = await navigator.mediaDevices.getUserMedia({
+					video: {
+						deviceId: selectedVideoInputDeviceId ? { exact: selectedVideoInputDeviceId } : undefined
+					}
+				});
+			}
+
+			if (cameraStream) {
+				await getVideoInputDevices();
+				video.srcObject = cameraStream;
+				await video.play();
+			}
+		}
+	};
+
+	const stopVideoStream = async () => {
+		if (cameraStream) {
+			const tracks = cameraStream.getTracks();
+			tracks.forEach((track) => track.stop());
+		}
+
+		cameraStream = null;
+	};
+
+	const takeScreenshot = () => {
+		const video = document.getElementById('camera-feed');
+		const canvas = document.getElementById('camera-canvas');
+
+		if (!canvas) {
+			return;
+		}
+
+		const context = canvas.getContext('2d');
+
+		// Make the canvas match the video dimensions
+		canvas.width = video.videoWidth;
+		canvas.height = video.videoHeight;
+
+		// Draw the image from the video onto the canvas
+		context.drawImage(video, 0, 0, video.videoWidth, video.videoHeight);
+
+		// Convert the canvas to a data base64 URL and console log it
+		const dataURL = canvas.toDataURL('image/png');
+		console.log(dataURL);
+
+		return dataURL;
+	};
+
+	const stopCamera = async () => {
+		await stopVideoStream();
+		camera = false;
+	};
+
+	const MIN_DECIBELS = -55;
+	const VISUALIZER_BUFFER_LENGTH = 300;
+
+	const transcribeHandler = async (audioBlob) => {
+		// Create a blob from the audio chunks
+
+		await tick();
+		const file = blobToFile(audioBlob, 'recording.wav');
+
+		const res = await transcribeAudio(localStorage.token, file).catch((error) => {
+			toast.error(error);
+			return null;
+		});
+
+		if (res) {
+			console.log(res.text);
+
+			if (res.text !== '') {
+				const _responses = await submitPrompt(res.text, { _raw: true });
+				console.log(_responses);
+			}
+		}
+	};
+
+	const stopRecordingCallback = async (_continue = true) => {
+		if ($showCallOverlay) {
+			console.log('%c%s', 'color: red; font-size: 20px;', '🚨 stopRecordingCallback 🚨');
+
+			// deep copy the audioChunks array
+			const _audioChunks = audioChunks.slice(0);
+
+			audioChunks = [];
+			mediaRecorder = false;
+
+			if (_continue) {
+				startRecording();
+			}
+
+			if (confirmed) {
+				loading = true;
+				emoji = null;
+
+				if (cameraStream) {
+					const imageUrl = takeScreenshot();
+
+					files = [
+						{
+							type: 'image',
+							url: imageUrl
+						}
+					];
+				}
+
+				const audioBlob = new Blob(_audioChunks, { type: 'audio/wav' });
+				await transcribeHandler(audioBlob);
+
+				confirmed = false;
+				loading = false;
+			}
+		} else {
+			audioChunks = [];
+			mediaRecorder = false;
+		}
+	};
+
+	const startRecording = async () => {
+		const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
+		mediaRecorder = new MediaRecorder(stream);
+
+		mediaRecorder.onstart = () => {
+			console.log('Recording started');
+			audioChunks = [];
+			analyseAudio(stream);
+		};
+
+		mediaRecorder.ondataavailable = (event) => {
+			if (hasStartedSpeaking) {
+				audioChunks.push(event.data);
+			}
+		};
+
+		mediaRecorder.onstop = (e) => {
+			console.log('Recording stopped', e);
+			stopRecordingCallback();
+		};
+
+		mediaRecorder.start();
+	};
+
+	// Function to calculate the RMS level from time domain data
+	const calculateRMS = (data: Uint8Array) => {
+		let sumSquares = 0;
+		for (let i = 0; i < data.length; i++) {
+			const normalizedValue = (data[i] - 128) / 128; // Normalize the data
+			sumSquares += normalizedValue * normalizedValue;
+		}
+		return Math.sqrt(sumSquares / data.length);
+	};
+
+	const analyseAudio = (stream) => {
+		const audioContext = new AudioContext();
+		const audioStreamSource = audioContext.createMediaStreamSource(stream);
+
+		const analyser = audioContext.createAnalyser();
+		analyser.minDecibels = MIN_DECIBELS;
+		audioStreamSource.connect(analyser);
+
+		const bufferLength = analyser.frequencyBinCount;
+
+		const domainData = new Uint8Array(bufferLength);
+		const timeDomainData = new Uint8Array(analyser.fftSize);
+
+		let lastSoundTime = Date.now();
+		hasStartedSpeaking = false;
+
+		console.log('🔊 Sound detection started', lastSoundTime, hasStartedSpeaking);
+
+		const detectSound = () => {
+			const processFrame = () => {
+				if (!mediaRecorder || !$showCallOverlay) {
+					return;
+				}
+
+				analyser.getByteTimeDomainData(timeDomainData);
+				analyser.getByteFrequencyData(domainData);
+
+				// Calculate RMS level from time domain data
+				rmsLevel = calculateRMS(timeDomainData);
+
+				// Check if initial speech/noise has started
+				const hasSound = domainData.some((value) => value > 0);
+				if (hasSound) {
+					// BIG RED TEXT
+					console.log('%c%s', 'color: red; font-size: 20px;', '🔊 Sound detected');
+
+					if (!hasStartedSpeaking) {
+						hasStartedSpeaking = true;
+						stopAllAudio();
+					}
+
+					lastSoundTime = Date.now();
+				}
+
+				// Start silence detection only after initial speech/noise has been detected
+				if (hasStartedSpeaking) {
+					if (Date.now() - lastSoundTime > 2000) {
+						confirmed = true;
+
+						if (mediaRecorder) {
+							console.log('%c%s', 'color: red; font-size: 20px;', '🔇 Silence detected');
+							mediaRecorder.stop();
+							return;
+						}
+					}
+				}
+
+				window.requestAnimationFrame(processFrame);
+			};
+
+			window.requestAnimationFrame(processFrame);
+		};
+
+		detectSound();
+	};
+
+	let finishedMessages = {};
+	let currentMessageId = null;
+	let currentUtterance = null;
+
+	const speakSpeechSynthesisHandler = (content) => {
+		if ($showCallOverlay) {
+			return new Promise((resolve) => {
+				let voices = [];
+				const getVoicesLoop = setInterval(async () => {
+					voices = await speechSynthesis.getVoices();
+					if (voices.length > 0) {
+						clearInterval(getVoicesLoop);
+
+						const voice =
+							voices
+								?.filter(
+									(v) => v.voiceURI === ($settings?.audio?.tts?.voice ?? $config?.audio?.tts?.voice)
+								)
+								?.at(0) ?? undefined;
+
+						currentUtterance = new SpeechSynthesisUtterance(content);
+
+						if (voice) {
+							currentUtterance.voice = voice;
+						}
+
+						speechSynthesis.speak(currentUtterance);
+						currentUtterance.onend = async (e) => {
+							await new Promise((r) => setTimeout(r, 200));
+							resolve(e);
+						};
+					}
+				}, 100);
+			});
+		} else {
+			return Promise.resolve();
+		}
+	};
+
+	const playAudio = (audio) => {
+		if ($showCallOverlay) {
+			return new Promise((resolve) => {
+				const audioElement = document.getElementById('audioElement');
+
+				if (audioElement) {
+					audioElement.src = audio.src;
+					audioElement.muted = true;
+
+					audioElement
+						.play()
+						.then(() => {
+							audioElement.muted = false;
+						})
+						.catch((error) => {
+							console.error(error);
+						});
+
+					audioElement.onended = async (e) => {
+						await new Promise((r) => setTimeout(r, 100));
+						resolve(e);
+					};
+				}
+			});
+		} else {
+			return Promise.resolve();
+		}
+	};
+
+	const stopAllAudio = async () => {
+		interrupted = true;
+
+		if (chatStreaming) {
+			stopResponse();
+		}
+
+		if (currentUtterance) {
+			speechSynthesis.cancel();
+			currentUtterance = null;
+		}
+
+		const audioElement = document.getElementById('audioElement');
+		if (audioElement) {
+			audioElement.muted = true;
+			audioElement.pause();
+			audioElement.currentTime = 0;
+		}
+	};
+
+	let audioAbortController = new AbortController();
+
+	// Audio cache map where key is the content and value is the Audio object.
+	const audioCache = new Map();
+	const emojiCache = new Map();
+
+	const fetchAudio = async (content) => {
+		if (!audioCache.has(content)) {
+			try {
+				// Set the emoji for the content if needed
+				if ($settings?.showEmojiInCall ?? false) {
+					const emoji = await generateEmoji(localStorage.token, modelId, content, chatId);
+					if (emoji) {
+						emojiCache.set(content, emoji);
+					}
+				}
+
+				if ($config.audio.tts.engine !== '') {
+					const res = await synthesizeOpenAISpeech(
+						localStorage.token,
+						$settings?.audio?.tts?.voice ?? $config?.audio?.tts?.voice,
+						content
+					).catch((error) => {
+						console.error(error);
+						return null;
+					});
+
+					if (res) {
+						const blob = await res.blob();
+						const blobUrl = URL.createObjectURL(blob);
+						audioCache.set(content, new Audio(blobUrl));
+					}
+				} else {
+					audioCache.set(content, true);
+				}
+			} catch (error) {
+				console.error('Error synthesizing speech:', error);
+			}
+		}
+
+		return audioCache.get(content);
+	};
+
+	let messages = {};
+
+	const monitorAndPlayAudio = async (id, signal) => {
+		while (!signal.aborted) {
+			if (messages[id] && messages[id].length > 0) {
+				// Retrieve the next content string from the queue
+				const content = messages[id].shift(); // Dequeues the content for playing
+
+				if (audioCache.has(content)) {
+					// If content is available in the cache, play it
+
+					// Set the emoji for the content if available
+					if (($settings?.showEmojiInCall ?? false) && emojiCache.has(content)) {
+						emoji = emojiCache.get(content);
+					} else {
+						emoji = null;
+					}
+
+					if ($config.audio.tts.engine !== '') {
+						try {
+							console.log(
+								'%c%s',
+								'color: red; font-size: 20px;',
+								`Playing audio for content: ${content}`
+							);
+
+							const audio = audioCache.get(content);
+							await playAudio(audio); // Here ensure that playAudio is indeed correct method to execute
+							console.log(`Played audio for content: ${content}`);
+							await new Promise((resolve) => setTimeout(resolve, 200)); // Wait before retrying to reduce tight loop
+						} catch (error) {
+							console.error('Error playing audio:', error);
+						}
+					} else {
+						await speakSpeechSynthesisHandler(content);
+					}
+				} else {
+					// If not available in the cache, push it back to the queue and delay
+					messages[id].unshift(content); // Re-queue the content at the start
+					console.log(`Audio for "${content}" not yet available in the cache, re-queued...`);
+					await new Promise((resolve) => setTimeout(resolve, 200)); // Wait before retrying to reduce tight loop
+				}
+			} else if (finishedMessages[id] && messages[id] && messages[id].length === 0) {
+				// If the message is finished and there are no more messages to process, break the loop
+				break;
+			} else {
+				// No messages to process, sleep for a bit
+				await new Promise((resolve) => setTimeout(resolve, 200));
+			}
+		}
+		console.log(`Audio monitoring and playing stopped for message ID ${id}`);
+	};
+
+	onMount(async () => {
+		startRecording();
+
+		const chatStartHandler = async (e) => {
+			const { id } = e.detail;
+
+			chatStreaming = true;
+
+			if (currentMessageId !== id) {
+				console.log(`Received chat start event for message ID ${id}`);
+
+				currentMessageId = id;
+				if (audioAbortController) {
+					audioAbortController.abort();
+				}
+				audioAbortController = new AbortController();
+
+				// Start monitoring and playing audio for the message ID
+				monitorAndPlayAudio(id, audioAbortController.signal);
+			}
+		};
+
+		const chatEventHandler = async (e) => {
+			const { id, content } = e.detail;
+			// "id" here is message id
+			// if "id" is not the same as "currentMessageId" then do not process
+			// "content" here is a sentence from the assistant,
+			// there will be many sentences for the same "id"
+
+			if (currentMessageId === id) {
+				console.log(`Received chat event for message ID ${id}: ${content}`);
+
+				try {
+					if (messages[id] === undefined) {
+						messages[id] = [content];
+					} else {
+						messages[id].push(content);
+					}
+
+					console.log(content);
+
+					fetchAudio(content);
+				} catch (error) {
+					console.error('Failed to fetch or play audio:', error);
+				}
+			}
+		};
+
+		const chatFinishHandler = async (e) => {
+			const { id, content } = e.detail;
+			// "content" here is the entire message from the assistant
+
+			chatStreaming = false;
+			finishedMessages[id] = true;
+		};
+
+		eventTarget.addEventListener('chat:start', chatStartHandler);
+		eventTarget.addEventListener('chat', chatEventHandler);
+		eventTarget.addEventListener('chat:finish', chatFinishHandler);
+
+		return async () => {
+			eventTarget.removeEventListener('chat:start', chatStartHandler);
+			eventTarget.removeEventListener('chat', chatEventHandler);
+			eventTarget.removeEventListener('chat:finish', chatFinishHandler);
+
+			audioAbortController.abort();
+			await tick();
+
+			await stopAllAudio();
+
+			await stopRecordingCallback(false);
+			await stopCamera();
+		};
+	});
+</script>
+
+{#if $showCallOverlay}
+	<div class=" absolute w-full h-screen max-h-[100dvh] flex z-[999] overflow-hidden">
+		<div
+			class="absolute w-full h-screen max-h-[100dvh] bg-white text-gray-700 dark:bg-black dark:text-gray-300 flex justify-center"
+		>
+			<div class="max-w-lg w-full h-screen max-h-[100dvh] flex flex-col justify-between p-3 md:p-6">
+				{#if camera}
+					<div class="flex justify-center items-center w-full h-20 min-h-20">
+						{#if emoji}
+							<div
+								class="  transition-all rounded-full"
+								style="font-size:{rmsLevel * 100 > 4
+									? '4.5'
+									: rmsLevel * 100 > 2
+									? '4.25'
+									: rmsLevel * 100 > 1
+									? '3.75'
+									: '3.5'}rem;width: 100%; text-align:center;"
+							>
+								{emoji}
+							</div>
+						{:else if loading}
+							<svg
+								class="size-12 text-gray-900 dark:text-gray-400"
+								viewBox="0 0 24 24"
+								fill="currentColor"
+								xmlns="http://www.w3.org/2000/svg"
+								><style>
+									.spinner_qM83 {
+										animation: spinner_8HQG 1.05s infinite;
+									}
+									.spinner_oXPr {
+										animation-delay: 0.1s;
+									}
+									.spinner_ZTLf {
+										animation-delay: 0.2s;
+									}
+									@keyframes spinner_8HQG {
+										0%,
+										57.14% {
+											animation-timing-function: cubic-bezier(0.33, 0.66, 0.66, 1);
+											transform: translate(0);
+										}
+										28.57% {
+											animation-timing-function: cubic-bezier(0.33, 0, 0.66, 0.33);
+											transform: translateY(-6px);
+										}
+										100% {
+											transform: translate(0);
+										}
+									}
+								</style><circle class="spinner_qM83" cx="4" cy="12" r="3" /><circle
+									class="spinner_qM83 spinner_oXPr"
+									cx="12"
+									cy="12"
+									r="3"
+								/><circle class="spinner_qM83 spinner_ZTLf" cx="20" cy="12" r="3" /></svg
+							>
+						{:else}
+							<div
+								class=" {rmsLevel * 100 > 4
+									? ' size-[4.5rem]'
+									: rmsLevel * 100 > 2
+									? ' size-16'
+									: rmsLevel * 100 > 1
+									? 'size-14'
+									: 'size-12'}  transition-all bg-black dark:bg-white rounded-full"
+							/>
+						{/if}
+						<!-- navbar -->
+					</div>
+				{/if}
+
+				<div class="flex justify-center items-center flex-1 h-full w-full max-h-full">
+					{#if !camera}
+						{#if emoji}
+							<div
+								class="  transition-all rounded-full"
+								style="font-size:{rmsLevel * 100 > 4
+									? '13'
+									: rmsLevel * 100 > 2
+									? '12'
+									: rmsLevel * 100 > 1
+									? '11.5'
+									: '11'}rem;width:100%;text-align:center;"
+							>
+								{emoji}
+							</div>
+						{:else if loading}
+							<svg
+								class="size-44 text-gray-900 dark:text-gray-400"
+								viewBox="0 0 24 24"
+								fill="currentColor"
+								xmlns="http://www.w3.org/2000/svg"
+								><style>
+									.spinner_qM83 {
+										animation: spinner_8HQG 1.05s infinite;
+									}
+									.spinner_oXPr {
+										animation-delay: 0.1s;
+									}
+									.spinner_ZTLf {
+										animation-delay: 0.2s;
+									}
+									@keyframes spinner_8HQG {
+										0%,
+										57.14% {
+											animation-timing-function: cubic-bezier(0.33, 0.66, 0.66, 1);
+											transform: translate(0);
+										}
+										28.57% {
+											animation-timing-function: cubic-bezier(0.33, 0, 0.66, 0.33);
+											transform: translateY(-6px);
+										}
+										100% {
+											transform: translate(0);
+										}
+									}
+								</style><circle class="spinner_qM83" cx="4" cy="12" r="3" /><circle
+									class="spinner_qM83 spinner_oXPr"
+									cx="12"
+									cy="12"
+									r="3"
+								/><circle class="spinner_qM83 spinner_ZTLf" cx="20" cy="12" r="3" /></svg
+							>
+						{:else}
+							<div
+								class=" {rmsLevel * 100 > 4
+									? ' size-52'
+									: rmsLevel * 100 > 2
+									? 'size-48'
+									: rmsLevel * 100 > 1
+									? 'size-[11.5rem]'
+									: 'size-44'}  transition-all bg-black dark:bg-white rounded-full"
+							/>
+						{/if}
+					{:else}
+						<div
+							class="relative flex video-container w-full max-h-full pt-2 pb-4 md:py-6 px-2 h-full"
+						>
+							<video
+								id="camera-feed"
+								autoplay
+								class="rounded-2xl h-full min-w-full object-cover object-center"
+								playsinline
+							/>
+
+							<canvas id="camera-canvas" style="display:none;" />
+
+							<div class=" absolute top-4 md:top-8 left-4">
+								<button
+									type="button"
+									class="p-1.5 text-white cursor-pointer backdrop-blur-xl bg-black/10 rounded-full"
+									on:click={() => {
+										stopCamera();
+									}}
+								>
+									<svg
+										xmlns="http://www.w3.org/2000/svg"
+										viewBox="0 0 16 16"
+										fill="currentColor"
+										class="size-6"
+									>
+										<path
+											d="M5.28 4.22a.75.75 0 0 0-1.06 1.06L6.94 8l-2.72 2.72a.75.75 0 1 0 1.06 1.06L8 9.06l2.72 2.72a.75.75 0 1 0 1.06-1.06L9.06 8l2.72-2.72a.75.75 0 0 0-1.06-1.06L8 6.94 5.28 4.22Z"
+										/>
+									</svg>
+								</button>
+							</div>
+						</div>
+					{/if}
+				</div>
+
+				<div class="flex justify-between items-center pb-2 w-full">
+					<div>
+						{#if camera}
+							<VideoInputMenu
+								devices={videoInputDevices}
+								on:change={async (e) => {
+									console.log(e.detail);
+									selectedVideoInputDeviceId = e.detail;
+									await stopVideoStream();
+									await startVideoStream();
+								}}
+							>
+								<button class=" p-3 rounded-full bg-gray-50 dark:bg-gray-900" type="button">
+									<svg
+										xmlns="http://www.w3.org/2000/svg"
+										viewBox="0 0 20 20"
+										fill="currentColor"
+										class="size-5"
+									>
+										<path
+											fill-rule="evenodd"
+											d="M15.312 11.424a5.5 5.5 0 0 1-9.201 2.466l-.312-.311h2.433a.75.75 0 0 0 0-1.5H3.989a.75.75 0 0 0-.75.75v4.242a.75.75 0 0 0 1.5 0v-2.43l.31.31a7 7 0 0 0 11.712-3.138.75.75 0 0 0-1.449-.39Zm1.23-3.723a.75.75 0 0 0 .219-.53V2.929a.75.75 0 0 0-1.5 0V5.36l-.31-.31A7 7 0 0 0 3.239 8.188a.75.75 0 1 0 1.448.389A5.5 5.5 0 0 1 13.89 6.11l.311.31h-2.432a.75.75 0 0 0 0 1.5h4.243a.75.75 0 0 0 .53-.219Z"
+											clip-rule="evenodd"
+										/>
+									</svg>
+								</button>
+							</VideoInputMenu>
+						{:else}
+							<Tooltip content={$i18n.t('Camera')}>
+								<button
+									class=" p-3 rounded-full bg-gray-50 dark:bg-gray-900"
+									type="button"
+									on:click={async () => {
+										await navigator.mediaDevices.getUserMedia({ video: true });
+										startCamera();
+									}}
+								>
+									<svg
+										xmlns="http://www.w3.org/2000/svg"
+										fill="none"
+										viewBox="0 0 24 24"
+										stroke-width="1.5"
+										stroke="currentColor"
+										class="size-5"
+									>
+										<path
+											stroke-linecap="round"
+											stroke-linejoin="round"
+											d="M6.827 6.175A2.31 2.31 0 0 1 5.186 7.23c-.38.054-.757.112-1.134.175C2.999 7.58 2.25 8.507 2.25 9.574V18a2.25 2.25 0 0 0 2.25 2.25h15A2.25 2.25 0 0 0 21.75 18V9.574c0-1.067-.75-1.994-1.802-2.169a47.865 47.865 0 0 0-1.134-.175 2.31 2.31 0 0 1-1.64-1.055l-.822-1.316a2.192 2.192 0 0 0-1.736-1.039 48.774 48.774 0 0 0-5.232 0 2.192 2.192 0 0 0-1.736 1.039l-.821 1.316Z"
+										/>
+										<path
+											stroke-linecap="round"
+											stroke-linejoin="round"
+											d="M16.5 12.75a4.5 4.5 0 1 1-9 0 4.5 4.5 0 0 1 9 0ZM18.75 10.5h.008v.008h-.008V10.5Z"
+										/>
+									</svg>
+								</button>
+							</Tooltip>
+						{/if}
+					</div>
+
+					<div>
+						<button type="button">
+							<div class=" line-clamp-1 text-sm font-medium">
+								{#if loading}
+									{$i18n.t('Thinking...')}
+								{:else}
+									{$i18n.t('Listening...')}
+								{/if}
+							</div>
+						</button>
+					</div>
+
+					<div>
+						<button
+							class=" p-3 rounded-full bg-gray-50 dark:bg-gray-900"
+							on:click={async () => {
+								showCallOverlay.set(false);
+							}}
+							type="button"
+						>
+							<svg
+								xmlns="http://www.w3.org/2000/svg"
+								viewBox="0 0 20 20"
+								fill="currentColor"
+								class="size-5"
+							>
+								<path
+									d="M6.28 5.22a.75.75 0 0 0-1.06 1.06L8.94 10l-3.72 3.72a.75.75 0 1 0 1.06 1.06L10 11.06l3.72 3.72a.75.75 0 1 0 1.06-1.06L11.06 10l3.72-3.72a.75.75 0 0 0-1.06-1.06L10 8.94 6.28 5.22Z"
+								/>
+							</svg>
+						</button>
+					</div>
+				</div>
+			</div>
+		</div>
+	</div>
+{/if}

src/lib/components/chat/MessageInput/CallOverlay/VideoInputMenu.svelte

@@ -0,0 +1,51 @@
+<script lang="ts">
+	import { DropdownMenu } from 'bits-ui';
+	import { flyAndScale } from '$lib/utils/transitions';
+	import { getContext, createEventDispatcher } from 'svelte';
+
+	const i18n = getContext('i18n');
+	const dispatch = createEventDispatcher();
+
+	import Dropdown from '$lib/components/common/Dropdown.svelte';
+
+	export let onClose: Function = () => {};
+	export let devices: any;
+
+	let show = false;
+</script>
+
+<Dropdown
+	bind:show
+	on:change={(e) => {
+		if (e.detail === false) {
+			onClose();
+		}
+	}}
+>
+	<slot />
+
+	<div slot="content">
+		<DropdownMenu.Content
+			class="w-full max-w-[180px] rounded-lg px-1 py-1.5 border border-gray-300/30 dark:border-gray-700/50 z-[9999] bg-white dark:bg-gray-900 dark:text-white shadow-sm"
+			sideOffset={6}
+			side="top"
+			align="start"
+			transition={flyAndScale}
+		>
+			{#each devices as device}
+				<DropdownMenu.Item
+					class="flex gap-2 items-center px-3 py-2 text-sm  cursor-pointer hover:bg-gray-50 dark:hover:bg-gray-800 rounded-md"
+					on:click={() => {
+						dispatch('change', device.deviceId);
+					}}
+				>
+					<div class="flex items-center">
+						<div class=" line-clamp-1">
+							{device?.label ?? 'Camera'}
+						</div>
+					</div>
+				</DropdownMenu.Item>
+			{/each}
+		</DropdownMenu.Content>
+	</div>
+</Dropdown>

src/lib/components/chat/MessageInput/Documents.svelte

@@ -101,18 +101,20 @@
 </script>
 
 {#if filteredItems.length > 0 || prompt.split(' ')?.at(0)?.substring(1).startsWith('http')}
-	<div class="md:px-2 mb-3 text-left w-full absolute bottom-0 left-0 right-0">
+	<div class="pl-1 pr-12 mb-3 text-left w-full absolute bottom-0 left-0 right-0">
 		<div class="flex w-full px-2">
 			<div class=" bg-gray-100 dark:bg-gray-700 w-10 rounded-l-xl text-center">
 				<div class=" text-lg font-semibold mt-2">#</div>
 			</div>
 
-			<div class="max-h-60 flex flex-col w-full rounded-r-xl bg-white">
+			<div
+				class="max-h-60 flex flex-col w-full rounded-r-xl bg-white dark:bg-gray-850 dark:text-gray-100"
+			>
 				<div class="m-1 overflow-y-auto p-1 rounded-r-xl space-y-0.5">
 					{#each filteredItems as doc, docIdx}
 						<button
 							class=" px-3 py-1.5 rounded-xl w-full text-left {docIdx === selectedIdx
-								? ' bg-gray-100 selected-command-option-button'
+								? ' bg-gray-100 dark:bg-gray-600 dark:text-gray-100 selected-command-option-button'
 								: ''}"
 							type="button"
 							on:click={() => {
@@ -126,17 +128,19 @@
 							on:focus={() => {}}
 						>
 							{#if doc.type === 'collection'}
-								<div class=" font-medium text-black line-clamp-1">
+								<div class=" font-medium text-black dark:text-gray-100 line-clamp-1">
 									{doc?.title ?? `#${doc.name}`}
 								</div>
 
-								<div class=" text-xs text-gray-600 line-clamp-1">{$i18n.t('Collection')}</div>
+								<div class=" text-xs text-gray-600 dark:text-gray-100 line-clamp-1">
+									{$i18n.t('Collection')}
+								</div>
 							{:else}
-								<div class=" font-medium text-black line-clamp-1">
+								<div class=" font-medium text-black dark:text-gray-100 line-clamp-1">
 									#{doc.name} ({doc.filename})
 								</div>
 
-								<div class=" text-xs text-gray-600 line-clamp-1">
+								<div class=" text-xs text-gray-600 dark:text-gray-100 line-clamp-1">
 									{doc.title}
 								</div>
 							{/if}

src/lib/components/chat/MessageInput/InputMenu.svelte

@@ -4,24 +4,33 @@
 	import { getContext } from 'svelte';
 
 	import Dropdown from '$lib/components/common/Dropdown.svelte';
-	import GarbageBin from '$lib/components/icons/GarbageBin.svelte';
-	import Pencil from '$lib/components/icons/Pencil.svelte';
 	import Tooltip from '$lib/components/common/Tooltip.svelte';
-	import Tags from '$lib/components/chat/Tags.svelte';
-	import Share from '$lib/components/icons/Share.svelte';
-	import ArchiveBox from '$lib/components/icons/ArchiveBox.svelte';
 	import DocumentArrowUpSolid from '$lib/components/icons/DocumentArrowUpSolid.svelte';
 	import Switch from '$lib/components/common/Switch.svelte';
 	import GlobeAltSolid from '$lib/components/icons/GlobeAltSolid.svelte';
 	import { config } from '$lib/stores';
+	import WrenchSolid from '$lib/components/icons/WrenchSolid.svelte';
 
 	const i18n = getContext('i18n');
 
 	export let uploadFilesHandler: Function;
+
+	export let selectedToolIds: string[] = [];
 	export let webSearchEnabled: boolean;
 
+	export let tools = {};
 	export let onClose: Function;
 
+	$: tools = Object.fromEntries(
+		Object.keys(tools).map((toolId) => [
+			toolId,
+			{
+				...tools[toolId],
+				enabled: selectedToolIds.includes(toolId)
+			}
+		])
+	);
+
 	let show = false;
 </script>
 
@@ -39,20 +48,48 @@
 
 	<div slot="content">
 		<DropdownMenu.Content
-			class="w-full max-w-[190px] rounded-xl px-1 py-1  border-gray-300/30 dark:border-gray-700/50 z-50 bg-white dark:bg-gray-850 dark:text-white shadow"
+			class="w-full max-w-[200px] rounded-xl px-1 py-1  border-gray-300/30 dark:border-gray-700/50 z-50 bg-white dark:bg-gray-850 dark:text-white shadow"
 			sideOffset={15}
 			alignOffset={-8}
 			side="top"
 			align="start"
 			transition={flyAndScale}
 		>
+			{#if Object.keys(tools).length > 0}
+				<div class="  max-h-28 overflow-y-auto scrollbar-hidden">
+					{#each Object.keys(tools) as toolId}
+						<div
+							class="flex gap-2 items-center px-3 py-2 text-sm font-medium cursor-pointer rounded-xl"
+						>
+							<div class="flex-1 flex items-center gap-2">
+								<WrenchSolid />
+								<Tooltip content={tools[toolId]?.description ?? ''} className="flex-1">
+									<div class=" line-clamp-1">{tools[toolId].name}</div>
+								</Tooltip>
+							</div>
+
+							<Switch
+								bind:state={tools[toolId].enabled}
+								on:change={(e) => {
+									selectedToolIds = e.detail
+										? [...selectedToolIds, toolId]
+										: selectedToolIds.filter((id) => id !== toolId);
+								}}
+							/>
+						</div>
+					{/each}
+				</div>
+
+				<hr class="border-gray-100 dark:border-gray-800 my-1" />
+			{/if}
+
 			{#if $config?.features?.enable_web_search}
 				<div
 					class="flex gap-2 items-center px-3 py-2 text-sm font-medium cursor-pointer rounded-xl"
 				>
 					<div class="flex-1 flex items-center gap-2">
 						<GlobeAltSolid />
-						<div class="flex items-center">{$i18n.t('Web Search')}</div>
+						<div class=" line-clamp-1">{$i18n.t('Web Search')}</div>
 					</div>
 
 					<Switch bind:state={webSearchEnabled} />
@@ -68,7 +105,7 @@
 				}}
 			>
 				<DocumentArrowUpSolid />
-				<div class="flex items-center">{$i18n.t('Upload Files')}</div>
+				<div class=" line-clamp-1">{$i18n.t('Upload Files')}</div>
 			</DropdownMenu.Item>
 		</DropdownMenu.Content>
 	</div>

src/lib/components/chat/MessageInput/Models.svelte

@@ -133,18 +133,18 @@
 
 {#if prompt.charAt(0) === '@'}
 	{#if filteredModels.length > 0}
-		<div class="md:px-2 mb-3 text-left w-full absolute bottom-0 left-0 right-0">
+		<div class="pl-1 pr-12 mb-3 text-left w-full absolute bottom-0 left-0 right-0">
 			<div class="flex w-full px-2">
 				<div class=" bg-gray-100 dark:bg-gray-700 w-10 rounded-l-xl text-center">
 					<div class=" text-lg font-semibold mt-2">@</div>
 				</div>
 
-				<div class="max-h-60 flex flex-col w-full rounded-r-xl bg-white">
+				<div class="max-h-60 flex flex-col w-full rounded-r-xl bg-white dark:bg-gray-850">
 					<div class="m-1 overflow-y-auto p-1 rounded-r-xl space-y-0.5">
 						{#each filteredModels as model, modelIdx}
 							<button
 								class=" px-3 py-1.5 rounded-xl w-full text-left {modelIdx === selectedIdx
-									? ' bg-gray-100 selected-command-option-button'
+									? ' bg-gray-100 dark:bg-gray-600 selected-command-option-button'
 									: ''}"
 								type="button"
 								on:click={() => {
@@ -155,7 +155,7 @@
 								}}
 								on:focus={() => {}}
 							>
-								<div class=" font-medium text-black line-clamp-1">
+								<div class=" font-medium text-black dark:text-gray-100 line-clamp-1">
 									{model.name}
 								</div>
 

src/lib/components/chat/MessageInput/PromptCommands.svelte

@@ -6,6 +6,7 @@
 
 	const i18n = getContext('i18n');
 
+	export let files;
 	export let prompt = '';
 	let selectedCommandIdx = 0;
 	let filteredPromptCommands = [];
@@ -35,6 +36,32 @@
 				return '{{CLIPBOARD}}';
 			});
 
+			console.log(clipboardText);
+
+			const clipboardItems = await navigator.clipboard.read();
+
+			let imageUrl = null;
+			for (const item of clipboardItems) {
+				// Check for known image types
+				for (const type of item.types) {
+					if (type.startsWith('image/')) {
+						const blob = await item.getType(type);
+						imageUrl = URL.createObjectURL(blob);
+						console.log(`Image URL (${type}): ${imageUrl}`);
+					}
+				}
+			}
+
+			if (imageUrl) {
+				files = [
+					...files,
+					{
+						type: 'image',
+						url: imageUrl
+					}
+				];
+			}
+
 			text = command.content.replaceAll('{{CLIPBOARD}}', clipboardText);
 		}
 
@@ -61,18 +88,18 @@
 </script>
 
 {#if filteredPromptCommands.length > 0}
-	<div class="md:px-2 mb-3 text-left w-full absolute bottom-0 left-0 right-0">
+	<div class="pl-1 pr-12 mb-3 text-left w-full absolute bottom-0 left-0 right-0">
 		<div class="flex w-full px-2">
 			<div class=" bg-gray-100 dark:bg-gray-700 w-10 rounded-l-xl text-center">
 				<div class=" text-lg font-semibold mt-2">/</div>
 			</div>
 
-			<div class="max-h-60 flex flex-col w-full rounded-r-xl bg-white">
+			<div class="max-h-60 flex flex-col w-full rounded-r-xl bg-white dark:bg-gray-850">
 				<div class="m-1 overflow-y-auto p-1 rounded-r-xl space-y-0.5">
 					{#each filteredPromptCommands as command, commandIdx}
 						<button
 							class=" px-3 py-1.5 rounded-xl w-full text-left {commandIdx === selectedCommandIdx
-								? ' bg-gray-100 selected-command-option-button'
+								? ' bg-gray-100 dark:bg-gray-600 selected-command-option-button'
 								: ''}"
 							type="button"
 							on:click={() => {
@@ -83,11 +110,11 @@
 							}}
 							on:focus={() => {}}
 						>
-							<div class=" font-medium text-black">
+							<div class=" font-medium text-black dark:text-gray-100">
 								{command.command}
 							</div>
 
-							<div class=" text-xs text-gray-600">
+							<div class=" text-xs text-gray-600 dark:text-gray-100">
 								{command.title}
 							</div>
 						</button>
@@ -95,7 +122,7 @@
 				</div>
 
 				<div
-					class=" px-2 pb-1 text-xs text-gray-600 bg-white rounded-br-xl flex items-center space-x-1"
+					class=" px-2 pb-1 text-xs text-gray-600 dark:text-gray-100 bg-white dark:bg-gray-850 rounded-br-xl flex items-center space-x-1"
 				>
 					<div>
 						<svg

src/lib/components/chat/MessageInput/VoiceRecording.svelte

@@ -0,0 +1,458 @@
+<script lang="ts">
+	import { toast } from 'svelte-sonner';
+	import { createEventDispatcher, tick, getContext } from 'svelte';
+	import { config, settings } from '$lib/stores';
+	import { blobToFile, calculateSHA256, findWordIndices } from '$lib/utils';
+
+	import { transcribeAudio } from '$lib/apis/audio';
+
+	const i18n = getContext('i18n');
+
+	const dispatch = createEventDispatcher();
+
+	export let recording = false;
+
+	let loading = false;
+	let confirmed = false;
+
+	let durationSeconds = 0;
+	let durationCounter = null;
+
+	let transcription = '';
+
+	const startDurationCounter = () => {
+		durationCounter = setInterval(() => {
+			durationSeconds++;
+		}, 1000);
+	};
+
+	const stopDurationCounter = () => {
+		clearInterval(durationCounter);
+		durationSeconds = 0;
+	};
+
+	$: if (recording) {
+		startRecording();
+	} else {
+		stopRecording();
+	}
+
+	const formatSeconds = (seconds) => {
+		const minutes = Math.floor(seconds / 60);
+		const remainingSeconds = seconds % 60;
+		const formattedSeconds = remainingSeconds < 10 ? `0${remainingSeconds}` : remainingSeconds;
+		return `${minutes}:${formattedSeconds}`;
+	};
+
+	let speechRecognition;
+
+	let mediaRecorder;
+	let audioChunks = [];
+
+	const MIN_DECIBELS = -45;
+	const VISUALIZER_BUFFER_LENGTH = 300;
+
+	let visualizerData = Array(VISUALIZER_BUFFER_LENGTH).fill(0);
+
+	// Function to calculate the RMS level from time domain data
+	const calculateRMS = (data: Uint8Array) => {
+		let sumSquares = 0;
+		for (let i = 0; i < data.length; i++) {
+			const normalizedValue = (data[i] - 128) / 128; // Normalize the data
+			sumSquares += normalizedValue * normalizedValue;
+		}
+		return Math.sqrt(sumSquares / data.length);
+	};
+
+	const normalizeRMS = (rms) => {
+		rms = rms * 10;
+		const exp = 1.5; // Adjust exponent value; values greater than 1 expand larger numbers more and compress smaller numbers more
+		const scaledRMS = Math.pow(rms, exp);
+
+		// Scale between 0.01 (1%) and 1.0 (100%)
+		return Math.min(1.0, Math.max(0.01, scaledRMS));
+	};
+
+	const analyseAudio = (stream) => {
+		const audioContext = new AudioContext();
+		const audioStreamSource = audioContext.createMediaStreamSource(stream);
+
+		const analyser = audioContext.createAnalyser();
+		analyser.minDecibels = MIN_DECIBELS;
+		audioStreamSource.connect(analyser);
+
+		const bufferLength = analyser.frequencyBinCount;
+
+		const domainData = new Uint8Array(bufferLength);
+		const timeDomainData = new Uint8Array(analyser.fftSize);
+
+		let lastSoundTime = Date.now();
+
+		const detectSound = () => {
+			const processFrame = () => {
+				if (!recording || loading) return;
+
+				if (recording && !loading) {
+					analyser.getByteTimeDomainData(timeDomainData);
+					analyser.getByteFrequencyData(domainData);
+
+					// Calculate RMS level from time domain data
+					const rmsLevel = calculateRMS(timeDomainData);
+					// Push the calculated decibel level to visualizerData
+					visualizerData.push(normalizeRMS(rmsLevel));
+
+					// Ensure visualizerData array stays within the buffer length
+					if (visualizerData.length >= VISUALIZER_BUFFER_LENGTH) {
+						visualizerData.shift();
+					}
+
+					visualizerData = visualizerData;
+
+					// if (domainData.some((value) => value > 0)) {
+					// 	lastSoundTime = Date.now();
+					// }
+
+					// if (recording && Date.now() - lastSoundTime > 3000) {
+					// 	if ($settings?.speechAutoSend ?? false) {
+					// 		confirmRecording();
+					// 	}
+					// }
+				}
+
+				window.requestAnimationFrame(processFrame);
+			};
+
+			window.requestAnimationFrame(processFrame);
+		};
+
+		detectSound();
+	};
+
+	const transcribeHandler = async (audioBlob) => {
+		// Create a blob from the audio chunks
+
+		await tick();
+		const file = blobToFile(audioBlob, 'recording.wav');
+
+		const res = await transcribeAudio(localStorage.token, file).catch((error) => {
+			toast.error(error);
+			return null;
+		});
+
+		if (res) {
+			console.log(res.text);
+			dispatch('confirm', res.text);
+		}
+	};
+
+	const saveRecording = (blob) => {
+		const url = URL.createObjectURL(blob);
+		const a = document.createElement('a');
+		document.body.appendChild(a);
+		a.style = 'display: none';
+		a.href = url;
+		a.download = 'recording.wav';
+		a.click();
+		window.URL.revokeObjectURL(url);
+	};
+
+	const startRecording = async () => {
+		startDurationCounter();
+
+		const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
+		mediaRecorder = new MediaRecorder(stream);
+		mediaRecorder.onstart = () => {
+			console.log('Recording started');
+			audioChunks = [];
+			analyseAudio(stream);
+		};
+		mediaRecorder.ondataavailable = (event) => audioChunks.push(event.data);
+		mediaRecorder.onstop = async () => {
+			console.log('Recording stopped');
+			if (($settings?.audio?.stt?.engine ?? '') === 'web') {
+				audioChunks = [];
+			} else {
+				if (confirmed) {
+					const audioBlob = new Blob(audioChunks, { type: 'audio/wav' });
+
+					await transcribeHandler(audioBlob);
+
+					confirmed = false;
+					loading = false;
+				}
+				audioChunks = [];
+				recording = false;
+			}
+		};
+		mediaRecorder.start();
+		if ($config.audio.stt.engine === 'web' || ($settings?.audio?.stt?.engine ?? '') === 'web') {
+			if ('SpeechRecognition' in window || 'webkitSpeechRecognition' in window) {
+				// Create a SpeechRecognition object
+				speechRecognition = new (window.SpeechRecognition || window.webkitSpeechRecognition)();
+
+				// Set continuous to true for continuous recognition
+				speechRecognition.continuous = true;
+
+				// Set the timeout for turning off the recognition after inactivity (in milliseconds)
+				const inactivityTimeout = 2000; // 3 seconds
+
+				let timeoutId;
+				// Start recognition
+				speechRecognition.start();
+
+				// Event triggered when speech is recognized
+				speechRecognition.onresult = async (event) => {
+					// Clear the inactivity timeout
+					clearTimeout(timeoutId);
+
+					// Handle recognized speech
+					console.log(event);
+					const transcript = event.results[Object.keys(event.results).length - 1][0].transcript;
+
+					transcription = `${transcription}${transcript}`;
+
+					await tick();
+					document.getElementById('chat-textarea')?.focus();
+
+					// Restart the inactivity timeout
+					timeoutId = setTimeout(() => {
+						console.log('Speech recognition turned off due to inactivity.');
+						speechRecognition.stop();
+					}, inactivityTimeout);
+				};
+
+				// Event triggered when recognition is ended
+				speechRecognition.onend = function () {
+					// Restart recognition after it ends
+					console.log('recognition ended');
+
+					confirmRecording();
+					dispatch('confirm', transcription);
+
+					confirmed = false;
+					loading = false;
+				};
+
+				// Event triggered when an error occurs
+				speechRecognition.onerror = function (event) {
+					console.log(event);
+					toast.error($i18n.t(`Speech recognition error: {{error}}`, { error: event.error }));
+					dispatch('cancel');
+
+					stopRecording();
+				};
+			}
+		}
+	};
+
+	const stopRecording = async () => {
+		if (recording && mediaRecorder) {
+			await mediaRecorder.stop();
+		}
+		stopDurationCounter();
+		audioChunks = [];
+	};
+
+	const confirmRecording = async () => {
+		loading = true;
+		confirmed = true;
+
+		if (recording && mediaRecorder) {
+			await mediaRecorder.stop();
+		}
+		clearInterval(durationCounter);
+	};
+</script>
+
+<div
+	class="{loading
+		? ' bg-gray-100/50 dark:bg-gray-850/50'
+		: 'bg-indigo-300/10 dark:bg-indigo-500/10 '} rounded-full flex p-2.5"
+>
+	<div class="flex items-center mr-1">
+		<button
+			type="button"
+			class="p-1.5
+
+            {loading
+				? ' bg-gray-200 dark:bg-gray-700/50'
+				: 'bg-indigo-400/20 text-indigo-600 dark:text-indigo-300 '} 
+
+
+             rounded-full"
+			on:click={async () => {
+				dispatch('cancel');
+				stopRecording();
+			}}
+		>
+			<svg
+				xmlns="http://www.w3.org/2000/svg"
+				fill="none"
+				viewBox="0 0 24 24"
+				stroke-width="3"
+				stroke="currentColor"
+				class="size-4"
+			>
+				<path stroke-linecap="round" stroke-linejoin="round" d="M6 18 18 6M6 6l12 12" />
+			</svg>
+		</button>
+	</div>
+
+	<div
+		class="flex flex-1 self-center items-center justify-between ml-2 mx-1 overflow-hidden h-6"
+		dir="rtl"
+	>
+		<div class="flex-1 flex items-center gap-0.5 h-6">
+			{#each visualizerData.slice().reverse() as rms}
+				<div
+					class="w-[2px]
+                    
+                    {loading
+						? ' bg-gray-500 dark:bg-gray-400   '
+						: 'bg-indigo-500 dark:bg-indigo-400  '} 
+                    
+                    inline-block h-full"
+					style="height: {Math.min(100, Math.max(14, rms * 100))}%;"
+				/>
+			{/each}
+		</div>
+	</div>
+
+	<div class="  mx-1.5 pr-1 flex justify-center items-center">
+		<div
+			class="text-sm
+        
+        
+        {loading ? ' text-gray-500  dark:text-gray-400  ' : ' text-indigo-400 '} 
+       font-medium flex-1 mx-auto text-center"
+		>
+			{formatSeconds(durationSeconds)}
+		</div>
+	</div>
+
+	<div class="flex items-center mr-1">
+		{#if loading}
+			<div class=" text-gray-500 rounded-full cursor-not-allowed">
+				<svg
+					width="24"
+					height="24"
+					viewBox="0 0 24 24"
+					xmlns="http://www.w3.org/2000/svg"
+					fill="currentColor"
+					><style>
+						.spinner_OSmW {
+							transform-origin: center;
+							animation: spinner_T6mA 0.75s step-end infinite;
+						}
+						@keyframes spinner_T6mA {
+							8.3% {
+								transform: rotate(30deg);
+							}
+							16.6% {
+								transform: rotate(60deg);
+							}
+							25% {
+								transform: rotate(90deg);
+							}
+							33.3% {
+								transform: rotate(120deg);
+							}
+							41.6% {
+								transform: rotate(150deg);
+							}
+							50% {
+								transform: rotate(180deg);
+							}
+							58.3% {
+								transform: rotate(210deg);
+							}
+							66.6% {
+								transform: rotate(240deg);
+							}
+							75% {
+								transform: rotate(270deg);
+							}
+							83.3% {
+								transform: rotate(300deg);
+							}
+							91.6% {
+								transform: rotate(330deg);
+							}
+							100% {
+								transform: rotate(360deg);
+							}
+						}
+					</style><g class="spinner_OSmW"
+						><rect x="11" y="1" width="2" height="5" opacity=".14" /><rect
+							x="11"
+							y="1"
+							width="2"
+							height="5"
+							transform="rotate(30 12 12)"
+							opacity=".29"
+						/><rect
+							x="11"
+							y="1"
+							width="2"
+							height="5"
+							transform="rotate(60 12 12)"
+							opacity=".43"
+						/><rect
+							x="11"
+							y="1"
+							width="2"
+							height="5"
+							transform="rotate(90 12 12)"
+							opacity=".57"
+						/><rect
+							x="11"
+							y="1"
+							width="2"
+							height="5"
+							transform="rotate(120 12 12)"
+							opacity=".71"
+						/><rect
+							x="11"
+							y="1"
+							width="2"
+							height="5"
+							transform="rotate(150 12 12)"
+							opacity=".86"
+						/><rect x="11" y="1" width="2" height="5" transform="rotate(180 12 12)" /></g
+					></svg
+				>
+			</div>
+		{:else}
+			<button
+				type="button"
+				class="p-1.5 bg-indigo-500 text-white dark:bg-indigo-500 dark:text-blue-950 rounded-full"
+				on:click={async () => {
+					await confirmRecording();
+				}}
+			>
+				<svg
+					xmlns="http://www.w3.org/2000/svg"
+					fill="none"
+					viewBox="0 0 24 24"
+					stroke-width="2.5"
+					stroke="currentColor"
+					class="size-4"
+				>
+					<path stroke-linecap="round" stroke-linejoin="round" d="m4.5 12.75 6 6 9-13.5" />
+				</svg>
+			</button>
+		{/if}
+	</div>
+</div>
+
+<style>
+	.visualizer {
+		display: flex;
+		height: 100%;
+	}
+
+	.visualizer-bar {
+		width: 2px;
+		background-color: #4a5aba; /* or whatever color you need */
+	}
+</style>

src/lib/components/chat/Messages/CompareMessages.svelte

@@ -109,7 +109,7 @@
 					class=" snap-center min-w-80 w-full max-w-full m-1 border {history.messages[
 						currentMessageId
 					].model === model
-						? 'border-gray-100 dark:border-gray-700 border-[1.5px]'
+						? 'border-gray-100 dark:border-gray-850 border-[1.5px]'
 						: 'border-gray-50 dark:border-gray-850 '} transition p-5 rounded-3xl"
 					on:click={() => {
 						currentMessageId = groupedMessages[model].messages[groupedMessagesIdx[model]].id;

src/lib/components/chat/Messages/Placeholder.svelte

@@ -1,11 +1,14 @@
 <script lang="ts">
 	import { WEBUI_BASE_URL } from '$lib/constants';
+	import { marked } from 'marked';
+
 	import { config, user, models as _models } from '$lib/stores';
 	import { onMount, getContext } from 'svelte';
 
 	import { blur, fade } from 'svelte/transition';
 
 	import Suggestions from '../MessageInput/Suggestions.svelte';
+	import { sanitizeResponseContent } from '$lib/utils';
 
 	const i18n = getContext('i18n');
 
@@ -29,7 +32,7 @@
 </script>
 
 {#key mounted}
-	<div class="m-auto w-full max-w-6xl px-8 lg:px-24 pb-16">
+	<div class="m-auto w-full max-w-6xl px-8 lg:px-24 pb-10">
 		<div class="flex justify-start">
 			<div class="flex -space-x-4 mb-1" in:fade={{ duration: 200 }}>
 				{#each models as model, modelIdx}
@@ -65,8 +68,12 @@
 
 				<div in:fade={{ duration: 200, delay: 200 }}>
 					{#if models[selectedModelIdx]?.info?.meta?.description ?? null}
-						<div class="mt-0.5 text-base font-normal text-gray-500 dark:text-gray-400 line-clamp-3">
-							{models[selectedModelIdx]?.info?.meta?.description}
+						<div
+							class="mt-0.5 text-base font-normal text-gray-500 dark:text-gray-400 line-clamp-3 markdown"
+						>
+							{@html marked.parse(
+								sanitizeResponseContent(models[selectedModelIdx]?.info?.meta?.description)
+							)}
 						</div>
 						{#if models[selectedModelIdx]?.info?.meta?.user}
 							<div class="mt-0.5 text-sm font-normal text-gray-400 dark:text-gray-500">
@@ -85,7 +92,7 @@
 							</div>
 						{/if}
 					{:else}
-						<div class=" font-medium text-gray-400 dark:text-gray-500">
+						<div class=" font-medium text-gray-400 dark:text-gray-500 line-clamp-1">
 							{$i18n.t('How can I help you today?')}
 						</div>
 					{/if}

src/lib/components/chat/Messages/RateComment.svelte

@@ -46,8 +46,8 @@
 	}
 
 	onMount(() => {
-		selectedReason = message.annotation.reason;
-		comment = message.annotation.comment;
+		selectedReason = message?.annotation?.reason ?? '';
+		comment = message?.annotation?.comment ?? '';
 		loadReasons();
 	});
 

src/lib/components/chat/Messages/ResponseMessage.svelte

@@ -211,82 +211,98 @@
 			speaking = null;
 			speakingIdx = null;
 		} else {
-			speaking = true;
-
-			if ($settings?.audio?.TTSEngine === 'openai') {
-				loadingSpeech = true;
-
-				const sentences = extractSentences(message.content).reduce((mergedTexts, currentText) => {
-					const lastIndex = mergedTexts.length - 1;
-					if (lastIndex >= 0) {
-						const previousText = mergedTexts[lastIndex];
-						const wordCount = previousText.split(/\s+/).length;
-						if (wordCount < 2) {
-							mergedTexts[lastIndex] = previousText + ' ' + currentText;
+			if ((message?.content ?? '').trim() !== '') {
+				speaking = true;
+
+				if ($config.audio.tts.engine === 'openai') {
+					loadingSpeech = true;
+
+					const sentences = extractSentences(message.content).reduce((mergedTexts, currentText) => {
+						const lastIndex = mergedTexts.length - 1;
+						if (lastIndex >= 0) {
+							const previousText = mergedTexts[lastIndex];
+							const wordCount = previousText.split(/\s+/).length;
+							if (wordCount < 2) {
+								mergedTexts[lastIndex] = previousText + ' ' + currentText;
+							} else {
+								mergedTexts.push(currentText);
+							}
 						} else {
 							mergedTexts.push(currentText);
 						}
-					} else {
-						mergedTexts.push(currentText);
-					}
-					return mergedTexts;
-				}, []);
-
-				console.log(sentences);
-
-				sentencesAudio = sentences.reduce((a, e, i, arr) => {
-					a[i] = null;
-					return a;
-				}, {});
-
-				let lastPlayedAudioPromise = Promise.resolve(); // Initialize a promise that resolves immediately
-
-				for (const [idx, sentence] of sentences.entries()) {
-					const res = await synthesizeOpenAISpeech(
-						localStorage.token,
-						$settings?.audio?.speaker,
-						sentence,
-						$settings?.audio?.model
-					).catch((error) => {
-						toast.error(error);
-
-						speaking = null;
-						loadingSpeech = false;
-
-						return null;
-					});
-
-					if (res) {
-						const blob = await res.blob();
-						const blobUrl = URL.createObjectURL(blob);
-						const audio = new Audio(blobUrl);
-						sentencesAudio[idx] = audio;
-						loadingSpeech = false;
-						lastPlayedAudioPromise = lastPlayedAudioPromise.then(() => playAudio(idx));
-					}
-				}
-			} else {
-				let voices = [];
-				const getVoicesLoop = setInterval(async () => {
-					voices = await speechSynthesis.getVoices();
-					if (voices.length > 0) {
-						clearInterval(getVoicesLoop);
+						return mergedTexts;
+					}, []);
+
+					console.log(sentences);
+
+					sentencesAudio = sentences.reduce((a, e, i, arr) => {
+						a[i] = null;
+						return a;
+					}, {});
 
-						const voice =
-							voices?.filter((v) => v.name === $settings?.audio?.speaker)?.at(0) ?? undefined;
+					let lastPlayedAudioPromise = Promise.resolve(); // Initialize a promise that resolves immediately
 
-						const speak = new SpeechSynthesisUtterance(message.content);
+					for (const [idx, sentence] of sentences.entries()) {
+						const res = await synthesizeOpenAISpeech(
+							localStorage.token,
+							$settings?.audio?.tts?.voice ?? $config?.audio?.tts?.voice,
+							sentence
+						).catch((error) => {
+							toast.error(error);
 
-						speak.onend = () => {
 							speaking = null;
-							if ($settings.conversationMode) {
-								document.getElementById('voice-input-button')?.click();
-							}
-						};
-						speak.voice = voice;
-						speechSynthesis.speak(speak);
+							loadingSpeech = false;
+
+							return null;
+						});
+
+						if (res) {
+							const blob = await res.blob();
+							const blobUrl = URL.createObjectURL(blob);
+							const audio = new Audio(blobUrl);
+							sentencesAudio[idx] = audio;
+							loadingSpeech = false;
+							lastPlayedAudioPromise = lastPlayedAudioPromise.then(() => playAudio(idx));
+						}
 					}
-				}, 100);
+				} else {
+					let voices = [];
+					const getVoicesLoop = setInterval(async () => {
+						voices = await speechSynthesis.getVoices();
+						if (voices.length > 0) {
+							clearInterval(getVoicesLoop);
+
+							const voice =
+								voices
+									?.filter(
+										(v) =>
+											v.voiceURI === ($settings?.audio?.tts?.voice ?? $config?.audio?.tts?.voice)
+									)
+									?.at(0) ?? undefined;
+
+							console.log(voice);
+
+							const speak = new SpeechSynthesisUtterance(message.content);
+
+							console.log(speak);
+
+							speak.onend = () => {
+								speaking = null;
+								if ($settings.conversationMode) {
+									document.getElementById('voice-input-button')?.click();
+								}
+							};
+
+							if (voice) {
+								speak.voice = voice;
+							}
+
+							speechSynthesis.speak(speak);
+						}
+					}, 100);
+				}
+			} else {
+				toast.error('No content to speak');
 			}
 		}
 	};
@@ -404,26 +420,29 @@
 				class="prose chat-{message.role} w-full max-w-full dark:prose-invert prose-headings:my-0 prose-headings:-mb-4 prose-p:m-0 prose-p:-mb-6 prose-pre:my-0 prose-table:my-0 prose-blockquote:my-0 prose-img:my-0 prose-ul:-my-4 prose-ol:-my-4 prose-li:-my-3 prose-ul:-mb-6 prose-ol:-mb-8 prose-ol:p-0 prose-li:-mb-4 whitespace-pre-line"
 			>
 				<div>
-					{#if message?.status}
+					{#if (message?.statusHistory ?? [...(message?.status ? [message?.status] : [])]).length > 0}
+						{@const status = (
+							message?.statusHistory ?? [...(message?.status ? [message?.status] : [])]
+						).at(-1)}
 						<div class="flex items-center gap-2 pt-1 pb-1">
-							{#if message?.status?.done === false}
+							{#if status.done === false}
 								<div class="">
 									<Spinner className="size-4" />
 								</div>
 							{/if}
 
-							{#if message?.status?.action === 'web_search' && message?.status?.urls}
-								<WebSearchResults urls={message?.status?.urls}>
+							{#if status?.action === 'web_search' && status?.urls}
+								<WebSearchResults {status}>
 									<div class="flex flex-col justify-center -space-y-0.5">
 										<div class="text-base line-clamp-1 text-wrap">
-											{message.status.description}
+											{status?.description}
 										</div>
 									</div>
 								</WebSearchResults>
 							{:else}
 								<div class="flex flex-col justify-center -space-y-0.5">
 									<div class=" text-gray-500 dark:text-gray-500 text-base line-clamp-1 text-wrap">
-										{message.status.description}
+										{status?.description}
 									</div>
 								</div>
 							{/if}
@@ -757,7 +776,7 @@
 										</Tooltip>
 
 										{#if $config?.features.enable_image_generation && !readOnly}
-											<Tooltip content="Generate Image" placement="bottom">
+											<Tooltip content={$i18n.t('Generate Image')} placement="bottom">
 												<button
 													class="{isLastMessage
 														? 'visible'
@@ -857,8 +876,8 @@
 												<button
 													class="{isLastMessage
 														? 'visible'
-														: 'invisible group-hover:visible'} p-1.5 hover:bg-black/5 dark:hover:bg-white/5 rounded-lg {message
-														?.annotation?.rating === 1
+														: 'invisible group-hover:visible'} p-1.5 hover:bg-black/5 dark:hover:bg-white/5 rounded-lg {(message
+														?.annotation?.rating ?? null) === 1
 														? 'bg-gray-100 dark:bg-gray-800'
 														: ''} dark:hover:text-white hover:text-black transition"
 													on:click={() => {
@@ -892,8 +911,8 @@
 												<button
 													class="{isLastMessage
 														? 'visible'
-														: 'invisible group-hover:visible'} p-1.5 hover:bg-black/5 dark:hover:bg-white/5 rounded-lg {message
-														?.annotation?.rating === -1
+														: 'invisible group-hover:visible'} p-1.5 hover:bg-black/5 dark:hover:bg-white/5 rounded-lg {(message
+														?.annotation?.rating ?? null) === -1
 														? 'bg-gray-100 dark:bg-gray-800'
 														: ''} dark:hover:text-white hover:text-black transition"
 													on:click={() => {
@@ -963,6 +982,7 @@
 														? 'visible'
 														: 'invisible group-hover:visible'} p-1.5 hover:bg-black/5 dark:hover:bg-white/5 rounded-lg dark:hover:text-white hover:text-black transition regenerate-response-button"
 													on:click={() => {
+														showRateComment = false;
 														regenerateResponse(message);
 													}}
 												>

src/lib/components/chat/Messages/ResponseMessage/WebSearchResults.svelte

@@ -1,10 +1,11 @@
 <script lang="ts">
 	import ChevronDown from '$lib/components/icons/ChevronDown.svelte';
 	import ChevronUp from '$lib/components/icons/ChevronUp.svelte';
+	import MagnifyingGlass from '$lib/components/icons/MagnifyingGlass.svelte';
 	import { Collapsible } from 'bits-ui';
 	import { slide } from 'svelte/transition';
 
-	export let urls = [];
+	export let status = { urls: [], query: '' };
 	let state = false;
 </script>
 
@@ -27,11 +28,45 @@
 		class=" text-sm border border-gray-300/30 dark:border-gray-700/50 rounded-xl"
 		transition={slide}
 	>
-		{#each urls as url, urlIdx}
+		{#if status?.query}
+			<a
+				href="https://www.google.com/search?q={status.query}"
+				target="_blank"
+				class="flex w-full items-center p-3 px-4 border-b border-gray-300/30 dark:border-gray-700/50 group/item justify-between font-normal text-gray-800 dark:text-gray-300 no-underline"
+			>
+				<div class="flex gap-2 items-center">
+					<MagnifyingGlass />
+
+					<div class=" line-clamp-1">
+						{status.query}
+					</div>
+				</div>
+
+				<div
+					class=" ml-1 text-white dark:text-gray-900 group-hover/item:text-gray-600 dark:group-hover/item:text-white transition"
+				>
+					<!--  -->
+					<svg
+						xmlns="http://www.w3.org/2000/svg"
+						viewBox="0 0 16 16"
+						fill="currentColor"
+						class="size-4"
+					>
+						<path
+							fill-rule="evenodd"
+							d="M4.22 11.78a.75.75 0 0 1 0-1.06L9.44 5.5H5.75a.75.75 0 0 1 0-1.5h5.5a.75.75 0 0 1 .75.75v5.5a.75.75 0 0 1-1.5 0V6.56l-5.22 5.22a.75.75 0 0 1-1.06 0Z"
+							clip-rule="evenodd"
+						/>
+					</svg>
+				</div>
+			</a>
+		{/if}
+
+		{#each status.urls as url, urlIdx}
 			<a
 				href={url}
 				target="_blank"
-				class="flex w-full items-center p-3 px-4 {urlIdx === urls.length - 1
+				class="flex w-full items-center p-3 px-4 {urlIdx === status.urls.length - 1
 					? ''
 					: 'border-b border-gray-300/30 dark:border-gray-700/50'} group/item justify-between font-normal text-gray-800 dark:text-gray-300"
 			>

src/lib/components/chat/ModelSelector/Selector.svelte

@@ -1,5 +1,6 @@
 <script lang="ts">
 	import { DropdownMenu } from 'bits-ui';
+	import { marked } from 'marked';
 
 	import { flyAndScale } from '$lib/utils/transitions';
 	import { createEventDispatcher, onMount, getContext, tick } from 'svelte';
@@ -152,6 +153,7 @@
 
 					toast.error(error);
 					// opts.callback({ success: false, error, modelName: opts.modelName });
+					break;
 				}
 			}
 
@@ -219,7 +221,7 @@
 	<DropdownMenu.Content
 		class=" z-40 {$mobile
 			? `w-full`
-			: `${className}`} max-w-[calc(100vw-1rem)] justify-start rounded-xl  bg-white dark:bg-gray-850 dark:text-white shadow-lg border border-gray-300/30 dark:border-gray-700/50  outline-none "
+			: `${className}`} max-w-[calc(100vw-1rem)] justify-start rounded-xl  bg-white dark:bg-gray-850 dark:text-white shadow-lg border border-gray-300/30 dark:border-gray-850/50  outline-none "
 		transition={flyAndScale}
 		side={$mobile ? 'bottom' : 'bottom-start'}
 		sideOffset={4}
@@ -265,7 +267,7 @@
 								</div>
 							{/if}
 							<div class="flex items-center gap-2">
-								<div class="flex items-center">
+								<div class="flex items-center min-w-fit">
 									<div class="line-clamp-1">
 										{item.label}
 									</div>
@@ -332,9 +334,12 @@
 
 								{#if item.model?.info?.meta?.description}
 									<Tooltip
-										content={`${sanitizeResponseContent(
-											item.model?.info?.meta?.description
-										).replaceAll('\n', '<br>')}`}
+										content={`${marked.parse(
+											sanitizeResponseContent(item.model?.info?.meta?.description).replaceAll(
+												'\n',
+												'<br>'
+											)
+										)}`}
 									>
 										<div class="">
 											<svg

src/lib/components/chat/Settings/About.svelte

@@ -92,7 +92,7 @@
 		</div>
 
 		{#if ollamaVersion}
-			<hr class=" dark:border-gray-700" />
+			<hr class=" dark:border-gray-850" />
 
 			<div>
 				<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Ollama Version')}</div>
@@ -104,7 +104,7 @@
 			</div>
 		{/if}
 
-		<hr class=" dark:border-gray-700" />
+		<hr class=" dark:border-gray-850" />
 
 		<div class="flex space-x-1">
 			<a href="https://discord.gg/5rJgQTnV4s" target="_blank">

src/lib/components/chat/Settings/Account.svelte

@@ -234,7 +234,7 @@
 			<UpdatePassword />
 		</div>
 
-		<hr class=" dark:border-gray-700 my-4" />
+		<hr class=" dark:border-gray-850 my-4" />
 
 		<div class="flex justify-between items-center text-sm">
 			<div class="  font-medium">{$i18n.t('API keys')}</div>

src/lib/components/chat/Settings/Advanced/AdvancedParams.svelte

@@ -5,21 +5,25 @@
 
 	const i18n = getContext('i18n');
 
+	export let admin = false;
+
 	export let params = {
 		// Advanced
-		seed: 0,
+		seed: null,
 		stop: null,
-		temperature: '',
-		frequency_penalty: '',
-		repeat_last_n: '',
-		mirostat: '',
-		mirostat_eta: '',
-		mirostat_tau: '',
-		top_k: '',
-		top_p: '',
-		tfs_z: '',
-		num_ctx: '',
-		max_tokens: '',
+		temperature: null,
+		frequency_penalty: null,
+		repeat_last_n: null,
+		mirostat: null,
+		mirostat_eta: null,
+		mirostat_tau: null,
+		top_k: null,
+		top_p: null,
+		tfs_z: null,
+		num_ctx: null,
+		num_batch: null,
+		num_keep: null,
+		max_tokens: null,
 		use_mmap: null,
 		use_mlock: null,
 		num_thread: null,
@@ -112,10 +116,10 @@
 				class="p-1 px-3 text-xs flex rounded transition"
 				type="button"
 				on:click={() => {
-					params.temperature = (params?.temperature ?? '') === '' ? 0.8 : '';
+					params.temperature = (params?.temperature ?? null) === null ? 0.8 : null;
 				}}
 			>
-				{#if (params?.temperature ?? '') === ''}
+				{#if (params?.temperature ?? null) === null}
 					<span class="ml-2 self-center"> {$i18n.t('Default')} </span>
 				{:else}
 					<span class="ml-2 self-center"> {$i18n.t('Custom')} </span>
@@ -123,7 +127,7 @@
 			</button>
 		</div>
 
-		{#if (params?.temperature ?? '') !== ''}
+		{#if (params?.temperature ?? null) !== null}
 			<div class="flex mt-0.5 space-x-2">
 				<div class=" flex-1">
 					<input
@@ -143,7 +147,7 @@
 						class=" bg-transparent text-center w-14"
 						min="0"
 						max="1"
-						step="0.05"
+						step="any"
 					/>
 				</div>
 			</div>
@@ -158,10 +162,10 @@
 				class="p-1 px-3 text-xs flex rounded transition"
 				type="button"
 				on:click={() => {
-					params.mirostat = (params?.mirostat ?? '') === '' ? 0 : '';
+					params.mirostat = (params?.mirostat ?? null) === null ? 0 : null;
 				}}
 			>
-				{#if (params?.mirostat ?? '') === ''}
+				{#if (params?.mirostat ?? null) === null}
 					<span class="ml-2 self-center">{$i18n.t('Default')}</span>
 				{:else}
 					<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@@ -169,7 +173,7 @@
 			</button>
 		</div>
 
-		{#if (params?.mirostat ?? '') !== ''}
+		{#if (params?.mirostat ?? null) !== null}
 			<div class="flex mt-0.5 space-x-2">
 				<div class=" flex-1">
 					<input
@@ -204,10 +208,10 @@
 				class="p-1 px-3 text-xs flex rounded transition"
 				type="button"
 				on:click={() => {
-					params.mirostat_eta = (params?.mirostat_eta ?? '') === '' ? 0.1 : '';
+					params.mirostat_eta = (params?.mirostat_eta ?? null) === null ? 0.1 : null;
 				}}
 			>
-				{#if (params?.mirostat_eta ?? '') === ''}
+				{#if (params?.mirostat_eta ?? null) === null}
 					<span class="ml-2 self-center">{$i18n.t('Default')}</span>
 				{:else}
 					<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@@ -215,7 +219,7 @@
 			</button>
 		</div>
 
-		{#if (params?.mirostat_eta ?? '') !== ''}
+		{#if (params?.mirostat_eta ?? null) !== null}
 			<div class="flex mt-0.5 space-x-2">
 				<div class=" flex-1">
 					<input
@@ -235,7 +239,7 @@
 						class=" bg-transparent text-center w-14"
 						min="0"
 						max="1"
-						step="0.05"
+						step="any"
 					/>
 				</div>
 			</div>
@@ -250,10 +254,10 @@
 				class="p-1 px-3 text-xs flex rounded transition"
 				type="button"
 				on:click={() => {
-					params.mirostat_tau = (params?.mirostat_tau ?? '') === '' ? 5.0 : '';
+					params.mirostat_tau = (params?.mirostat_tau ?? null) === null ? 5.0 : null;
 				}}
 			>
-				{#if (params?.mirostat_tau ?? '') === ''}
+				{#if (params?.mirostat_tau ?? null) === null}
 					<span class="ml-2 self-center">{$i18n.t('Default')}</span>
 				{:else}
 					<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@@ -261,7 +265,7 @@
 			</button>
 		</div>
 
-		{#if (params?.mirostat_tau ?? '') !== ''}
+		{#if (params?.mirostat_tau ?? null) !== null}
 			<div class="flex mt-0.5 space-x-2">
 				<div class=" flex-1">
 					<input
@@ -281,7 +285,7 @@
 						class=" bg-transparent text-center w-14"
 						min="0"
 						max="10"
-						step="0.5"
+						step="any"
 					/>
 				</div>
 			</div>
@@ -296,10 +300,10 @@
 				class="p-1 px-3 text-xs flex rounded transition"
 				type="button"
 				on:click={() => {
-					params.top_k = (params?.top_k ?? '') === '' ? 40 : '';
+					params.top_k = (params?.top_k ?? null) === null ? 40 : null;
 				}}
 			>
-				{#if (params?.top_k ?? '') === ''}
+				{#if (params?.top_k ?? null) === null}
 					<span class="ml-2 self-center">{$i18n.t('Default')}</span>
 				{:else}
 					<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@@ -307,7 +311,7 @@
 			</button>
 		</div>
 
-		{#if (params?.top_k ?? '') !== ''}
+		{#if (params?.top_k ?? null) !== null}
 			<div class="flex mt-0.5 space-x-2">
 				<div class=" flex-1">
 					<input
@@ -327,7 +331,7 @@
 						class=" bg-transparent text-center w-14"
 						min="0"
 						max="100"
-						step="0.5"
+						step="any"
 					/>
 				</div>
 			</div>
@@ -342,10 +346,10 @@
 				class="p-1 px-3 text-xs flex rounded transition"
 				type="button"
 				on:click={() => {
-					params.top_p = (params?.top_p ?? '') === '' ? 0.9 : '';
+					params.top_p = (params?.top_p ?? null) === null ? 0.9 : null;
 				}}
 			>
-				{#if (params?.top_p ?? '') === ''}
+				{#if (params?.top_p ?? null) === null}
 					<span class="ml-2 self-center">{$i18n.t('Default')}</span>
 				{:else}
 					<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@@ -353,7 +357,7 @@
 			</button>
 		</div>
 
-		{#if (params?.top_p ?? '') !== ''}
+		{#if (params?.top_p ?? null) !== null}
 			<div class="flex mt-0.5 space-x-2">
 				<div class=" flex-1">
 					<input
@@ -373,7 +377,7 @@
 						class=" bg-transparent text-center w-14"
 						min="0"
 						max="1"
-						step="0.05"
+						step="any"
 					/>
 				</div>
 			</div>
@@ -388,10 +392,10 @@
 				class="p-1 px-3 text-xs flex rounded transition"
 				type="button"
 				on:click={() => {
-					params.frequency_penalty = (params?.frequency_penalty ?? '') === '' ? 1.1 : '';
+					params.frequency_penalty = (params?.frequency_penalty ?? null) === null ? 1.1 : null;
 				}}
 			>
-				{#if (params?.frequency_penalty ?? '') === ''}
+				{#if (params?.frequency_penalty ?? null) === null}
 					<span class="ml-2 self-center">{$i18n.t('Default')}</span>
 				{:else}
 					<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@@ -399,7 +403,7 @@
 			</button>
 		</div>
 
-		{#if (params?.frequency_penalty ?? '') !== ''}
+		{#if (params?.frequency_penalty ?? null) !== null}
 			<div class="flex mt-0.5 space-x-2">
 				<div class=" flex-1">
 					<input
@@ -419,7 +423,7 @@
 						class=" bg-transparent text-center w-14"
 						min="0"
 						max="2"
-						step="0.05"
+						step="any"
 					/>
 				</div>
 			</div>
@@ -434,10 +438,10 @@
 				class="p-1 px-3 text-xs flex rounded transition"
 				type="button"
 				on:click={() => {
-					params.repeat_last_n = (params?.repeat_last_n ?? '') === '' ? 64 : '';
+					params.repeat_last_n = (params?.repeat_last_n ?? null) === null ? 64 : null;
 				}}
 			>
-				{#if (params?.repeat_last_n ?? '') === ''}
+				{#if (params?.repeat_last_n ?? null) === null}
 					<span class="ml-2 self-center">{$i18n.t('Default')}</span>
 				{:else}
 					<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@@ -445,7 +449,7 @@
 			</button>
 		</div>
 
-		{#if (params?.repeat_last_n ?? '') !== ''}
+		{#if (params?.repeat_last_n ?? null) !== null}
 			<div class="flex mt-0.5 space-x-2">
 				<div class=" flex-1">
 					<input
@@ -480,10 +484,10 @@
 				class="p-1 px-3 text-xs flex rounded transition"
 				type="button"
 				on:click={() => {
-					params.tfs_z = (params?.tfs_z ?? '') === '' ? 1 : '';
+					params.tfs_z = (params?.tfs_z ?? null) === null ? 1 : null;
 				}}
 			>
-				{#if (params?.tfs_z ?? '') === ''}
+				{#if (params?.tfs_z ?? null) === null}
 					<span class="ml-2 self-center">{$i18n.t('Default')}</span>
 				{:else}
 					<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@@ -491,7 +495,7 @@
 			</button>
 		</div>
 
-		{#if (params?.tfs_z ?? '') !== ''}
+		{#if (params?.tfs_z ?? null) !== null}
 			<div class="flex mt-0.5 space-x-2">
 				<div class=" flex-1">
 					<input
@@ -511,7 +515,7 @@
 						class=" bg-transparent text-center w-14"
 						min="0"
 						max="2"
-						step="0.05"
+						step="any"
 					/>
 				</div>
 			</div>
@@ -526,10 +530,10 @@
 				class="p-1 px-3 text-xs flex rounded transition"
 				type="button"
 				on:click={() => {
-					params.num_ctx = (params?.num_ctx ?? '') === '' ? 2048 : '';
+					params.num_ctx = (params?.num_ctx ?? null) === null ? 2048 : null;
 				}}
 			>
-				{#if (params?.num_ctx ?? '') === ''}
+				{#if (params?.num_ctx ?? null) === null}
 					<span class="ml-2 self-center">{$i18n.t('Default')}</span>
 				{:else}
 					<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@@ -537,7 +541,7 @@
 			</button>
 		</div>
 
-		{#if (params?.num_ctx ?? '') !== ''}
+		{#if (params?.num_ctx ?? null) !== null}
 			<div class="flex mt-0.5 space-x-2">
 				<div class=" flex-1">
 					<input
@@ -556,7 +560,7 @@
 						type="number"
 						class=" bg-transparent text-center w-14"
 						min="-1"
-						step="10"
+						step="1"
 					/>
 				</div>
 			</div>
@@ -565,16 +569,16 @@
 
 	<div class=" py-0.5 w-full justify-between">
 		<div class="flex w-full justify-between">
-			<div class=" self-center text-xs font-medium">{$i18n.t('Max Tokens (num_predict)')}</div>
+			<div class=" self-center text-xs font-medium">{$i18n.t('Batch Size (num_batch)')}</div>
 
 			<button
 				class="p-1 px-3 text-xs flex rounded transition"
 				type="button"
 				on:click={() => {
-					params.max_tokens = (params?.max_tokens ?? '') === '' ? 128 : '';
+					params.num_batch = (params?.num_batch ?? null) === null ? 512 : null;
 				}}
 			>
-				{#if (params?.max_tokens ?? '') === ''}
+				{#if (params?.num_batch ?? null) === null}
 					<span class="ml-2 self-center">{$i18n.t('Default')}</span>
 				{:else}
 					<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@@ -582,27 +586,26 @@
 			</button>
 		</div>
 
-		{#if (params?.max_tokens ?? '') !== ''}
+		{#if (params?.num_batch ?? null) !== null}
 			<div class="flex mt-0.5 space-x-2">
 				<div class=" flex-1">
 					<input
 						id="steps-range"
 						type="range"
-						min="-2"
-						max="16000"
-						step="1"
-						bind:value={params.max_tokens}
+						min="256"
+						max="8192"
+						step="256"
+						bind:value={params.num_batch}
 						class="w-full h-2 rounded-lg appearance-none cursor-pointer dark:bg-gray-700"
 					/>
 				</div>
 				<div class="">
 					<input
-						bind:value={params.max_tokens}
+						bind:value={params.num_batch}
 						type="number"
 						class=" bg-transparent text-center w-14"
-						min="-2"
-						max="16000"
-						step="1"
+						min="256"
+						step="256"
 					/>
 				</div>
 			</div>
@@ -611,56 +614,18 @@
 
 	<div class=" py-0.5 w-full justify-between">
 		<div class="flex w-full justify-between">
-			<div class=" self-center text-xs font-medium">{$i18n.t('use_mmap (Ollama)')}</div>
-
-			<button
-				class="p-1 px-3 text-xs flex rounded transition"
-				type="button"
-				on:click={() => {
-					params.use_mmap = (params?.use_mmap ?? null) === null ? true : null;
-				}}
-			>
-				{#if (params?.use_mmap ?? null) === null}
-					<span class="ml-2 self-center">{$i18n.t('Default')}</span>
-				{:else}
-					<span class="ml-2 self-center">{$i18n.t('On')}</span>
-				{/if}
-			</button>
-		</div>
-	</div>
-
-	<div class=" py-0.5 w-full justify-between">
-		<div class="flex w-full justify-between">
-			<div class=" self-center text-xs font-medium">{$i18n.t('use_mlock (Ollama)')}</div>
-
-			<button
-				class="p-1 px-3 text-xs flex rounded transition"
-				type="button"
-				on:click={() => {
-					params.use_mlock = (params?.use_mlock ?? null) === null ? true : null;
-				}}
-			>
-				{#if (params?.use_mlock ?? null) === null}
-					<span class="ml-2 self-center">{$i18n.t('Default')}</span>
-				{:else}
-					<span class="ml-2 self-center">{$i18n.t('On')}</span>
-				{/if}
-			</button>
-		</div>
-	</div>
-
-	<div class=" py-0.5 w-full justify-between">
-		<div class="flex w-full justify-between">
-			<div class=" self-center text-xs font-medium">{$i18n.t('num_thread (Ollama)')}</div>
+			<div class=" self-center text-xs font-medium">
+				{$i18n.t('Tokens To Keep On Context Refresh (num_keep)')}
+			</div>
 
 			<button
 				class="p-1 px-3 text-xs flex rounded transition"
 				type="button"
 				on:click={() => {
-					params.num_thread = (params?.num_thread ?? null) === null ? 2 : null;
+					params.num_keep = (params?.num_keep ?? null) === null ? 24 : null;
 				}}
 			>
-				{#if (params?.num_thread ?? null) === null}
+				{#if (params?.num_keep ?? null) === null}
 					<span class="ml-2 self-center">{$i18n.t('Default')}</span>
 				{:else}
 					<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@@ -668,26 +633,25 @@
 			</button>
 		</div>
 
-		{#if (params?.num_thread ?? null) !== null}
+		{#if (params?.num_keep ?? null) !== null}
 			<div class="flex mt-0.5 space-x-2">
 				<div class=" flex-1">
 					<input
 						id="steps-range"
 						type="range"
-						min="1"
-						max="256"
+						min="-1"
+						max="10240000"
 						step="1"
-						bind:value={params.num_thread}
+						bind:value={params.num_keep}
 						class="w-full h-2 rounded-lg appearance-none cursor-pointer dark:bg-gray-700"
 					/>
 				</div>
 				<div class="">
 					<input
-						bind:value={params.num_thread}
+						bind:value={params.num_keep}
 						type="number"
 						class=" bg-transparent text-center w-14"
-						min="1"
-						max="256"
+						min="-1"
 						step="1"
 					/>
 				</div>
@@ -697,16 +661,16 @@
 
 	<div class=" py-0.5 w-full justify-between">
 		<div class="flex w-full justify-between">
-			<div class=" self-center text-xs font-medium">{$i18n.t('Template')}</div>
+			<div class=" self-center text-xs font-medium">{$i18n.t('Max Tokens (num_predict)')}</div>
 
 			<button
 				class="p-1 px-3 text-xs flex rounded transition"
 				type="button"
 				on:click={() => {
-					params.template = (params?.template ?? null) === null ? '' : null;
+					params.max_tokens = (params?.max_tokens ?? null) === null ? 128 : null;
 				}}
 			>
-				{#if (params?.template ?? null) === null}
+				{#if (params?.max_tokens ?? null) === null}
 					<span class="ml-2 self-center">{$i18n.t('Default')}</span>
 				{:else}
 					<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
@@ -714,17 +678,151 @@
 			</button>
 		</div>
 
-		{#if (params?.template ?? null) !== null}
+		{#if (params?.max_tokens ?? null) !== null}
 			<div class="flex mt-0.5 space-x-2">
 				<div class=" flex-1">
-					<textarea
-						class="px-3 py-1.5 text-sm w-full bg-transparent border dark:border-gray-600 outline-none rounded-lg -mb-1"
-						placeholder="Write your model template content here"
-						rows="4"
-						bind:value={params.template}
+					<input
+						id="steps-range"
+						type="range"
+						min="-2"
+						max="16000"
+						step="1"
+						bind:value={params.max_tokens}
+						class="w-full h-2 rounded-lg appearance-none cursor-pointer dark:bg-gray-700"
+					/>
+				</div>
+				<div class="">
+					<input
+						bind:value={params.max_tokens}
+						type="number"
+						class=" bg-transparent text-center w-14"
+						min="-2"
+						max="16000"
+						step="1"
 					/>
 				</div>
 			</div>
 		{/if}
 	</div>
+
+	{#if admin}
+		<div class=" py-0.5 w-full justify-between">
+			<div class="flex w-full justify-between">
+				<div class=" self-center text-xs font-medium">{$i18n.t('use_mmap (Ollama)')}</div>
+
+				<button
+					class="p-1 px-3 text-xs flex rounded transition"
+					type="button"
+					on:click={() => {
+						params.use_mmap = (params?.use_mmap ?? null) === null ? true : null;
+					}}
+				>
+					{#if (params?.use_mmap ?? null) === null}
+						<span class="ml-2 self-center">{$i18n.t('Default')}</span>
+					{:else}
+						<span class="ml-2 self-center">{$i18n.t('On')}</span>
+					{/if}
+				</button>
+			</div>
+		</div>
+
+		<div class=" py-0.5 w-full justify-between">
+			<div class="flex w-full justify-between">
+				<div class=" self-center text-xs font-medium">{$i18n.t('use_mlock (Ollama)')}</div>
+
+				<button
+					class="p-1 px-3 text-xs flex rounded transition"
+					type="button"
+					on:click={() => {
+						params.use_mlock = (params?.use_mlock ?? null) === null ? true : null;
+					}}
+				>
+					{#if (params?.use_mlock ?? null) === null}
+						<span class="ml-2 self-center">{$i18n.t('Default')}</span>
+					{:else}
+						<span class="ml-2 self-center">{$i18n.t('On')}</span>
+					{/if}
+				</button>
+			</div>
+		</div>
+
+		<div class=" py-0.5 w-full justify-between">
+			<div class="flex w-full justify-between">
+				<div class=" self-center text-xs font-medium">{$i18n.t('num_thread (Ollama)')}</div>
+
+				<button
+					class="p-1 px-3 text-xs flex rounded transition"
+					type="button"
+					on:click={() => {
+						params.num_thread = (params?.num_thread ?? null) === null ? 2 : null;
+					}}
+				>
+					{#if (params?.num_thread ?? null) === null}
+						<span class="ml-2 self-center">{$i18n.t('Default')}</span>
+					{:else}
+						<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
+					{/if}
+				</button>
+			</div>
+
+			{#if (params?.num_thread ?? null) !== null}
+				<div class="flex mt-0.5 space-x-2">
+					<div class=" flex-1">
+						<input
+							id="steps-range"
+							type="range"
+							min="1"
+							max="256"
+							step="1"
+							bind:value={params.num_thread}
+							class="w-full h-2 rounded-lg appearance-none cursor-pointer dark:bg-gray-700"
+						/>
+					</div>
+					<div class="">
+						<input
+							bind:value={params.num_thread}
+							type="number"
+							class=" bg-transparent text-center w-14"
+							min="1"
+							max="256"
+							step="1"
+						/>
+					</div>
+				</div>
+			{/if}
+		</div>
+
+		<!-- <div class=" py-0.5 w-full justify-between">
+			<div class="flex w-full justify-between">
+				<div class=" self-center text-xs font-medium">{$i18n.t('Template')}</div>
+
+				<button
+					class="p-1 px-3 text-xs flex rounded transition"
+					type="button"
+					on:click={() => {
+						params.template = (params?.template ?? null) === null ? '' : null;
+					}}
+				>
+					{#if (params?.template ?? null) === null}
+						<span class="ml-2 self-center">{$i18n.t('Default')}</span>
+					{:else}
+						<span class="ml-2 self-center">{$i18n.t('Custom')}</span>
+					{/if}
+				</button>
+			</div>
+
+			{#if (params?.template ?? null) !== null}
+				<div class="flex mt-0.5 space-x-2">
+					<div class=" flex-1">
+						<textarea
+							class="px-3 py-1.5 text-sm w-full bg-transparent border dark:border-gray-600 outline-none rounded-lg -mb-1"
+							placeholder="Write your model template content here"
+							rows="4"
+							bind:value={params.template}
+						/>
+					</div>
+				</div>
+			{/if}
+		</div> -->
+	{/if}
 </div>

src/lib/components/chat/Settings/Audio.svelte

@@ -1,6 +1,5 @@
 <script lang="ts">
-	import { getAudioConfig, updateAudioConfig } from '$lib/apis/audio';
-	import { user, settings } from '$lib/stores';
+	import { user, settings, config } from '$lib/stores';
 	import { createEventDispatcher, onMount, getContext } from 'svelte';
 	import { toast } from 'svelte-sonner';
 	import Switch from '$lib/components/common/Switch.svelte';
@@ -11,26 +10,15 @@
 	export let saveSettings: Function;
 
 	// Audio
-
-	let OpenAIUrl = '';
-	let OpenAIKey = '';
-	let OpenAISpeaker = '';
-
-	let STTEngines = ['', 'openai'];
-	let STTEngine = '';
-
 	let conversationMode = false;
 	let speechAutoSend = false;
 	let responseAutoPlayback = false;
 	let nonLocalVoices = false;
 
-	let TTSEngines = ['', 'openai'];
-	let TTSEngine = '';
+	let STTEngine = '';
 
 	let voices = [];
-	let speaker = '';
-	let models = [];
-	let model = '';
+	let voice = '';
 
 	const getOpenAIVoices = () => {
 		voices = [
@@ -43,10 +31,6 @@
 		];
 	};
 
-	const getOpenAIVoicesModel = () => {
-		models = [{ name: 'tts-1' }, { name: 'tts-1-hd' }];
-	};
-
 	const getWebAPIVoices = () => {
 		const getVoicesLoop = setInterval(async () => {
 			voices = await speechSynthesis.getVoices();
@@ -58,21 +42,6 @@
 		}, 100);
 	};
 
-	const toggleConversationMode = async () => {
-		conversationMode = !conversationMode;
-
-		if (conversationMode) {
-			responseAutoPlayback = true;
-			speechAutoSend = true;
-		}
-
-		saveSettings({
-			conversationMode: conversationMode,
-			responseAutoPlayback: responseAutoPlayback,
-			speechAutoSend: speechAutoSend
-		});
-	};
-
 	const toggleResponseAutoPlayback = async () => {
 		responseAutoPlayback = !responseAutoPlayback;
 		saveSettings({ responseAutoPlayback: responseAutoPlayback });
@@ -83,76 +52,35 @@
 		saveSettings({ speechAutoSend: speechAutoSend });
 	};
 
-	const updateConfigHandler = async () => {
-		if (TTSEngine === 'openai') {
-			const res = await updateAudioConfig(localStorage.token, {
-				url: OpenAIUrl,
-				key: OpenAIKey,
-				model: model,
-				speaker: OpenAISpeaker
-			});
-
-			if (res) {
-				OpenAIUrl = res.OPENAI_API_BASE_URL;
-				OpenAIKey = res.OPENAI_API_KEY;
-				model = res.OPENAI_API_MODEL;
-				OpenAISpeaker = res.OPENAI_API_VOICE;
-			}
-		}
-	};
-
 	onMount(async () => {
 		conversationMode = $settings.conversationMode ?? false;
 		speechAutoSend = $settings.speechAutoSend ?? false;
 		responseAutoPlayback = $settings.responseAutoPlayback ?? false;
 
-		STTEngine = $settings?.audio?.STTEngine ?? '';
-		TTSEngine = $settings?.audio?.TTSEngine ?? '';
-		nonLocalVoices = $settings.audio?.nonLocalVoices ?? false;
-		speaker = $settings?.audio?.speaker ?? '';
-		model = $settings?.audio?.model ?? '';
+		STTEngine = $settings?.audio?.stt?.engine ?? '';
+		voice = $settings?.audio?.tts?.voice ?? $config.audio.tts.voice ?? '';
+		nonLocalVoices = $settings.audio?.tts?.nonLocalVoices ?? false;
 
-		if (TTSEngine === 'openai') {
+		if ($config.audio.tts.engine === 'openai') {
 			getOpenAIVoices();
-			getOpenAIVoicesModel();
 		} else {
 			getWebAPIVoices();
 		}
-
-		if ($user.role === 'admin') {
-			const res = await getAudioConfig(localStorage.token);
-
-			if (res) {
-				OpenAIUrl = res.OPENAI_API_BASE_URL;
-				OpenAIKey = res.OPENAI_API_KEY;
-				model = res.OPENAI_API_MODEL;
-				OpenAISpeaker = res.OPENAI_API_VOICE;
-				if (TTSEngine === 'openai') {
-					speaker = OpenAISpeaker;
-				}
-			}
-		}
 	});
 </script>
 
 <form
 	class="flex flex-col h-full justify-between space-y-3 text-sm"
 	on:submit|preventDefault={async () => {
-		if ($user.role === 'admin') {
-			await updateConfigHandler();
-		}
 		saveSettings({
 			audio: {
-				STTEngine: STTEngine !== '' ? STTEngine : undefined,
-				TTSEngine: TTSEngine !== '' ? TTSEngine : undefined,
-				speaker:
-					(TTSEngine === 'openai' ? OpenAISpeaker : speaker) !== ''
-						? TTSEngine === 'openai'
-							? OpenAISpeaker
-							: speaker
-						: undefined,
-				model: model !== '' ? model : undefined,
-				nonLocalVoices: nonLocalVoices
+				stt: {
+					engine: STTEngine !== '' ? STTEngine : undefined
+				},
+				tts: {
+					voice: voice !== '' ? voice : undefined,
+					nonLocalVoices: $config.audio.tts.engine === '' ? nonLocalVoices : undefined
+				}
 			}
 		});
 		dispatch('save');
@@ -162,53 +90,25 @@
 		<div>
 			<div class=" mb-1 text-sm font-medium">{$i18n.t('STT Settings')}</div>
 
-			<div class=" py-0.5 flex w-full justify-between">
-				<div class=" self-center text-xs font-medium">{$i18n.t('Speech-to-Text Engine')}</div>
-				<div class="flex items-center relative">
-					<select
-						class="dark:bg-gray-900 w-fit pr-8 rounded px-2 p-1 text-xs bg-transparent outline-none text-right"
-						bind:value={STTEngine}
-						placeholder="Select a mode"
-						on:change={(e) => {
-							if (e.target.value !== '') {
-								navigator.mediaDevices.getUserMedia({ audio: true }).catch(function (err) {
-									toast.error(
-										$i18n.t(`Permission denied when accessing microphone: {{error}}`, {
-											error: err
-										})
-									);
-									STTEngine = '';
-								});
-							}
-						}}
-					>
-						<option value="">{$i18n.t('Default (Web API)')}</option>
-						<option value="whisper-local">{$i18n.t('Whisper (Local)')}</option>
-					</select>
+			{#if $config.audio.stt.engine !== 'web'}
+				<div class=" py-0.5 flex w-full justify-between">
+					<div class=" self-center text-xs font-medium">{$i18n.t('Speech-to-Text Engine')}</div>
+					<div class="flex items-center relative">
+						<select
+							class="dark:bg-gray-900 w-fit pr-8 rounded px-2 p-1 text-xs bg-transparent outline-none text-right"
+							bind:value={STTEngine}
+							placeholder="Select an engine"
+						>
+							<option value="">{$i18n.t('Default')}</option>
+							<option value="web">{$i18n.t('Web API')}</option>
+						</select>
+					</div>
 				</div>
-			</div>
-
-			<div class=" py-0.5 flex w-full justify-between">
-				<div class=" self-center text-xs font-medium">{$i18n.t('Conversation Mode')}</div>
-
-				<button
-					class="p-1 px-3 text-xs flex rounded transition"
-					on:click={() => {
-						toggleConversationMode();
-					}}
-					type="button"
-				>
-					{#if conversationMode === true}
-						<span class="ml-2 self-center">{$i18n.t('On')}</span>
-					{:else}
-						<span class="ml-2 self-center">{$i18n.t('Off')}</span>
-					{/if}
-				</button>
-			</div>
+			{/if}
 
 			<div class=" py-0.5 flex w-full justify-between">
 				<div class=" self-center text-xs font-medium">
-					{$i18n.t('Auto-send input after 3 sec.')}
+					{$i18n.t('Instant Auto-Send After Voice Transcription')}
 				</div>
 
 				<button
@@ -230,50 +130,6 @@
 		<div>
 			<div class=" mb-1 text-sm font-medium">{$i18n.t('TTS Settings')}</div>
 
-			<div class=" py-0.5 flex w-full justify-between">
-				<div class=" self-center text-xs font-medium">{$i18n.t('Text-to-Speech Engine')}</div>
-				<div class="flex items-center relative">
-					<select
-						class=" dark:bg-gray-900 w-fit pr-8 rounded px-2 p-1 text-xs bg-transparent outline-none text-right"
-						bind:value={TTSEngine}
-						placeholder="Select a mode"
-						on:change={(e) => {
-							if (e.target.value === 'openai') {
-								getOpenAIVoices();
-								OpenAISpeaker = 'alloy';
-								model = 'tts-1';
-							} else {
-								getWebAPIVoices();
-								speaker = '';
-							}
-						}}
-					>
-						<option value="">{$i18n.t('Default (Web API)')}</option>
-						<option value="openai">{$i18n.t('Open AI')}</option>
-					</select>
-				</div>
-			</div>
-
-			{#if $user.role === 'admin'}
-				{#if TTSEngine === 'openai'}
-					<div class="mt-1 flex gap-2 mb-1">
-						<input
-							class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
-							placeholder={$i18n.t('API Base URL')}
-							bind:value={OpenAIUrl}
-							required
-						/>
-
-						<input
-							class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
-							placeholder={$i18n.t('API Key')}
-							bind:value={OpenAIKey}
-							required
-						/>
-					</div>
-				{/if}
-			{/if}
-
 			<div class=" py-0.5 flex w-full justify-between">
 				<div class=" self-center text-xs font-medium">{$i18n.t('Auto-playback response')}</div>
 
@@ -293,23 +149,23 @@
 			</div>
 		</div>
 
-		<hr class=" dark:border-gray-700" />
+		<hr class=" dark:border-gray-850" />
 
-		{#if TTSEngine === ''}
+		{#if $config.audio.tts.engine === ''}
 			<div>
 				<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Set Voice')}</div>
 				<div class="flex w-full">
 					<div class="flex-1">
 						<select
 							class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
-							bind:value={speaker}
+							bind:value={voice}
 						>
-							<option value="" selected={speaker !== ''}>{$i18n.t('Default')}</option>
-							{#each voices.filter((v) => nonLocalVoices || v.localService === true) as voice}
+							<option value="" selected={voice !== ''}>{$i18n.t('Default')}</option>
+							{#each voices.filter((v) => nonLocalVoices || v.localService === true) as _voice}
 								<option
-									value={voice.name}
+									value={_voice.name}
 									class="bg-gray-100 dark:bg-gray-700"
-									selected={speaker === voice.name}>{voice.name}</option
+									selected={voice === _voice.name}>{_voice.name}</option
 								>
 							{/each}
 						</select>
@@ -325,7 +181,7 @@
 					</div>
 				</div>
 			</div>
-		{:else if TTSEngine === 'openai'}
+		{:else if $config.audio.tts.engine === 'openai'}
 			<div>
 				<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Set Voice')}</div>
 				<div class="flex w-full">
@@ -333,7 +189,7 @@
 						<input
 							list="voice-list"
 							class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
-							bind:value={OpenAISpeaker}
+							bind:value={voice}
 							placeholder="Select a voice"
 						/>
 
@@ -345,25 +201,6 @@
 					</div>
 				</div>
 			</div>
-			<div>
-				<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Set Model')}</div>
-				<div class="flex w-full">
-					<div class="flex-1">
-						<input
-							list="model-list"
-							class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
-							bind:value={model}
-							placeholder="Select a model"
-						/>
-
-						<datalist id="model-list">
-							{#each models as model}
-								<option value={model.name} />
-							{/each}
-						</datalist>
-					</div>
-				</div>
-			</div>
 		{/if}
 	</div>
 

src/lib/components/chat/Settings/Chats.svelte

@@ -161,7 +161,7 @@
 			</div>
 		</div>
 
-		<hr class=" dark:border-gray-700" />
+		<hr class=" dark:border-gray-850" />
 
 		<div class="flex flex-col">
 			<input
@@ -218,7 +218,7 @@
 			</button>
 		</div>
 
-		<hr class=" dark:border-gray-700" />
+		<hr class=" dark:border-gray-850" />
 
 		<div class="flex flex-col">
 			{#if showArchiveConfirm}

src/lib/components/chat/Settings/General.svelte

@@ -4,7 +4,7 @@
 	import { getLanguages } from '$lib/i18n';
 	const dispatch = createEventDispatcher();
 
-	import { models, settings, theme } from '$lib/stores';
+	import { models, settings, theme, user } from '$lib/stores';
 
 	const i18n = getContext('i18n');
 
@@ -43,19 +43,21 @@
 
 	let params = {
 		// Advanced
-		seed: 0,
-		temperature: '',
-		frequency_penalty: '',
-		repeat_last_n: '',
-		mirostat: '',
-		mirostat_eta: '',
-		mirostat_tau: '',
-		top_k: '',
-		top_p: '',
+		seed: null,
+		temperature: null,
+		frequency_penalty: null,
+		repeat_last_n: null,
+		mirostat: null,
+		mirostat_eta: null,
+		mirostat_tau: null,
+		top_k: null,
+		top_p: null,
 		stop: null,
-		tfs_z: '',
-		num_ctx: '',
-		max_tokens: ''
+		tfs_z: null,
+		num_ctx: null,
+		num_batch: null,
+		num_keep: null,
+		max_tokens: null
 	};
 
 	const toggleRequestFormat = async () => {
@@ -79,12 +81,6 @@
 		requestFormat = $settings.requestFormat ?? '';
 		keepAlive = $settings.keepAlive ?? null;
 
-		params.seed = $settings.seed ?? 0;
-		params.temperature = $settings.temperature ?? '';
-		params.frequency_penalty = $settings.frequency_penalty ?? '';
-		params.top_k = $settings.top_k ?? '';
-		params.top_p = $settings.top_p ?? '';
-		params.num_ctx = $settings.num_ctx ?? '';
 		params = { ...params, ...$settings.params };
 		params.stop = $settings?.params?.stop ? ($settings?.params?.stop ?? []).join(',') : null;
 	});
@@ -146,6 +142,7 @@
 						<option value="dark">🌑 {$i18n.t('Dark')}</option>
 						<option value="oled-dark">🌃 {$i18n.t('OLED Dark')}</option>
 						<option value="light">☀️ {$i18n.t('Light')}</option>
+						<option value="her">🌷 Her</option>
 						<!-- <option value="rose-pine dark">🪻 {$i18n.t('Rosé Pine')}</option>
 						<option value="rose-pine-dawn light">🌷 {$i18n.t('Rosé Pine Dawn')}</option> -->
 					</select>
@@ -203,7 +200,7 @@
 			</div>
 		</div>
 
-		<hr class=" dark:border-gray-700 my-3" />
+		<hr class=" dark:border-gray-850 my-3" />
 
 		<div>
 			<div class=" my-2.5 text-sm font-medium">{$i18n.t('System Prompt')}</div>
@@ -227,8 +224,8 @@
 			</div>
 
 			{#if showAdvanced}
-				<AdvancedParams bind:params />
-				<hr class=" dark:border-gray-700" />
+				<AdvancedParams admin={$user?.role === 'admin'} bind:params />
+				<hr class=" dark:border-gray-850" />
 
 				<div class=" py-1 w-full justify-between">
 					<div class="flex w-full justify-between">
@@ -300,20 +297,25 @@
 				saveSettings({
 					system: system !== '' ? system : undefined,
 					params: {
-						seed: (params.seed !== 0 ? params.seed : undefined) ?? undefined,
+						seed: (params.seed !== null ? params.seed : undefined) ?? undefined,
 						stop: params.stop ? params.stop.split(',').filter((e) => e) : undefined,
-						temperature: params.temperature !== '' ? params.temperature : undefined,
+						temperature: params.temperature !== null ? params.temperature : undefined,
 						frequency_penalty:
-							params.frequency_penalty !== '' ? params.frequency_penalty : undefined,
-						repeat_last_n: params.repeat_last_n !== '' ? params.repeat_last_n : undefined,
-						mirostat: params.mirostat !== '' ? params.mirostat : undefined,
-						mirostat_eta: params.mirostat_eta !== '' ? params.mirostat_eta : undefined,
-						mirostat_tau: params.mirostat_tau !== '' ? params.mirostat_tau : undefined,
-						top_k: params.top_k !== '' ? params.top_k : undefined,
-						top_p: params.top_p !== '' ? params.top_p : undefined,
-						tfs_z: params.tfs_z !== '' ? params.tfs_z : undefined,
-						num_ctx: params.num_ctx !== '' ? params.num_ctx : undefined,
-						max_tokens: params.max_tokens !== '' ? params.max_tokens : undefined
+							params.frequency_penalty !== null ? params.frequency_penalty : undefined,
+						repeat_last_n: params.repeat_last_n !== null ? params.repeat_last_n : undefined,
+						mirostat: params.mirostat !== null ? params.mirostat : undefined,
+						mirostat_eta: params.mirostat_eta !== null ? params.mirostat_eta : undefined,
+						mirostat_tau: params.mirostat_tau !== null ? params.mirostat_tau : undefined,
+						top_k: params.top_k !== null ? params.top_k : undefined,
+						top_p: params.top_p !== null ? params.top_p : undefined,
+						tfs_z: params.tfs_z !== null ? params.tfs_z : undefined,
+						num_ctx: params.num_ctx !== null ? params.num_ctx : undefined,
+						num_batch: params.num_batch !== null ? params.num_batch : undefined,
+						num_keep: params.num_keep !== null ? params.num_keep : undefined,
+						max_tokens: params.max_tokens !== null ? params.max_tokens : undefined,
+						use_mmap: params.use_mmap !== null ? params.use_mmap : undefined,
+						use_mlock: params.use_mlock !== null ? params.use_mlock : undefined,
+						num_thread: params.num_thread !== null ? params.num_thread : undefined
 					},
 					keepAlive: keepAlive ? (isNaN(keepAlive) ? keepAlive : parseInt(keepAlive)) : undefined
 				});

src/lib/components/chat/Settings/Interface.svelte

@@ -14,19 +14,18 @@
 	// Addons
 	let titleAutoGenerate = true;
 	let responseAutoCopy = false;
-	let titleAutoGenerateModel = '';
-	let titleAutoGenerateModelExternal = '';
 	let widescreenMode = false;
-	let titleGenerationPrompt = '';
 	let splitLargeChunks = false;
 
 	// Interface
 	let defaultModelId = '';
-	let promptSuggestions = [];
 	let showUsername = false;
+
 	let chatBubble = true;
 	let chatDirection: 'LTR' | 'RTL' = 'LTR';
 
+	let showEmojiInCall = false;
+
 	const toggleSplitLargeChunks = async () => {
 		splitLargeChunks = !splitLargeChunks;
 		saveSettings({ splitLargeChunks: splitLargeChunks });
@@ -47,6 +46,11 @@
 		saveSettings({ showUsername: showUsername });
 	};
 
+	const toggleEmojiInCall = async () => {
+		showEmojiInCall = !showEmojiInCall;
+		saveSettings({ showEmojiInCall: showEmojiInCall });
+	};
+
 	const toggleTitleAutoGenerate = async () => {
 		titleAutoGenerate = !titleAutoGenerate;
 		saveSettings({
@@ -85,36 +89,19 @@
 	};
 
 	const updateInterfaceHandler = async () => {
-		if ($user.role === 'admin') {
-			promptSuggestions = await setDefaultPromptSuggestions(localStorage.token, promptSuggestions);
-			await config.set(await getBackendConfig());
-		}
-
 		saveSettings({
-			title: {
-				...$settings.title,
-				model: titleAutoGenerateModel !== '' ? titleAutoGenerateModel : undefined,
-				modelExternal:
-					titleAutoGenerateModelExternal !== '' ? titleAutoGenerateModelExternal : undefined,
-				prompt: titleGenerationPrompt ? titleGenerationPrompt : undefined
-			},
 			models: [defaultModelId]
 		});
 	};
 
 	onMount(async () => {
-		if ($user.role === 'admin') {
-			promptSuggestions = $config?.default_prompt_suggestions;
-		}
-
 		titleAutoGenerate = $settings?.title?.auto ?? true;
-		titleAutoGenerateModel = $settings?.title?.model ?? '';
-		titleAutoGenerateModelExternal = $settings?.title?.modelExternal ?? '';
-		titleGenerationPrompt =
-			$settings?.title?.prompt ??
-			`Create a concise, 3-5 word phrase as a header for the following query, strictly adhering to the 3-5 word limit and avoiding the use of the word 'title': {{prompt}}`;
+
 		responseAutoCopy = $settings.responseAutoCopy ?? false;
 		showUsername = $settings.showUsername ?? false;
+
+		showEmojiInCall = $settings.showEmojiInCall ?? false;
+
 		chatBubble = $settings.chatBubble ?? true;
 		widescreenMode = $settings.widescreenMode ?? false;
 		splitLargeChunks = $settings.splitLargeChunks ?? false;
@@ -217,6 +204,26 @@
 				</div>
 			</div>
 
+			<div>
+				<div class=" py-0.5 flex w-full justify-between">
+					<div class=" self-center text-xs font-medium">{$i18n.t('Display Emoji in Call')}</div>
+
+					<button
+						class="p-1 px-3 text-xs flex rounded transition"
+						on:click={() => {
+							toggleEmojiInCall();
+						}}
+						type="button"
+					>
+						{#if showEmojiInCall === true}
+							<span class="ml-2 self-center">{$i18n.t('On')}</span>
+						{:else}
+							<span class="ml-2 self-center">{$i18n.t('Off')}</span>
+						{/if}
+					</button>
+				</div>
+			</div>
+
 			{#if !$settings.chatBubble}
 				<div>
 					<div class=" py-0.5 flex w-full justify-between">
@@ -304,162 +311,6 @@
 				</select>
 			</div>
 		</div>
-
-		<hr class=" dark:border-gray-850" />
-
-		<div>
-			<div class=" mb-2.5 text-sm font-medium flex">
-				<div class=" mr-1">{$i18n.t('Set Task Model')}</div>
-				<Tooltip
-					content={$i18n.t(
-						'A task model is used when performing tasks such as generating titles for chats and web search queries'
-					)}
-				>
-					<svg
-						xmlns="http://www.w3.org/2000/svg"
-						fill="none"
-						viewBox="0 0 24 24"
-						stroke-width="1.5"
-						stroke="currentColor"
-						class="w-5 h-5"
-					>
-						<path
-							stroke-linecap="round"
-							stroke-linejoin="round"
-							d="m11.25 11.25.041-.02a.75.75 0 0 1 1.063.852l-.708 2.836a.75.75 0 0 0 1.063.853l.041-.021M21 12a9 9 0 1 1-18 0 9 9 0 0 1 18 0Zm-9-3.75h.008v.008H12V8.25Z"
-						/>
-					</svg>
-				</Tooltip>
-			</div>
-			<div class="flex w-full gap-2 pr-2">
-				<div class="flex-1">
-					<div class=" text-xs mb-1">Local Models</div>
-					<select
-						class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
-						bind:value={titleAutoGenerateModel}
-						placeholder={$i18n.t('Select a model')}
-					>
-						<option value="" selected>{$i18n.t('Current Model')}</option>
-						{#each $models.filter((m) => m.owned_by === 'ollama') as model}
-							<option value={model.id} class="bg-gray-100 dark:bg-gray-700">
-								{model.name}
-							</option>
-						{/each}
-					</select>
-				</div>
-
-				<div class="flex-1">
-					<div class=" text-xs mb-1">External Models</div>
-					<select
-						class="w-full rounded-lg py-2 px-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none"
-						bind:value={titleAutoGenerateModelExternal}
-						placeholder={$i18n.t('Select a model')}
-					>
-						<option value="" selected>{$i18n.t('Current Model')}</option>
-						{#each $models as model}
-							<option value={model.id} class="bg-gray-100 dark:bg-gray-700">
-								{model.name}
-							</option>
-						{/each}
-					</select>
-				</div>
-			</div>
-
-			<div class="mt-3 mr-2">
-				<div class=" mb-2.5 text-sm font-medium">{$i18n.t('Title Generation Prompt')}</div>
-				<textarea
-					bind:value={titleGenerationPrompt}
-					class="w-full rounded-lg p-4 text-sm dark:text-gray-300 dark:bg-gray-850 outline-none resize-none"
-					rows="3"
-				/>
-			</div>
-		</div>
-
-		{#if $user.role === 'admin'}
-			<hr class=" dark:border-gray-700" />
-
-			<div class=" space-y-3 pr-1.5">
-				<div class="flex w-full justify-between mb-2">
-					<div class=" self-center text-sm font-semibold">
-						{$i18n.t('Default Prompt Suggestions')}
-					</div>
-
-					<button
-						class="p-1 px-3 text-xs flex rounded transition"
-						type="button"
-						on:click={() => {
-							if (promptSuggestions.length === 0 || promptSuggestions.at(-1).content !== '') {
-								promptSuggestions = [...promptSuggestions, { content: '', title: ['', ''] }];
-							}
-						}}
-					>
-						<svg
-							xmlns="http://www.w3.org/2000/svg"
-							viewBox="0 0 20 20"
-							fill="currentColor"
-							class="w-4 h-4"
-						>
-							<path
-								d="M10.75 4.75a.75.75 0 00-1.5 0v4.5h-4.5a.75.75 0 000 1.5h4.5v4.5a.75.75 0 001.5 0v-4.5h4.5a.75.75 0 000-1.5h-4.5v-4.5z"
-							/>
-						</svg>
-					</button>
-				</div>
-				<div class="flex flex-col space-y-1">
-					{#each promptSuggestions as prompt, promptIdx}
-						<div class=" flex border dark:border-gray-600 rounded-lg">
-							<div class="flex flex-col flex-1">
-								<div class="flex border-b dark:border-gray-600 w-full">
-									<input
-										class="px-3 py-1.5 text-xs w-full bg-transparent outline-none border-r dark:border-gray-600"
-										placeholder={$i18n.t('Title (e.g. Tell me a fun fact)')}
-										bind:value={prompt.title[0]}
-									/>
-
-									<input
-										class="px-3 py-1.5 text-xs w-full bg-transparent outline-none border-r dark:border-gray-600"
-										placeholder={$i18n.t('Subtitle (e.g. about the Roman Empire)')}
-										bind:value={prompt.title[1]}
-									/>
-								</div>
-
-								<input
-									class="px-3 py-1.5 text-xs w-full bg-transparent outline-none border-r dark:border-gray-600"
-									placeholder={$i18n.t('Prompt (e.g. Tell me a fun fact about the Roman Empire)')}
-									bind:value={prompt.content}
-								/>
-							</div>
-
-							<button
-								class="px-2"
-								type="button"
-								on:click={() => {
-									promptSuggestions.splice(promptIdx, 1);
-									promptSuggestions = promptSuggestions;
-								}}
-							>
-								<svg
-									xmlns="http://www.w3.org/2000/svg"
-									viewBox="0 0 20 20"
-									fill="currentColor"
-									class="w-4 h-4"
-								>
-									<path
-										d="M6.28 5.22a.75.75 0 00-1.06 1.06L8.94 10l-3.72 3.72a.75.75 0 101.06 1.06L10 11.06l3.72 3.72a.75.75 0 101.06-1.06L11.06 10l3.72-3.72a.75.75 0 00-1.06-1.06L10 8.94 6.28 5.22z"
-									/>
-								</svg>
-							</button>
-						</div>
-					{/each}
-				</div>
-
-				{#if promptSuggestions.length > 0}
-					<div class="text-xs text-left w-full mt-2">
-						{$i18n.t('Adjusting these settings will apply changes universally to all users.')}
-					</div>
-				{/if}
-			</div>
-		{/if}
 	</div>
 
 	<div class="flex justify-end text-sm font-medium">

src/lib/components/chat/Settings/Models.svelte

@@ -541,7 +541,7 @@
 			]);
 		} else {
 			ollamaEnabled = false;
-			toast.error('Ollama API is disabled');
+			toast.error($i18n.t('Ollama API is disabled'));
 		}
 	});
 </script>
@@ -1063,7 +1063,7 @@
 				</div>
 			{/if}
 		{:else if ollamaEnabled === false}
-			<div>Ollama API is disabled</div>
+			<div>{$i18n.t('Ollama API is disabled')}</div>
 		{:else}
 			<div class="flex h-full justify-center">
 				<div class="my-auto">

src/lib/components/chat/Settings/Personalization.svelte

@@ -35,7 +35,9 @@
 		<div>
 			<div class="flex items-center justify-between mb-1">
 				<Tooltip
-					content="This is an experimental feature, it may not function as expected and is subject to change at any time."
+					content={$i18n.t(
+						'This is an experimental feature, it may not function as expected and is subject to change at any time.'
+					)}
 				>
 					<div class="text-sm font-medium">
 						{$i18n.t('Memory')}
@@ -57,8 +59,9 @@
 
 		<div class="text-xs text-gray-600 dark:text-gray-400">
 			<div>
-				You can personalize your interactions with LLMs by adding memories through the 'Manage'
-				button below, making them more helpful and tailored to you.
+				{$i18n.t(
+					"You can personalize your interactions with LLMs by adding memories through the 'Manage' button below, making them more helpful and tailored to you."
+				)}
 			</div>
 
 			<!-- <div class="mt-3">
@@ -79,7 +82,7 @@
 					showManageModal = true;
 				}}
 			>
-				Manage
+				{$i18n.t('Manage')}
 			</button>
 		</div>
 	</div>

src/lib/components/chat/Settings/Personalization/AddMemoryModal.svelte

@@ -2,13 +2,12 @@
 	import { createEventDispatcher, getContext } from 'svelte';
 
 	import Modal from '$lib/components/common/Modal.svelte';
-	import { addNewMemory } from '$lib/apis/memories';
+	import { addNewMemory, updateMemoryById } from '$lib/apis/memories';
 	import { toast } from 'svelte-sonner';
 
 	const dispatch = createEventDispatcher();
 
 	export let show;
-
 	const i18n = getContext('i18n');
 
 	let loading = false;
@@ -38,7 +37,9 @@
 <Modal bind:show size="sm">
 	<div>
 		<div class=" flex justify-between dark:text-gray-300 px-5 pt-4 pb-2">
-			<div class=" text-lg font-medium self-center">{$i18n.t('Add Memory')}</div>
+			<div class=" text-lg font-medium self-center">
+				{$i18n.t('Add Memory')}
+			</div>
 			<button
 				class="self-center"
 				on:click={() => {
@@ -75,7 +76,7 @@
 						/>
 
 						<div class="text-xs text-gray-500">
-							ⓘ Refer to yourself as "User" (e.g., "User is learning Spanish")
+							ⓘ {$i18n.t('Refer to yourself as "User" (e.g., "User is learning Spanish")')}
 						</div>
 					</div>
 

src/lib/components/chat/Settings/Personalization/EditMemoryModal.svelte

@@ -0,0 +1,136 @@
+<script>
+	import { createEventDispatcher, getContext } from 'svelte';
+	import { toast } from 'svelte-sonner';
+
+	import { updateMemoryById } from '$lib/apis/memories';
+
+	import Modal from '$lib/components/common/Modal.svelte';
+
+	const dispatch = createEventDispatcher();
+
+	export let show;
+	export let memory = {};
+
+	const i18n = getContext('i18n');
+
+	let loading = false;
+	let content = '';
+
+	$: if (show) {
+		setContent();
+	}
+
+	const setContent = () => {
+		content = memory.content;
+	};
+
+	const submitHandler = async () => {
+		loading = true;
+
+		const res = await updateMemoryById(localStorage.token, memory.id, content).catch((error) => {
+			toast.error(error);
+
+			return null;
+		});
+
+		if (res) {
+			console.log(res);
+			toast.success('Memory updated successfully');
+			dispatch('save');
+			show = false;
+		}
+
+		loading = false;
+	};
+</script>
+
+<Modal bind:show size="sm">
+	<div>
+		<div class=" flex justify-between dark:text-gray-300 px-5 pt-4 pb-2">
+			<div class=" text-lg font-medium self-center">
+				{$i18n.t('Edit Memory')}
+			</div>
+			<button
+				class="self-center"
+				on:click={() => {
+					show = false;
+				}}
+			>
+				<svg
+					xmlns="http://www.w3.org/2000/svg"
+					viewBox="0 0 20 20"
+					fill="currentColor"
+					class="w-5 h-5"
+				>
+					<path
+						d="M6.28 5.22a.75.75 0 00-1.06 1.06L8.94 10l-3.72 3.72a.75.75 0 101.06 1.06L10 11.06l3.72 3.72a.75.75 0 101.06-1.06L11.06 10l3.72-3.72a.75.75 0 00-1.06-1.06L10 8.94 6.28 5.22z"
+					/>
+				</svg>
+			</button>
+		</div>
+
+		<div class="flex flex-col md:flex-row w-full px-5 pb-4 md:space-x-4 dark:text-gray-200">
+			<div class=" flex flex-col w-full sm:flex-row sm:justify-center sm:space-x-6">
+				<form
+					class="flex flex-col w-full"
+					on:submit|preventDefault={() => {
+						submitHandler();
+					}}
+				>
+					<div class="">
+						<textarea
+							bind:value={content}
+							class=" bg-transparent w-full text-sm resize-none rounded-xl p-3 outline outline-1 outline-gray-100 dark:outline-gray-800"
+							rows="3"
+							placeholder={$i18n.t('Enter a detail about yourself for your LLMs to recall')}
+						/>
+
+						<div class="text-xs text-gray-500">
+							ⓘ {$i18n.t('Refer to yourself as "User" (e.g., "User is learning Spanish")')}
+						</div>
+					</div>
+
+					<div class="flex justify-end pt-1 text-sm font-medium">
+						<button
+							class=" px-4 py-2 bg-emerald-700 hover:bg-emerald-800 text-gray-100 transition rounded-3xl flex flex-row space-x-1 items-center {loading
+								? ' cursor-not-allowed'
+								: ''}"
+							type="submit"
+							disabled={loading}
+						>
+							{$i18n.t('Update')}
+
+							{#if loading}
+								<div class="ml-2 self-center">
+									<svg
+										class=" w-4 h-4"
+										viewBox="0 0 24 24"
+										fill="currentColor"
+										xmlns="http://www.w3.org/2000/svg"
+										><style>
+											.spinner_ajPY {
+												transform-origin: center;
+												animation: spinner_AtaB 0.75s infinite linear;
+											}
+											@keyframes spinner_AtaB {
+												100% {
+													transform: rotate(360deg);
+												}
+											}
+										</style><path
+											d="M12,1A11,11,0,1,0,23,12,11,11,0,0,0,12,1Zm0,19a8,8,0,1,1,8-8A8,8,0,0,1,12,20Z"
+											opacity=".25"
+										/><path
+											d="M10.14,1.16a11,11,0,0,0-9,8.92A1.59,1.59,0,0,0,2.46,12,1.52,1.52,0,0,0,4.11,10.7a8,8,0,0,1,6.66-6.61A1.42,1.42,0,0,0,12,2.69h0A1.57,1.57,0,0,0,10.14,1.16Z"
+											class="spinner_ajPY"
+										/></svg
+									>
+								</div>
+							{/if}
+						</button>
+					</div>
+				</form>
+			</div>
+		</div>
+	</div>
+</Modal>

src/lib/components/chat/Settings/Personalization/ManageModal.svelte

@@ -10,18 +10,24 @@
 	import { deleteMemoriesByUserId, deleteMemoryById, getMemories } from '$lib/apis/memories';
 	import Tooltip from '$lib/components/common/Tooltip.svelte';
 	import { error } from '@sveltejs/kit';
+	import EditMemoryModal from './EditMemoryModal.svelte';
 
 	const i18n = getContext('i18n');
 
 	export let show = false;
 
 	let memories = [];
+	let loading = true;
 
 	let showAddMemoryModal = false;
+	let showEditMemoryModal = false;
 
-	$: if (show) {
+	let selectedMemory = null;
+
+	$: if (show && memories.length === 0 && loading) {
 		(async () => {
 			memories = await getMemories(localStorage.token);
+			loading = false;
 		})();
 	}
 </script>
@@ -62,7 +68,9 @@
 								>
 									<tr>
 										<th scope="col" class="px-3 py-2"> {$i18n.t('Name')} </th>
-										<th scope="col" class="px-3 py-2 hidden md:flex"> {$i18n.t('Created At')} </th>
+										<th scope="col" class="px-3 py-2 hidden md:flex">
+											{$i18n.t('Last Modified')}
+										</th>
 										<th scope="col" class="px-3 py-2 text-right" />
 									</tr>
 								</thead>
@@ -76,11 +84,38 @@
 											</td>
 											<td class=" px-3 py-1 hidden md:flex h-[2.5rem]">
 												<div class="my-auto whitespace-nowrap">
-													{dayjs(memory.created_at * 1000).format($i18n.t('MMMM DD, YYYY'))}
+													{dayjs(memory.updated_at * 1000).format(
+														$i18n.t('MMMM DD, YYYY hh:mm:ss A')
+													)}
 												</div>
 											</td>
 											<td class="px-3 py-1">
 												<div class="flex justify-end w-full">
+													<Tooltip content="Edit">
+														<button
+															class="self-center w-fit text-sm px-2 py-2 hover:bg-black/5 dark:hover:bg-white/5 rounded-xl"
+															on:click={() => {
+																selectedMemory = memory;
+																showEditMemoryModal = true;
+															}}
+														>
+															<svg
+																xmlns="http://www.w3.org/2000/svg"
+																fill="none"
+																viewBox="0 0 24 24"
+																stroke-width="1.5"
+																stroke="currentColor"
+																class="w-4 h-4 s-FoVA_WMOgxUD"
+																><path
+																	stroke-linecap="round"
+																	stroke-linejoin="round"
+																	d="m16.862 4.487 1.687-1.688a1.875 1.875 0 1 1 2.652 2.652L6.832 19.82a4.5 4.5 0 0 1-1.897 1.13l-2.685.8.8-2.685a4.5 4.5 0 0 1 1.13-1.897L16.863 4.487Zm0 0L19.5 7.125"
+																	class="s-FoVA_WMOgxUD"
+																/></svg
+															>
+														</button>
+													</Tooltip>
+
 													<Tooltip content="Delete">
 														<button
 															class="self-center w-fit text-sm px-2 py-2 hover:bg-black/5 dark:hover:bg-white/5 rounded-xl"
@@ -136,7 +171,7 @@
 					class=" px-3.5 py-1.5 font-medium hover:bg-black/5 dark:hover:bg-white/5 outline outline-1 outline-gray-300 dark:outline-gray-800 rounded-3xl"
 					on:click={() => {
 						showAddMemoryModal = true;
-					}}>Add memory</button
+					}}>{$i18n.t('Add Memory')}</button
 				>
 				<button
 					class=" px-3.5 py-1.5 font-medium text-red-500 hover:bg-black/5 dark:hover:bg-white/5 outline outline-1 outline-red-300 dark:outline-red-800 rounded-3xl"
@@ -150,7 +185,7 @@
 							toast.success('Memory cleared successfully');
 							memories = [];
 						}
-					}}>Clear memory</button
+					}}>{$i18n.t('Clear memory')}</button
 				>
 			</div>
 		</div>
@@ -163,3 +198,11 @@
 		memories = await getMemories(localStorage.token);
 	}}
 />
+
+<EditMemoryModal
+	bind:show={showEditMemoryModal}
+	memory={selectedMemory}
+	on:save={async () => {
+		memories = await getMemories(localStorage.token);
+	}}
+/>

src/lib/components/chat/SettingsModal.svelte

@@ -8,16 +8,14 @@
 	import Modal from '../common/Modal.svelte';
 	import Account from './Settings/Account.svelte';
 	import About from './Settings/About.svelte';
-	import Models from './Settings/Models.svelte';
 	import General from './Settings/General.svelte';
 	import Interface from './Settings/Interface.svelte';
 	import Audio from './Settings/Audio.svelte';
 	import Chats from './Settings/Chats.svelte';
-	import Connections from './Settings/Connections.svelte';
-	import Images from './Settings/Images.svelte';
 	import User from '../icons/User.svelte';
 	import Personalization from './Settings/Personalization.svelte';
 	import { updateUserSettings } from '$lib/apis/users';
+	import { goto } from '$app/navigation';
 
 	const i18n = getContext('i18n');
 
@@ -90,55 +88,32 @@
 					<div class=" self-center">{$i18n.t('General')}</div>
 				</button>
 
-				{#if $user?.role === 'admin'}
-					<button
-						class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
-						'connections'
-							? 'bg-gray-200 dark:bg-gray-700'
-							: ' hover:bg-gray-300 dark:hover:bg-gray-800'}"
-						on:click={() => {
-							selectedTab = 'connections';
-						}}
-					>
-						<div class=" self-center mr-2">
-							<svg
-								xmlns="http://www.w3.org/2000/svg"
-								viewBox="0 0 16 16"
-								fill="currentColor"
-								class="w-4 h-4"
-							>
-								<path
-									d="M1 9.5A3.5 3.5 0 0 0 4.5 13H12a3 3 0 0 0 .917-5.857 2.503 2.503 0 0 0-3.198-3.019 3.5 3.5 0 0 0-6.628 2.171A3.5 3.5 0 0 0 1 9.5Z"
-								/>
-							</svg>
-						</div>
-						<div class=" self-center">{$i18n.t('Connections')}</div>
-					</button>
-
+				{#if $user.role === 'admin'}
 					<button
 						class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
-						'models'
+						'admin'
 							? 'bg-gray-200 dark:bg-gray-700'
 							: ' hover:bg-gray-300 dark:hover:bg-gray-800'}"
-						on:click={() => {
-							selectedTab = 'models';
+						on:click={async () => {
+							await goto('/admin/settings');
+							show = false;
 						}}
 					>
 						<div class=" self-center mr-2">
 							<svg
 								xmlns="http://www.w3.org/2000/svg"
-								viewBox="0 0 20 20"
+								viewBox="0 0 24 24"
 								fill="currentColor"
-								class="w-4 h-4"
+								class="size-4"
 							>
 								<path
 									fill-rule="evenodd"
-									d="M10 1c3.866 0 7 1.79 7 4s-3.134 4-7 4-7-1.79-7-4 3.134-4 7-4zm5.694 8.13c.464-.264.91-.583 1.306-.952V10c0 2.21-3.134 4-7 4s-7-1.79-7-4V8.178c.396.37.842.688 1.306.953C5.838 10.006 7.854 10.5 10 10.5s4.162-.494 5.694-1.37zM3 13.179V15c0 2.21 3.134 4 7 4s7-1.79 7-4v-1.822c-.396.37-.842.688-1.306.953-1.532.875-3.548 1.369-5.694 1.369s-4.162-.494-5.694-1.37A7.009 7.009 0 013 13.179z"
+									d="M4.5 3.75a3 3 0 0 0-3 3v10.5a3 3 0 0 0 3 3h15a3 3 0 0 0 3-3V6.75a3 3 0 0 0-3-3h-15Zm4.125 3a2.25 2.25 0 1 0 0 4.5 2.25 2.25 0 0 0 0-4.5Zm-3.873 8.703a4.126 4.126 0 0 1 7.746 0 .75.75 0 0 1-.351.92 7.47 7.47 0 0 1-3.522.877 7.47 7.47 0 0 1-3.522-.877.75.75 0 0 1-.351-.92ZM15 8.25a.75.75 0 0 0 0 1.5h3.75a.75.75 0 0 0 0-1.5H15ZM14.25 12a.75.75 0 0 1 .75-.75h3.75a.75.75 0 0 1 0 1.5H15a.75.75 0 0 1-.75-.75Zm.75 2.25a.75.75 0 0 0 0 1.5h3.75a.75.75 0 0 0 0-1.5H15Z"
 									clip-rule="evenodd"
 								/>
 							</svg>
 						</div>
-						<div class=" self-center">{$i18n.t('Models')}</div>
+						<div class=" self-center">{$i18n.t('Admin Settings')}</div>
 					</button>
 				{/if}
 
@@ -210,34 +185,6 @@
 					<div class=" self-center">{$i18n.t('Audio')}</div>
 				</button>
 
-				{#if $user.role === 'admin'}
-					<button
-						class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
-						'images'
-							? 'bg-gray-200 dark:bg-gray-700'
-							: ' hover:bg-gray-300 dark:hover:bg-gray-800'}"
-						on:click={() => {
-							selectedTab = 'images';
-						}}
-					>
-						<div class=" self-center mr-2">
-							<svg
-								xmlns="http://www.w3.org/2000/svg"
-								viewBox="0 0 16 16"
-								fill="currentColor"
-								class="w-4 h-4"
-							>
-								<path
-									fill-rule="evenodd"
-									d="M2 4a2 2 0 0 1 2-2h8a2 2 0 0 1 2 2v8a2 2 0 0 1-2 2H4a2 2 0 0 1-2-2V4Zm10.5 5.707a.5.5 0 0 0-.146-.353l-1-1a.5.5 0 0 0-.708 0L9.354 9.646a.5.5 0 0 1-.708 0L6.354 7.354a.5.5 0 0 0-.708 0l-2 2a.5.5 0 0 0-.146.353V12a.5.5 0 0 0 .5.5h8a.5.5 0 0 0 .5-.5V9.707ZM12 5a1 1 0 1 1-2 0 1 1 0 0 1 2 0Z"
-									clip-rule="evenodd"
-								/>
-							</svg>
-						</div>
-						<div class=" self-center">{$i18n.t('Images')}</div>
-					</button>
-				{/if}
-
 				<button
 					class="px-2.5 py-2.5 min-w-fit rounded-lg flex-1 md:flex-none flex text-right transition {selectedTab ===
 					'chats'
@@ -325,15 +272,6 @@
 							toast.success($i18n.t('Settings saved successfully!'));
 						}}
 					/>
-				{:else if selectedTab === 'models'}
-					<Models {getModels} />
-				{:else if selectedTab === 'connections'}
-					<Connections
-						{getModels}
-						on:save={() => {
-							toast.success($i18n.t('Settings saved successfully!'));
-						}}
-					/>
 				{:else if selectedTab === 'interface'}
 					<Interface
 						{saveSettings}
@@ -355,13 +293,6 @@
 							toast.success($i18n.t('Settings saved successfully!'));
 						}}
 					/>
-				{:else if selectedTab === 'images'}
-					<Images
-						{saveSettings}
-						on:save={() => {
-							toast.success($i18n.t('Settings saved successfully!'));
-						}}
-					/>
 				{:else if selectedTab === 'chats'}
 					<Chats {saveSettings} />
 				{:else if selectedTab === 'account'}

src/lib/components/common/CodeEditor.svelte

@@ -0,0 +1,135 @@
+<script lang="ts">
+	import { basicSetup, EditorView } from 'codemirror';
+	import { keymap, placeholder } from '@codemirror/view';
+	import { Compartment, EditorState } from '@codemirror/state';
+
+	import { acceptCompletion } from '@codemirror/autocomplete';
+	import { indentWithTab } from '@codemirror/commands';
+
+	import { indentUnit } from '@codemirror/language';
+	import { python } from '@codemirror/lang-python';
+	import { oneDark } from '@codemirror/theme-one-dark';
+
+	import { onMount, createEventDispatcher } from 'svelte';
+	import { formatPythonCode } from '$lib/apis/utils';
+	import { toast } from 'svelte-sonner';
+
+	const dispatch = createEventDispatcher();
+
+	export let boilerplate = '';
+	export let value = '';
+
+	let codeEditor;
+
+	let isDarkMode = false;
+	let editorTheme = new Compartment();
+
+	export const formatPythonCodeHandler = async () => {
+		if (codeEditor) {
+			const res = await formatPythonCode(value).catch((error) => {
+				toast.error(error);
+				return null;
+			});
+
+			if (res && res.code) {
+				const formattedCode = res.code;
+				codeEditor.dispatch({
+					changes: [{ from: 0, to: codeEditor.state.doc.length, insert: formattedCode }]
+				});
+
+				toast.success('Code formatted successfully');
+				return true;
+			}
+			return false;
+		}
+		return false;
+	};
+
+	let extensions = [
+		basicSetup,
+		keymap.of([{ key: 'Tab', run: acceptCompletion }, indentWithTab]),
+		python(),
+		indentUnit.of('    '),
+		placeholder('Enter your code here...'),
+		EditorView.updateListener.of((e) => {
+			if (e.docChanged) {
+				value = e.state.doc.toString();
+			}
+		}),
+		editorTheme.of([])
+	];
+
+	onMount(() => {
+		console.log(value);
+		if (value === '') {
+			value = boilerplate;
+		}
+
+		// Check if html class has dark mode
+		isDarkMode = document.documentElement.classList.contains('dark');
+
+		// python code editor, highlight python code
+		codeEditor = new EditorView({
+			state: EditorState.create({
+				doc: value,
+				extensions: extensions
+			}),
+			parent: document.getElementById('code-textarea')
+		});
+
+		if (isDarkMode) {
+			codeEditor.dispatch({
+				effects: editorTheme.reconfigure(oneDark)
+			});
+		}
+
+		// listen to html class changes this should fire only when dark mode is toggled
+		const observer = new MutationObserver((mutations) => {
+			mutations.forEach((mutation) => {
+				if (mutation.type === 'attributes' && mutation.attributeName === 'class') {
+					const _isDarkMode = document.documentElement.classList.contains('dark');
+
+					if (_isDarkMode !== isDarkMode) {
+						isDarkMode = _isDarkMode;
+						if (_isDarkMode) {
+							codeEditor.dispatch({
+								effects: editorTheme.reconfigure(oneDark)
+							});
+						} else {
+							codeEditor.dispatch({
+								effects: editorTheme.reconfigure()
+							});
+						}
+					}
+				}
+			});
+		});
+
+		observer.observe(document.documentElement, {
+			attributes: true,
+			attributeFilter: ['class']
+		});
+
+		const keydownHandler = async (e) => {
+			if ((e.ctrlKey || e.metaKey) && e.key === 's') {
+				e.preventDefault();
+				dispatch('save');
+			}
+
+			// Format code when Ctrl + Shift + F is pressed
+			if ((e.ctrlKey || e.metaKey) && e.shiftKey && e.key === 'f') {
+				e.preventDefault();
+				await formatPythonCodeHandler();
+			}
+		};
+
+		document.addEventListener('keydown', keydownHandler);
+
+		return () => {
+			observer.disconnect();
+			document.removeEventListener('keydown', keydownHandler);
+		};
+	});
+</script>
+
+<div id="code-textarea" class="h-full w-full" />

src/lib/components/common/ConfirmDialog.svelte

@@ -0,0 +1,109 @@
+<script lang="ts">
+	import { onMount, createEventDispatcher } from 'svelte';
+	import { fade } from 'svelte/transition';
+
+	import { flyAndScale } from '$lib/utils/transitions';
+
+	const dispatch = createEventDispatcher();
+
+	export let title = 'Confirm your action';
+	export let message = 'This action cannot be undone. Do you wish to continue?';
+
+	export let cancelLabel = 'Cancel';
+	export let confirmLabel = 'Confirm';
+
+	export let show = false;
+	let modalElement = null;
+	let mounted = false;
+
+	const handleKeyDown = (event: KeyboardEvent) => {
+		if (event.key === 'Escape') {
+			console.log('Escape');
+			show = false;
+		}
+	};
+
+	onMount(() => {
+		mounted = true;
+	});
+
+	$: if (mounted) {
+		if (show) {
+			window.addEventListener('keydown', handleKeyDown);
+			document.body.style.overflow = 'hidden';
+		} else {
+			window.removeEventListener('keydown', handleKeyDown);
+			document.body.style.overflow = 'unset';
+		}
+	}
+</script>
+
+{#if show}
+	<!-- svelte-ignore a11y-click-events-have-key-events -->
+	<!-- svelte-ignore a11y-no-static-element-interactions -->
+	<div
+		bind:this={modalElement}
+		class=" fixed top-0 right-0 left-0 bottom-0 bg-black/60 w-full min-h-screen h-screen flex justify-center z-[9999] overflow-hidden overscroll-contain"
+		in:fade={{ duration: 10 }}
+		on:mousedown={() => {
+			show = false;
+		}}
+	>
+		<div
+			class=" m-auto rounded-2xl max-w-full w-[32rem] mx-2 bg-gray-50 dark:bg-gray-950 shadow-3xl border border-gray-850"
+			in:flyAndScale
+			on:mousedown={(e) => {
+				e.stopPropagation();
+			}}
+		>
+			<div class="px-[1.75rem] py-6">
+				<div class=" text-lg font-semibold dark:text-gray-200 mb-2.5">{title}</div>
+
+				<slot>
+					<div class=" text-sm text-gray-500">
+						{message}
+					</div>
+				</slot>
+
+				<div class="mt-6 flex justify-between gap-1.5">
+					<button
+						class="bg-gray-100 hover:bg-gray-200 text-gray-800 dark:bg-gray-850 dark:hover:bg-gray-800 dark:text-white font-medium w-full py-2.5 rounded-lg transition"
+						on:click={() => {
+							show = false;
+						}}
+						type="button"
+					>
+						{cancelLabel}
+					</button>
+					<button
+						class="bg-gray-900 hover:bg-gray-850 text-gray-100 dark:bg-gray-100 dark:hover:bg-white dark:text-gray-800 font-medium w-full py-2.5 rounded-lg transition"
+						on:click={() => {
+							show = false;
+							dispatch('confirm');
+						}}
+						type="button"
+					>
+						{confirmLabel}
+					</button>
+				</div>
+			</div>
+		</div>
+	</div>
+{/if}
+
+<style>
+	.modal-content {
+		animation: scaleUp 0.1s ease-out forwards;
+	}
+
+	@keyframes scaleUp {
+		from {
+			transform: scale(0.985);
+			opacity: 0;
+		}
+		to {
+			transform: scale(1);
+			opacity: 1;
+		}
+	}
+</style>

src/lib/components/common/Selector.svelte

@@ -48,7 +48,7 @@
 		<ChevronDown className="absolute end-2 top-1/2 -translate-y-[45%] size-3.5" strokeWidth="2.5" />
 	</Select.Trigger>
 	<Select.Content
-		class="w-full rounded-lg  bg-white dark:bg-gray-900 dark:text-white shadow-lg border border-gray-300/30 dark:border-gray-700/50  outline-none"
+		class="w-full rounded-lg  bg-white dark:bg-gray-900 dark:text-white shadow-lg border border-gray-300/30 dark:border-gray-850/50  outline-none"
 		transition={flyAndScale}
 		sideOffset={4}
 	>

src/lib/components/common/Tooltip.svelte

@@ -1,5 +1,7 @@
 <script lang="ts">
 	import { onDestroy } from 'svelte';
+	import { marked } from 'marked';
+
 	import tippy from 'tippy.js';
 
 	export let placement = 'top';

src/lib/components/documents/Settings/QueryParams.svelte

@@ -95,7 +95,7 @@
 					)}
 				</div>
 
-				<hr class=" dark:border-gray-700 my-3" />
+				<hr class=" dark:border-gray-850 my-3" />
 			{/if}
 
 			<div>

src/lib/components/documents/Settings/WebParams.svelte

@@ -11,7 +11,7 @@
 	export let saveHandler: Function;
 
 	let webConfig = null;
-	let webSearchEngines = ['searxng', 'google_pse', 'brave', 'serpstack', 'serper'];
+	let webSearchEngines = ['searxng', 'google_pse', 'brave', 'serpstack', 'serper', 'serply'];
 
 	let youtubeLanguage = 'en';
 	let youtubeTranslation = null;
@@ -68,10 +68,10 @@
 						<select
 							class="dark:bg-gray-900 w-fit pr-8 rounded px-2 p-1 text-xs bg-transparent outline-none text-right"
 							bind:value={webConfig.search.engine}
-							placeholder="Select a engine"
+							placeholder={$i18n.t('Select a engine')}
 							required
 						>
-							<option disabled selected value="">Select a engine</option>
+							<option disabled selected value="">{$i18n.t('Select a engine')}</option>
 							{#each webSearchEngines as engine}
 								<option value={engine}>{engine}</option>
 							{/each}

src/lib/components/icons/Headphone.svelte

@@ -0,0 +1,20 @@
+<script lang="ts">
+	export let className = 'w-4 h-4';
+	export let strokeWidth = '0';
+</script>
+
+<svg
+	aria-hidden="true"
+	xmlns="http://www.w3.org/2000/svg"
+	fill="currentColor"
+	viewBox="0 0 24 24"
+	stroke-width={strokeWidth}
+	stroke="currentColor"
+	class={className}
+>
+	<path
+		fill-rule="evenodd"
+		d="M12 5a7 7 0 0 0-7 7v1.17c.313-.11.65-.17 1-.17h2a1 1 0 0 1 1 1v6a1 1 0 0 1-1 1H6a3 3 0 0 1-3-3v-6a9 9 0 0 1 18 0v6a3 3 0 0 1-3 3h-2a1 1 0 0 1-1-1v-6a1 1 0 0 1 1-1h2c.35 0 .687.06 1 .17V12a7 7 0 0 0-7-7Z"
+		clip-rule="evenodd"
+	/>
+</svg>

src/lib/components/icons/MagnifyingGlass.svelte

@@ -0,0 +1,19 @@
+<script lang="ts">
+	export let className = 'size-4';
+	export let strokeWidth = '2';
+</script>
+
+<svg
+	xmlns="http://www.w3.org/2000/svg"
+	fill="none"
+	viewBox="0 0 24 24"
+	stroke-width={strokeWidth}
+	stroke="currentColor"
+	class={className}
+>
+	<path
+		stroke-linecap="round"
+		stroke-linejoin="round"
+		d="m21 21-5.197-5.197m0 0A7.5 7.5 0 1 0 5.196 5.196a7.5 7.5 0 0 0 10.607 10.607Z"
+	/>
+</svg>