0. Introduction

The project is open-sourced on GitHub. Here are some fundamental technical explanations.

https://github.com/ouyangyanhuo/ModelChat

1. Architecture Design: A Plugin-Based Project

The project is built with a plugin-based architecture. By leveraging Ncatbot's plugin system, we can cleanly separate the main program from the features.

The advantages of a plugin architecture are clear module boundaries, high dependency isolation, and strong system extensibility. A standard plugin approach significantly decouples the code, making both development and deployment much friendlier.

The project's directory structure is as follows:

ModuleChat/
├── main.py            # Plugin main entry point, responsible for command registration and dispatch logic
├── chat.py            # Model adaptation layer, encapsulating calls to local and cloud models
├── config.yml         # Configuration file, centrally controlling model parameters and enabled options
├── requirements.txt   # Dependency libraries
└── cache/
    └── history.json  # Chat history memory file

chat.py is the core module of the system, while the main program main.py handles receiving and parsing commands, and routes messages to the model module for processing. This is a very developer-friendly structure during development, allowing you to focus on developing and debugging individual functional modules, greatly reducing maintenance complexity.

Configuration items are centralized in config.yml, further enhancing flexibility and adaptability to different environments.

By using a temporary JSON file to record command invocations and replies, and passing this data to the API interface, the large model can gain a degree of short-term memory. However, this approach isn't entirely ideal for the system; I believe a database would be a better solution, but using a database significantly increases system complexity. Therefore, using a JSON file is a good alternative.

2. Plugin Main Program: Command Decoupling and Routing Hub

main.py is the plugin's main entry point. It registers two commands via the register_user_func method: /chat and /clear chat_history, corresponding to the chat function and history clearing, respectively.

Additionally, the main program supports automatic recognition of image messages, extracting the image URL and passing it to the chat_model_instance.recognize_image method to automatically obtain a visual description.

if image_url and self.chat_model.get('enable_vision', True) and not self.chat_model.get('use_local_model'):
    # Use image recognition feature
    image_description = await chat_model_instance.recognize_image(image_url)
    user_input = f"用户发送了一张图片，图片描述是：{image_description}。用户说：{user_input}"
elif image_url and not self.chat_model.get('enable_vision', True):
    # Image recognition is disabled, but check if it's a local model
    if self.chat_model.get('use_local_model'):
        user_input = f"用户发送了一张图片，但用户使用的是本地模型，无法进行图像识别。用户说：{user_input}"
    else:
        user_input = f"用户发送了一张图片，但图像识别功能未开启。用户说：{user_input}"

After obtaining the visual description, it's passed to the language model for output. This is actually a great solution. In the current usage scenario, the need is more about recognizing the image and then analyzing the content, rather than processing the image itself. This approach can significantly reduce API calls, improve cache hit rates, and reduce TOKEN usage, thereby lowering API call costs. Furthermore, you can use a cloud model for recognition and then hand it over to a local model for the answer, further compressing costs.

For error handling, the entire chat logic is wrapped in a try...except block, preventing image decoding failures or API exceptions from crashing the main flow, keeping the plugin robust.

Overall, main.py follows a typical "light controller" pattern. It only coordinates the various components without handling business logic details, giving the entire plugin good engineering readability.

3. Model Adaptation Module: Multi-Model Encapsulation and Semantic Consistency

chat.py is the core logic of the plugin. It handles model calls, chat history memory, image recognition, and other tasks. To be compatible with various model interfaces (such as the OpenAI API and the local Ollama service), a unified encapsulation interface strategy is adopted. This allows external callers to ignore model details and simply use the useCloudModel() or useLocalModel() methods to conduct conversations.

async def useLocalModel(self, msg: BaseMessage, user_input: str):
    """Use local model to process messages"""
    try:
        # Build message list, including history
        messages = self._build_messages(user_input, msg.user_id if hasattr(msg, 'user_id') else None)
        response: ChatResponse = chat(
            model=self.config['model'],
            messages=messages
        )
        reply = response.message.content.strip()

        # Save current conversation to history
        if hasattr(msg, 'user_id'):
            self._update_user_history(msg.user_id, {"role": "user", "content": user_input})
            self._update_user_history(msg.user_id, {"role": "assistant", "content": reply})
    except Exception as e:
        reply = f"请求出错了：{str(e)}"
    return reply

It's worth noting that since the OpenAI interface call and the Ollama call differ slightly, and some models have incomplete parameters, using the OpenAI interface to call cloud models can actually provide a better experience. For example, we can control the model's temperature to make it more imaginative or more grounded, reducing hallucinations.

All user history is stored in the cache/history.json file. This is a persistent storage solution that also offers a degree of traceability. The history is dynamically updated via the _update_user_history method, keeping it within the maximum number of turns set in the configuration file. This approach prevents performance issues caused by overly large contexts, while ensuring the model understands continuous context, improving answer quality, and providing a near-memory capability even when interfacing with APIs.

The class also integrates OpenAI's image recognition model, building multimodal message structures through _build_vision_messages. In the design, I've separated functions like image processing, message construction, exception handling, and model invocation, making it faster to locate issues during development and easier for other developers to read after open-sourcing.

4. Cloud Model Integration (OpenAI): Standardized Encapsulation

Cloud model calls are primarily encapsulated using the official openai library, utilizing the chat.completions.create method to construct context and generate replies. In each call, the _build_messages() method constructs the complete conversation context, adding a system prompt and using the history saved in cache/history.json, enabling multi-turn memory-based conversations.

def _build_messages(self, user_input: str, user_id: str = None):
    """Build message list"""
    messages = []
        
    # Add system prompt
    system_prompt = self.config.get('system_prompt', "你是一名聊天陪伴机器人")
    messages.append({"role": "system", "content": system_prompt})

    if user_id:
        history = self._get_user_history(user_id)
        messages.extend(history)
        
    # Add current user input
    messages.append({"role": "user", "content": user_input})
    return messages

The calling logic encapsulates the temperature parameter, supporting flexible control over the randomness of model output through the configuration file.

When encountering bug reports, we uniformly use return to feed errors back to the user side. This reduces a lot of error-handling development and provides clearer feedback for common issues caused by configuration mistakes. In essence, it's a unified model + rule-based handling approach to report runtime problems.

if "401" in str(fallback_error) or "Unauthorized" in str(fallback_error):
    raise Exception("模型API认证失败，请检查配置文件")
raise Exception(f"图像识别出错: {str(e)}, 备用方法也失败: {str(fallback_error)}")

After returning the result, the current Q&A is synchronized to the user's history cache and saved to a local file, ensuring the context can be retrieved correctly in the next round. This reduces memory dependency, enhances cache hit rates, and provides a basis for future debugging and behavior reproduction.

5. Local Model Invocation (Ollama): Lightweight Inference and Unified Interface

Local model calls are completed via ollama.chat(), reusing the _build_messages() context-building logic to ensure the calling logic is consistent with the cloud and maintain interface consistency.

The advantages of this local inference mechanism are clear: it allows the use of intelligent chat features even in offline or private deployment environments, greatly enhancing the plugin's deployment flexibility and security. Even in privacy-sensitive scenarios, local deployment and operation are possible.

By design, the local and cloud invocation interfaces are kept consistent (both encapsulated as use*Model()), so external callers don't need to determine the model source, reducing complexity. Additionally, it implements the same history update and exception capture mechanisms, giving the local model the same feature completeness and stability as the cloud version.

6. Image Recognition Logic: Semantic Enhancement Strategy for Multimodal Input

Image recognition is a major highlight of this plugin. The plugin supports recognizing image messages and processing them through OpenAI's vision model. The entire flow is as follows:

• 1. Extract the URL from the image message;

     PYTHON

     for segment in msg.message:
        if isinstance(segment, dict) and segment.get("type") == "image":
            image_url = segment.get("data", {}).get("url")
            break
     

• 2. Fetch the image content via an HTTP request and encode it in Base64;
• 3. Construct the vision input format (including image_url and text prompt);

response = requests.get(image_url)
response.raise_for_status()
return base64.b64encode(response.content).decode('utf-8')

• 4. Call the vision model to complete the image description;

# Get and encode the image
image_data = self._encode_image_from_url(image_url)
            
# Build messages
messages = self._build_vision_messages(image_data, prompt)
            
# Call the vision model
response = self.vision_client.chat.completions.create(
    model=self.config.get('vision_model'),
    messages=messages,
    temperature=self.config.get('model_temperature', 0.6),
    stream=False,
    max_tokens=2048
)

• 5. Concatenate the image description into the user input to enhance the semantic completeness of the context.

This mechanism effectively addresses the information asymmetry problem in mixed image-text input scenarios. Additionally, through tiered call management, you can use cloud high-computing power only for complex problems and then hand off the simplified problem to the local model, greatly reducing TOKEN usage.

For exception handling, we've designed a two-level degradation strategy: if the primary call fails, we attempt a pure text fallback prompt; if that also fails, we prompt the user to check the API key or model status. This fault-tolerant design ensures the plugin can maintain service continuity even during partial failures.

7. Chat History System: Memory Window Control

Chat history is stored in the cache/history.json file, managed on a per-user basis. This design allows the system to serve multiple users simultaneously while maintaining an independent context for each user. Through the _get_user_history and _update_user_history methods, the plugin automatically injects historical information into each conversation round, achieving a "memory-like" Q&A experience.

We've implemented a window limit on history length (default 10 rounds) to control context size and avoid excessive processing pressure on the model and over-consumption of TOKENs. Cache updates are synchronous write operations, ensuring no information is lost during system crashes, power outages, or other abnormal situations.

async def clear_user_history(self, user_id: str):
    """Clear the history for a specified user"""
    user_id = str(user_id)
    if user_id in self.history:
        del self.history[user_id]
        self._save_history()
        reply = "已清空聊天记录"
    else:
        reply = "没有找到用户的聊天记录"
    return reply

Additionally, the command /clear chat_history is supported to actively clear user history, providing convenience for privacy or starting a new conversation. This mechanism gives the plugin both persistence and user control.

8. DEBUG & LOG

Setting breakpoints and using print flags during debugging are good testing habits. I also picked up a trick from WeChat development: print("FUCK"). When the system occasionally crashes during long-term operation, you can output a specific character in the log. When reviewing logs to locate issues, you can directly search for that string to quickly pinpoint the problem. FUCK is certainly an interesting way to do it.