shoko/randebu
1
0
Fork 0
Code Issues 6 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

[Refactor] Split conversational.py into Modular Structure #63

New Issue
Closed
opened 2026-04-14 04:15:40 +02:00 by shoko · 0 comments
shoko commented 2026-04-14 04:15:40 +02:00
Owner
Copy Link

Detailed Guideline for Refactoring conversational.py

Objective

Split conversational.py (2271 lines) into modular files to improve:

  • Testability
  • Debuggability
  • Reviewability
  • Parallel development

Current Structure

ai_agent/
└── conversational.py (2271 lines)
    ├── TOOL_REGISTRY (constants, ~100 lines)
    ├── get_tool_registry()
    ├── format_tools_list()
    ├── format_general_help()
    ├── format_tool_help()
    ├── format_skill_acknowledgment()
    ├── ... (more formatters ~200 lines)
    └── ConversationalAgent class (~1800 lines)
        ├── __init__
        ├── _is_error_output()
        ├── _handle_slash_command()
        ├── _call_llm()
        ├── _parse_response()
        ├── _execute_tool()
        ├── process_message()
        └── ... (more methods)

Target Structure

ai_agent/
├── __init__.py           # Public exports
├── tools.py              # TOOL_REGISTRY, get_tool_registry()
├── help.py               # All format_* functions
├── client.py             # MiniMax API calls
├── agent.py              # ConversationalAgent class
└── conversational.py     # REMOVED (replaced by above)

Implementation Steps

Phase 1: Create New Files

1.1 Create tools.py

# File: src/backend/app/services/ai_agent/tools.py
"""Tool registry and definitions for the conversational agent."""

from typing import Dict, Any

# Copy TOOL_REGISTRY here
TOOL_REGISTRY: Dict[str, Any] = {...}

def get_tool_registry() -> Dict[str, Any]:
    """Return the tool registry."""
    return TOOL_REGISTRY

def get_tools_by_category(category: str) -> list:
    """Get tools filtered by category."""
    ...

Check: Verify imports still work (no circular imports)

1.2 Create help.py

# File: src/backend/app/services/ai_agent/help.py
"""Help formatters for slash commands and tool documentation."""

from typing import Optional
from .tools import get_tool_registry

def format_tools_list() -> str:
    """Format all available tools as a help message."""
    ...

def format_general_help() -> str:
    """Format general help message."""
    ...

def format_tool_help(tool_name: str) -> str:
    """Format help for a specific tool."""
    ...

def format_skill_acknowledgment(tool_name: str, description: str) -> str:
    """Format acknowledgment when a skill is loaded."""
    ...

Check: Test each function independently

1.3 Create client.py

# File: src/backend/app/services/ai_agent/client.py
"""MiniMax API client for the conversational agent."""

import requests
from typing import Dict, Any, Optional

class MiniMaxClient:
    """Client for MiniMax extended thinking API."""
    
    def __init__(self, api_key: str, model: str = "MiniMax-M2.7"):
        self.api_key = api_key
        self.model = model
        self.endpoint = "https://api.minimax.io/v1/text/chatcompletion_v2"
    
    def chat(
        self,
        messages: list,
        system_prompt: str,
        tools: Optional[list] = None,
        **kwargs
    ) -> Dict[str, Any]:
        """Send a chat request to MiniMax API."""
        ...
    
    def check_connection(self) -> bool:
        """Check if API is reachable."""
        ...

Check: Test API calls with mock data

1.4 Create agent.py

# File: src/backend/app/services/ai_agent/agent.py
"""Conversational trading agent."""

from typing import Dict, Any, Optional
from .client import MiniMaxClient
from .help import format_tools_list, format_general_help
from .tools import get_tool_registry

class ConversationalAgent:
    """Agent that handles trading bot conversations."""
    
    def __init__(self, api_key: str, model: str = "MiniMax-M2.7", bot_id: str = None):
        self.api_key = api_key
        self.model = model
        self.bot_id = bot_id
        
        # Use the client
        self.client = MiniMaxClient(api_key, model)
        
        # Track pending command
        self.pending_command = None
        self.recent_search_results = []
    
    def _is_error_output(self, code: int, output: str) -> bool:
        """Check if command output contains an error."""
        ...
    
    def _handle_slash_command(self, user_message: str) -> Dict[str, Any]:
        """Handle slash command help requests."""
        # Use help.py functions
        ...
    
    def process_message(self, user_message: str) -> Dict[str, Any]:
        """Process a user message and return response."""
        ...

Check: Import and use help.py and client.py

1.5 Create __init__.py

# File: src/backend/app/services/ai_agent/__init__.py
"""AI Agent module for conversational trading."""

from .agent import ConversationalAgent
from .client import MiniMaxClient
from .tools import get_tool_registry, TOOL_REGISTRY
from .help import (
    format_tools_list,
    format_general_help,
    format_tool_help,
)

__all__ = [
    "ConversationalAgent",
    "MiniMaxClient",
    "get_tool_registry",
    "TOOL_REGISTRY",
    "format_tools_list",
    "format_general_help",
    "format_tool_help",
]

Phase 2: Update Import Sites

Find all files that import from conversational.py:

# Find imports
grep -r "from.*conversational import" src/backend/
grep -r "import.*conversational" src/backend/

Expected files to update (verify by searching):

  • api/ave.py
  • api/config.py
  • services/backtest/engine.py (if used there)
  • Any other file using ConversationalAgent

Update imports to:

# Old
from ...services.ai_agent.conversational import ConversationalAgent

# New
from ...services.ai_agent import ConversationalAgent

Phase 3: Delete Old File

After all imports updated:

rm src/backend/app/services/ai_agent/conversational.py

What to Check/Verify

Per Module

Module Check How
tools.py No circular imports Run python -c "from app.services.ai_agent import tools"
help.py Functions return strings Unit test each function
client.py API calls work Test with mock API key
agent.py Uses other modules Import and instantiate
__init__.py Exports correct from app.services.ai_agent import *

Integration

Check How
All imports work Run app, check no import errors
Slash commands work Send /help, verify response
Agent processes messages Test agent.process_message("hello")
Tools execute Run a tool via agent

How to Test

Unit Tests

# tests/test_ai_agent/test_tools.py
def test_get_tool_registry():
    registry = get_tool_registry()
    assert "randebu" in registry
    assert "ave" in registry

def test_get_tools_by_category():
    tools = get_tools_by_category("Randebu Built-in")
    assert len(tools) > 0

# tests/test_ai_agent/test_help.py
def test_format_tools_list():
    result = format_tools_list()
    assert "/backtest" in result
    assert "/search" in result

def test_format_tool_help():
    result = format_tool_help("backtest")
    assert "description" in result.lower()

# tests/test_ai_agent/test_client.py
def test_client_initialization():
    client = MiniMaxClient("test-key")
    assert client.model == "MiniMax-M2.7"

# tests/test_ai_agent/test_agent.py
def test_agent_initialization():
    agent = ConversationalAgent("test-key")
    assert agent.bot_id is None

Integration Tests

# tests/test_ai_agent/test_integration.py
def test_slash_command_help():
    agent = ConversationalAgent("test-key")
    result = agent.process_message("/help")
    assert result["success"] is True
    assert "/backtest" in result["response"]

def test_slash_command_list():
    agent = ConversationalAgent("test-key")
    result = agent.process_message("/")
    assert result["success"] is True

How to Debug

If Import Fails

# Check for circular imports
python -c "from app.services.ai_agent import ConversationalAgent"

# Check path
python -c "import sys; sys.path.insert(0, 'src/backend'); from app.services.ai_agent import *"

If Agent Not Working

# Debug: Check client works
from app.services.ai_agent import MiniMaxClient
client = MiniMaxClient("your-key")
# Test with print statements

# Debug: Check help functions
from app.services.ai_agent import format_general_help
print(format_general_help())

If Slash Commands Fail

# Debug: Test slash command directly
from app.services.ai_agent import ConversationalAgent
agent = ConversationalAgent("your-key")
result = agent._handle_slash_command("/help")
print(result)

What NOT to Change (Invariants)

Item Must Stay Same
ConversationalAgent class API __init__, process_message() signature unchanged
Tool registry structure Keys, format unchanged
Slash command responses /, /help, /tool output same as before
Response format process_message() returns same dict structure
MiniMax API behavior Same endpoint, same params

If tests fail because response format changed, that's a bug — fix the refactor, not the tests.

Rollback Plan

If refactor breaks something:

  1. Don't delete conversational.py immediately — keep it until refactor verified
  2. If issues: git checkout conversational.py
  3. Compare: What changed between old and new?
  4. Fix module by module, not all at once

Acceptance Criteria

  • tools.py exists with TOOL_REGISTRY and get_tool_registry()
  • help.py exists with all format_* functions
  • client.py exists with MiniMaxClient class
  • agent.py exists with ConversationalAgent class
  • __init__.py exports everything correctly
  • Old conversational.py deleted
  • All import sites updated
  • No circular imports
  • Slash commands work (/, /help, /search, etc.)
  • Agent can process messages
  • Unit tests pass
  • Integration tests pass

Files to Modify/Create

Action File
CREATE ai_agent/tools.py
CREATE ai_agent/help.py
CREATE ai_agent/client.py
CREATE ai_agent/agent.py
CREATE ai_agent/__init__.py
UPDATE All files importing conversational.py
DELETE ai_agent/conversational.py

Dependencies

  • None (pure refactor)

Priority

HIGH — Should be done BEFORE implementing #59 (Backend Chat System)

Notes for Developer

  1. Work incrementally — Don't move everything at once
  2. Test after each module — Don't wait until end to test
  3. Keep response format identical — Users shouldn't notice difference
  4. Check invariants — Don't change behavior, only structure
  5. Use type hints — Helps with understanding
  6. Add docstrings — This is the chance to improve documentation
# Detailed Guideline for Refactoring conversational.py ## Objective Split `conversational.py` (2271 lines) into modular files to improve: - Testability - Debuggability - Reviewability - Parallel development --- ## Current Structure ``` ai_agent/ └── conversational.py (2271 lines) ├── TOOL_REGISTRY (constants, ~100 lines) ├── get_tool_registry() ├── format_tools_list() ├── format_general_help() ├── format_tool_help() ├── format_skill_acknowledgment() ├── ... (more formatters ~200 lines) └── ConversationalAgent class (~1800 lines) ├── __init__ ├── _is_error_output() ├── _handle_slash_command() ├── _call_llm() ├── _parse_response() ├── _execute_tool() ├── process_message() └── ... (more methods) ``` --- ## Target Structure ``` ai_agent/ ├── __init__.py # Public exports ├── tools.py # TOOL_REGISTRY, get_tool_registry() ├── help.py # All format_* functions ├── client.py # MiniMax API calls ├── agent.py # ConversationalAgent class └── conversational.py # REMOVED (replaced by above) ``` --- ## Implementation Steps ### Phase 1: Create New Files #### 1.1 Create `tools.py` ```python # File: src/backend/app/services/ai_agent/tools.py """Tool registry and definitions for the conversational agent.""" from typing import Dict, Any # Copy TOOL_REGISTRY here TOOL_REGISTRY: Dict[str, Any] = {...} def get_tool_registry() -> Dict[str, Any]: """Return the tool registry.""" return TOOL_REGISTRY def get_tools_by_category(category: str) -> list: """Get tools filtered by category.""" ... ``` **Check:** Verify imports still work (no circular imports) --- #### 1.2 Create `help.py` ```python # File: src/backend/app/services/ai_agent/help.py """Help formatters for slash commands and tool documentation.""" from typing import Optional from .tools import get_tool_registry def format_tools_list() -> str: """Format all available tools as a help message.""" ... def format_general_help() -> str: """Format general help message.""" ... def format_tool_help(tool_name: str) -> str: """Format help for a specific tool.""" ... def format_skill_acknowledgment(tool_name: str, description: str) -> str: """Format acknowledgment when a skill is loaded.""" ... ``` **Check:** Test each function independently --- #### 1.3 Create `client.py` ```python # File: src/backend/app/services/ai_agent/client.py """MiniMax API client for the conversational agent.""" import requests from typing import Dict, Any, Optional class MiniMaxClient: """Client for MiniMax extended thinking API.""" def __init__(self, api_key: str, model: str = "MiniMax-M2.7"): self.api_key = api_key self.model = model self.endpoint = "https://api.minimax.io/v1/text/chatcompletion_v2" def chat( self, messages: list, system_prompt: str, tools: Optional[list] = None, **kwargs ) -> Dict[str, Any]: """Send a chat request to MiniMax API.""" ... def check_connection(self) -> bool: """Check if API is reachable.""" ... ``` **Check:** Test API calls with mock data --- #### 1.4 Create `agent.py` ```python # File: src/backend/app/services/ai_agent/agent.py """Conversational trading agent.""" from typing import Dict, Any, Optional from .client import MiniMaxClient from .help import format_tools_list, format_general_help from .tools import get_tool_registry class ConversationalAgent: """Agent that handles trading bot conversations.""" def __init__(self, api_key: str, model: str = "MiniMax-M2.7", bot_id: str = None): self.api_key = api_key self.model = model self.bot_id = bot_id # Use the client self.client = MiniMaxClient(api_key, model) # Track pending command self.pending_command = None self.recent_search_results = [] def _is_error_output(self, code: int, output: str) -> bool: """Check if command output contains an error.""" ... def _handle_slash_command(self, user_message: str) -> Dict[str, Any]: """Handle slash command help requests.""" # Use help.py functions ... def process_message(self, user_message: str) -> Dict[str, Any]: """Process a user message and return response.""" ... ``` **Check:** Import and use help.py and client.py --- #### 1.5 Create `__init__.py` ```python # File: src/backend/app/services/ai_agent/__init__.py """AI Agent module for conversational trading.""" from .agent import ConversationalAgent from .client import MiniMaxClient from .tools import get_tool_registry, TOOL_REGISTRY from .help import ( format_tools_list, format_general_help, format_tool_help, ) __all__ = [ "ConversationalAgent", "MiniMaxClient", "get_tool_registry", "TOOL_REGISTRY", "format_tools_list", "format_general_help", "format_tool_help", ] ``` --- ### Phase 2: Update Import Sites Find all files that import from conversational.py: ```bash # Find imports grep -r "from.*conversational import" src/backend/ grep -r "import.*conversational" src/backend/ ``` Expected files to update (verify by searching): - `api/ave.py` - `api/config.py` - `services/backtest/engine.py` (if used there) - Any other file using `ConversationalAgent` Update imports to: ```python # Old from ...services.ai_agent.conversational import ConversationalAgent # New from ...services.ai_agent import ConversationalAgent ``` --- ### Phase 3: Delete Old File After all imports updated: ```bash rm src/backend/app/services/ai_agent/conversational.py ``` --- ## What to Check/Verify ### Per Module | Module | Check | How | |--------|-------|-----| | `tools.py` | No circular imports | Run `python -c "from app.services.ai_agent import tools"` | | `help.py` | Functions return strings | Unit test each function | | `client.py` | API calls work | Test with mock API key | | `agent.py` | Uses other modules | Import and instantiate | | `__init__.py` | Exports correct | `from app.services.ai_agent import *` | ### Integration | Check | How | |-------|------| | All imports work | Run app, check no import errors | | Slash commands work | Send `/help`, verify response | | Agent processes messages | Test `agent.process_message("hello")` | | Tools execute | Run a tool via agent | --- ## How to Test ### Unit Tests ```python # tests/test_ai_agent/test_tools.py def test_get_tool_registry(): registry = get_tool_registry() assert "randebu" in registry assert "ave" in registry def test_get_tools_by_category(): tools = get_tools_by_category("Randebu Built-in") assert len(tools) > 0 # tests/test_ai_agent/test_help.py def test_format_tools_list(): result = format_tools_list() assert "/backtest" in result assert "/search" in result def test_format_tool_help(): result = format_tool_help("backtest") assert "description" in result.lower() # tests/test_ai_agent/test_client.py def test_client_initialization(): client = MiniMaxClient("test-key") assert client.model == "MiniMax-M2.7" # tests/test_ai_agent/test_agent.py def test_agent_initialization(): agent = ConversationalAgent("test-key") assert agent.bot_id is None ``` ### Integration Tests ```python # tests/test_ai_agent/test_integration.py def test_slash_command_help(): agent = ConversationalAgent("test-key") result = agent.process_message("/help") assert result["success"] is True assert "/backtest" in result["response"] def test_slash_command_list(): agent = ConversationalAgent("test-key") result = agent.process_message("/") assert result["success"] is True ``` --- ## How to Debug ### If Import Fails ```bash # Check for circular imports python -c "from app.services.ai_agent import ConversationalAgent" # Check path python -c "import sys; sys.path.insert(0, 'src/backend'); from app.services.ai_agent import *" ``` ### If Agent Not Working ```python # Debug: Check client works from app.services.ai_agent import MiniMaxClient client = MiniMaxClient("your-key") # Test with print statements # Debug: Check help functions from app.services.ai_agent import format_general_help print(format_general_help()) ``` ### If Slash Commands Fail ```python # Debug: Test slash command directly from app.services.ai_agent import ConversationalAgent agent = ConversationalAgent("your-key") result = agent._handle_slash_command("/help") print(result) ``` --- ## What NOT to Change (Invariants) | Item | Must Stay Same | |------|----------------| | `ConversationalAgent` class API | `__init__`, `process_message()` signature unchanged | | Tool registry structure | Keys, format unchanged | | Slash command responses | `/`, `/help`, `/tool` output same as before | | Response format | `process_message()` returns same dict structure | | MiniMax API behavior | Same endpoint, same params | **If tests fail because response format changed, that's a bug — fix the refactor, not the tests.** --- ## Rollback Plan If refactor breaks something: 1. **Don't delete** conversational.py immediately — keep it until refactor verified 2. If issues: `git checkout conversational.py` 3. Compare: What changed between old and new? 4. Fix module by module, not all at once --- ## Acceptance Criteria - [ ] `tools.py` exists with `TOOL_REGISTRY` and `get_tool_registry()` - [ ] `help.py` exists with all `format_*` functions - [ ] `client.py` exists with `MiniMaxClient` class - [ ] `agent.py` exists with `ConversationalAgent` class - [ ] `__init__.py` exports everything correctly - [ ] Old `conversational.py` deleted - [ ] All import sites updated - [ ] No circular imports - [ ] Slash commands work (`/`, `/help`, `/search`, etc.) - [ ] Agent can process messages - [ ] Unit tests pass - [ ] Integration tests pass --- ## Files to Modify/Create | Action | File | |--------|------| | CREATE | `ai_agent/tools.py` | | CREATE | `ai_agent/help.py` | | CREATE | `ai_agent/client.py` | | CREATE | `ai_agent/agent.py` | | CREATE | `ai_agent/__init__.py` | | UPDATE | All files importing `conversational.py` | | DELETE | `ai_agent/conversational.py` | --- ## Dependencies - None (pure refactor) --- ## Priority **HIGH** — Should be done BEFORE implementing #59 (Backend Chat System) --- ## Notes for Developer 1. **Work incrementally** — Don't move everything at once 2. **Test after each module** — Don't wait until end to test 3. **Keep response format identical** — Users shouldn't notice difference 4. **Check invariants** — Don't change behavior, only structure 5. **Use type hints** — Helps with understanding 6. **Add docstrings** — This is the chance to improve documentation
shoko closed this issue 2026-04-14 05:57:16 +02:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
AI/ML
AI or ML related tasks
 backend
Backend related tasks
 enhancement
Feature or enhancement
 frontend
Frontend related tasks
 infrastructure
Infrastructure or deployment
 meta
Meta issue for tracking
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
shoko
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: shoko/randebu#63
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?