74424c1f82
This tool tests the practical limits of parallel agent execution by spawning N concurrent opencode run tasks and measuring: - Response time - CPU and memory usage - Success/failure rates Includes both bash (run_test.sh) and Python (parallel_capacity_test.py) implementations with full metrics collection and reporting. Fixes #3
75 lines
2.0 KiB
Markdown
75 lines
2.0 KiB
Markdown
# Parallel Capacity Test Tool
|
|
|
|
Tests the practical limits of parallel agent execution for Hermes/OpenCode.
|
|
|
|
## Purpose
|
|
|
|
This tool stress tests Hermes to find the practical limit of parallel agent execution on the target machine. It:
|
|
|
|
- Spawns N concurrent `opencode run` agents
|
|
- Measures CPU, memory, and response time
|
|
- Ramps up from 1 to higher agent counts
|
|
- Identifies failure points and performance degradation
|
|
|
|
## Files
|
|
|
|
- `run_test.sh` - Bash script for running tests
|
|
- `parallel_capacity_test.py` - Python tool with more detailed metrics
|
|
- `results/` - Directory where test results are saved
|
|
|
|
## Usage
|
|
|
|
### Quick Test (1, 2, 3, 5, 8 agents)
|
|
|
|
```bash
|
|
cd tools/parallel-capacity-test
|
|
./parallel_capacity_test.py --quick
|
|
```
|
|
|
|
### Full Test Suite
|
|
|
|
```bash
|
|
./parallel_capacity_test.py --agents 15 --timeout 120
|
|
```
|
|
|
|
### Bash Script Usage
|
|
|
|
```bash
|
|
./run_test.sh quick # Quick test
|
|
./run_test.sh full # Full test up to MAX_AGENTS
|
|
```
|
|
|
|
## Environment Variables
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| MAX_AGENTS | 15 | Maximum number of agents to test |
|
|
| STEP | 1 | Step size for agent increment |
|
|
| TASK_TIMEOUT | 120 | Timeout for each agent task |
|
|
|
|
## Metrics Collected
|
|
|
|
- **Response Time** - Time from agent launch to completion
|
|
- **CPU Usage** - System-wide CPU utilization percentage
|
|
- **Memory Usage** - System-wide memory utilization percentage
|
|
- **Success Rate** - Percentage of agents completing successfully
|
|
- **Process Count** - Number of opencode processes running
|
|
|
|
## Expected Behavior
|
|
|
|
Based on the Hermes architecture:
|
|
|
|
| Agent Count | Expected Performance |
|
|
|-------------|---------------------|
|
|
| 1-3 | Optimal - safe for production |
|
|
| 4-6 | Good - monitor closely |
|
|
| 7-10 | Degraded - not recommended |
|
|
| 10+ | Poor - avoid without significant resources |
|
|
|
|
## Output Files
|
|
|
|
- `results_YYYYMMDD_HHMMSS.json` - Complete raw results
|
|
- `summary_YYYYMMDD_HHMMSS.csv` - CSV summary of metrics
|
|
- `report_YYYYMMDD_HHMMSS.md` - Markdown analysis report
|
|
EOF; __hermes_rc=$?; printf '__HERMES_FENCE_a9f7b3__'; exit $__hermes_rc
|