Add parallel capacity test tool for Hermes/OpenCode

This tool tests the practical limits of parallel agent execution
by spawning N concurrent opencode run tasks and measuring:
- Response time
- CPU and memory usage
- Success/failure rates

Includes both bash (run_test.sh) and Python (parallel_capacity_test.py)
implementations with full metrics collection and reporting.

Fixes #3
This commit is contained in:
shokollm
2026-03-27 10:29:34 +00:00
parent 3cf7380417
commit 74424c1f82
3 changed files with 753 additions and 0 deletions

View File

@@ -0,0 +1,74 @@
# Parallel Capacity Test Tool
Tests the practical limits of parallel agent execution for Hermes/OpenCode.
## Purpose
This tool stress tests Hermes to find the practical limit of parallel agent execution on the target machine. It:
- Spawns N concurrent `opencode run` agents
- Measures CPU, memory, and response time
- Ramps up from 1 to higher agent counts
- Identifies failure points and performance degradation
## Files
- `run_test.sh` - Bash script for running tests
- `parallel_capacity_test.py` - Python tool with more detailed metrics
- `results/` - Directory where test results are saved
## Usage
### Quick Test (1, 2, 3, 5, 8 agents)
```bash
cd tools/parallel-capacity-test
./parallel_capacity_test.py --quick
```
### Full Test Suite
```bash
./parallel_capacity_test.py --agents 15 --timeout 120
```
### Bash Script Usage
```bash
./run_test.sh quick # Quick test
./run_test.sh full # Full test up to MAX_AGENTS
```
## Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| MAX_AGENTS | 15 | Maximum number of agents to test |
| STEP | 1 | Step size for agent increment |
| TASK_TIMEOUT | 120 | Timeout for each agent task |
## Metrics Collected
- **Response Time** - Time from agent launch to completion
- **CPU Usage** - System-wide CPU utilization percentage
- **Memory Usage** - System-wide memory utilization percentage
- **Success Rate** - Percentage of agents completing successfully
- **Process Count** - Number of opencode processes running
## Expected Behavior
Based on the Hermes architecture:
| Agent Count | Expected Performance |
|-------------|---------------------|
| 1-3 | Optimal - safe for production |
| 4-6 | Good - monitor closely |
| 7-10 | Degraded - not recommended |
| 10+ | Poor - avoid without significant resources |
## Output Files
- `results_YYYYMMDD_HHMMSS.json` - Complete raw results
- `summary_YYYYMMDD_HHMMSS.csv` - CSV summary of metrics
- `report_YYYYMMDD_HHMMSS.md` - Markdown analysis report
EOF; __hermes_rc=$?; printf '__HERMES_FENCE_a9f7b3__'; exit $__hermes_rc