feat(queue): add queue system with background daemon #140

Merged
shoko merged 2 commits from feat/queue-daemon into main 2026-04-05 06:49:13 +02:00
Showing only changes of commit 379d53cedc - Show all commits

View File

@@ -49,6 +49,9 @@ A default config file is created during `kugetsu init` with commented examples:
| `MAX_CONCURRENT_AGENTS` | 3 | Maximum number of concurrent dev agents |
| `KUGETSU_TEMP_DIR` | `~/.local/share/opencode/tool-output` | Temp directory for subagent tool output (useful in headless environments where /tmp is restricted) |
| `KUGETSU_VERBOSITY` | `default` | PM agent verbosity level: `verbose`, `default`, or `quiet` |
| `QUEUE_DAEMON_INTERVAL_MINUTES` | 5 | How often daemon polls queue (in minutes) |
| `QUEUE_DAEMON_BATCH_SIZE` | 2 | How many tasks daemon picks per poll |
| `QUEUE_CLEANUP_AGE_DAYS` | 7 | Auto-cleanup completed/error items older than N days |
### Environment Variables for Agents
@@ -111,6 +114,10 @@ Each issue session gets its own git worktree to prevent conflicts:
├── worktrees/
│ ├── github.com-shoko-kugetsu-14/ # Isolated workdir for issue #14
│ └── github.com-shoko-kugetsu-15/ # Isolated workdir for issue #15
├── queue/
│ ├── items/ # Queue item JSON files
│ ├── daemon.pid # Daemon process ID
│ └── daemon.log # Daemon log output
└── index.json # Maps session IDs and issue refs to session files
```
@@ -258,16 +265,17 @@ kugetsu destroy --base -y
### kugetsu delegate `<message>`
Send a message to the PM agent for task coordination (fire-and-forget):
Send a message to the PM agent for task coordination via queue:
```bash
kugetsu delegate "work on issue #14"
kugetsu delegate "review PR #92"
```
- Non-blocking: returns immediately, runs in background
- PM agent processes the message asynchronously
- Uses `KUGETSU_VERBOSITY` env var to control PM agent output verbosity
- Log output stored in `~/.kugetsu/logs/delegate-<timestamp>.log`
- **Always enqueues** (fire-and-forget): returns immediately
- Queue daemon polls queue and invokes PM when slots available
- Tasks are processed FIFO (first-in-first-out)
- Use `kugetsu queue list` to see pending tasks
- Use `kugetsu queue-daemon logs` to debug queue processing
### kugetsu logs [n]
@@ -328,35 +336,79 @@ kugetsu server default github # Set default server
kugetsu server get github # Get server URL
```
### kugetsu queue <list|enqueue|dequeue|clear>
### kugetsu queue <list|stats|clear>
Manage task queue for autonomous PM operation:
```bash
kugetsu queue list # Show queued tasks
kugetsu queue enqueue "task" # Add task to queue
kugetsu queue dequeue # Remove next task from queue
kugetsu queue clear # Clear all queued tasks
kugetsu queue list # Show queued tasks with status
kugetsu queue stats # Show queue statistics (total, pending, notified, completed, error)
kugetsu queue clear # Clean up old completed/error items
kugetsu queue enqueue <issue-ref> <message> # Manually enqueue a task
```
- Queue stored in `~/.kugetsu/queue.json`
**Queue Item States:**
- `pending` - Waiting in queue, daemon can pick up
- `notified` - PM agent has picked up the task
- `completed` - Dev agent finished, PR created
- `error` - Timeout or failure
### kugetsu queue-daemon <start|stop|restart|status|logs>
Manage the queue daemon background process:
```bash
kugetsu queue-daemon start # Start daemon in background
kugetsu queue-daemon stop # Stop daemon
kugetsu queue-daemon restart # Restart daemon
kugetsu queue-daemon status # Check if daemon is running
kugetsu queue-daemon logs # Show recent daemon logs
```
**Daemon Behavior:**
1. Runs at configurable interval (default: 5 minutes)
2. Checks if active agents < MAX_CONCURRENT_AGENTS
3. Picks 1-N pending items (configurable batch size)
4. Forks PM session for each picked item
5. PM decides whether to use `start` or `continue`
**Queue Directory:**
```
~/.kugetsu/queue/
├── items/ # Queue item JSON files
│ ├── q_1234567890.json # One file per queued task
│ └── q_1234567891.json
├── daemon.pid # Daemon process ID
├── daemon.lock # Daemon lock file
└── daemon.log # Daemon log output
```
## Workflow Example
### First-time Setup
```bash
# First-time setup (requires TTY)
# Initialize kugetsu (requires TTY)
kugetsu init
# Creates: base session + pm-agent session
# Start work on issue
kugetsu start github.com/shoko/kugetsu#14 "implement feature X"
# Creates: worktree at ~/.kugetsu/worktrees/github.com-shoko-kugetsu-14/
# Start the queue daemon (for autonomous operation)
kugetsu queue-daemon start
```
# Continue later
### Normal Workflow
```bash
# Enqueue tasks via delegate - agents will process them automatically
kugetsu delegate "work on issue #14"
kugetsu delegate "review PR #92"
# Check queue status
kugetsu queue list # See pending tasks
kugetsu queue stats # See statistics
# Debug queue daemon
kugetsu queue-daemon status # Is daemon running?
kugetsu queue-daemon logs # See daemon logs
# Continue work on existing issue
kugetsu continue github.com/shoko/kugetsu#14 "add tests"
# Continue again
kugetsu continue github.com/shoko/kugetsu#14 "fix failing test"
# List all sessions
kugetsu list
@@ -367,6 +419,21 @@ kugetsu prune --force
kugetsu destroy github.com/shoko/kugetsu#14
```
### Queue Daemon Management
```bash
# Check if daemon is running
kugetsu queue-daemon status
# View daemon logs for debugging
kugetsu queue-daemon logs
# Restart daemon if needed
kugetsu queue-daemon restart
# Stop daemon
kugetsu queue-daemon stop
```
## Headless Operation
This design solves the headless CLI limitation discovered in Issue #14: