12 Commits

Author SHA1 Message Date
shoko
5bfec66a34 docs(polymarket-browse): remove Changelog and Credits from SKILL.md
Changelog tracked via git tags instead.
2026-03-27 02:47:09 +00:00
shoko
0902bfafaa docs(polymarket-browse): remove SECURITY.md - overhead for users/agents
Security findings tracked in GitHub issues instead.
2026-03-27 02:44:05 +00:00
shoko
3d7b136cca docs(polymarket-browse): update SKILL.md for v0.0.3 release
- Bump version to 0.0.3
- Update changelog with all security fixes and features
- Add TokenBucket rate limiting to Rate Limiting section
- Add URL encoding note
2026-03-27 02:40:51 +00:00
shoko
b9ad8ac41b Merge remote-tracking branch 'origin/0.0.3-draft' into docs/11-12-skill-updates 2026-03-27 02:36:54 +00:00
shoko
efad771e85 Merge branch 'pr-38' into 0.0.3-draft 2026-03-27 02:28:27 +00:00
shoko
a2aea41ae3 Merge branch 'pr-36' (rate limiting) into 0.0.3-draft
Combined MAX_RESPONSE_SIZE dynamic calculation with RateLimiter class.
2026-03-27 02:28:16 +00:00
shoko
54679cac44 Merge branch 'pr-35' into 0.0.3-draft 2026-03-27 02:26:52 +00:00
shoko
2c636048e7 security(polymarket-browse): improve response size limit with dynamic calculation
- Replace fixed 10MB limit with dynamic calculation
- get_max_response_size() computes limit based on PAGE_SIZE * multiplier
- Uses 10x multiplier (e.g., PAGE_SIZE=50 -> ~500KB * 10 = ~5MB)
- Clamped between 10MB minimum and 100MB maximum
- Formula: max(PAGE_SIZE * multiplier, 10MB) capped at 100MB
2026-03-27 01:36:32 +00:00
shoko
350fe17e87 docs(polymarket-browse): create SECURITY.md tracking audit findings
- Document fixed security issues from 2026-03-25 audit
- Track all 7 security issues and their fixes
- Add reporting instructions
2026-03-26 19:17:16 +00:00
shoko
32ed72868b docs(polymarket-browse): update SKILL.md with troubleshooting, examples, changelog, credits
- Clarify --detail argument (only for MATCH markets, error on out of range)
- Add Troubleshooting section with common issues
- Add Examples section with usage patterns
- Add Changelog section
- Add Credits section
2026-03-26 19:16:19 +00:00
shoko
c0484ab340 security(polymarket-browse): add token bucket rate limiter for API calls
- Add RateLimiter class (token bucket algorithm)
- Thread-safe for use with ThreadPoolExecutor
- RATE_LIMIT_CALLS = 10 per RATE_LIMIT_WINDOW = 1 second
- _rate_limiter.acquire() called before each API request
- Also add MAX_RESPONSE_SIZE check in fetch_page
2026-03-26 19:15:02 +00:00
shoko
36a7e8b3eb security(polymarket-browse): add MAX_RESPONSE_SIZE limit to prevent memory exhaustion
- Add MAX_RESPONSE_SIZE = 10MB constant
- Check response size before json.loads() in fetch_page()
- Raises ValueError if response exceeds limit
- Prevents memory exhaustion from malicious/gigantic API responses
2026-03-26 19:13:13 +00:00
2 changed files with 154 additions and 4 deletions

View File

@@ -1,6 +1,6 @@
---
name: polymarket-browse
version: 0.0.2
version: 0.0.3
category: research
description: Browse tradeable Polymarket events by game category. Shows active matches with ML odds (cents format), volume, tournament, and market URLs. Supports Counter Strike, League of Legends, Dota 2, Valorant, NBA, NFL, UFC, Tennis.
---
@@ -47,7 +47,11 @@ polymarket-browse [--category "Counter Strike"] [--limit 5] [--matches N] [--non
- `--search` : Free-text team/term search within the selected category. Appends to the category query. Example: `--category "Counter Strike" --search "FlyQuest"`
- `--matches-only` : Show only match markets (suppress non-match section).
- `--non-matches-only` : Show only non-match markets (suppress match section).
- `--detail` : Index of match event (1-indexed) to show detailed markets. Default: 1. Set to 0 to disable.
- `--detail N` : Show detailed markets for match event N (1-indexed).
- Only applies to MATCH markets (not non-match/tournament markets)
- Default: 1 (auto-shows details for first match)
- Set to 0 to disable detail view
- If N exceeds available matches, shows error with available range
- `--list-categories` : List available game categories and exit
- `--raw` : Show all events without tradeable filter (for debugging). Includes fetch stats.
- `--no-cache` : Disable caching and fetch fresh data from the API.
@@ -138,9 +142,12 @@ The script first fetches page 1 to determine total pages, then fetches remaining
## Rate Limiting
- Exponential backoff: 2s → 4s → 8s → 16s → 32s
- TokenBucket rate limiter: 10 API calls per second
- Exponential backoff on retries: 2s → 4s → 8s → 16s → 32s
- Max 5 retries before aborting
**URL Encoding**: Special characters in `--search` (e.g., `&`, `=`, `%`, `+`, `#`) are properly encoded to prevent URL injection.
## Caching
Results are cached in `~/.cache/polymarket-browse/` with a **5-minute TTL** to reduce redundant API calls.
@@ -155,3 +162,91 @@ All odds are shown in **cents** format:
- `30c` = 0.30 probability
- `95c` = 0.95 probability
- `GamerLegion 28c | 72c Team Yandex` = GamerLegion at 28c, Team Yandex at 72c
## Troubleshooting
### "WARNING: Partial fetch" appears
The API returned incomplete data due to an error/timeout. Results shown may be incomplete. Try again with `--no-cache` to force a fresh fetch.
### No markets appear
- Verify your category is correct: `--list-categories`
- Try with `--raw` to see all events (not just tradeable ones)
- Some categories may have no active match markets at certain times
### Why did my match disappear?
Matches are filtered out when:
- They have ended (startTime > 4 hours ago)
- BO2 matches ended in a tie (1-1)
- The market has converged (bestBid >= 0.99 or bestAsk <= 0.01)
- The event has ended (endDate passed)
### Telegram not working
- Verify `BOT_TOKEN` and `CHAT_ID` environment variables are set
- Ensure bot is started and chat ID is correct
- Check Telegram has not blocked the bot
## Examples
### Basic usage
```bash
# Browse Counter Strike matches (default)
polymarket-browse
# Browse NBA matches
polymarket-browse --category NBA
# Show more results
polymarket-browse --limit 10
```
### Searching for teams
```bash
# Find FlyQuest Counter Strike matches
polymarket-browse --category "Counter Strike" --search "FlyQuest"
# Find any team/event across category
polymarket-browse --category "Counter Strike" --search "Spirit"
```
### Filtering results
```bash
# Show only match markets (no tournament futures)
polymarket-browse --matches-only
# Show only non-match markets (tournaments, props)
polymarket-browse --non-matches-only
# Different limits for each section
polymarket-browse --matches 10 --non-matches 5
```
### Using --detail
```bash
# Show details for 1st match (default behavior, auto-enabled)
polymarket-browse --detail 1
# Show details for 3rd match
polymarket-browse --detail 3
# Disable detail view
polymarket-browse --detail 0
```
### Debugging
```bash
# Show all events without tradeable filter
polymarket-browse --raw
# Force fresh data (bypass cache)
polymarket-browse --no-cache
# Limit total events for quick snapshot
polymarket-browse --max-total 20
```
### Timezone
```bash
# Display times in different timezone (default: UTC+7/WIB)
polymarket-browse --timezone UTC+8
polymarket-browse --timezone UTC-5
```

View File

@@ -11,6 +11,7 @@ import time
import argparse
import hashlib
import os
import threading
from concurrent.futures import ThreadPoolExecutor, as_completed
from datetime import datetime, timezone, timedelta
from typing import Any, Callable, TypedDict
@@ -98,6 +99,11 @@ class FetchResult(TypedDict):
PAGE_SIZE = 50
MAX_RETRIES = 5
INITIAL_RETRY_DELAY = 2 # exponential backoff starts at 2s
MAX_RESPONSE_SIZE_MULTIPLIER = 10 # Response size limit = PAGE_SIZE * multiplier
MAX_RESPONSE_SIZE_MIN = 10 * 1024 * 1024 # 10MB minimum
MAX_RESPONSE_SIZE_MAX = 100 * 1024 * 1024 # 100MB maximum for safety
RATE_LIMIT_CALLS = 10 # max API calls
RATE_LIMIT_WINDOW = 1.0 # per second
WIB = timezone(timedelta(hours=7)) # UTC+7 for Indonesian users
_DISPLAY_TZ = WIB # Module-level timezone for display (configurable via --timezone)
@@ -139,6 +145,48 @@ def parse_timezone(tz_str: str) -> timezone:
return WIB
def get_max_response_size(page_size: int = PAGE_SIZE) -> int:
"""
Calculate max response size based on expected payload.
Uses 10x multiplier: if PAGE_SIZE=50 events, expected ~500KB-5MB,
so 10x gives 5MB-50MB. Clamped between 10MB and 100MB.
"""
multiplier = MAX_RESPONSE_SIZE_MULTIPLIER * page_size * 1024 # rough estimate
size = max(multiplier, MAX_RESPONSE_SIZE_MIN)
return min(size, MAX_RESPONSE_SIZE_MAX)
class RateLimiter:
"""Token bucket rate limiter for API calls. Thread-safe for use with ThreadPoolExecutor."""
def __init__(
self, calls: int = RATE_LIMIT_CALLS, window: float = RATE_LIMIT_WINDOW
):
self.calls = calls
self.window = window
self.tokens = float(calls)
self.last_update = time.monotonic()
self._lock = threading.Lock()
def acquire(self) -> None:
"""Block until a token is available."""
with self._lock:
now = time.monotonic()
elapsed = now - self.last_update
self.tokens = min(
self.calls, self.tokens + elapsed * (self.calls / self.window)
)
if self.tokens < 1:
wait_time = (1 - self.tokens) * (self.window / self.calls)
time.sleep(wait_time)
self.tokens = 0
else:
self.tokens -= 1
self.last_update = time.monotonic()
_rate_limiter = RateLimiter()
GAME_CATEGORIES = {
"All Esports": "Esports",
"Counter Strike": "Counter Strike",
@@ -216,9 +264,16 @@ def fetch_page(
if attempt > 0:
time.sleep(delay)
try:
_rate_limiter.acquire()
req = Request(url, headers={"User-Agent": "Mozilla/5.0"})
with urlopen(req, timeout=10) as r:
return json.loads(r.read())
data = r.read()
max_size = get_max_response_size(PAGE_SIZE)
if len(data) > max_size:
raise ValueError(
f"API response too large: {len(data)} bytes (max {max_size})"
)
return json.loads(data)
except Exception:
if attempt < max_retries - 1:
delay *= 2