Add parallel fetching, caching, and max_total parameter #26

shoko · 2026-03-26T17:29:52+01:00

shoko commented

2026-03-26 17:29:52 +01:00

Summary

Parallel page fetching with ThreadPoolExecutor (concurrency=5)
File-based cache with 5 min TTL in ~/.cache/polymarket-browse/
New --no-cache flag to bypass cache
New --max-total parameter for early exit
All 61 tests pass

## Summary - Parallel page fetching with ThreadPoolExecutor (concurrency=5) - File-based cache with 5 min TTL in ~/.cache/polymarket-browse/ - New --no-cache flag to bypass cache - New --max-total parameter for early exit - All 61 tests pass

shoko added 1 commit 2026-03-26 17:29:52 +01:00

Add parallel fetching, caching, and max_total parameter eafbdba4a5

- Parallel page fetching with ThreadPoolExecutor (concurrency=5)
- File-based cache with 5 min TTL in ~/.cache/polymarket-browse/
- New --no-cache flag to bypass cache
- New --max-total parameter for early exit
- Updated tests to work with new implementation

shoko added 1 commit 2026-03-26 17:43:21 +01:00

Add unit tests for parallelization, cache, and max_total bab373ab8f

- TestParallelFetchConcurrency: verify batch size of 5 and concurrency limit
- TestCacheFunctions: test cache read/write error handling
- TestMaxTotalParameter: test max_total event limiting

shoko added 1 commit 2026-03-26 17:54:48 +01:00

Fix total_pages calculation bug and add tests 1ae60f5661

- Fixed total_pages calculation: API returns 5 events/page, not PAGE_SIZE
- This was causing partial=false positives when max_total was used
- Updated tests to use correct pagination values

shoko added 1 commit 2026-03-26 18:06:35 +01:00

Add comment explaining total_pages ceiling division calculation 09f3cb9066

shoko added 1 commit 2026-03-26 18:15:37 +01:00

Make page size calculation dynamic based on first API response 9d1e328f53

- Uses actual event count from page 1 to calculate total_pages
- Removes hardcoded '5' for events per page
- API changes to page size will be handled automatically
- Updated tests to match real API behavior (5 events per page)

shoko added 1 commit 2026-03-26 18:19:14 +01:00

Update SKILL.md with new caching and parallel fetching documentation c75d123dfd

han approved these changes 2026-03-26 18:21:03 +01:00

han left a comment

lgtm, lets merge it

shoko merged commit f468a12ea4 into master

2026-03-26 18:23:20 +01:00

shoko referenced this issue from a commit

2026-03-26 18:23:21 +01:00

Merge pull request 'Add parallel fetching, caching, and max_total parameter' (#26) from feat/parallel-fetch-cache into master

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: shoko/jujutsu-skills#26