[Analysis] API Efficiency Issues and Optimization Opportunities #25

New Issue

shoko · 2026-03-26T16:51:35+01:00

shoko commented

2026-03-26 16:51:35 +01:00

Executive Summary

Assumption: CONFIRMED. The current implementation is inefficient. The Polymarket API does NOT support server-side filtering or sorting. We must fetch all events and filter/sort client-side.

API Behavior Findings

API returns only 5 events per page regardless of limit parameter
No server-side sort - sort=volume, order=desc are ignored
No server-side filter - Cannot filter by volume, tradeable status, or event type
Counter Strike: 155 events across 31 pages, only 50 are tradeable matches

Current Inefficiencies

Scenario	Events Fetched	Efficiency
matches=5, non_matches=5	145/155	6.5% waste
matches=10, non_matches=10	150/155	3.2% waste
matches_only=5	20/155	87% waste
sort_by=volume	155/155	100% required

Rate Limit Testing

50 rapid burst requests: 100% success
Parallel requests (8 categories): ~4x faster
Conclusion: Parallel fetching appears safe, but production limits unknown

Proposed Solutions

Parallel Page Fetches - Use ThreadPoolExecutor, ~4x speedup
Smarter Early Exit - OR logic instead of AND
Caching Layer - Cache for 5 min to reduce redundant calls
Volume-Weighted Sampling - Fetch first 10-15 pages only
Accept Inefficiency - Current behavior is correct, just slow

Questions for Discussion

Is fetching 93% of data for simple queries acceptable?
Should we prioritize speed over completeness?
Would caching help the use case?

Full analysis: docs/api-efficiency-analysis.md

## Executive Summary **Assumption: CONFIRMED.** The current implementation is inefficient. The Polymarket API does NOT support server-side filtering or sorting. We must fetch all events and filter/sort client-side. --- ### API Behavior Findings 1. **API returns only 5 events per page** regardless of limit parameter 2. **No server-side sort** - sort=volume, order=desc are ignored 3. **No server-side filter** - Cannot filter by volume, tradeable status, or event type 4. **Counter Strike: 155 events across 31 pages**, only 50 are tradeable matches ### Current Inefficiencies | Scenario | Events Fetched | Efficiency | |----------|---------------|------------| | matches=5, non_matches=5 | 145/155 | 6.5% waste | | matches=10, non_matches=10 | 150/155 | 3.2% waste | | matches_only=5 | 20/155 | 87% waste | | sort_by=volume | 155/155 | 100% required | ### Rate Limit Testing - 50 rapid burst requests: 100% success - Parallel requests (8 categories): ~4x faster - **Conclusion:** Parallel fetching appears safe, but production limits unknown ### Proposed Solutions 1. **Parallel Page Fetches** - Use ThreadPoolExecutor, ~4x speedup 2. **Smarter Early Exit** - OR logic instead of AND 3. **Caching Layer** - Cache for 5 min to reduce redundant calls 4. **Volume-Weighted Sampling** - Fetch first 10-15 pages only 5. **Accept Inefficiency** - Current behavior is correct, just slow --- ### Questions for Discussion 1. Is fetching 93% of data for simple queries acceptable? 2. Should we prioritize speed over completeness? 3. Would caching help the use case? --- *Full analysis: `docs/api-efficiency-analysis.md`*

shoko added the enhancement medium labels 2026-03-26 16:56:20 +01:00

han commented

2026-03-26 17:03:24 +01:00

Parallel Page Fetches

for parallelization, how much parallel should we do in a single process? I meant for example if there is 50 pages. should we just run 50 parallelization? or we will get 4 pages at a time? how to define how many parallezation? is it regarding cpu or memory?

Smarter Early Exit

Changing it to OR will make it to be a different purpose. AND means we want both of them. OR means we want either of them. AND needed when user want to get both of them. while OR needed when user doesn't care which one is it as long as they got for example 10 markets.

Caching Layer
yes, I was going to suggest caching. can you propose and benchmark also put tradeoff is there better option than caching it for 5 mins? new markets doesn't appear in every seconds or something. I think if user want to get new data without caching, they should put an extra flag like --refresh or something. what do you think?

Volume-Weighted Sampling
I don't quite understand what is the purpose of it

I want to keep both speed and completeness. I don't want to have 1 second speed but only 1% completeness. I want something that doesn't take too long but also completeness is reach the threshold. what do you think about it?

**Parallel Page Fetches** for parallelization, how much parallel should we do in a single process? I meant for example if there is 50 pages. should we just run 50 parallelization? or we will get 4 pages at a time? how to define how many parallezation? is it regarding cpu or memory? **Smarter Early Exit** Changing it to OR will make it to be a different purpose. AND means we want both of them. OR means we want either of them. AND needed when user want to get both of them. while OR needed when user doesn't care which one is it as long as they got for example 10 markets. **Caching Layer** yes, I was going to suggest caching. can you propose and benchmark also put tradeoff is there better option than caching it for 5 mins? new markets doesn't appear in every seconds or something. I think if user want to get new data without caching, they should put an extra flag like --refresh or something. what do you think? **Volume-Weighted Sampling** I don't quite understand what is the purpose of it I want to keep both speed and completeness. I don't want to have 1 second speed but only 1% completeness. I want something that doesn't take too long but also completeness is reach the threshold. what do you think about it?

shoko commented

2026-03-26 17:05:36 +01:00

Thanks for the detailed questions. Here are my responses:

Parallel Page Fetches

how much parallel should we do in a single process?

It is about network I/O and avoiding overwhelming the API, not CPU/memory-bound. A good starting point is min(5, total_pages) concurrent requests. With 50 pages, we would do batches of 5. This balances speed and being respectful to the API.

Smarter Early Exit

Changing it to OR will make it to be a different purpose

You are right. The current AND logic is semantically correct for "I want both match and non-match markets." The issue is it is too conservative. We could add an early_exit_mode parameter: early_exit_mode="any" (stop when either quota met) vs early_exit_mode="all" (current behavior). But given your requirements below, this may not be needed.

Caching Layer

if user want to get new data without caching, they should put an extra flag like --refresh or something

Agreed. Proposal: cache_ttl=300 (5 min default) with --no-cache or --refresh flag to bypass. Also consider stale-while-revalidate pattern: serve cached data immediately while fetching fresh data in background.

Volume-Weighted Sampling

I do not quite understand what is the purpose of it

Given the API returns only 5 events/page and does not sort by volume, I was exploring "what if we only fetch first 10 pages (50 events) instead of all 155?" We would get ~35% of events but possibly 70%+ of high-volume ones. But honestly this is a poor tradeoff. Discarding it.

Revised Proposal (based on your requirements)

You want BOTH speed AND completeness, not a tradeoff.

Parallel fetching (batches of 5) + conservative early exit = ~2-3x faster without sacrificing completeness
Background refresh cache - serve stale data instantly, update in background when cache expires
Keep completeness - do not implement OR logic or sampling that reduces data

This gives you speed (parallel + caching) AND completeness (full data when needed).

Shall we proceed with this hybrid approach?

Thanks for the detailed questions. Here are my responses: --- **Parallel Page Fetches** > how much parallel should we do in a single process? It is about network I/O and avoiding overwhelming the API, not CPU/memory-bound. A good starting point is `min(5, total_pages)` concurrent requests. With 50 pages, we would do batches of 5. This balances speed and being respectful to the API. --- **Smarter Early Exit** > Changing it to OR will make it to be a different purpose You are right. The current AND logic is semantically correct for "I want both match and non-match markets." The issue is it is too conservative. We could add an **early_exit_mode** parameter: `early_exit_mode="any"` (stop when either quota met) vs `early_exit_mode="all"` (current behavior). But given your requirements below, this may not be needed. --- **Caching Layer** > if user want to get new data without caching, they should put an extra flag like --refresh or something Agreed. Proposal: `cache_ttl=300` (5 min default) with `--no-cache` or `--refresh` flag to bypass. Also consider **stale-while-revalidate** pattern: serve cached data immediately while fetching fresh data in background. --- **Volume-Weighted Sampling** > I do not quite understand what is the purpose of it Given the API returns only 5 events/page and does not sort by volume, I was exploring "what if we only fetch first 10 pages (50 events) instead of all 155?" We would get ~35% of events but possibly 70%+ of high-volume ones. But honestly this is a poor tradeoff. **Discarding it.** --- **Revised Proposal (based on your requirements)** You want BOTH speed AND completeness, not a tradeoff. 1. **Parallel fetching** (batches of 5) + conservative early exit = ~2-3x faster without sacrificing completeness 2. **Background refresh cache** - serve stale data instantly, update in background when cache expires 3. **Keep completeness** - do not implement OR logic or sampling that reduces data This gives you speed (parallel + caching) AND completeness (full data when needed). Shall we proceed with this hybrid approach?

han commented

2026-03-26 17:13:49 +01:00

Parallel Page Fetches

yes I think going with min(5, total_pages) is good for now. we'll see later if we need further enchantment or not

Smarter Early Exit

I think once we got caching, we don't really need this one? maybe an optional parameter of (max_total)? so basically if max_matches is 10 and then max_non_matches is 10 but max_total is 15, it means that once we reached 15 we will exit early. what do you think?

**Parallel Page Fetches** yes I think going with min(5, total_pages) is good for now. we'll see later if we need further enchantment or not **Smarter Early Exit** I think once we got caching, we don't really need this one? maybe an optional parameter of (max_total)? so basically if max_matches is 10 and then max_non_matches is 10 but max_total is 15, it means that once we reached 15 we will exit early. what do you think?

shoko commented

2026-03-26 17:14:25 +01:00

Parallel Page Fetches

Agreed. min(5, total_pages) as the initial concurrency level.

Smarter Early Exit / max_total

I like the max_total idea. It is simpler and more intuitive than early_exit_mode. Semantics:

max_total=15 means "fetch at most 15 events total, then stop early"
Useful for quick snapshots without fetching everything
Works well with caching - cached data is always complete, but displayed results are capped

Implementation:

def fetch_all_pages(q, matches_max=None, non_matches_max=None, max_total=None):
    # ...
    for e in events:
        if is_match_market(e):
            match_count += 1
        else:
            non_match_count += 1
        
        # New: total count early exit
        if max_total is not None and (match_count + non_match_count) >= max_total:
            break
    # ...

This gives users a new way to get quick results without changing the existing max_matches/max_non_matches behavior.

Shall I proceed with this plan?

**Parallel Page Fetches** Agreed. min(5, total_pages) as the initial concurrency level. --- **Smarter Early Exit / max_total** I like the max_total idea. It is simpler and more intuitive than early_exit_mode. Semantics: - `max_total=15` means "fetch at most 15 events total, then stop early" - Useful for quick snapshots without fetching everything - Works well with caching - cached data is always complete, but displayed results are capped Implementation: ```python def fetch_all_pages(q, matches_max=None, non_matches_max=None, max_total=None): # ... for e in events: if is_match_market(e): match_count += 1 else: non_match_count += 1 # New: total count early exit if max_total is not None and (match_count + non_match_count) >= max_total: break # ... ``` This gives users a new way to get quick results without changing the existing max_matches/max_non_matches behavior. Shall I proceed with this plan?

han commented

2026-03-26 17:17:56 +01:00

yes lets proceed

shoko commented

2026-03-26 17:18:21 +01:00

Great. Starting implementation now.

Plan:

Parallel page fetching with min(5, total_pages) concurrency
Add max_total parameter for early exit
Add caching layer with stale-while-revalidate pattern and --no-cache flag

Will create PR once done.

Great. Starting implementation now. Plan: 1. Parallel page fetching with min(5, total_pages) concurrency 2. Add max_total parameter for early exit 3. Add caching layer with stale-while-revalidate pattern and --no-cache flag Will create PR once done.

shoko closed this issue

2026-03-26 18:53:17 +01:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: shoko/jujutsu-skills#25