Let systems find their own limits
Quote
it just self adjust as it goes until it finds each until it finds the happy place for each aggregator.
Episode 135: Maximizing the Pull (episode page, opens in a new tab) 00:23:08 (play audio from this timestamp, opens in a new tab)
Speaker-labeled
Summary
Each Podcast Index aggregator starts with a big batch, backs off by 100 feeds whenever the parser crashes, and after 48 stable hours tries to move up again. Each one finds its own 'happy place' and runs at maximum utilization.