When using proxy for browser automation, data collection or interface requests, timeout and retry settings will directly affect stability. Too high concurrency, too short timeouts, or infinite retries may slow down tasks or generate a large number of failed requests.
Recommended basic settings
- Connection timeout: It is recommended to start testing from 10 to 30 seconds.
- Read timeout: According to the target site response speed settings, slow sites can be appropriately relaxed.
- Number of retries: Usually 2 to 3 times is sufficient. Infinite retries are not recommended.
- Concurrency quantity: Test in small batches first, and then gradually increase according to the success rate.
When is it necessary to retry
When the network fluctuates temporarily, the target site is temporarily unavailable, or a single egress connection fails, limited retry can be performed. It is recommended to wait briefly between each retry to avoid continuous requests putting pressure on the target site and local tasks.
When not to retry
- Wrong username or password.
- The package has expired or the balance is insufficient.
- The proxy protocol or port is incorrectly filled in.
These issues require a configuration correction rather than continuing to retry. Repeated retries only amplify errors.
Troubleshooting process
- First use cURL to test whether a single proxy is available.
- Rerun small-scale tasks with 1 to 5 concurrent runs.
- Record success rate, average response time and failure reasons.
- If the success rate is stable, then gradually increase the concurrency.
A stable proxy task usually does not pursue the highest concurrency from the beginning, but first finds a reasonable request rhythm suitable for the target site, protocol and package.