Proposals for alleviating API strain

Question

Problem

With a recent influx of stack-exchange-api challenges, the poor old API has been taking a bit of a hammering and some of us are finding ourselves hitting the throttle limit and being blocked from accessing the API for extended periods of times.

The answers below propose possible solutions to counter this problem and, hopefully, from them we can come up with something viable.

Answer 1

Proposal

To solve the problem and as (with very rare exception) live data is not required in order to complete API based challenges, I propose the creation of a website, built and maintained by the community, to implement the API as a search on SEDE's data dump, which is updated every week.

---

How it would work

• A user posting a new stack-exchange-api challenge would enter the URL(s) of the API request required to complete the challenge, except under the domain of the website instead of that of Stack Exchange. e.g. instead of https://api.stackexchange.com/2.2/comments?order=desc&min=10&sort=votes&site=codegolf users would enter something like https://semirrorapi.com/2.2/comments?order=desc&min=10&sort=votes&site=codegolf
• The site would then execute a database query to retrieve the response for that request.
• All Stack Exchange API questions would be required to use this website instead of the pfficial Stack Exchange API.

Answer 2

Proposal

To solve the problem and as (with very rare exception) live data is not required in order to complete API based challenges, I propose the creation of a website, built & maintained by the community, to cache responses from the API that members can then use in their solutions.

---

How it Would Work

• A user posting a new stack-exchange-api challenge would enter the URL of the API request required to complete the challenge.
• The site would retrieve the response for that request and save it locally as a JSON file.
• A local URL would be returned to the user for inclusion in their question.
  • If a user enters a URL for a request with a response already saved then the existing URL would be returned and, optionally, the local file would be updated with fresh data if it was older than a set period of time.
• All users attempting to answer the challenge would use that all users answering it must use in their solution.

---

Questions

• Isn't figuring out the API request usually part of the challenge?
  Yes, it can be seen to be, however, once one user figures out the optimal URL to use then all users attempting the challenge will use the same URL so it's not really an issue.
• What about challenges that do require live data?
  Hopefully, they will remain so few and far between and only require one request to be made to the API that we could conceivably still use the API for those particular challenges.
• What about questions that necessitate requesting multiple pages from the API?
  In most cases, removing the need to request multiple pages does not detract from the challenge. In fact, in my opinion, it improves it. Consider the "Whose rep is tied?" and "PPCG's Unanswered Questions" challenges; both require the requesting of multiple pages from the API but neither is improved by having that requirement nor would either be worse for the its removal.