Say you have a webapp that lets users upload a file to S3, immediately followed by a download request for the same file. A user’s profile picture is a common example. As soon as the user has uploaded their profile picture, you have to download it to show it on their profile page. If the download is via CloudFront, not directly from S3, there are a few caveats you should be aware of.
CloudFront is a pull-through CDN. CloudFront edge locations only know about your site’s location & configuration. They don’t know about the site’s content until they receive requests for it. When that happens, the CloudFront edge fetches the requested content from the origin server, & caches it for serving subsequent requests.
If you hit the CloudFront download URL for the file you just uploaded, before the upload completes, CloudFront will try to fetch the file from S3, get a 404 in response & cache this 404 status for a few minutes. All subsequent download requests for the same object in the next few minutes will return the cached 404, even if the object has finished uploading & is indeed available for download. This is called “negative caching” — caching the fact that a request won’t work. This is done to avoid hammering the origin with requests that are likely to fail anyway.
CloudFront always caches the following HTTP 4xx and 5xx status codes returned by your origin — 404, 405, 414, & 500 to 504.
— HTTP Status Codes that CloudFront Always Caches — CloudFront Documentation
Note that some browsers like Chrome will preload URLs typed into the address bar. So if you typed the object’s download URL & waited for the upload to complete, Chrome has already made a request to CloudFront for that URL in the background & CloudFront has already cached the non-existence of the object.
Chrome starts loading some web pages in the background, even before you’ve finished typing the URL in the omnibox. If the URL auto-completes to a site you’re very likely to visit, Chrome will begin to prerender the page. Prerendering reduces the time between when you hit Enter and when you see your fully-loaded web page–in some cases, the web page appears instantly.
— Speed & Security — Chrome Blog
Harish KM 