Huhm, so doesn't it imply that your server is so slow that the file is not loaded immediately then?
BTW, after the first time loading it, your web browser should have cached that specific file as well and doesn't need to reload from the server.