I am sure you already have the answer to this by now since its 6 months old, but I thought I'd add this for the benefit of people who face this problem in the future, as I did today.
I too removed the https and that increased the speed a little but not enough. In the end I moved the box.js line to the bottom of the page. This means your entire web page will load before it attempts to download the box.js line and the user can be happily reading your web page while the box.js loads in the background. Its not perfect buts it a reasonable work around and give the illusion that your page is downloading faster.