mark nottingham

recent entries all entries feed

Why Our New TV Doesn't Like the Web

Thursday, 3 June 2010

HTTP Caching

A while back we used an absurd amount of reward points from our credit card to get some Myer gift certificates, and on the weekend these miraculously turned into a new TV, the Sony 32EX600.

Overall, we really like it; while I’m still trying to find the exact recipe to successfully encode video to feed to it via DLNA, it’s beautiful to look at, and (unlike most TVs these days), the UI is a pleasure to use.

Except.

One of the big selling features of the TV is that it offers “Internet Streaming” as well as DLNA. Basically, this means that it can stream directly from YouTube, the Onion, and a number of other places that will soon include back episodes of most programs on at least two major Australian networks.

That’s very cool, and kudos to Sony for making arrangements for content in the local market. However, actually using this feature from Australia — the same market they’re customising a TV for — is less-than-impressive.

That’s because every time you access one of these “Internet Streaming” channels, the TV makes not one but up to five SSL connections serially to a server in the US.

1275275765.332 1044 192.168.1.19 TCP_MISS/200 4157 CONNECT ssm.internet.sony.tv:443 - DIRECT/64.37.180.11 -
1275275766.432 1012 192.168.1.19 TCP_MISS/200 3845 CONNECT treb.internet.sony.tv:443 - DIRECT/64.37.180.15 -
1275275767.456 905 192.168.1.19 TCP_CLIENT_REFRESH_MISS/200 4139 GET http://ssm.internet.sony.tv/BIVL/icons/service_23/sub_2/h.png - DIRECT/64.37.180.11 image/png
1275275767.836 1290 192.168.1.19 TCP_MISS/200 5036 CONNECT treb.internet.sony.tv:443 - DIRECT/64.37.180.15 -
1275275769.276 1294 192.168.1.19 TCP_MISS/200 5139 CONNECT treb.internet.sony.tv:443 - DIRECT/64.37.180.15 -
1275275771.066 1745 192.168.1.19 TCP_MISS/200 32170 CONNECT treb.internet.sony.tv:443 - DIRECT/64.37.180.15 -

Since they’re serialised, it means we have to wait each time for the TCP connection to come up, the SSL context to be established, and the HTTP request and response to be received before you see a byte; hence the latencies of anywhere from about one to two seconds (second column, in milliseconds) from Australia, since their servers are in the US:

7 ge-7-11.car3.losangeles1.level3.net (4.71.32.61) 188.653 ms 187.361 ms 187.985 ms
 8 ae-71-70.ebr1.losangeles1.level3.net (4.69.144.114) 212.445 ms 211.550 ms 215.080 ms
 9 ae-5-5.car1.sandiego1.level3.net (4.69.133.205) 200.850 ms 200.829 ms 200.435 ms
10 ge-1-2.hsa2.sandiego1.level3.net (4.69.142.162) 201.915 ms 200.725 ms 201.477 ms
11 vl862.sdtermswitch-2.sonyonline.net (63.212.173.146) 201.194 ms 201.141 ms 201.253 ms
12 vl832.sdkollsw-2.sonyonline.net (64.37.144.90) 191.396 ms 192.539 ms 191.285 ms
13 * * *

Which means it’s about five seconds before you see anything come up in this interface, despite the fact that there’s nothing personalised or particularly dynamic in the content. Ouch.

After that, you get a screen with a bunch of icons for different shows and/or episodes on it, but again the TV doesn’t want to play nicely; not only are many of the responses uncacheable, but the TV also sends Pragma: no-cache on everything:

GET http://www.videodetective.net/utils/dynamicthumb.aspx?filepath=6747/28341022_.jpg&width=128&height=96 HTTP/1.1
Host: www.videodetective.net
Pragma: no-cache
Accept: */*
Proxy-Connection: Keep-Alive

…which results in a lot of cache misses:

1275275772.242 809 192.168.1.19 TCP_CLIENT_REFRESH_MISS/200 3043 GET http://www.videodetective.net/utils/dynamicthumb.aspx?filepath=2808/11795832_.jpg&width=128&height=96 - DIRECT/65.52.12.234 image/jpeg
1275275772.276 906 192.168.1.19 TCP_CLIENT_REFRESH_MISS/200 4502 GET http://treb.internet.sony.tv/content/thumbs/videodetective/CAT27178471f2c4019c872d1545a0f154c7.png - DIRECT/64.37.180.15 image/png
1275275772.337 909 192.168.1.19 TCP_CLIENT_REFRESH_MISS/200 5727 GET http://treb.internet.sony.tv/content/thumbs/videodetective/CATf2407056c089170d4df9dac66a284f38.png - DIRECT/64.37.180.15 image/png
1275275772.882 1509 192.168.1.19 TCP_CLIENT_REFRESH_MISS/200 4388 GET http://www.videodetective.net/utils/dynamicthumb.aspx?filepath=6452/27101922_.jpg&width=128&height=96 - DIRECT/65.52.12.234 image/jpeg
1275275773.213 1757 192.168.1.19 TCP_CLIENT_REFRESH_MISS/200 2901 GET http://www.videodetective.net/utils/dynamicthumb.aspx?filepath=6340/26630733_.jpg&width=128&height=96 - DIRECT/65.52.12.234 image/jpeg
1275275773.491 2030 192.168.1.19 TCP_CLIENT_REFRESH_MISS/200 4355 GET http://www.videodetective.net/utils/dynamicthumb.aspx?filepath=6706/28166929_.jpg&width=128&height=96 - DIRECT/65.52.12.234 image/jpeg
1275275773.887 366 192.168.1.19 TCP_CLIENT_REFRESH_MISS/200 3372 GET http://www.videodetective.net/utils/dynamicthumb.aspx?filepath=6767/28425441_.jpg&width=128&height=96 - DIRECT/65.52.12.234 image/jpeg

…and so on. All up, it takes about twenty — yes, twenty — seconds to load a page with a few thumbnails on it.

It’s not just the TV’s fault, to be fair; the content providers are providing things like this — which as per above takes nearly a second to load from Australia, and is uncacheable. Looking at it in REDbot, we see why; they send Cache-Control: private, meaning that shared caches like mine can’t store this static, non-personalised image.

It’s possible to fix some of these problems in Squid, luckily. The recipe I have so far is:

refresh_pattern ^http://ssm\.internet.sony\.tv/BIVL/icons/ 2880 50% 10800 ignore-reload
refresh_pattern ^http://brevia\.condenet\.com/.*\.jpg 2880 50% 10800 ignore-reload
refresh_pattern ^http://treb\.internet\.sony\.tv/content/thumbs/ 2880 50% 10800 ignore-reload
refresh_pattern ^http://www\.videodetective\.net/utils/dynamicthumb.aspx 2880 50% 10800 ignore-private override-expire ignore-reload
refresh_pattern ^http://images\.onnetworks\.com/ 2880 50% 10800 ignore-reload

Here, you can see the ignore-reload option which tells Squid to ignore the TV’s Pragma: no-cache, as well as ignore-private for the videodetective URLs.

Of course, I shouldn’t have to do this, and to really improve things, Sony needs to use something other than that dance of CONNECTs to view content; making multiple serialised SSL connections from halfway around the world is just not good user experience.


4 Comments

Subbu Allamaraju said:

Atleast they used GETs! When I last checked, the bing app on IPhone uses POST+POX. There are others like that - no matter what - they won’t get it.

Thursday, June 3 2010 at 11:14 AM

Steve Clay said:

With your squid rules, how much faster is it?

Friday, June 4 2010 at 11:28 AM

Mark Nottingham said:

The initial connection dance means that it still takes about five seconds to get a menu screen up, but the icons load in less than a second, rather than 10-15 seconds (provided they’re cached, of course).

Friday, June 4 2010 at 12:47 PM

daniel.haxx said:

Just a little curiosity:

The look of that HTTP request (including the pragma) does match libcurl’s request exactly (as in contents and order of headers) as how it used to do it…

Friday, November 12 2010 at 5:45 AM

about blog home