Web Players Seen Working Together More for Network Efficiency

Wireline ISPs have been approached by CDNs seeking to install gear in broadband networks, to move frequently requested video streams like movies and TV shows closer to subscribers. That’s according to responses to our informal survey of a few dozen websites, makers of consumer electronics, CDNs, telcos and cable operators. There’s also more of a focus on so-called federations to move content closer to end-users, according to many of the respondents. “We're at a point of mass experimentation -- everyone is experimenting,” said Vice President Chris Osika of Cisco’s Internet Business Solutions Group, a consultant to ISPs and media companies. “It will benefit all of them to be part of one of these federated models.”

Subscribers will eventually face changes in how they're billed for broadband, agreed executives from the CE, telco and website industries. They said the current model that still predominates at many U.S. cable and telco companies of not charging fast-Web customers for the amount of bandwidth they use may not be sustainable if those consumers want to use increasingly more data during peak periods of heavy downloads. “There has not been a lot of innovation in pricing” by wireline ISPs, said Osika. Those companies “have to have some way to recapture those costs” of adding capacity, said Senior Vice President Michael Golob of Frontier Communications. The telco’s average fast Internet customer uses 12 to 15 gigabytes of data monthly, a figure it expects to continue rising, he said.

Regulation’s a “wild card” in how these changes pan out, and “the landscape will change” accordingly, said David Reisfeld, general manager-enterprise solutions for Limelight. It works with websites and ISPs to speed video delivery. Rules are “a possible component to sway a winner of technology” here, Reisfeld said. Executives said there’s wide agreement within their industries that net neutrality rules aren’t violated by the various business and technology arrangements their companies are pursuing, nor by changing how broadband subscribers pay their provider. “How you charge your users, as long as you're doing it for the purposes of being profitable, is not really related to network neutrality,” said Chief Technology Officer Don Bowman of Sandvine, a vendor of traffic management and other products to ISPs.

Usage-based billing seems likely to be employed by more U.S. wireline ISPs, though it’s mainly been wireless carriers that charge customers for monthly packages of data, executives said. They said cable and telco ISPs may even charge broadband subscribers for activity based on whether it occurs at times of peak network use, though that’s unlikely anytime soon. Usage-based billing is “going to come eventually,” Golob said. “It’s not going to be one price and you get as much as you could possibly want. I think there will be different billing tiers."

Usage-based billing won’t likely much cut peak network traffic, not dissuading many from downloads of what they wouldn’t want to watch during other parts of the day, Bowman said. It’s a “means of performing discipline and traffic control,” he said. “We do not believe that usage management and traffic management should be confused with each other.” Dynamic-based pricing where rates vary by network capacity addresses bandwidth management and perceived consumer fairness, Bowman said. “If the market would accept that and it could be delivered fairly, everyone’s interests would be aligned perfectly,” he said of users and ISPs. “It’s on the cusp of being technically possible.” That traffic management techniques not applied full-time across a network must be avoided “is something we learned the hard way with the concept of network neutrality in 2008,” Bowman said of the FCC’s order that year against Comcast’s blocking of some peer-to-peer downloads (CD Aug 4/08 p1).

It’s “important” that ISPs better align costs with consumers’ motivations, said Vice President Mark Taylor of Level 3, an Internet backbone provider and CDN. “It’s something the industry has struggled to figure out what’s acceptable to consumers.” Charging users for peak-time usage as some highways do with high-occupancy vehicle lanes may make sense, he said. “But these are big, big changes that we as consumers would have to understand and accept, and the probability is if someone does that first and people don’t understand it, they'll lose customers.” Wireline ISPs seem likely to “default to exactly what the mobile phone industry does, which is ‘we'll bill you for the total amount of megabytes you consume in a month,'” Taylor said, citing what Comcast is doing. “The industry seems to be heading that way.” Comcast, which had no comment for this story, said in May it’s getting rid of a 250 GB monthly broadband usage cap it set in 2008 and in one strategy being tested will have a 300 GB minimum threshold for the lowest tiers (CD May 18 p3).

ISPs “are experimenting with different ways” of bandwidth management “and seeing what can they get away with from a consumer tolerance perspective and what can they get away with from a net neutrality or other regulatory constraints” viewpoint, said Vice President Ted Middleton of EdgeCast, which accelerates websites’ content delivery. “Wireless is obviously on the cutting edge of that” because there are physical limits to spectrum availability, he said.

CDNs ‘Courting’ ISPs

With any change to monthly bills going to be gradual, because residential broadband subscribers have grown accustomed to paying for how fast their service is and not necessarily how much data they consume, ISPs and CDNs said they're focusing on other areas to avert a capacity crunch. Cable operators and telcos are buying more circuits to increase capacity, bringing content closer to regional headends and central offices so it doesn’t need to travel on a backbone to be consumed, and testing out alliances with websites and CDNs, executives said. The ISPs are continuing to deploy more fiber, buy more capacity and move content closer on their networks to end-users, activities that won’t change even with a gradually increasing focus on alliances between the providers and other companies, executives said.

Frontier’s experimenting in its labs with putting CDNs in its network, much like multiple telcos form a “carrier hotel” to exchange traffic with various third-parties, said Golob. Websites that run CDNs have spoken with the telco about putting some of their servers in its network, he said. “They have actually been courting us.” CDNs would provide their own servers, which would run in a Frontier central office or other regional network facility, he said.

"The business model for that is still being worked out -- it does take some traffic off” Frontier’s network, said Golob. “But typically they want free power, they want free space, it’s something in our network that is not really controlled by us, so as an engineer it goes against some of our principles. So that relationship hasn’t been totally fleshed out yet.” Any deals must benefit both sides, and Frontier’s “done a lot of talking to a lot of folks about this, but I don’t think we've come to a business arrangement,” Golob said. Osika predicts “you're going to see a lot of the service providers collaborating” to bring Netflix and other large over-the-top video providers into ISP networks. “It benefits all the providers on there from an operational cost aspect,” he said. “If I'm Disney or if I'm News Corp., I don’t want to strike a deal with 30 ISPs in America” to move those media companies’ online programming closer to broadband subscribers, which can be accomplished through a CDN federation, Osika said. Disney and News Corp. representatives had no comment.

Federations’ operational models leave a lot to be desired, despite the arrangements seeming “great,” “Democratic” and “the right thing to do,” Reisfeld said. The deals don’t always have clear lines of demarcation about how to quickly fix unforeseen problems and ensure service quality, he said. “When something goes wrong, who do I call?” Such arrangements have been “somewhat muted,” since a website doesn’t want to enter into 20 different federations or CDN deals across the U.S., Reisfeld said. For the accords “to really take hold, it needs to be ubiquitous,” he said: It’s now “patchwork,” which “sort of creates a disincentive” to enter into an arrangement.

Costs for ISPs to buy more bandwidth capacity have been declining more slowly than what it costs CDNs to cache content in a cable operator or telco’s network, said Taylor. “What’s happened fairly predictably over the last four, five years is the CDN layer is getting cheaper than the bandwidth layer.” Broadband providers do “an awful lot of planning” to ensure there’s enough capacity per user, “constantly reducing the number of people that connect to a particular pool of bandwidth,” he said. Osika noted that “the bottleneck” of broadband capacity “still remains in the access network” of ISPs.

Frontier has been buying circuits, which let it add capacity “fairly easily,” Golob said: “The issue we have there is older equipment from when we first started” deploying DSL more than a decade ago. The goal is to increase “capacity to handle the customer needs, or being able to put less customers on a particular device” that’s in the network, he said. Frontier had 1.7 million broadband customers March 31. The company is also testing in its labs deep packet inspection, Golob said. “It gives you a pretty granular view of what customers are doing, to help manage your capacity a little bit better."

Cox Communications’ “focus is to make sure we have enough capacity to meet the needs of our subscribers to utilize bandwidth when and where they need it,” said a spokesman for the cable ISP with more than 4 million broadband customers. Ensuring there’s adequate capacity is “part our of ongoing infrastructure planning and management process,” he said. “The two types of customers we serve -- residential and business -- often have usage peaks at different times.” Businesses tend to use the most bandwidth during the workday, and residential users in the evening, the spokesman noted. Cox has more than 275,000 business customers, he said. ISPs are adding capacity between a central office or headends and data backbones, Osika said. The goal is to “preload” content by putting it closer to the end-user, so “I can push that further and further out to the node” or a regional headend, he said: “They're going to push it deeper and deeper” -- closer to customers.

Smart Devices

Devices that can predict what data a consumer will want and when they'll use it are becoming possible, executives said. That’s much like TiVo DVRs and Netflix’s online video website can predict what types of movies and shows users want to see. Such devices, though, are costly, could require cooperation among programmers, distributors and CE makers, and may not go mainstream in the foreseeable future, executives said. The smart devices could download data during off-peak times that would be consumed during busy network periods.

"Major network elements” are capable of traffic prediction, such as the pieces that go on network equipment that are made by companies like Alcatel-Lucent, Cisco and Juniper, Golob said. More devices at the consumer level that have such capabilities are “coming, and it really goes back to a business decision and how do you want to treat customers,” he said. “If customers understood” that downloading things like software upgrades during off-peak hours makes for “a better experience on-peak,” they could embrace it, he said. “There’s room for good traffic shaping, and there are some bad actors that have done other things, so I think you have to think about it from a customers’ perspective."

"As the networks do get more congested, you will see those models emerge, the device as a node” where CE products can relieve network congestion, Osika said: The move of content to the cloud contrasts with that push by placing more data away from devices, “so you've got a little bit of a pull and a push.” A cloud-based smart device could be in the cards in the future, Osika said. The move toward residential gateways, networking devices that serve all CE devices in a residence, makes such smart devices increasingly possible, said executives including Middleton. “If the consumer device or the player or the application could know you better, and have the ability to know you, it could download that during off-peak hours” what you'll watch later, said Taylor.

It’s very complex “to join up multiple parts of the distribution chain,” though it has the potential to make networks much more efficient, Taylor said. “That is the most elegant technical solution” to dealing with bandwidth constraints, he said of smart devices. “It’s not a perfect answer, but it would offer the opportunity to mitigate some of the costs of continuing to add capacity to the network which is driven only by the peak,” Taylor said: “The downside is if your predictions are not correct and you don’t end up watching the thing, you've actually” wasted bandwidth, though at a time of day when demand is lower. Instead of downloading large amounts of data at off-peak times, it may be more efficient to transmit streaming video in chunks of say five minutes even during peak times, so bandwidth won’t be wasted at off-peak times if an entire program that’s transmitted predictably then isn’t later watched, said Reisfeld. “The downside of doing any predictive ‘I am going to push content to you on your set-top box’ is I am going to consume bandwidth and I have no idea whether you're going to watch it,” he said of the premise that’s “on the hope and prayer that you will watch."

CDN/ISP deals and federation formation have greatly outpaced smart device efforts, and all those initiatives are at an early stage, said executives. Cooperation between websites and ISPs took “a while to take hold, and a bunch of dominoes had to line up, but they have lined up,” Middleton said. Increasing numbers of U.S. broadband subscribers, content providers “embracing” Internet Protocol and over-the-top delivery, and “successful commercial offerings like Netflix with a big subscriber base and a reasonable content library” are among the dominoes that have fallen in Middleton’s view. CDNs are becoming “as visible a part of network buildout as fiber and switches and optical gear” were “traditionally,” he said.

Websites’ Efforts

Websites also are stepping up efforts to fashion IP streams in ways more amenable to being viewed on a variety of devices at different speeds, executives said. They said the techniques include varying bitrates so video can be viewed with connections of different speeds, using HTTP streaming so content can be broken up into files of varying sizes that can be sent using the quickest routes between networks, and having routers with the content inside ISPs’ networks. Although websites generally have no economic incentive to use networks most efficiently, because they want video to be seen in as high quality as possible, they're increasingly working with ISPs to better use resources or making changes on their own, industry officials said. Websites will “get very creative in how they deliver content” in terms of using network resources, especially as ISPs change pricing structures, predicted Osika: “Three years ago I saw a much more adversarial relationship” between websites and ISPs, he said. “That’s growing much more collaborative."

Google’s YouTube and Netflix are among websites using techniques like pacing so streaming video gets to broadband customers as they're about to consume the particular chunk of a video, Bowman said. He said Sandvine’s research shows that the amount of U.S. wireline broadband network capacity consumed by Netflix at peak times has topped out at around 33 percent, and other functions are growing more quickly in the amount of network capacity they consume (http://xrl.us/bnbvvp). Other research has had similar findings, said Osika. All “real-time entertainment” accounts for about 65 percent of peak downstream traffic, including Amazon, Netflix, Pandora and YouTube, Bowman said. “Over the last few years, that’s been the landslide shift -- it’s driving higher peaks and more valleys. So the network is actually getting less efficient.” Amazon, Google and Netflix representatives had no comment for this story.

It’s hard to predict what streaming formats are likely to predominate in the long run among websites, what types of broadband customer billing arrangements will become most common or whether CE devices that can predict Internet usage will ever proliferate, all executives we spoke to agreed. “It’s a very cloudy question” of where efforts coalesce, Reisfeld said. “The market is definitely in a state of flux.” Since “I don’t know what format is going to win,” Limelight’s customers “don’t need to figure that out,” he said. “We'll wait for the dust to settle. Meanwhile, I'll mux it on the fly for you,” he said of changing content formats “on the fly.” The move toward HTTP-based streaming formats “is one of the biggest remaining single things that will help the industry, because it’s inherently cache and CDN friendly,” said Middleton.

Increasingly websites are effectively throttling their own content, boosting or decreasing the streaming speeds depending on how it’s being consumed, executives said. “It moves a bunch of the intelligence about how to consume the stream into the player, where the eyeball is” that’s watching streaming media, Middleton said. “The player is taking into account ‘what’s my network connectivity right now.” It would “throttle the actual amount of streamed bandwidth being requested from the server,” he said: “The players kind of self-govern themselves based on what’s available” and “the fact that the network might be degraded."

Peak-based billing or time-of-day billing make the most economic sense, when customer preferences aren’t taken into account, because it lines up financial incentives of users and ISPs best, executives said. Dynamic peak-time pricing where broadband customers pay for what they use depending on how much capacity’s available is fair from both a network management and consumer perspective, Bowman said: But there’s a “lack of obvious market acceptance and demand.” Regardless of what billing method predominates, consumers must “recognize that bandwidth is a limited or finite resource and ... really treat it that way,” Middleton said. “Particularly in the upcoming generation that expects it to just be there and the Internet is like a utility, the question becomes is that a trainable behavior?”