UMTS: Truly, you have a dizzying intellect

We have spent a lot of time in recent months studying the specifications for "Uu", the UMTS 3G subscriber interface. The spec is an obfuscated mess and the experts who publish books seem to disagree on a lot of critical details. In fact, the literature is so inconsistent that we are starting a public wiki-based documentation project.

In its basic form (non-HSPA), UMTS can deliver 384 kb/s per channel with up to 4 such channels active at once, assuming good link margins, small delay spreads, good power control, proper phase of moon, low traffic levels in surrounding cells, and generally clean living and happy thoughts on the part of everyone involved. That's just over 1.5 Mb/s on a really, really good day. To do this, UMTS musters just about all of the fancy math that the 1990's had to offer: orthogonal spreading codes (at least, orthogonal until they hit the real world), turbo codes, rake receivers, multisensor diversity demodulators. In exchange for all of this complexity you get roughly half (yes, half) the usable bit rate that you could get from an EDGE-style PSK/TDMA interface in the same bandwidth, except that the PSK/TDMA approach would provide more solid QoS guarantees, use a lot fewer transistors and be more power-efficient due to smaller crest factors in the amplifiers.

So, you might ask, why all of the new complexity if we get no performance advantage? Because simple approaches don't produce enough new intellectual property to keep the usual suspects in business. If you just take the GSM/GPRS/EDGE specification and change some parameters (like symbol rate and number of timeslots) you can get a very efficient, flexible system, but you won't have a much fodder for IEEE papers. Worse yet, you don't generate a lot of new patents, and with a lot of the patents on 2.xG systems expiring early in this decade, the NEPs needed a new gravy train. This complexity also deters small players from building their own UMTS implementations. (We have taken that as a challenge, but then we are unreasonable people.)

And what did UMTS do for the carriers? It nearly killed them. They overbid for the spectrum, barely had enough money left to roll out these really expensive new networks and when it was over they discovered that all their customers really wanted was better coverage and lower bills, neither of which have anything to do with UMTS. (The iPhone later proved that lack of demand for data was mostly due to the carriers' walled-garden approach to the itnernet, but that's fodder for a different post.) The killer app for the next five years turned out to be text messaging and economists got to write lots of papers saying things like "We conclude that the rationalization of bidding in the United Kingdom’s UMTS auction remains problematic." And just as the carriers are starting to recover from UMTS, the usual suspects start pushing the next shiny, new thing: LTE/IMS.

Making GSM Future-Compatible

Over the last few weeks, I have been reading through the 3GPP IMS specifications. IMS is the core network for next-generation 4G/LTE mobile data and telephony. Going through the specs is more like bush-whacking than reading; I still can't look at most of the network diagrams without getting dizzy. But I am starting to get a feel for it. In it's essence, IMS is a SIP core network for cellular. Granted, it still looks way more complicated than it needs to be to serve that function, but that's what it is.

Lucky for us, one of the key ideas of OpenBTS is to also use a SIP core network for cellular. So in terms of core networks, we are about five years ahead of the industry, even if the air interface is Um or Uu. We expect the commercial release of OpenBTS to "just plug in" to IMS core networks within a few weeks. IMS compatibility has two big implications for OpenBTS moving forward.

First, it means that there is an application for OpenBTS in incumbent carrier networks that are moving to 4G in the next few years. I've had the opportunity to talk to executives and network engineers from a few carriers who are planning their 4G transitions and have heard the same story over and over. Here it is: "The 4G rollout is expensive, but the performance improvements justify the cost. Except in rural areas, where the subscriber density is too low to justify the expense. But if we keep running GSM/EPGRS or 3G in those areas, then we will have to continue running the old SS7-style core network in addition to the new IMS core network. So we either waste money running two core networks or we waste money installing 4G basestations in the middle of nowhere." The OpenBTS approach offers a solution: Refit your rural sites with an inexpensive OpenBTS-based RAN and then turn off all those BSCs and MSCs.

Second, it eases the minds of carriers looking at greenfield rollouts in the developing world. These carriers need inexpensive networks, but don't want to feel like they are installing obsolete technology. Installing some low-end BTS/BSC/MSC combination just because it's cheap is installing obsolete technology because it will saddle you with an end-of-life core network that you will need to continue to support for years. Sure, you might run your circuit-switched protocols over 802.whatever, a la SIGTRAN, but all that means is that you're not completely stupid; Abis-over-IP is so 1998. On the other hand, installing an IMS-compatible OpenBTS-based network is a first step toward 4G, even if the initial rollout only supports 2G handsets. When the future arrives in your corner of the world, you'll be ready.

FAKALOFA LAHI ATU

"FAKALOFA LAHI ATU! Please respond with your provisioning code..."

There is now an OpenBTS pilot site in Niue, installed with the cooperation of Telecom Niue under a license from the government. The system is still in a closed evaluation, but when the evaluation phase ends the Niue system will probably be the first OpenBTS installation to provide common-carrier service to the general public. This is a very big step for the project and will bring a much-missed service to the residents, many of whom already own GSM handsets when they travel in New Zealand. It will be a learning process for everyone involved.

Installation took two weeks and is still incomplete, mostly due to customs delays in New Zealand and incomplete documentation on the installation site. We also had serious problems coordinating spectrum with a large public wifi system who's operators seem to think that they can use whatever spectrum they want without consulting the regulators. I would have blogged about all of this on the spot, but the public internet service was unusable most of the time we were there. (Naturally, they blamed us. More on that later. UPDATED BONUS: They are STILL BLAMING US.) If you need a blog fix right away though, Tim Panton managed to squeeze a posting out.

The short status summary is this: Telecom Niue's technicians put a 13 dBi sector antenna about 53 meters up on a platform. From there, we should be able to get reasonably good coverage over Alofi, 3-5 km away, once the wifi people quit jamming our uplink with their unlicensed 900 MHz gear. We managed to make a few international calls from cellphones in Alofi and we sent a lot of text messages among ourselves around the island. We look forward to working with Telecom Niue over the next few weeks to get the system better configured and tied-in to their existing wireline switch. The details will follow over the next few days.

I also want to say that most of the people we encountered in Niue were remarkably nice to us and that the natural beauty of the island's coastline is stunning ... even for someone who lives in California.

(Kone Magatogia setting the antenna, 53 meters AGL. Thanks to Toki Talagi for this photo.)

The Problem of Spectrum Granularity

[BTW, Greetings from eComm 2009.]

One of the most serious challenges to providing low-cost cellular service in rural areas is the lack of available cellular spectrum. Just about everywhere in the world, all of the spectrum is already locked up by incumbent carriers. So, you might ask, if the spectrum is already held, why don't the people living under it have service? The problem is one of granularity.

Rural areas have lower population density and less infrastructure than urban areas. You need taller towers to get greater range. Your cell sites might not have grid power. The best sites may not be near paved roads. These factors make rural areas more expensive to serve. As the same time, perversely, the people who live in these rural areas have less income, and there are a lot less of them. So if you are a cellular carrier with licenses in both rural and urban areas, you have good motives to concentrate on urban service and ignore the rural areas.

Basic physics shows us that urban and rural areas might require different technical approaches. Basic demographics shows us that expectations of profitability are much lower in rural areas than in urban areas. So how do regulators deal with that? They make it nearly impossible to get a cellular license in a rural area without having to get a license in an urban area at the same time. No, that's not supposed to make sense, but it is true nearly everywhere in the world.

Here in the US, the FCC auctioned most cellular licenses by "metropolitan statistical area" (MSA) or "rural statistical area" (RSA). Despite those promising names, more often than not an MSA or RSA is just a county or group of counties. (Here's the map in PDF.)  That's why I can't get a license for rural Solano County, California, which is mostly sheep pasture and marshes, without getting licenses for several cities totaling nearly 500,000 people at the same time. That's why I can't get a license for Gerlach, Nevada, an isolated town of about 200 people, without getting a license for Reno, a distant city of more than 200,000, in the bargain. What if you want to serve Gerlach but can't afford a license for Reno? TFB (too ... bad). No license for you!

It's bad enough to do business that way in the US, where even the country folk are affluent by world standards, but in developing countries, where the urban-rural disparity is even greater, most licenses are national.  For example, if you want to provide cellular service anywhere in Kenya, you probably need a license for Nairobi.  And since median income in Nairobi is around US$160/mo and the median income in the coutryside is less than US$30/mo, you can imagine what that does for the prospects of a small rural carrier ever happening.

If you wanted a licensing system to discourage rural service, it would be hard to design a more effective spectrum allocation policy.  Some countries are making noises about changing these policies soon.  Let's hope.

GSM WLLs and Carrier Acceptance

The biggest challenge to the deployment of OpenBTS is that all of the world's cellular spectrum is already licensed, most of it to very big companies.  These big companies don't have strong motivation to deploy low-cost services in rural areas.  First, their actual cost of operation is fairly high in rural areas.  Second, even if that cost of operation could be lowered dramatically, it would create a marketing problem.  Solving the first problem will only magnify the second.

Suppose you're "Big Cellular" and you run a GSM network in the developing world.  It costs you $4-$8 per subscriber per month to operate, costing less in urban areas and more in rural areas.  But the people who actually live in rural areas can only afford about $2/month, so you mostly avoid those areas, unless a major road happens to pass through them, carrying your richer urban customers between cities.  Government regulators may pressure you to serve the rural areas, but you can always just show them your balance sheets and argue (honestly, even) that you are already giving the broadest service that can reasonably be expected for a profitable network.  Everyone's happy -- expect for the rural poor who, will never get telephone service under this model.

This is all cozy until a disruptive technology makes $2/month rural service a real possibility.  If you're Big Cellular, that's not good news.  You already have a legacy network that you're still paying for and the new technology is not directly compatible with it.  Even if it were compatible, the new technology creates a marketing problem because your urban customers paying $12/month will soon be demanding to know why they can't get $2 service like their country cousins.  You can try starting a second brand, but that's very expensive and you fear that your new, cheap brand will simply erode your existing market along the urban-rural edge.

The solution here is to make sure that the new service is not a viable substitute for normal cellular.  I'm not saying give the rural poor broken service.  I'm saying give them what they really need, which is reliable telephone service at a very low price, which is not the same thing as cellular, even if the "subscriber terminal" was built to be a cellphone.

The purpose of the new network is to provide basic telephone service in rural areas.  You don't need full cellular functionality to do that.  For example, maybe you don't implement handovers of active calls between cells.  Maybe you don't allow your rural subscribers to roam into "real" cellular networks.  If you are really cheap, maybe you even bind each SIM to a specific cell site, eliminating all of the mobility management functions.  This functionality already has a name: wireless local loop (WLL).  You use GSM like you might use DECT or WiFi, but with much larger service areas and much cheaper handsets.

Operating in WLL mode offers several advantages in this scenario.  There is the technical advantage of a much simpler core network, although a carrier can still support roaming for conventional cellular subscribers if it chooses.  There is the business advantage of no longer being a direct competitor to legacy cellular networks.  And depending on what country you are in, there may be regulatory advantages as well.

If you are Big Cellular, this new low-cost WLL is not a particular threat to your existing business.  It serves a market you would rather not deal with.  Maybe you can open a new subsidiary to operate WLL networks, or, depending on your local regulations, you can lease your fallow rural spectrum to a WLL carrier.  The WLL becomes a modest source of profit.  Universal service can be someone else's problem while you, Big Cellular, can do what comes naturally: market ever more complex services to the cities and gouge tourists with crazy roaming fees.  Everyone is happy again, and maybe this time we can spread it around a bit more.

What Stuff Costs, Part 2: CAPEX

There are no "list prices" in the global telecom industry. Every purchase is a negotiated deal with the details covered by NDAs. Prices are arbitrary.  How do the equipment providers get away with that? Let's take a look...

Consider the cost of installing a BTS in rural site.  Something like this:

That costs $200k-$250k, depending on what part of the world you're in.  Most of that money is for "civil installation": site prep, concrete pads, backup power, the mast, that little shack, etc.  There's well over $150k worth of stuff there just to support the BTS.  So what should the actual BTS cost?  As long as it's a lot less than the infrastructure cost, the buyer doesn't care because it won't be a significant part of the total site cost. The baseband processors, transceivers, power supplies and amplifiers for a 3-sector 3-TRX ("1/1/1") kit typically run $20k-$50k, depending on the vendor, the buyer, the specific product and whatever side deals the vendor can offer.  That will give 21 Bm channels at full rate.  There's no point in going below $20k because the savings to the carrier are insignificant below that point.  And the price can't go much above $50k before the BTS becomes significant in the total.  Notice that this price range has nothing to do with the actual cost of producing a BTS, as long as that cost is well below $20k.  The total installed cost is around $75k per fielded TRX, or around $11k per Bm channel.

That's the equipment in the field.  You also need a core network.  The core network gets installed carrier-grade data centers.  As long as the equipment costs less than the data centers, prices just don't matter much.  Together, the BSCs, MSCs and location registers in the core network can easily cost over $5k per fielded TRX, or about $700 per Bm channel.  The civil part probably costs twice that, bringing to total to around $15k/TRX or $2,100 per Bm channel.  The core network also creates a floor for a viable network size, since even a "small" MSC is built to support hundreds of cell sites and priced accordingly.

So the rollout cost is around $15k/TRX for electronics and totals around $90k/TRX for a low-density network when you include all of the civil infrastructure.  One TRX can serve about 1,000 subscribers in the developing world so your rollout capital is least $90 per subscriber, not counting counting other costs ignored here.  Note, though, that the dominant cost is civil infrastructure.  Even if the electronics were free, the total capital would not change by more than about 25%.

The only way to dramatically change the cost of a cellular network is to simplify the infrastructure, something that the existing equipment providers have little motivation to do.  For example, if the whole BTS package can be mounted directly onto the mast and left out in the weather, you can get rid of that air conditioned shack.  If you cut the power requirements, you also cut the cost of the backup power systems.  OpenBTS is radical, though, in its approach to the core network: get rid of it and run BTS units as peers.  Don't just reduce the cost of equipment.  Reduce the amount of equipment.

This is one way that OpenBTS hopes to change the economics of rural cellular service: reducing the capital requirements to build a network. The OpenBTS model can reduce the rollout capital from over $90/sub to around $25/sub, not by offering a "cheap BTS" but by eliminating most of the steel and concrete and generators that a conventional GSM network requires.  OpenBTS can also reduce the minimum size of a viable network to something as small as a single cell site, allowing a carrier to start service with an initial capital investment of less than $30k.  Will carriers go for it, though?  Is there any spectrum available for this new kind of carrier to emerge?  We're working on it...

What Stuff Costs, Part 1: OPEX

Most African cellular carriers are partly owned by corporations like Millicom and Vodaphone that are traded on stock exchanges in Europe and America.  They publish regular financial reports.  From those reports we can tell that the typical 2007 African cellular subscriber paid $10-$12/month to talk on the phone for just over half an hour.  That sounds like a rip-off until you do a little more math and realize that it actually cost the carrier about $6/month to provide the service, not counting the cost of internetworking.  What the heck?

Let's say, for simplicity, that all of the traffic is compressed into 6 hours each day, so that you see a load of about 0.003 Erlang per subscriber during this peak traffic time.  A minimum 3-sector GSM BTS site provides about 10.5 Erlangs at 2% blocking and thus serves 3,500 subscribers at your typical daily peak load.  If your cost of operation is $6/sub/mo, that corresponds to a cost of about $252k/year per BTS site to run your network, with most of that cost in the BTS site itself: about $200k/year. (!)  When we first estimated this, we though we'd misplaced a decimal point somewhere.  Then we did we read this article in Balancing Act that put the cost of operating an off-grid BTS site in Africa at around $210k/year.  Then we talked to some telecom people from Africa who said the cost was well over $150k/yr but they didn't know by how much.  So it probably really is around $200k/yr.  Why?

It's all about power.  Suppose you have a BTS that draws 5 kW.  And since it's in the tropics you have to cool it, which brings your power budget up to 7 kW.  To supply that, you need a generator.  And since a generator is a target for theft, you need security lighting and cameras, which drive up your power budget and add at least 1 Mb/s to your backhaul requirement, which requires yet more power.  Before long, the site is drawing over 1o kW continuously and you are burning at least 25 gallons of diesel fuel every day.  Now you need a crew with a truck to drive around fixing generators and fences and filling fuel tanks, which is complicated by the fact that most of these sites aren't even near roads.  It starts looking like war logistics, where Sun Tzu tells us that every sack of rice at the front cost 10 more just to get there.  By the time you have everything in place you're spending nearly $20k/mo to keep this beast running.

This matters a lot to the long term development of these countries, because most of the people who live out in the countryside cannot afford $6/mo for anything, meaning that they will never get telephone service, not even on a non-profit basis.  To achieve universal service, someone will need to try something completely different.

So here's the good news: if you can keep site power consumption down to just a few hundred Watts, this all changes dramatically.  Instead of a generator, you can run the whole site on solar panels or microturbines in many parts of the world.  No more diesel fuel.  No more crews in trucks.  Every two years, you replace the batteries in the power system.  That's all.  That's why the design target for OpenBTS is 75 Watts per transceiver, a target that we are very near already just using off the shelf equipment.

Other other cost components in the subscriber rate are internetworking and capital amortization. Most connections between African carriers happen in Europe. That means that if you call from your MTN cell phone to a wired phone down the street that call may well get routed through France at French long distance rates. And the capital cost of rolling out a rural GSM network is at least $100/subscriber.  But those are topics for other posts.

A Lie Agreed Upon: Getting Hybrid Cellular into the Field

I just got back from 25C3 in Berlin. Thanks to Delta, I had a miserable time getting there, but I'm still glad I went. I met a lot of good people. I tried Club-Mate. I tried my best mangled German with the Reisegepäck staff at Flughafen Tegel. On the last night, Harald Welte treated me to a steak dinner, one of the few proper sit-down meals I had on the whole trip.

I met a lot of telecommunications professionals and we discussed the problem of carrier acceptance of the OpenBTS approach: providing a simple set of services at minimal cost and replacing the GSM "core network" with collection of peer-to-peer SIP applications.

There was a general consensus that the OpenBTS approach was technically feasible, even on large scales, and could be integrated into existing GSM core networks if needed.  There was also a general consensus that most incumbent carriers would reject the technology, even if integration into core networks is easy, largely on economic grounds.  The simple truth is that nobody in the telecommunications industry is really interested in making a modest profit by serving large numbers of very poor people.  The typical cellular executive would rather talk about extending 3G or 4G networks into rural Africa and then not do it.  Talking about bringing scorching fast networks to the poor is much more exciting than actually building a 2G system that they can afford to use.  

This consensus is not new to me.  I have had nearly the same conversation several times over the last two years with a handful of ex-employees from African cellular carriers. The big carriers will continue to concentrate on squeezing more revenue from their more affluent customers by offering more complex services. In the meantime, these big carriers will continue to sit on spectrum that is completely unavailable to the people who live under it, or, in the case of South Africa, cover the whole country with services that only a small minority can actually afford to access.  To borrow a phrase from Mark Twain, universal service is "a lie agreed upon" for the telecommunications industry.  It won't happen, even at modestly profitable levels, unless regulators force it.

This does not mean that OpenBTS will not find a commercial market.  It just means that it will not find a market with incumbent commercial carriers until regulators force them to get serious about universal service. That won't happen until early adopters, most likely small rural carriers, use OpenBTS to demonstrate that self-sustaining universal service really is possible.  So we're looking for early adopters.