July 30, 2005

About Those Root Servers

There is an interesting note on the ITU Strategy and Policy Unit Newslog about Root Servers, Anycast, DNSSEC, WGIG and WSIS about a presentation to ICANN's GAC.  (The GAC website appears to be offline or inaccessible today.)

The interesting sentence is this:

"Lack of formal relationship with root server operators" is a public policy issue relevant to Internet governance. It is stated that this is "wrong" and "not a way to solve the issues about who edits the [root] zone file."

Let's look at that lack of a formal relationship.

But before we begin, I'd like to raise the following question: Where does the money come from (and where does it go) to provide DNS root services?

Over the years I've put together estimates of what it would take to deliver root services, and I've probably always undershot the actual costs.  The raw hardware for a root server site isn't all that much - server computers, firewalls, load balancers, network switches/routers, and power distribution gear don't really cost all that much - a few tens of thousands of dollars in capital and installation costs per site depending on the desired capacity and the availability of reliable power.  But there are recurring costs that can be rather higher, particularly the costs for bandwidth, funds for replacement and upgrade, and maintenance.  If one wants to throw in physical security beyond that found in a typical high-quality shared facility or use dedicated links to multiple providers, the one-time and recurring costs will rise.

And that's just one site.  Today, because of the use of anycast to replicate many of the 13 legacy servers, there are more than 100 root server sites spread around the world.  Compared to the cost of an aircraft carrier, the total isn't that much.  But we're still talking about a system that in total costs several millions of dollars per year.

So where does the money to operate this system come from?

Much of it is donated.  But donated money is fickle.  And it often comes with hidden strings.  Unfortunately the root server operators have been very secretive about such matters.

We know that some of the root server operators are run by for-profit commercial corporations that are answerable to their stockholders and that may be acquired on the open stock exchanges.  And some root operators are operated by the United States military establishment - which is ultimately obligated to protect the United States, any obligation to others is subordinate.  There are root servers operated by university and non-profit entities.  In the case of the former, there is little to guarantee that the trustees of the university will continue to want to expend money to provide DNS root services as educational costs continue to rise and educational budgets become ever more difficult to balance.  As for the latter, they are under the control of trustees of boards that may have insular points of views or subtle biases in favor of certain industrial segments they consider to be "stakeholders".

In this whole system the flows of cash, the fiscal constraints and pressures, the ultimate allegiances, the chains of authority, and the hierarchies of authority are as unclear and vague as the flow of water through a Louisiana bayou.

All in all we can see that the root server operators are like a herd of cats - they may act in concert today but they could scatter to the four winds tomorrow as each responds to the pressures it feels and the attractions it sees.

There is no denying that to date the root server operators have done a job that deserves great praise.

But the internet community is building its future on nothing more than faith that the status quo will endure.

Suppose a root server operator found itself in a tough financial situation.  There are ways they could use their position to raise money:

  • An operator could charge for root services or adopt the more subtle method of charging for preferred root server access and relegate the rest of us to fight over the left-overs.

  • An operator could mine the incoming query stream for marketing data.  The full domain name being resolved is visible in the queries that go to the root servers, and even though the number of queries that reaches those servers represents a fraction of the total number of queries made by users it still forms a stream of raw data that can be mined using statistical techniques to form a rich lode of data about what domains are of interest and to whom.

  • An operator could sell response rates, much like a search engine sells words, so that queries for sponsored names are given priority over queries for names that are unsponsored.

  • And operator could skimp on protection, backups, and recovery planning.  This is like skipping payments on an insurance policy - it feels like a good idea as long as nothing bad happens.

Or suppose one of the military root server operators received a command from its government, say perhaps, that that government declared itself to be at war with some country or some group of people.  That root server operator would find itself in a position to observe enough of "the enemy's" queries to generate intelligence data.  And that operator would also be in a position to poison the responses to those queries so that, for example, some portion of "enemy" VOIP or web traffic was vectored through a man-in-the-middle that observes that traffic.

Some may consider these scenarios to be hyperbole and unlikely.  But those same people can not deny that what I have said above is possible.

And all of us have observed the unlikely turn into reality.  Take for example the Pacific Lumber Company.

The Pacific Lumber Company is in the business of growing and producing redwood lumber.  The best of this lumber comes from old-growth trees.  The Pacific Lumber Company held a large inventory of such trees and protected that inventory and its market value.  The company cut just enough trees to satisfy the demand of the upper tier of the market.  As a result the company had a good balance sheet with good long-term prospects and a very good reputation for environmental protection.  However the company was acquired via a leveraged buyout - that's a technique that uses the company's own assets to pay much of the purchase price.  The Pacific Lumber Company found itself suddenly having to liquidate its assets to pay for the buyout.  The company swiftly switched from careful conservation to massive clear cutting.  Assets that would have lasted decades or longer and brought top dollar were liquidated as fast as the loggers could cut and sold into a glutted market.

There is no reason to believe that the commercial root server operators are immune to the kind of involuntary reversal of personality such as was suffered by the Pacific Lumber Company.

And there is no reason to believe that the US military won't decide that the US should use all of its weapons, including its root servers, in its wars.

So the question we need to ask is this: How do we institutionalize root server operations so that the community of internet users has the assurance that it will be able to obtain root server services continuously, equitably, and without its activities being observed (or manipulated) for commercial or other purposes?

It seems to me that contracts - clearly enforceable and clearly binding contracts - are the appropriate vehicle.  The notion of contract is, with only relatively minor variations, recognized by every nation on the planet.

We know that in the extreme we can never contractually bind sovereign national governments - or their military operations.  And that may mean that it is time to thank and excuse the military root server operators and replace them with providers who are willing to enter into enforceable agreements.

What should these agreements require, with whom should they be made, and who should be allowed to demand that the obligations be enforced?

I will address these in reverse order:

We want to make the right to require enforcement to be as broad as possible.  Far too frequently people who are affected by a contact obligation find themselves locked out because they lack standing.  For this reason any root server contracts should explicitly recognize that the users of the internet are third-party beneficiaries with explicit powers to require that the parties to the contract live up to their obligations.  There is, of course, a danger that some people could use this right to become nuisances in order to obtain unwarranted settlements.  So some careful thought would be needed when crafting this third-party right.

There needs to be some body with whom the root server operators make these contracts.  I have no clear idea who or what this body is, but I do feel that this body will also need to hold the strings over the contents of the root zone file that the root servers will be obligated to publish.  This linkage to the root zone file is necessary so that the oversight body can exercise final authority over who is and who is not a root server for its root zone file.  My own personal feeling is that ICANN has disqualified itself from consideration for this role.

And finally - what should be the terms in those agreements?  My list is found below.  Most of the obligations in that list are things that the root servers do already; most of the obligations have no affect on current operations.  Rather most of the obligations ensure that the status quo remains the status quo into the future.  I've listed these obligations in qualitative terms; in practice these obligations should be restated into quantitative service level agreements.

  • Servers must be operated to ensure high availability of individual servers, of anycast server clusters, and of network access paths.

  • Root zone changes should be propagated reasonably quickly as they become available.

  • User query packets should be answered with dispatch but without prejudice to the operator's ability to protect itself against ill formed queries or queries that are obviously intended to cause harm or overload.

  • User query packets should be answered accurately and without manipulation that interferes with the user's right to enjoy the end-to-end principle and to be free from the undesired introduction of intermediary proxies or man-in-the-middle systems.

  • Operators should coordinate with one another to ensure reasonably consistent responses to queries made to different root servers at approximately the same time.

  • There should be no discrimination either for or against any query source.

  • Queries should be given equal priority no matter what name the query is seeking to resolve.

  • There should be no ancillary data mining (e.g. using the queries to generate marketing data) except for purposes of root service capacity planning and protection.

  • The operator must operate its service to be reasonably robust against threats, both natural and human.

  • The operator must demonstrate at reasonable intervals that it has adequate backup and recovery plans.  Part of this demonstration ought to require that the plans have been realistically tested.

  • The operator must demonstrate at reasonable intervals that it has adequate financial reserves and human resources so that should an ill event occur the operator has the capacity (and obligation) to recover.

Obligations go two-ways.  The oversight body should ensure that there is wide and free dissemination of the root zone file so that people, entities, and local communities can cache the data and, when necessary, create local temporary DNS roots during times of emergency when those local communities are cut-off from the larger part of the internet.

Posted by karl at July 30, 2005 6:49 PM