CaveBear Blog

IP/TV vs IPTV

It was only a small surprise to see the announcement of an agreement between Verizon and Microsoft to do TV over the net.

But it was a rather larger surprise to see them call it "IPTV".

IP/TV is an active, registered US trademark - but not a mark owned by Microsoft of Verizon.  Instead the mark is owned by Cisco and covers a product line to move TV over the internet.

I know this because I wrote much of the real-time core of that product back in the 1995-1998 timeframe.

I kind'a would'a thought that Microsoft or Verizon would have discovered the existence of the "IP/TV" mark.

So Whatever Happened To .Org Last Year?

On July 1, 2004 the .org TLD stopped working.

The outage was noticed far and wide - names in .org simply stopped working.  The perception of the outage varied from place to place, suggesting that the problem was related to, exacerbated by, or partially masked by the use of anycast technology by PIR's subcontractor.

I noted this event in my blog entry "Did ICANN Even Notice .Org's Problems?"

It has taken several months for PIR, the registry operator for .org, to publish its month report for the period.

The June report from PIR claims that no name resolution outages that month.  PIR's July report indicates 60 minutes of "unplanned" outage.

Was that July outage of 60 minutes the same failure as the one that occurred on July 1?  We can not tell because the PIR July report says absolutely nothing about the date of the outage, its cause, or what corrective measures were taken to ensure that it does not recur.

PIR's subcontractor is associated with a bid for the .net redelegation currently underway in ICANN.  Failures, even unplanned ones, are a fact of life - bad things happen to everyone.  An unplanned outage - that I can forgive.  However an apparent unwillingness to learn from failure or to make the lessons visible to other registry operators - that is not forgivable.  Unless the July .org outage is clearly explained (along with a satisfactory statement of why  a timely report was not made) the operator involved should be eliminated from contention for .net.

ICANN and Its Fairy Tale

ICANN's recent Loyalty Oath, recites the tired old claim that "ICANN is responsible for ensuring the stable and secure operation of the Internet's unique identifier systems".

That claim is false and deceptive.

ICANN might recognize an Internet unique identifier one bit 'em and the IETF wrote an RFC about it.

But ICANN's role with regard to such identifiers is empty and without form.

ICANN does nothing about IP addresses.

ICANN does nothing about domain names except to regulate the domain name business place and define DNS products and prices.   ICANN engages in no DNS technical matters.  ICANN does nothing about DNS security.  ICANN has no role in or over DNS root operations or service standards.

IANA, not ICANN, writes numbers down in the big book of internet protocol numbers, and the IETF instructs IANA what to write.

ICANN's claim that it is "ICANN is responsible for ensuring the stable and secure operation of the Internet's unique identifier systems" is false on several grounds:

ICANN is neither responsible nor does it ensure anything beyond the business profits of DNS registries and registrars and protect a bloated assertion of intellectual property rights.

ICANN has no role with regard IP addresses, the single most important kind of internet identifier.  ICANN abandoned that part of its job years ago to the regional internet address registries.

ICANN has no role over the technical stability over DNS.  The root servers are operated by an entirely independent group over which ICANN has no authority.  ICANN has repeatedly avoided any role in establishing performance or security requirements for DNS root operations.  And ICANN doesn't have the mental horsepower, much less the willingness, to ask question and obtain answers, about changes in DNS operations, such as the deployment of anycast or the introduction of IPv6 records.  ICANN doesn't even bother to ask questions about why major failures - like last summer's outage of .org - have occurred, much less how they can be prevented in the future.

It is unsettling to hear ICANN repeat these falsehoods.  It is unsettling to know that ICANN, which was supposed to be our internet fireman to ensure that our DNS and IP address houses don't burn down has, instead, thrown away its hoses and axes and has decided to turn the firehouse into a stock brokerage.

We have been fortunate that the RIRs and root server operators are competent and that the internet has a lot of resilience.  However, ICANN's abandonment of its role of technical oversight and its assumption of the role of business regulator has weakened the internet and made the internet more vulnerable to failure and attack.

The Non-Commercial User Constituency - probably the smartest and most thoughtful of ICANN's appointed "constituencies" - has been sniffing ICANN's snake oil.  In its draft comment on ICANN's strategic plan the NCUC makes this statement:

At the same time, we recognize the dangers of overly intrusive and arbitrary forms of governmental (or intergovernmental) intervention in the management of Internet identifiers.

While I agree that in the abstract such a concern about governmental incompetence is valid, in the concrete situation of ICANN and its "strategic plan" there is no need to worry: ICANN has so far indicated absolutely no interest in actually engaging in the management of internet identifiers.

It is long past the time when we should be believing the fairy tale that ICANN manages internet identifiers or has any role in the technical stability of the internet.

We should by now all clearly recognize that ICANN has become nothing more than an industrial cartel that has as its goal the management of the domain name industry by defining the products offered by that industry, setting prices of those products, and selecting who may and who may not participate in that industry.

We should also clearly recognize the the ICANN cartel is opposed to any form of innovation that might disturb the profits of the domain name industry or that might tarnish its mutual assistance pact with the intellectual property industry.

Anybody who actually believes ICANN's claim that "ICANN is responsible for ensuring the stable and secure operation of the Internet's unique identifier systems" needs their head examined.

Wow, I Must Be Scary

I notice how much energy the US Government is expending in order to endorse and support relatively open and public elections in Iraq despite the potential that people who oppose the status quo government might be elected.

By comparison I note how little energy the US Government (via the US Department of Commerce and its sub-agency NTIA) have expended to endorse and support the restoration of relatively open and public elections in NTIA's foster child, ICANN.

There are a lot of really scary people - people who might have more than a passing relationship with the kind of nasty folks who shoot guns, fire RPG's, and launch mortar rounds into their opponents or innocents - who could win in Iraq.  Yet the US and Iraq are moving forward.  (We all might want to pause for a moment this weekend and launch into the luminiferous ether a thought of peace and good will with a hope for a stable outcome to the election.)

ICANN, with the backing of the US Government, dropped public elections.  The unstated reason was that they were afraid that more people like myself or Andy Mueller-Maguhn might be elected.  (There is little doubt in my mind that I would have been re-elected had ICANN permitted an election.)

I guess that in the world of ICANN and the US Department of Commerce, the chance that Andy Mueller-Maguhn or I might be re-elected to ICANN is more to be feared than the chance that some unsavory folks might be elected in Iraq.

It is pretty obvious that ICANN's "reformed" board selection process has resulted in exactly what it was intended to do: fill the Board of Directors with quiet timid creatures who are afraid to ask questions, afraid to demand accountability, afraid to focus ICANN, afraid to impose onto ICANN a clear job description, and afraid hold ICANN to that description.  The individual directors of ICANN have made themselves so insignificant that it is hard to remember who they are.

ICANN is in at least as much need of publicly elected board members as Iraq is in need of a publicly elected government.

Trains and Automobiles Do Not Mix

This morning a nutcase in a jeep automobile caused a major train wreck in Los Angeles.

I'm associated with the California Trolley and Railroad Corporation (CTRC).  One of our projects is the restoration of a 1923 mainline steam locomotive.

On February 12, 1937 that locomotive (and the train it was pulling) were involved in a similar wreck - a drunk driver and his wife drove their car onto the tracks and walked away.  The engineer and fireman were killed.  The Interstate Commerce Commission report and photos are visible online.

India is often considered a lesser developed nation when compared to the United States.  However, in India grade crossing - places where roadways and tracks intersect - are relatively rare.  In the United States grade crossings are quite common, and so are the collisions.

311 people died in the USA in 2002 at railroad grade crossings  We are spending millions of dollars per year on worthless security measures at airports; we could get a lot more value for that money by improving railroad grade crossings.  And if trains can run faster and more safely then perhaps more people would get out of their automobiles and we might actually reduce our national addiction to oil.

Wanna Bet He Won't Apply It To His Own Government?

In President Bush's Second Inaugural Address the President said:

"it is the policy of the United States to seek and support the growth of democratic movements and institutions in every nation and culture"

Does he really mean this?  If so then let him begin at home, with his own Department of Commerce, in particular the National Telecommunications and Information Administration (NTIA) and its internet guild, ICANN.

ICANN, as we well know, was established through the efforts of NTIA.  The Department of Commerce has several agreements with ICANN.  NTIA and the Dep't of Commerce yearly, if not more frequently, endorse ICANN's actions and promote ICANN as a model institution of internet governance.  If ICANN is not technically an arm of the Executive (Presidential) branch of the US Government, then it is certainly the well tended fruit of that government.

Those governmental contracts and endorsements have included an acceptance of ICANN's repudiation of ICANN express obligation to bring the public into its decision-making forums and processes.

If President Bush means what he says then he ought to require his own Department of Commerce to bring the principle of "democratic movements and institutions" to the internet and require ICANN to live up to its promises to let the public elect at least half of ICANN's Board of Directors.

The .net Top Level Domain and Cross-Coupled Failures

The .net Top Level Domain (TLD) contains the names of the main group of DNS root servers as well as the names of the servers for several other large TLDs, such as .com, .org, .arpa and .mil.

Most of the focus about the .net redelegation has concerned the quality of the registration systems.  But that is a minor matter next to the quality of the name server operation.  If registration problems occur then the only people affected are those who are engaged in obtaining or transferring a name.  But if the name servers go awry then the entire net will be strongly affected.

Because .net contains the name servers for so many other TLDs, any weakness in the .net servers could sweep across the net like a tsunami.

Perhaps part of the .net redelegation should include an effort to reduce the dependency of other TLDs on .net.  Perhaps the name servers for other TLDs should be moved out of .net 

Yes, this will have an impact on the effectiveness of DNS name compression.  But should we not be trying to reduce the cross-dependencies in DNS and protect against cascading net failures and reduce the interdependencies that could hinder recovery from any major failure?

What Is The Internet Distance From Hither To Yon?

A few days ago I wrote an item entitled How Soon We Forget (Technology).

In the interest of reviving lost ideas I've obtained permission from Cisco to revive some work that I had left unfinished and unpublished.  My intention is to refine and extend this idea and submit it to the IETF.

I was, and continue to be, interested in the issue of network control.  For example see my talk From Barnstorming to Boeing - Transforming the Internet Into a Lifeline Utility (powerpoint)    (speaker's notes in .pdf)

One piece of this continent-sized chunk of internet terra incognita are mechanisms to learn the shape and quality of the pathways through the net.  This information is necessary for troubleshooting, content management and placement, and service level assurances.

It is possible to squeeze a lot of useful information out of tools like Traceroute and pchar, and through inspection of internal and external (BGP) routing information.  (For example, take a look at CAIDA's Macroscopic Topology AS Adjacencies.  But more precise data and more extensive information is needed.

I was intrigued by Pathcar and its sucessor pchar.  But these tools depend on measuring the external behaviour of routers; these tools deeply refine and analyze the noisy raw data and winnow out the information.  Having come from the SNMP community I wondered whether it would be easier, faster, more efficient, and more accurate to simply see if the information could be obtained directly from the routers along a pathway.  The mechanisms used in mtrace struck me as a useful architectural approach.

The result is FPCP - the Fast Path Characterization Protocol.

I stopped work on it back in year 2000 because running-for and being-on the ICANN board of directors began to consume too much of my time.

My stopping point from several years ago is visible at the following URL: http://www.cavebear.com/fpcp/fpcp-sept-19-2000.html

I am going to pick up where I left off and turn the existing draft into a more complete proposal that can be sent to the IETF.  If anyone is interested in helping in this work, please let me know.

SELinux and MovableType

I'm still rasslin' with SELinux on Fedora Core 3.  The latest problem was with Movabletype (I'm still using version 2.661.)

This entry is being written partially to be helpful to others and partially to test whether my hack actually works.

SElinux was not allowing Perl (the language in which MovableType is written) to follow the symlink from /usr/tmp to /var/tmp.

Rather than mucking around with the SELinux permissions I simply went into the MovableType Perl files and changed 'em to use /var/tmp rather than /usr/tmp

This was a change to file CGI.pm in the extlib directory.  I changed the code (near line 25) so that it looks like the following:

# HARD-CODED LOCATION FOR FILE UPLOAD TEMPORARY FILES.
# UNCOMMENT THIS ONLY IF YOU KNOW WHAT YOU'RE DOING.
# $CGITempFile::TMPDIRECTORY = '/usr/tmp';
$CGITempFile::TMPDIRECTORY = '/var/tmp';

Slightly paraphrasing Shakespeare (from "As You Like It"):

   a poor hack, sir, an ill-favoured thing, sir, but mine own

DNS, Apache, SELinux, Fedora Core 3

I ran into a problem when trying to run Bind on as a secondary/slave server on Fedora Core 3.

Fedora Core 3 contains SELinux. SELinux adds an additional layer of access control on top of the traditional Unix owner, group, and world protection bits.

You can run into this problem if you run Bind and your named.conf file specifies a slave zone and also specifies a file into which the downloaded zone file should be placed.

Suppose you have an entry in your named.conf file that looks something like this:

zone "example.com" {
    type slave;
    file "2nd/db.example.com.2nd";
    masters {
        192.168.1.1;
    };
};

You can get an error in your logs that looks something like the following:

Jan 16 20:40:26 p3 kernel: audit(1105936826.400:0): avc:  denied  { write } for  pid=7216 exe=/usr/sbin/named name=2nd dev=md4 ino=491657 scontext=root:system_r:named_t tcontext=root:object_r:named_zone_t tclass=dir
Jan 16 20:40:26 p3 named[7215]: transfer of 'example.com/IN' from 192.168.1.1#53: failed while receiving responses: permission denied

Apparently someone decided that the downloading of zone files can be a security hole and should be opened only through the explicit action of the system administrator.  Unfortunately nobody bothered to tell the system administrator how to do this.

Here's how I fixed the problem.  As superuser (root) I ran the following command:

setsebool -P named_write_master_zones true

While on the subject of SELinux, I also ran into another access failure when trying to put my web server's DocumentRoot onto a mounted filesystem rather than in /var/www.

In my case I created a new RAID volume that I mount as /vol0 and wanted my DocumentRoot to be in /vol0/www.

I had to run two commands (as root) to establish the correct security contexts.  The second of these commands was needed because the root of that file system was unlabled, i.e. had no security labels.  I had to use the form of the command with the elements of the security context joined by colons rather than using the -t -r and -u options because the chcon command doesn't realize that it has enough information to go forward when all three options are present.

chcon -R -t httpd_sys_content_t /vol0/www
chcon system_u:object_r:httpd_sys_content_t /vol0/

I don't know if these are the proper solutions, but they did seem to work.

How Soon We Forget (Technology)

I just got home from a very snowy (nearly 3 meters of snowfall in 3 days) weekend up in the Sequoia National Park - where the really big trees are.  The world's tallest species of tree grows along the California coast, including around my house in Santa Cruz.  But the biggest species of tree, in terms of mass, lives up in the Sierra Nevada, particularly in Sequoia National Park.  And when I say "big", I mean big.  Here's a photo of my wife and her Christmas tree.  It probably sprouted while Rome was an empire.  And it's not nearly the largest, or oldest, tree in the vicinity!

These trees exist because they (or rather their ancestors) are highly successful innovators.  When these trees come up with a useful evolutionary trait they don't shoot themselves in the foot (root?) by discarding the new trait simply because it might change the status quo.

On the other hand we humans tend to institutionalize the old and endow it with an enormous power to resist change.

When I got back from the mountains (and snow) I read with dismay of the continuing Media-Luddite attack on BitTorrent.

BitTorrent is the latest target in the sights of the movie and recording industry.  I'd hate to see BitTorrent be squished - I routinely use it to download open source distributions.

BitTorrent is an interesting bit of technology - it operates through a kind of mutual assistance between receivers: a receiver that has obtained a file block from the originator makes that block available to other receivers, thus reducing the burden on the originator and also spreading the traffic load across the net so that there is no traffic hot-spot focused on the originator.

In the IP multicast world reliable multicast streaming and file transfers have been around for decades, including protocols that use mutual assistance among receivers.

But IP multicast has become a lost technology - the MBONE, a worldwide multicast overlay on top of the internet, has faded.  And cross-ISP dissemination of IP multicast routing information never really happened.

There has been talk about the development of new media distribution networks based on BitTorrent.  It would be a much more efficient use of net resources if that media distribution were to occur using IP multicast.

Could it work - yes.  How do I know this?  Because back in 1995 we built it and it worked.  We could efficiently move DVD quality audio/video over the net, even over multi-vendor, multi-provider networks.  And yes, even in that earlier and more innocent era we did try to build-in mechanisms to honor the rights of copyright owners.  (By-the-way, the first movie we watched over our lab net was BladeRunner off of one of those now ancient LP sized video disks.)

IP multicast is not the only lost technology.  A once promising technique for reliability and security, capability based hardware and operating systems, also seems to have faded.

I have my own favorite lost technologies:

• A video conferencing system that begins with an exchange of photos of the participants in a set of standard poses.  Once the conference begins, rather than actually moving images of the participants, morphing directives are sent so that the pre-exchanged poses can be morphed in real-time to mimic the actual motions of the people.  The system could be tweaked so that a participant could give the morphing commands a kind of stereotypical inflection: in the Italian dialect the morphing would be extravagant while in the English dialect the morphing would be reserved.

• A resource advertisement and detection system that uses "network pheromones", a kind of packet containing resource advertisements that is wafted, aggregated, and relayed around the net.

So what are the points I'm trying to make?

First: Innovation isn't just the art of the new idea, it is also the resurrection of the old and seemingly lost idea.

Second: New ideas must be given a fair chance to succeed and not be nipped in the bud simply because some pre-existing interest claims that it might be harmed.  (See my proposed First Law of the Internet.)

Spam Load

On my most heavily instrumented domain the incoming e-mail is now 97% spam.

Most of my domains are under constant Rumplestiltskin attacks (email containing addresses to hundreds and even thousands of possible names).

My sendmail backlog (3 day timeout) of bounces/double-bounces is typically on the order of 10,000 pending items per domain.

There are an increasing number of obvious zombie machines that send me the same joe-job mails once a day every day.

I see no reason to expect the situation to improve and see many reasons why it may get worse.

Thoughts For the New Year.

My iPod selected an interesting way to begin the year: Virgil Thompson's suite (Stokowski conducting) from the 1936 film The Plow That Broke the Plains.

I've owned many copies of this music through the years - going back to an album of 78 rpm disks.  And from these copies I've learned the difference between conductors (such as Stokowski) who can bring out the emotion of a piece and perform a work of art and those (Mariner) who boil it to death and produce meaningless, boring mush.

This film and this music are the result of government.  The world is a better place because the government of the United States commissioned these works.

In the film we see the combined activities of private farmers plowing, spoiling, and eventually destroying, the great plains grasslands of the United States.  It is no great stretch to imagine ICANN - an epitome of private action for the self-centered benefit of a few industrial actors - driving those tractors across the plains of the internet; plains that are still fertile but whose soil for innovation is drying up and blowing away with each seasonal plowing.

It is simply wrong to think, as many today do, that government is a bad thing.

Yes, there are bad governments - the US Department of Commerce has demonstrated how easily a governmental body can replace responsible decision-making  with dogma.  The internet is far less stable and more vulnerable today than it was before the US Department of Commerce came aboard in 1997 and created ICANN as its private, secular arm.

Despite the few rotten apples, government is where the aspirations of a nation come together, in the words of the US Constitution, to "establish Justice, insure domestic Tranquillity, provide for the common defence, promote the general Welfare, and secure the Blessings of Liberty to ourselves and our Posterity".

2005 is the year in which the combined governments of the world intend to act on questions of internet governance.  This is a matter that transcends the internet; what is done with regard to the internet will establish the model for what will occur as the clarity of national boundaries and national powers continues to erode.

There are many things that are polluting the debate.  ICANN has adopted a stance in which its own self-preservation supersedes truth and the actual stability of the internet.  The US Department of Commerce continues to promote an obsolete Reaganesque view of Corporate America First.  Voices from smaller countries are afraid to speak for fear of being derided as "clueless" because they are not fluent in the vernacular of internet technology and are unwilling to require that internet technologists state their assumptions and values, disclose their logic, and separate those aspects of their conclusions that are disguised social policy making from those that are actually technical choices.

Eric Hobsbawm, in his book The Age of Extremes : A History of the World, 1914-1991, argues that the system of capitalistic private enterprise nearly collapsed during the 1930's just as the USSR's rigid systems of centralized economic planning eventually did later in the century.  He goes further to argue that what we have today is a blended system - unconstrained private enterprise has been moderated by government - through laws and central planning and administrative bodies (such as the US Federal Reserve and US Environmental Protection Agency.)

That kind of blended system is not "a public-private partnership" for the simple reason that it is in the nature of private enterprise to promote its own economic self interest as far as is allowed by public law and public regulation.  Public and private interests are in opposition.  The public sector is not the partner of private enterprise but, rather, is its keeper.

The worldwide debate on internet governance should not fall victim to the false idols of "private governance" or "public-private partnership".  Just as the private farmers of the 1930's turned the grasslands of the United States into "the dustbowl" through the cumulative weight of private choices, the internet is in danger of becoming a wasteland of industrial strip malls if its governance is left in the hands of those who respond to private interests rather than those of the community of internet users.