I had the great pleasure of being in DC yesterday for the inauguration (though I chose to watch it inside on a big screen tv). Obviously, there will be huge changes in how our government works and acts and speaks. One small sign of this is on the web where the Obama administration just unveiled the all new WhiteHouse.gov, the official website of the White House.

Web designer Bryce Tugwell noticed a small, technical change to the WhiteHouse.gov website that in effect sums up a lot that will be different with the new administration:

Just as an additional side note on this conversation [about the new site], here’s a small and nerdy measure of the huge change in the executive branch of the US government today. Here’s the robots.txt file from whitehouse.gov the day before yesterday:

User-agent: *
Disallow: /cgi-bin
Disallow: /search
Disallow: /query.html
Disallow: /omb/search
Disallow: /omb/query.html
Disallow: /expectmore/search
Disallow: /expectmore/query.html
Disallow: /results/search
Disallow: /results/query.html
Disallow: /earmarks/search
Disallow: /earmarks/query.html
Disallow: /help
Disallow: /360pics/text
Disallow: /911/911day/text
Disallow: /911/heroes/text

And it goes on like that for almost 2400 lines! Effectively locking search engines out from the vast majority of the site.
Here’s the new Obamafied robots.txt file :

User-agent: *
Disallow: /includes/

That’s it! BTW, the robots.txt file tells search engines what to include and not include in their indexes.

Bryce