15KB of Fame

How To Save The Web

2008-08-26T02:29:00.012-04:00

There is no such thing as public record on the internet. Maybe that sounds strange -- the internet is the most public thing around. But consider this: it would be impossible for, say, the New York Times to change or obliterate something it printed in 1997. There are hardcopies all over the world. But for nyt.com it's as simple as a mouse click, and it happens all of the time. The internet we have today is public but it's not really a record.

This is pretty important. Public record, the ability to prove that someone said something on a certain date, is the basis of a free and literate society.

It doesn't have to be this way

Some disappeared things that got noticed.

sports.yahoo.com ...a New York computer security expert who found official Chinese documents that list He's age as 14 years and 220 days... The spreadsheets were taken down off the site recently and He's name had been removed...

www.talkingpointsmemo.com "When we went to the page for the photograph of President Bush and Abramoff, the page in question had disappeared from the site..."

perezhilton.com "Pinksky's photo disappeared from the hospital's site on Friday, as the scandal story started to get more legs..."

lots more...

For historical reasons, stuff published on the web lives primarily on servers controlled by the original publisher. This means that if they go away that data does too. It also means that they get to choose how it is accessed. Every time some politician is disgraced, his or her name is removed from the website of any organization that can't afford the embarrassment. When a publisher announces that all back issues will go behind the "pay wall", behind the wall it goes.

Revisionism is sometimes risky. If you are caught in the act it only takes a few people to make sure copies pop up like dandelions. But these cases are the exception and I think Cory Doctorow is far too smug about this point. The daily loss of data is much greater than what bloggers can hope to rescue. And what is important to us now may not be the bits our descendants care about. Archaeologists learn more from garbage dumps than from Genesis.

There are a few coherent public archives, for example, Yahoo's and Google's cache and archive.org. But Google's cache exists for Google's purposes and it is not designed for the long term. Archive.org has two handicaps: a centralized cache of the web is very expensive to maintain, and they are forced to take down stuff all of the time. We need something that can scale, something both lawyer- and disaster-resistant.

The good news

Isn't it interesting how many of these problems are solved by having lots of copies in lots of places?

In a sense this archive already exists, though in a low-energy state. Part of it is in your browser's cache right now. These caches are not coherent, organized, searchable, or public. They also have a lot of stuff in there that is better left private. So we have to work around that. But it's a start.

What the Archive should be

Decentralized & Redundant
Centralized is too expensive and too fragile. Redundancy increases the odds of survival.
Long-Term
If it's not long-term there's not much point. Open, stable formats.
Locally curated
There should be at least as many opinions about what should go into the archive as people using it.
Public & Coherent
A cache is useless if no one can get to it. It should also contain only things that are already public.
Verifiable
Whether by digital signatures or by comparing copies, or both, the archive must be resistant to tampering.
Useful for User 0
It has to be useful even if you are the only user, otherwise there is no incentive to keep the flame alive.

Fellow Travelers

One way to do this is with desktop software. My old project Dowser was an attempt to make personal archiving easy for the average user under the guise of a "research tool". I'm working on a new version called Dowser2. If you want to help, drop me a line at my first name at bueno dot org.

Network Distance

2008-06-09T17:04:00.009-04:00

When Sydney is closer than Sao Paulo

It's quite possible to make your living from the internet without really considering how it's constructed. I came across this talking with my friend Aaron. He's a bright guy but like many people, assumes the net is done "with satellites or whatever", or never think about it.

The physical structure of the internet very similar to the global air network. Picture those glowing arcs connecting cities on the back of airline magazines. Then recall the hour you spent in line before boarding, the hour spent driving to the airport, the two hours from the big hub airport to the smaller city you really want to go to, and how relieved you are to be traveling now instead of last week when a storm in Chicago somehow messed up flights to Los Angeles. That's basically how your data feels, too.

The long-haul internet is, in fact, a series of tubes. Very fragile tubes.

The long-haul cables that run the global economy are less than 2 inches in diameter, buried under railroad tracks, highways, or ocean sediment. Within densely-settled areas the network is relatively redundant. But between countries and across oceans, data flows through an uncomfortably small set of bottlenecks.

Any minor disaster can damage large portions of the 'net. I recall one incident in 2004 when Miami/Sao Paulo traffic was suddenly re-routed through Washington DC, then New York, then across the ocean to Brussels(!), then back to SP. As late as 1998, a train wreck or wildfire in northern Florida could cut off the whole peninsula for days. This year one of the most exposed points, Suez, suffered multiple failures. The re-routing flooded the already overloaded Europe-Asia network for several weeks.

Even on a good day you can see the problem. Look at how a packet of data might travel from San Francisco to Hong Kong:


(start)
Folsom St, San Francisco (1 mile)
Pine St, San Francisco (2 miles)
Pine St, San Francisco
Oakland, CA (10 miles)
Sacramento, CA (80 miles)
San Jose, CA (120 miles)
Oakland, CA (40 miles)
San Jose, CA (40 miles)
Hong Kong (7,000 miles)
(finish)

That poor little packet of data rattled all around California, looking for an uncongested cable over the Pacific. For each hop, a decision is made to send it on to some other place that may have better luck. The system works pretty well under stress. The point is that the stress is there all of the time.

The long-haul internet is a map of trade volume between cities, that lags up to 20 years behind reality.

This is actually true for all forms of high-volume transport, so there is a lot of history to learn from. Infrastructure is insanely expensive and slow to build even though it almost always pays off in the long run. Short hops between financial/military/political/industrial hubs tend to get built up first. Just look at how many ways there are to travel between New York, Boston, and DC, for example.

Or look at area codes. At the time the precursor to the modern phone system was built, dialing a 1 took 1/9th the time of dialing a 9. Silly, but true. So there was a premium on lower numbers. New York's area code was 212, DC 202, Los Angeles 310, Chicago 312. El Paso, Texas? 915. Anchorage, Alaska? 907. Miami was definitely not a hub at the time, but it was important to the Navy and Air Force. Miami's code is 305.

So if several factors of demography, geography, and politics align, there may form a route of sufficient capacity between two points. If not, too bad. It takes years to build up demand, more years to begin the project, and more and more years to finish it. There are bribes, labor riots, sabotage, political chicanery, etc, and that's when it's a good idea. The first trans-continental rail link in the US was completed in 1869. Twenty years earlier, people had been hijacking ships in Louisiana to sail around South America and crash-land ashore at San Francisco. Other people walked.

Almost as often you'll see huge infrastructure built for political reasons. For example, there is a highway that runs directly from Orlando's airport to Disney World and Cape Canaveral. In those cases trade can grow because the infrastructure is already there.

So what does this have to do with the internet? Quite a lot. The same drama plays out when cables are planned and laid. The connection between Seattle and Tokyo is excellent. Ciudad Mexico and Dallas? Fairly new and really fast. New York to London? World-class. But try to get an email from Barcelona to Bangalore, and often you'll find that it routes through America. Companies are scrambling to build up Europe-Asia links. They've been scrambling since the late 1980's, and it will be some time before they get there.

The long-haul internet is not a magic leprechaun.

Let's imagine a perfectly balanced world-wide network. You have a business based in San Francisco. Your people are in SF, your technology suppliers are in SF, most of your customers are in the United States. You have a small but growing customer base in Hong Kong and China. We have a perfect 'net, so there are no silly congestion problems and there is just as much bandwidth across oceans as between cities. Rack space in HK is twice the price as in SF.

So what is the no-brainer place to put your next server farm, hire people to maintain it, set up office space, pay property and business taxes, etc? Correct. Hong Kong. No matter how good the internet gets it will never be faster than a small fraction of the speed of light.

Recently I was looking at the server logs of a site located in the US. The response times had a very high variance, which indicates a severe bottleneck somewhere. After a lot of poking around, trying to find the problem within the server farm, I had the bright idea to segment the logs by source country. And there it was: the response times were all over the map because the users were all over the map. From the perspective of the 'net, Stockholm and Singapore are just down the block while India and Sao Paulo are past the moon.

Rules of thumb:
- Light takes about 100 milliseconds to travel 10,000 miles and back.

- A network packet on a good route may take 3 to 10 times that long.

- A network packet has not "arrived" until a return acknowledgment has been sent all the way back.

- The longer and more complicated the route from here to there, the higher the chance (often more than 10%!) that a packet will get lost.

- The larger the file you send, the more packets it has to be chopped up into, the more likely one will be lost and have to be re-transmitted. All else being equal the transit time of a file increases more than linearly to its size.

Evolved Code vs Enterprise Code

2008-02-01T20:51:00.000-05:00

Once upon a time, Adrian Thompson accidentally created a magical artifact. He used a genetic algorithm to evolve a FPGA circuit that would distinguish between signals of two frequencies. The winning specimen was pretty unlikely: it worked on that chip only, and only at certain temperatures. It contained circuits that were not connected to the others, but when he turned those off the damned thing stopped working. The neighboring gates depended on that extra bit of inductance, which varied with a slight flaw in another gate... maybe. No one really knows.

Once upon a time, a small group of hackers evolved a piece of software. They focused so much on doing more with less that they themselves became crucial parts of the implementation: Leo took over ops because he stayed late to avoid traffic anyway. These support emails routed to Bianca because she spoke Spanish and Portuguese. Tatiana had a nicer touch with angry customers. Carlos had the pager because he's good at troubleshooting. Tom made the changes to the template parser because Tom's the genius who wrote it. The bug tracking system was a whiteboard. Every little detail seemed to depend on some idiosyncrasy of the people or their environment. It worked -- somehow. Every startup is a freak accident.

This is why I think Jaron Lanier's bomb-throwing about closed- versus open-source focuses on the wrong thing. When you're talking about "radical creativity" in a program, it doesn't matter how many people are allowed to read the source code, but how many are required to understand it. Having too many junior hackers on your source tree is just as bad as too many customers pestering you about compatability.

The fundamental difference between the first two systems and the ones that come out of large organizations is that there was no difference between design and implementation. Enterprise code is designed, vetted, reviewed, implemented, deployed, life-cycled, etc. The purpose is to have no irreplaceable parts. On the other hand, the magic chip and the startup were evolved as fast as possible in a way that could never be exactly repeated.

When an enterprizey programmer and an alpha-hacker look at each other's code, the same thought occurs: "This is a rambling mess! How do they get away with this crap?" The documentation is the code, alpha-hackers say. Don't you see that you're writing your programs in PowerPoint? The process gurus grumble about "heroics", maintainability, knowledge transfer, bus accidents. The enterprizey people regard design as something that must be as separate as possible from the expression-as-code. How else can it be maintained by anyone? Take away Tom and the thing no longer works. True, but if enterprizey was always better, BigCo wouldn't get their lunch eaten by wittle bitty startups.

It gets more fun when the big ones turn around and eat the little ones. Most startup acquisitions play out like a spider dissecting a fly: gruesomely impersonal and tedious. You just can't get over the image of that fat spider pick pick picking away at a violated husk. Yahoo's epic rewrite of RTML, or its painful digestion of my company's system, are the same phenomenon as that spider, or that poor bastard trying to understand his magical chip. There are bits that work only there & then, bits that may be impossible for an outsider to grok completely, bits that even the same people would do differently now.

It's the Economy, Stupid.

2007-12-03T12:11:00.000-05:00

At least 50.7% of the voters in Venezuela would rather not give their president dictatorial powers and a shot at life in office.

The "socialist paradise" bribe in the constitutional referendum was transparent: a 6-hour day and some worker's benefits in exchange for astonishing new powers for Chavez, including no term limit and an extension of the term from 6 to 7 years. The 7-year thing really bothered me. The only reason I can think for it is that, being a prime number, a 7-year term would put him out of step. At the times most other leaders would be sweating re-election he would have lots of elbow room.

But Chavez was out of step on the economy. The bribes were to be funded by oil exports, which are strongly linked to the dollar. With the dollar going down, there is enormous inflation pressure in oil-producing countries. Chavez' answer to that was price and currency controls. The official rate is 2,200-odd Bolivares to 1 Dollar, with tight limits on how much you can buy or sell. As of last week the black-market rate is over 5,000:1. To wish inflation away he put in price controls on basic foodstuffs, with the unsurprising result that there is less and less stuff on the shelves.

Just as Solidarity in Poland was partly fueled by the price of sausage, the NO campaign got boost from the scarcity of Doña Arepa. You can shut down the TV stations, you can shoot the protesters are arrest the "traitors", but you can't monkey with the price of milk.

Almost Famous

2007-11-01T18:09:00.000-04:00

Check out this cutie: Linda Goodwin-Nichols

Ms. Nichols is the Mayor of Kissimee, Florida, a real estate agent and a local philanthropist. She is what we at Spock call a semi-famous person. Her data was automatically picked out from thousands of "Linda Nichols" pages and assembled from over 25 separate sources on the web. Pictures, co-workers, political contributions, fax number, mailing address, press articles, etc etc and so forth. A trained researcher at a newspaper would take hours to gather this much information.

Forget about trawling social networks and Wikipedia. Tagging your friends is fun but not very useful outside of your own circle. Famous people are easy because there is so much unambiguous information about them. Regular joes like you and I are easy because there is so little. The real challenge and value of "people search" is getting this level of depth for the several million people who are somewhere in between.

Persai: failed in the cradle

2007-09-03T02:41:00.000-04:00

Persai is Ted D's new startup. Details are still sketchy, and will remain so until these e-tards finally wither away. The basic idea is that Persai will suck on everyone's RSS feed pipe, swish it around and then spit it out. The killer feature is "recommendations", probably based on a classifying algorithm they cribbed from an old IR paper. So not only will it suck for user #1, it will suck for every other user until their personal filter is trained.

A shitty idea deserves a shitty implementation. The whole thing depends on Rails plus a wobbly collection of Java search-engine-y stuff that keeps changing its name and is nowhere near as efficient or reliable as the Google software it tries to imitate. Bonus: Persai will be hosted on Amazon's EC2 for maximum falability. Read my lips, guys: $73/month plus bandwidth for a lamed webserver with no N-O S-L-A. This is what happens when math nerds-turned-Google interns design a server farm.

Congratulations! Your company is pursuing a

(X) pseudo-technical ( ) web 2.0 ( ) ad-based ( ) acquisition-based

approach to making money. Your idea fails. Here is why it fails. (One or more of the following may apply to your particular idea, and it may have other flaws which we couldn't be arsed to point out.)

(X) 80% of blog pages are robot-generated rips and spam
(X) of the remaining 20%, 95% are metoo posts
(X) "Authority" (i.e. "groupthink") is easily gamed and useless in any event
(X) storage and bandwidth are cheap, but not that cheap
( ) moving data to the CPU is still not cheap
(X) No one will pay to use this
( ) Google will not buy you
(X) Nicely-organized shit is still shit
( ) A dating site that actually works means no repeat customers, dummy.

Specifically, your plan fails to account for:

( ) Lack of central controlling authority for blogs
(X) Asshats
( ) Fakesters
(X) Providing actual value
(X) DMCA takedown notices
(X) Criminal investigations & liability
( ) Thundering Herds (not that you have any users)
(X) Scalability
( ) Hyperventilation about paedophiles/kiddie porn/bomb instructions
(X) Undocumented, alpha-quality, Open Source knock-offs of Google infrastructure
(X) Continued availability of Amazon compute services
( ) Languages other than English (you provincial e-tard)
(X) Extremely low marginal cost of blogspam
(X) Clickfraud
( ) Inherent failure of geocoding IP addresses
( ) Armies of worm-riddled, broadband-connected computers
(X) Eternal arms race involved in all filtering approaches
(X) Unpopularity of weird new search engines
(X) User Fatigue And Internet Login Syndrome (UFAILS)
( ) No one wants to be on the same social network as their mom
( ) Extreme stupidity of the internet public
( ) Extreme cupidity of the internet public
(X) Badly-coded blog readers
(X) Badly-coded feeds
(X) False positives that result in the replication of spammy data at your expense. (Thanks, asshole!)
(X) Search Engine payola
(X) Facebook

and the following philosophical objections may also apply:

(X) Ideas similar to yours are easy to come up with, yet all have failed
( ) Blacklists suck
( ) Whitelists suck
(X) PageRank-style ideas are patented, and also suck
(X) Clustering algorithms run in k^2 time or worse, and, furthermore, suck
( ) Other people should be able to talk about Ron Paul without us having to hear it
( ) Profitability should not involve AdSense
( ) Profitability should not involve banner ads or affiliate programs
( ) Userbase growth should not involve spamming address books
( ) Searching should be free
(X) Why should we have to trust you and your servers?
(X) Why should I listen to your sucky algorithm instead of my friends?

Furthermore, this is what I think about you:

( ) Sorry dude, but I don’t think it would work.
( ) This is a stupid idea, and you’re stupid for suggesting it.
(X) Nice try, assh0le! Go back to wanking on 4chan.

Think Twitter is frivolous? Ask the protesters in Venezuela.

2007-06-01T12:11:00.000-04:00

Twitting your bathroom breaks is frivolous. Like the user who used reddit to report on last year's coup in Thailand, the kids on the streets of Caracas who are protesting Hugo Chavez have a real use for it. They are using Twitter, SMS, camera phones, etc to coordinate, keep people safe, and get the word out as things happen on the scene.

Periodismo De Paz (Spanish)

Update: Sis reminded me about the Usenet posts from "kremvax" during the Russian coup in 1991.

Helprin Wants Eternal Copyright

2007-05-22T11:54:00.000-04:00

I have to respond to this. Mark Helprin, an author and occasional pundit, has really jumped the shark in his New York Times Op-Ed about copyrights (cached). He is seriously (though not too intelligently) arguing for eternal copyright protection.

Let's skip over the emotional appeals in the essay, the strange images of a copyleft junta that steals yachts from the mouths of children yet unborn. Helprin has made two errors in his thinking. One is that real property can be equated with other kinds. The other is that ideas can be easily separated from works.

The term "intellectual property" has no legal basis. There is only real property, trademarks, patents, and copyrights. Trademarks are already treated almost the same as real property (with good reason), and he does not seem to be too interested in patents, so let's stick to copyright.

A story is demonstrably different from an office building: I can copy a story with no material loss to the original copy. It is impossible to "own" or "steal" a copyrighted work, in the sense of denying others the use of it by possession. Thus most real property laws simply do not apply, and that is why they are treated differently under the law. A story cannot be contained, moved, hidden, or destroyed. Pretending otherwise is as silly as charging Xerox or Sony with grand theft. (Which, by the way, was actually tried by numbskulls such as the late Jack Valenti.)

Helprin claims that this is unfair. Ford can keep his car factory for as long as it stands. Ay, but there's the rub. Ford owns real things that must be staffed and amortized and upgraded for them to produce more value. A story has no such pesky problems -- it is indeed forever. By his own logic, wouldn't eternal ownership of maintenance-free, non-material, value-producing things be unfair to owners of material ones?

On to the second assertion: "ideas are immaterial to the question of copyright." Pardon? An entire phylum of lawyers lives off of derivative-work litigation. The separation of "idea" from "work" is a matter of judgement (ie expensive trials), not law. I invite Helprin to invest his hard-earned cash in a cartoon featuring a happy-go-lucky mouse in red shorts. I think it'll be a winner.

His conclusion that "no good case can exist for treating with special disfavor the work of the spirit and the mind" is insupportable and false-to-fact. Copyrights and patents already have extraordinary monopoly protection --enjoyed by no other kind of work-- as compensation for their immateriality. Simply by breathing an author gets exclusive right to exploit the industry called copyright, backed up by the full might of every Berne Convention government. To balance this enormous grant of power against the needs of our shared culture, the culture that enriches all of our lives, it is limited in time.

Now we have some deathless, rapacious corporations that would like to have this protection forever, to hold a veto on our culture unto the seventh generation. These same corporations take the lion's share of profit from authors, and then enlist them in their cause! The arrogance of Ozymandias was a rabbit fart in comparison.

The Financial Times weighs in savagely.

Lawrence Lessig's copyleft hordes do too.

Software for the World

2007-04-06T13:05:00.001-04:00

Q: How many Californians does it take to change a light bulb?
A: Just one. He holds it up and the world rotates around him.

In the US border states there used to be a phenomenon called "sombrero politics". During election season the gringo politicians would go to the Mexican parts of town, put on a gaudy sombrero, eat a burrito and give a short speech (in English). Then they would leave until the next election season. That's about where we are now with software. We take a bit of time out of our busy lives, code some functions, test the Unicode support, and go back to the important work.

When a typical US programmer thinks of "internationalizing" software, he does it superficially. Porting your software to another culture is much more complex than porting to another OS or browser. Like the plates of the Earth's crust, language and mythology sit unseen underneath, shaping the mental landscape.

Language indicates profound differences in how people think. For example, there is no satisfying equivalent for the Spanish word "travieso". It denotes a young, puckish, clever, likable little bastard. The trickster archetype is an important part of that culture; in English there is only a fuzzy cloud of adjectives ^[1]. There are emotional states that would take a dozen words to translate. Time is expressed as a quantity; the Anglophone time-as-distance is slightly weird. Compass directions (i.e., "turn North") are deeply weird and complex. The 12-hour clock and mm/dd/yy format are as archaic as feet and pounds. Why are they the default in so much software? Cultural bias in the programmers.

Bias seeps in everywhere. I grew up in the desert. I don't think much about seafood. If I were creating a recipe site I would lump all seafood into one category and put beef, lamb, etc into their own categories. To me, fish is fish. To someone from the coast that's backwards: meat is meat. Fish is one type, shellfish another. There are mussels, lake fish, river fish, seawater and briny. That is a cultural disconnect at work. If only desert-people wrote software, their view would appear to dominate even though there are many more people who live on coasts.

The LikeBetter guys have a test that compares you against all the YCombinator startup founder's scores. From a series of pictures, it inferred a lot of things about me: I am a female who doesn't like spicy food, is uninterested in politics and who doesn't like pranks. Wrong, wrong, wrong and wrong. What I really don't like is snow, and 7 or 8 of the pictures featured brain-damaged people up to their hips in that stuff. Without feedback likebetter is just shouting in an echo chamber.

They got my arrogance right, at least.

Add all these differences up and you get a kind of cultural accent. As with verbal accents, the thicker it is the less people take you seriously. You might still succeed internationally but only by chance or lack of choice. The most popular chat software in South America and much of Europe is MSN. Do you know why? MSN doesn't either. Google's social network, Orkut, was flooded by Brasilians in 2004, even though the site was in English. Even the Brasilians don't agree on why. But consider that Orkut was invite-only, that three big signs of cultural status in Brasil are sexuality, popularity, and connections with the United States... and there are a lot of brasileiros working for Google ^[2].

US programmers often reply that it's the features that matter, not the funny squiggles over the vowels. Business folk will come back with the size of the US market. But that's short-term thinking. Treating entire continents as rounding errors only works when you are the only game in town.

^[1] A reader suggested "rascal", which captures the flavor very well. Try to remember the last time you've heard that word in casual speech.

^[2] Do you know who is big in Japan? Pat Metheney, Frank Zappa, and all the outré jazz composers you would only know if you are a hardcore music geek. In this case it's not US cachet. Experimental jazz has never been widely popular in the States. The conventional wisdom in music schools is that average ear doesn't get it, that Anglophones focus on lyrics over melody. But the average Asian ear is trained from birth on tonal languages in which a slight inflection changes meaning.

Also: Globalization Test

Globalization Test

2007-04-02T10:00:00.000-04:00

If you write software assuming any of the following:

English
Acronyms
Fixed timezone offsets (pssst... it's summer in Brasil and it's tomorrow in Australia)
"AM" and "PM"
Knowledge of US geography (Quick -- name all of the Canadian "states")
Latin-1 or ASCII-7 (We'll just add the funny things over the vowels later)
US-centric telephone & address formats (You mean it's not called a Zip code?)
US-centric methods of payment
US-centric legal frameworks (You can drink at 16 in England?)
Feet, Pounds, Fahrenheit (How's that Mars probe doing?)
Commas for decimals and periods for thousands-markers
"Soccer"
"Foreign" or "International" anything

...report to my cube immediately for re-education with this baseball bat.

Why Popular Sites Are "Ugly"

2007-03-22T18:25:00.000-04:00

It seems as though popular websites, books, songs, movies, etc offend a lot of principles of quality and taste. What's more, it seems to be by design. It is. The funny thing is that this is exactly as it should be. It can't be any other way.

The reason is that people are more different than you usually like to think. The form of a message has to be tuned to the intended effect, to the semantic receptors of the audience. When your audience is large and the common thread between them so thin, the form of a successful message becomes self-parody, an appeal to the basest desires. Nothing escapes this rule. Just look at the highest of highbrow-but-popular stuff (say, Shakespeare or Cervantes) and count the profound ideas shoved next to diarrhea jokes.

I spent a few years designing ads for the phone book and direct mail. It's a no-nonesense business. Personal aesthetic takes a backseat to performance. In a way it's great: you have a single test of effectiveness (i.e., the phone rings or does not ring), reinforced by fast iteration and feedback. Direct Marketing is a real-world example of design by genetic algorithm.

This kind of design does not always produce ugly results. It seems that way because, by definition, you see more wide-audience-targeted material than other kinds. Yachting and Spa ads are pretty posh but odds are good you're not on the list.

Nor is popularity a good measure of quality. The books Tom Sawyer and Huckleberry Finn have the same characters, same kinds of silly scenes, pirates, farce. Yet Sawyer is a kid's book while Finn is a masterpiece. The difference is the stuff Mark Twain wove in between the crowd-pleasers.

I sympathize with esthetes. I am am one. But if you are trying to make a popular website, remember that you are a commercial artist. Commercial art is the most demanding because of its constraints: the worst being that you do not have final say. The audience does. Would Monty Python be as popular as it is without the flying sheep or the road-testing they did on each sketch?

Also: An essay on Whitespace by Mark Boulton

Weird things you must believe in to be an applied mathematician:

2007-03-22T09:41:00.000-04:00

1) Any theorem that is disproved by contradiction must be thrown out.

2) ...except for Bivalence and LEM even though Russel & Gödel kicked the bottom out of them three generations ago. It's just too scary.

3) Probability, that fluffy blue blankie we use where bivalence breaks down, is axiomatic even though it produces no significant results.

4) A logical tautology cannot also be a factual truth.

5) ...but the Principle of Induction always holds, except when it doesn't, and when it doesn't it means there is something new to learn. Why? Because that's the way it has always happened in the past.

Bueno's Law Of Geocoding:

2007-03-19T15:48:00.000-04:00

The accuracy of a given company's mapping application is 100% for the company headquarters, greater than 75% for the homes and hometowns of company employees, and undefined everywhere else.

Startup School & Startup Culture

2007-03-09T10:39:00.000-05:00

Well, I've made it into Startup School this year. I'm genuinely excited, not just for the speakers but to meet the other 649 hackers.

I'm curious to meet Paul Graham in particular. Over email he's always gracious & thoughtful. But --and this is a little seed of a thought that's been growing for a couple of years-- I wonder about the effect he's having on the industry: whether YCombinator is doing good for the art of software as it is for the business.

Take music, the other creative pursuit linked with garages. People learn it on their own or at expensive schools. Groups condense out of the social soup of young people, and it tends to clump in certain cities. They have goofy names. They do it because they love it. The best are able to work magic. A lot of them fail, some make a decent living, a very few become rock stars.

That's the myth. But that's not what actually happens. In the background are competing factions of promoters, managers, coaches, ghost writers, hired experts, financial backers. Without the cooperation of this network your chances of making it are thin enough to cut your wrist. Without them the Beatles would have just been a really good bar band. Generations of jazz and blues geniuses died in poverty because their looks were not marketable. Talent is not closely correlated with success.

So ask yourself: Who makes money in this regime? The backers, the promoters, the hired guns, the music schools, and a pinch of the talents. How does this regime further the art of music? Do you have a radio? Have you used it lately? The internet is a great end-run around the music cartels, but your startup is already on the internet.

YC is backed by serious & smart people. The pitch is soft, the benefits are compelling. They didn't create this situation. They are doing a lot of good: directly to the kids coming to Boston and as inspiration to everyone else. But if you love the art of software, keep your eyes open. Don't assume that what's best for your success is best for your art, or what's best for them is best for you.

Our Discontented Bones

2007-02-26T22:23:00.000-05:00

Today the Miami sun is a hungover showgirl, leaving pink and purple smears on her pillow-clouds. She washes with lazy rain. It drips on Mediterranean collonades and quirky Arabian facades: memories ancient Persia by way of Spain. It drizzles into your Calle Ocho cafe con leche. It puddles the driveways of a hundred thousand SUVs and a dozen kinds of palm trees -- all of them, like most of us, recent imports.

The lure of Miami is how it embraces the international and mocks the cosmopolitan. Another block is another dimesion: Little Haiti, Little Havana, pockets of Europe and Argentina and Hong Kong, people trading driving, praying, building, teeming in and out of one of the Grand Caravanserais of the New World.

On occasion some 4th-generation cracker will read too much into my complexion. He will confide the unmoored feeling that comes every time he sees a billboard written in another language. I will drop into an accent that is comfortable to my tounge yet alien to my ears. It's an unconcious habit. I will share with him in the Brotherhood of Expatriates: a club with no rites or roster but which calls itself into session wherever beleaguered men find the familiar in a strange place. We will sit and sip and I will listen to how is it's a shame to have to taste the Brotherhood while on our native soil.

The first time it happened I threw the ignorant prick out of my house. But that was during the golden time between callow youth and heavy drinking, before I realized that the unmoored feeling was with me always.

People of recently mixed heritage have no homeland in this sense: there is no place, however distant in space and time, peopled with those who share every one of your values. Is the answer a permanent Wanderjahr?

Quoth Nathaniel Hawthorne, a man who got paid by the comma:

"...the years, after all, have a kind of emptiness, when we spend too many of them on a foreign shore. We defer the reality of life, in such cases, until a future moment, when we shall again breathe our native air; but, by and by, there are no future moments; or, if we do return, we find that the native air has lost its invigorating quality, and that life has shifted its reality to the spot where we have deemed ourselves only temporary residents. Thus, between two countries, we have none at all, or only that little space of either, in which we finally lay down our discontented bones."

I didn't come to love this place until I had left, seen what was around, and came back. I needed distance to appreciate this mad soup that seems to be self-organizing into a cultural capital.