I've come to believe that a lot of what's wrong with the Internet has to do with memory. The Internet somehow contrives to remember too much and too little at the same time, and it maps poorly on our concepts of how memory should work.

I don't know if they did this in Germany, but in our elementary schools in America, if we did something particularly heinous, they had a special way of threatening you. They would say: "This is going on your permanent record". 

The permanent record would follow you through life, and whenever you changed schools, or looked for a job or moved to a new house, people would see the shameful things you had done in fifth grade. How wonderful it felt when I first realized the permanent record didn't exist. They were bluffing! Nothing I did was going to matter! We were free!

And then when I grew up, I helped build it for real.

Anyone who works with computers learns to fear their capacity to forget. Like so many things with computers, memory is strictly binary. There is either perfect recall or total oblivion, with nothing in between. It doesn't matter how important or trivial the information is. The computer can forget anything in an instant. If it remembers, it remembers for keeps.

Every programmer has firsthand experience of accidentally deleting something important. Our folklore as programmers is filled with stories of lost data, failed backups, inadvertently clobbering some vital piece of information, undoing months of work with a single keystroke. We learn to be afraid. And because we live in a time when storage grows ever cheaper, we learn to save everything, log everything, and keep it forever. You never know what will come in use. Deleting is dangerous. There are no horror stories—yet—about keeping too much data for too long.

Our lives have become split between two worlds with two very different norms around memory. 

The offline world works like it always has. I saw many of you talking yesterday between sessions; I bet none of you has a verbatim transcript of those conversations. If you do, then I bet the people you were talking to would find that extremely creepy. I saw people taking pictures, but there's a nice set of gestures and conventions in place for that. You lift your camera or phone when you want to record, and people around you can see that. All in all, it works pretty smoothly.

The online world is very different. Online, everything is recorded by default, and you may not know where or by whom. If you've ever wondered why Facebook is such a joyless place, even though we've theoretically surrounded ourselves with friends and loved ones, it's because of this need to constantly be wearing our public face.

It's interesting to watch what happens when these two worlds collide. Somehow it's always Google that does it.

One reason there's a backlash against Google glasses is that they try to bring the online rules into the offline world. Suddenly, anything can be recorded, and there's the expectation (if the product succeeds) that everything will be recorded.

Well, people hate the online rules!

Google's answer is, wake up, grandpa, this is the new normal. But all they're doing is trying to port a bug in the Internet over to the real world, and calling it progress. You can dress up a bug and call it a feature. You can also put dog crap in the freezer and call it ice cream. But people can taste the difference.

There's another reason, besides fear, that's driving us to save everything. That reason is hubris. 

You've all seen those TV shows where the cops are viewing a scene from space, and someone keeps hitting "ENHANCE", until pretty soon you can count the bacteria on the criminal's license plate.

We all dream of building that 'enhance' button. In the past, we were going to build it with artificial intelligence. Now we believe in "Big Data". This is the classic programmer's delusion, the belief that if you look deep enough, there's a hidden deterministic pattern. Tap the chisel on the right spot and the rock will crack open.

The belief in Big Data turns out to be true, although in an unexpected way. If you collect enough data, you really can find anything you want.

'Big data' has this intoxicating effect. We start collecting it out of fear, but then it seduces us into thinking that it will give us power. In the end, it's just a mirror, reflecting whatever assumptions we approach it with.

But collecting it drives this dynamic of relentless surveillance.