The Latest from TechCrunch

The Anatomy Of The Twitter Attack
FasterWeb Wants To Make The Entire Web Up To Ten Times Faster In 2010
Start Saving: PocketSmith Is A Crystal Ball For Your Bank Accounts
Handshake Horror: The Awareness Spreads.
AT&T Is A Big, Steaming Heap Of Failure

Posted: 19 Jul 2009 03:40 AM PDT

The Twitter document leak fiasco started with a simple story that personal accounts of Twitter employees were hacked. Twitter CEO Evan Williams commented on that story, saying that Twitter itself was mostly unaffected. No personal accounts were compromised, and “most of the sensitive information was personal rather than company-related,” he said. The individual behind the attacks, known as Hacker Croll, wasn’t happy with that response. Lots of Twitter corporate information was compromised, and he wanted the world to know about it. So he sent us all of the documents that he obtained, some 310 of them, and the story developed from there.

It’s clear that Twitter was completely unaware of how deeply they were affected as a company - when Williams said that most of the information wasn’t company related he believed it. It wasn’t until later that he realized just how much and what kind of information was taken. It included things like financial projections and executive meeting notes that contained highly confidential information.

We’ve already said a lot about all of this and the related “server password = password” story that was discovered by another individual last week. But we’ve got two more stories to tell. The first, this post, is exactly how the hacks took place, based on information gathered from hours of conversations with Hacker Croll. The second is what was happening behind he scenes with Twitter as the story unfolded. We’ll post that later this week.

When the story first broke the true scope of what had taken place and how it occurred was not understood. Various bloggers speculated about the cause of the attack - with some placing the blame on Google while others blaming the rising trend of hosting documents in the cloud.

We immediately informed Twitter of the information we had in our possession (and forwarded it to them), and at the same time reached out to the attacker. With some convincing, the attacker responsible for the intrusion at Twitter began a dialog with us. I spent days communicating with the attacker in an effort to gain insight into how the attack took place, what the true scope of it was and how we could learn from it.

We’ve waited to post exactly what happened until Twitter had time to close all of these security holes.

Some Background

In the security industry there is a generally accepted philosophy that no system or network is completely secure - a competent attacker with enough time, patience and resources will eventually find a way into a target. Some of the more famous information security breaches have relied on nothing more than elementary issues exploited by an attacker with enough time and patience at hand to see their goal through. A classic example is the case of Gary McKinnon, a self-confessed “bumbling computer nerd” who while usually drunk and high on cannabis would spend days randomly dialing or attempting to login to government servers using default passwords. His efforts led to the compromise of almost 100 servers within a number of government departments. After McKinnon spent a number of years trawling through servers looking for evidence of alien life (long story), somebody within the government finally wised up to his activities which lead to not only the arrest and attempted extradition of McKinnon from the United Kingdom, but a massive re-evaluation of the security methods employed to protect government information.

A more recent example is the case of Kendall Myers, who after being recruited to work for the Cuban government by an anonymous stranger they met while on holiday in that country, set out to obtain a high ranking position within the State Department specifically to obtain access to US government secrets. Kendall dedicated his entire life to obtaining state secrets, and up until he was recently caught by the FBI had successfully passed on secret information and internal documents to the Cuban government for 30 years. He relied only on his memory, his education credentials and sheer dedication.

The Twitter Attack: How The Ecosystem Failed

Like other successful attacks, Hacker Croll used the same combination of patience, sheer determination and somewhat elementary methods to gain access to a frightening number of accounts and services related to Twitter and Twitter employees. The list of services affected either directly, or indirectly, are some of the most popular web applications in use today - Gmail, Google Apps, MobileMe, AT&T, Amazon, Hotmail, Paypal and iTunes . Taken individually, most of these services have reasonable security precautions against intrusion. But there are huge weaknesses when they are looked at together, as an ecosystem. Like dominoes, once one fell (Gmail was the first to go), the others all tumbled as well. The end result was chaos, and raises important questions about how private corporate and personal information is managed and secured in a time when the trend is towards more data, applications and entire user identities being hosted on the web and ‘in the cloud’.

“Hacker Croll” is a Frenchman in his early 20’s. He currently resides in a European country and first discovered his interest in web security over two years ago. Currently in between jobs, he has made use of the additional time he now has, along with his acquired skillset, to break into both corporate and personal accounts across the web. His knowledge of web security has been attained through a combination of materials available to the public and from within a tight-knit group of fellow crackers who exchange details of new, and sometimes unknown, techniques and vulnerabilities. Despite the significance and impact a successful attack has, the cracker claims that his primary motivation is a combination of curiosity, exploration and an interest in web security. There is almost a voyeuristic tendency amongst these individuals, as they revel in the thought of gaining privileged access to information about the inner lives of individuals and corporations. The “high” of access and gaining unauthorized knowledge must be big enough to carry a cracker’s motivation through the long hours, days and months of effort it may take to hit the next pot of gold.

For Hacker Croll, his first port of call in setting out to gain access to a target network is to make use of public search engines and public information to build a profile of a company or individual. In the case of the Twitter attacks, this public information allowed him to create a rich catalog of data that included a list of employee names, their associated email addresses and their roles within the company. Information like birth dates, names of pets and other seemingly innocent pieces of data were also found and logged. This dragnet across the millions of pages on the web picked up both work and personal information on each of the names that were discovered. Public information on the web has no concept of, or ability to, distinguish between the work and personal details of a person’s identity - so from the perspective of a cracker on a research mission, having both the business and personal aspects of a target’s digital life intertwined only serves to provide additional potential entry points.

With his target mapped out, Hacker Croll knew that he likely only needed a single entry point in any one of the business or personal accounts in his list in order to penetrate the network and then spread into other accounts and other parts of the business. This is because the web was designed at a time where there was implicit trust between its participants - requiring no central or formal identification mechanism. In order to keep private data private, modern web applications have built out their own systems and policies that require a user to register and then manage their identities separately with each app. The identifier that most applications use is an email address, and it is this common factor that creates a de facto trust relationship between a user’s applications. The second factor is a password: a random string that only the user knows, is unique to each application, and in theory should take even a computer months or years to figure out if it started guessing. These two elements would work well enough for most cases, were it not for what is often the single weakest factor: human habit.

Look at the front page of almost any web application and you will see hints at just how hopeless and helpless we are in managing our digital lives: “forgot my password”, “forgot my username”, “keep me logged in”, “do not keep me logged in”, “forgot my name”, “who am i?”. Features that were designed and built as a compromise since we are often unable to remember and recall a single four-digit PIN number, let alone a unique password for every application we ever sign up for. Each new service that a user signs up for creates a management overhead that collapses quickly into a common dirty habit of using simple passwords, everywhere. At that point, the security of that user’s entire online identity is only as strong as the weakest application they use - which often is to say, very weak.

Now going back to Hacker Croll and his list of Twitter employees and other information. Twitter just happens to be one of a number of a new breed of companies where almost the entire business exists online. Each of these employees, as part of their work, share data with other employees - be it through a feature of a particular application or simply through email. As these users become interwoven, it adds a whole new attack vector whereby the weak point in the chain is no longer just the weakest application - it is the weakest application used by the weakest user. For an attacker such as Hacker Croll looking to exploit the combination of bad user habit, poorly implemented features and users mixing their personal and business data - his chances of success just got exponentially greater. Companies that are heavily web based rely largely on users being able to manage themselves - the odds are not only stacked against Twitter, they are stacked against most companies adopting this model.

Unfortunately for Twitter, Hacker Croll found such a weak point. An employee who has online habits that are probably no different than those of 98% of other web users. It began with the personal Gmail account of this employee. As with most other web applications, the personal edition of Gmail has a password recovery feature that presents a user with a number of challenges to prove their identity so that their password can be reset. It likely wasn’t the first account from a Twitter employee that Hacker Croll had attempted to access - but in the case of this particular account he discovered a kink in the armor that gave him the big first step. On requesting to recover the password, Gmail informed him that an email had been sent to the user’s secondary email account. In an effort to balance usability with security, Gmail offered a hint as to which account the email to reset the password was being sent to, in case the user required a gentle reminder. In this case the obfuscated pointer to the location of the secondary email account was ******@h******.com. The natural best guess was that the secondary email account was hosted at hotmail.com.

At Hotmail, Hacker Croll again attempted the password recovery procedure - making an educated guess of what the username would be based on what he already knew. This is the point where the chain of trust broke down, as the attacker discovered that the account specified as a secondary for Gmail, and hosted at Hotmail was no longer active. This is due to a policy at Hotmail where old and dormant accounts are removed and recycled. He registered the account, re-requested the password recovery feature at Gmail and within a few moments had access to the personal Gmail account of a Twitter employee. The first domino had fallen.

Well designed web applications will never just give a user their password if they forget it, they will force the user to pick a new one. Hacker Croll had access to the account, but with a password he had specified. To not alert the account owner that their account had been compromised, he had to somehow find out what the old Gmail password was and to set it back. He now had a bevy of information at his fingertips, a complete mailbox and control of an email account. It wasn’t long before he found an email that would have looked something like this:

To: Lazy User
From: Super Duper Web Service
Subject: Thank you for signing up to Super Duper Web Service

Dear Lazy User,

Thank you for signing up to Super Duper Web Service. For the benefit of our support department (and anybody else who is reading this), please find your account information below:

username: LazyUser
password: funsticks

To reset your password please follow the link to.. ahh forget it, nobody does this anyway.

Regards,

Super Duper Web Service

Bad human habit #1: Using the same passwords everywhere. We are all guilty of it. Search your own inbox for a password of your own. Hacker Croll reset the password of the Gmail account to the password he found associated with some random web service the user had subscribed to and that sent a confirmation with the password in clear text (and he found the same password more than once). He then waited, to check that the user was still able to access their account. Not too long later there was obvious activity in the email account from the account owner - incoming email read, replies sent and new messages drafted. The account owner never would have noticed that a complete stranger was lurking in the background. The second domino falls.

From here it was easy.

Hacker Croll now sifts through the new set of information he has access to - using the emails from this user’s personal Gmail account to further fill in his information map of his target. He extends his access out to all the other services he finds that this user has signed up for. In some instances, the password is again the same - that led Croll into this user’s work email account, hosted on Google Apps for Domains. It turns out that this employee (and in fact most/all Twitter employees and everyone else) used the same password for their Google Apps email (the Twitter email account) as he did with his personal Gmail account. With other sites, where the original password may not work - he takes advantage of a feature many sites have implemented to help users recover passwords: the notorious “secret question”.

Fork the story here for a moment because there is a real issue here with the “secret question” (from here on abbreviated more appropriately as just “secret ?”). For some strange reason, some sites refer to the “secret ?” as an additional layer of security - when it is often the complete opposite. In the story of Hacker Croll and Twitter, the internal documents that we now all know about were only a few steps away from the first account he gained access to. In addition to that, this attacker, and certainly others just like him, have been able to demonstrate that some of the biggest and most popular applications on the web contain fundamental weaknesses that alone might seem harmless, but in combination with other factors can cause an attacker to completely tear through the accounts of users, even those who maintain good password policy.

This is not the first time that the issue of “secret ?” being used in password recovery systems has been raised. Last September, US Republican Vice Presidential candidate and former governor of Alaska, Sarah Palin, had screenshots of her personal Yahoo mail account published to Wikileaks. A hacker or group known only as ‘Anonymous’ claimed credit for the hack, which was carried out by the attacker making an educated guess in response to the security question used to recover passwords. In early 2005, celebrity Paris Hilton suffered a similar incident when her T-Mobile sidekick account was broken into, and the details of her call log, messages (some with private pictures of Hilton) and contact list were leaked to the media. The culprit, again, was “secret ?”.

Giving the user an option to guess the name of a pet in lieu of actually knowing a password is just dramatically shortening the odds for the attacker. The service is essentially telling the attacker: “we understand that guessing passwords is hard, so let us help you narrow it down from potentially millions of combinations to around a dozen, or even better, if you know how to Google, just one”. The problem is not the concept of having an additional authorization token, such as mothers maiden name, that can be used to authenticate in addition to a password, the problem arises when it is relied on alone, when the answer is stored in the clear in account settings, and when users end up using the same question and answer combination on all of their accounts.

From this point, with a single personal account as a starting point, the intrusion spread like a virus - infecting a number of accounts on a number of different services both inside and outside of Twitter. Once Hacker Croll had access to the employee’s Twitter email account hosted by Google, he was able to download attachments to email that included lots of sensitive information, including more passwords and usernames. He quickly took over the accounts of at least three senior execs, including Evan Williams and Biz Stone. Perusing their email attachments led to lots more sensitive data being downloaded.

He then spidered out and accessed AT&T for phone logs, Amazon for purchasing history, MobileMe for more personal emails and iTunes for full credit card information (iTunes has a security hole that shows credit card information in clear text - we’ve notified Apple but have not heard back, so we won’t publish the still-open exploit now).

Basically, when he was done, Hacker Croll had enough personal and work information on key Twitter executives to make their lives a living hell.

Just to summarize the attack:

HC accessed Gmail for a Twitter employee by using the password recovery feature that sends a reset link to a secondary email. In this case the secondary email was an expired Hotmail account, he simply registered it, clicked the link and reset the password. Gmail was then owned.
HC then read emails to guess what the original Gmail password was successfully and reset the password so the Twitter employee would not notice the account had changed.
HC then used the same password to access the employee’s Twitter email on Google Apps for your domain, getting access to a gold mine of sensitive company information from emails and, particularly, email attachments.
HC then used this information along with additional password guesses and resets to take control of other Twitter employee personal and work emails.
HC then used the same username/password combinations and password reset features to access AT&T, MobileMe, Amazon and iTunes, among other services. A security hole in iTunes gave HC access to full credit card information in clear text.
Even at this point, Twitter had absolutely no idea they had been compromised.

What could have happened next is that Hacker Croll could have used or sold this information for profit. He didn’t do that, and says he never intended to. All he wanted to do, he says, was to highlight the weaknesses in Twitter’s data security policies and get them and other startups to consider more robust security measures.

He also says he’s sorry for causing Twitter so much trouble. We asked Hacker Croll if he had any message he wants to deliver to Twitter, and he sent me the following:

Je tiens à présenter toutes mes excuses au personnel de Twitter. Je trouve que cette société a beaucoup d’avenir devant elle.

J’ai fait cela dans un but non lucratif. La sécurité est un domaine qui me passionne depuis de longues années et je voudrais en faire mon métier. Dans mon quotidien, il m’arrive d’aider des gens à se prémunir contre les dangers de l’internet. Je leur apprend les règles de base.. Par exemple : Faire attention où on clique, les fichiers que l’on télécharge et ce que l’on tape au clavier. S’assurer que l’ordinateur est équipé d’une protection efficace contre les virus, attaques extérieures, spam, phishing… Mettre à jour le système d’exploitation, les logiciels fréquemment utilisés… Penser à utiliser des mots de passe sans aucune similitude entre eux. Penser à les changer régulièrement… Ne jamais stocker d’informations confidentielles sur l’ordinateur…

J’espère que mes interventions répétées auront permis de montrer à quel point il peut être facile à une personne mal intentionnée d’accéder à des informations sensibles sans trop de connaissances.

Hacker Croll.

This roughly translates to:

I would like to offer my personal apology to Twitter. I think this company has a great future ahead of it.

I did not do this to profit from the information. Security is an area that fascinated me for many years and I want to do my job. In my everyday life, I help people to guard against the dangers of the Internet. I learned the basic rules .. For example: Be careful where you click the files that you download and what you type on the keyboard. Ensure that the computer is equipped with effective protection against viruses, external attacks, spam, phishing … Upgrading the operating system, software commonly used … Remember to use passwords without any similarity between them. Remember to change them regularly … Never store confidential information on the computer …

I hope that my intervention will be repeated to show how easy it can be for a malicious person to gain access to sensitive information without too much knowledge.

Croll hacker.

What’s the takeaway from all this? Cloud services are convenient and cheap, and can help a company grow more quickly. But security infrastructure is still nascent. And while any single service can be fairly secure, the important thing is that the ecosystem most certainly is not. Combine the fact that so much personal information about individuals is so easily findable on the web with the reality that most people have merged their work and personal identities and you’ve got the seed of a problem. A single Gmail account falls, and soon the security integrity of an entire startup crumbles. So for a start, reset those passwords and don’t use the same passwords for different services. Don’t use password recovery questions that can easily be answered with a simple web search (an easy solution is to answer those questions falsely). And just in general be paranoid about data security. You may be happy you were.

Crunch Network: CrunchGear drool over the sexiest new gadgets and hardware.

FasterWeb Wants To Make The Entire Web Up To Ten Times Faster In 2010

Posted: 19 Jul 2009 03:00 AM PDT

As the web matures, it’s also getting more complex. Yet much of it is still fundamentally based on things like HTML which are 30 years old. A new startup, FasterWeb, aims to bring these old technologies up to speed — as it were — making the web faster, by optimizing the old standards for doing new things. And in doing so, it claims that it can increase the performance of any site by 2 to 10 times — something which would obviously be a huge leap forward, if it can deliver.

One VC firm, YL Ventures, believes that it can. And they’ve seen it in action, so we’ll just have to take their word for it, for now. We spoke with Yoav Andrew Leitersdorf, managing parter at YL, and he tells us that the different between the regular web versus a site optimized with FasterWeb, is pretty staggering. And that’s why his firm had no hesitation in pouring an undisclosed amount of money into the Israeli-based venture.

So how does FasterWeb claim to work? Leitersdorf wouldn’t go into the details, saying that’s the company’s secret, but he would say that it uses 45 different techniques to optimize the web. He also said that this is done either on the end of the content provider or the ISP. In other words, the end user doesn’t have to do a thing to experience the increase in web speed. And FasterWeb will work across all the major web browsers, starting with Internet Explorer and Firefox immediately, and expanding to the rest, including Opera, Chrome and Safari, when it’s ready for its widespread release next year.

But some sites won’t have to wait until next year to get the speed boost. Over the next several weeks, the first sites optimized with FasterWeb will begin hitting our browsers, Leitersdorf says. He would not say which ones, but notes that some will be known entities in the U.S. and worldwide.

And all of this will work for the mobile web too. “That’s one of the biggest opportunities here,” Leitersdorf says. He went on to note that they’re thinking a lot about mobile ISPs in particular.

Obviously, a two to ten fold increase in speed is a big difference, but Leitersdorf notes that the more complex a page is, the higher the magnitude of optimization will be. This optimization occurs across HTTP, HTML, JavaScript, CSS and images on a page, to achieve the results.

The business model for the project seems sound as well. FasterWeb has a multi-pronged approach depending on the situation of the website or ISP. That means it can either charge a one-time fee, or do a revenue sharing model. “What we found out as a VC fund going into this business is that by selling this to websites, it’s going to increase their revenues. And these sites are willing to spend 20-30% of their increase in revenues on our solution,” Leitersdorf says.

He also notes that in their research, YL only found two companies even come close to doing what these guys are doing. But Leitersdorf declined to name them. Seeing as this is all on the backend, and requires nothing from the consumers, it seems safe to assume this will be significantly better than something like the Google Web Accelerator toolbar.

Naturally, all of this sounds great, but it will be another thing to deliver on a massive scale across much of the web. “We’ve talked to the customers, they’re excited. But FasterWeb wants to make sure they’re ready,” according to Leitersdorf. And that’s why we won’t see wide-scale deployment until next year.

The Israel-based FasterWeb was started by Ofer Gadish, Gil Shai, Ofir Ehrlich and Leonid Fainberg.

Crunch Network: CrunchGear drool over the sexiest new gadgets and hardware.

Start Saving: PocketSmith Is A Crystal Ball For Your Bank Accounts

Posted: 18 Jul 2009 07:12 PM PDT

Saving money is tough work, even when you’re working towards something you really want, like a new car or a vacation. First, there’s the business of actually finding someone to write you a paycheck each month, and then you have to look at that pile of cash sitting in your bank account, just begging to be spent on the latest DVD, book, or gadget that suits your fancy. And there’s always the predictable but expensive costs like rent and insurance that keep eating away at those savings, not to mention the issues you can’t plan for.

PocketSmith is a new startup that’s looking to help. The site offers a range of tools for managing your financials both now and in the future, hopefully helping you reach your financial goals in the process. This week it’s leaving beta, and is offering the first 50 TechCrunch readers to Email contact@pocketsmith.com a free premium account for six months.

There are a number of well known financial services already on the web, including Wesabe and Mint, which won the to prize at 2007’s TechCrunch 40 conference. But whereas Mint is really about looking at your spending habits and figuring out ways to save, PocketSmith is more of a calendar for finances that lets you set financial goals and track your progress over time.

The other big difference from Mint is that you don’t directly connect your bank accounts to the service. Instead, you can download your transaction history from your bank’s website and upload it, or you can manually enter your transactions. If you’re used these other services and are okay with allowing a startup to access your banking data, this is a bit of an inconvenience, but the upload method really only takes a minute.

The site iself is very well done, sporting a polished interface and nifty effects that make otherwise mundane tasks a bit more fun. There are also videos for many of the site’s common functions, explaining how you should be using them. And because many of the service’s functions are calendar-based, you can import them into iCal and Outlook, as well as Google Calendar.

Most of the features are broken into one of four sections: Forecast, which lets you create a calendar of your financial activity, updating with regular costs (like rent) and other major expenses that you know are coming up. This section also projects how much money you’ll have months (or a year) down the line. The Accounts section lets you can either manually input or upload your transaction history, while ‘Compares’ lets you visually contrast how your projected financials are matching up with what you actually have. Finally, there’s Goals, where you can list off things that you’re saving for (say, a new car).

It’s the last section that will probably be the most useful for people, as it determines how many days you’ll have to maintain your current savings plan until you can meet a goal. There’s something about being able to watch that countdown tick downwards that can really become a powerful psychological tool, which can help you resist those financial splurges.

PocketSmith offers three different pricing plans: a free option, which allows you to maintain two calendars and track six events; a premium version for around $5 a month that allows for unlimited events and five calendars, and finally a $12 version that allows for unlimited events and ten calendars (Note: the prices on the site are in New Zealand Dollars, so I’ve converted them to US).

All in all the site seems solid, but it’s probably going to appeal to a somewhat different audience than Mint does. This is primarily because using Mint can be a passive experience — you enter your account data once, and it does the rest for you. With PocketSmith, you need to proactively set your goals and make sure that you’re entering any upcoming expenses, which requires a bit more effort. Still, the use case for the sites are pretty different, and there may well be a significant market for PocketSmith.

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

Handshake Horror: The Awareness Spreads.

Posted: 18 Jul 2009 05:56 PM PDT

Now even mainstream journalists are picking up the no handshake banner and running with it. Neil Swidey, writing for the Boston Globe, says “Last month, swine flu officially became a pandemic. Public health officials have said so-called "social distancing" strategies — sharply reducing contact with others — have proved most effective in slowing the spread of previous outbreaks, such as the 1918 flu pandemic. And they told us to cut down on our handshakes as much as we could. Northeastern University heeded the advice, asking its graduates not to shake hands when receiving their diplomas during the school's commencement ceremony in May.”

Swidley also points to Brad Feld’s promise earlier this year to end handshakes, and asked Feld how that was going. Feld said "My campaign was a total failure. I found that I was having the same conversation over and over, explaining why I wasn't shaking hands. I got tired of it and decided it was easier to just shake everyone's hands and then wash mine a bunch throughout the day."

I obviously agree that handshakes need to go. My first post on handshakes was in May, and after I noted that some startups and venture capitalists were trying to end the barbaric practice at board meetings. I piped up again on National Handshake Day.

Like Feld, I too have mostly given up on this. People just get pissed when you don’t shake their hand. But 30% or so of people I meet with know how I feel about it and offer a friendly fist bump. The best moments I have are when people say how much they love TechCrunch and read it every day, and then stick out their sweaty palm to shake hands. They obviously were just being polite about reading this blog. I shake their hand with a smile, and remember to wash my hands at the next opportunity.

Crunch Network: CrunchBase the free database of technology companies, people, and investors

AT&T Is A Big, Steaming Heap Of Failure

Posted: 18 Jul 2009 04:33 PM PDT

picture-217 When Om Malik of GigaOM said he was breaking up with his iPhone 5 months ago because of the failures of AT&T, I must admit, I thought he was overreacting. I was wrong.

Since I switched to AT&T from Verizon just over 2 years ago to get the iPhone (which, of course, AT&T has exclusively in the U.S.), there have been no shortage of shortcomings by AT&T. But as of late, I’ve been noticing things getting much, much worse. And I’m hardly the only one. And so it’s time to call out AT&T on those failures. And plead with Apple not to renew its exclusive contract with AT&T when it expires next year.

In my mind, the most recent AT&T failure is completely inexcusable. Its visual voicemail system — which is the only way to be notified of voicemails on the iPhone — has been down for many users for days, if not weeks. And AT&T apparently didn’t bother to tell anyone. What does this mean? Thousands, or hundreds of thousands or maybe even millions of missed connections, that could be vital for personal lives, business and a host of other things. I’m simply dumbfounded by the failure.

Here’s how I found out about it. While I was coming home from the office yesterday, I all of a sudden got bombarded by visual voicemails. It was only then that I realized that I had not received one in a while. How long? Since sometime before July 3, apparently. Yes, 2 weeks without a single voicemail.

Even better is that not only did I get bombarded by these weeks old voicemails at once, but I still cannot listen to them. It has been over a day since the notifications finally came in, and visual voicemail is still down. I’ve had to manually call the AT&T voicemail service — not a huge deal, except that I’ve never done it before, so I didn’t know how, and that I didn’t receive any kind of notice that I had to do that.

Once I did that, sure enough, I had a a range of voicemails from personal ones, to pretty important ones for appointments and work that I just totally missed, to a voicemail from my 90-year-old grandma, who probably thinks I’m avoiding her now. I’m not grandma, AT&T just is a complete and utter failure.

new-att-logo2 Oh, and did I mention that half of those missed voicemails don’t even show up in my call logs as missed calls? So who knows what else I’ve missed from people who didn’t bother to leave voicemails.

I’m so pissed off that I kind of want to call AT&T and demand that they call each of the people I missed calls from and personally apologize. Instead, I’m writing them this very public condemnation.

This is really, really bad any way you look at it. But it’s compounded by a host of other failures over the past several months and years on AT&T’s behalf.

Even since the iPhone launched on AT&T’s network, there have been reports of problems. But things really got bad with the launch of the iPhone 3G last year, when basically no one could activate their phones. Okay, so AT&T learned from that mistake, right? Nope — the same thing happened this year. And immediately after that post, AT&T contacted us to suggest that it wasn’t its fault, but when we asked for some sort of proof or statement to that effect, they did not get back to us. Yeah.

And let’s not forget the total failure of AT&T’s network during this year’s SXSW festival. AT&T tried to pat itself on the back for rushing to turn up the bandwidth — something which still didn’t really work all that well, and came far too late. Sure, there were a ton of iPhones in one place that were accessing the network, but AT&T has one job: To provide service to its customers, and it failed at it.

And it fails at it far too often. Depending on where you are here in Bay Area (I’m using that as an example because that’s where I live, but the problems are hardly confined to here), there is basically no AT&T reception. This is what Malik noticed all those months ago. And as more iPhones are being sold, it’s getting progressively worse.

AT&T promises that network upgrades are coming, but the fact is, the company has had over 2 years to fix these issues (that have arisen since the launch of the iPhone) and they have not. Hell, they can’t even get basic services like MMS and tethering working, even as their carrier counterparts in other countries already have them up and running. And now you can add visual voicemail to the list. Pathetic.

And something else that’s not talked about nearly enough is that the newest iPhone, the 3GS, is built to handle data download rates twice that of older iPhones. But it doesn’t. Why? Because AT&T’s network isn’t yet equipped to handle it. And won’t be for most places until 2011. There will likely be two more versions of the iPhone by then.

And even where AT&T is testing the new faster network, in Chicago, there is apparently no data transfer speed difference, tests performed by Gizmodo have confirmed. Again, nice job AT&T.

As someone who writes about the iPhone a lot, I often get asked by people if I think they should get one now or wait to see if it ever gets on another carrier. That answer becomes easier everyday: If you can, wait.

As great as the iPhone is as a mobile computing device, it is still first and foremost a phone. But with AT&T’s shortcomings, it has basically turned the iPhone into an iPod touch. So why not just buy one of those? After all, you can get much of the actual working functionality, without having to pay a high monthly bill.

picture-310 AT&T’s exclusive deal with Apple is set to expire next year, and they’re trying to extend it right now. I will say right now that if Apple does re-up with AT&T it will easily be one of the most disappointing things it has ever done. And I think ultimately that would prove to be a huge blunder from a business perspective.

I understand why Apple went exclusively with AT&T at first (though it had first offered the device to Verizon, which turned it down) — it got a pretty sweet deal, and was able to use it to put it in a position of power over the entire industry. And I even understand why they re-upped the first time — to get an even sweeter deal (the subsidy from AT&T for each phone sold). But now AT&T is a liability for Apple that will inhibit its huge potential for growth in the U.S.

Apple no longer needs AT&T. Thanks to its huge success, it can dictate its own terms to other carriers now, and ensure it controls the iPhone ecosystem — its top priority. Verizon, as the nation’s largest carrier, is likely to give it the most resistance. But that resistance is futile. The iPhone will eventually be on Verizon, on Apple’s terms. It’s just a question of when.

If that’s by the end of next year, many of us will be happy campers. I don’t care what I have to pay to break an AT&T contract, I will do so in a heartbeat.

If it’s not next year, will I consider switching carriers and getting another phone? Yes. As I indicated, I’d be happy carrying around an iPod touch and having some other phone — even a crappy one — that actually works. Or more likely, I’ll just unlock the iPhone and use it on another carrier. At this point, I don’t care how much that costs, I just want a working phone.

But I don’t think I’ll have to do that. Because I truly believe that Apple has to know that it needs to expand its carrier roster in the U.S. to continue growth. And if I were a betting man, I would bet on that happening next year.

Let’s all do what we can to ensure that happens — to ensure Apple gets the message. Every time there is one of these ridiculous AT&T failures, tweet about it, blog about it, write Apple about it, or scream about it. Do whatever you can, but don’t just sit there and take it any more.

It’s time to send a message, since AT&T can’t provide us with ours with any sort of reliability.

Update: The plot thickens (gets worse for AT&T)… From Molly Wood on Twitter: