HORROR! Imagine if anyone could find out EVERYTHING you EVER tweeted

Cripes. Engineers open up ALL microblurts since 2006

Remember those dreadful, toe-curling, never-to-be-referred-to-again tweets you tweeted years back? The ones that, thank everything that's good and kind in the universe, eventually vanished from human knowledge? Twitter remembers. Now everyone else will too.

The post-profit website, founded in 2006, has built an index of every public tweet submitted in the last eight years, and will soon allow everyone to search for really old posts. Up until now, the search engine has only worked on very recent tweets.

In a post on the company's engineering blog, project lead Yi Zhuang said building the catalog of messages was a challenge because the website was designed to keep an index of tweets in server RAM for speed.

RAM capacity is limited, therefore people could only search for stuff that had appeared within the past week or so, rather than everything ever submitted, which was archived in long-term storage.

Twitter's engineers had to redesign the processing pipeline generating the real-time RAM index to instead operate on slower but much larger SSD storage.

"Our search engine excelled at surfacing breaking news and events in real time, and our search index infrastructure reflected this strong emphasis on recency," Zhuang said.

"But our long-standing goal has been to let people search through every Tweet ever published.

"For now, complete results from the full index will appear in the 'All' tab of search results on the Twitter web client and Twitter for iOS & Twitter for Android apps. Over time, you’ll see more Tweets from this index appearing in the 'Top' tab of search results and in new product experiences powered by this index."

You can find all the gory details on how Twitter designed its new indexing system in the blog post above. ®