A new tweak Twitter made to its internal instructions to search engines now specifically allows the big-time engines, including Google, Bing, Yahoo and Yandex, to crawl through Twitter search result pages. The results were previously inaccessible and could potentially impact local search engine optimization.
The change occurred when search engine specialists noticed a small change to Twitter's robots.txt file, a tool that webmasters use to either prevent or allow a search engine from indexing parts of their sites.
Now, Twitter has opened up its content search pages, including tweets and hashtags for indexing. However, while the social media platform has increased its visibility in major search engine results, it is still preventing bots from rifling through searches for users, pictures and videos.
Even though the new change greatly improves the ability to index search results, Twitter never fully prohibited search engines from accessing its content. However, its search result visibility was markedly decreased in the past year after a spat with Google. The two had a previous agreement that gave exclusive priority to Google to display real-time Twitter updates, but the deal expired in July 2011 and effectively killed off Google's Realtime Search product.