In the past few hours, Google apparently updated its database and moved blogs way, way down the list of search results. First Atrios mentioned it and thought it strange, and then Sisyphus Shrugged found it had happened to her. That prompted me to google my own site, and it's true for me, too: this blog was the first search result for "Fables of the reconstruction" and the third (I recall) for "Mithras" until now. Now, it is the 53rd result for "Fables of the reconstruction" and does not appear at all in the first 200 results for "Mithras" (although, strangely, my profiles on other websites do show up in those results.)
Strangely, this blog still has a google pagerank of 5 (out of 10), which is google's numerical way of denoting importance.
We'll see how this shakes out. But if it persists, this is very bad news for blogs. I have written elsewhere that blogs are only an important phenomenon (important other than for the intrinsic satisfaction of writing) to the extent that search engines prioritize blogs. There are, by my estimate, only about 500,000 people who actively search out political and current events blogs. That number is not likely to grow much; all of the people who are likely to take an active interest have had a chance to join us by now. While not nothing, such a number is quite small in cultural terms. What makes blogging potentially culturally relevant is being indexed by search engines. Until today, if you googled any topic of current interest, the first page of results included blogs. That made blogs relevant to general internet users, but much more importantly, it made blogs vital for journalists researching stories. The prominent placement of blogs in google search results meant that bloggers, rightly or wrongly, were paid attention to. It also meant that blogs occupied valuable internet real estate that attracts advertising revenue.
Take search engines away, and the entire enterprise dries up. No google hits, no blogs. It's that simple.
As I said, we'll see how this shakes out.
Update 2/16/05: Moved up from comments:
Before this change, googling "Fables of the reconstruction" returned this blog as the first result. When I wrote this post, it had fallen to 53. Well, now it's not in the first 200 results, which is where I stop looking. Links to this blog and comments I have left on other blogs rank higher than the blog itself.
I don't know what google did, but I have to say, it seems fucked up.
Update 2/18/05: Based on a completely unscientific, nonrandom sample, it appears that this affects blogs that use their blogging software's domain, such as typepad.com, blogs.com and blogspot.com. Blogs with their own domains do not appear to be affected. What this means, I can only speculate, and I am down with the flu today so I am not going to research it.
Actually -- maybe the fact that my blog has disappeared from view is a direct the result of the fact that I have installed the nofollow plugin. Is that true for other self-hosted blogs out there who have diminished in the rankings?
Posted by: paul | February 19, 2005 at 07:07 PM
Paul, maybe it's the other way around. Not that you installed the nofollow plugin, but that everyone else did. See, I was just thinking (in my non-tech-literate way) that the nofollow thing wasn't a plausible explanation since it only affected links in comments and trackbacks. I thought, how many of all links could be in comments and trackbacks? Then I realized I was assuming that the nofollow thing only works prospectively, and went back to a comment on here from last May and checked the page source. Sure enough, it had nofollow in it. So, when nofollow went active in MT blogs, all comments from the beginning of the blog were coded with nofollow.
I assume that's a big number of links. If that's true, then that may explain the drop. Is it that simple?
Posted by: Mithras | February 19, 2005 at 08:10 PM
Possibly, Paul. But there is a much larger thing that is happening. Take a listen to my podcast I talked about above and note it was about a month ago when I predicted this.
I will be doing the update podcast tomorrow afternoon. I'll try to go through each of the questions raised here and answer them one by one.
I can sense a bit of frustration and a little bewilderment about the SE's (Google and Yahoo!) and that's my expertise.
Posted by: Scott | February 19, 2005 at 08:11 PM
Scott, excuse me, but I am not going to listen to an 18 minute podcast. I did give the first few minutes a listen and wondered about your implied premise that nofollow will affect all links on a blog. As far as I can tell, it only affects links in comments and trackbacks. So, the example you gave of the "miserable failure" googlebomb is inapt, because that happened primarily in the body of posts, not in comments.
Posted by: Mithras | February 19, 2005 at 09:12 PM
Mithras -- the problem with this hypothesis is that some blogs seem not to be affected at this time, in particular many (but not all) of the blogs that self-host. If the nofollow has been adopted widely, wouldn't it result in the same effects for all blogs, whether or not they are self-hosted? I think it's much more likely that self-hosting is an imperfect surrogate for not having installed nofollow, since the installation could have been automated in blogger and typepad blogs, but has to be installed manually by those who run their own MT. Anyway, it's a real mystery.
Posted by: paul | February 20, 2005 at 01:39 AM
If the nofollow has been adopted widely, wouldn't it result in the same effects for all blogs, whether or not they are self-hosted?
You would think.
I think it's much more likely that self-hosting is an imperfect surrogate for not having installed nofollow ....
Yes, and it's tempting to draw the conclusion that that somehow explains the apparent difference, except that I don't see why search engines would change my ranking in search results based on the number of links that I have to others. I always thought it was based (in part) on how many others link to me. So, I agree there is correlation; I don't understand what the causation is, though.
Posted by: Mithras | February 20, 2005 at 08:56 AM
I don't see why search engines would change my ranking in search results based on the number of links that I have to others. I always thought it was based (in part) on how many others link to me. So, I agree there is correlation; I don't understand what the causation is, though.
Agreed. A side note: right after I installed nofollow I got an increase in my Google page rank, and I actually wondered (not too seriously) if Google had built that in as an incentive to install nofollow. I don't think in general Google has done this kind of direct behavior reinforcement, but in some cases (as in the above) it might be useful for overcoming collective action problems.
Posted by: paul | February 20, 2005 at 11:18 AM
Mithras: I certainly apologize if you think what I'm stating is not true. My full time job is to understand changes like this and explain them (use them). I'm just trying to explain to everyone 'why' this is going on in laymans terms.
I really AM just trying to help to answer these questions and help clear up some confusion.
I also apppreciate you allowing me to post here. Thank you!
Posted by: Scott | February 20, 2005 at 12:26 PM
I acknowledge I'm a laymen, Scott. But I am still confused about what you're saying. Do MT-blogs' links have nofollow in them? I don't see it in the page source.
Posted by: Mithras | February 20, 2005 at 12:31 PM
A couple of things are in play right now. We believe LSI (Latent semantic indexing) is being introduced into Google as of update 'Allegra' during the first part of Feb.
We saw this coming, somewhat, with Google's purchase of Applied Semantics in early 2003. We just didn't know exactly how Google was going to use it at the time. We think an early version of this was floated in late 2003 with update 'Florida', although it was not quite clear at that point.
In hindsight, it makes sense, but at the time the panic was across the board with major players losing rankings. They weren't so much interested in 'what' happened, but 'how' to get around it.
So, that's a small part of it.
Secondly, Google (and other major SE's) are a little embarrassed by the 'miserable failure' thing. The major SE's won't admit this publically, but it is a sore spot for them.
The SE's KNOW this is result of blogs...and hence the need to develop what we like to call a 'blog knob'(filter) into the algo.
Blogs helped develop this little 'blog knob' tremendously by asking for the 'no-follow' attribute. All the major search engines signed on to it quite quickly.
The real question bloggers should ask is, "WHY?"
WHY would the major SE's jump all over themselves to 'help' the bloggers? Because they LOVE blogs?
Think again.
The SE's do what's best for them, not bloggers. This is an important point. Never assume that the SE's do anything like this for your benefit. They ALWAYS have a different angle.
I think it was mentioned in this thread that maybe there are 500,000 blogs or so? That's nothing in the scheme of things. There are 10-20 times that amount of webmasters out there. We've dealt with SE's since they were invented, AND we've come to realize that when an SE 'helps' us, there is a price at the end of it.
What I'm trying to do here is give some perspective to folks reading this, not to belittle anyone, but to provide a little history.
Back to the 'blog knob'. What the SE's couldn't fix, the bloggers did for them. They really loved this. THEY couldn't stop blog pages from ranking where it wasn't relevant AND influencing rankings of other pages...and the bloggers helped them fix it. (Well, that combined with LSI in Google's case.)
So, now that the SE's have a 'blog knob', they can tweak it however they want.
Kinda neat, huh?
Turn it to the left...blogs go down in rankings. Turn it to the right..they go up. Don't feel singled out. There are a LOT of knobs (we call them filters) for about every type of page out there and we (webmasters) have been dealing with them for years.
The SE's don't rank sites, they rank pages. It is important to remember that.
PR is only one factor in 100+ in Google's algo, and has probably been devalued. Most blogs are more than one page and all blogs are on a domain. I could go into more detail here about how all that plays into what you are seeing, but it probably is getting into details that could get boring for most of you:>)
The third thing to be aware of is the overall linking structure of the web, in general.
Links come in and links go out.
There is a 'butterfly effect' on every link. Both outward and inward. Again, the 'no follow' attribute, (I almost typed 'tag', I'm so used to seeing it written that way!), is a fairly large butterfly and that effect is happpening as we speak.
I could go into ROS links, PR sales, nav structure, duplication, duplicate content, 302 hijacks, scrapers and many other concepts, but I'm trying to be basic here.
Although all of the above sounds as though I have something against SE's, nothing could be further from the truth. I've thoroughly enjoyed talking with the SE reps and discussing ideas. We both have businesses to run. I need them, they need me (us).
Going back to the original post by Mithras. I have a couple of thought questions for everyone:
If you removed the words, "Fables of the Reconstruction" entirely from your blog/website, would LSI still think your page is the most relevant for that term? Does the context support it? Is it the most relevant page on the Internet to find out about that term?
In closing I want to recommend something for the future, bloggers (we webmaster guys had to learn this lesson the hard way):
If you have a problem and the SE's have a problem, let the SE's solve their own problems. Yes, that means you need to solve your own, but then again, you might very well be part of the SE's problem. And, folks, sometimes that's a GOOD thing.
Posted by: Scott | February 20, 2005 at 03:55 PM
Whoops. I forgot another quick tip:
Google and the other SE's have a 'poison word' filter.
This includes any type of 'adult language'.
In otherwords: If you have adult language in your blog or on your pages, expect to be treated as an adult page, and to be filtered as such.
Posted by: Scott | February 20, 2005 at 04:04 PM
Mithras -- "Entartete kunst," my blog, comes up second on Google. I suspect that an unique name helps -- and that's no bull (heungh, heungh).
Posted by: Jim Shirk | February 20, 2005 at 04:32 PM
Blogs ? never mind them, what about other sites. Google’s completely stuffed up. I index 5 sites for a month and nothing !. And all the shit is No 1. Arseholes who have nothing to do with a search topic pop up everywhere. The sooner Microsoft launches something useful the better. I’m finished with the fools at Google. Watch them go down.
Posted by: Les | November 07, 2005 at 09:42 AM