supplemental index

Google Bowling via Proxy

Marketing Fan — Thu, 16 Aug 2007 20:00:22 +0200

So with SES San Jose just around the corner, Dan Thies put up a great post detailling all the headaches that a website owner could get when looking at his serps a bit closer or with the right tools to do so …

Dan calls this “Google Proxy Hacking”, but frankly, we are not hacking any of Google’s proxies – so I’m talking about Google Bowling via Proxy Sites – related to the older black hat term “Google Bowling” for buying too many / bad links for competitor sites to knock them off the serps. Yes, it IS possible to knock a competitor site off the SERPs, altought Google says= there is ~~nothing~~ almost nothing a competitor can do to harm you (yeah, right – the Google folks weakened this message some months ago, because the “nothing” was plain wrong – and they knew it).

If you read thru Dan’s post you might get headaches just like this guy from all those details and the partly wrong promises for a cure for it with two solutions that BOTH address only the outdated part of the problem.

So I though I have to illustrate to you what’s going on and how Google Bowling via Proxies actually looks like

The above results are returned if you search for the unique phrase

which is was only found on my company site cemper.com … (ok – now it’s also found on this marketingfan.com

blog and on marketingfan.at as soon as we translate it)

But WTH is Proxy Dust ???

As you can see this unique phrase which should id if my page is healthy does not show my own site but “one of those PITA sites”: run by a guy called Matt Twine from the UK (if that IS his real name…)

and as you can image the url http://www.proxydust.com/index.php?q=aHR0cDovL3d3dy5jZW1wZXIuY29t has an EXACT copy of my company site’s home page there…

Did I hear Spam Report? yadda yadda – don’t bother – the Googlers don’t seem to care, because I submitted that 2 weeks ago…

But it get’s worse

Now clicking that “filter=0” to reveal all search results we see this HUGE list of pages – cemper.com coming second…. as a filtered result right after that proxy site used for google bowling…

[... pages cut out here … ]

But also we have a couple more scumbags stealing my content and trying to hijack my site…

In fact only the ProxyDust copy wins big time over CEMPER.COM because … believe it or not…

that fricking domain registered in January 2007 got a Wikipedia backlink

And my site does not.

I currently think that’s the main reason why Google chose them over my own site – which is from 2000, not heavily SEOed, but I bet a handful more trusted than this Mark “Thief” Twine’s site.

Well, it might well be that Mark has NO CLUE about what he does, but all those ads plastered around my site indicate different.

In fact it appears the whole strategy of running those proxy sites is to earn money from the ads placed on other’s content and cashing in on their work…

What we (legit webmasters) can do…

Frankly, I love Dan’s general post as an introduction to this post, because I would have hated to explain it in all length as he did. ~~But what he points out as “solutions” are somewhat old school methods to identify bots that pretend to be Google, Yahoo or MSNbot….~~

Dan’s post ALSO contains the 2nd method for sending ALL visitors a “noindex, nofollow” that do NOT

1) Identify as spiders

2) Pass a “valid IP address” test

Pretty cool – I think that might work – and will test this ASAP, in addition to my own method of blocking those scumbags.

What about you?

Has YOUR site been hijacked? Do you know?

How you could know? Just follow Jim’s post to find if a page is in supplemental … but actually make sure you look at the results closely… because what you might find is that somebody is stealing your content….

You should do that for EVERY PAGE of your site – best case – if you got the right tools for it…. but it costs a lot of resources either way – by hand or by machine tool.

Let me know about YOUR hijack experiences !

(and I’m sure people should talk about this at the SES in San Jose , however I fear they won’t too much…)

Update: You could of course get around the initial problem of having too less trust in Google by getting real juicy authority links using MSN’s linkfromdomain command by effectively even letting your competitor link indirect to you ... obviously you still want to make sure you get only the juicy pages and not spend your time with dead meat.

Why Removed Supplemental Index Labels are good for my business

Marketing Fan — Sat, 04 Aug 2007 12:50:18 +0200

Earlier this week Google has removed the “supplemental index” labels from the SERPs, and as with every major poops from Google the whole SEO scene freaked out on this!

Me too – because I have to thank Google for giving me – and my clients a new competitive advantage.

Why do I thank Google for removing interesting signals?

Well, until last week every wannabeo-seo and his mother could see (in the SERPs, see grandfathered sample below)

if a page had a problem with ranking…

There have been huge posts by Jim , Halfdeck and Halfdeck again and a lot more on what/why/where supplementals are. Even I posted about Supplemental Hell and one PayPerPost link buying penalty bringing pages into the to supplemental index.

Today however…

People need to put more effort into detecting if a page has problem due to being in the supplemental index.

That means a certain (large) amount of SEOs just won’t be able to do this in their everyday job.

Halfdeck has his own Supplemenal Detector which is a fancy JAVA application that is in fact a “pagerank emulator” – ~~and all pages below a certain threshold are marked as supplemental. I encourage you to download this data scraper, and I’m sure it works nicely – but haven’t tried it.~~

After playing with Halfdeck’s Pagerank emulator I must say that it’s a great way to simulate how the “link juice” flows thru your site and where you are actually wasting precious link juice (i.e. on useless stats pages). Halfdeck even implemented a “backlink emulator” where you can judge on the effects of an additional PRx link to any page you like… pretty cool tool – it just lacks TBPR live queries, but I hope he can add that in the next version.

But, in fact I never cared much about the TOTAL number of supplementals, but always if a single page is in supplemental. Why that?

Well, I guess Link Ninja Master Jim Boykin knows why – it’s because you don’t want to get links on pages in the supplemental index because they won’t get crawled as often.

Jim’s recent explanation on finding if a page is in supplemental pretty well details how to detect if a page is “healthy” at all – i.e. ranks for obscure terms.

If a page does not even rank for an obscure terms on it, you don’t need a link there.

So you ask again, why is this cool for your business?

Because the way to check if a page is worthy to spend time to get a link on it has just become a bit harder. You will need more work, time, effort, unless you automate it. Just as we do here.

And this is the perfect situation to use an “internal tool” (as many SEOs have) as a competitie advantage to get more and better links in a shorter time… heck – some link builders might spend another couple clicks on each page to find out if it qualifies for hunting for link.

We Don’t :-)

In fact we already see the “Supplemental Index” on top of our brower toolbar bar when we visit a page.

In fact we already see the “Supplemental Index” label as it used to be printed for ALL google users in the past, nicely embedded in the SERPs.

And in fact Google has bought itself now 10-20 more Google queries per SERP page my Link Arbeiter team screens when looking for links – plus we are inflating the pageloads of all those sites we screen by one…

Do you think that hurts Google? Nah – enough resources.

Do you think it hurts me? Nah – just got a bunch more proxy IPs to make up for the bigger Google scraping load.

Do you think it will hurt the SEO scene building links? Well …

I would assume a huge bunch of people won’t even notice a difference – after all even SEOmoz de-classified 70% of SEO companies for not knowing the SEO basic

Furthermore a couple of smaller equipped companies will struggle as they will need to do more a lot more work (as per Jim’s description) to get the same results…. and I mean – A LOT MORE.

Now this moves the benefits to larger scale companies (as Google is) that DO have an intact SEO infrastructure for their daily SEO work.

But the small scale link builders will first have to build that infrastructure, browser plugins and knowledge to make visible what Google has just taken from the public.

Thanks Google !

Google Infrastructure update - pagerank, supplementals, indexing

Marketing Fan — Thu, 11 Jan 2007 17:50:57 +0100

Today I found that Matt Cutts, mr. GoogleGuy himself had two really super posts on his blog

First Pagerank update 2007 – for those obsessed, yes, they are updating it once again… some new values here and there… who cares at all?

All remaining words on the green bar are here at Jim’s blog who is bitching again about people caring too much (or anything at all) about page rank (but he’s also got a nice post today with his internal link valuation tool )

Matt Cutts also gives another official explanation on what supplemental pages are, how they work and that they are improving their crawl frequency…

so supplementals are not older than 4 MONTHS now anymore… :-)

Another funny thing is Matt bitching about email authentication including pointing to concepts like domainkeys that I already bitched about some weeks before

Looks like Mr. Cutts was hit by the same amount of RE: my crap spam that was hitting my own (G)mail accounts – undetected :-)

LOL

Death of the SEO Copywriters - Spam Detection with Phrase Based Information Retrieval

Marketing Fan — Fri, 29 Dec 2006 14:03:12 +0100

Bill Slawski of SEObytheSea has a great post up explaining a concept of how search engines (Google, man!) do a phrase based analysis – of your content to assign quality measures to it and possibly put it into the wastebasket or at least supplemental index.

The idea is that quality documents have a different co-occurrence of certain phrases (“money-words”) than spammy or low quality articles you bought for two dollars each from that low-quality writer in India recently who wasn’t even aware of how to use Word properly, not to speak about creating quality content…

Certainly a “SEOed” article around a phrase, let’s say “President of the united states” would use that term in all variations, word order and such.

A quality article really talking about the President of the united states would probably mention other “unimportant” things like names of past presidents, non-important things like amorous adventures, hollywood careers or other generally bad habits of those big guys that nobody would place an Adwords bid on for example.

The search engines just create a co-occurance matrix for all phrases in the document and match those statistics against other quality documents.

patent,bill wrote:

From the foregoing, the number of the related phrases present in a given document will be known. A normal, non-spam document will generally have a relatively limited number of related phrases, typically on the order of between 8 and 20, depending on the document collection. By contrast, a spam document will have an excessive number of related phrases, for example on the order of between 100 and 1000 related phrases. Thus, the present invention takes advantage of this discovery by identifying as spam documents those documents that have a statistically significant deviation in the number of related phrases relative to an expected number of related phrases for documents in the document collection.

So short – that patent and the wonderful clear exlpanation by Bill outlines pretty well, that Google & co DO have the means and technology to judge on content quality …

and that is the death for all SEO “copywriters” just focussing on keyword density, repetition and keyword stuffing.

What does that mean for you if you HIRE a writer for creating content?

DO NOT overdo your specifications concerning keyword phrases to use!

Especially in the last months I have seen content rank GREAT on Google (if on the right domains) for related phrases versus phrases that were really used in the content… you don’t need to have an exact mention of a keyword phrase for it to be found on Google anymore!

NO, it could even harm you nowadays – that’s the next phase of overoptimization penalties – create good, natural content and RANK!