research

Death of the SEO Copywriters - Spam Detection with Phrase Based Information Retrieval

Marketing Fan — Fri, 29 Dec 2006 14:03:12 +0100

Bill Slawski of SEObytheSea has a great post up explaining a concept of how search engines (Google, man!) do a phrase based analysis – of your content to assign quality measures to it and possibly put it into the wastebasket or at least supplemental index.

The idea is that quality documents have a different co-occurrence of certain phrases (“money-words”) than spammy or low quality articles you bought for two dollars each from that low-quality writer in India recently who wasn’t even aware of how to use Word properly, not to speak about creating quality content…

Certainly a “SEOed” article around a phrase, let’s say “President of the united states” would use that term in all variations, word order and such.

A quality article really talking about the President of the united states would probably mention other “unimportant” things like names of past presidents, non-important things like amorous adventures, hollywood careers or other generally bad habits of those big guys that nobody would place an Adwords bid on for example.

The search engines just create a co-occurance matrix for all phrases in the document and match those statistics against other quality documents.

patent,bill wrote:

From the foregoing, the number of the related phrases present in a given document will be known. A normal, non-spam document will generally have a relatively limited number of related phrases, typically on the order of between 8 and 20, depending on the document collection. By contrast, a spam document will have an excessive number of related phrases, for example on the order of between 100 and 1000 related phrases. Thus, the present invention takes advantage of this discovery by identifying as spam documents those documents that have a statistically significant deviation in the number of related phrases relative to an expected number of related phrases for documents in the document collection.

So short – that patent and the wonderful clear exlpanation by Bill outlines pretty well, that Google & co DO have the means and technology to judge on content quality …

and that is the death for all SEO “copywriters” just focussing on keyword density, repetition and keyword stuffing.

What does that mean for you if you HIRE a writer for creating content?

DO NOT overdo your specifications concerning keyword phrases to use!

Especially in the last months I have seen content rank GREAT on Google (if on the right domains) for related phrases versus phrases that were really used in the content… you don’t need to have an exact mention of a keyword phrase for it to be found on Google anymore!

NO, it could even harm you nowadays – that’s the next phase of overoptimization penalties – create good, natural content and RANK!

Indirect linking - Truncated Page Rank and getting rid of link buying penalties

Marketing Fan — Wed, 06 Dec 2006 15:11:42 +0100

I figured over the last weeks, that some partners or clients are not aware of the great possbility of natural link building "the other way round" ... getting a presell page and then boosting that page with a few links to get the effect of those links indirectly to their own site… that’s why I call this concept indirect linking

The mindset "I need more PR" was born when every direct link to a site contributed to it’s direct pagerank and even better, it’s direct ranking – that was somewhere in 2001 or so. Well – those days are gone ... the search engine engineers know that buying links is mainstream, and have two strategies to fight that kind of link popularity manipulation – a) say that buying links is bad (you bad bad boy paying somebody money for linking to you… ) and b) and more effective – discount the value of direct links...

There is a concept that was documented as "truncated page rank" in the paper "Using Rank Propagation and Probabilistic Counting for Link-Based Spam Detection" which I also had on my TO-READ list of TOP SEO resarch papers since back in July (you should really bookmark that page and check back often :-)

What you read there is of course by no means a confirmation what Google or others implemented, but it gives you an idea of how they work – and I like their conclusions on how to filter "link spam" with the method of applying a heavy damping factor to the direct links.

It is based on the assumption that people that pay money for links, only direct it to their own sites… (that was a crazy though in 2002 – why pay that guy just to link to me … "oh my god, they pay me 5 bucks for a link on my blog – but today it’s common thinking)

[...]where is a damping factor between 0 and 1, typically 0.85.

A page participating in a link farm can gain a high Page-

Rank score because it has many in-links, this is, supporters

that are topologically "close" to the target node. Intuitively,

a way of demoting those pages could be to consider a damp-

ing function that ignores the direct contribution of the

first levels of links,

[...]

This function penalizes pages that obtain a large share

of their PageRank from the first few levels of links; we call

the corresponding functional ranking the Truncated Page-

Rank of a page. This is similar to PageRank, except that

the supporters that are too "close" to a target node, do not

contribute towards its ranking.

So while the first level links are discounted, the second and third level links are not, or at least not to that extend. And the reason is simpe as that

Why should I pay for a link to a site that is NOT mine?

Please note that I’m not saying that direct links (be they bought or leased) dont work… they work just over time, at a discounted strength – and IMHO we are already used to link damping over time for 2 years now… But still, getting strong links to (old) pages that link to you help both – the page you link to AND your own site!

So here’s my advice to get the most out of your presell page (AKA hosted marketign page):

once you get a pre sell page on an authority site, be it an old .edu site or another old and trusted domain, don’t let it there alone – include it in your link building plan
yes, that means linking to a site that might not be owned by you.. so what? you pay people to link to let those links help you!
chances are VERY GOOD that this presell page will RANK. and even OUTRANK your own site! good – you will get traffic AND link juice from that page
so you will increase your traffic from both – indirect traffic from human visitors from the presell page and your own rankings from the links of that high ranking page of course

Talking about the thing called "reputation management" – that’s a similar concept… buy pages on old sites that talk good about you or your company and let them rank ... voila’ that means your reputation on a typical Google search will increase :-) ...

oh – and did I mention your chance of getting traffic increases by 30-100% by having a second entry in the search results?

Now THAT is an incentive, eh?

Note: this post was also sent out end of August to my client-only newsletter … if you are interested in getting such stuff earlier, you might consider becoming a client, too.

SpamRank - my interpretation

Marketing Fan — Sun, 06 Aug 2006 19:59:10 +0200

One of the papers I listed in The Top 15 SEO research papers is about SpamRank with the goal of Fully Automatic Link Spam Detection

Preparing for the SES tomorrow, I want to share my observations and interpretations of that WWW2005 paper

Goal

obviously search engines need a good and automatic way to identifying link networks and link farms – sites that are built soley for inflating page rank or a group of webmasters that join together for a link rink or a link exchange network

Method

research paper wrote:

key assumption is that

supporters of an honest page should not be overly dependent on one another, i.e. they should be spread

across sources of different quality

This means having too many high ranking sites is bad, while having only low ranking sites is bad too – emulate natural web

Quote:

as in the case of the entire Web, the PageRank distribution of an

honest set of supporters should be power law.

You want a natural linking pattern for your sites

The two key observations in detecting link farms

assumption 1 wrote:

Portions of the Web are self-similar; an honest set of supporter pages arise by independent actions of individuals and organizations that build a structure with properties similar to the entire Web. In particular, the PageRank of the supporters (ed: the linking sites) follows a power law distribution just as the case for the entire Web.

Again, you want a natural linking pattern for your sites

assumption 2 wrote:

Link spammers have a limited budget; when boosting the PageRank of a target page, “unimportant” structures are not replicated.

If you need “unimportant” structures for your site, then go for low-value links like PR0, PR1, PR2, links from uncached pages, links from new sites, hell – any link – just make sure you build a GOOD, NATURAL (again) mix

So how can I lower my SpamRank ?

These are my interpretations & recommendations only:

Get a good mix of backlinks, where the PR follows a power-principle, say a power of 3 have 3^5 = 243 PR0, 81 PR1, 27x PR2, 9 PR3, 3 PR4 and 1 PR5 … you get the idea?
THEREFORE: Don’t bother buying links on High PR pages like PR9,PR8 that sure will stick out of your link profile immediately
Look at your competitors and their backlink profiles and try to emulate that
Get links from within content (presell pages), not only sidebar, navigational or footer links
Add “useless” pages like About, Sitemap, Contact us etc to your sites

Add “useless” links to your sites – some nofollow links, some PR0, some uncached

Hell – and don’t count on every single link… make your site evolve naturally, the days where you bought 3 PR7 and got a PR6 next month are over for YEARS... (I still get questions from people who believe those 2003ish myths that are burried in some old abandoned webmaster forums and minds)

Two other algorithms are cited and I’m sure they are in development or production already

B. Wu and B. D. Davison. Identifying link farm pages.

and with 22 scientific citations and only via payables from Springs this here

H. Zhang, A. Goel, R. Govindan, K. Mason, and B. V. Roy. Making eigenvector-based reputation systems robust to collusion.

More to read on the topic of link farms and link spam detection is listed here

Yahoo’s Link Farm detection patent

Using Rank Propagation and Probabilistic Counting for Link-Based Spam Detection

Undue Influence: Eliminating the Impact of Link Plagiarism on Web Search Rankings

Sidenote: Barry and others already reported from last years’ Chicago show (I think it was Chicago) that Matt Cutts from Google was playing with

his notebook to point out rented links and links from link farms.

While it’s not clear that thes algos are in effect for all search queries, all websites, all countries or languages, it’s sure Google & Co have implemented MANY of them – as the pure calculation is pretty simple once you have the huge database of web site vectors (interlink data) as Google & Co have.