In a previous post, I pointed out the vast majority of malicious spam SEO sites check the referrer string of the visitor. If this string does not include bing.com, yahoo.com or google.com, the user is not redirected to a malicious page.
We are releasing a Firefox add-on that uses this trick to protect Firefox users against most spam SEO threats, including fake AV and fake Video pages. This plugin works with Firefox 3.x. Click on the image below to install it. It is called "Search Engine Security".
 |
| Search Engine Security add-on installed |
How it works
This Firefox add-on handles Bing, Yahoo and Google search engines, in all languages. Normally, if a user clicks on a link within search engine results, the HTTP request to the external site contains a Referrer string from the search engine within the HTTP header. For example, if a user searches for "this is a test" in Google, any request to a search result will include the following Referrer:
Referer: http://www.google.com?q=this+is+a+test&hl=en&safe=active
For these requests, the add-on changes the Referrer header to a different value. This means that the requested page does not know that a given request came from a Google, Yahoo or Bing search. This is critical as Blackhat SEO pages only deliver malicious content (fake AV, Flash/Java updates, codecs, etc.) when requests come from the SEO results. Changing the Referer header, breaks the attack.The add-on does not change the referrer if you navigate within the same site (for example, inside google.com), or if your Referrer does not include a Bing, Yahoo or Google domain.
Configuration
You can customize the behavior of the Search Engine Security add-on. In the preferences menu, you can change the following values:
 |
| Search Engine Security preferences |
-Protect
Select the search engines for which you wish to enable protection.
- Use Referer header
Choose the Referer value to use for overriding the Google/Bing/Yahoo Referer. You can use an empty value, but it is recommended that you use a valid URL.
- Modify User-Agent (NEW in 1.0.8)
Most spam pages look at the Referrer value to decide whether or not to redirect users to a malicious page. However, in some cases like the Hot Video pages, only the User-Agent value is used. One common check is to look for "slurp" in the user-agent string to flag the request as coming form the Yahoo crawler. If you check the "Modify User-Agent" checkbox in the options, the string "slurp" is added to the User-Agent header when you leave Google/Bing/Yahoo in addition to overriding the Referrer header.
This option provides additional protection against malicious spam SEO.
-Whitelist
Some websites display a different page if you come from a search engine. When you use this add-on, the websites can no longer detect that you come from Google/Yahoo/Bing. If you are sure that a website is safe, you can add it to the whitelist. This will disable the add-on for this website.
If the URL matches any of the elements in the whitelist, the add-on does not change the Referer value. This is a string match and the match occurs if the URL includes one element of the whitelist. For example, http://www.expert-exchange.com/ can be whitelisted by adding:
- http://www.expert-exchange.com/ (also matches http://www.expert-exchange.com/foo)
- expert-exchange.com/ (matches any subdomain)
- expert-exchange. (matches the domains expert-exchange.net, expert-exchange.org, and paths like http://example.com/expert-exchange.html/)
- etc.
Notification (NEW in 1.0.4)
A notification is shown on Bing, Yahoo, and Google to let users know whether the SES protection is enabled for this search engine. The notification is shown under the search input.
 |
| Search Engine Security notification in Google search |
 |
| Search Engine Security notification in Bing search |
If you find any problem with this add-on, please let me know at jsobrier@zscaler.com.
-- Julien
Posts
37 comments:
I have the feeling you will piss of a lot of web masters who use the Google referrer to determine how many people are coming from Google, what search terms they are using, etc.
why are there a whole host of versions? The firefox add site has 1.03. This site has 1.04, updated to 1.05?
There are an older, widly customizable addon, named RefControl:
https://addons.mozilla.org/hu/firefox/addon/953/
And yeah, and sorry, I'll piss on all webmasters, who modify their webpage's content by the visitors referrer.
The current versions, as of 08/13/2010, is 1.0.5. I have updated the post to point to the latest version.
I released a couple of updates to the plugins:
* 1.0.1, 1.0.2: Better descriptions
* 1.0.3: link to this blog post
* 1.0.4: SES on/off shown in Google searches
* 1.0.5: SES on/off show in Yahoo and Bing as well
If you want the latest version, go to Tools > Add-ons > extensions. Look for Search Engine Security and right click > Find Update.
@ julian
Thanks for the info. I downloaded the add on via the official Firefox add on site. Right now it is still 1.0.3 and it says no update found when I try forced update. I guess the addon from the Firefox add on site is not updated yet to 1.0.5 or even 1.0.4.
https://addons.mozilla.org/en-US/firefox/addon/207647/
You can also disable Firefox's referrer manually: type about:config in the address bar and press Enter. Click through the reminder to be careful and navigate in the resulting list to network.http.sendRefererHeader. Double-click the entry to open the "Edit integer value" dialog. Change the value to 0 and click OK. The change will take effect when you restart Firefox.
Please update this add-on on addons.mozilla.org to the latest version.
Search Engine Security is still being reviewed by the official Firefox Add-ons sites. I did not realize that it was actually available to anybody before it was reviewed. I've uploaded version 1.0.6 to our website and the official Add-on site, so you should be able to update the plugin. 1.0.6 is functionally the same as 1.0.5, it only contains some internal code refactoring.
"You can also disable Firefox's referrer manually[...]"
This solution is less intrusive. It changes the Referrer only when the user is leaving Google/Yahoo/Bing. It is also easier to whitelist websites which display different content when you come from a search engine (like expert-exchange.com)
Does this add-on work with Google SSL Search? Google SSL Search removes referer when going to a http site, but not when going to a https site.
I'm using Firefox 4 beta 3 with Add-on Compatibility Reporter.
The "SES on" is visible on Google search page, but how can I test if it works? How can I know the referer isn't sent?
About "Google SSL Search": the add-on does modify the Referer for https website in Google SSL Search.
"I'm using Firefox 4 beta 3 with Add-on"
The add-on is set to work with Firefox 3.0 to 3.6.*, so I do not know how it will behave with Firefox 4.0
The easiest way to test SES is to install the Firefox plugin "Live HTTP Headers" This extension will show you the headers sent with each request. Or you can use a proxy such a Fiddler.
Hi,
Any idea as to when Mozilla will have finished testing this Addon?
I'm patiently waiting for it to get the green light.
The extensions has been approved by Mozilla today: https://addons.mozilla.org/en-US/firefox/addon/207647/
I need to upload a newer version.
Thanks julien, installing now.
Nice work btw :).
Hello Julien,
Sorry for my ignorance, but I'm not the developer, like you. :)
You wrote: 'There are currently no ultimate solutions for end-users to protect themselves against fake AV pages, fake videos and other malicious spam SEO ..'.
My experience tells me two easy solutions against these fake antivirus automatic redirections.
If I pass my infected link by a web proxy anonymizer, I do not get the malicious content, but innocent spam page only.
If I click on malicious link with a search engine using the New Window feature(eg. Yippy Search, Zuula Search), I do not get the malicious content, but innocent page ..
So: by proxy - no infection, by New Window in search engine - no infection.
But why - this is my question for you ..
Thank you for your response,
PROROOTECT
Yes, I wait patiently for your response, sorry :)
Prorootect :)
Sorry for my English, yes, it's me who asks the question: how to act proxies and search engines with 'New Window' feature - to do their job, which I described.
P.
@anonymous I don't think using a free anonymizer is a solution. You have no idea what they do with your data: do they add malicious content to the reply, how do they handle credentials and other private information you may need to enter on the final site, etc.
I have this on 3.6.13 Mac OS X. With it installed Firefox uses a lot more CPU.
@ae6dx, can you elaborate? The addon itself is only 11 k, so I doubt it.
However, I would like Julien Sobrier for this free addon. Became security conscious after hitting a fake AV link from a google search.
Now In my current browser config, Firefox+ Norton DNS+ WOT+ Zscaler SES = pretty secure browser all for free.
This seems like a great idea - my users are getting hit with these SEO malware a lot. Is there any way to do something like this in IE?
Please update the addon and mark it compatible with Firefox 5.*
This may seem like a stupid question, but WHY does the referral heading (e.g., Bing, Google, Yahoo, (or whoever), determine the instance of "Blackhat spam SEO"?
@all: the plug-in has been updated to work with Firefox 5.*
@anonymous: check out the this post to learn how Blackhat SEO posts differentiate between users and bots:http://research.zscaler.com/2010/09/hot-video-pages-analysis-of-hijacked_29.html
Please update Search Engine Security on AMO, and mark it compatible with Firefox 6
Please update the addon and mark it compatible with Firefox 6.*
ZScaler SES needs an update for FF6....
@Anonymous Thank you for the reminder, I've updated the plugin.
Why isn't version 1.2.1 on AMO?
The "Modify User-Agent" doesn't work. It has no effect, the user agent doesn't get modified.
Version 1.2.1 was released to include a newer maximum version of Firefox. There is no functional change. AMO took care of increasing the version number, so there was no need to create a new version.
@anonymyzer: the add-on adds "slurp" at the user agent string. Look for "referer mobilefish" in Google. Click on the first search result. The page from mobilefish.com shows your User Agent string as well as Referer. You can try it with Search Engine Security truned on and off for Google to see the different values.
Just thank you note for the addon. Precisely what I wanted--don't want to get myself notified for private search keyword but also keep most websites working untouched. For me the reason choosing was not because of advertisement websites but more of privacy. Regardless it still works fine for my use. Refcontrol was popular result when I searched referer for extension, but it's impossible to set refcontrol like SES or vice versa. So please keep up your good work (which I understand to be not as eas with current FF numbering scheme).
SES is marked incompatible and disabled in Firefox for Mobile 9
Please fix!
@All SES 1.2.3 is marked as compatible with Firefox Mobile 11 on our website, and version 15 on the Mozilla Add-on site
Post a Comment