使用Nginx对百度蜘蛛的301重定向跳转处理

先简单说下我碰到的情况。我一个域名的http页面被墙，国内用户可以通过https方式打开。为了seo，也遵循尽量不换网站域名的前提下，现在我想实现的目标是对国外蜘蛛（google...）和不在墙内的用户访问http页面时301重定向跳转到https，而百度蜘蛛访问原页面时301到一个新域名，新域名反向代理原来的网页，用户通过百度搜索点击打开的是原域名https页面。

1、修改原域名nginx配置成类似如下所示的内容

server {
listen 80 default;
server_name www.nbhao.org;
index index.html index.htm index.php default.html default.htm default.php;
root /home/wwwroot/nbhao;

#通过$http_user_agent判断是否是百度蜘蛛
if ($http_user_agent ~* Baiduspider){
return 301 http://www.hellonet8.com$request_uri;
}
#排除对一个网址的301重定向
if ($request_uri !~ ^/.*/.*notify_url\.php.*){
return 301 https://$server_name$request_uri;
}
location ~ .*\.(php|php5)?$
{
try_files $uri =404;
fastcgi_pass unix:/tmp/php-cgi.sock;
fastcgi_index index.php;
include fcgi.conf;
}

access_log /home/wwwlogs/www.nbhao.org.log www.nbhao.org;
}
server {
listen 443 ;
......
#https配置详见http://www.sijitao.net/1835.html
}

2、修改新域名nginx配置成如下

server
{
listen 80;
#listen [::]:80;
server_name www.hellonet8.com;
index index.html index.htm index.php default.html default.htm default.php;
root /home/wwwroot/hellonet8.com/www;
#如果用户不是百度蜘蛛且访问的网址不是baidu.*\.html格式
set $flag 0;
if ($http_user_agent !~* Baiduspider){
set $flag "${flag}1";
}
#用做百度站长平台的验证
if ($request_uri !~ ^.*/baidu.*\.html.*){
set $flag "${flag}2";
}
if ($flag = "012") {
return 301 https://www.nbhao.org$request_uri;
}
#接上不，如果是百度蜘蛛，执行proxy_pass反代。
location ^~ / {
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
if ($request_uri !~ ^.*/baidu.*\.html.*){
proxy_pass https://www.nbhao.org;
}
}
......
access_log /home/wwwlogs/www.hellonet8.com.log access;
}

3、通过curl模拟百度蜘蛛、google、普通用户验证

[stocky@s1 ~]$ curl -I -A Baiduspider www.nbhao.org
HTTP/1.1 301 Moved Permanently
Server: nginx/1.0.15
Date: Wed, 04 Mar 2015 08:59:27 GMT
Content-Type: text/html
Content-Length: 185
Connection: keep-alive
Location: http://www.hellonet8.com/
[stocky@s1 ~]$ curl -I -A Baiduspider www.hellonet8.com
HTTP/1.1 200 OK
Server: nginx
Date: Wed, 04 Mar 2015 09:01:29 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Vary: Accept-Encoding
Vary: Accept-Encoding
X-Powered-By: PHP/5.2.17p1
[stocky@s1 ~]$ curl -I -A google www.nbhao.org
HTTP/1.1 301 Moved Permanently
Server: nginx/1.0.15
Date: Wed, 04 Mar 2015 09:00:08 GMT
Content-Type: text/html
Content-Length: 185
Connection: keep-alive
Location: https://www.nbhao.org/
[stocky@s1 ~]$ curl -I -A google www.hellonet8.com
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Wed, 04 Mar 2015 09:02:07 GMT
Content-Type: text/html
Content-Length: 178
Connection: keep-alive
Location: https://www.nbhao.org/
[stocky@s1 ~]$ curl -I -A Mozilla www.nbhao.org
HTTP/1.1 301 Moved Permanently
Server: nginx/1.0.15
Date: Wed, 04 Mar 2015 09:01:59 GMT
Content-Type: text/html
Content-Length: 185
Connection: keep-alive
Location: https://www.nbhao.org/
[stocky@s1 ~]$ curl -I -A Mozilla www.hellonet8.com
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Wed, 04 Mar 2015 09:04:00 GMT
Content-Type: text/html
Content-Length: 178
Connection: keep-alive
Location: https://www.nbhao.org/

测试后发现满足要求。