前言:相信许多人都听说过.net开发过程中基于Lucene.net实现的全文索引,而Solr是一个高性能,基于Lucene的全文搜索服务器。同时对其进行了扩展,提供了比Lucene更为丰富的查询语言,同时实现了可配置、可扩展并对查询性能进行了优化,并且提供了一个完善的功能管理界面,是一款非常优秀的全文搜索引引擎,这里我就绕过Lucene,直接说Solr的应用了,总之,Solr比Lucene更加方便简洁好用,而且上手快,开发效率高。
Solr应用场景:涉及到大数据的全文搜索。尤其是电子商务平台还有现在流行的云计算,物联网等都是需要强大的数据量作为支撑的,使用Solr来进行数据 检索最合适不过了,而且Solr是免费开源的,门槛低、投资少见效快。关于Solr的一些优点我这里就不在累赘陈述了,园子里也有很多大神也写了很多关于 Solr的技术博文,我这里也只是抛砖引玉,见笑了。
好了,这里就开始Solr的奇幻之旅吧
基于.NET平台下的Solr开发步骤
一、搭建Solr服务器,具体步骤如下:
1.安装JDK,因为是.NET平台,不需要安装JRE、JAVA虚拟机,只安装JDK即可,而且安装JDK不需要手动去配置环境变量,它会自动帮我们配置好环境变量,很方便,这里我安装的是jdk1.7,官网地址:http://www.oracle.com/technetwork/java/javase/downloads/index.html
2.安装Tomcat8.0,官网地址:http://tomcat.apache.org/download-80.cgi,安装完成后启动Monitor Tomcat,浏览器地址栏输入http://localhost:8080/,能进入说明安装成功
3.下载Solr,这里我用的是Solr4.4版本,下载后进行下列配置
(1)解压Solr4.4,创建Solr目录,比如D:/SorlServer/one,将解压后的Solr4.4中的example目录下的Solr文件夹中的所有文件拷贝到创建的目录中
(2)创建Solr Web应用,具体步骤,将解压后的Solr4.4中的dist目录下的Solr-4.4.0.war文件拷贝到Tomcat下,比如C:\Program Files\Apache Software Foundation\Tomcat 7.0\webapps下,重命名为one.war,启动Tomcat后该文件会自动解压,进入到D:\SorlServer\one\collection1\conf下,打开solrconfig.xml文件,找到 节点改为${solr.data.dir:c:/SorlServer/one/data}
注意:这一步很重要:打开C:\Program Files\Apache Software Foundation\Tomcat 7.0\webapps\One\WEB-INF下的web.xml文件,找到节点开启,
将env-entry-value值改为D:/SorlServer/one,如下:
solr/home
D:/SorlServer/one
java.lang.String
(3)将解压后的Solr4.4下的/dist/solrj-lib目录中的所有jar包拷贝到C:\Program Files\Apache Software Foundation\Tomcat 7.0\lib中
(4)停止Tomcat,然后再启动,访问http://localhost:8080/one,即可打开
注意:如果是开发英文网站,我们就不需要使用第三方的分词配置,Solr本身就内置支持英文分词,如果是其他语种比如小语种(日语、意大利、法语等等),大家可以去网上找相关的分词包,这里我们以中文分词为例,毕竟国内大部分网站都是中文为主的。
4.配置中文分词,国内常用的分词器(庖丁解牛、mmseg4j、IKAnalyzer),这里我用的是IKAnalyzer,这个分词器比较活跃而且更新也快,挺好用的,具体步骤如下:
(1)将IKAnalyzer的jar包以及IKAnalyzer.cfg.xml都复制到C:\Program Files\Apache Software Foundation\Tomcat 7.0\webapps\one\WEB-INF\lib下
(2)配置D:\SorlServer\one\collection1\conf下的schema.xml,添加如下配置:
(3)停止Tomcat,然后再启动,访问http://localhost:8080/one/#/collection1/analysis,即可进行测试
以上是Solr服务器端的相关配置工作
二、开始基于.NET平台的Solr开发:
1.下载Solr客户端组件,我用的是园子里的Terry大哥的EasyNet.Solr,地址在微软开源站:http://easynet.codeplex.com/,
Terry大哥已经把solr客户端封装的很完善了,里面封装了很多现成的方法和参数配置,我们直接可以拿过来用,利用Easynet.solr创建索引,然后再查询索引,具体使用方法如下:
(1)下载EasyNet.Solr源码直接放到项目中,也可以将源码生成Dll组件后添加到项目引用进行使用,把源码放到项目中最好不过了,我们也可以对其进行调整来满足自己的需要
(2)创建索引实体类,就是我们要保存的索引数据,比如创建一个产品实体类
using System;using System.Collections.Generic;namespace Seek.SearchIndex{ public partial class IndexProductModel { public IndexProductModel() { } #region Properties public int ID { get; set; } public int ProductID { get; set; } public string ClassPath { get; set; } public int ClassID1 { get; set; } public int ClassID2 { get; set; } public int ClassID3 { get; set; } public string Title { get; set; } public string Model { get; set; } public string PriceRange { get; set; } public string AttributeValues { get; set; } public string ProductImages { get; set; } public int MemberID { get; set; } public System.DateTime CreateDate { get; set; } public System.DateTime LastEditDate { get; set; } public string FileName { get; set; } public string ProductType { get; set; } public string Summary { get; set; } public string Details { get; set; } public string RelatedKeywords { get; set; } public int MemberGrade { get; set; } #endregion }}
(3)配置Solr服务器端的xml,就是将咱们的这个索引实体类配置到Solr服务器上,进入D:\SorlServer\one\collection1\conf,打开schema.xml文件,配置如下
(4)开始创建索引,最好能写一个生成索引的客户端程序,我这里提供一下自己的索引器关键代码
using System;using System.Collections.Generic;using System.Linq;using System.Text;using Seek.SearchIndex;using System.Data;using System.Threading;using System.Configuration;using System.Reflection;using EasyNet.Solr;using EasyNet.Solr.Impl;using EasyNet.Solr.Commons;using System.Xml.Linq;using EasyNet.Solr.Commons.Params;using System.Threading.Tasks;namespace Seek.SearchIndex{ /// /// 索引器 /// public class Indexer { private readonly static OptimizeOptions optimizeOptiOns= new OptimizeOptions(); private readonly static CommitOptions commitOptiOns= new CommitOptions() { SoftCommit = true }; private readonly static ISolrResponseParser binaryRespOnseHeaderParser= new BinaryResponseHeaderParser(); private readonly static IUpdateParametersConvert updateParametersCOnvert= new BinaryUpdateParametersConvert(); private readonly static ISolrQueryConnection cOnnection= new SolrQueryConnection() { ServerUrl = ConfigurationManager.AppSettings["SolrServer"] }; private readonly static ISolrUpdateConnection solrUpdateCOnnection= new SolrUpdateConnection() { ServerUrl = ConfigurationManager.AppSettings["SolrServer"], COntentType= "application/javabin" }; private readonly static ISolrUpdateOperations solr = new SolrUpdateOperations(solrUpdateConnection, updateParametersConvert) { RespOnseWriter= "javabin" }; private readonly static ISolrQueryOperations solrQuery = new SolrQueryOperations(connection) { RespOnseWriter= "javabin" }; public enum State { /// /// 运行中 /// Runing, /// /// 停止 /// Stop, /// /// 中断 /// Break } /// /// 窗口 /// private Main form; /// /// 线程 /// public Thread t; /// /// 消息状态 /// public State state = State.Stop; /// /// 当前索引 /// private long currentIndex = 0; public long CurrentIndex { get { return currentIndex; } set { currentIndex = value; } } private int _startId = AppCongfig.StartId; public int StartId { get { return _startId; } set { _startId = value; } } /// /// 产品总数 /// private int productsCount = 0; /// /// 起始时间 /// private DateTime startTime = DateTime.Now; /// /// 结束时间 /// private DateTime endTime = DateTime.MinValue; private static object syncLock = new object(); #region 单利模式 private static Indexer instance = null; private Indexer(Main _form) { form = _form; productsCount = DataAccess.GetCount(0); //产品数统计 form.fullerTsslMaxNum.Text = productsCount.ToString(); form.fullerProgressBar.Minimum = 0; form.fullerProgressBar.Maximum = productsCount; } public static Indexer GetInstance(Main form) { if (instance == null) { lock (syncLock) { if (instance == null) { instance = new Indexer(form); } } } return instance; } #endregion /// /// 启动 /// public void Start() { ThreadStart ts = new ThreadStart(FullerRun); t = new Thread(ts); t.Start(); } /// /// 停止 /// public void Stop() { state = State.Stop; } /// /// 中断 /// public void Break() { state = State.Break; } /// /// 创建索引 /// public void InitIndex(object data) { var docs = new List(); DataTable list = data as DataTable; foreach (DataRow pro in list.Rows) { var model = new SolrInputDocument(); PropertyInfo[] properites = typeof(IndexProductModel).GetProperties();//得到实体类属性的集合 string[] dateFields = { "CreateDate", "LastEditDate" }; string field = string.Empty;//存储fieldname foreach (PropertyInfo propertyInfo in properites)//遍历数组 { object val = pro[propertyInfo.Name]; if (val != DBNull.Value) { model.Add(propertyInfo.Name, new SolrInputField(propertyInfo.Name, val)); } } docs.Add(model); StartId = Convert.ToInt32(pro["ID"]); } GetStartId(); lock (syncLock) { if (currentIndex <= productsCount) { form.fullerProgressBar.Value = (int)currentIndex; } form.fullerTsslCurrentNum.Text = currentIndex.ToString(); } var result = solr.Update("/update", new UpdateOptions() { Docs = docs }); } /// /// 创建索引 /// public void CreateIndexer(DataTable dt) { GetStartId(); Parallel.ForEach(dt.AsEnumerable(), (row) => { //从数据库查询商品详细属性 if (row != null) { var docs = new List(); var model = new SolrInputDocument(); PropertyInfo[] properites = typeof(IndexProductModel).GetProperties();//得到实体类属性的集合 string[] dateFields = { "CreateDate", "LastEditDate" }; string field = string.Empty;//存储fieldname foreach (PropertyInfo propertyInfo in properites)//遍历数组 { object val = row[propertyInfo.Name]; if (val != DBNull.Value) { model.Add(propertyInfo.Name, new SolrInputField(propertyInfo.Name, val)); } } docs.Add(model); StartId = Convert.ToInt32(row["ID"]); var result = solr.Update("/update", new UpdateOptions() { Docs = docs }); } }); //GetStartId(); lock (syncLock) { if (currentIndex <= productsCount) { form.fullerProgressBar.Value = (int)currentIndex; } form.fullerTsslCurrentNum.Text = currentIndex.ToString(); } } /// /// 全部索引运行 /// public void FullerRun() { //GetStartId(); //form.fullerTsslCurrentNum.Text = currentIndex.ToString(); DataTable dt = DataAccess.GetNextProductsInfo(StartId); StartId = AppCongfig.StartId; if (state == State.Break) { this.SendMesasge("完全索引已继续,起始ID[" + StartId + "]..."); } else { startTime = DateTime.Now; this.SendMesasge("完全索引已启动,起始ID[" + StartId + "]..."); } state = State.Runing; form.btnInitIndex.Enabled = false; form.btnSuspend.Enabled = true; form.btnStop.Enabled = true; while (dt != null //单线程 // CreateIndexer(dt);//多线程 } catch (Exception ex) { state = State.Stop; form.btnInitIndex.Enabled = true; form.btnSuspend.Enabled = false; form.btnStop.Enabled = false; GetStartId(); this.SendMesasge(ex.Message.ToString()); } form.fullerTsslTimeSpan.Text = "已运行 :" + GetTimeSpanShow(DateTime.Now - startTime) + ",预计还需:" + GetTimeSpanForecast(); try { dt = DataAccess.GetNextProductsInfo(StartId);//获取下一组产品 } catch (Exception err) { this.SendMesasge("获取下一组产品出错,起始ID[" + StartId + "]:" + err.Message); } } if (state == State.Runing) { state = State.Stop; form.btnInitIndex.Enabled = true; form.btnSuspend.Enabled = false; form.btnStop.Enabled = false; AppCongfig.SetValue("StartId", StartId.ToString()); this.SendMesasge("完全索引已完成,总计索引数[" + currentIndex + "]结束的产品Id" + StartId); } else if (state == State.Break) { GetStartId(); state = State.Break; form.btnInitIndex.Enabled = true; form.btnSuspend.Enabled = false; form.btnStop.Enabled = false; AppCongfig.SetValue("StartId", StartId.ToString()); this.SendMesasge("完全索引已暂停,当前索引位置[" + currentIndex + "]结束的产品Id" + StartId); } else if (state == State.Stop) { GetStartId(); state = State.Stop; this.SendMesasge("完全索引已停止,已索引数[" + currentIndex + "]结束的产品Id" + StartId); form.btnInitIndex.Enabled = true; form.btnSuspend.Enabled = false; form.btnStop.Enabled = false; AppCongfig.SetValue("StartId", StartId.ToString()); productsCount = DataAccess.GetCount(StartId); //产品数统计 form.fullerTsslMaxNum.Text = productsCount.ToString(); form.fullerProgressBar.Minimum = 0; form.fullerProgressBar.Maximum = productsCount; } endTime = DateTime.Now; } /// /// 多线程构建索引数据方法 /// /// public void MultiThreadCreateIndex(object threadDataParam) { InitIndex(threadDataParam); } /// /// 获取最大的索引id /// private void GetStartId() { IDictionary
(5)运行索引器,创建索引,这里是我的索引器界面,如图
可以随时跟踪索引生成的情况
(6)索引创建完毕后,可以进入Solr服务器界面http://localhost:8080/one/#/collection1/query进行测试
1.全文搜索,分词配置,以及类似于谷歌和百度那种输入关键字自动完成功能
2.Facet查询
.NET开发过程中的全文索引使用技巧之Solr(转)