Error message here!

Hide Error message here!

忘记密码?

Error message here!

请输入正确邮箱

Hide Error message here!

密码丢失?请输入您的电子邮件地址。您将收到一个重设密码链接。

Error message here!

返回登录

Close

zookeeper配置中心实战--solrcloud zookeeper配置中心原理及源码分析

一天不进步,就是退步 2019-02-25 16:05:00 阅读数:376 评论数:0 点赞数:0 收藏数:0

程序的发展,需要引入集中配置

  • 随着程序功能的日益复杂,程序的配置日益增多:各种功能的开关、参数的配置、服务器的地址……
  • 并且对配置的期望也越来越高,配置修改后实时生效,灰度发布,分环境、分集群管理配置,完善的权限、审核机制……
  • 并且随着采用分布式的开发模式,项目之间的相互引用随着服务的不断增多,相互之间的调用复杂度成指数升高,每次投产或者上线新的项目时苦不堪言,因此需要引用配置中心治理。

有哪些开源配置中心

1.spring-cloud/spring-cloud-confighttps://github.com/spring-cloud/spring-cloud-configspring出品,可以和spring cloud无缝配合1.淘宝 diamondhttps://github.com/takeseem/diamond已经不维护1.disconfhttps://github.com/knightliao/disconfjava开发,蚂蚁金服技术专家发起,业界使用广泛1.ctrip apollohttps://github.com/ctripcorp/apollo/Apollo(阿波罗)是携程框架部门研发的开源配置管理中心,具备规范的权限、流程治理等特性。

配置中心的实现方式可以使用数据库如mysql,可以使用缓存数据如redis,mongodb等,也可以使用zookeeper,zookeeper的watcher特性使它天然具有配置中心的属性。

1.solr zookeeper配置中心搭建(windows环境)

1.1 下载solr 

下载最新的solr https://lucene.apache.org/solr/mirrors-solr-latest-redir.html

我此时下载的最新版本为 solr-7.7.0.zip

解压到本地目录

E:demosolr-7.7.0

 

       1.2 启动solr

   CMD进入bin目录下,执行 

solr.cmd start -e cloud

按照提示创建solr cloud实例和分片和collection:techproducts

,详细参考官方文档:https://lucene.apache.org/solr/guide/7_6/solr-tutorial.html

回到E:demosolr-7.7.0目录,CMD执行导入数据命令: java -jar -Dc=techproducts -Dauto exampleexampledocspost.jar exampleexampledocs/*

       1.3  访问admin 

           http://localhost:8983

         查看内置zookeeper状态

       

 

  创建了一个9983端口的zk实例

    1.4 使用ZooInspector监控查看

运行脚本@echo off cd D:softwarezookeeper-3.4.6ZooInspectorbuild d: Java -Dfile.encoding=UTF-8 -jar zookeeper-dev-ZooInspector.jar

 

2.配置中心文件的上传,下载功能实现

   本文仅实现上传功能,下载功能由读者自行实现

  2.1 上传配置文件:进入E:demosolr-7.7.0/bin目录,CMD执行solr.cmd zk cp ../LICENSE.txt zk:/test/LICENSE.txt -z localhost:9983

此时在zk下面创建了一个test目录,目录下面有一个license.txt节点,数据即为license.txt文件内容。

 2.2 源码分析

   用记事本打开solr.cmd命令,找到parsezkargs参数的地方,发现解析完zk参数就启动了运行zk命令方法run_zk:runzk IF"!ZKOP!"==""( set"ERRORMSG=Invalid command specified for zk sub-command"goto zkshortusage ) IF"!ZKHOST!"==""( set"ERRORMSG=Must specify -z zkHost"goto zkshortusage ) IF"!ZKOP!"=="-upconfig"( set ZKOP="upconfig") IF"!ZKOP!"=="-downconfig"( set ZKOP="downconfig") IF"!ZKOP!"=="upconfig"( IF"!CONFIGSETNAME!"==""( set ERRORMSG="-n option must be set for upconfig"goto zkshortusage ) IF"!CONFIGSETDIR!"==""( set ERRORMSG="The -d option must be set for upconfig."goto zkshortusage )"%JAVA%" %SOLRSSLOPTS% %AUTHCOPTS% %SOLRZKCREDSANDACLS% -Dsolr.install.dir="%SOLRTIP%" ^ -Dlog4j.configurationFile="file:///%DEFAULTSERVERDIR%resourceslog4j2-console.xml" ^ -classpath "%DEFAULTSERVERDIR%solr-webappwebappWEB-INFlib/;%DEFAULTSERVERDIR%libext/" ^org.apache.solr.util.SolrCLI!ZKOP! -confname !CONFIGSETNAME! -confdir !CONFIGSETDIR! -zkHost !ZKHOST! %ZKVERBOSE%^ -configsetsDir "%SOLRTIP%/server/solr/configsets") ELSE IF"!ZKOP!"=="downconfig"( IF"!CONFIGSETNAME!"==""( set ERRORMSG="-n option must be set for downconfig"goto zkshortusage ) IF"!CONFIGSETDIR!"==""( set ERRORMSG="The -d option must be set for downconfig."goto zkshortusage )"%JAVA%" %SOLRSSLOPTS% %AUTHCOPTS% %SOLRZKCREDSANDACLS% -Dsolr.install.dir="%SOLRTIP%" ^ -Dlog4j.configurationFile="file:///%DEFAULTSERVERDIR%resourceslog4j2-console.xml" ^ -classpath "%DEFAULTSERVERDIR%solr-webappwebappWEB-INFlib/*;%DEFAULTSERVERDIR%libext/*" ^org.apache.solr.util.SolrCLI!ZKOP! -confname !CONFIGSETNAME! -confdir !CONFIGSETDIR! -zkHost !ZKHOST! %ZKVERBOSE%) ELSE IF"!ZKOP!"=="cp" ( IF "%ZKSRC%"=="" ( set ERRORMSG=" must be specified for 'cp' command" goto zkshortusage ) IF "%ZKDST%"=="" ( set ERRORMSG= must be specified for 'cp' command" goto zkshortusage ) IF NOT "!ZKSRC:~0,3!"=="zk:" ( IF NOT "!%ZKDST:~0,3!"=="zk:" ( set ERRORMSG="At least one of src or dst must be prefixed by 'zk:'" goto zkshortusage ) ) "%JAVA%" %SOLRSSLOPTS% %AUTHCOPTS% %SOLRZKCREDSANDACLS% -Dsolr.install.dir="%SOLRTIP%" ^ -Dlog4j.configurationFile="file:///%DEFAULTSERVERDIR%resourceslog4j2-console.xml" ^ -classpath "%DEFAULTSERVERDIR%solr-webappwebappWEB-INFlib/;%DEFAULTSERVERDIR%libext/" ^ org.apache.solr.util.SolrCLI !ZKOP! -zkHost !ZKHOST! -src !ZKSRC! -dst !ZKDST! -recurse !ZKRECURSE! %ZKVERBOSE%) ELSE IF"!ZKOP!"=="mv"( IF"%ZKSRC%"==""( set ERRORMSG=" must be specified for 'mv' command"goto zkshortusage ) IF"%ZKDST%"==""( set ERRORMSG=" must be specified for 'mv' command"goto zkshortusage )"%JAVA%" %SOLRSSLOPTS% %AUTHCOPTS% %SOLRZKCREDSANDACLS% -Dsolr.install.dir="%SOLRTIP%" ^ -Dlog4j.configurationFile="file:///%DEFAULTSERVERDIR%resourceslog4j2-console.xml" ^ -classpath "%DEFAULTSERVERDIR%solr-webappwebappWEB-INFlib/*;%DEFAULTSERVERDIR%libext/*" ^org.apache.solr.util.SolrCLI!ZKOP! -zkHost !ZKHOST! -src !ZKSRC! -dst !ZKDST! %ZKVERBOSE%) ELSE IF"!ZKOP!"=="rm"( IF"%ZKSRC"==""( set ERRORMSG="Zookeeper path to remove must be specified when using the 'rm' command"goto zkshortusage )"%JAVA%" %SOLRSSLOPTS% %AUTHCOPTS% %SOLRZKCREDSANDACLS% -Dsolr.install.dir="%SOLRTIP%" ^ -Dlog4j.configurationFile="file:///%DEFAULTSERVERDIR%resourceslog4j2-console.xml" ^ -classpath "%DEFAULTSERVERDIR%solr-webappwebappWEB-INFlib/*;%DEFAULTSERVERDIR%libext/*" ^org.apache.solr.util.SolrCLI!ZKOP! -zkHost !ZKHOST! -path !ZKSRC! -recurse !ZKRECURSE! %ZKVERBOSE%) ELSE IF"!ZKOP!"=="ls"( IF"%ZKSRC"==""( set ERRORMSG="Zookeeper path to remove must be specified when using the 'ls' command"goto zkshortusage )"%JAVA%" %SOLRSSLOPTS% %AUTHCOPTS% %SOLRZKCREDSANDACLS% -Dsolr.install.dir="%SOLRTIP%" ^ -Dlog4j.configurationFile="file:///%DEFAULTSERVERDIR%resourceslog4j2-console.xml" ^ -classpath "%DEFAULTSERVERDIR%solr-webappwebappWEB-INFlib/*;%DEFAULTSERVERDIR%libext/*" ^org.apache.solr.util.SolrCLI!ZKOP! -zkHost !ZKHOST! -path !ZKSRC! -recurse !ZKRECURSE! %ZKVERBOSE%) ELSE IF"!ZKOP!"=="mkroot"( IF"%ZKSRC"==""( set ERRORMSG="Zookeeper path to create must be specified when using the 'mkroot' command"goto zkshortusage )"%JAVA%" %SOLRSSLOPTS% %AUTHCOPTS% %SOLRZKCREDSANDACLS% -Dsolr.install.dir="%SOLRTIP%" ^ -Dlog4j.configurationFile="file:///%SOLRSERVERDIR%resourceslog4j2-console.xml" ^ -classpath "%DEFAULTSERVERDIR%solr-webappwebappWEB-INFlib/*;%DEFAULTSERVERDIR%libext/*" ^org.apache.solr.util.SolrCLI!ZKOP! -zkHost !ZKHOST! -path !ZKSRC! %ZKVERBOSE%) ELSE ( set ERRORMSG="Unknown zk option !ZKOP!"goto zkshort_usage ) gotodone

红色即为zk cp命令时执行 

org.apache.solr.util.SolrCLI类

 不同的命令,使用不同的Tool类

 //Creates an instance of the requested tool, using classpath scanning if necessary private static Tool newTool(String toolType) throwsException {if ("healthcheck".equals(toolType))return newHealthcheckTool();else if ("status".equals(toolType))return newStatusTool();else if ("api".equals(toolType))return newApiTool();else if ("createcollection".equals(toolType))return newCreateCollectionTool();else if ("createcore".equals(toolType))return newCreateCoreTool();else if ("create".equals(toolType))return newCreateTool();else if ("delete".equals(toolType))return newDeleteTool();else if ("config".equals(toolType))return newConfigTool();else if ("run_example".equals(toolType))return newRunExampleTool();else if ("upconfig".equals(toolType))return newConfigSetUploadTool();else if ("downconfig".equals(toolType))return newConfigSetDownloadTool();else if ("rm".equals(toolType))return newZkRmTool();else if ("mv".equals(toolType))return newZkMvTool();else if ("cp".equals(toolType))return newZkCpTool();else if ("ls".equals(toolType))return newZkLsTool();else if ("mkroot".equals(toolType))return newZkMkrootTool();else if ("assert".equals(toolType))return newAssertTool();else if ("utils".equals(toolType))return newUtilsTool();else if ("auth".equals(toolType))return new AuthTool();

cp执行ZkCpTool的runImpl方法

protected void runImpl(CommandLine cli) throwsException { raiseLogLevelUnlessVerbose(cli); String zkHost=getZkHost(cli);if (zkHost == null) {throw new IllegalStateException("Solr at " + cli.getOptionValue("solrUrl") + " is running in standalone server mode, cp can only be used when running in SolrCloud mode.n"); }try (SolrZkClient zkClient = new SolrZkClient(zkHost, 30000)) { echoIfVerbose("nConnecting to ZooKeeper at " + zkHost + " ...", cli); String src= cli.getOptionValue("src"); String dst= cli.getOptionValue("dst"); Boolean recurse= Boolean.parseBoolean(cli.getOptionValue("recurse")); echo("Copying from '" + src + "' to '" + dst + "'. ZooKeeper at " +zkHost);boolean srcIsZk = src.toLowerCase(Locale.ROOT).startsWith("zk:");boolean dstIsZk = dst.toLowerCase(Locale.ROOT).startsWith("zk:"); String srcName=src;if(srcIsZk) { srcName= src.substring(3); }else if (srcName.toLowerCase(Locale.ROOT).startsWith("file:")) { srcName= srcName.substring(5); } String dstName=dst;if(dstIsZk) { dstName= dst.substring(3); }else{if (dstName.toLowerCase(Locale.ROOT).startsWith("file:")) { dstName= dstName.substring(5); } } zkClient.zkTransfer(srcName, srcIsZk, dstName, dstIsZk, recurse); }catch(Exception e) { log.error("Could not complete the zk operation for reason: " +e.getMessage());throw(e); } }

调用SolrZkClient的zkTransfer

public voidzkTransfer(String src, Boolean srcIsZk, String dst, Boolean dstIsZk, Boolean recurse)throwsSolrServerException, KeeperException, InterruptedException, IOException { ZkMaintenanceUtils.zkTransfer(this, src, srcIsZk, dst, dstIsZk, recurse); }

实现由ZkMaintenanceUtils来做

//// Copy between local file system and Zookeeper, or from one Zookeeper node to another, / optionally copying recursively. / /@paramsrc Source to copy from. Both src and dst may be Znodes. However, both may NOT be local /@paramdst The place to copy the files too. Both src and dst may be Znodes. However both may NOT be local /@paramrecurse if the source is a directory, reccursively copy the contents iff this is true. /@throwsSolrServerException Explanatory exception due to bad params, failed operation, etc. /@throwsKeeperException Could not perform the Zookeeper operation. /@throwsInterruptedException Thread interrupted// public static voidzkTransfer(SolrZkClient zkClient, String src, Boolean srcIsZk, String dst, Boolean dstIsZk, Boolean recurse)throwsSolrServerException, KeeperException, InterruptedException, IOException {if (srcIsZk == false && dstIsZk == false) {throw new SolrServerException("One or both of source or destination must specify ZK nodes."); }//Make sure -recurse is specified if the source has children. if (recurse == false) {if(srcIsZk) {if (zkClient.getChildren(src, null, true).size() != 0) {throw new SolrServerException("Zookeeper node " + src + " has children and recurse is false"); } }else if(Files.isDirectory(Paths.get(src))) {throw new SolrServerException("Local path " + Paths.get(src).toAbsolutePath() + " is a directory and recurse is false"); } }if (dstIsZk && dst.length() == 0) { dst= "/"; //for consistency, one can copy from zk: and send to zk:/ } dst=normalizeDest(src, dst, srcIsZk, dstIsZk);//ZK -> ZK copy. if (srcIsZk &&dstIsZk) { traverseZkTree(zkClient, src, VISITORDER.VISITPRE,newZkCopier(zkClient, src, dst));return; }//local -> ZK copy if(dstIsZk) { uploadToZK(zkClient, Paths.get(src), dst,null);return; }//Copying individual files from ZK requires special handling since downloadFromZK assumes the node has children.//This is kind of a weak test for the notion of "directory" on Zookeeper.//ZK -> local copy where ZK is a parent node if (zkClient.getChildren(src, null, true).size() > 0) { downloadFromZK(zkClient, src, Paths.get(dst));return; }//Single file ZK -> local copy where ZK is a leaf node if(Files.isDirectory(Paths.get(dst))) {if (dst.endsWith(File.separator) == false) dst +=File.separator; dst=normalizeDest(src, dst, srcIsZk, dstIsZk); }byte[] data = zkClient.getData(src, null, null, true); Path filename=Paths.get(dst); Files.createDirectories(filename.getParent()); log.info("Writing file {}", filename); Files.write(filename, data); }

注意:这个copy 可以是本地文件到znode之间的copy,也可以是znode之间的copy

本文以本地文档到znode之间的copypublic static void uploadToZK(SolrZkClient zkClient, final Path fromPath, finalString zkPath,final Pattern filenameExclusions) throwsIOException { String path=fromPath.toString();if (path.endsWith("/*")) { path= path.substring(0, path.length() - 1); }final Path rootPath =Paths.get(path);if (!Files.exists(rootPath))throw new IOException("Path " + rootPath + " does not exist"); Files.walkFileTree(rootPath,new SimpleFileVisitor() { @Overridepublic FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throwsIOException { String filename=file.getFileName().toString();if (filenameExclusions != null &&filenameExclusions.matcher(filename).matches()) { log.info("uploadToZK skipping '{}' due to filenameExclusions '{}'", filename, filenameExclusions);returnFileVisitResult.CONTINUE; } String zkNode=createZkNodeName(zkPath, rootPath, file);try{//if the path exists (and presumably we're uploading data to it) just set its data if (file.toFile().getName().equals(ZKNODE_DATA_FILE) && zkClient.exists(zkNode, true)) { zkClient.setData(zkNode, file.toFile(),true); }else{ zkClient.makePath(zkNode, file.toFile(),false, true); } }catch (KeeperException |InterruptedException e) {throw new IOException("Error uploading file " + file.toString() + " to zookeeper path " +zkNode, SolrZkClient.checkInterrupted(e)); }returnFileVisitResult.CONTINUE; } @Overridepublic FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throwsIOException {if (dir.getFileName().toString().startsWith(".")) returnFileVisitResult.SKIP_SUBTREE;returnFileVisitResult.CONTINUE; } }); }

文件转换为byte流数据,设置到znode

//// Write file to ZooKeeper - default system encoding used. / /@parampath path to upload file to e.g. /solr/conf/solrconfig.xml /@paramfile path to file to be uploaded// public Stat setData(String path, File file, boolean retryOnConnLoss) throwsIOException, KeeperException, InterruptedException { log.debug("Write to ZooKeeper: {} to {}", file.getAbsolutePath(), path);byte[] data =FileUtils.readFileToByteArray(file);returnsetData(path, data, retryOnConnLoss);//// Returns node's state/*/ public Stat setData(final String path, final byte data[], final int version, booleanretryOnConnLoss)throwsKeeperException, InterruptedException {if(retryOnConnLoss) {return zkCmdExecutor.retryOperation(() ->keeper.setData(path, data, version)); }else{returnkeeper.setData(path, data, version); } }

具体实现为ZooKeeper本身的客户端ZooKeeper.java

//// Set the data for the node of the given path if such a node exists and the / given version matches the version of the node (if the given version is / -1, it matches any node's versions). Return the stat of the node. /

/ This operation, if successful, will trigger all the watches on the node / of the given path left by getData calls. /

/ A KeeperException with error code KeeperException.NoNode will be thrown / if no node with the given path exists. /

/ A KeeperException with error code KeeperException.BadVersion will be / thrown if the given version does not match the node's version. /

/ The maximum allowable size of the data array is 1 MB (1,048,576 bytes). / Arrays larger than this will cause a KeeperException to be thrown. / /@parampath / the path of the node /@paramdata / the data to set /@paramversion / the expected matching version /@returnthe state of the node /@throwsInterruptedException If the server transaction is interrupted. /@throwsKeeperException If the server signals an error with a non-zero error code. /@throwsIllegalArgumentException if an invalid path is specified/*/ public Stat setData(final String path, byte data[], intversion)throwsKeeperException, InterruptedException {final String clientPath =path; PathUtils.validatePath(clientPath);final String serverPath =prependChroot(clientPath); RequestHeader h= newRequestHeader(); h.setType(ZooDefs.OpCode.setData); SetDataRequest request= newSetDataRequest(); request.setPath(serverPath); request.setData(data); request.setVersion(version); SetDataResponse response= newSetDataResponse(); ReplyHeader r= cnxn.submitRequest(h, request, response, null);if (r.getErr() != 0) {throwKeeperException.create(KeeperException.Code.get(r.getErr()), clientPath); }returnresponse.getStat(); }

  1. 小结

   1.solrcloud使用zookeeper实现了配置中心,入口函数为ZkCLI.java ,它使用命令模式内部封装了一系列命令,如:healthcheck,status,api,config,upconfig,downconfig,mv,cp,ls,mkroot等命令

   2.最终调用的是apache zookeeper本身的api,如果为文件的话,需要先转为byte流,然后存入znode节点中。

参考文献:

【1】https://www.cnblogs.com/xiaoqi/p/configserver-compair.html

【2】https://lucene.apache.org/solr/guide/7_6/solr-tutorial.html

      

  

版权声明
本文为[一天不进步,就是退步]所创,转载请带上原文链接,感谢
https://www.cnblogs.com/davidwang456/p/10431431.html

编程之旅,人生之路,不止于编程,还有诗和远方。
阅代码原理,看框架知识,学企业实践;
赏诗词,读日记,踏人生之路,观世界之行;

支付宝红包,每日可领