問題背景
再數(shù)據(jù)平臺(tái)中，項(xiàng)目搭建需要使用es和HBASE搭建數(shù)據(jù)查詢接口，整合的過程中出現(xiàn)jar包沖突的bug :com.google.common.base.Stopwatch.()V from class org.apache.hadoop.hbase.zookeeper.MetaTableLocator
|
1
2
3
4
5
6
7
8
9
10
11
|
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.hbase.zookeeper.MetaTableLocator at org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:239) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:150) ... at java.lang.Thread.run(Thread.java:745)Caused by: java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.hbase.zookeeper.MetaTableLocator at org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntil ... at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:75) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134) ... 45 more |
解決辦法
經(jīng)排查，確認(rèn)是因?yàn)?com.google.guava 這個(gè)包引起的沖突。es依賴于18及以上版本，而HBASE只支持16及以上版本。而guava這個(gè)包從17開始內(nèi)部發(fā)生變化，方法發(fā)生改變，所以18并不會(huì)向下兼容16版本。在項(xiàng)目運(yùn)行的過程中，如果同時(shí)引入16、18版本，es及hbase的調(diào)用過程會(huì)發(fā)生混亂。那么接下來就好辦了，我們可以重新打包，將guava18打入es內(nèi)部，再在pom文件中顯示引用guava16版本的包。這樣，es調(diào)用包內(nèi)部打入的guava18，而hbase調(diào)用外部的guava16。
重新打包
新建maven工程，在pom文件中進(jìn)行如下配置：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
|
<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <!-- this maven project is aim to fix the jar comflic in hbase & elasticsearch. in es, required guava 18+ or up. but in hbase you should use guava 16- or blow. cd the project directory and run 'sh ./cleanbuild.sh',you will get a new-self jar. then, adjust your project pom.xml whth: <dependency> <groupId>douguo.shaded.elasticsearch</groupId> <artifactId>douguo_shaded_elasticsearch</artifactId> <version>1.0-SNAPSHOT</version> <exclusions> <exclusion> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch</artifactId> </exclusion> </exclusions> </dependency> finally, your jar comflic will be fixed. @date:2017-11-30 @author:zhangjianfei @since:douguo.shaded.elasticsearch-1.0.0 --> <groupId>douguo.shaded.elasticsearch</groupId> <artifactId>douguo_shaded_elasticsearch</artifactId> <version>1.0-SNAPSHOT</version> <properties> <elasticsearch.version>2.4.1</elasticsearch.version> </properties> <dependencies> <dependency> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch</artifactId> <version>${elasticsearch.version}</version> </dependency> <dependency> <groupId>org.elasticsearch.plugin</groupId> <artifactId>shield</artifactId> <version>${elasticsearch.version}</version> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>2.4.1</version> <configuration> <createDependencyReducedPom>false</createDependencyReducedPom> </configuration> <executions> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> <relocations> <relocation> <pattern>com.google.guava</pattern> <shadedPattern>douguo.shaded.elasticsearch.guava</shadedPattern> </relocation> <relocation> <pattern>org.joda</pattern> <shadedPattern>douguo.shaded.elasticsearch.joda</shadedPattern> </relocation> <relocation> <pattern>com.google.common</pattern> <shadedPattern>douguo.shaded.elasticsearch.common</shadedPattern> </relocation> <relocation> <pattern>com.google.thirdparty</pattern> <shadedPattern>douguo.shaded.elasticsearch.thirdparty</shadedPattern> </relocation> </relocations> <transformers> <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer" /> </transformers> </configuration> </execution> </executions> </plugin> </plugins> </build> <repositories> <repository> <id>elasticsearch-releases</id> <url>http://maven.elasticsearch.org/releases</url> <releases> <enabled>true</enabled> <updatePolicy>daily</updatePolicy> </releases> <snapshots> <enabled>false</enabled> </snapshots> </repository> </repositories></project> |
將org.joda等4個(gè)可能有沖突的jar包通過maven-shade-plugin插件遷移后重新打個(gè)jar包從而使得在引入這個(gè)jar包時(shí)能夠使用該jar包自己的依賴而不是使用外部依賴。這里需要注意的是，需要將com.google.common等4個(gè)包全部重新遷移，否則會(huì)出現(xiàn)java.lang.IllegalAccessError: tried to access method com.google.common.base的錯(cuò)誤
項(xiàng)目打包
|
1
|
mvn clean install |
新的依賴包會(huì)在.m2 maven倉庫中，如果公司搭建了倉庫的話，需要上傳jar包。如果直接運(yùn)行jar包的話，記得重新編譯項(xiàng)目，并替換lib目錄
項(xiàng)目載入新包
只需要在pom文件中配置：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
|
<!-- douguo.shaded.elasticsearch --><dependency> <groupId>douguo.shaded.elasticsearch</groupId> <artifactId>douguo_shaded_elasticsearch</artifactId> <version>1.0-SNAPSHOT</version> <exclusions> <exclusion> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch</artifactId> </exclusion> </exclusions></dependency><!-- https://mvnrepository.com/artifact/com.google.guava/guava --><!-- this guava is only used in habse in es, 18.0+ is required, but hbase only supported 16.0 or blow. clean as install douguo.shaded.elasticsearch --><dependency> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> <version>16.0</version></dependency> |
這樣，guava18包就在douguo.shaded.elasticsearch下，es會(huì)優(yōu)先調(diào)用。而外部配置的guava16就會(huì)被HBASE調(diào)用。2個(gè)版本的jar包互相獨(dú)立存在！
到此，問題就解決了！
以上就是本文的全部內(nèi)容，希望對(duì)大家的學(xué)習(xí)有所幫助，也希望大家多多支持服務(wù)器之家。
原文鏈接：http://blog.csdn.net/qq_31573519/article/details/78679831
