在OpenShift中添加Cassandra节点到现有的集群

我在CentOS 6上安装了3个cassandra节点(S1,S2,S3)。它们中的两个(S1,S2)位于数据中心A中,其中一个(S3)位于openshift中。 我正在尝试使用多个数据中心中的多个节点群集进行基准testing。

Cassandra节点S1和S2上的安装和configuration基于在基于RHEL的系统上安装DataStax社区另一方面,节点S3(openshift)上的cassandra安装基于如何在OpenShift上configuration和运行Cassandra 。

所有的服务器运行良好,但我不能添加节点S3到群集。 所有服务器都有完全相同的configuration文件(cassandra.yaml和cassandra-topology.properties)

节点S1和S2具有cassandra的1.2.18版本。

[root@node1 ~]# cassandra -v 1.2.18 

节点S3有cassandra的1.1.6版本。

 [cassandra-xxxx.rhcloud.com xxxxxxxx]\> app-root/data/cassandra/bin/cassandra -v xss = -ea -javaagent:app-root/data/cassandra/bin/../lib/jamm-0.2.5.jar -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms1834M -Xmx1834M -Xmn256M -XX:+HeapDumpOnOutOfMemoryError -Xss180k 1.1.6 

所有节点具有相同的cassandra.yaml和cassandra-topology.propertiesconfiguration,如下所示(除了监听ip地址)。

cassandra.yaml

 cluster_name: 'MyDemoCluster' num_tokens: 256 seed_provider: - class_name: org.apache.cassandra.locator.SimpleSeedProvider parameters: - seeds: "xxx236" listen_address: xxxx endpoint_snitch: PropertyFileSnitch 

cassandra-topology.properties

 # Cassandra Node IP=Data Center:Rack xxx236=XXX:RAC1 xxx237=XXX:RAC2 yyyyyy=OPS:RAC1 

首先,我开始种子节点,而其他节点停止。 成功启动后,我启动了其他节点(日志文件中没有任何错误消息)。

然后,当我在种子节点上运行节点工具,我只看到节点S1和S2,而不是节点S3。

 [root@node1]# nodetool status Datacenter: XXX =============== Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns (effective) Host ID Rack UN xxx236 79.25 KB 256 100.0% xxxxxxxx-xxxx-404a-aa3f-fbddfb5da124 RAC2 UN xxx237 79.56 KB 256 100.0% xxxxxxxx-xxxx-4b27-970b-76944c382c9f RAC1 

节点S2的cassandra日志

  INFO [HANDSHAKE-/xxx236] 2014-08-08 11:25:26,120 OutboundTcpConnection.java (line 418) Handshaking version with /xxx 

在节点S3上运行cassandra

  [cassandra-biberltd.rhcloud.com data]\> cassandra/bin/cassandra -f xss = -ea -javaagent:cassandra/bin/../lib/jamm-0.2.5.jar -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms1834M -Xmx1834M -Xmn256M -XX:+HeapDumpOnOutOfMemoryError -Xss180k INFO 11:36:31,296 Logging initialized INFO 11:36:31,305 JVM vendor/version: OpenJDK Server VM/1.7.0_65 INFO 11:36:31,305 Heap size: 1896284160/1896284160 INFO 11:36:31,305 Classpath: cassandra/bin/../conf:cassandra/bin/../build/classes/main:cassandra/bin/../build/classes/thrift:cassandra/bin/../lib/antlr-3.2.jar:cassandra/bin/../lib/apache-cassandra-1.1.6.jar:cassandra/bin/../lib/apache-cassandra-clientutil-1.1.6.jar:cassandra/bin/../lib/apache-cassandra-thrift-1.1.6.jar:cassandra/bin/../lib/avro-1.4.0-fixes.jar:cassandra/bin/../lib/avro-1.4.0-sources-fixes.jar:cassandra/bin/../lib/commons-cli-1.1.jar:cassandra/bin/../lib/commons-codec-1.2.jar:cassandra/bin/../lib/commons-lang-2.4.jar:cassandra/bin/../lib/compress-lzf-0.8.4.jar:cassandra/bin/../lib/concurrentlinkedhashmap-lru-1.3.jar:cassandra/bin/../lib/guava-r08.jar:cassandra/bin/../lib/high-scale-lib-1.1.2.jar:cassandra/bin/../lib/jackson-core-asl-1.9.2.jar:cassandra/bin/../lib/jackson-mapper-asl-1.9.2.jar:cassandra/bin/../lib/jamm-0.2.5.jar:cassandra/bin/../lib/jline-0.9.94.jar:cassandra/bin/../lib/json-simple-1.1.jar:cassandra/bin/../lib/libthrift-0.7.0.jar:cassandra/bin/../lib/log4j-1.2.16.jar:cassandra/bin/../lib/metrics-core-2.0.3.jar:cassandra/bin/../lib/servlet-api-2.5-20081211.jar:cassandra/bin/../lib/slf4j-api-1.6.1.jar:cassandra/bin/../lib/slf4j-log4j12-1.6.1.jar:cassandra/bin/../lib/snakeyaml-1.6.jar:cassandra/bin/../lib/snappy-java-1.0.4.1.jar:cassandra/bin/../lib/snaptree-0.1.jar:cassandra/bin/../lib/jamm-0.2.5.jar INFO 11:36:31,308 JNA not found. Native methods will be disabled. INFO 11:36:31,329 Loading settings from file:/var/lib/openshift/53e4cf805004462c540000a9/app-root/data/cassandra/conf/cassandra.yaml INFO 11:36:31,649 32bit JVM detected. It is recommended to run Cassandra on a 64bit JVM for better performance. INFO 11:36:31,650 DiskAccessMode 'auto' determined to be standard, indexAccessMode is standard INFO 11:36:32,322 Global memtable threshold is enabled at 602MB INFO 11:36:33,196 Initializing key cache with capacity of 90 MBs. INFO 11:36:33,218 Scheduling key cache save to each 14400 seconds (going to save all keys). INFO 11:36:33,220 Initializing row cache with capacity of 0 MBs and provider org.apache.cassandra.cache.SerializingCacheProvider INFO 11:36:33,228 Scheduling row cache save to each 0 seconds (going to save all keys). INFO 11:36:33,595 Couldn't detect any schema definitions in local storage. INFO 11:36:33,596 Found table data in data directories. Consider using the CLI to define your schema. INFO 11:36:33,664 No commitlog files found; skipping replay INFO 11:36:33,730 Cassandra version: 1.1.6 INFO 11:36:33,730 Thrift API version: 19.32.0 INFO 11:36:33,735 CQL supported versions: 2.0.0,3.0.0-beta1 (default: 2.0.0) INFO 11:36:33,835 Loading persisted ring state INFO 11:36:33,839 Starting up server gossip INFO 11:36:33,862 Enqueuing flush of Memtable-LocationInfo@8039520(135/168 serialized/live bytes, 3 ops) INFO 11:36:33,863 Writing Memtable-LocationInfo@8039520(135/168 serialized/live bytes, 3 ops) INFO 11:36:33,945 Completed flushing $HOME/app-root/data/cassandra-data/data/system/LocationInfo/system-LocationInfo-hf-1-Data.db (243 bytes) for commitlog position ReplayPosition(segmentId=1407512193559, position=595) INFO 11:36:33,982 Starting Messaging Service on port 17000 INFO 11:36:33,995 JOINING: waiting for ring information INFO 11:37:04,035 JOINING: schema complete, ready to bootstrap INFO 11:37:04,036 JOINING: getting bootstrap token INFO 11:37:04,045 Enqueuing flush of Memtable-LocationInfo@21839213(53/66 serialized/live bytes, 2 ops) INFO 11:37:04,046 Writing Memtable-LocationInfo@21839213(53/66 serialized/live bytes, 2 ops) INFO 11:37:04,079 Completed flushing $HOME/app-root/data/cassandra-data/data/system/LocationInfo/system-LocationInfo-hf-2-Data.db (163 bytes) for commitlog position ReplayPosition(segmentId=1407512193559, position=776) INFO 11:37:04,082 JOINING: sleeping 30000 ms for pending range setup INFO 11:37:34,083 JOINING: Starting to bootstrap... INFO 11:37:34,103 Enqueuing flush of Memtable-LocationInfo@21895329(53/66 serialized/live bytes, 2 ops) INFO 11:37:34,103 Writing Memtable-LocationInfo@21895329(53/66 serialized/live bytes, 2 ops) INFO 11:37:34,140 Completed flushing $HOME/app-root/data/cassandra-data/data/system/LocationInfo/system-LocationInfo-hf-3-Data.db (163 bytes) for commitlog position ReplayPosition(segmentId=1407512193559, position=957) INFO 11:37:34,154 Node /yyyyyy state jump to normal INFO 11:37:34,155 Bootstrap/Replace/Move completed! Now serving reads. INFO 11:37:34,221 Binding thrift service to /yyyyyy:19160 INFO 11:37:34,226 Using TFastFramedTransport with a max frame size of 15728640 bytes. INFO 11:37:34,232 Using synchronous/threadpool thrift server on /yyyyyy : 19160 INFO 11:37:34,233 Listening for thrift clients... 

我只find了

  INFO 11:36:31,308 JNA not found. Native methods will be disabled. 

我的问题是

  1. 我怎样才能将openshift节点添加到群集?
  2. openshift是否支持多个数据中心集群成员的节点?
  3. 所有的节点必须有相同版本的cassandra?