Home | 简体中文 | 繁体中文 | 杂文 | 打赏(Donations) | ITEYE 博客 | OSChina 博客 | Facebook | Linkedin | 知乎专栏 | Search | Email

第 49 章 Hypertable

目录

49.1. Hypertable 安装
49.1.1. Hypertable standalone 单机安装
49.1.2. Hypertable on HDFS(hadoop) 安装
49.1.3. MapR
49.1.4. Ceph
49.1.5. 检验安装
49.2. Code examples
49.2.1. PHP
49.3. HQL
49.3.1. namespace 命名空间管理
49.3.2. Table 表
49.4. FAQ
49.4.1. 切换 DFS Broker

http://hypertable.org/

49.1. Hypertable 安装

49.1.1. Hypertable standalone 单机安装

过程 49.1. Hypertable standalone 安装过程

  1. 安装 Hypertable 软件包

    # cd /usr/local/src/
    # wget http://cdn.hypertable.com/packages/0.9.7.0/hypertable-0.9.7.0-linux-x86_64.rpm
    					
  2. 安装 Hypertable, 我个人比较喜欢用yum localinstall他会解决软件之间的依赖关系

    # yum localinstall hypertable-0.9.7.0-linux-x86_64.rpm
    					

    相关的软件会自动安装

    Dependencies Resolved
    
    ====================================================================================================================
     Package                       Arch          Version                  Repository                               Size
    ====================================================================================================================
    Installing:
     hypertable                    x86_64        0.9.7.0-1                /hypertable-0.9.7.0-linux-x86_64        288 M
    Installing for dependencies:
     mailcap                       noarch        2.1.31-2.el6             base                                     27 k
     perl-Bit-Vector               x86_64        7.1-2.el6                base                                    169 k
     perl-Carp-Clan                noarch        6.03-2.el6               base                                     25 k
     perl-Compress-Raw-Zlib        x86_64        1:2.020-127.el6          base                                     68 k
     perl-Compress-Zlib            x86_64        2.020-127.el6            base                                     43 k
     perl-HTML-Parser              x86_64        3.64-2.el6               base                                    109 k
     perl-HTML-Tagset              noarch        3.20-4.el6               base                                     17 k
     perl-IO-Compress-Base         x86_64        2.020-127.el6            base                                     67 k
     perl-IO-Compress-Zlib         x86_64        2.020-127.el6            base                                    134 k
     perl-IO-String                noarch        1.08-9.el6               base                                     15 k
     perl-URI                      noarch        1.40-2.el6               base                                    117 k
     perl-libwww-perl              noarch        5.833-2.el6              base                                    387 k
    
    Transaction Summary
    ====================================================================================================================
    Install      13 Package(s)
    					
  3. Hypertable 默认安装在 /opt/hypertable/0.9.7.0

    备份配置文件,

    # cd /opt/hypertable/0.9.7.0/conf
    # cp hypertable.cfg hypertable.cfg.original
    					
  4. FHS-IZE 安装

    # bin/fhsize.sh
    Setting up /var/opt/hypertable
    Setting up /etc/opt/hypertable
    fshize /opt/hypertable/0.9.7.0:  success
    					
  5. 设计 "CURRENT" 连接

    # cd /opt/hypertable
    # ln -s 0.9.7.0 current
    					
  6. 安装 notification-hook.sh 脚本.

    # cp conf/notification-hook.tmpl conf/notification-hook.sh
    # chmod o+x conf/notification-hook.sh
    					

    测试 notification-hook.sh脚本 .

    /opt/hypertable/current/conf/notification-hook.sh "Test Message" "This is a test."
    					
  7. 启动 hypertable

    # /opt/hypertable/current/bin/start-all-servers.sh local
    DFS broker: available file descriptors: 1024
    Started DFS Broker (local)
    Started Hyperspace
    Started Hypertable.Master
    /proc/sys/vm/swappiness = 60
    Started Hypertable.RangeServer
    Started ThriftBroker
    					
    					
    # /opt/hypertable/current/bin/ht shell
    
    Welcome to the hypertable command interpreter.
    For information about Hypertable, visit http://hypertable.com
    
    Type 'help' for a list of commands, or 'help shell' for a
    list of shell meta commands.
    
    hypertable>
    					
    					
  8. 测试安装是否有效

    					
    # /opt/hypertable/current/bin/ht shell
    
    Welcome to the hypertable command interpreter.
    For information about Hypertable, visit http://hypertable.com
    
    Type 'help' for a list of commands, or 'help shell' for a
    list of shell meta commands.
    
    hypertable> help
    
    USE ................ Sets the current namespace
    CREATE NAMESPACE ... Creates a new namespace
    DROP NAMESPACE ..... Removes a namespace
    EXISTS TABLE ....... Check if table exists
    CREATE TABLE ....... Creates a table
    DELETE ............. Deletes all or part of a row from a table
    DESCRIBE TABLE ..... Displays a table's schema
    DROP TABLE ......... Removes a table
    RENAME TABLE ....... Renames a table
    DUMP TABLE ......... Create efficient backup file
    ALTER TABLE ........ Add/remove column family from existing table
    INSERT ............. Inserts data into a table
    LOAD DATA INFILE ... Loads data from a TSV input file into a table
    SELECT ............. Selects (and display) cells from a table
    SHOW CREATE TABLE .. Displays CREATE TABLE command used to create table
    SHOW TABLES ........ Displays only the list of tables in the current namespace
    GET LISTING ........ Displays the list of tables and namespace in the current namespace
    
    Statements must be terminated with ';'.  For more information on
    a specific statement, type 'help <statement>', where <statement> is from
    the preceding list.
    
    hypertable>quit
    					
    					
  9. 停止 hypertable

    运行下列命令停止 Hypertable

    $ /opt/hypertable/current/bin/stop-servers.sh
    					

49.1.2. Hypertable on HDFS(hadoop) 安装

Hadoop - HDFS 安装指南

过程 49.2. Hypertable on HDFS

  1. 创建工作目录

    $ hadoop fs -mkdir /hypertable
    $ hadoop fs -chmod 777 /hypertable
    					
  2. 安装 Java 运行环境

    yum install java-1.7.0-openjdk
    yum localinstall http://ftp.cuhk.edu.hk/pub/packages/apache.org/hadoop/common/hadoop-1.1.2/hadoop-1.1.2-1.x86_64.rpm
    					
  3. 修改 jrun bug

    cp /opt/hypertable/current/bin/jrun /opt/hypertable/current/bin/jrun.old
    
    vim /opt/hypertable/current/bin/jrun
    #HT_JAR=`ls -1 /opt/hypertable/doug/current/lib/java/*.jar | grep "hypertable-[^-]*.jar" | awk 'BEGIN {FS="/"} {print $NF}'`
    HT_JAR=`ls -1 /opt/hypertable/current/lib/java/*.jar | grep "hypertable-[^-]*.jar" | awk 'BEGIN {FS="/"} {print $NF}'`
    					
     export JAVA_HOME=/usr
     export HADOOP_HOME=/usr
     export HYPERTABLE_HOME=/opt/hypertable/current
    					
  4. hypertable.cfg

    # cat conf/hypertable.cfg
    #
    # hypertable.cfg
    #
    
    # HDFS Broker
    #HdfsBroker.Hadoop.ConfDir=/etc/hadoop/conf
    HdfsBroker.Hadoop.ConfDir=/etc/hadoop
    
    # Ceph Broker
    CephBroker.MonAddr=192.168.6.2:6789
    
    # Local Broker
    DfsBroker.Local.Root=fs/local
    
    # DFS Broker - for clients
    DfsBroker.Port=38030
    
    # Hyperspace
    Hyperspace.Replica.Host=localhost
    Hyperspace.Replica.Port=38040
    Hyperspace.Replica.Dir=hyperspace
    
    # Hypertable.Master
    #Hypertable.Master.Host=localhost
    Hypertable.Master.Port=38050
    
    # Hypertable.RangeServer
    Hypertable.RangeServer.Port=38060
    
    Hyperspace.KeepAlive.Interval=30000
    Hyperspace.Lease.Interval=1000000
    Hyperspace.GracePeriod=200000
    
    # ThriftBroker
    ThriftBroker.Port=38080
    					

    Hadoop 配置文件 /etc/hadoop/core-site.xml

    					
    # cat /etc/hadoop/core-site.xml
    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    
    <!-- Put site-specific property overrides in this file. -->
    
    <configuration>
        <property>
             <name>fs.default.name</name>
             <value>hdfs://namenode.example.com:9000</value>
        </property>
        <property>
             <name>hadoop.tmp.dir</name>
             <value>/var/tmp/hadoop</value>
        </property>
    </configuration>
    					
    					

    Hadoop 配置文件 /etc/hadoop/hdfs-site.xml

    					
    # cat /etc/hadoop/hdfs-site.xml
    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    
    <!-- Put site-specific property overrides in this file. -->
    
    <configuration>
        <property>
            <name>dfs.name.dir</name>
            <value>/var/hadoop/name1</value>
            <description>  </description>
        </property>
        <property>
            <name>dfs.data.dir</name>
            <value>/var/hadoop/hdfs/data1</value>
            <description> </description>
        </property>
        <property>
            <name>dfs.replication</name>
            <value>2</value>
        </property>
    </configuration>
    					
    					
  5. 启动 dfsbroker

    # /opt/hypertable/current/bin/set-hadoop-distro.sh cdh4
    Hypertable successfully configured for Hadoop cdh4
    					
    # /opt/hypertable/current/bin/start-dfsbroker.sh hadoop
    DFS broker: available file descriptors: 1024
    Started DFS Broker (hadoop)
    					

    查看启动日志

    # tail -f /opt/hypertable/current/log/DfsBroker.hadoop.log
    log4j:WARN No appenders could be found for logger (org.apache.hadoop.conf.Configuration).
    log4j:WARN Please initialize the log4j system properly.
    HdfsBroker.dfs.client.read.shortcircuit=false
    HdfsBroker.dfs.replication=2
    HdfsBroker.Server.fs.default.name=hdfs://namenode.example.com:9000
    Apr 23, 2013 6:43:18 PM org.hypertable.AsyncComm.IOHandler DeliverEvent
    INFO: [/192.168.6.25:53556 ; Tue Apr 23 18:43:18 HKT 2013] Connection Established
    Apr 23, 2013 6:43:18 PM org.hypertable.DfsBroker.hadoop.ConnectionHandler handle
    INFO: [/192.168.6.25:53556 ; Tue Apr 23 18:43:18 HKT 2013] Disconnect - COMM broken connection : Closing all open handles from /192.168.6.25:53556
    Closed 0 input streams and 0 output streams for client connection /192.168.6.25:53556
    					

49.1.3. MapR

49.1.4. Ceph

修改 CephBroker.MonAddr 对应的服务器与端口号即可

# cat hypertable.cfg
#
# hypertable.cfg
#

# HDFS Broker
HdfsBroker.Hadoop.ConfDir=/etc/hadoop/conf

# Ceph Broker
CephBroker.MonAddr=192.168.6.2:6789

# Local Broker
DfsBroker.Local.Root=fs/local

# DFS Broker - for clients
DfsBroker.Port=38030

# Hyperspace
Hyperspace.Replica.Host=localhost
Hyperspace.Replica.Port=38040
Hyperspace.Replica.Dir=hyperspace

# Hypertable.Master
Hypertable.Master.Port=38050

# Hypertable.RangeServer
Hypertable.RangeServer.Port=38060

Hyperspace.KeepAlive.Interval=30000
Hyperspace.Lease.Interval=1000000
Hyperspace.GracePeriod=200000

# ThriftBroker
ThriftBroker.Port=38080
			

启动 dfsbroker

# /opt/hypertable/current/bin/start-dfsbroker.sh ceph
			

49.1.5. 检验安装

创建一个表

			
# echo "USE '/'; CREATE TABLE foo ( c1, c2 ); GET LISTING;" \
>     | /opt/hypertable/current/bin/ht shell --batch
foo
sys	(namespace)
tmp	(namespace)
			
			

插入一些数据

			
# echo "USE '/'; INSERT INTO foo VALUES('001', 'c1', 'very'), \
>     ('000', 'c1', 'Hypertable'), ('001', 'c2', 'easy'), ('000', 'c2', 'is');" \
>     | /opt/hypertable/current/bin/ht shell --batch
			
			

查询数据

			
# echo "USE '/'; SELECT * FROM foo;" \
>     | /opt/hypertable/current/bin/ht shell --batch
000	c1	Hypertable
000	c2	is
001	c1	very
001	c2	easy
			
			

如果你想清楚所有表运行下面命令

$ /opt/hypertable/current/bin/stop-servers.sh
$ /opt/hypertable/current/bin/clean-database.sh