在 Windows 上运行 Apache Storm 2.0.0

本文主要参考 [Running Apache Storm on Windows][6]（其使用的是 *0.9.1-incubating* 版）。

前置需要安装 Java 和 Python，[Storm 2.0.0][9] 需要安装 Java 8。

安装后不要忘记配置环境变量。

| 变量名 | 变量值 |
| --- | --- |
| `JAVA_HOME` | *C:\Progra~1\Java\jdk1.8.0_111* |
| `Path` （追加） | *C:\Python37\Scripts\;C:\Python37\;%JAVA_HOME%\bin;* |
| `PATHEXT` （追加） | *.PY;* |

安装后可使用命令 `java -version` 和 `py -V` 确认安装的版本。

```bash
C:\Users\liujiajia>java -version
java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)
```

```bash
C:\Users\liujiajia>py -V
Python 3.7.3
```

## 安装 Zookeeper

下载 [zookeeper-3.4.14.tar.gz][8] 解压到 *D:\zookeeper\zookeeper-3.4.14* 目录后执行如下命令。

```bash
D:
cd D:\zookeeper\zookeeper-3.4.14
copy conf\zoo_sample.cfg conf\zoo.cfg
bin\zkServer.cmd
```

执行成功的窗口打印日志：

```java
D:\zookeeper\zookeeper-3.4.14>call "C:\Progra~1\Java\jdk1.8.0_111"\bin\java "-Dz
ookeeper.log.dir=D:\zookeeper\zookeeper-3.4.14\bin\.." "-Dzookeeper.root.logger=
INFO,CONSOLE" -cp "D:\zookeeper\zookeeper-3.4.14\bin\..\build\classes;D:\zookeep
er\zookeeper-3.4.14\bin\..\build\lib\*;D:\zookeeper\zookeeper-3.4.14\bin\..\*;D:
\zookeeper\zookeeper-3.4.14\bin\..\lib\*;D:\zookeeper\zookeeper-3.4.14\bin\..\co
nf" org.apache.zookeeper.server.quorum.QuorumPeerMain "D:\zookeeper\zookeeper-3.
4.14\bin\..\conf\zoo.cfg"
2019-09-05 14:13:04,838 [myid:] - INFO  [main:QuorumPeerConfig@136] - Reading co
nfiguration from: D:\zookeeper\zookeeper-3.4.14\bin\..\conf\zoo.cfg
2019-09-05 14:13:04,856 [myid:] - INFO  [main:DatadirCleanupManager@78] - autopu
rge.snapRetainCount set to 3
2019-09-05 14:13:04,857 [myid:] - INFO  [main:DatadirCleanupManager@79] - autopu
rge.purgeInterval set to 0
2019-09-05 14:13:04,857 [myid:] - INFO  [main:DatadirCleanupManager@101] - Purge
 task is not scheduled.
2019-09-05 14:13:04,860 [myid:] - WARN  [main:QuorumPeerMain@116] - Either no co
nfig or no quorum defined in config, running  in standalone mode
2019-09-05 14:13:04,935 [myid:] - INFO  [main:QuorumPeerConfig@136] - Reading co
nfiguration from: D:\zookeeper\zookeeper-3.4.14\bin\..\conf\zoo.cfg
2019-09-05 14:13:04,936 [myid:] - INFO  [main:ZooKeeperServerMain@98] - Starting
 server
2019-09-05 14:13:04,951 [myid:] - INFO  [main:Environment@100] - Server environm
ent:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on
03/06/2019 16:18 GMT
2019-09-05 14:13:04,951 [myid:] - INFO  [main:Environment@100] - Server environm
ent:host.name=EDG2GYKVF8G5P6Z.octopus.888ly.cn
2019-09-05 14:13:04,952 [myid:] - INFO  [main:Environment@100] - Server environm
ent:java.version=1.8.0_111
2019-09-05 14:13:04,952 [myid:] - INFO  [main:Environment@100] - Server environm
ent:java.vendor=Oracle Corporation
2019-09-05 14:13:04,953 [myid:] - INFO  [main:Environment@100] - Server environm
ent:java.home=C:\Progra~1\Java\jdk1.8.0_111\jre
2019-09-05 14:13:04,953 [myid:] - INFO  [main:Environment@100] - Server environm
ent:java.class.path=D:\zookeeper\zookeeper-3.4.14\bin\..\build\classes;D:\zookee
per\zookeeper-3.4.14\bin\..\build\lib\*;D:\zookeeper\zookeeper-3.4.14\bin\..\zoo
keeper-3.4.14.jar;D:\zookeeper\zookeeper-3.4.14\bin\..\lib\audience-annotations-
0.5.0.jar;D:\zookeeper\zookeeper-3.4.14\bin\..\lib\jline-0.9.94.jar;D:\zookeeper
\zookeeper-3.4.14\bin\..\lib\log4j-1.2.17.jar;D:\zookeeper\zookeeper-3.4.14\bin\
..\lib\netty-3.10.6.Final.jar;D:\zookeeper\zookeeper-3.4.14\bin\..\lib\slf4j-api
-1.7.25.jar;D:\zookeeper\zookeeper-3.4.14\bin\..\lib\slf4j-log4j12-1.7.25.jar;D:
\zookeeper\zookeeper-3.4.14\bin\..\conf
2019-09-05 14:13:04,953 [myid:] - INFO  [main:Environment@100] - Server environm
ent:java.library.path=C:\Progra~1\Java\jdk1.8.0_111\bin;C:\Windows\Sun\Java\bin;
C:\Windows\system32;C:\Windows;C:\ProgramData\Oracle\Java\javapath;C:\Windows\sy
stem32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell
\v1.0\;C:\Windows\system32\config\systemprofile\.dnx\bin;C:\Program Files\Micros
oft DNX\Dnvm\;C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;C:\Program F
iles\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files\Git\cmd;C:\Program Fi
les\dotnet\;C:\Program Files\TortoiseSVN\bin;C:\Program Files\Microsoft SQL Serv
er\120\DTS\Binn\;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\110\Tools
\Binn\;C:\Program Files (x86)\Microsoft SQL Server\120\Tools\Binn\;C:\Program Fi
les (x86)\Microsoft SQL Server\120\Tools\Binn\ManagementStudio\;C:\Program Files
 (x86)\Microsoft SQL Server\120\DTS\Binn\;C:\Program Files\Microsoft SQL Server\
Client SDK\ODBC\170\Tools\Binn\;C:\Program Files\PuTTY\;C:\PROGRA~2\Groovy\GROOV
Y~1.0\bin;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\Window
sPowerShell\v1.0\;C:\Program Files\Redis\;D:\hadoop-3.2.0\bin;D:\hadoop-3.2.0\sb
in;C:\Users\liujiajia\AppData\Local\Programs\Python\Python37\Scripts\;C:\Users\l
iujiajia\AppData\Local\Programs\Python\Python37\;C:\Users\liujiajia\AppData\Loca
l\Programs\Microsoft VS Code\bin;C:\Users\liujiajia\.dotnet\tools;;C:\Program Fi
les\Docker Toolbox;.
2019-09-05 14:13:04,954 [myid:] - INFO  [main:Environment@100] - Server environm
ent:java.io.tmpdir=C:\Users\LIUJIA~1\AppData\Local\Temp\
2019-09-05 14:13:04,955 [myid:] - INFO  [main:Environment@100] - Server environm
ent:java.compiler=<NA>
2019-09-05 14:13:04,956 [myid:] - INFO  [main:Environment@100] - Server environm
ent:os.name=Windows 7
2019-09-05 14:13:04,956 [myid:] - INFO  [main:Environment@100] - Server environm
ent:os.arch=amd64
2019-09-05 14:13:04,957 [myid:] - INFO  [main:Environment@100] - Server environm
ent:os.version=6.1
2019-09-05 14:13:04,957 [myid:] - INFO  [main:Environment@100] - Server environm
ent:user.name=liujiajia
2019-09-05 14:13:04,957 [myid:] - INFO  [main:Environment@100] - Server environm
ent:user.home=C:\Users\liujiajia
2019-09-05 14:13:04,958 [myid:] - INFO  [main:Environment@100] - Server environm
ent:user.dir=D:\zookeeper\zookeeper-3.4.14
2019-09-05 14:13:04,988 [myid:] - INFO  [main:ZooKeeperServer@836] - tickTime se
t to 2000
2019-09-05 14:13:04,988 [myid:] - INFO  [main:ZooKeeperServer@845] - minSessionT
imeout set to -1
2019-09-05 14:13:04,989 [myid:] - INFO  [main:ZooKeeperServer@854] - maxSessionT
imeout set to -1
2019-09-05 14:13:05,200 [myid:] - INFO  [main:ServerCnxnFactory@117] - Using org
.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory
2019-09-05 14:13:05,202 [myid:] - INFO  [main:NIOServerCnxnFactory@89] - binding
 to port 0.0.0.0/0.0.0.0:2181
```

## 安装 Storm

下载 [apache-storm-2.0.0.zip][5] 解压到 *D:\storm\apache-storm-2.0.0* 目录

## 配置环境变量

| 变量名 | 变量值 |
| --- | --- |
| `STORM_HOME` | *D:\storm\apache-storm-2.0.0* |
| `Path` （追加） | *%STORM_HOME%\bin;* |

## 启动 Nimbus、Supervisor 和 Storm UI

- Nimbus

```bash
    D:
    cd %STORM_HOME%
    storm.py nimbus
    ```

- Supervisor

```bash
    D:
    cd %STORM_HOME%
    storm.py supervisor
    ```

- Storm UI

```bash
    D:
    cd %STORM_HOME%
    storm.py ui
    ```

若启动失败，可在 *logs* 目录下查看对应的日志文件。

## 访问 StormUI

默认地址：[http://localhost:8080/](http://localhost:8080/)

若 *8080* 端口被占用，可在 *conf/storm.yaml* 指定端口号。

```yaml
ui.port: 8071
```

这里指定为 *8071* 端口，然后访问 [http://localhost:8071/](http://localhost:8071/) 。

![](/wp-content/uploads/2019/09/storm-ui.png)

## 部署 Storm 自带的 examples 中的 "WordCount" Topology

代码在 *D:\storm\apache-storm-2.0.0\examples\storm-starter* 目录，需要自己打包。
打包后生成 *storm-starter-2.0.0.jar* 文件（172MB - 这么大难怪下载包里不提供）。

执行如下命令部署：

```bash
storm.py jar D:\storm\apache-storm-2.0.0\examples\storm-starter\target\storm-starter-2.0.0.jar org.apache.storm.starter.WordCountTopology WordCount -c nimbus.host=localhost
```

控制台输出：

```bash
D:\storm\apache-storm-2.0.0>storm.py jar D:\storm\apache-storm-2.0.0\examples\storm-starter\target\storm-starter-2.0.0.jar org.apache.storm.starter.WordCountTopology WordCount -c nimbus.host=localhost
Running: C:\Progra~1\Java\jdk1.8.0_111\bin\java.exe -client -Ddaemon.name= -Dstorm.options=nimbus.host%3Dlocalhost -Dstorm.home=D:\storm\apache-storm-2.0.0 -Dstorm.log.dir=D:\storm\apache-storm-2.0.0\logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64 -Dstorm.conf.file= -cp D:\storm\apache-storm-2.0.0\*;D:\storm\apache-storm-2.0.0\lib\*;D:\storm\apache-storm-2.0.0\extlib\*;D:\storm\apache-storm-2.0.0\examples\storm-starter\target\storm-starter-2.0.0.jar;D:\storm\apache-storm-2.0.0\conf;D:\storm\apache-storm-2.0.0\bin; -Dstorm.jar=D:\storm\apache-storm-2.0.0\examples\storm-starter\target\storm-starter-2.0.0.jar -Dstorm.dependency.jars= -Dstorm.dependency.artifacts={} org.apache.storm.starter.WordCountTopology WordCount
b'10:17:19.107 [main] INFO  o.a.s.StormSubmitter - Generated ZooKeeper secret payload for MD5-digest: -8881801371475247195:-5380596067515307374\r\n10:17:19.153 [main] WARN  o.a.s.v.ConfigValidation - task.heartbeat.frequency.secs is a deprecated config please see class org.apache.storm.Config.TASK_HEARTBEAT_FREQUENCY_SECS for more information.\r\n10:17:19.227 [main] INFO  o.a.s.u.NimbusClient - Found leader nimbus : EDG2GYKVF8G5P6Z.octopus.888ly.cn:6627\r\n10:17:19.228 [main] INFO  o.a.s.s.a.ClientAuthUtils - Got AutoCreds []\r\n10:17:19.256 [main] INFO  o.a.s.StormSubmitter - Uploading dependencies - jars...\r\n10:17:19.257
[main] INFO  o.a.s.StormSubmitter - Uploading dependencies - artifacts...\r\n10:17:19.257 [main] INFO  o.a.s.StormSubmitter - Dependency Blob keys - jars : [] / artifacts : []\r\n10:17:19.267 [main] INFO  o.a.s.StormSubmitter - Uploading topology jar D:\\storm\\apache-storm-2.0.0\\examples\\storm-starter\\target\\storm-starter-2.0.0.jar to assigned location: D:\\storm\\apache-storm-2.0.0\\storm-local\\nimbus\\inbox/stormjar-ebe35cf8-cbc5-4821-a95f-f604bcebe442.jar\r\n10:17:26.576 [main] INFO  o.a.s.StormSubmitter - Successfully uploaded topology jar to assigned location: D:\\storm\\apache-storm-2.0.0\\storm-local\\nimbus\\inbox/stormjar-ebe35cf8-cbc5-4821-a95f-f604bcebe442.jar\r\n10:17:26.576 [main] INFO  o.a.s.StormSubmitter - Submitting topology WordCount in distributed mode with conf {"storm.zookeeper.topology.auth.scheme":"digest","storm.zookeeper.topology.auth.payload":"-8881801371475247195:-5380596067515307374","nimbus.host":"localhost","topology.workers":3,"topology.debug":true}\r\n10:17:27.604 [main] INFO  o.a.s.StormSubmitter - Finished submitting topology: WordCount\r\n'
```

在 Storm UI 上查看 Topology summary

![](/wp-content/uploads/2019/09/storm-ui-word-count.png)

[1]: http://storm.apache.org/ (Apache Storm)
[2]: http://storm.apache.org/releases/2.0.0/index.html (Documentation Version: 2.0.0)
[3]: https://github.com/ppat/storm-rabbitmq (storm-rabbitmq)
[4]: http://storm.apache.org/downloads.html (Apache Storm downloads)
[5]: https://www.apache.org/dyn/closer.lua/storm/apache-storm-2.0.0/apache-storm-2.0.0.zip (apache-storm-2.0.0.zip)
[6]: http://ptgoetz.github.io/blog/2013/12/18/running-apache-storm-on-windows/ (Running Apache Storm on Windows)
[7]: http://zookeeper.apache.org/ (Apache ZooKeeper)
[8]: http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz (zookeeper-3.4.14.tar.gz)
[9]: http://storm.apache.org/2019/05/30/storm200-released.html (Apache Storm 2.0.0 Released)
[10]: https://stackoverflow.com/questions/28473309/storm-ui-not-working (Storm UI not working)

在 Windows 上运行 Apache Storm 2.0.0

回复内容 *

目录

AUTHOR