Jekyll2018-08-25T14:44:11+08:00https://j.ixiaozhi.com/有爱的小止ixiaozhi's blog
ixiaozhiblog@ixiaozhi.comInstall Ghost in DigitalOcean2017-08-05T00:00:00+08:002017-08-05T00:00:00+08:00https://j.ixiaozhi.com/install-ghost-in-centos-digitalocean<h2 id="前言">前言</h2>
<p>惯例,换博客平台第一件事便是发篇搭建过程。本次使用的服务器是 DigitalOcean,CentOS,Blog 程序使用开源的 Ghost。</p>
<h2 id="下载-ghost">下载 Ghost</h2>
<p>截止至目前,最高版本为0.7.0,那就它了。
文件是一个 zip 包,因为我的 DigitalOcean 上没有安装 Unzip,所以解压前我还得安装 Unzip。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo yum install unzip
$ unzip ghost-0.7.0.zip
</code></pre></div></div>
<h2 id="下载-nodejs">下载 Node.js</h2>
<p>Ghost 0.7.0 需要 Node.js v0.10~v0.12 版本,因些我们去 nodejs.org previous releases 下载旧版本(下载地址见<code class="highlighter-rouge">参考</code>)。这里下载的是 v0.12.7 版本。之后便是解压:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ tar zxvf node-v0.12.7-linux-x64.tar.gz
</code></pre></div></div>
<p>配置环境变量:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo vi /etc/profile
</code></pre></div></div>
<p>文件末尾添加:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>PATH=$PATH:/root/ghost/node-v0.12.7-linux-x64/bin
</code></pre></div></div>
<p>生效配置并验证:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ source /etc/profile
$ echo $PATH
$ node -v
$ npm -v
</code></pre></div></div>
<p>PATH 中已经有 nodejs/bin 且 node、npm返回正确的版本号 v0.12.7 便是配置成功了。</p>
<h2 id="运行-ghost">运行 Ghost</h2>
<p>Ghost 需要依赖 sqlite 数据库,在安装时,sqlite 需要依赖 gcc,g++。所以先安装g++库:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo yum install gcc gcc-c++
</code></pre></div></div>
<p>npm install</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ npm install --production
</code></pre></div></div>
<p>修改配置</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ cd /root/ghost/ghost
$ vi config.js
</code></pre></div></div>
<p>修改 Production 节点,添加 MailServer 等配置,示例如下:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>production: {
url: 'http://www.ixiaozhi.com',
mail: {
transport: 'SMTP',
options: {
service: 'smtp.xxxx.com',
auth: {
user: 'service@ixiaozhi.com',
pass: 'xxxxxxxxxx'
}
}
},
database: {
client: 'sqlite3',
connection: {
filename: path.join(__dirname, '/content/data/ghost.db')
},
debug: false
},
server: {
host: 'xxxxxx',
port: '80'
}
}
</code></pre></div></div>
<p>运行</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ npm start --production
</code></pre></div></div>
<p>访问并进行管理员账号密码设置</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>http://www.ixiaozhi.com/ghost
</code></pre></div></div>
<p>访问</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>http://www.ixiaozhi.com
</code></pre></div></div>
<p>至此,程序已经在运行了。</p>
<h2 id="forever-运行">Forever 运行</h2>
<p>为了 Node.js 以 daemon 运行并能在错误时自动重启,我选择使用 Forever 组件,当然也有其他的选择~</p>
<p>安装</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo npm install -g forever
</code></pre></div></div>
<p>运行并记录日志( /root/ghost/log 目录要求先存在)</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ NODE_ENV=production forever -l /root/ghost/log/forever.log start index.js
</code></pre></div></div>
<hr />
<p><em>参考</em></p>
<ul>
<li>Node.js 下载: https://nodejs.org/en/download/releases/</li>
<li>Ghost 下载: https://ghost.org/download/</li>
<li>Ghost Git:https://github.com/TryGhost/Ghost</li>
</ul>ixiaozhiblog@ixiaozhi.com前言 惯例,换博客平台第一件事便是发篇搭建过程。本次使用的服务器是 DigitalOcean,CentOS,Blog 程序使用开源的 Ghost。 下载 Ghost 截止至目前,最高版本为0.7.0,那就它了。 文件是一个 zip 包,因为我的 DigitalOcean 上没有安装 Unzip,所以解压前我还得安装 Unzip。 $ sudo yum install unzip $ unzip ghost-0.7.0.zip 下载 Node.js Ghost 0.7.0 需要 Node.js v0.10~v0.12 版本,因些我们去 nodejs.org previous releases 下载旧版本(下载地址见参考)。这里下载的是 v0.12.7 版本。之后便是解压: $ tar zxvf node-v0.12.7-linux-x64.tar.gz 配置环境变量: $ sudo vi /etc/profile 文件末尾添加: PATH=$PATH:/root/ghost/node-v0.12.7-linux-x64/bin 生效配置并验证: $ source /etc/profile $ echo $PATH $ node -v $ npm -v PATH 中已经有 nodejs/bin 且 node、npm返回正确的版本号 v0.12.7 便是配置成功了。 运行 Ghost Ghost 需要依赖 sqlite 数据库,在安装时,sqlite 需要依赖 gcc,g++。所以先安装g++库: $ sudo yum install gcc gcc-c++ npm install $ npm install --production 修改配置 $ cd /root/ghost/ghost $ vi config.js 修改 Production 节点,添加 MailServer 等配置,示例如下: production: { url: 'http://www.ixiaozhi.com', mail: { transport: 'SMTP', options: { service: 'smtp.xxxx.com', auth: { user: 'service@ixiaozhi.com', pass: 'xxxxxxxxxx' } } }, database: { client: 'sqlite3', connection: { filename: path.join(__dirname, '/content/data/ghost.db') }, debug: false }, server: { host: 'xxxxxx', port: '80' } } 运行 $ npm start --production 访问并进行管理员账号密码设置 http://www.ixiaozhi.com/ghost 访问 http://www.ixiaozhi.com 至此,程序已经在运行了。 Forever 运行 为了 Node.js 以 daemon 运行并能在错误时自动重启,我选择使用 Forever 组件,当然也有其他的选择~ 安装 $ sudo npm install -g forever 运行并记录日志( /root/ghost/log 目录要求先存在) $ NODE_ENV=production forever -l /root/ghost/log/forever.log start index.js 参考 Node.js 下载: https://nodejs.org/en/download/releases/ Ghost 下载: https://ghost.org/download/ Ghost Git:https://github.com/TryGhost/Ghostmotan 实现调用授权2017-04-01T00:00:00+08:002017-04-01T00:00:00+08:00https://j.ixiaozhi.com/motan-filter-login<p>motan 提供官方的 spi 扩展方法。 <a href="https://github.com/weibocom/motan/wiki/zh_userguide#%E4%BD%BF%E7%94%A8mock%E5%8D%8F%E8%AE%AE">点击查看</a></p>
<p>方式如下:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>1、实现自定义mock协议类,继承 AbstractMockRpcProtocol,实现 processRequest 方法(自定义 mock 逻辑)。
2、添加spi声明 @SpiMeta(name = "your_mock_protocol") ,在 META-INF/services/com.weibo.api.motan.rpc.Protocol 文件中添加 mock 协议类的类全名。
3、配置 motan:protocol 为 SpiMeta 中声明的名字,即
name=your_mock_protocol,如果在 client 端 mock,就在
basicReferer 或 Referer 中设置对应 protocl;如果在 server
端 mock,则在 export 中配置 ${mock协议的id}:port
</code></pre></div></div>
<h2 id="扩展方式">扩展方式</h2>
<h3 id="扩展点">扩展点</h3>
<ul>
<li>Filter 发送/接收请求过程中增加切面逻辑,默认提供日志统计等功能</li>
<li>HAStrategy 扩展可用性策略,默认提供快速失败等策略</li>
<li>LoadBalance 扩展负载均衡策略,默认提供轮询等策略</li>
<li>Serialization 扩展序列化方式,默认使用 Hession 序列化</li>
<li>Protocol 扩展通讯协议,默认使用 Motan 自定义协议</li>
<li>Registry 扩展服务发现机制,默认支持 Zookeeper、Consul 等服务发现机制</li>
<li>Transport 扩展通讯框架,默认使用 Netty 框架</li>
</ul>
<h3 id="编写一个-motan-扩展">编写一个 Motan 扩展</h3>
<ol>
<li>实现 SPI 扩展点接口</li>
<li>实现类增加注解</li>
</ol>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Spi(scope = Scope.SINGLETON) // 扩展加载形式,单例或多例
@SpiMeta(name = "motan") // name 表示扩展点的名称,根据name加载对应扩展
@Activation(sequence = 100) // 同类型扩展生效顺序,部分扩展点支持。非必填
</code></pre></div></div>
<h2 id="实现">实现</h2>
<p>这里使用 Filter 过滤器扩展点,来现在对 Motan 接口登录授权。 其他的工作就简单了,大致代码如下:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@SpiMeta(name = "baseFilter")
@Activation(sequence = 99)
public class BaseFilter implements Filter {
@Override
public Response filter(Caller<?> caller, Request request) {
Object[] args = request.getArguments();
ApplicationContext applicationContext = 取出全局 Spring Context 对象
ITokenService tokenService = (ITokenService) applicationContext.getBean("tokenServiceHandler");
// 认证逻辑
if (tokenService.isAllow(String.valueOf(args[0]))) {
throw new RuntimeException("RPC manage token expired.\r\nRequest token: " + args[0]); // throw exception to client
}
Response response = caller.call(request);
return response;
}
}
</code></pre></div></div>
<p>然后,添加该扩展至 Motan。在 <code class="highlighter-rouge">resources</code> 下新建路径及文件 <code class="highlighter-rouge">META-INF.services/com.weibo.api.motan.filter.Filter</code>,并往中添加内容</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>com.ixiaozhi.motan.BaseFilter
</code></pre></div></div>
<p>至此,所有的工作已经完成。</p>ixiaozhiblog@ixiaozhi.commotan 提供官方的 spi 扩展方法。 点击查看 方式如下: 1、实现自定义mock协议类,继承 AbstractMockRpcProtocol,实现 processRequest 方法(自定义 mock 逻辑)。 2、添加spi声明 @SpiMeta(name = "your_mock_protocol") ,在 META-INF/services/com.weibo.api.motan.rpc.Protocol 文件中添加 mock 协议类的类全名。 3、配置 motan:protocol 为 SpiMeta 中声明的名字,即 name=your_mock_protocol,如果在 client 端 mock,就在 basicReferer 或 Referer 中设置对应 protocl;如果在 server 端 mock,则在 export 中配置 ${mock协议的id}:port 扩展方式 扩展点 Filter 发送/接收请求过程中增加切面逻辑,默认提供日志统计等功能 HAStrategy 扩展可用性策略,默认提供快速失败等策略 LoadBalance 扩展负载均衡策略,默认提供轮询等策略 Serialization 扩展序列化方式,默认使用 Hession 序列化 Protocol 扩展通讯协议,默认使用 Motan 自定义协议 Registry 扩展服务发现机制,默认支持 Zookeeper、Consul 等服务发现机制 Transport 扩展通讯框架,默认使用 Netty 框架 编写一个 Motan 扩展 实现 SPI 扩展点接口 实现类增加注解 @Spi(scope = Scope.SINGLETON) // 扩展加载形式,单例或多例 @SpiMeta(name = "motan") // name 表示扩展点的名称,根据name加载对应扩展 @Activation(sequence = 100) // 同类型扩展生效顺序,部分扩展点支持。非必填 实现 这里使用 Filter 过滤器扩展点,来现在对 Motan 接口登录授权。 其他的工作就简单了,大致代码如下: @SpiMeta(name = "baseFilter") @Activation(sequence = 99) public class BaseFilter implements Filter { @Override public Response filter(Caller<?> caller, Request request) { Object[] args = request.getArguments(); ApplicationContext applicationContext = 取出全局 Spring Context 对象 ITokenService tokenService = (ITokenService) applicationContext.getBean("tokenServiceHandler"); // 认证逻辑 if (tokenService.isAllow(String.valueOf(args[0]))) { throw new RuntimeException("RPC manage token expired.\r\nRequest token: " + args[0]); // throw exception to client } Response response = caller.call(request); return response; } } 然后,添加该扩展至 Motan。在 resources 下新建路径及文件 META-INF.services/com.weibo.api.motan.filter.Filter,并往中添加内容 com.ixiaozhi.motan.BaseFilter 至此,所有的工作已经完成。API 签名简谈2017-03-07T00:00:00+08:002017-03-07T00:00:00+08:00https://j.ixiaozhi.com/api-simple-signature<p>介绍一种简单的 API 调用时,各参数的完整性效验。仅使用服务器签发 App Key 与 App Secret 和客户进行交互。</p>
<h2 id="方式">方式</h2>
<p>Client 端使用 Server 提供的 App Secret 以及指定的方法进行签名,生成 signature 签名字符串,Server 收到请求先进行验签,确认请求完整。反之亦然。</p>
<h2 id="服务器维护客户端列表">服务器维护客户端列表</h2>
<p>服务器端需要一个客户端、App Key、App Secret 的对应关系。</p>
<p>假使我们有以下客户端:</p>
<table>
<thead>
<tr>
<th>Client</th>
<th>App Key</th>
<th>App Secret</th>
</tr>
</thead>
<tbody>
<tr>
<td>官方客户端</td>
<td>0001</td>
<td>3F2504E0-4F89-11D3-9A0C-0305E82C3301</td>
</tr>
</tbody>
</table>
<p>服务端提供 App Key 与 App Secret 给用户客户端程序。</p>
<p>假设使用参数名称以及当前时间的时间戳 <code class="highlighter-rouge">timestamp</code> (可以规定,服务器与客户端的时间差需要小于10分钟)先按字典序进行排列;然后再拼上 App Secret 进行 md5 signature 后,把 <code class="highlighter-rouge">appkey</code> 及 <code class="highlighter-rouge">signature</code> 附加在字段后传输。</p>
<h2 id="客户端发送请求">客户端发送请求</h2>
<p>某请求有三个参数</p>
<table>
<thead>
<tr>
<th>参数名</th>
<th>值</th>
</tr>
</thead>
<tbody>
<tr>
<td>ab</td>
<td>test</td>
</tr>
<tr>
<td>aa</td>
<td>1234</td>
</tr>
<tr>
<td>zz</td>
<td>haha</td>
</tr>
</tbody>
</table>
<ol>
<li>request params 的字典序为(包含当时时间戳): <code class="highlighter-rouge">aa=1234&ab=test&timestamp=1488857536&zz=haha</code></li>
<li>拼接上 App Secret 后为 <code class="highlighter-rouge">aa=1234&ab=test&timestamp=1488857536&zz=haha3F2504E0-4F89-11D3-9A0C-0305E82C3301</code> ; 对该字符串进行 md5 散列哈希后为 <code class="highlighter-rouge">C194D39F31D115B7F115631237E9CD5E</code></li>
<li>因此最终的请求 request params 为:<code class="highlighter-rouge">aa=1234&ab=test&timestamp=1488857536&zz=haha&appkey=0001&signature=C194D39F31D115B7F115631237E9CD5E</code></li>
</ol>
<h2 id="服务器验签">服务器验签</h2>
<p>先对接收到的参数进行拆解,分成四个部分</p>
<ul>
<li>原请求再进行字典序 <code class="highlighter-rouge">aa=1234&ab=test&timestamp=1488857536&zz=haha</code></li>
<li>App Key <code class="highlighter-rouge">0001</code></li>
<li>signature <code class="highlighter-rouge">C194D39F31D115B7F115631237E9CD5E</code></li>
<li>timestamp <code class="highlighter-rouge">1488857536</code></li>
</ul>
<p>然后进行验签</p>
<ol>
<li>判断时间是否过期</li>
<li>利用 App Key 查表得出 App Secret;把原请求进行相同的算法,把得到的 md5 哈希字符串与收到的 signature 字段串进行比较</li>
</ol>
<h2 id="反向">反向</h2>
<p>服务器返回给客户端的数据,可以进行同样的方式进行签名与验签。</p>ixiaozhiblog@ixiaozhi.com介绍一种简单的 API 调用时,各参数的完整性效验。仅使用服务器签发 App Key 与 App Secret 和客户进行交互。 方式 Client 端使用 Server 提供的 App Secret 以及指定的方法进行签名,生成 signature 签名字符串,Server 收到请求先进行验签,确认请求完整。反之亦然。 服务器维护客户端列表 服务器端需要一个客户端、App Key、App Secret 的对应关系。 假使我们有以下客户端: Client App Key App Secret 官方客户端 0001 3F2504E0-4F89-11D3-9A0C-0305E82C3301 服务端提供 App Key 与 App Secret 给用户客户端程序。 假设使用参数名称以及当前时间的时间戳 timestamp (可以规定,服务器与客户端的时间差需要小于10分钟)先按字典序进行排列;然后再拼上 App Secret 进行 md5 signature 后,把 appkey 及 signature 附加在字段后传输。 客户端发送请求 某请求有三个参数 参数名 值 ab test aa 1234 zz haha request params 的字典序为(包含当时时间戳): aa=1234&ab=test&timestamp=1488857536&zz=haha 拼接上 App Secret 后为 aa=1234&ab=test&timestamp=1488857536&zz=haha3F2504E0-4F89-11D3-9A0C-0305E82C3301 ; 对该字符串进行 md5 散列哈希后为 C194D39F31D115B7F115631237E9CD5E 因此最终的请求 request params 为:aa=1234&ab=test&timestamp=1488857536&zz=haha&appkey=0001&signature=C194D39F31D115B7F115631237E9CD5E 服务器验签 先对接收到的参数进行拆解,分成四个部分 原请求再进行字典序 aa=1234&ab=test&timestamp=1488857536&zz=haha App Key 0001 signature C194D39F31D115B7F115631237E9CD5E timestamp 1488857536 然后进行验签 判断时间是否过期 利用 App Key 查表得出 App Secret;把原请求进行相同的算法,把得到的 md5 哈希字符串与收到的 signature 字段串进行比较 反向 服务器返回给客户端的数据,可以进行同样的方式进行签名与验签。Spring-Simple-memcached 增加支持只取键值2016-08-02T00:00:00+08:002016-08-02T00:00:00+08:00https://j.ixiaozhi.com/spring-simple-memcached-only-get<p>这里使用的 Spring-Simple-Memcached 的版本为:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>compile 'com.google.code.simple-spring-memcached:spymemcached:2.8.4'
compile 'com.google.code.simple-spring-memcached:spymemcached-provider:3.1.0'
compile 'com.google.code.simple-spring-memcached:simple-spring-memcached:3.1.0'
</code></pre></div></div>
<p>ssm 支持的读取相关的方法有:</p>
<p>@ReadThroughAssignCache: 读取指定key缓存
@ReadThroughSingleCache: 读取单个缓存
@ReadThroughMultiCache: 读取多个缓存</p>
<p>在 get(key) key 值不存在时,该方法会默认把返回的 value 值添加至该 key 值的缓存。</p>
<p>假设,我们有需求要只取某 key 的缓存值,不存在时也不需要增加缓存,以下是对 ssm 包的扩展。</p>
<p>google.code.ssm.ReadOnlyThroughSingleCacheAdvice.java</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">package</span> <span class="n">com</span><span class="p">.</span><span class="n">google</span><span class="p">.</span><span class="n">code</span><span class="p">.</span><span class="n">ssm</span><span class="p">.</span><span class="n">aop</span><span class="p">;</span>
<span class="n">import</span> <span class="n">com</span><span class="p">.</span><span class="n">google</span><span class="p">.</span><span class="n">code</span><span class="p">.</span><span class="n">ssm</span><span class="p">.</span><span class="n">aop</span><span class="p">.</span><span class="n">support</span><span class="p">.</span><span class="n">AnnotationData</span><span class="p">;</span>
<span class="n">import</span> <span class="n">com</span><span class="p">.</span><span class="n">google</span><span class="p">.</span><span class="n">code</span><span class="p">.</span><span class="n">ssm</span><span class="p">.</span><span class="n">api</span><span class="p">.</span><span class="n">ReadOnlyThroughSingleCache</span><span class="p">;</span>
<span class="n">import</span> <span class="n">org</span><span class="p">.</span><span class="n">aspectj</span><span class="p">.</span><span class="n">lang</span><span class="p">.</span><span class="n">ProceedingJoinPoint</span><span class="p">;</span>
<span class="n">import</span> <span class="n">org</span><span class="p">.</span><span class="n">aspectj</span><span class="p">.</span><span class="n">lang</span><span class="p">.</span><span class="n">annotation</span><span class="p">.</span><span class="n">Around</span><span class="p">;</span>
<span class="n">import</span> <span class="n">org</span><span class="p">.</span><span class="n">aspectj</span><span class="p">.</span><span class="n">lang</span><span class="p">.</span><span class="n">annotation</span><span class="p">.</span><span class="n">Aspect</span><span class="p">;</span>
<span class="n">import</span> <span class="n">org</span><span class="p">.</span><span class="n">aspectj</span><span class="p">.</span><span class="n">lang</span><span class="p">.</span><span class="n">annotation</span><span class="p">.</span><span class="n">Pointcut</span><span class="p">;</span>
<span class="n">import</span> <span class="n">org</span><span class="p">.</span><span class="n">slf4j</span><span class="p">.</span><span class="n">Logger</span><span class="p">;</span>
<span class="n">import</span> <span class="n">org</span><span class="p">.</span><span class="n">slf4j</span><span class="p">.</span><span class="n">LoggerFactory</span><span class="p">;</span>
<span class="p">/**</span>
<span class="p">*</span> <span class="n">Created</span> <span class="n">by</span> <span class="n">ixiaozhi</span> <span class="n">on</span> <span class="m">16</span><span class="p">/</span><span class="m">7</span><span class="p">/</span><span class="m">27.</span>
<span class="p">*/</span>
<span class="p">@</span><span class="n">Aspect</span>
<span class="k">public</span> <span class="n">class</span> <span class="n">ReadOnlyThroughSingleCacheAdvice</span> <span class="n">extends</span> <span class="n">SingleReadCacheAdviceExtend</span><span class="p"><</span><span class="n">ReadOnlyThroughSingleCache</span><span class="p">></span> <span class="p">{</span>
<span class="n">private</span> <span class="n">static</span> <span class="n">final</span> <span class="n">Logger</span> <span class="n">LOG</span> <span class="p">=</span> <span class="n">LoggerFactory</span><span class="p">.</span><span class="n">getLogger</span><span class="p">(</span><span class="n">ReadThroughSingleCacheAdvice</span><span class="p">.</span><span class="n">class</span><span class="p">);</span>
<span class="k">public</span> <span class="n">ReadOnlyThroughSingleCacheAdvice</span><span class="p">()</span> <span class="p">{</span>
<span class="n">super</span><span class="p">(</span><span class="n">ReadOnlyThroughSingleCache</span><span class="p">.</span><span class="n">class</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">@</span><span class="n">Pointcut</span><span class="p">(</span><span class="s2">"@annotation(com.google.code.ssm.api.ReadOnlyThroughSingleCache)"</span><span class="p">)</span>
<span class="k">public</span> <span class="n">void</span> <span class="n">getSingle</span><span class="p">()</span> <span class="p">{</span>
<span class="p">}</span>
<span class="p">@</span><span class="n">Around</span><span class="p">(</span><span class="s2">"getSingle()"</span><span class="p">)</span>
<span class="k">public</span> <span class="n">Object</span> <span class="n">cacheGetSingle</span><span class="p">(</span><span class="n">final</span> <span class="n">ProceedingJoinPoint</span> <span class="n">pjp</span><span class="p">)</span> <span class="n">throws</span> <span class="n">Throwable</span> <span class="p">{</span>
<span class="n">return</span> <span class="n">cache</span><span class="p">(</span><span class="n">pjp</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">@</span><span class="n">Override</span>
<span class="n">protected</span> <span class="k">String</span> <span class="n">getCacheKey</span><span class="p">(</span><span class="n">final</span> <span class="n">AnnotationData</span> <span class="n">data</span><span class="p">,</span> <span class="n">final</span> <span class="n">Object</span><span class="p">[]</span> <span class="n">args</span><span class="p">,</span> <span class="n">final</span> <span class="k">String</span> <span class="n">methodDesc</span><span class="p">)</span> <span class="n">throws</span> <span class="n">Exception</span> <span class="p">{</span>
<span class="n">return</span> <span class="n">getCacheBase</span><span class="p">().</span><span class="n">getCacheKeyBuilder</span><span class="p">().</span><span class="n">getCacheKey</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">args</span><span class="p">,</span> <span class="n">methodDesc</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">@</span><span class="n">Override</span>
<span class="n">protected</span> <span class="n">Logger</span> <span class="n">getLogger</span><span class="p">()</span> <span class="p">{</span>
<span class="n">return</span> <span class="n">LOG</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>com.google.code.ssm.aop.SingleReadCacheAdviceExtend.java</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">package</span> <span class="n">com</span><span class="p">.</span><span class="n">google</span><span class="p">.</span><span class="n">code</span><span class="p">.</span><span class="n">ssm</span><span class="p">.</span><span class="n">aop</span><span class="p">;</span>
<span class="n">import</span> <span class="n">com</span><span class="p">.</span><span class="n">google</span><span class="p">.</span><span class="n">code</span><span class="p">.</span><span class="n">ssm</span><span class="p">.</span><span class="n">aop</span><span class="p">.</span><span class="n">support</span><span class="p">.</span><span class="n">AnnotationData</span><span class="p">;</span>
<span class="n">import</span> <span class="n">com</span><span class="p">.</span><span class="n">google</span><span class="p">.</span><span class="n">code</span><span class="p">.</span><span class="n">ssm</span><span class="p">.</span><span class="n">aop</span><span class="p">.</span><span class="n">support</span><span class="p">.</span><span class="n">AnnotationDataBuilder</span><span class="p">;</span>
<span class="n">import</span> <span class="n">com</span><span class="p">.</span><span class="n">google</span><span class="p">.</span><span class="n">code</span><span class="p">.</span><span class="n">ssm</span><span class="p">.</span><span class="n">api</span><span class="p">.</span><span class="n">format</span><span class="p">.</span><span class="n">SerializationType</span><span class="p">;</span>
<span class="n">import</span> <span class="n">org</span><span class="p">.</span><span class="n">aspectj</span><span class="p">.</span><span class="n">lang</span><span class="p">.</span><span class="n">ProceedingJoinPoint</span><span class="p">;</span>
<span class="n">import</span> <span class="n">java</span><span class="p">.</span><span class="n">lang</span><span class="p">.</span><span class="n">annotation</span><span class="p">.</span><span class="n">Annotation</span><span class="p">;</span>
<span class="n">import</span> <span class="n">java</span><span class="p">.</span><span class="n">lang</span><span class="p">.</span><span class="n">reflect</span><span class="p">.</span><span class="n">Method</span><span class="p">;</span>
<span class="p">/**</span>
<span class="p">*</span> <span class="n">Created</span> <span class="n">by</span> <span class="n">ixiaozhi</span> <span class="n">on</span> <span class="m">16</span><span class="p">/</span><span class="m">7</span><span class="p">/</span><span class="m">27.</span>
<span class="p">*/</span>
<span class="n">abstract</span> <span class="n">class</span> <span class="n">SingleReadCacheAdviceExtend</span><span class="p"><</span><span class="n">T</span> <span class="n">extends</span> <span class="n">Annotation</span><span class="p">></span> <span class="n">extends</span> <span class="n">CacheAdvice</span> <span class="p">{</span>
<span class="n">private</span> <span class="n">final</span> <span class="n">Class</span><span class="p"><</span><span class="n">T</span><span class="p">></span> <span class="n">annotationClass</span><span class="p">;</span>
<span class="n">protected</span> <span class="n">SingleReadCacheAdviceExtend</span><span class="p">(</span><span class="n">final</span> <span class="n">Class</span><span class="p"><</span><span class="n">T</span><span class="p">></span> <span class="n">annotationClass</span><span class="p">)</span> <span class="p">{</span>
<span class="n">this</span><span class="p">.</span><span class="n">annotationClass</span> <span class="p">=</span> <span class="n">annotationClass</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">protected</span> <span class="n">Object</span> <span class="n">cache</span><span class="p">(</span><span class="n">final</span> <span class="n">ProceedingJoinPoint</span> <span class="n">pjp</span><span class="p">)</span> <span class="n">throws</span> <span class="n">Throwable</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">isDisabled</span><span class="p">())</span> <span class="p">{</span>
<span class="n">getLogger</span><span class="p">().</span><span class="n">info</span><span class="p">(</span><span class="s2">"Cache disabled"</span><span class="p">);</span>
<span class="n">return</span> <span class="n">pjp</span><span class="p">.</span><span class="n">proceed</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">//</span> <span class="n">This</span> <span class="n">is</span> <span class="n">injected</span> <span class="n">caching</span><span class="p">.</span> <span class="k">If</span> <span class="n">anything</span> <span class="n">goes</span> <span class="n">wrong</span> <span class="k">in</span> <span class="n">the</span> <span class="n">caching</span><span class="p">,</span> <span class="n">LOG</span>
<span class="p">//</span> <span class="n">the</span> <span class="n">crap</span> <span class="n">outta</span> <span class="n">it</span><span class="p">,</span> <span class="n">but</span> <span class="k">do</span> <span class="k">not</span> <span class="n">let</span> <span class="n">it</span> <span class="n">surface</span> <span class="n">up</span> <span class="n">past</span> <span class="n">the</span> <span class="n">AOP</span> <span class="n">injection</span> <span class="n">itself</span><span class="p">.</span>
<span class="n">final</span> <span class="n">T</span> <span class="n">annotation</span><span class="p">;</span>
<span class="n">final</span> <span class="n">AnnotationData</span> <span class="n">data</span><span class="p">;</span>
<span class="n">final</span> <span class="n">SerializationType</span> <span class="n">serializationType</span><span class="p">;</span>
<span class="k">String</span> <span class="n">cacheKey</span> <span class="p">=</span> <span class="n">null</span><span class="p">;</span>
<span class="n">try</span> <span class="p">{</span>
<span class="n">final</span> <span class="n">Method</span> <span class="n">methodToCache</span> <span class="p">=</span> <span class="n">getCacheBase</span><span class="p">().</span><span class="n">getMethodToCache</span><span class="p">(</span><span class="n">pjp</span><span class="p">);</span>
<span class="n">getCacheBase</span><span class="p">().</span><span class="n">verifyReturnTypeIsNoVoid</span><span class="p">(</span><span class="n">methodToCache</span><span class="p">,</span> <span class="n">annotationClass</span><span class="p">);</span>
<span class="n">verifyNoUseJsonAnnotation</span><span class="p">(</span><span class="n">methodToCache</span><span class="p">);</span>
<span class="n">annotation</span> <span class="p">=</span> <span class="n">methodToCache</span><span class="p">.</span><span class="n">getAnnotation</span><span class="p">(</span><span class="n">annotationClass</span><span class="p">);</span>
<span class="n">serializationType</span> <span class="p">=</span> <span class="n">getCacheBase</span><span class="p">().</span><span class="n">getSerializationType</span><span class="p">(</span><span class="n">methodToCache</span><span class="p">);</span>
<span class="n">data</span> <span class="p">=</span> <span class="n">AnnotationDataBuilder</span><span class="p">.</span><span class="n">buildAnnotationData</span><span class="p">(</span><span class="n">annotation</span><span class="p">,</span> <span class="n">annotationClass</span><span class="p">,</span> <span class="n">methodToCache</span><span class="p">);</span>
<span class="n">cacheKey</span> <span class="p">=</span> <span class="n">getCacheKey</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">pjp</span><span class="p">.</span><span class="n">getArgs</span><span class="p">(),</span> <span class="n">methodToCache</span><span class="p">.</span><span class="n">toString</span><span class="p">());</span>
<span class="n">final</span> <span class="n">Object</span> <span class="n">result</span> <span class="p">=</span> <span class="n">getCacheBase</span><span class="p">().</span><span class="n">getCache</span><span class="p">(</span><span class="n">data</span><span class="p">).</span><span class="n">get</span><span class="p">(</span><span class="n">cacheKey</span><span class="p">,</span> <span class="n">serializationType</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">result</span> <span class="c1">!= null) {
</span> <span class="n">getLogger</span><span class="p">().</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Cache hit."</span><span class="p">);</span>
<span class="n">return</span> <span class="n">getCacheBase</span><span class="p">().</span><span class="n">getResult</span><span class="p">(</span><span class="n">result</span><span class="p">);</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="n">return</span> <span class="n">null</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span> <span class="n">catch</span> <span class="p">(</span><span class="n">Throwable</span> <span class="n">ex</span><span class="p">)</span> <span class="p">{</span>
<span class="n">warn</span><span class="p">(</span><span class="n">ex</span><span class="p">,</span> <span class="s2">"Caching on method %s and key [%s] aborted due to an error."</span><span class="p">,</span> <span class="n">pjp</span><span class="p">.</span><span class="n">toShortString</span><span class="p">(),</span> <span class="n">cacheKey</span><span class="p">);</span>
<span class="n">return</span> <span class="n">pjp</span><span class="p">.</span><span class="n">proceed</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="n">protected</span> <span class="n">abstract</span> <span class="k">String</span> <span class="n">getCacheKey</span><span class="p">(</span><span class="n">final</span> <span class="n">AnnotationData</span> <span class="n">data</span><span class="p">,</span> <span class="n">final</span> <span class="n">Object</span><span class="p">[]</span> <span class="n">args</span><span class="p">,</span> <span class="n">final</span> <span class="k">String</span> <span class="n">methodDesc</span><span class="p">)</span> <span class="n">throws</span> <span class="n">Exception</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>com.google.code.ssm.api.ReadOnlyThroughSingleCache.java</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">package</span> <span class="n">com</span><span class="p">.</span><span class="n">google</span><span class="p">.</span><span class="n">code</span><span class="p">.</span><span class="n">ssm</span><span class="p">.</span><span class="n">api</span><span class="p">;</span>
<span class="n">import</span> <span class="n">java</span><span class="p">.</span><span class="n">lang</span><span class="p">.</span><span class="n">annotation</span><span class="p">.</span><span class="n">ElementType</span><span class="p">;</span>
<span class="n">import</span> <span class="n">java</span><span class="p">.</span><span class="n">lang</span><span class="p">.</span><span class="n">annotation</span><span class="p">.</span><span class="n">Retention</span><span class="p">;</span>
<span class="n">import</span> <span class="n">java</span><span class="p">.</span><span class="n">lang</span><span class="p">.</span><span class="n">annotation</span><span class="p">.</span><span class="n">RetentionPolicy</span><span class="p">;</span>
<span class="n">import</span> <span class="n">java</span><span class="p">.</span><span class="n">lang</span><span class="p">.</span><span class="n">annotation</span><span class="p">.</span><span class="n">Target</span><span class="p">;</span>
<span class="p">/**</span>
<span class="p">*</span> <span class="n">Created</span> <span class="n">by</span> <span class="n">ixiaozhi</span> <span class="n">on</span> <span class="m">16</span><span class="p">/</span><span class="m">7</span><span class="p">/</span><span class="m">27.</span>
<span class="p">*/</span>
<span class="p">@</span><span class="n">Retention</span><span class="p">(</span><span class="n">RetentionPolicy</span><span class="p">.</span><span class="n">RUNTIME</span><span class="p">)</span>
<span class="p">@</span><span class="n">Target</span><span class="p">(</span><span class="n">ElementType</span><span class="p">.</span><span class="n">METHOD</span><span class="p">)</span>
<span class="k">public</span> <span class="p">@</span><span class="n">interface</span> <span class="n">ReadOnlyThroughSingleCache</span> <span class="p">{</span>
<span class="k">String</span> <span class="n">namespace</span><span class="p">()</span> <span class="n">default</span> <span class="n">AnnotationConstants</span><span class="p">.</span><span class="n">DEFAULT_STRING</span><span class="p">;</span>
<span class="n">int</span> <span class="n">expiration</span><span class="p">()</span> <span class="n">default</span> <span class="m">0</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>配置文件中的 simplesm-context.xml 添加以下实例化配置:(如果有的话;如果不存在,在自己的 Spring 配置中添加也是一样的)</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><bean id="readOnlyThroughSingleCache" class="com.google.code.ssm.aop.ReadOnlyThroughSingleCacheAdvice">
<property name="cacheBase" ref="cacheBase" />
</bean>
</code></pre></div></div>
<p>使用时,使用自定义的注解: <code class="highlighter-rouge">@ReadOnlyThroughSingleCache</code>,参数与 ReadThroughSingleCache 相同,有 namespace 与 expiration。</p>ixiaozhiblog@ixiaozhi.com这里使用的 Spring-Simple-Memcached 的版本为: compile 'com.google.code.simple-spring-memcached:spymemcached:2.8.4' compile 'com.google.code.simple-spring-memcached:spymemcached-provider:3.1.0' compile 'com.google.code.simple-spring-memcached:simple-spring-memcached:3.1.0' ssm 支持的读取相关的方法有: @ReadThroughAssignCache: 读取指定key缓存 @ReadThroughSingleCache: 读取单个缓存 @ReadThroughMultiCache: 读取多个缓存 在 get(key) key 值不存在时,该方法会默认把返回的 value 值添加至该 key 值的缓存。 假设,我们有需求要只取某 key 的缓存值,不存在时也不需要增加缓存,以下是对 ssm 包的扩展。 google.code.ssm.ReadOnlyThroughSingleCacheAdvice.java package com.google.code.ssm.aop; import com.google.code.ssm.aop.support.AnnotationData; import com.google.code.ssm.api.ReadOnlyThroughSingleCache; import org.aspectj.lang.ProceedingJoinPoint; import org.aspectj.lang.annotation.Around; import org.aspectj.lang.annotation.Aspect; import org.aspectj.lang.annotation.Pointcut; import org.slf4j.Logger; import org.slf4j.LoggerFactory; /** * Created by ixiaozhi on 16/7/27. */ @Aspect public class ReadOnlyThroughSingleCacheAdvice extends SingleReadCacheAdviceExtend<ReadOnlyThroughSingleCache> { private static final Logger LOG = LoggerFactory.getLogger(ReadThroughSingleCacheAdvice.class); public ReadOnlyThroughSingleCacheAdvice() { super(ReadOnlyThroughSingleCache.class); } @Pointcut("@annotation(com.google.code.ssm.api.ReadOnlyThroughSingleCache)") public void getSingle() { } @Around("getSingle()") public Object cacheGetSingle(final ProceedingJoinPoint pjp) throws Throwable { return cache(pjp); } @Override protected String getCacheKey(final AnnotationData data, final Object[] args, final String methodDesc) throws Exception { return getCacheBase().getCacheKeyBuilder().getCacheKey(data, args, methodDesc); } @Override protected Logger getLogger() { return LOG; } } com.google.code.ssm.aop.SingleReadCacheAdviceExtend.java package com.google.code.ssm.aop; import com.google.code.ssm.aop.support.AnnotationData; import com.google.code.ssm.aop.support.AnnotationDataBuilder; import com.google.code.ssm.api.format.SerializationType; import org.aspectj.lang.ProceedingJoinPoint; import java.lang.annotation.Annotation; import java.lang.reflect.Method; /** * Created by ixiaozhi on 16/7/27. */ abstract class SingleReadCacheAdviceExtend<T extends Annotation> extends CacheAdvice { private final Class<T> annotationClass; protected SingleReadCacheAdviceExtend(final Class<T> annotationClass) { this.annotationClass = annotationClass; } protected Object cache(final ProceedingJoinPoint pjp) throws Throwable { if (isDisabled()) { getLogger().info("Cache disabled"); return pjp.proceed(); } // This is injected caching. If anything goes wrong in the caching, LOG // the crap outta it, but do not let it surface up past the AOP injection itself. final T annotation; final AnnotationData data; final SerializationType serializationType; String cacheKey = null; try { final Method methodToCache = getCacheBase().getMethodToCache(pjp); getCacheBase().verifyReturnTypeIsNoVoid(methodToCache, annotationClass); verifyNoUseJsonAnnotation(methodToCache); annotation = methodToCache.getAnnotation(annotationClass); serializationType = getCacheBase().getSerializationType(methodToCache); data = AnnotationDataBuilder.buildAnnotationData(annotation, annotationClass, methodToCache); cacheKey = getCacheKey(data, pjp.getArgs(), methodToCache.toString()); final Object result = getCacheBase().getCache(data).get(cacheKey, serializationType); if (result != null) { getLogger().debug("Cache hit."); return getCacheBase().getResult(result); } else { return null; } } catch (Throwable ex) { warn(ex, "Caching on method %s and key [%s] aborted due to an error.", pjp.toShortString(), cacheKey); return pjp.proceed(); } } protected abstract String getCacheKey(final AnnotationData data, final Object[] args, final String methodDesc) throws Exception; } com.google.code.ssm.api.ReadOnlyThroughSingleCache.java package com.google.code.ssm.api; import java.lang.annotation.ElementType; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target; /** * Created by ixiaozhi on 16/7/27. */ @Retention(RetentionPolicy.RUNTIME) @Target(ElementType.METHOD) public @interface ReadOnlyThroughSingleCache { String namespace() default AnnotationConstants.DEFAULT_STRING; int expiration() default 0; } 配置文件中的 simplesm-context.xml 添加以下实例化配置:(如果有的话;如果不存在,在自己的 Spring 配置中添加也是一样的) <bean id="readOnlyThroughSingleCache" class="com.google.code.ssm.aop.ReadOnlyThroughSingleCacheAdvice"> <property name="cacheBase" ref="cacheBase" /> </bean> 使用时,使用自定义的注解: @ReadOnlyThroughSingleCache,参数与 ReadThroughSingleCache 相同,有 namespace 与 expiration。Spring 与 Quartz 动态配置2016-06-29T00:00:00+08:002016-06-29T00:00:00+08:00https://j.ixiaozhi.com/spring-quartz-cluster-dynamic-configuration<p>因为项目的需求,需要有动态配置计划任务的功能。
本文在 Quartz JobBean 中获取配置的 Quartz cronExpression 时间表达式及 Spring Bean 的对象名、方法名并运行。</p>
<h2 id="准备">准备</h2>
<h3 id="环境">环境</h3>
<ul>
<li>quartz : 2.2.2</li>
<li>spring : 4.2.3.RELEASE</li>
</ul>
<h3 id="配置">配置</h3>
<p>假设已经配置好数据源,且在数据库中已经建好相关的 Quartz 表。</p>
<p>Spring 配置文件配置好单机器的 Quartz 任务。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><bean id="localQuartzScheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean"></bean>
</code></pre></div></div>
<p>去除原有的 quartz 的 jobDetail 等其他设置,下面我们将把这些改为动态设置。</p>
<h2 id="集群">集群</h2>
<p>Spring 增加 cluster quartz 配置。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><!-- Quartz集群Scheduler -->
<bean id="clusterQuartzScheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<!-- quartz配置文件路径-->
<property name="configLocation" value="classpath:quartz.properties"/>
<!-- 启动时延期3秒开始任务 -->
<property name="startupDelay" value="3"/>
<!-- 保存Job数据到数据库所需的数据源 -->
<property name="dataSource" ref="dataSource"/>
<!-- Job接受applicationContext的成员变量名 -->
<property name="applicationContextSchedulerContextKey" value="applicationContext"/>
<property name="overwriteExistingJobs" value="true"/>
</bean>
</code></pre></div></div>
<p>配置中使用的 dataSource 数据源,需要提前配置;quartz.properties 属性文件自行配置。集群定时任务的任务会序列化后储存至数据库,在某机器 crash 后,可以快速的切换到新的机器去运行,并且保证有仅只有一台机器运行计划任务。</p>
<p>先写 QuartzRunnable 文件,这个文件我用来启动 Quartz 定时任务。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>import org.quartz.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.ApplicationContext;
/**
* Created by ixiaozhi on 16/6/27.
*/
public class QuartzRunnable {
private static final Logger logger = LoggerFactory.getLogger(QuartzRunnable.class);
private ApplicationContext context;
/**
* 构造函数, 传入 applicationContext
*
* @param context
*/
public QuartzRunnable(ApplicationContext context) {
this.context = context;
}
public void work() throws SchedulerException {
logger.info("quartz is running ...");
// scheduler 对象
Scheduler schedulerCluster = (Scheduler) context.getBean("clusterQuartzScheduler");
Scheduler schedulerLocal = (Scheduler) context.getBean("localQuartzScheduler");
List<ScheduleJob> allQuartzJobs = ......; // 从数据库或者配置文件或者其他任何地方取得 Quartz 任务的配置文件, ScheduleJob 对象为自定义的 Quartz 任务设置,对象的属性见下文
// 启动定时任务
for (ScheduleJob job : allQuartzJobs) {
// 区分本机运行或集群运行
Scheduler scheduler;
if (job.getIsCluster() == 1) {
scheduler = schedulerCluster;
} else {
scheduler = schedulerLocal;
}
TriggerKey triggerKey = TriggerKey.triggerKey(job.getJobName(), job.getJobGroup());
CronTrigger trigger = (CronTrigger) scheduler.getTrigger(triggerKey);
//不存在,创建一个
if (null == trigger) {
JobDetail jobDetail = JobBuilder.newJob(MyDetailQuartzJobBean.class).withIdentity(job.getJobName(), job.getJobGroup()).build();
JobDataMap dataMap = jobDetail.getJobDataMap();
dataMap.put("scheduleJob", job); // 传递 job 对象至执行的方法体
//表达式调度构建器
CronScheduleBuilder scheduleBuilder = CronScheduleBuilder.cronSchedule(job.getCronExpression());
//按新的cronExpression表达式构建一个新的trigger
trigger = TriggerBuilder.newTrigger().withIdentity(job.getJobName(), job.getJobGroup()).withSchedule(scheduleBuilder).withDescription(job.getDescription()).build();
scheduler.scheduleJob(jobDetail, trigger);
} else {
// Trigger已存在,那么更新相应的定时设置
//表达式调度构建器
CronScheduleBuilder scheduleBuilder = CronScheduleBuilder.cronSchedule(job.getCronExpression());
//按新的cronExpression表达式重新构建trigger
trigger = trigger.getTriggerBuilder().withIdentity(triggerKey).withSchedule(scheduleBuilder).build();
//按新的trigger重新设置job执行
scheduler.rescheduleJob(triggerKey, trigger);
}
}
}
}
</code></pre></div></div>
<p>从 Spring ApplicationContext 上下文对象中取得本文上面配置的两个 Quartz Scheduler,分别用于启动本机的定时任务与集群定时任务。再根据自己的配置,构造对应的 Trigger 与 Job,加入不同的计划任务中执行。</p>
<p>对于计划任务,都使用同一个类 <code class="highlighter-rouge">MyDetailQuartzJobBean</code> 进行启动。在配置中可以根据配置反射启动相应的方法。</p>
<p>我的 ScheduleJob 计划任务配置的属性有以下。 <code class="highlighter-rouge">targetObject</code> 为 Spring 中注入的 bean 名称, <code class="highlighter-rouge">targetMethod</code> 用于定时任务启动的方法入口。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> public class ScheduleJob implements Serializable {
private static final long serialVersionUID = -4166311089940333025L;
private String jobId; // 任务 ID
private String jobName; // 任务名称
private String jobGroup; // 任务分组
private String cronExpression; // 时间表达式
private String description; // 任务描述
private String targetObject; // Spring 注入的类名
private String targetMethod; // 方法
private int isCluster;// 是否集群运行
// getter and setter
... ...
}
</code></pre></div></div>
<p>比如测试的 ScheduleJob 对象:
jobId=1;
jobName=”Test”;
jobGroup=”DEFAULT”;
cronExpression=”1/30 * * * * ?”; // 从1秒开始,每30秒执行一次
description=”测试任务”;
targetObject=”testService”;
targetMethod=”quartzTest”;
isCluster=true;
含义为,将从 1 秒开始,每 30 秒在集群中的某一台机器运行一次,从 targetObject 的注入对象中的 targetMethod 方法。</p>
<p>现在来实现 JobBean 计划任务执行类,我命名为 <code class="highlighter-rouge">MyDetailQuartzJobBean.java</code>。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>import org.quartz.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.scheduling.quartz.QuartzJobBean;
import java.lang.reflect.Method;
/**
* 动态运行方法
*/
//@PersistJobDataAfterExecution
//@DisallowConcurrentExecution //确保多个任务不会同时运行
public class MyDetailQuartzJobBean extends QuartzJobBean {
private static final Logger logger = LoggerFactory.getLogger(MyDetailQuartzJobBean.class);
private ScheduleJob scheduleJob;
protected void executeInternal(JobExecutionContext context)
throws JobExecutionException {
try {
Object targetObject = ApplicationContextUtil.getBean(scheduleJob.getTargetObject());
Method m;
try {
m = targetObject.getClass().getMethod(scheduleJob.getTargetMethod(), new Class[]{});
m.invoke(targetObject, new Object[]{});
} catch (SecurityException e) {
logger.error(e.getMessage(), e);
} catch (NoSuchMethodException e) {
logger.error(e.getMessage(), e);
}
} catch (Exception e) {
logger.error(e.getMessage(), e);
throw new JobExecutionException(e);
}
}
public void setScheduleJob(ScheduleJob scheduleJob) {
this.scheduleJob = scheduleJob;
}
}
</code></pre></div></div>
<p>JobBean 需要继承至 QuartzJobBean,并重写 executeInternal 方法。而且,Quartz 集群中运行的 QuartzJobBean 必须实现序列化。但是,applicationContext 并不支持序列化,在这里面直接注入对象会报 exception 且无法使用。</p>
<p>因此我使用静态化来保存 applicationContext 对象,实现类 <code class="highlighter-rouge">ApplicationContextUtil</code> 。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Component
public class ApplicationContextUtil implements ApplicationContextAware {
private static ApplicationContext applicationContext; // Spring应用上下文环境
public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
ApplicationContextUtil.applicationContext = applicationContext;
}
public static ApplicationContext getApplicationContext() {
return applicationContext;
}
public static Object getBean(String beanName) {
return applicationContext.getBean(beanName);
}
@SuppressWarnings("unchecked")
public static <T> T getBeanDetail(String beanName) throws BeansException {
return (T) applicationContext.getBean(beanName);
}
}
</code></pre></div></div>
<p>Spring 配置文件中注册该 Bean。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><bean id="applicationContextUtil" class="com.ixiaozhi.util.ApplicationContextUtil"/>
</code></pre></div></div>
<p>利用 getBean 可直接从 Spring 上下文中取得注入的对象,如上述的 <code class="highlighter-rouge">MyDetailQuartzJobBean</code> 使用该方法绕过 Spring ApplicationContext 无法序列化的问题,且取得 Spring Bean 并反射调用其中的方法。</p>
<h2 id="测试">测试</h2>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> /**
* 程序入口
*
* @param args
*/
public void test() {
// 初始化 Spring
ApplicationContext applicationContext = new ClassPathXmlApplicationContext("/applicationContext.xml");
// 启动定时任务
QuartzRunnable quartz = new QuartzRunnable(applicationContext);
quartz.work();
}
</code></pre></div></div>
<p>其他更好的实现方案,欢迎留言一起探讨。</p>
<h2 id="参考">参考</h2>
<ul>
<li><a href="http://itindex.net/detail/53315-spring-quartz-%E7%AE%A1%E7%90%86">Spring+quartz 实现动态管理任务</a></li>
<li><a href="http://tech.meituan.com/mt-crm-quartz.html">Quartz应用与集群原理分析</a></li>
<li><a href="http://shift-alt-ctrl.iteye.com/blog/2215983">Spring与Quartz Cluster备忘</a></li>
</ul>ixiaozhiblog@ixiaozhi.com因为项目的需求,需要有动态配置计划任务的功能。 本文在 Quartz JobBean 中获取配置的 Quartz cronExpression 时间表达式及 Spring Bean 的对象名、方法名并运行。 准备 环境 quartz : 2.2.2 spring : 4.2.3.RELEASE 配置 假设已经配置好数据源,且在数据库中已经建好相关的 Quartz 表。 Spring 配置文件配置好单机器的 Quartz 任务。 <bean id="localQuartzScheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean"></bean> 去除原有的 quartz 的 jobDetail 等其他设置,下面我们将把这些改为动态设置。 集群 Spring 增加 cluster quartz 配置。 <!-- Quartz集群Scheduler --> <bean id="clusterQuartzScheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean"> <!-- quartz配置文件路径--> <property name="configLocation" value="classpath:quartz.properties"/> <!-- 启动时延期3秒开始任务 --> <property name="startupDelay" value="3"/> <!-- 保存Job数据到数据库所需的数据源 --> <property name="dataSource" ref="dataSource"/> <!-- Job接受applicationContext的成员变量名 --> <property name="applicationContextSchedulerContextKey" value="applicationContext"/> <property name="overwriteExistingJobs" value="true"/>
</bean> 配置中使用的 dataSource 数据源,需要提前配置;quartz.properties 属性文件自行配置。集群定时任务的任务会序列化后储存至数据库,在某机器 crash 后,可以快速的切换到新的机器去运行,并且保证有仅只有一台机器运行计划任务。 先写 QuartzRunnable 文件,这个文件我用来启动 Quartz 定时任务。 import org.quartz.*; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.context.ApplicationContext; /** * Created by ixiaozhi on 16/6/27. */ public class QuartzRunnable { private static final Logger logger = LoggerFactory.getLogger(QuartzRunnable.class); private ApplicationContext context; /** * 构造函数, 传入 applicationContext * * @param context */ public QuartzRunnable(ApplicationContext context) { this.context = context; } public void work() throws SchedulerException { logger.info("quartz is running ..."); // scheduler 对象 Scheduler schedulerCluster = (Scheduler) context.getBean("clusterQuartzScheduler"); Scheduler schedulerLocal = (Scheduler) context.getBean("localQuartzScheduler"); List<ScheduleJob> allQuartzJobs = ......; // 从数据库或者配置文件或者其他任何地方取得 Quartz 任务的配置文件, ScheduleJob 对象为自定义的 Quartz 任务设置,对象的属性见下文 // 启动定时任务 for (ScheduleJob job : allQuartzJobs) { // 区分本机运行或集群运行 Scheduler scheduler; if (job.getIsCluster() == 1) { scheduler = schedulerCluster; } else { scheduler = schedulerLocal; } TriggerKey triggerKey = TriggerKey.triggerKey(job.getJobName(), job.getJobGroup()); CronTrigger trigger = (CronTrigger) scheduler.getTrigger(triggerKey); //不存在,创建一个 if (null == trigger) { JobDetail jobDetail = JobBuilder.newJob(MyDetailQuartzJobBean.class).withIdentity(job.getJobName(), job.getJobGroup()).build(); JobDataMap dataMap = jobDetail.getJobDataMap(); dataMap.put("scheduleJob", job); // 传递 job 对象至执行的方法体 //表达式调度构建器 CronScheduleBuilder scheduleBuilder = CronScheduleBuilder.cronSchedule(job.getCronExpression()); //按新的cronExpression表达式构建一个新的trigger trigger = TriggerBuilder.newTrigger().withIdentity(job.getJobName(), job.getJobGroup()).withSchedule(scheduleBuilder).withDescription(job.getDescription()).build(); scheduler.scheduleJob(jobDetail, trigger); } else { // Trigger已存在,那么更新相应的定时设置 //表达式调度构建器 CronScheduleBuilder scheduleBuilder = CronScheduleBuilder.cronSchedule(job.getCronExpression()); //按新的cronExpression表达式重新构建trigger trigger = trigger.getTriggerBuilder().withIdentity(triggerKey).withSchedule(scheduleBuilder).build(); //按新的trigger重新设置job执行 scheduler.rescheduleJob(triggerKey, trigger); } } } } 从 Spring ApplicationContext 上下文对象中取得本文上面配置的两个 Quartz Scheduler,分别用于启动本机的定时任务与集群定时任务。再根据自己的配置,构造对应的 Trigger 与 Job,加入不同的计划任务中执行。 对于计划任务,都使用同一个类 MyDetailQuartzJobBean 进行启动。在配置中可以根据配置反射启动相应的方法。 我的 ScheduleJob 计划任务配置的属性有以下。 targetObject 为 Spring 中注入的 bean 名称, targetMethod 用于定时任务启动的方法入口。 public class ScheduleJob implements Serializable { private static final long serialVersionUID = -4166311089940333025L; private String jobId; // 任务 ID private String jobName; // 任务名称 private String jobGroup; // 任务分组 private String cronExpression; // 时间表达式 private String description; // 任务描述 private String targetObject; // Spring 注入的类名 private String targetMethod; // 方法 private int isCluster;// 是否集群运行 // getter and setter ... ... } 比如测试的 ScheduleJob 对象: jobId=1; jobName=”Test”; jobGroup=”DEFAULT”; cronExpression=”1/30 * * * * ?”; // 从1秒开始,每30秒执行一次 description=”测试任务”; targetObject=”testService”; targetMethod=”quartzTest”; isCluster=true; 含义为,将从 1 秒开始,每 30 秒在集群中的某一台机器运行一次,从 targetObject 的注入对象中的 targetMethod 方法。 现在来实现 JobBean 计划任务执行类,我命名为 MyDetailQuartzJobBean.java。 import org.quartz.*; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.scheduling.quartz.QuartzJobBean; import java.lang.reflect.Method; /** * 动态运行方法 */ //@PersistJobDataAfterExecution //@DisallowConcurrentExecution //确保多个任务不会同时运行 public class MyDetailQuartzJobBean extends QuartzJobBean { private static final Logger logger = LoggerFactory.getLogger(MyDetailQuartzJobBean.class); private ScheduleJob scheduleJob; protected void executeInternal(JobExecutionContext context) throws JobExecutionException { try { Object targetObject = ApplicationContextUtil.getBean(scheduleJob.getTargetObject()); Method m; try { m = targetObject.getClass().getMethod(scheduleJob.getTargetMethod(), new Class[]{}); m.invoke(targetObject, new Object[]{}); } catch (SecurityException e) { logger.error(e.getMessage(), e); } catch (NoSuchMethodException e) { logger.error(e.getMessage(), e); } } catch (Exception e) { logger.error(e.getMessage(), e); throw new JobExecutionException(e); } } public void setScheduleJob(ScheduleJob scheduleJob) { this.scheduleJob = scheduleJob; } } JobBean 需要继承至 QuartzJobBean,并重写 executeInternal 方法。而且,Quartz 集群中运行的 QuartzJobBean 必须实现序列化。但是,applicationContext 并不支持序列化,在这里面直接注入对象会报 exception 且无法使用。 因此我使用静态化来保存 applicationContext 对象,实现类 ApplicationContextUtil 。 @Component public class ApplicationContextUtil implements ApplicationContextAware { private static ApplicationContext applicationContext; // Spring应用上下文环境 public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { ApplicationContextUtil.applicationContext = applicationContext; } public static ApplicationContext getApplicationContext() { return applicationContext; } public static Object getBean(String beanName) { return applicationContext.getBean(beanName); } @SuppressWarnings("unchecked") public static <T> T getBeanDetail(String beanName) throws BeansException { return (T) applicationContext.getBean(beanName); } } Spring 配置文件中注册该 Bean。 <bean id="applicationContextUtil" class="com.ixiaozhi.util.ApplicationContextUtil"/> 利用 getBean 可直接从 Spring 上下文中取得注入的对象,如上述的 MyDetailQuartzJobBean 使用该方法绕过 Spring ApplicationContext 无法序列化的问题,且取得 Spring Bean 并反射调用其中的方法。 测试 /** * 程序入口 * * @param args */ public void test() { // 初始化 Spring ApplicationContext applicationContext = new ClassPathXmlApplicationContext("/applicationContext.xml"); // 启动定时任务 QuartzRunnable quartz = new QuartzRunnable(applicationContext); quartz.work(); } 其他更好的实现方案,欢迎留言一起探讨。 参考 Spring+quartz 实现动态管理任务 Quartz应用与集群原理分析 Spring与Quartz Cluster备忘CORS JavaScript 跨域方案 Java 实现2016-05-12T00:00:00+08:002016-05-12T00:00:00+08:00https://j.ixiaozhi.com/javascript-cors-implemented-in-java-web<h2 id="jsonp-方案">JSONP 方案</h2>
<h3 id="实现">实现</h3>
<p>在 Controller 中,直接返回 com.fasterxml.jackson.databind.util.JSONPObject 对象,前端代码中便可使用 JSONP 的方案接受。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@RequestMapping(value = "/test")
@ResponseBody
public Object test(@RequestParam(required = false,name = "callback") String callback) {
// 程序需要返回的数据
Map<String,String> result = new HashMap<>();
result.put("result","返回的结果");
if (callback == null || "".equals(callback)) {
return result; // 非 JSONP 请求,返回正常的 JOSN 数据
} else {
return new JSONPObject(callback, result); // JSONP 请求,返回 JOSNP 数据
}
}
</code></pre></div></div>
<h3 id="缺点">缺点</h3>
<p>JSONP 仅能使用 GET 请求的方式。 对于 RESTful 的 API 来说,发送 POST/PUT/DELET 请求将成为问题,不利于接口的统一。</p>
<h2 id="共享-cors-方案">共享 CORS 方案</h2>
<h3 id="实现-1">实现</h3>
<p>CORS需要浏览器和服务器同时支持。目前,所有浏览器都支持该功能,IE浏览器不能低于IE10。
前端代理无需作处理,后端代码中,Header 头中添加 <code class="highlighter-rouge">Access-Control-Allow-Origin: https://static.ixiaozhi.com</code> 来告知浏览器允许的域名。</p>
<p>CORS 有简单与非简单请求,其中区别可见参考资料中的阮一峰的网络日志。</p>
<p>Java 服务器端,我们使用 Filter 来批量添加相关的 Header 头信息。</p>
<p>CORSFilter 类代码:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">package</span> <span class="n">com</span><span class="p">.</span><span class="n">ixiaozhi</span><span class="p">.</span><span class="n">filter</span><span class="p">;</span><span class="err">
</span>
<span class="n">import</span> <span class="n">org</span><span class="p">.</span><span class="n">springframework</span><span class="p">.</span><span class="n">web</span><span class="p">.</span><span class="n">filter</span><span class="p">.</span><span class="n">OncePerRequestFilter</span><span class="p">;</span>
<span class="n">import</span> <span class="n">javax</span><span class="p">.</span><span class="n">servlet</span><span class="p">.</span><span class="n">FilterChain</span><span class="p">;</span>
<span class="err">
</span><span class="n">import</span> <span class="n">javax</span><span class="p">.</span><span class="n">servlet</span><span class="p">.</span><span class="n">ServletException</span><span class="p">;</span>
<span class="err">
</span><span class="n">import</span> <span class="n">javax</span><span class="p">.</span><span class="n">servlet</span><span class="p">.</span><span class="n">http</span><span class="p">.</span><span class="n">HttpServletRequest</span><span class="p">;</span>
<span class="err">
</span><span class="n">import</span> <span class="n">javax</span><span class="p">.</span><span class="n">servlet</span><span class="p">.</span><span class="n">http</span><span class="p">.</span><span class="n">HttpServletResponse</span><span class="p">;</span>
<span class="n">import</span> <span class="n">java</span><span class="p">.</span><span class="n">io</span><span class="p">.</span><span class="n">IOException</span><span class="p">;</span>
<span class="err">
</span><span class="p">/**</span><span class="err">
</span>
<span class="p">*</span> <span class="n">Created</span> <span class="n">by</span> <span class="n">ixiaozhi</span><span class="err">
</span>
<span class="p">*/</span>
<span class="k">public</span> <span class="n">class</span> <span class="n">CORSFilter</span> <span class="n">extends</span> <span class="n">OncePerRequestFilter</span> <span class="p">{</span>
<span class="p">@</span><span class="n">Override</span>
<span class="n">protected</span> <span class="n">void</span> <span class="n">doFilterInternal</span><span class="p">(</span><span class="n">HttpServletRequest</span> <span class="n">request</span><span class="p">,</span> <span class="n">HttpServletResponse</span> <span class="n">response</span><span class="p">,</span> <span class="n">FilterChain</span> <span class="n">filterChain</span><span class="p">)</span> <span class="n">throws</span> <span class="n">ServletException</span><span class="p">,</span> <span class="n">IOException</span> <span class="p">{</span>
<span class="p">//</span> <span class="n">CORS</span> <span class="err">的域名白名单,不支持正规,允许所有可以用</span> <span class="p">*</span>
<span class="n">response</span><span class="p">.</span><span class="n">addHeader</span><span class="p">(</span><span class="s2">"Access-Control-Allow-Origin"</span><span class="p">,</span> <span class="s2">"https://static.ixiaozhi.com"</span><span class="p">);</span><span class="err">
</span>
<span class="p">//</span> <span class="err">对于非简单请求,浏览器会自动发送一个</span> <span class="k">OPTIONS</span> <span class="err">请求,利用</span> <span class="n">Header</span> <span class="err">来告知浏览器可以使用的请求方式及</span> <span class="n">Header</span> <span class="err">的类型</span>
<span class="k">if</span> <span class="p">(</span><span class="n">request</span><span class="p">.</span><span class="n">getHeader</span><span class="p">(</span><span class="s2">"Access-Control-Request-Method"</span><span class="p">)</span> <span class="c1">!= null && "OPTIONS".equals(request.getMethod())) {
</span> <span class="err">
</span> <span class="n">response</span><span class="p">.</span><span class="n">addHeader</span><span class="p">(</span><span class="s2">"Access-Control-Allow-Methods"</span><span class="p">,</span> <span class="s2">"GET, POST, PUT, DELETE"</span><span class="p">);</span>
<span class="err">
</span> <span class="n">response</span><span class="p">.</span><span class="n">addHeader</span><span class="p">(</span><span class="s2">"Access-Control-Allow-Headers"</span><span class="p">,</span> <span class="s2">"Content-Type"</span><span class="p">);</span>
<span class="err">
</span> <span class="n">response</span><span class="p">.</span><span class="n">addHeader</span><span class="p">(</span><span class="s2">"Access-Control-Max-Age"</span><span class="p">,</span> <span class="s2">"1"</span><span class="p">);</span>
<span class="err">
</span> <span class="p">}</span><span class="err">
</span>
<span class="n">filterChain</span><span class="p">.</span><span class="n">doFilter</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="n">response</span><span class="p">);</span>
<span class="p">}</span>
<span class="err">
</span><span class="p">}</span>
</code></pre></div></div>
<p>该过滤器在 <code class="highlighter-rouge">web.xml</code> 中配置:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<filter>
<filter-name>cors</filter-name>
<filter-class>com.ixiaozhi.filter.CORSFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>cors</filter-name>
<url-pattern>/*</url-pattern><!--需要允许CORS跨域的地址-->
</filter-mapping>
</code></pre></div></div>
<p>其他,前端的代码使用正常的 Ajax 调用方式就可以,无需关心跨域问题。</p>
<h2 id="参考资料">参考资料</h2>
<ul>
<li><a href="http://www.ruanyifeng.com/blog/2016/04/cors.html?hmsr=toutiao.io&utm_medium=toutiao.io&utm_source=toutiao.io">跨域资源共享 CORS 详解</a></li>
</ul>ixiaozhiblog@ixiaozhi.comJSONP 方案 实现 在 Controller 中,直接返回 com.fasterxml.jackson.databind.util.JSONPObject 对象,前端代码中便可使用 JSONP 的方案接受。 @RequestMapping(value = "/test")
@ResponseBody public Object test(@RequestParam(required = false,name = "callback") String callback) { // 程序需要返回的数据 Map<String,String> result = new HashMap<>(); result.put("result","返回的结果");
if (callback == null || "".equals(callback)) {
return result; // 非 JSONP 请求,返回正常的 JOSN 数据
} else {
return new JSONPObject(callback, result); // JSONP 请求,返回 JOSNP 数据
}
} 缺点 JSONP 仅能使用 GET 请求的方式。 对于 RESTful 的 API 来说,发送 POST/PUT/DELET 请求将成为问题,不利于接口的统一。 共享 CORS 方案 实现 CORS需要浏览器和服务器同时支持。目前,所有浏览器都支持该功能,IE浏览器不能低于IE10。 前端代理无需作处理,后端代码中,Header 头中添加 Access-Control-Allow-Origin: https://static.ixiaozhi.com 来告知浏览器允许的域名。 CORS 有简单与非简单请求,其中区别可见参考资料中的阮一峰的网络日志。 Java 服务器端,我们使用 Filter 来批量添加相关的 Header 头信息。 CORSFilter 类代码: package com.ixiaozhi.filter;
import org.springframework.web.filter.OncePerRequestFilter; import javax.servlet.FilterChain;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse; import java.io.IOException;
/**
* Created by ixiaozhi
*/ public class CORSFilter extends OncePerRequestFilter { @Override protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) throws ServletException, IOException { // CORS 的域名白名单,不支持正规,允许所有可以用 * response.addHeader("Access-Control-Allow-Origin", "https://static.ixiaozhi.com");
// 对于非简单请求,浏览器会自动发送一个 OPTIONS 请求,利用 Header 来告知浏览器可以使用的请求方式及 Header 的类型 if (request.getHeader("Access-Control-Request-Method") != null && "OPTIONS".equals(request.getMethod())) {
response.addHeader("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE");
response.addHeader("Access-Control-Allow-Headers", "Content-Type");
response.addHeader("Access-Control-Max-Age", "1");
}
filterChain.doFilter(request, response); }
} 该过滤器在 web.xml 中配置:
<filter> <filter-name>cors</filter-name> <filter-class>com.ixiaozhi.filter.CORSFilter</filter-class>
</filter>
<filter-mapping> <filter-name>cors</filter-name> <url-pattern>/*</url-pattern><!--需要允许CORS跨域的地址-->
</filter-mapping> 其他,前端的代码使用正常的 Ajax 调用方式就可以,无需关心跨域问题。 参考资料 跨域资源共享 CORS 详解Gradle 打包实现生产环境与测试环境配置分离2016-05-12T00:00:00+08:002016-05-12T00:00:00+08:00https://j.ixiaozhi.com/java-gradle-archive-different-profile<h1 id="gradle-打包实现生产环境与测试环境配置分离">Gradle 打包实现生产环境与测试环境配置分离</h1>
<p>前篇:<a href="https://ixiaozhi.com/java-maven-archive-different-profile/">Maven 打包实现生产环境与测试环境配置分离</a></p>
<p>前篇是使用 Maven 进行的包管理,这次我们使用 Gradle 进行 Java Web Server 的包管理的配置。</p>
<h2 id="配置-gradle-配置文件">配置 Gradle 配置文件</h2>
<p>build.gradle 中配置相关的 resources 配置文件的目录。不同的资源文件放置在 <code class="highlighter-rouge">src/main/filters/$env</code> 目录下,其中 $env 目录为环境名,例如:dev、test、product 等等。且定义了默认环境为 dev 环境。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>def env = System.getProperty("profile") ?: "dev"
sourceSets {
main {
resources {
srcDirs = ["src/main/resources", "src/main/filters/$env"]
}
}
}
</code></pre></div></div>
<p>把不同环境的 properties 的文件,分别放在 filters 目录下的不同的环境文件中,如下图。</p>
<p><img src="https://raw.githubusercontent.com/zhoujiajun88/zhoujiajun88.github.io/master/images/2016/gradle-filters-properties.png" alt="alt" /></p>
<p>在使用 Gradle 编译的时候,添加参数 <code class="highlighter-rouge">-Dprofile=dev</code> 来指定编译的最终代码为何环境。如:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code># 把程序编译成生产环境
./gradlew bootRepackage -Dprofile=product
</code></pre></div></div>
<h2 id="使用-intellij-idea-启动不同的-gradle-环境">使用 Intellij IDEA 启动不同的 Gradle 环境</h2>
<p>这里的方式同本文前篇所讲述的方式,可以直接参见 maven 的使用方式。</p>ixiaozhiblog@ixiaozhi.comGradle 打包实现生产环境与测试环境配置分离 前篇:Maven 打包实现生产环境与测试环境配置分离 前篇是使用 Maven 进行的包管理,这次我们使用 Gradle 进行 Java Web Server 的包管理的配置。 配置 Gradle 配置文件 build.gradle 中配置相关的 resources 配置文件的目录。不同的资源文件放置在 src/main/filters/$env 目录下,其中 $env 目录为环境名,例如:dev、test、product 等等。且定义了默认环境为 dev 环境。 def env = System.getProperty("profile") ?: "dev" sourceSets { main { resources { srcDirs = ["src/main/resources", "src/main/filters/$env"] } } } 把不同环境的 properties 的文件,分别放在 filters 目录下的不同的环境文件中,如下图。 在使用 Gradle 编译的时候,添加参数 -Dprofile=dev 来指定编译的最终代码为何环境。如: # 把程序编译成生产环境 ./gradlew bootRepackage -Dprofile=product 使用 Intellij IDEA 启动不同的 Gradle 环境 这里的方式同本文前篇所讲述的方式,可以直接参见 maven 的使用方式。Spring Cache use Guava2016-04-13T00:00:00+08:002016-04-13T00:00:00+08:00https://j.ixiaozhi.com/spring-cache-use-guava<p>前篇:<a href="https://ixiaozhi.com/spring-cache-annotation-useage/">Spring Cache 缓存注解实现</a></p>
<h2 id="需求">需求</h2>
<p>现在缓存需要指定的过期策略,比如某段时间后过期或者多久没有使用过后过期。</p>
<h2 id="开始">开始</h2>
<p>同样的,spring 配置中文件中,需要启用 cache 相关的注解。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><cache:annotation-driven/>
</code></pre></div></div>
<p>TestCache 示例:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>public class TestCache {
@Cacheable(value = "testCache")
public String get(String name) {
return System.currentTimeMillis() + "";
}
}
</code></pre></div></div>
<p>并注册该缓存的 Bean。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><bean id="testCache" class="com.ixiaozhi.TestCache"/>
</code></pre></div></div>
<p>对缓存管理的设置,改用 GuavaCache 并把配置移到 class 中进行配置。新建一个 CacheConfig 文件,并注册。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Configuration
@EnableCaching
public class CacheConfig {
@Bean
public CacheManager cacheManager() {
SimpleCacheManager cacheManager = new SimpleCacheManager();
// 设置 testCache 的缓存有效期为创建后的1天
GuavaCache testCache = new GuavaCache("testCache", CacheBuilder.newBuilder().expireAfterWrite(1, TimeUnit.DAYS).build());
// 设置 testCache 的缓存有效期为访问过后的 1 个小时
// GuavaCache testCache = new GuavaCache("testCache", CacheBuilder.newBuilder().expireAfterAccess(1, TimeUnit.HOURS).build());
// 把缓存加入 cacheManager
cacheManager.setCaches(Arrays.asList(testCache));
return cacheManager;
}
</code></pre></div></div>
<p>GuavaCache 中可以设置像创建后固定时间过期或者是闲置固定时间后过期等缓存的策略,并可以对单个缓存类进行单独的配置。</p>ixiaozhiblog@ixiaozhi.com前篇:Spring Cache 缓存注解实现 需求 现在缓存需要指定的过期策略,比如某段时间后过期或者多久没有使用过后过期。 开始 同样的,spring 配置中文件中,需要启用 cache 相关的注解。 <cache:annotation-driven/> TestCache 示例: public class TestCache {
@Cacheable(value = "testCache")
public String get(String name) {
return System.currentTimeMillis() + "";
}
} 并注册该缓存的 Bean。 <bean id="testCache" class="com.ixiaozhi.TestCache"/> 对缓存管理的设置,改用 GuavaCache 并把配置移到 class 中进行配置。新建一个 CacheConfig 文件,并注册。 @Configuration
@EnableCaching
public class CacheConfig {
@Bean
public CacheManager cacheManager() {
SimpleCacheManager cacheManager = new SimpleCacheManager(); // 设置 testCache 的缓存有效期为创建后的1天
GuavaCache testCache = new GuavaCache("testCache", CacheBuilder.newBuilder().expireAfterWrite(1, TimeUnit.DAYS).build()); // 设置 testCache 的缓存有效期为访问过后的 1 个小时 // GuavaCache testCache = new GuavaCache("testCache", CacheBuilder.newBuilder().expireAfterAccess(1, TimeUnit.HOURS).build());
// 把缓存加入 cacheManager cacheManager.setCaches(Arrays.asList(testCache));
return cacheManager;
} GuavaCache 中可以设置像创建后固定时间过期或者是闲置固定时间后过期等缓存的策略,并可以对单个缓存类进行单独的配置。LetsEncrypt SSL 证书签发(Nginx)2016-04-09T00:00:00+08:002016-04-09T00:00:00+08:00https://j.ixiaozhi.com/lets-encrypt-ssl-use-on-nginx<h2 id="概述">概述</h2>
<p>官方文档参考: <a href="https://letsencrypt.org/getting-started/">let’s encrypt getting started</a></p>
<p>域名认证过程有自动认证与手动认证,自动认证会启动一个监听 80 端口的程序来完成自动认证。手动认证使用参数 <code class="highlighter-rouge">--webroot</code> 来进行使用网站访问手动认证,认证时,会访问网址 <code class="highlighter-rouge">/.well-known/acme-challenge/xxxxxxxx</code>。</p>
<p>每次签发的证书有 90 天的有效期,所以我们还得在每个月去重新签发一个新的证书。</p>
<p>本文的操作是基于系统 CentOS 6.7 操作进行。</p>
<h2 id="准备工作">准备工作</h2>
<p>在取得官方代码前,得先查看系统环境中是否安装全所需要的工具软件。</p>
<ul>
<li>Git</li>
</ul>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>yum -y install git
</code></pre></div></div>
<ul>
<li>python 2.7 检查</li>
</ul>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>/usr/bin/python -V #查看版本
</code></pre></div></div>
<ul>
<li>安装编译需要的工具</li>
</ul>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>yum install zlib-devel bzip2-devel openssl-devel xz-libs wget xz
</code></pre></div></div>
<ul>
<li>安装 Python2.7.8</li>
</ul>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>wget http://www.python.org/ftp/python/2.7.8/Python-2.7.8.tar.xz
xz -d Python-2.7.8.tar #下载源码
tar -xvf Python-2.7.8.tar #解压
cd Python-2.7.8 #进入目录
./configure --prefix=/usr/local #运行配置
make
make altinstall #编译及安装
python2.7 -V #检查版本
export PATH="/usr/local/bin:$PATH"
cd ../
</code></pre></div></div>
<ul>
<li>安装 pip 及 virtualenv</li>
</ul>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>wget --no-check-certificate https://pypi.python.org/packages/source/s/setuptools/setuptools-1.4.2.tar.gz #下载源码
tar -xvf setuptools-1.4.2.tar.gz #解压
cd setuptools-1.4.2
python2.7 setup.py install #用 Python2.7.8安装setuptools
cd ../
curl https://bootstrap.pypa.io/get-pip.py | python2.7 - #安装pip
pip2.7 install virtualenv #安装virtualenv
</code></pre></div></div>
<h2 id="域名认证并生成证书">域名认证并生成证书</h2>
<ul>
<li>从官方 Git 库中取得代码。</li>
</ul>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git clone https://github.com/letsencrypt/letsencrypt
cd letsencrypt
</code></pre></div></div>
<ul>
<li>自动认证 standalone 模式</li>
</ul>
<p>运行认证程序。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>service nginx stop #停止 Nginx 服务器
./letsencrypt-auto certonly --standalone -d ixiaozhi.com -d www.ixiaozhi.com
</code></pre></div></div>
<p>letsencrypt-auto 按照提示输入 E-mail 和域名即可。在运行认证程序前,要先停用 nginx,因为接下来的环节需要占用80等端口。之后证书会生成到 <code class="highlighter-rouge">/etc/letsencrypt/live/ixiaozhi.com/</code> 下,其中的 ixiaozhi.com 改为自己的域名。</p>
<ul>
<li>手动认证 webroot 模式</li>
</ul>
<p>Nginx 添加目录访问:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>location /.well-known/acme-challenge/ {
default_type text/plain;
root /home/ixiaozhi/acme-challenge/;
}
</code></pre></div></div>
<p>添加目录,并重启服务:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cd /home/ixiaozhi/
mkdir acme-challenge
service nginx restart
</code></pre></div></div>
<p>进行认证并生成证书:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>service nginx reload
./letsencrypt-auto certonly --webroot -w /home/ixiaozhi/acme-challenge -d ixiaozhi.com -d www.ixiaozhi.com
</code></pre></div></div>
<p>认真阅读输出信息,输入邮箱且同意协议后,成功后会输出:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at
/etc/letsencrypt/live/ixiaozhi.com/fullchain.pem. Your cert will
expire on 2016-05-29. To obtain a new version of the certificate in
the future, simply run Let's Encrypt again.
- If you like Let's Encrypt, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le
</code></pre></div></div>
<p>看到 Congratulations 我们就放心了。之后证书会生成到 <code class="highlighter-rouge">/etc/letsencrypt/live/ixiaozhi.com/</code> 下。</p>
<h2 id="配置-nginx">配置 Nginx</h2>
<p>修改 Nginx 的 nginx.conf,添加配置 ssl。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>listen 80;
listen 443 ssl;
ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
server_name ixiaozhi.com;
ssl_certificate /etc/letsencrypt/live/ixiaozhi.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ixiaozhi.com/privkey.pem;
</code></pre></div></div>
<p>重启 nginx 即可。</p>
<h2 id="定时任务定时签发证书">定时任务定时签发证书</h2>
<p>我们可以先测试一个 renew 证书是否可以成功。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./letsencrypt-auto renew --email admin@ixiaozhi.com --dry-run --agree-tos
</code></pre></div></div>
<p>当看到 <code class="highlighter-rouge">Congratulations, all renewals succeeded. The following certs have been renewed</code> 时,测试就是通过的。使用 <code class="highlighter-rouge">--dry-run</code> 参数来测试并不会保存任何证书。</p>
<p>如果需要自动更新,先使用“crontab -e”,选择编辑器后,在最底部加入</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>0 0 1 * * ./letsencrypt-auto renew --email admin@ixiaozhi.com --agree-tos --force-renewal
</code></pre></div></div>
<p>crontab 的时间格式为:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>* * * * * command
分 时 日 月 周 命令
</code></pre></div></div>
<p>添加成功后,可以使用 <code class="highlighter-rouge">crontab -l</code> 查看当前用户的定时任务,确认是否已经生效。</p>
<h2 id="转发设置">转发设置</h2>
<p>如果希望访问 http 都跳转至 https 进行访问,可以通过两种方法进行转发。(如果是使用 –webroot 进行认证的,在 nginx 设置中要把 <code class="highlighter-rouge">/.well-known/acme-challenge/</code> 例外不进行转发)</p>
<p>一个是直接利用 nginx 进行转发。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>server {
listen 80;
server_name ixiaozhi.com;
return https://ixiaozhi.com$request_uri;
...
</code></pre></div></div>
<p>一个是设置HSTS。</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>server {
add_header Strict-Transport-Security "max-age=63072000;includeSubdomains; preload";
#添加一行
...
</code></pre></div></div>
<p>别忘了上述设置都需要重启 Nginx。</p>
<h2 id="参考资料">参考资料</h2>
<ul>
<li><a href="https://letsencrypt.org/getting-started/">let’s encrypt getting started</a></li>
<li><a href="https://github.com/letsencrypt/letsencrypt">let’s encrypt github</a></li>
<li><a href="https://mitnk.com/2015/11/lets_encrypt/">Nginx 安装 Let’s Encrypt 免费 SSL 证书</a></li>
<li><a href="http://www.tennfy.com/3911.html">免费 SSL 证书 Let’s Encrypt 安装使用教程</a></li>
<li><a href="https://yuan.ga/ghost-blogqi-yong-letsencrypt-sslzheng-shu/">Ghost Blog 启用 HTTPS 使用 Let’s Encrypt SSL 证书</a></li>
<li><a href="https://undefinedblog.com/lets-encrypt/">Let’s Encrypt 花三分钟免费接入 SSL 证书</a></li>
</ul>ixiaozhiblog@ixiaozhi.com概述 官方文档参考: let’s encrypt getting started 域名认证过程有自动认证与手动认证,自动认证会启动一个监听 80 端口的程序来完成自动认证。手动认证使用参数 --webroot 来进行使用网站访问手动认证,认证时,会访问网址 /.well-known/acme-challenge/xxxxxxxx。 每次签发的证书有 90 天的有效期,所以我们还得在每个月去重新签发一个新的证书。 本文的操作是基于系统 CentOS 6.7 操作进行。 准备工作 在取得官方代码前,得先查看系统环境中是否安装全所需要的工具软件。 Git yum -y install git python 2.7 检查 /usr/bin/python -V #查看版本 安装编译需要的工具 yum install zlib-devel bzip2-devel openssl-devel xz-libs wget xz 安装 Python2.7.8 wget http://www.python.org/ftp/python/2.7.8/Python-2.7.8.tar.xz xz -d Python-2.7.8.tar #下载源码 tar -xvf Python-2.7.8.tar #解压 cd Python-2.7.8 #进入目录 ./configure --prefix=/usr/local #运行配置 make make altinstall #编译及安装 python2.7 -V #检查版本 export PATH="/usr/local/bin:$PATH" cd ../ 安装 pip 及 virtualenv wget --no-check-certificate https://pypi.python.org/packages/source/s/setuptools/setuptools-1.4.2.tar.gz #下载源码 tar -xvf setuptools-1.4.2.tar.gz #解压 cd setuptools-1.4.2 python2.7 setup.py install #用 Python2.7.8安装setuptools cd ../ curl https://bootstrap.pypa.io/get-pip.py | python2.7 - #安装pip pip2.7 install virtualenv #安装virtualenv 域名认证并生成证书 从官方 Git 库中取得代码。 git clone https://github.com/letsencrypt/letsencrypt cd letsencrypt 自动认证 standalone 模式 运行认证程序。 service nginx stop #停止 Nginx 服务器 ./letsencrypt-auto certonly --standalone -d ixiaozhi.com -d www.ixiaozhi.com letsencrypt-auto 按照提示输入 E-mail 和域名即可。在运行认证程序前,要先停用 nginx,因为接下来的环节需要占用80等端口。之后证书会生成到 /etc/letsencrypt/live/ixiaozhi.com/ 下,其中的 ixiaozhi.com 改为自己的域名。 手动认证 webroot 模式 Nginx 添加目录访问: location /.well-known/acme-challenge/ { default_type text/plain; root /home/ixiaozhi/acme-challenge/; } 添加目录,并重启服务: cd /home/ixiaozhi/ mkdir acme-challenge service nginx restart 进行认证并生成证书: service nginx reload ./letsencrypt-auto certonly --webroot -w /home/ixiaozhi/acme-challenge -d ixiaozhi.com -d www.ixiaozhi.com 认真阅读输出信息,输入邮箱且同意协议后,成功后会输出: IMPORTANT NOTES: - Congratulations! Your certificate and chain have been saved at /etc/letsencrypt/live/ixiaozhi.com/fullchain.pem. Your cert will expire on 2016-05-29. To obtain a new version of the certificate in the future, simply run Let's Encrypt again. - If you like Let's Encrypt, please consider supporting our work by: Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate Donating to EFF: https://eff.org/donate-le 看到 Congratulations 我们就放心了。之后证书会生成到 /etc/letsencrypt/live/ixiaozhi.com/ 下。 配置 Nginx 修改 Nginx 的 nginx.conf,添加配置 ssl。 listen 80; listen 443 ssl; ssl_protocols TLSv1.2 TLSv1.1 TLSv1; server_name ixiaozhi.com; ssl_certificate /etc/letsencrypt/live/ixiaozhi.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/ixiaozhi.com/privkey.pem; 重启 nginx 即可。 定时任务定时签发证书 我们可以先测试一个 renew 证书是否可以成功。 ./letsencrypt-auto renew --email admin@ixiaozhi.com --dry-run --agree-tos 当看到 Congratulations, all renewals succeeded. The following certs have been renewed 时,测试就是通过的。使用 --dry-run 参数来测试并不会保存任何证书。 如果需要自动更新,先使用“crontab -e”,选择编辑器后,在最底部加入 0 0 1 * * ./letsencrypt-auto renew --email admin@ixiaozhi.com --agree-tos --force-renewal crontab 的时间格式为: * * * * * command 分 时 日 月 周 命令 添加成功后,可以使用 crontab -l 查看当前用户的定时任务,确认是否已经生效。 转发设置 如果希望访问 http 都跳转至 https 进行访问,可以通过两种方法进行转发。(如果是使用 –webroot 进行认证的,在 nginx 设置中要把 /.well-known/acme-challenge/ 例外不进行转发) 一个是直接利用 nginx 进行转发。 server { listen 80; server_name ixiaozhi.com; return https://ixiaozhi.com$request_uri; ... 一个是设置HSTS。 server { add_header Strict-Transport-Security "max-age=63072000;includeSubdomains; preload"; #添加一行 ... 别忘了上述设置都需要重启 Nginx。 参考资料 let’s encrypt getting started let’s encrypt github Nginx 安装 Let’s Encrypt 免费 SSL 证书 免费 SSL 证书 Let’s Encrypt 安装使用教程 Ghost Blog 启用 HTTPS 使用 Let’s Encrypt SSL 证书 Let’s Encrypt 花三分钟免费接入 SSL 证书转 使用 Java 8 中的 Stream2016-04-09T00:00:00+08:002016-04-09T00:00:00+08:00https://j.ixiaozhi.com/java-8-steam<p>来源: <a href="http://nkcoder.github.io/2016/01/24/java-8-stream-api/">Think And COde</a></p>
<p>Stream是Java 8 提供的高效操作集合类(Collection)数据的 API。</p>
<h2 id="从-iterator-到-stream">从 Iterator 到 Stream</h2>
<p>有一个字符串的 list,要统计其中长度大于 7 的字符串的数量,用迭代来实现:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>List<String> wordList = Arrays.asList("regular", "expression", "specified", "as", "a",
"string", "must");
int countByIterator = 0;
for (String word: wordList) {
if (word.length() > 7) {
countByIterator++;
}
}
</code></pre></div></div>
<p>用 Stream 实现:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>long countByStream = wordList.stream().filter(w -> w.length() > 7).count();
</code></pre></div></div>
<p>显然,用 stream 实现更简洁,不仅如此,stream 很容易实现并发操作,比如:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>long countByParallelStream = wordList.parallelStream().filter(w -> w.length() > 7).count();
</code></pre></div></div>
<p>stream 遵循的原则是:告诉我做什么,不用管我怎么做。比如上例:告诉 stream 通过多线程统计字符串长度,至于以什么顺序、在哪个线程中执行,由 stream 来负责;而在迭代实现中,由于计算的方式已确定,很难优化了。</p>
<p>Stream 和 Collection 的区别主要有:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>* stream 本身并不存储数据,数据是存储在对应的 collection 里,或者在需要的时候才生成的;
* stream 不会修改数据源,总是返回新的 stream;
* stream 的操作是懒执行 (lazy) 的:仅当最终的结果需要的时候才会执行,比如上面的例子中,结果仅需要前3个长度大于7
的字符串,那么在找到前3个长度符合要求的字符串后,`filter()` 将停止执行;
</code></pre></div></div>
<p>使用stream的步骤如下:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>* 创建stream;
* 通过一个或多个中间操作(intermediate operations)将初始stream转换为另一个stream;
* 通过中止操作(terminal operation)获取结果;该操作触发之前的懒操作的执行,中止操作后,该stream关闭,不能再
使用了;
</code></pre></div></div>
<p>在上面的例子中, <code class="highlighter-rouge">wordList.stream()</code> 和 <code class="highlighter-rouge">wordList.parallelStream()</code> 是创建 stream,<code class="highlighter-rouge">filter()</code> 是中间操作,过滤后生成一个新的 <code class="highlighter-rouge">stream,count()</code> 是中止操作,获取结果。</p>
<h2 id="创建stream的方式">创建Stream的方式</h2>
<p>1) 从 array 或 list 创建 stream:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Stream<Integer> integerStream = Stream.of(10, 20, 30, 40);
String[] cityArr = {"Beijing", "Shanghai", "Chengdu"};
Stream<String> cityStream = Stream.of(cityArr);
Stream<String> nameStream = Arrays.asList("Daniel", "Peter", "Kevin").stream();
Stream<String> cityStream2 = Arrays.stream(cityArr, 0, 1);
Stream<String> emptyStream = Stream.empty();
</code></pre></div></div>
<p>2) 通过 generate 和 iterate 创建无穷 stream:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Stream<String> echos = Stream.generate(() -> "echo");
Stream<Integer> integers = Stream.iterate(0, num -> num + 1);
</code></pre></div></div>
<p>3) 通过其它 API 创建 stream:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Stream<String> lines = Files.lines(Paths.get("test.txt"))
String content = "AXDBDGXC";
Stream<String> contentStream = Pattern.compile("[ABC]{1,3}").splitAsStream(content);
</code></pre></div></div>
<h2 id="stream转换">Stream转换</h2>
<p>1) <code class="highlighter-rouge">filter()</code> 用于过滤,即使原 stream 中满足条件的元素构成新的 stream:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>List<String> langList = Arrays.asList("Java", "Python", "Swift", "HTML");
Stream<String> filterStream = langList.stream().filter(lang -> lang.equalsIgnoreCase("java"));
</code></pre></div></div>
<p>2) map()用于映射,遍历原stream中的元素,转换后构成新的stream:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>List<String> langList = Arrays.asList("Java", "Python", "Swift", "HTML");
Stream<String> mapStream = langList.stream().map(String::toUpperCase);
</code></pre></div></div>
<p>3) <code class="highlighter-rouge">flatMap()</code> 用于将 <code class="highlighter-rouge">[["ABC", "DEF"], ["FGH", "IJK"]]</code> 的形式转换为<code class="highlighter-rouge">["ABC", "DEF", "FGH", "IJK"]</code>:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Stream<String> cityStream = Stream.of("Beijing", "Shanghai", "Shenzhen");
// [['B', 'e', 'i', 'j', 'i', 'n', 'g'], ['S', 'h', 'a', 'n', 'g', 'h', 'a', 'i'], ...]
Stream<Stream<Character>> characterStream1 = cityStream.map(city -> characterStream(city));
Stream<String> cityStreamCopy = Stream.of("Beijing", "Shanghai", "Shenzhen");
// ['B', 'e', 'i', 'j', 'i', 'n', 'g', 'S', 'h', 'a', 'n', 'g', 'h', 'a', 'i', ...]
Stream<Character> characterStreamCopy = cityStreamCopy.flatMap(city -> characterStream(city));
</code></pre></div></div>
<p>其中, <code class="highlighter-rouge">characterStream()</code> 返回有参数字符串的字符构成的Stream;</p>
<p>4) <code class="highlighter-rouge">limit()</code> 表示限制 stream 中元素的数量,<code class="highlighter-rouge">skip()</code> 表示跳过 stream 中前几个元素, concat 表示将多个 stream 连接起来,<code class="highlighter-rouge">peek()</code> 主要用于 debug 时查看 stream 中元素的值:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Stream<Integer> limitStream = Stream.of(18, 20, 12, 35, 89).sorted().limit(3);
Stream<Integer> skipStream = Stream.of(18, 20, 12, 35, 89).sorted(Comparator.reverseOrder())
.skip(1);
Stream<Integer> concatStream = Stream.concat(Stream.of(1, 2, 3), Stream.of(4, 5, 6));
concatStream.peek(i -> System.out.println(i)).count();
</code></pre></div></div>
<p><code class="highlighter-rouge">peek()</code> 是 intermediate operation,所以后面需要一个terminal operation,如 <code class="highlighter-rouge">count()</code> 才能在输出中看到结果;</p>
<p>5) 有状态的 (stateful) 转换,即元素之间有依赖关系,如 <code class="highlighter-rouge">distinct()</code> 返回由唯一元素构成的 stream,<code class="highlighter-rouge">sorted()</code> 返回排序后的 stream:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Stream<String> distinctStream = Stream.of("Beijing", "Tianjin", "Beijing").distinct();
Stream<String> sortedStream = Stream.of("Beijing", "Shanghai", "Chengdu").sorted(Comparator.comparing(String::length).reversed());
</code></pre></div></div>
<h2 id="stream-reduction">Stream reduction</h2>
<p>reduction 就是从 stream 中取出结果,是 terminal operation,因此经过 reduction 后的 stream 不能再使用了。</p>
<h3 id="optional">Optional</h3>
<p>Optional 表示或者有一个T类型的对象,或者没有值;</p>
<p>1) 创建 Optional 对象:</p>
<p>直接通过 Optional 的类方法:<code class="highlighter-rouge">of()/empty()/ofNullable()</code>:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Optional<Integer> intOpt = Optional.of(10);
Optional<String> emptyOpt = Optional.empty();
Optional<Double> doubleOpt = Optional.ofNullable(5.5);
</code></pre></div></div>
<p>2) 使用 Optional 对象:</p>
<p>你当然可以这么使用:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>if (intOpt.isPresent()) {
intOpt.get();
}
</code></pre></div></div>
<p>但是,最好这么使用:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>doubleOpt.orElse(0.0);
doubleOpt.orElseGet(() -> 1.0);
doubleOpt.orElseThrow(RuntimeException::new);
List<Double> doubleList = new ArrayList<>();
doubleOpt.ifPresent(doubleList::add);
map()方法与ifPresent()用法相同,就是多个返回值,flatMap()用于Optional的链式表达:
Optional<Boolean> addOk = doubleOpt.map(doubleList::add);
Optional.of(4.0).flatMap(num -> Optional.ofNullable(num * 100)).flatMap(num -> Optional.ofNullable(Math.sqrt(num)));
</code></pre></div></div>
<h3 id="简单的-reduction">简单的 reduction</h3>
<p>主要包含以下操作: <code class="highlighter-rouge">findFirst()/findAny()/allMatch/anyMatch()/noneMatch</code> ,比如:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Optional<String> firstWord = wordStream.filter(s -> s.startsWith("Y")).findFirst();
Optional<String> anyWord = wordStream.filter(s -> s.length() > 3).findAny();
wordStream.allMatch(s -> s.length() > 3);
wordStream.anyMatch(s -> s.length() > 3);
wordStream.noneMatch(s -> s.length() > 3);
</code></pre></div></div>
<h3 id="reduce-方法">reduce 方法</h3>
<p>1) reduce(accumulator):参数是一个执行双目运算的 Functional Interface,假如这个参数表示的操作为 op,stream中的元素为x, y, z, …,则 <code class="highlighter-rouge">reduce()</code> 执行的就是 x op y op z …,所以要求op这个操作具有结合性 (associative),即满足:(x op y) op z = x op (y op z),满足这个要求的操作主要有:求和、求积、求最大值、求最小值、字符串连接、集合并集和交集等。另外,该函数的返回值是 Optional 的:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Optional<Integer> sum1 = numStream.reduce((x, y) -> x + y);
</code></pre></div></div>
<p>2) reduce(identity, accumulator):可以认为第一个参数为默认值,但需要满足 identity op x = x,所以对于求和操作,identity 的值为0,对于求积操作,identity 的值为1。返回值类型是 stream 元素的类型:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Integer sum2 = numStream.reduce(0, Integer::sum);
</code></pre></div></div>
<h2 id="collect-结果">collect 结果</h2>
<p>1) collect() 方法:</p>
<p><code class="highlighter-rouge">reduce()</code> 和 <code class="highlighter-rouge">collect()</code> 的区别是:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> * reduce()的结果是一个值;
* collect()可以对stream中的元素进行各种处理后,得到stream中元素的值;
</code></pre></div></div>
<p><code class="highlighter-rouge">Collectors</code> 接口提供了很方便的创建 <code class="highlighter-rouge">Collector</code> 对象的工厂方法:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>// collect to Collection
Stream.of("You", "may", "assume").collect(Collectors.toList());
Stream.of("You", "may", "assume").collect(Collectors.toSet());
Stream.of("You", "may", "assume").collect(Collectors.toCollection(TreeSet::new));
// join element
Stream.of("You", "may", "assume").collect(Collectors.joining());
Stream.of("You", "may", "assume").collect(Collectors.joining(", "));
// summarize element
IntSummaryStatistics summary = Stream.of("You", "may", "assume").collect(Collectors.summarizingInt(String::length));
summary.getMax();
</code></pre></div></div>
<p>2) foreach() 方法:</p>
<p><code class="highlighter-rouge">foreach()</code> 用于遍历 stream 中的元素,属于 terminal operation;
<code class="highlighter-rouge">forEachOrdered()</code> 是按照 stream 中元素的顺序遍历,也就无法利用并发的优势;</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Stream.of("You", "may", "assume", "you", "can", "fly").parallel().forEach(w -> System.out.println(w));
Stream.of("You", "may", "assume", "you", "can", "fly").forEachOrdered(w -> System.out.println(w));
</code></pre></div></div>
<p>3) toArray() 方法:</p>
<p>得到由 stream 中的元素得到的数组,默认是 Object[],可以通过参数设置需要结果的类型:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Object[] words1 = Stream.of("You", "may", "assume").toArray();
String[] words2 = Stream.of("You", "may", "assume").toArray(String[]::new);
</code></pre></div></div>
<p>4) toMap() 方法:</p>
<p>toMap: 将 stream 中的元素映射为的形式,两个参数分别用于生成对应的 key 和 value 的值。比如有一个字符串 stream,将首字母作为 key,字符串值作为 value,得到一个 map:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Stream<String> introStream = Stream.of("Get started with UICollectionView and the photo library".split(" "));
Map<String, String> introMap = introStream.collect(Collectors.toMap(s -> s.substring(0, 1), s -> s));
</code></pre></div></div>
<p>如果一个 key 对应多个 value,则会抛出异常,需要使用第三个参数设置如何处理冲突,比如仅使用原来的 value、使用新的 value,或者合并:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Stream<String> introStream = Stream.of("Get started with UICollectionView and the photo library".split(" "));
Map<Integer, String> introMap2 = introStream.collect(Collectors.toMap(s -> s.length(), s -> s, (existingValue, newValue) -> existingValue));
</code></pre></div></div>
<p>如果 value 是一个集合,即将 key 对应的所有 value 放到一个集合中,则需要使用第三个参数,将多个 value 合并:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Stream<String> introStream3 = Stream.of("Get started with UICollectionView and the photo library".split(" "));
Map<Integer, Set<String>> introMap3 = introStream3.collect(Collectors.toMap(s -> s.length(), s -> Collections.singleton(s), (existingValue, newValue) -> {
HashSet<String> set = new HashSet<>(existingValue);
set.addAll(newValue);
return set;
}
));
introMap3.forEach((k, v) -> System.out.println(k + ": " + v));
</code></pre></div></div>
<p>如果 value 是对象自身,则使用 Function.identity(),如:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Map<Integer, Person> idToPerson = people.collect(Collectors.toMap(Person::getId, Function.identity()));
</code></pre></div></div>
<p>toMap() 默认返回的是 HashMap,如果需要其它类型的 map ,比如 TreeMap ,则可以在第四个参数指定构造方法:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Map<Integer, String> introMap2 = introStream.collect(Collectors.toMap(s -> s.length(), s -> s, (existingValue, newValue) -> existingValue, TreeMap::new));
</code></pre></div></div>
<h2 id="grouping-和-partitioning">Grouping 和 Partitioning</h2>
<p>1) <code class="highlighter-rouge">groupingBy()</code> 表示根据某一个字段或条件进行分组,返回一个 Map,其中 key 为分组的字段或条件,value 默认为 list,<code class="highlighter-rouge">groupingByConcurrent()</code> 是其并发版本:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Map<String, List<Locale>> countryToLocaleList = Stream.of(Locale.getAvailableLocales()).collect(Collectors.groupingBy(l -> l.getDisplayCountry()));
</code></pre></div></div>
<p>2) 如果 <code class="highlighter-rouge">groupingBy()</code> 分组的依据是一个 bool 条件,则 key 的值为 true/false ,此时与 <code class="highlighter-rouge">partitioningBy()</code> 等价,且
<code class="highlighter-rouge">partitioningBy()</code> 的效率更高:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>// predicate
Map<Boolean, List<Locale>> englishAndOtherLocales = Stream.of(Locale.getAvailableLocales()).collect(Collectors.groupingBy(l -> l.getDisplayLanguage().equalsIgnoreCase("English")));
// partitioningBy
Map<Boolean, List<Locale>> englishAndOtherLocales2 = Stream.of(Locale.getAvailableLocales()).collect(Collectors.partitioningBy(l -> l.getDisplayLanguage().equalsIgnoreCase("English")));
</code></pre></div></div>
<p>3) <code class="highlighter-rouge">groupingBy()</code> 提供第二个参数,表示 downstream,即对分组后的 value 作进一步的处理:</p>
<p>返回 set,而不是 list:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Map<String, Set<Locale>> countryToLocaleSet = Stream.of(Locale.getAvailableLocales()).collect(Collectors.groupingBy(l -> l.getDisplayCountry(), Collectors.toSet()));
</code></pre></div></div>
<p>返回 value 集合中元素的数量:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Map<String, Long> countryToLocaleCounts = Stream.of(Locale.getAvailableLocales()).collect(Collectors.groupingBy(l -> l.getDisplayCountry(), Collectors.counting()));
</code></pre></div></div>
<p>对 value 集合中的元素求和:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Map<String, Integer> cityToPopulationSum = Stream.of(cities).collect(Collectors.groupingBy(City::getName, Collectors.summingInt(City::getPopulation)));
</code></pre></div></div>
<p>对 value 的某一个字段求最大值,注意 value 是 Optional 的:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Map<String, Optional<City>> cityToPopulationMax = Stream.of(cities).collect(Collectors.groupingBy(City::getName, Collectors.maxBy(Comparator.comparing(City::getPopulation))));
</code></pre></div></div>
<p>使用 mapping 对 value 的字段进行 map 处理:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Map<String, Optional<String>> stateToNameMax = Stream.of(cities).collect(Collectors.groupingBy(City::getState, Collectors.mapping(City::getName, Collectors.maxBy(Comparator.comparing(String::length)))));
Map<String, Set<String>> stateToNameSet = Stream.of(cities).collect(Collectors.groupingBy(City::getState, Collectors.mapping(City::getName, Collectors.toSet())));
</code></pre></div></div>
<p>通过 summarizingXXX 获取统计结果:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Map<String, IntSummaryStatistics> stateToPopulationSummary = Stream.of(cities).collect(Collectors.groupingBy(City::getState, Collectors.summarizingInt(City::getPopulation)));
</code></pre></div></div>
<p><code class="highlighter-rouge">reducing()</code> 可以对结果作更复杂的处理,但是 <code class="highlighter-rouge">reducing()</code> 却并不常用:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Map<String, String> stateToNameJoining = Stream.of(cities).collect(Collectors.groupingBy(City::getState, Collectors.reducing("", City::getName, (s, t) -> s.length() == 0 ? t : s + ", " + t)));
</code></pre></div></div>
<p>比如上例可以通过 mapping 达到同样的效果:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Map<String, String> stateToNameJoining2 = Stream.of(cities).collect(Collectors.groupingBy(City::getState, Collectors.mapping(City::getName, Collectors.joining(", "))));
</code></pre></div></div>
<h2 id="primitive-stream">Primitive Stream</h2>
<p>Stream<Integer> 对应的 Primitive Stream 就是 IntStream,类似的还有 DoubleStream 和 LongStream。</Integer></p>
<p>1) Primitive Stream的构造:<code class="highlighter-rouge">of(), range(), rangeClosed(), Arrays.stream()</code>:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>IntStream intStream = IntStream.of(10, 20, 30);
IntStream zeroToNintyNine = IntStream.range(0, 100);
IntStream zeroToHundred = IntStream.rangeClosed(0, 100);
double[] nums = {10.0, 20.0, 30.0};
DoubleStream doubleStream = Arrays.stream(nums, 0, 3);
</code></pre></div></div>
<p>2) Object Stream 与 Primitive Stream 之间的相互转换,通过mapToXXX() 和 boxed():</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>// map to
Stream<String> cityStream = Stream.of("Beijing", "Tianjin", "Chengdu");
IntStream lengthStream = cityStream.mapToInt(String::length);
// box
Stream<Integer> oneToNine = IntStream.range(0, 10).boxed();
</code></pre></div></div>
<p>3) 与 Object Stream 相比,Primitive Stream 的特点:</p>
<p>toArray() 方法返回的是对应的 Primitive 类型:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>int[] intArr = intStream.toArray();
</code></pre></div></div>
<p>自带统计类型的方法,如:<code class="highlighter-rouge">max(), average(), summaryStatistics()</code>:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>OptionalInt maxNum = intStream.max();
IntSummaryStatistics intSummary = intStream.summaryStatistics();
</code></pre></div></div>
<h2 id="parallel-stream">Parallel Stream</h2>
<p>1) Stream 支持并发操作,但需要满足以下几点:</p>
<p>构造一个 paralle stream,默认构造的 stream 是顺序执行的,调用 <code class="highlighter-rouge">paralle()</code> 构造并行的 stream:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>IntStream scoreStream = IntStream.rangeClosed(10, 30).parallel();
</code></pre></div></div>
<p>要执行的操作必须是可并行执行的,即并行执行的结果和顺序执行的结果是一致的,而且必须保证 stream 中执行的操作是线程安全的:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>int[] wordLength = new int[12];
Stream.of("It", "is", "your", "responsibility").parallel().forEach(s -> {
if (s.length() < 12) wordLength[s.length()]++;
});
</code></pre></div></div>
<p>这段程序的问题在于,多线程访问共享数组 wordLength,是非线程安全的。解决的思路有:</p>
<ul>
<li>构造AtomicInteger数组</li>
<li>使用groupingBy()根据length统计</li>
</ul>
<p>2) 可以通过并行提高效率的常见场景:</p>
<p>使 stream 无序:对于 <code class="highlighter-rouge">distinct()</code> 和 <code class="highlighter-rouge">limit()</code> 等方法,如果不关心顺序,则可以使用并行:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>LongStream.rangeClosed(5, 10).unordered().parallel().limit(3);
IntStream.of(14, 15, 15, 14, 12, 81).unordered().parallel().distinct();
</code></pre></div></div>
<p>在 <code class="highlighter-rouge">groupingBy()</code> 的操作中,map 的合并操作是比较重的,可以通过 <code class="highlighter-rouge">groupingByConcurrent()</code> 来并行处理,不过前提是 parallel stream:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Stream.of(cities).parallel().collect(Collectors.groupingByConcurrent(City::getState));
</code></pre></div></div>
<p>在执行 stream 操作时不能修改 stream 对应的 collection ;</p>
<p>stream 本身是不存储数据的,数据保存在对应的 collection 中,所以在执行 stream 操作的同时修改对应的 collection ,结果
是未定义的:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>// ok
Stream<String> wordStream = wordList.stream();
wordList.add("number");
wordStream.distinct().count();
// ConcurrentModificationException
Stream<String> wordStream = wordList.stream();
wordStream.forEach(s -> { if (s.length() >= 6) wordList.remove(s);});
</code></pre></div></div>
<h2 id="functional-interface">Functional Interface</h2>
<p>仅包含一个抽象方法的 interface 被成为 Functional Interface,比如:Predicate, Function, Consumer 等。
此时我们一般传入一个 lambda 表达式或 Method Reference。
常见的 Functional Interface 有:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Functional Interface Parameter Return Type Description Types
Supplier<T> None T Supplies a value of type T
Consumer<T> T void Consumes a value of type T
BiConsumer<T, U> T,U void Consumes values of types T and U
Predicate<T> T boolean A Boolean-valued function
ToIntFunction<T> T int An int-, long-, or double-valued function
ToLongFunction<T> T long
ToDoubleFunction<T> T double
IntFunction<R> int R A function with argument of type int, long, or double
LongFunction<R> long
DoubleFunction<R> double
Function<T, R> T R A function with argument of type T
BiFunction<T, U, R> T,U R A function with arguments of types T and U
UnaryOperator<T> T T A unary operator on the type T
BinaryOperator<T> T,T T A binary operator on the type T
</code></pre></div></div>
<h2 id="参考">参考</h2>
<ul>
<li><a href="http://www.amazon.cn/Java-SE8-for-the-Really-Impatient-A-Short-Course-on-the-Basics-Horstmann-Cay-S/dp/0321927761/ref=sr_1_2">Java SE8 for the Really Impatient</a></li>
<li><a href="https://gist.github.com/nkcoder/50c115a96c4e67164580#file-java-8-stream-api-java">Stream示例代码</a></li>
</ul>ixiaozhiblog@ixiaozhi.com来源: Think And COde Stream是Java 8 提供的高效操作集合类(Collection)数据的 API。 从 Iterator 到 Stream 有一个字符串的 list,要统计其中长度大于 7 的字符串的数量,用迭代来实现: List<String> wordList = Arrays.asList("regular", "expression", "specified", "as", "a", "string", "must"); int countByIterator = 0; for (String word: wordList) { if (word.length() > 7) { countByIterator++; } } 用 Stream 实现: long countByStream = wordList.stream().filter(w -> w.length() > 7).count(); 显然,用 stream 实现更简洁,不仅如此,stream 很容易实现并发操作,比如: long countByParallelStream = wordList.parallelStream().filter(w -> w.length() > 7).count(); stream 遵循的原则是:告诉我做什么,不用管我怎么做。比如上例:告诉 stream 通过多线程统计字符串长度,至于以什么顺序、在哪个线程中执行,由 stream 来负责;而在迭代实现中,由于计算的方式已确定,很难优化了。 Stream 和 Collection 的区别主要有: * stream 本身并不存储数据,数据是存储在对应的 collection 里,或者在需要的时候才生成的; * stream 不会修改数据源,总是返回新的 stream; * stream 的操作是懒执行 (lazy) 的:仅当最终的结果需要的时候才会执行,比如上面的例子中,结果仅需要前3个长度大于7
的字符串,那么在找到前3个长度符合要求的字符串后,`filter()` 将停止执行; 使用stream的步骤如下: * 创建stream; * 通过一个或多个中间操作(intermediate operations)将初始stream转换为另一个stream; * 通过中止操作(terminal operation)获取结果;该操作触发之前的懒操作的执行,中止操作后,该stream关闭,不能再
使用了; 在上面的例子中, wordList.stream() 和 wordList.parallelStream() 是创建 stream,filter() 是中间操作,过滤后生成一个新的 stream,count() 是中止操作,获取结果。 创建Stream的方式 1) 从 array 或 list 创建 stream: Stream<Integer> integerStream = Stream.of(10, 20, 30, 40); String[] cityArr = {"Beijing", "Shanghai", "Chengdu"}; Stream<String> cityStream = Stream.of(cityArr); Stream<String> nameStream = Arrays.asList("Daniel", "Peter", "Kevin").stream(); Stream<String> cityStream2 = Arrays.stream(cityArr, 0, 1); Stream<String> emptyStream = Stream.empty(); 2) 通过 generate 和 iterate 创建无穷 stream: Stream<String> echos = Stream.generate(() -> "echo"); Stream<Integer> integers = Stream.iterate(0, num -> num + 1); 3) 通过其它 API 创建 stream: Stream<String> lines = Files.lines(Paths.get("test.txt")) String content = "AXDBDGXC"; Stream<String> contentStream = Pattern.compile("[ABC]{1,3}").splitAsStream(content); Stream转换 1) filter() 用于过滤,即使原 stream 中满足条件的元素构成新的 stream: List<String> langList = Arrays.asList("Java", "Python", "Swift", "HTML"); Stream<String> filterStream = langList.stream().filter(lang -> lang.equalsIgnoreCase("java")); 2) map()用于映射,遍历原stream中的元素,转换后构成新的stream: List<String> langList = Arrays.asList("Java", "Python", "Swift", "HTML"); Stream<String> mapStream = langList.stream().map(String::toUpperCase); 3) flatMap() 用于将 [["ABC", "DEF"], ["FGH", "IJK"]] 的形式转换为["ABC", "DEF", "FGH", "IJK"]: Stream<String> cityStream = Stream.of("Beijing", "Shanghai", "Shenzhen"); // [['B', 'e', 'i', 'j', 'i', 'n', 'g'], ['S', 'h', 'a', 'n', 'g', 'h', 'a', 'i'], ...] Stream<Stream<Character>> characterStream1 = cityStream.map(city -> characterStream(city)); Stream<String> cityStreamCopy = Stream.of("Beijing", "Shanghai", "Shenzhen"); // ['B', 'e', 'i', 'j', 'i', 'n', 'g', 'S', 'h', 'a', 'n', 'g', 'h', 'a', 'i', ...] Stream<Character> characterStreamCopy = cityStreamCopy.flatMap(city -> characterStream(city)); 其中, characterStream() 返回有参数字符串的字符构成的Stream; 4) limit() 表示限制 stream 中元素的数量,skip() 表示跳过 stream 中前几个元素, concat 表示将多个 stream 连接起来,peek() 主要用于 debug 时查看 stream 中元素的值: Stream<Integer> limitStream = Stream.of(18, 20, 12, 35, 89).sorted().limit(3); Stream<Integer> skipStream = Stream.of(18, 20, 12, 35, 89).sorted(Comparator.reverseOrder()) .skip(1); Stream<Integer> concatStream = Stream.concat(Stream.of(1, 2, 3), Stream.of(4, 5, 6)); concatStream.peek(i -> System.out.println(i)).count(); peek() 是 intermediate operation,所以后面需要一个terminal operation,如 count() 才能在输出中看到结果; 5) 有状态的 (stateful) 转换,即元素之间有依赖关系,如 distinct() 返回由唯一元素构成的 stream,sorted() 返回排序后的 stream: Stream<String> distinctStream = Stream.of("Beijing", "Tianjin", "Beijing").distinct(); Stream<String> sortedStream = Stream.of("Beijing", "Shanghai", "Chengdu").sorted(Comparator.comparing(String::length).reversed()); Stream reduction reduction 就是从 stream 中取出结果,是 terminal operation,因此经过 reduction 后的 stream 不能再使用了。 Optional Optional 表示或者有一个T类型的对象,或者没有值; 1) 创建 Optional 对象: 直接通过 Optional 的类方法:of()/empty()/ofNullable(): Optional<Integer> intOpt = Optional.of(10); Optional<String> emptyOpt = Optional.empty(); Optional<Double> doubleOpt = Optional.ofNullable(5.5); 2) 使用 Optional 对象: 你当然可以这么使用: if (intOpt.isPresent()) { intOpt.get(); } 但是,最好这么使用: doubleOpt.orElse(0.0); doubleOpt.orElseGet(() -> 1.0); doubleOpt.orElseThrow(RuntimeException::new); List<Double> doubleList = new ArrayList<>(); doubleOpt.ifPresent(doubleList::add); map()方法与ifPresent()用法相同,就是多个返回值,flatMap()用于Optional的链式表达: Optional<Boolean> addOk = doubleOpt.map(doubleList::add); Optional.of(4.0).flatMap(num -> Optional.ofNullable(num * 100)).flatMap(num -> Optional.ofNullable(Math.sqrt(num))); 简单的 reduction 主要包含以下操作: findFirst()/findAny()/allMatch/anyMatch()/noneMatch ,比如: Optional<String> firstWord = wordStream.filter(s -> s.startsWith("Y")).findFirst(); Optional<String> anyWord = wordStream.filter(s -> s.length() > 3).findAny(); wordStream.allMatch(s -> s.length() > 3); wordStream.anyMatch(s -> s.length() > 3); wordStream.noneMatch(s -> s.length() > 3); reduce 方法 1) reduce(accumulator):参数是一个执行双目运算的 Functional Interface,假如这个参数表示的操作为 op,stream中的元素为x, y, z, …,则 reduce() 执行的就是 x op y op z …,所以要求op这个操作具有结合性 (associative),即满足:(x op y) op z = x op (y op z),满足这个要求的操作主要有:求和、求积、求最大值、求最小值、字符串连接、集合并集和交集等。另外,该函数的返回值是 Optional 的: Optional<Integer> sum1 = numStream.reduce((x, y) -> x + y); 2) reduce(identity, accumulator):可以认为第一个参数为默认值,但需要满足 identity op x = x,所以对于求和操作,identity 的值为0,对于求积操作,identity 的值为1。返回值类型是 stream 元素的类型: Integer sum2 = numStream.reduce(0, Integer::sum); collect 结果 1) collect() 方法: reduce() 和 collect() 的区别是: * reduce()的结果是一个值; * collect()可以对stream中的元素进行各种处理后,得到stream中元素的值; Collectors 接口提供了很方便的创建 Collector 对象的工厂方法: // collect to Collection Stream.of("You", "may", "assume").collect(Collectors.toList()); Stream.of("You", "may", "assume").collect(Collectors.toSet()); Stream.of("You", "may", "assume").collect(Collectors.toCollection(TreeSet::new)); // join element Stream.of("You", "may", "assume").collect(Collectors.joining()); Stream.of("You", "may", "assume").collect(Collectors.joining(", ")); // summarize element IntSummaryStatistics summary = Stream.of("You", "may", "assume").collect(Collectors.summarizingInt(String::length)); summary.getMax(); 2) foreach() 方法: foreach() 用于遍历 stream 中的元素,属于 terminal operation;
forEachOrdered() 是按照 stream 中元素的顺序遍历,也就无法利用并发的优势; Stream.of("You", "may", "assume", "you", "can", "fly").parallel().forEach(w -> System.out.println(w)); Stream.of("You", "may", "assume", "you", "can", "fly").forEachOrdered(w -> System.out.println(w)); 3) toArray() 方法: 得到由 stream 中的元素得到的数组,默认是 Object[],可以通过参数设置需要结果的类型: Object[] words1 = Stream.of("You", "may", "assume").toArray(); String[] words2 = Stream.of("You", "may", "assume").toArray(String[]::new); 4) toMap() 方法: toMap: 将 stream 中的元素映射为的形式,两个参数分别用于生成对应的 key 和 value 的值。比如有一个字符串 stream,将首字母作为 key,字符串值作为 value,得到一个 map: Stream<String> introStream = Stream.of("Get started with UICollectionView and the photo library".split(" ")); Map<String, String> introMap = introStream.collect(Collectors.toMap(s -> s.substring(0, 1), s -> s)); 如果一个 key 对应多个 value,则会抛出异常,需要使用第三个参数设置如何处理冲突,比如仅使用原来的 value、使用新的 value,或者合并: Stream<String> introStream = Stream.of("Get started with UICollectionView and the photo library".split(" ")); Map<Integer, String> introMap2 = introStream.collect(Collectors.toMap(s -> s.length(), s -> s, (existingValue, newValue) -> existingValue)); 如果 value 是一个集合,即将 key 对应的所有 value 放到一个集合中,则需要使用第三个参数,将多个 value 合并: Stream<String> introStream3 = Stream.of("Get started with UICollectionView and the photo library".split(" ")); Map<Integer, Set<String>> introMap3 = introStream3.collect(Collectors.toMap(s -> s.length(), s -> Collections.singleton(s), (existingValue, newValue) -> { HashSet<String> set = new HashSet<>(existingValue); set.addAll(newValue); return set; } )); introMap3.forEach((k, v) -> System.out.println(k + ": " + v)); 如果 value 是对象自身,则使用 Function.identity(),如: Map<Integer, Person> idToPerson = people.collect(Collectors.toMap(Person::getId, Function.identity())); toMap() 默认返回的是 HashMap,如果需要其它类型的 map ,比如 TreeMap ,则可以在第四个参数指定构造方法: Map<Integer, String> introMap2 = introStream.collect(Collectors.toMap(s -> s.length(), s -> s, (existingValue, newValue) -> existingValue, TreeMap::new)); Grouping 和 Partitioning 1) groupingBy() 表示根据某一个字段或条件进行分组,返回一个 Map,其中 key 为分组的字段或条件,value 默认为 list,groupingByConcurrent() 是其并发版本: Map<String, List<Locale>> countryToLocaleList = Stream.of(Locale.getAvailableLocales()).collect(Collectors.groupingBy(l -> l.getDisplayCountry())); 2) 如果 groupingBy() 分组的依据是一个 bool 条件,则 key 的值为 true/false ,此时与 partitioningBy() 等价,且
partitioningBy() 的效率更高: // predicate Map<Boolean, List<Locale>> englishAndOtherLocales = Stream.of(Locale.getAvailableLocales()).collect(Collectors.groupingBy(l -> l.getDisplayLanguage().equalsIgnoreCase("English"))); // partitioningBy Map<Boolean, List<Locale>> englishAndOtherLocales2 = Stream.of(Locale.getAvailableLocales()).collect(Collectors.partitioningBy(l -> l.getDisplayLanguage().equalsIgnoreCase("English"))); 3) groupingBy() 提供第二个参数,表示 downstream,即对分组后的 value 作进一步的处理: 返回 set,而不是 list: Map<String, Set<Locale>> countryToLocaleSet = Stream.of(Locale.getAvailableLocales()).collect(Collectors.groupingBy(l -> l.getDisplayCountry(), Collectors.toSet())); 返回 value 集合中元素的数量: Map<String, Long> countryToLocaleCounts = Stream.of(Locale.getAvailableLocales()).collect(Collectors.groupingBy(l -> l.getDisplayCountry(), Collectors.counting())); 对 value 集合中的元素求和: Map<String, Integer> cityToPopulationSum = Stream.of(cities).collect(Collectors.groupingBy(City::getName, Collectors.summingInt(City::getPopulation))); 对 value 的某一个字段求最大值,注意 value 是 Optional 的: Map<String, Optional<City>> cityToPopulationMax = Stream.of(cities).collect(Collectors.groupingBy(City::getName, Collectors.maxBy(Comparator.comparing(City::getPopulation)))); 使用 mapping 对 value 的字段进行 map 处理: Map<String, Optional<String>> stateToNameMax = Stream.of(cities).collect(Collectors.groupingBy(City::getState, Collectors.mapping(City::getName, Collectors.maxBy(Comparator.comparing(String::length))))); Map<String, Set<String>> stateToNameSet = Stream.of(cities).collect(Collectors.groupingBy(City::getState, Collectors.mapping(City::getName, Collectors.toSet()))); 通过 summarizingXXX 获取统计结果: Map<String, IntSummaryStatistics> stateToPopulationSummary = Stream.of(cities).collect(Collectors.groupingBy(City::getState, Collectors.summarizingInt(City::getPopulation))); reducing() 可以对结果作更复杂的处理,但是 reducing() 却并不常用: Map<String, String> stateToNameJoining = Stream.of(cities).collect(Collectors.groupingBy(City::getState, Collectors.reducing("", City::getName, (s, t) -> s.length() == 0 ? t : s + ", " + t))); 比如上例可以通过 mapping 达到同样的效果: Map<String, String> stateToNameJoining2 = Stream.of(cities).collect(Collectors.groupingBy(City::getState, Collectors.mapping(City::getName, Collectors.joining(", ")))); Primitive Stream Stream 对应的 Primitive Stream 就是 IntStream,类似的还有 DoubleStream 和 LongStream。 1) Primitive Stream的构造:of(), range(), rangeClosed(), Arrays.stream(): IntStream intStream = IntStream.of(10, 20, 30); IntStream zeroToNintyNine = IntStream.range(0, 100); IntStream zeroToHundred = IntStream.rangeClosed(0, 100); double[] nums = {10.0, 20.0, 30.0}; DoubleStream doubleStream = Arrays.stream(nums, 0, 3); 2) Object Stream 与 Primitive Stream 之间的相互转换,通过mapToXXX() 和 boxed(): // map to Stream<String> cityStream = Stream.of("Beijing", "Tianjin", "Chengdu"); IntStream lengthStream = cityStream.mapToInt(String::length); // box Stream<Integer> oneToNine = IntStream.range(0, 10).boxed(); 3) 与 Object Stream 相比,Primitive Stream 的特点: toArray() 方法返回的是对应的 Primitive 类型: int[] intArr = intStream.toArray(); 自带统计类型的方法,如:max(), average(), summaryStatistics(): OptionalInt maxNum = intStream.max(); IntSummaryStatistics intSummary = intStream.summaryStatistics(); Parallel Stream 1) Stream 支持并发操作,但需要满足以下几点: 构造一个 paralle stream,默认构造的 stream 是顺序执行的,调用 paralle() 构造并行的 stream: IntStream scoreStream = IntStream.rangeClosed(10, 30).parallel(); 要执行的操作必须是可并行执行的,即并行执行的结果和顺序执行的结果是一致的,而且必须保证 stream 中执行的操作是线程安全的: int[] wordLength = new int[12]; Stream.of("It", "is", "your", "responsibility").parallel().forEach(s -> { if (s.length() < 12) wordLength[s.length()]++; }); 这段程序的问题在于,多线程访问共享数组 wordLength,是非线程安全的。解决的思路有: 构造AtomicInteger数组 使用groupingBy()根据length统计 2) 可以通过并行提高效率的常见场景: 使 stream 无序:对于 distinct() 和 limit() 等方法,如果不关心顺序,则可以使用并行: LongStream.rangeClosed(5, 10).unordered().parallel().limit(3); IntStream.of(14, 15, 15, 14, 12, 81).unordered().parallel().distinct(); 在 groupingBy() 的操作中,map 的合并操作是比较重的,可以通过 groupingByConcurrent() 来并行处理,不过前提是 parallel stream: Stream.of(cities).parallel().collect(Collectors.groupingByConcurrent(City::getState)); 在执行 stream 操作时不能修改 stream 对应的 collection ; stream 本身是不存储数据的,数据保存在对应的 collection 中,所以在执行 stream 操作的同时修改对应的 collection ,结果
是未定义的: // ok Stream<String> wordStream = wordList.stream(); wordList.add("number"); wordStream.distinct().count(); // ConcurrentModificationException Stream<String> wordStream = wordList.stream(); wordStream.forEach(s -> { if (s.length() >= 6) wordList.remove(s);}); Functional Interface 仅包含一个抽象方法的 interface 被成为 Functional Interface,比如:Predicate, Function, Consumer 等。
此时我们一般传入一个 lambda 表达式或 Method Reference。 常见的 Functional Interface 有: Functional Interface Parameter Return Type Description Types Supplier<T> None T Supplies a value of type T Consumer<T> T void Consumes a value of type T BiConsumer<T, U> T,U void Consumes values of types T and U Predicate<T> T boolean A Boolean-valued function ToIntFunction<T> T int An int-, long-, or double-valued function ToLongFunction<T> T long ToDoubleFunction<T> T double IntFunction<R> int R A function with argument of type int, long, or double LongFunction<R> long DoubleFunction<R> double Function<T, R> T R A function with argument of type T BiFunction<T, U, R> T,U R A function with arguments of types T and U UnaryOperator<T> T T A unary operator on the type T BinaryOperator<T> T,T T A binary operator on the type T 参考 Java SE8 for the Really Impatient Stream示例代码