OnlyOffice Document Server 容器部署和监控

大体架构是:前端 Nginx 卸载SSL后反代给 OnlyOffice,OnlyOffice 将监控信息喂给 statsd-exporter,Prometheus 从 statsd-exporter 拉取数据,最终用 Grafana 可视化展示。

先来 docker-compose.yml 配置如下,有一些注意修改的地方:

  1. 可以把一些onlyoffice插件挂进容器中 /var/www/onlyoffice/documentserver/sdkjs-plugins/ 目录下,启动时会自动加载。
  2. onlyoffice自带的中文字体少的可怜,一定要整理一些常用字体(特别是Windows/Office自带的字体),将ttf/otf字体文件挂进容器中 /usr/share/fonts/ 目录下,启动时会自动加载。
  3. onlyoffice的持久化配置挂进容器 /etc/onlyoffice/documentserver/local-production-linux.json,这样升级重启的时候配置才能保留下来;容器里面的配置文件改了也没用,重建就全丢了。
  4. JWT_SECRET 配置一个足够强的Token,和应用软件配置要一样。
services:
  onlyoffice:
    image: onlyoffice/documentserver-ee
    container_name: onlyoffice
    volumes:
      - ./onlyoffice/logs:/var/log/onlyoffice
      - ./onlyoffice/data:/var/www/onlyoffice/Data
      - ./onlyoffice/lib:/var/lib/onlyoffice
      - ./onlyoffice/db:/var/lib/postgresql
      - ./onlyoffice/plugin/plugin-autocomplete:/var/www/onlyoffice/documentserver/sdkjs-plugins/plugin-autocomplete
      - ./onlyoffice/plugin/plugin-html:/var/www/onlyoffice/documentserver/sdkjs-plugins/plugin-html
      - ./onlyoffice/plugin/plugin-languagetool:/var/www/onlyoffice/documentserver/sdkjs-plugins/plugin-languagetool
      - ./onlyoffice/plugin/plugin-macros:/var/www/onlyoffice/documentserver/sdkjs-plugins/plugin-macros
      - ./onlyoffice/plugin/plugin-typograf:/var/www/onlyoffice/documentserver/sdkjs-plugins/plugin-typograf
      - ./onlyoffice/plugin/plugin-wordscounter:/var/www/onlyoffice/documentserver/sdkjs-plugins/plugin-wordscounter
      - ./onlyoffice/fonts/dejavu:/usr/share/fonts/dejavu
      - ./onlyoffice/fonts/founder:/usr/share/fonts/founder
      - ./onlyoffice/fonts/liberation:/usr/share/fonts/liberation
      - ./onlyoffice/fonts/libertinus:/usr/share/fonts/libertinus
      - ./onlyoffice/fonts/noto-cjk:/usr/share/fonts/noto-cjk
      - ./onlyoffice/fonts/noto-emoji:/usr/share/fonts/noto-emoji
      - ./onlyoffice/fonts/sarasa-gothic:/usr/share/fonts/sarasa-gothic
      - ./onlyoffice/fonts/source-code-pro:/usr/share/fonts/source-code-pro
      - ./onlyoffice/fonts/source-han-sans:/usr/share/fonts/source-han-sans
      - ./onlyoffice/fonts/source-han-serif:/usr/share/fonts/source-han-serif
      - ./onlyoffice/fonts/source-sans:/usr/share/fonts/source-sans
      - ./onlyoffice/fonts/source-serif:/usr/share/fonts/source-serif
      - ./onlyoffice/fonts/windows:/usr/share/fonts/windows
      - ./onlyoffice/local-production-linux.json:/etc/onlyoffice/documentserver/local-production-linux.json
    environment:
      - TZ=Asia/Shanghai
      - JWT_ENABLED=true
      - JWT_SECRET=yaoge123
  statsd-exporter:
    image: prom/statsd-exporter
    container_name: statsd-exporter
    ports:
      - 9102:9102
    depends_on:
      - onlyoffice 
  nginx:
    image: nginx:alpine
    container_name: nginx
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./nginx/conf.d/:/etc/nginx/conf.d/:ro
      - ./nginx/ssl/:/etc/nginx/ssl/:ro
    environment:
      - TZ=Asia/Shanghai
  nginx-exporter:
    image: nginx/nginx-prometheus-exporter
    container_name: nginx-prometheus-exporter
    ports:
      - 9113:9113
    command:
      - -nginx.scrape-uri
      - http://nginx:8080/stub_status
    depends_on:
      - nginx
……

在 local-production-linux.json 对 OnlyOffice 进行持久化配置,主要是三部分:

  1. 开启statsd监控推送,填写statsd-exporter的主机名和端口
  2. 开启自动保存
  3. 打开文件尺寸扩大至1GB(默认是100MB)
{
        "statsd": {
                "useMetrics": true,
                "host": "statsd-exporter",
                "port": "9125",
                "prefix": "ds."
        },
	"services": {
		"CoAuthoring": {
			"autoAssembly": {
				"enable": true,
				"interval": "5m"
			}
		}
	},
	"FileConverter": {
		"converter": {
			"maxDownloadBytes": 1073741824,
			"downloadAttemptMaxCount": 3
		}
	}
}

Nginx 配置反代卸载SSL

map $http_host $this_host {
    "" $host;
    default $http_host;
}

map $http_x_forwarded_proto $the_scheme {
     default $http_x_forwarded_proto;
     "" $scheme;
}

map $http_x_forwarded_host $the_host {
    default $http_x_forwarded_host;
    "" $this_host;
}

map $http_upgrade $proxy_connection {
    default upgrade;
    "" close;
}

server {
	listen 80;
	listen [::]:80;
	server_name onlyoffice.nju.edu.cn;
	rewrite ^ https://$http_host$request_uri? permanent;
	server_tokens off;
}

server {
        listen 443 ssl http2;
        listen [::]:443 ssl http2;
	server_name onlyoffice.nju.edu.cn;
	server_tokens off;

        include ssl/nju_edu_cn.conf;

	add_header X-Content-Type-Options nosniff;

	location / {
		proxy_pass http://onlyoffice;
		proxy_http_version 1.1;
		proxy_set_header Upgrade $http_upgrade;
		proxy_set_header Connection $proxy_connection;
		proxy_set_header X-Forwarded-Host $the_host;
		proxy_set_header X-Forwarded-Proto $the_scheme;
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	}
}

server {
	listen 8080;
	listen [::]:8080;
	server_name localhost;

	location /stub_status {
		stub_status on;
		access_log off;
	}
}

在 Consul 中注册 OnlyOffice 和 Nginx 的监控

[yaoge123 ~]$ curl -X PUT -d '{"id": "onlyoffice.nju.edu.cn_statsd-exporter","name": "statsd_exporter","address": "onlyoffice.nju.edu.cn","port": 9102,"tags": ["prometheus","vm"],"checks": [{"http": "http://onlyoffice.nju.edu.cn:9102/metrics","interval": "30s"}]}' http://consul:8500/v1/agent/service/register

[yaoge123 ~]$ curl -X PUT -d '{"id": "onlyoffice.nju.edu.cn_nginx-exporter","name": "nginx_exporter","address": "onlyoffice.nju.edu.cn","port": 9113,"tags": ["prometheus","vm"],"checks": [{"http": "http://onlyoffice.nju.edu.cn:9113/metrics","interval": "30s"}]}' http://consul:8500/v1/agent/service/register

在Grafana中导入基于官方Dashboard的修改版

https://grafana.com/grafana/dashboards/17048

山石网科 Hillstone 防火墙QoS详解

防火墙固件StoneOS 5.5R6,两路ISP PPPoE动态IP接入和静态IP接入,需要提供SSLVPN接入并将内网服务器发布到ISP的公网IP上,对公网直接访问和SSLVPN进行限速。山石网科的原厂工程师都搞不清楚怎么配置及其含义,所以只能通过自己实验来解决。
 
接口:
  • PPPoE接入:配置接口绑定三层安全域(untrust),IP配置为PPPoE,因为是动态IP所以必须配置DDNS,关闭逆向路由。
  • 静态IP接入:配置接口绑定三层安全域(untrust),IP配置为静态IP,关闭逆向路由。
目的NAT:
  • 源Any,目的地址对于动态IP选物理端口、对于静态IP填写IP地址,服务为对外发布的端口,转换为填写内网服务器IP和端口。
安全策略:
  • 源Any,目的安全域trust,目的地址为DDNS的动态域名和静态IP,服务对应目的NAT的服务。

VPN:

  • 为每个ISP接入创建独立的VPN,绑定各自的接入接口,为每个ISP创建独立的不同子网的隧道接口和地址池。
iQoS配置:
  • 不勾选”启用NAT IP匹配“。
iQoS策略:
  • 为每一个IPS接入创建单独的第一层流控管道。
  • 管道模式为整形,此模式支持带宽借用。
  • 匹配条件创建两条,一条只设置目的接口为ISP接防火墙的物理端口(无论动态还是静态IP),服务选择目的NAT中的服务;另一条只设置目的地址条目为vpn的地址池。
  • 流控动作是重点,在上述匹配条件下,正向流量指客户端下载、防火墙向ISP上传的流量;反向流量指客户端上传,防火墙从ISP下载的流量。
  • 管道带宽是ISP分配给的带宽,特别注意对于上下行非对等的宽带一定不要设置错。
  • 限制类型选限每IP,同IP下NAT的多个客户端会按照整个IP限制。
  • 限流选目的IP,这里的目的IP是客户端IP、源IP是服务器IP,因要限速每个客户端所以选目的IP,正向和反向是一样的。
  • 设置最小带宽,最小带宽是保证带宽,保证客户端有最基本的带宽可用。不设置最大带宽,尽可能的提高带宽利用率。
  • 不勾选平均带宽,这样不能很好地利用总带宽,应该设置最小保证带宽,不限制最大带宽可以充分利用带宽。如果勾选了平均带宽,会出现20Mb总带宽,4个IP连接,只有2个IP传输数据的时候,总带宽只能跑到10Mb。如果设置了最大带宽,但最大带宽小于总带宽,在单一客户端使用的情况下,跑不满总带宽会有浪费。

在以上配置的情况下,单一客户端可以跑满总带宽,多个客户端时可以保证每个客户端至少能获得最小带宽。

FortiWeb中配置HTTPS

以FortiWeb 6.3.1导入GoGetSSL颁发的RSA/ECC双证书为例

  1. 导入根证书和OCSP证书:菜单System->Certificates->CA,CA标签页,导入AAA Certificate Services、USERTrust RSA Certification Authority、USERTrust ECC Certification Authority、GoGetSSL RSA DV CA、GoGetSSL ECC DV CA
  2. 导入中级证书:菜单System->Certificates->Intermediate CA,Intermediate CA标签页,导入AAA Certificate Services签发的USERTrust RSA Certification Authority和USERTrust ECC Certification Authority,再导入USERTrust RSA Certification Authority签发的GoGetSSL RSA DV CA和USERTrust ECC Certification Authority签发的GoGetSSL ECC DV CA
  3. 创建中级证书组:菜单System->Certificates->Intermediate CA,Intermediate CA Group标签页,创建证书组GoGedtSSL RSA,先添加GoGetSSL RSA DV CA,再添加USERTrust RSA Certification Authority,ID 1应为签发服务器证书的中级证书,ID 2应为根证书签发的中级证书;同样的创建证书组GoGetSSL ECC,加入中级证书GoGetSSL ECC DV CA和USERTrust ECC Certification Authority;再创建一个证书组GoGetSSL把这四个中级证书都加进去
  4. 导入服务器证书:菜单System->Certificates->Local,分别导入证书颁发机构签发的服务器RSA和ECC证书及其对应的Key
  5. 添加OCSP stapling:菜单System->Certificates->OCSP stapling,为每一个服务器证书创建对应的OCSP,其中CA Certificate和OCSP URL按照服务器证书内容选择和填写,CA要先导入
  6. 创建多证书:菜单System->Certificates->Multi-certificate,创建一个多证书,分别选择RSA和ECDSA两个服务器证书
  7. 在Server里面如果使用多证书,就要选择有RSA和ECC中级证书的证书组,这是因为当前FortiWeb的系统的多证书只能选择一个中级证书组,所以就需要这个中级证书组里面放置所有的中级证书,将这些中级证书全部发送给客户端,让客户端自己选择

用 openssl s_client -showcerts -status -tlsextdebug -connect www.yaoge123.com:443 和 SSL Labs 检查证书链和OCSP

Seafile集成卡巴斯基

防病毒脚本 /opt/kaspersky/kav4fs_scan.sh

#!/bin/bash

VIRUS_FOUND=1
CLEAN=0
UNDEFINED=2
KAV4FS='/opt/kaspersky/kav4fs/bin/kav4fs-control'
if [ ! -x $KAV4FS ]
then
    echo "Binary not executable"
    exit $UNDEFINED
fi

SCAN_OUTPUT=`$KAV4FS --scan-file "$1"`
if [ "$?" -ne 0 ]
then
    echo "Error due to check file '$1'"
    exit 3
fi

while read line
do
	OUT1=`echo $line|cut -d':' -f 1`
	OUT2=`echo $line|cut -d':' -f 2|sed 's/ //g'`
	case "$OUT1" in
        "Threats found" )
                THREATS_C=$OUT2
                ;;
        "Riskware found" )
                RISKWARE_C=$OUT2
                ;;
        "Infected" )
                INFECTED=$OUT2
                ;;
        "Suspicious" )
                SUSPICIOUS=$OUT2
                ;;
        "Scan errors" )
                SCAN_ERRORS_C=$OUT2
                ;;
        "Password protected" )
                PASSWORD_PROTECTED=$OUT2
                ;;
        "Corrupted" )
                CORRUPTED=$OUT2
                ;;
	esac
done <<< "$SCAN_OUTPUT"

if [ $INFECTED -gt 0 ]
then
    exit $VIRUS_FOUND
elif [ $THREATS_C -gt 0 -o $RISKWARE_C -gt 0 -o $SUSPICIOUS -gt 0 -o $SCAN_ERRORS_C -gt 0 -o $CORRUPTED -gt 0 ]
then
    exit $UNDEFINED
else
    exit $CLEAN
fi

/opt/seafile/conf/seafile.conf 添加防病毒配置

[virus_scan]
scan_command = /opt/kaspersky/kav4fs_scan.sh
virus_code = 1
nonvirus_code = 0
scan_interval = 60

每天crontab清除kav4fs的日志/etc/cron.d/kav

30 0 * * * root find /var/log/kaspersky/kav4fs/supervisor_trace.log* -exec rm {} \;
40 0 * * * root /opt/kaspersky/kav4fs/bin/kav4fs-control -S --clean-stat

Ubuntu 16.04 改变 Transmission 运行用户

//先停止服务
sudo systemctl stop transmission-daemon.service

//修改文件
vim /etc/init/transmission-daemon.conf
setuid yaoge123
setgid yaoge123

//添加信息
sudo systemctl edit transmission-daemon.service
[Service]
User=yaoge123

//重启服务
sudo systemctl daemon-reload
sudo systemctl start transmission-daemon.service

 

安装和使用 Drush

apt安装drush版本是5.x,已被官方标记为不支持;采用pear安装虽然版本为6,但是小版本较老,所以:

curl -sS https://getcomposer.org/installer | php
mv composer.phar /usr/local/bin/composer
composer global require drush/drush:6.*
echo 'export PATH="$HOME/.composer/vendor/bin:$PATH"' >>  ~/.bashrc
//用于升级:
composer global update

下载和启用模块,并且en的时候会自动处理依赖

drush dl views
drush en views

 

Nginx HTTPS配置

配置很简单,但是需要注意的有:

  1. 应使用listen的ssl参数取代ssl on;
  2. cert文件应包含整个证书链,内容顺序必须是证书链的逆序,即cert文件头是本服务器的证书、中间是中间CA、最后是根CA。可以用下述命令检查
    openssl s_client -connect www.example.com:443
  3. ssl_protocols中只能包含TLS
    server {
            listen          [::]:80;
            listen          [::]:443 ssl;
    ……
            ssl_certificate /etc/ssl/microstructures_bundle.crt;
            ssl_certificate_key /etc/ssl/microstructures_org.key;
            ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
            ssl_prefer_server_ciphers on;
    ……
    }

 

Linux DHCP 下自定义路由和网关

主机IP必须通过DHCP获得,但是因故需要重新指定网关并做策略路由。例如主机DHCP获取IP段192.168.1.0/24,DHCP获取网关192.168.1.1,拟将默认路由改为192.168.1.2,本地IP仍然走网关192.168.1.1

RHEL(CentOS) 6/7

/etc/sysconfig/network-scripts/ifcfg-eth0
BOOTPROTO=dhcp
NM_CONTROLLED="no"
ONBOOT=yes
GATEWAY=192.168.1.2
……

/etc/sysconfig/network-scripts/route-eth0
192.168.0.0/16 via 192.168.1.1

/etc/sysconfig/network
NETWORKING=yes
……

Debian 7

/etc/network/interfaces
……
up route del default dev eth0
up route add -net 192.168.1.0 netmask 255.255.255.0 gw 192.168.1.1 dev eth0
up route add default gw 192.168.1.2 dev eth0
……

Suse 11

/etc/sysconfig/network/routes
……
192.168.0.0 192.168.1.1 255.255.0.0 eth0
default 192.168.1.2 - -