2017-12-08

JSON Web Token and Base64Url

我在讀維基百科的條目也有跟著使用 Tcl 驗證,我卡住了一段時間,因為最後的 Token 跟我程式計算出來的不同,不過最後我發現是 base64 和 Base64Url 有一些小小的不同,最後在搜尋 Tcler's wiki 以後知道要怎麼做了,下面是測試程式:

package require sha256

proc base64url_encode {string} {
    tailcall string map {+ - / _ = {}} [binary encode base64 $string]
}

proc base64url_decode {string} {
    tailcall binary decode base64 [string map {- + _ /} $string]
}


set header {{"alg":"HS256","typ":"JWT"}}
set payload {{"loggedInAs":"admin","iat":1422779638}}
set key {secretkey}

set unsignedToken "[base64url_encode $header].[base64url_encode $payload]"
set signature [base64url_encode [::sha2::hmac -bin -key $key $unsignedToken]]
set token "$unsignedToken.$signature"
puts $token

所以按照參考資料,寫了一個簡單的修正,然後編碼就正確了。

小知識: base64 != base64Url

* 更新:加入 decode 的部份

2017-12-06

Apache Rivet: status code and content type

最近在學習 Vue.js,所以有做一些 server 端的測試。寫法大致上是這樣:

Vue.js + Axios  -> send POST request to server -> get response

也就是所謂的前後端分離。在這個情況下,server 端(例如 Apache HTTP server + Apache Rivet 的組合)就變成比較像 web API 的部份。

XML 的範例可以看 Apache Rivet 的手冊:
#
# Ajax query servelet: a pseudo database is built into the dictionary 'composers' with the
# purpose of emulating the role of a real data source. 
# The script answers with  2 types of responses: a catalog of the record ids and a database 
# entry matching a given rec_id. The script obviously misses the error handling and the
# likes. Just an example to see rivet sending xml data to a browser. The full Tcl, JavaScript
# and HTML code are available from http://people.apache.org/~mxmanghi/rivet-ajax.tar.gz

# This example requires Tcl8.5 or Tcl8.4 with package 'dict' 
# (http://pascal.scheffers.net/software/tclDict-8.5.2.tar.gz)
# 

# A pseudo database. rec_id matches a record in the db

set composers [dict create  \
                1 {first_name Claudio middle_name "" last_name Monteverdi   \
                    lifespan 1567-1643 era Renaissance/Baroque}             \
                2 {first_name Johann middle_name Sebastian last_name Bach   \
                    lifespan 1685-1750 era Baroque }                        \
                3 {first_name Ludwig middle_name "" last_name "van Beethoven" \
                    lifespan 1770-1827 era Classical/Romantic}              \
                4 {first_name Wolfgang middle_name Amadeus last_name Mozart \
                    lifespan 1756-1791 era Classical }                      \
                5 {first_name Robert middle_name "" last_name Schumann      \
                    lifespan 1810-1856 era Romantic} ]

# we use the 'load' argument in order to determine the type of query
#
# load=catalog:         we have to return a list of the names in the database
# load=composer&amp;res_id=<id>: the script is supposed to return the record
#               having <id> as record id

if {[::rivet::var exists load]} {

# the xml declaration is common to every message (error messages included)

    set xml "<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?>\n"
    switch [::rivet::var get load] {
        catalog {
            append xml "<catalog>\n"
            foreach nm [dict keys $composers] {
                set first_name  [dict get $composers $nm first_name]
                set middle_name [dict get $composers $nm middle_name]
                set last_name   [dict get $composers $nm last_name]
                append xml "    <composer key=\"$nm\">$first_name "
                if {[string length [string trim $middle_name]] > 0} {
                    append xml "$middle_name "
                }
                append xml "$last_name</composer>\n"
            }
            append xml "</catalog>\n"
        }
        composer {
            append xml "<composer>\n"
            if {[::rivet::var exists rec_id]} {
                set rec_id [::rivet::var get rec_id]
                if {[dict exists $composers $rec_id]} {
                    foreach {k v} [dict get $composers $rec_id] {
                        append xml "<$k>$v</$k>\n"
                    }
                }
            }
            append xml "</composer>\n"
        }
    }

# we have to tell the client this is an XML message. Failing to do so
# would result in an XMLResponse property set to null

    ::rivet::headers type "text/xml"
    ::rivet::headers add Content-Length [string length $xml]
    puts $xml
}

只要知道怎麼拿取 request 的資料跟如何回應,就可以組成 web API。下面是我快速驗證的 prototype:
<?
    package require rl_json
    package require tdbc::postgres

    ::rivet::load_response response

    if {[info exists response(sqlstring)]==1} {
        set sqlstring $response(sqlstring)
    }

    if {[info exists response(sqlstring)]==0 || [info exists sqlstring]==0} {
        ::rivet::headers  numeric 400
    } else {
        ::rivet::headers numeric 200
        ::rivet::headers type application/json

        tdbc::postgres::connection create db -user danilo -password danilo -port 5432 -database danilo

        set stmt [db prepare $sqlstring]
        set resultset [$stmt execute]

        set cols [$resultset columns]
        set param_colitem [list]
        foreach col $cols {
            lappend param_colitem "string \"$col\""
        }
        rl_json::json set data "column" [rl_json::json new "array" {*}$param_colitem]

        set param_rowitems [list]
        $resultset foreach -as lists result {
            set param_rowitem [list]
            lappend param_rowitem "array"
            foreach row $result {
                lappend param_rowitem "string \"$row\""
            }

            lappend param_rowitems $param_rowitem
        }

        $resultset close
        $stmt close
        db close

        set data [rl_json::json set data "row" [rl_json::json new "array" {*}$param_rowitems]]
        puts $data
    }
?>

Apache Rivet 有自己的 Database API,可是因為我只是要寫一個測試用的程式,所以直接使用 TDBC。這個範例是送出 SQL statement,然後 POST 到 server 上,server 執行以後回傳結果。

因為並沒有做使用者資料驗證和檢查,這個測試程式請不要在測試目的以外的地方使用,這只是用來說明,我們可以使用 Apache Rivet 用來實作 web api。

2017-11-26

NaviServer: enable gzip compression

參考連結:
How to enable gzip compression

NaviServer 在 nsd-config.tcl 中的 ns/server/${server} 加入下列的設定,動態網頁 ADP 的部份就可以支援 gzip compression:
ns_section     "ns/server/default"
   
# Compress response character data: ns_return, ADP etc.
#
ns_param        compressenable  on      ;# false, use "ns_conn compress" to override
ns_param        compresslevel   4       ;# 4, 1-9 where 9 is high compression, high overhead
ns_param        compressminsize 512     ;# Compress responses larger than this
# ns_param      compresspreinit true    ;# false, if true then initialize and allocate buffers at startup


一開始的時候我以為和 AOLServer 一樣,需要 nszlib module,但是我寫好了 RPM spec 並且安裝好了以後,才發現 NaviServer 已經整合了 zlib 的部份,所以在一開始 configure 的時候就需要 --with-zlib 設定才行(我是往前翻安裝的筆記並且參考連結以後才發現不用安裝 nszlib module)。

經過實測,NaviServer 只要針對 ADP 的部份進行 gzip compression,而靜態網頁看起來是沒有壓縮的(PS. 我只有加這篇文章的設定進行測試)。

2017-11-25

NaviServer: 設定 HTTPS (self-signed certificate)

nsssl module 是 NaviServer 用來處理 SSL 部份的 module。我目前使用的 NaviServer 4.99.15 已經整合 nsssl 的功能進入核心。

下面是參考 nsssl 文件上如何產生  self-signed certificate 與我之前其它 server 的設定,綜合之後的筆記。

Creating self-signed certificate,首先是建立 localhost.conf 設定檔案:
[req]
default_bits       = 2048
default_keyfile    = localhost.key
distinguished_name = req_distinguished_name
req_extensions     = req_ext
x509_extensions    = v3_ca

[req_distinguished_name]
countryName                 = Country Name (2 letter code)
countryName_default         = US
stateOrProvinceName         = State or Province Name (full name)
stateOrProvinceName_default = New York
localityName                = Locality Name (eg, city)
localityName_default        = Rochester
organizationName            = Organization Name (eg, company)
organizationName_default    = localhost
organizationalUnitName      = organizationalunit
organizationalUnitName_default = Development
commonName                  = Common Name (e.g. server FQDN or YOUR name)
commonName_default          = localhost
commonName_max              = 64

[req_ext]
subjectAltName = @alt_names

[v3_ca]
subjectAltName = @alt_names

[alt_names]
DNS.1   = localhost
DNS.2   = 127.0.0.1

Run the following commands using OpenSSL to create a self-signed certificate in Linux or Mac OSX with OpenSSL :
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout localhost.key -out localhost.crt -config localhost.conf -passin pass:YourSecurePassword

cat localhost.crt localhost.key > server.pem
rm -rf localhost.crt localhost.key
openssl dhparam 2048 >> server.pem

然後將 server.pem 複製到 /var/lib/naviserver/modules/nsssl 目錄下。
接下來設定 NaviServer 的部份(修改 conf 目錄下的 nsd-config.tcl,或者是你的設定檔):
#
# NSSSL
#

ns_section    ns/server/default/modules
ns_param      nsssl                nsssl.so

ns_section    ns/server/default/module/nsssl
ns_param      certificate      $home/modules/nsssl/server.pem
ns_param      address          127.0.0.1
ns_param      port             8081
ns_param      ciphers              "ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!RC4"
ns_param      protocols            "!SSLv2:!SSLv3"
ns_param      verify                0

ns_param      extraheaders {
   Strict-Transport-Security "max-age=31536000; includeSubDomains"
   X-Frame-Options SAMEORIGIN
   X-Content-Type-Options nosniff
}

原本範例 port 是設定為 443,不過因為 1024 以下的 port 需要 root 權限,所以我改成 8081 來進行測試。

注意:default 要視你的設定而定,因為我是使用一開始的 nsd-config.tcl 來測試,所以 server name 是 default。

2017-11-24

Next Scripting Framework

Next Scripting Framework

NX is a highly flexible, Tcl-based, object-oriented scripting language. It is a descendant of XOTcl and was designed based on 10 years of experience with XOTcl in projects containing several hundred thousand lines of code.


上面是引述官網的介紹,也就是 NSF 是 XOTcl 的後繼者。在我的認知中,以前 Tcl 最主要提供物件導向支援的套件,一個是 Incr Tcl,一個就是 XOTcl。

我並不是 NSF 的使用者,不過有需要研究一下要怎麼安裝在 openSUSE 才對,然後我會試寫一個 RPM spec 來安裝。

因為 NaviServer 的 WebSocket 實作是以 NSF 寫的,所以如果我要知道目前 NaviServer 的實作情況,那麼先置條件是要裝 NSF。等裝完並且沒有問題以後,才能夠開始 NaviServer 的 WebSocket 功能測試。



配合維基百科上的 Handshake 說明:

用戶端請求
GET / HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Host: example.com
Origin: http://example.com
Sec-WebSocket-Key: sN9cRrP/n9NdMgdcy2VJFQ==
Sec-WebSocket-Version: 13

伺服器回應
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: fFBooB7FAkLlXgRSz0BT3v4hq5s=
Sec-WebSocket-Location: ws://example.com/


更新:
安裝 NSF 成功。
接下來執行 chat 範例成功(需要設定 host 為 localhost)。


更新 2017/11/25
我發現 revproxy module (Reverse Proxy module) 也是使用 NSF 寫的,所以 NaviServer 也有提供 Reverse Proxy 的功能,裝上 Reverse Proxy module 就好了。

tcl-lmdb v0.3.6

檔案放置網頁

tcl-lmdb - Tcl interface to the Lightning Memory-Mapped Database

About

This is the Lightning Memory-Mapped Database (LMDB) extension for Tcl using the Tcl Extension Architecture (TEA).

LMDB is a Btree-based database management library with an API similar to BerkeleyDB. The library is thread-aware and supports concurrent read/write access from multiple processes and threads. The DB structure is multi-versioned, and data pages use a copy-on-write strategy, which also provides resistance to corruption and eliminates the need for any recovery procedures. The database is exposed in a memory map, requiring no page cache layer of its own. This extension provides an easy to use interface for accessing LMDB database files from Tcl.

Main Change

  • Fix Segfault when you use an env without opening it

一些說明

merge gahr 的 solution。

2017-11-23

Install NaviServer on openSUSE

NaviServer is a high performance web server written in C and Tcl.

如果要自己手動安裝:
./configure --prefix=/var/lib/naviserver --with-tcl=/usr/lib64 --with-zlib=/usr/lib64 --with-openssl=/usr/lib64
make
sudo make install

如果在 make 的時候遇到 -fstack-protector-strong 或者是 -fstack-clash-protection 無法辨識的問題, 修改 include/Makefile.global, 將 -fstack-protector-strong 或者是 -fstack-clash-protection 改為 -fstack-protector。 這是編譯套件時 GCC 新舊版本不同所造成的狀況。
sed -i s/stack-protector-strong/stack-protector/g include/Makefile.global
sed -i s/stack-clash-protection/stack-protector/g include/Makefile.global

NaviServer 預設的使用者是 nsadmin,所以需要使用下列的指令來增加使用者:
sudo useradd -d/var/lib/naviserver -U -M -s/bin/bash nsadmin

再來需要修改 NaviServer 目錄的擁用者設定,修改擁有者 user:group 為 nsadmin:nsadmin。
sudo chown -R nsadmin:nsadmin /var/lib/naviserver

再來要設定 nsadmin 帳號的密碼,使用 passwd 改變 nsadmin 使用者的密碼:
sudo passwd nsadmin

再來是執行 NaviServer
sudo -i -u nsadmin  /var/lib/naviserver/bin/nsd -f -t /var/lib/naviserver/conf/nsd-config.tcl

再來瀏覽 http://localhost:8080/,如果出現 NaviServer 的訊息就表示成功了。

/var/lib/naviserver/conf 下還有其它的設定範例,包含 openacs-config.tcl, sample-config.tcl, simple-config.tcl, 也可以用來參考。


以上手動安裝的部份。在知道怎麼手動安裝以後,我就寫了 script 來產生 RPM 檔案,成果請看 naviserver-spec

(更新:我發現和裝到 /usr/local/ns 的狀況不同,所以不用設定 log 目錄位置,所以我移除了怎麼設定 log 目錄的部份)

2017-11-21

Wub systemd service

假設  Wub 目錄是放在 /var/opt/wub,將 user:group 設定為 wwwrun:www,所以試寫第一版的 wub.service 如下:

[Unit]
Description=Tcl Wub web-server
After=network.target

[Service]
WorkingDirectory=/var/opt/wub
ExecStart=/usr/bin/tclsh Wub.tcl
Restart=always
# Restart service after 10 seconds if Wub service crashes
RestartSec=10
User=wwwrun
Group=www
SyslogIdentifier=wub

[Install]
WantedBy=multi-user.target

如果是 openSUSE,檔名是 wub.service,然後放在 /etc/systemd/system 目錄下。這樣就可以將 wub 作為一個系統服務來管理。

也就是說,可以這樣啟動 Wub:
sudo service wub start

關閉 Wub:
sudo service wub stop

目前狀況:
sudo service wub status


更新 2017/11/22:
加入 After= 和 WantedBy=

2017-11-20

TclVFS @ core.tcl.tk

TclVFS 目前主要開發的位址已經被移到 core.tcl.tk,所以如果需要拿取最新原始碼或者是對這個套件有興趣的人需要更新網址,不過看起來還在整理的階投。還有就是我注意 mpexpr 也被放到 core.tcl.tk 的列表中。

如果 TclVFS 有人繼續維護的話,我想是一件好事,因為在一些情況下, Tcl Virtual Filesystem 是一個很好用的工具。

2017-11-14

Apache 2, Apache Rivet and openSUSE

這篇假設你已經在 openSUSE 安裝了 Tcl,如果沒有,那需要先安裝才行。測試的環境為 openSUSE Leap 42.3。

Apache HTTP Server 2 在 openSUSE 安裝的方式:
sudo zypper in apache2

如果要啟動 Apache 2:
sudo systemctl start apache2

如果要停止 Apache 2:
sudo systemctl stop apache2

如果要開機的時候就啟動服務,使用:
sudo chkconfig apache2 on

如果不要,使用:
sudo chkconfig apache2 off

如果需要公開在網路上,還需要設定防火牆,否則只能在 localhost 使用。


再來安裝 Apache Rivet,如果要在 openSUSE 安裝,先設定軟體庫:
sudo zypper addrepo https://download.opensuse.org/repositories/Apache:/Modules/openSUSE_Leap_42.3/ Apache-Modules

更新軟體庫:
sudo zypper refresh

更新以後,使用下列的指令安裝:
sudo zypper install apache2-mod_rivet

加入 Apache Rivet module 到 Apache 2:
sudo a2enmod rivet
(* 如果要移除,使用 sudo a2dismod rivet 來移除)

接下來重新啟動 Apache 2:
sudo systemctl restart apache2

到 /srv/www/htdocs/ 目錄下,建立 hello.rvt,內容如下:
<? set hello_message "Hello world" ?>
<html>
  <head>
    <title><?= $hello_message ?></title>
  </head>
  <body><?= [::rivet::html $hello_message pre b] ?></body>
</html>
瀏覽 http://localhost/hello.rvt,如果有看到訊息表示成功安裝。


再來設定 Apache2 HTTPS 的部份。一開始先確定 mod_ssl 有開啟:
sudo a2enmod ssl

再來是設定 Self-Signed Certificates 的部份,在 Linux 上使用 OpenSSL 建立。Create a config file for your certificate :
 [req]
default_bits       = 2048
default_keyfile    = localhost.key
distinguished_name = req_distinguished_name
req_extensions     = req_ext
x509_extensions    = v3_ca

[req_distinguished_name]
countryName                 = Country Name (2 letter code)
countryName_default         = US
stateOrProvinceName         = State or Province Name (full name)
stateOrProvinceName_default = New York
localityName                = Locality Name (eg, city)
localityName_default        = Rochester
organizationName            = Organization Name (eg, company)
organizationName_default    = localhost
organizationalUnitName      = organizationalunit
organizationalUnitName_default = Development
commonName                  = Common Name (e.g. server FQDN or YOUR name)
commonName_default          = localhost
commonName_max              = 64

[req_ext]
subjectAltName = @alt_names

[v3_ca]
subjectAltName = @alt_names

[alt_names]
DNS.1   = localhost
DNS.2   = 127.0.0.1
Run the following 2 commands using OpenSSL to create a self-signed certificate in openSUSE with OpenSSL :
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout localhost.key -out localhost.crt -config localhost.conf -passin pass:YourSecurePassword

sudo openssl pkcs12 -export -out localhost.pfx -inkey localhost.key -in localhost.crt

然後將我們製造的檔案複製到 Apache 的目錄下:
sudo cp localhost.crt /etc/apache2/ssl.crt/server.crt
sudo cp localhost.key /etc/apache2/ssl.key/server.key

如果是 Officially Signed Certificate,可以參考 OpenSUSE: Setting Up a Secure Web Server with SSL

再來修改 /etc/sysconfig/apache2 的設定,
APACHE_START_TIMEOUT="10" 
APACHE_SERVER_FLAGS="SSL"

複製 /etc/apache2/vhost.d/vhost-ssl.template 到 /etc/apache2/vhost.d/vhost-ssl.conf,主要的設定如下:
<IfDefine SSL>
<IfDefine !NOSSL>

##
## SSL Virtual Host Context
##

<VirtualHost *:443>

    #  General setup for the virtual host
    DocumentRoot "/srv/www/htdocs"
    #ServerName www.example.com:443
    #ServerAdmin webmaster@example.com
    ErrorLog /var/log/apache2/error_log
    TransferLog /var/log/apache2/access_log

    #   SSL Engine Switch:
    #   Enable/Disable SSL for this virtual host.
    SSLEngine on

    #   You can use per vhost certificates if SNI is supported.
    SSLCertificateFile /etc/apache2/ssl.crt/server.crt
    SSLCertificateKeyFile /etc/apache2/ssl.key/server.key
    #SSLCertificateChainFile /etc/apache2/ssl.crt/vhost-example-chain.crt

    #   Per-Server Logging:
    #   The home of a custom SSL log file. Use this when you want a
    #   compact non-error SSL logfile on a virtual host basis.
    CustomLog /var/log/apache2/ssl_request_log   ssl_combined

</VirtualHost>

</IfDefine>
</IfDefine>

接下來讓 main site 使用 HTTPS,編輯 /etc/apache2/default-server.conf,加入下面的設定:
IncludeOptional /etc/apache2/conf.d/*.conf
IncludeOptional /etc/apache2/vhosts.d/*.conf

接下來重新開啟 Apache 2,
sudo systemctl restart apache2

瀏覽 https://localhost/hello.rvt, 如果有看到訊息表示成功安裝(PS. 因為是 Self-Signed Certificates,所以 Firefox 會看到警告訊息)。

若要強制使用 SSL,需要開啟 mod_rewrite 才行。
sudo a2enmod rewrite

再來修改 /etc/apache2/vhost.d/vhost-ssl.conf
<VirtualHost *:80>
    RewriteEngine On
    RewriteCond %{HTTPS} !=on
    RewriteRule (.*) https://%{SERVER_NAME}/$1 [R,L] 
</VirtualHost>

<VirtualHost *:443>

    #  General setup for the virtual host
    DocumentRoot "/srv/www/htdocs"
    #ServerName www.example.com:443
    #ServerAdmin webmaster@example.com
    ErrorLog /var/log/apache2/error_log
    TransferLog /var/log/apache2/access_log

    #   SSL Engine Switch:
    #   Enable/Disable SSL for this virtual host.
    SSLEngine on

    #   You can use per vhost certificates if SNI is supported.
    SSLCertificateFile /etc/apache2/ssl.crt/server.crt
    SSLCertificateKeyFile /etc/apache2/ssl.key/server.key
    #SSLCertificateChainFile /etc/apache2/ssl.crt/vhost-example-chain.crt

    #   Per-Server Logging:
    #   The home of a custom SSL log file. Use this when you want a
    #   compact non-error SSL logfile on a virtual host basis.
    CustomLog /var/log/apache2/ssl_request_log   ssl_combined

</VirtualHost>

* 2017/11/18 更新:更新 rewrite 的規則

接下來重新開啟 Apache 2,
sudo systemctl restart apache2

這樣就會強制都使用 HTTPS 瀏覽。