diff --git a/content/post/install-nvidia-driver-cuda-pgstrom-in-ubuntu-1804.md b/content/post/install-nvidia-driver-cuda-pgstrom-in-ubuntu-1804.md new file mode 100644 index 00000000..2ec42d46 --- /dev/null +++ b/content/post/install-nvidia-driver-cuda-pgstrom-in-ubuntu-1804.md @@ -0,0 +1,255 @@ +--- +title: "[筆記] 在ubuntu 18.04 下安裝nvidia 顯示卡驅動程式以及 pgstrom / Install Nvidia Driver Cuda Pgstrom in Ubuntu 1804" +date: 2019-08-20T14:51:54+08:00 +noSummary: false +featuredImage: "https://h.cowbay.org/images/post-default-7.jpg" +categories: ['筆記'] +tags: ['nvidia'] +author: "Eric Chang" +keywords: + - nvidia + - cuda + - pgstrom +--- + +因為老闆說要試試看用GPU 來跑postgresql 威力 + +手邊剛好有一張 geforce gt 720 + +一開始沒想太多，看到有這張卡的驅動程式，然後CUDA也有支援 + +就直接從桌機拔下來，接去LAB Server ，然後就開始一連串的難關了... + + + +整個過程大致上分為四個步驟 + +### 安裝 nvidia driver +### 安裝 CUDA +### 安裝 postgresql +### 安裝 pgstrom + + +************ +#### 安裝 nvidia driver + +試過幾種方法，最後還是覺得用apt來安裝比較妥當 +先新增repository、update、裝driver +``` +sudo add-apt-repository ppa:graphics-drivers/ppa +sudo apt update +``` +然後用這個指令 +``` +ubuntu-drivers devices +``` +看一下現在的驅動程式狀態 +``` +administrator@hqdc032:~/pg-strom$ ubuntu-drivers devices +== /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0 == +modalias : pci:v000010DEd00001288sv0000174Bsd0000326Bbc03sc00i00 +vendor : NVIDIA Corporation +model : GK208B [GeForce GT 720] +driver : nvidia-driver-410 - third-party free +driver : nvidia-driver-418 - third-party free +driver : nvidia-340 - distro non-free +driver : nvidia-driver-430 - third-party free recommended +driver : nvidia-driver-390 - third-party free +driver : nvidia-driver-415 - third-party free +driver : xserver-xorg-video-nouveau - distro free builtin +``` +我這張卡，可以裝到 430 的版本， +接下來就繼續安裝驅動程式、裝完之後重開機 +``` +sudo apt install nvidia-driver-430 +sudo reboot +``` +這時候，應該可以看到一些基本資訊 +``` +lsmod|grep nvidia +``` + +大概長這樣 +``` +nvidia_uvm 798720 0 +nvidia_drm 45056 3 +nvidia_modeset 1093632 7 nvidia_drm +nvidia 18194432 258 nvidia_uvm,nvidia_modeset +drm_kms_helper 172032 1 nvidia_drm +drm 401408 6 drm_kms_helper,nvidia_drm +ipmi_msghandler 53248 2 ipmi_devintf,nvidia +``` + +到這邊 nvidia 驅動程式安裝完成，接下來安裝 cuda + +#### 安裝 CUDA +同樣採用官網下載deb 回來安裝的方法 + +到這邊 https://developer.nvidia.com/cuda-downloads + +依照自己的系統選擇 + +我選擇 Linux -- x86_64 -- Ubuntu -- 18.04 -- deb(local) + +畫面上就會有安裝步驟，照著做就沒問題了 +``` +wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin +sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600 +wget http://developer.download.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda-repo-ubuntu1804-10-1-local-10.1.243-418.87.00_1.0-1_amd64.deb +sudo dpkg -i cuda-repo-ubuntu1804-10-1-local-10.1.243-418.87.00_1.0-1_amd64.deb +sudo apt-key add /var/cuda-repo-10-1-local-10.1.243-418.87.00/7fa2af80.pub +sudo apt-get update +sudo apt-get -y install cuda +``` + +一樣，安裝完成後重新開機，然後來編譯一個 deviceQuery 的小程式來看看資訊 + +``` +cd /usr/local/cuda-10.1/samples/1_Utilities/deviceQuery +sudo make +``` + +會產生一個叫 deviceQuery 的執行檔，執行後，會有相關資訊 + +``` +administrator@hqdc032:/usr/local/cuda-10.1/samples/1_Utilities/deviceQuery$ ./deviceQuery +./deviceQuery Starting... + + CUDA Device Query (Runtime API) version (CUDART static linking) + +Detected 1 CUDA Capable device(s) + +Device 0: "GeForce GT 720" + CUDA Driver Version / Runtime Version 10.1 / 10.1 + CUDA Capability Major/Minor version number: 3.5 + Total amount of global memory: 1996 MBytes (2093416448 bytes) + ( 1) Multiprocessors, (192) CUDA Cores/MP: 192 CUDA Cores + GPU Max Clock rate: 797 MHz (0.80 GHz) + Memory Clock rate: 900 Mhz + Memory Bus Width: 64-bit + L2 Cache Size: 524288 bytes + Maximum Texture Dimension Size (x,y,z) 1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096) + Maximum Layered 1D Texture Size, (num) layers 1D=(16384), 2048 layers + Maximum Layered 2D Texture Size, (num) layers 2D=(16384, 16384), 2048 layers + Total amount of constant memory: 65536 bytes + Total amount of shared memory per block: 49152 bytes + Total number of registers available per block: 65536 + Warp size: 32 + Maximum number of threads per multiprocessor: 2048 + Maximum number of threads per block: 1024 + Max dimension size of a thread block (x,y,z): (1024, 1024, 64) + Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535) + Maximum memory pitch: 2147483647 bytes + Texture alignment: 512 bytes + Concurrent copy and kernel execution: Yes with 1 copy engine(s) + Run time limit on kernels: Yes + Integrated GPU sharing Host Memory: No + Support host page-locked memory mapping: Yes + Alignment requirement for Surfaces: Yes + Device has ECC support: Disabled + Device supports Unified Addressing (UVA): Yes + Device supports Compute Preemption: No + Supports Cooperative Kernel Launch: No + Supports MultiDevice Co-op Kernel Launch: No + Device PCI Domain ID / Bus ID / location ID: 0 / 1 / 0 + Compute Mode: + < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) > + +deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 10.1, CUDA Runtime Version = 10.1, NumDevs = 1 +Result = PASS +``` + +到這邊， CUDA 也安裝完成，再來是簡單的 postgresql 11 + +#### 安裝 postgresql 11 + +步驟差不多，就是新增repository，然後選擇要安裝的套件，不過套件的選擇和平常安裝postgresql 不太一樣 + +``` +sudo apt install wget ca-certificates +wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add - +sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" >> /etc/apt/sources.list.d/pgdg.list' +sudo apt update +sudo apt install postgresql-client-11 postgresql-11 postgresql-server-dev-11 postgresql-common libicu-dev +``` + +跑完之後，就直接啟動 postgresql service 就可以了 + +再來是最麻煩的 pgstorm + +#### pgstorm + +話說，這軟體到底叫啥名字？ pgstrom , pg-strom ? 看起來就像是拼錯字啊！ =.= + +https://github.com/heterodb/pg-strom + +先 git clone 回來，然後make、make install + +講是很簡單，但是一開始碰到很多問題，有去github 跟開發團隊回報，幸好有回覆我.. + +總之，目前在ubuntu 18.04 + postgresql-11 的環境下編譯是沒有問題了 + +``` +git clone https://github.com/heterodb/pg-strom.git +cd pg-strom +make && sudo make install +``` + +再來設定一下 postgresql + +#### postgresql 相關設定 + +需要修改一下postgresql.conf，來指定載入 pgstrom 的 library + +官方是這麼說的 + +``` +PG-Strom module must be loaded on startup of the postmaster process by the shared_preload_libraries. Unable to load it on demand. Therefore, you must add the configuration below. +``` +修改 /etc/postgresql/11/main/postgresql.conf +加入一行 +``` +shared_preload_libraries = '$libdir/pg_strom' +``` + +然後還有其他三個要修改，不過這個不改不會影響pgstrom 的啟動 + +看狀況要不要修改吧 + +``` +max_worker_processes = 100 +shared_buffers = 10GB +work_mem = 1GB +``` + +修改完後，重新啟動 postgresql service 有沒有很感動！？我終於可以享受用GPU跑SQL Query 的快感啦！！！ + +咦？？等等，為什麼postgresql service 沒起來！？ + +看一下 /var/log/syslog + +``` +Aug 20 14:23:43 hqdc032 postgresql@11-main[11801]: Error: /usr/lib/postgresql/11/bin/pg_ctl /usr/lib/postgresql/11/bin/pg_ctl start -D /var/lib/postgresql/11/main -l /var/log/postgresql/postgresql-11-main.log -s -o -c config_file="/etc/postgresql/11/main/postgresql.conf" exited with status 1: +Aug 20 14:23:43 hqdc032 postgresql@11-main[11801]: 2019-08-20 14:23:43.538 CST [11806] LOG: PG-Strom version 2.2 built for PostgreSQL 11 +Aug 20 14:23:43 hqdc032 postgresql@11-main[11801]: 2019-08-20 14:23:43.565 CST [11806] LOG: PG-Strom: GPU0 GeForce GT 720 - CC 3.5 is not supported +Aug 20 14:23:43 hqdc032 postgresql@11-main[11801]: 2019-08-20 14:23:43.565 CST [11806] FATAL: PG-Strom: no supported GPU devices found +Aug 20 14:23:43 hqdc032 postgresql@11-main[11801]: 2019-08-20 14:23:43.565 CST [11806] LOG: database system is shut down +Aug 20 14:23:43 hqdc032 postgresql@11-main[11801]: pg_ctl: could not start server +Aug 20 14:23:43 hqdc032 postgresql@11-main[11801]: Examine the log output. +Aug 20 14:23:43 hqdc032 systemd[1]: postgresql@11-main.service: Can't open PID file /run/postgresql/11-main.pid (yet?) after start: No such file or directory +Aug 20 14:23:43 hqdc032 systemd[1]: postgresql@11-main.service: Failed with result 'protocol'. +Aug 20 14:23:43 hqdc032 systemd[1]: Failed to start PostgreSQL Cluster 11-main. +``` + +啊幹！pg-strom 不支援這張GT 720啦！ + +https://github.com/heterodb/pg-strom/wiki/001:-GPU-Availability-Matrix + +簡單說，就是至少從 GTX 1080 起跳，其他都不用想了 + +幹，花了兩天的時間在弄這東西，結果明明一開始就應該要先檢查的相容列表卻沒有檢查... + +好了，現在就看准不准我買一張 GTX 1080 來測試了.... + +只是這價格嘛...嗯咳，不是我該煩惱的問題了.. + diff --git a/content/post/pgbarman-in-ubuntu-1804-postgresql-10-via-ssh.md b/content/post/pgbarman-in-ubuntu-1804-postgresql-10-via-ssh.md new file mode 100644 index 00000000..02777470 --- /dev/null +++ b/content/post/pgbarman-in-ubuntu-1804-postgresql-10-via-ssh.md @@ -0,0 +1,166 @@ +--- +title: "[筆記] 在Ubuntu 18.04 下 透過 pgbarman rsync/ssh backup 備份 postgresql 10 / backup postgresql 10 with pgbarman via ssh/rsync in ubuntu 18.04" +date: 2019-08-23T14:54:13+08:00 +noSummary: false +featuredImage: "https://h.cowbay.org/images/post-default-4.jpg" +categories: ['筆記'] +tags: ['postgresql','pgbarman'] +author: "Eric Chang" +keywords: + - pgbarman + - postgresql +--- + +這篇繼續講 pgbarman 透過 rsync/ssh 來備份 postgresql 資料庫的方式 + + + +其實呢，透過 ssh 的方式來做備份，雖然最後有弄出來，但是我不知道到底做了什麼事才搞定 + +也許要重新安裝一台來測試看看 + +所以就簡單說一下邏輯、記得的指令、還有最後的config + +##### ssh + +在db server 上，讓 barman server 可以用barman 帳號透過ssh key 登入 postgres 帳號 +然後在 barman server 上，讓 db server 可以用postgres帳號透過 ssh key 登入 barman 帳號 + +##### barman.d + +在/etc/barman.d/ 底下新增一個hqs019-ssh.conf + +內容如下 +``` +[hqs019-ssh] +description = "hqs019 (via SSH)" +ssh_command = ssh postgres@hqs019 +conninfo = host=hqs019 user=barman dbname=database_name +backup_method = rsync +reuse_backup = link +parallel_jobs = 5 +archiver = on +``` + +##### postgresql.conf + +新增跟barman有關的設定如下 +``` +### for barman test +wal_level = 'archive' +archive_mode = on +archive_command = 'rsync -a %p barman@192.168.11.192:/var/lib/barman/hqs019-ssh/incoming/%f' +``` + +理論上這樣就可以了，但是實際上碰到很多問題 + +主要都是這個錯誤訊息 + +``` +WAL archive: FAILED (please make sure WAL shipping is setup) +``` + +碰到這狀況，google 都會告訴你，在 postgresql.conf 裡面 archive_command 的路徑，要和 barman server 上的設定一致 +![](https://i.imgur.com/3dMdytn.png) + +但是我的路徑就沒有問題，還是一直跳這個錯誤 + +接著我又在 barman server 這邊做了這些事情 + +``` +barman check hqs019-ssh +barman check hqs019-ssh +barman check hqs019-ssh +ssh postgres@hqs019 +barman check hqs019-ssh +barman show-server hqs019-ssh |grep incoming_wals_directory +barman show-server all +barman cron +barman switch-xlog hqs019-ssh +barman check hqs019-ssh +psql -c 'SELECT version()' -U postgres -h hqs019 +psql -c 'SELECT version()' -U barman -h hqs019 +psql -c 'SELECT version()' -U barman -h hqs019 -d database_name +barman check hqs019-ssh +barman check hqs019-ssh +barman bachup hqs019-ssh +barman backup hqs019-ssh +barman list-backup hqs019-ssh +df -h +barman backup hqs019-ssh +barman show-server hqs019 +barman check hqs019 + +``` +前面幾次 barman check 一直都不通過 + +然後在 barman cron / barman switch-xlog hqs019-ssh (其實這個動作我做過好多次) + +想說確認一下psql 連接是不是正確(也的確正確無誤) + +接著第一次 check 還是FAILED 喔，過沒多久我再跑一次 check 又正常了... WTF !!! + +只要check 正常，接著跑 backup 應該就會很順利 +``` +barman@barman:~$ barman backup hqs019-ssh +Starting backup using rsync-exclusive method for server hqs019-ssh in /var/lib/barman/hqs019-ssh/base/20190823T113229 +Backup start at LSN: 264/B7000028 (0000000100000264000000B7, 00000028) +This is the first backup for server hqs019-ssh +WAL segments preceding the current backup have been found: + 0000000100000264000000B5 from server hqs019-ssh has been removed +Starting backup copy via rsync/SSH for 20190823T113229 (5 jobs) +Copy done (time: 1 hour, 5 minutes, 39 seconds) +This is the first backup for server hqs019-ssh +WAL segments preceding the current backup have been found: + 0000000100000264000000B6 from server hqs019-ssh has been removed +Asking PostgreSQL server to finalize the backup. +Backup size: 132.9 GiB. Actual size on disk: 132.9 GiB (-0.00% deduplication ratio). +Backup end at LSN: 264/B7000130 (0000000100000264000000B7, 00000130) +Backup completed (start time: 2019-08-23 11:32:30.078310, elapsed time: 1 hour, 5 minutes, 43 seconds) +Processing xlog segments from file archival for hqs019-ssh + 0000000100000264000000B7 + 0000000100000264000000B7.00000028.backup +barman@barman:~$ +``` + +檢查一下狀態 +``` +barman@barman:~$ barman list-backup hqs019-ssh +hqs019-ssh 20190823T113229 - Thu Aug 22 20:38:13 2019 - Size: 132.9 GiB - WAL Size: 0 B (tablespaces: tablespace_a:/var/lib/postgresql/10/main/tablespace_A, tablespace_b:/var/lib/postgresql/10/main/tablespace_B) +``` + +然後為了驗證是不是跑增量備份，所以再執行一次 backup +``` +barman@barman:~$ barman backup hqs019-ssh +Starting backup using rsync-exclusive method for server hqs019-ssh in /var/lib/barman/hqs019-ssh/base/20190823T132124 +Backup start at LSN: 264/B9000028 (0000000100000264000000B9, 00000028) +Starting backup copy via rsync/SSH for 20190823T132124 (5 jobs) +Copy done (time: 5 seconds) +Asking PostgreSQL server to finalize the backup. +Backup size: 132.9 GiB. Actual size on disk: 14.2 KiB (-100.00% deduplication ratio). +Backup end at LSN: 264/B9000130 (0000000100000264000000B9, 00000130) +Backup completed (start time: 2019-08-23 13:21:24.455819, elapsed time: 9 seconds) +Processing xlog segments from file archival for hqs019-ssh + 0000000100000264000000B8 + 0000000100000264000000B9 + 0000000100000264000000B9.00000028.backup +barman@barman:~$ +``` + +可以發現第一次跑了一個多小時，第二次只跑了五秒鐘 (因為資料庫根本沒異動) + +到這邊雖然功能驗證沒有問題，可是不知道怎麼弄出來的，還是讓我很阿雜.. + +應該會再找時間來重作一台，然後順便測試看看restore + + +update + +剛剛又做了一次測試，config 都一樣 + +果然要先做 barman cron/barman switch-xlog hqs019-ssh + +然後再做 barman check 就可以通過了 + +在文件裡面很少提到這部份，筆記一下，用ansible 去跑的時候才不會忘記 + diff --git a/content/post/pgbarman-in-ubuntu-1804-postgresql-10.md b/content/post/pgbarman-in-ubuntu-1804-postgresql-10.md new file mode 100644 index 00000000..443171db --- /dev/null +++ b/content/post/pgbarman-in-ubuntu-1804-postgresql-10.md @@ -0,0 +1,330 @@ +--- +title: "[筆記] 在Ubuntu 18.04 下 透過 pgbarman streaming backup 備份 postgresql 10/ backup postgresql 10 with pgbarman straming backup in ubuntu 18.04" +date: 2019-08-23T13:53:40+08:00 +noSummary: false +featuredImage: "https://h.cowbay.org/images/post-default-4.jpg" +categories: ['筆記'] +tags: ['postgresql','pgbarman'] +author: "Eric Chang" +keywords: + - pgbarman + - postgresql +--- + +很久以前就有看到這個用來備份postgresql 的 pgbarman + +https://www.pgbarman.org/ + +前幾天老闆在slack 上面又提到，所以這次就花了點時間來玩玩看 + +不過呢，雖然有弄起來，但是還真不知道有些問題是怎麼解決的... + + + +pgbarman 的備份有分兩種 + +streaming && rsync/SSH + +原理就不說了，我一直沒搞懂 postgresql 的 streaming .. + +依照官方網站上的說法，比較推薦 streaming 備份的方式 + +原因是，設定相對簡單，WTF ! + +``` +On a general basis, starting from Barman 2.0, backup over streaming replication is the recommended setup for PostgreSQL 9.4 or higher +The reason why we recommend streaming backup is that, based on our experience, it is easier to setup than the traditional one +``` + +事實上呢，設定的確是很簡單，可是有個致命的缺點 +``` +Because Barman transparently makes use of pg_basebackup, features such as incremental backup, parallel backup, deduplication, and network compression are currently not available. In this case, bandwidth limitation has some restrictions - compared to the traditional method via rsync. +``` + +如果要做差異/增量備份， streaming backup 不能做 + +所以每次備份都是完整備份，也因此，barman server 需要準備很大的硬碟空間 + +以我測試的資料庫來說，一次備份，目前是133G ，如果一天四次，保留七天 + +就需要 133 x 4 x 7 = 3724G + +咦，這樣看起來，其實也還好啦 XDD + +______ + +現在開始設定的部份 +#### 設定 postgresql server +IP: 192.168.11.19 +hostname: hqs019 + + +##### 在postgresql 建立相關帳號 + +streaming backup 需要先在postgresql Server 上建立一個具有 superuser 權限的帳號 + +以及一個用來做replication 的資料庫帳號 + +這裡就簡單帶過 + +``` +sudo su - postgres +psql + +create user barman with login superuser login password 'barmanpassword'; +CREATE ROLE stream_barman WITH REPLICATION PASSWORD 'password' LOGIN; +``` + +##### 鄉改 pg_hba.conf + +然後修改 pg_hba.conf，加入底下兩行 +``` +# for barman test +host database_name barman 192.168.11.192/32 md5 +host replication stream_barman 192.168.11.192/32 md5 +``` +當然，如果不考慮安全性問題， md5 直接改成用 trust ，可以省去一些麻煩。 + +##### 修改 postgresql.conf + +接著修改postgresql.conf +``` +### for barman test +max_wal_senders = 5 +max_replication_slots = 3 +wal_level = 'archive' +archive_mode = on +``` + +重起 postgresql service + +#### 設定barman server + +IP: 192.168.11.192 +hostname: barman + +##### 安裝 barman + +barman 在18.04 中，已經被放到標準repository 中 + +所以只要直接 +``` +sudo apt install barman +``` +就可以了 + + +##### 設定 barman.conf + +安裝完成後，在/etc/barman.d/ 底下會有兩個範例檔案 +``` +streaming-server.conf-template +ssh-server.conf-template +``` +複製 streaming-server 檔案 +``` +sudo cp /etc/barman.d/streaming-server.conf-template /etc/barman.d/hqs019.conf +``` +內容如下 +``` +[hqs019] +description = "hqs019 " +conninfo = host=192.168.11.19 user=barman dbname=database_name password=barmanpassword +streaming_conninfo = host=192.168.11.19 user=stream_barman dbname=database_name password=password +backup_method = postgres +retention_policy_mode = auto +streaming_archiver = on +slot_name = barman +``` + +接著修改 /etc/barman.conf + +``` +[barman] +barman_user = barman +configuration_files_directory = /etc/barman.d +barman_home = /var/lib/barman +log_file = /var/log/barman/barman.log +log_level = DEBUG +compression = gzip +immediate_checkpoint = true +basebackup_retry_times = 3 +basebackup_retry_sleep = 30 +last_backup_maximum_age = 1 DAYS +``` + +基本上這樣就設定完成了 + +##### 檢查設定 + +barman 有一些指令可以用來檢查目前的設定 + +barman show-server hqs019 可以看到所有的設定，這裡的 hqs019 跟 barman.d/hqs019.conf 裡面用"[ ]" 包起來的名稱要一致 +``` +barman@barman:~$ barman show-server hqs019 +Server hqs019: + active: True + archiver: False + archiver_batch_size: 0 + backup_directory: /var/lib/barman/hqs019 + backup_method: postgres + backup_options: BackupOptions(['concurrent_backup']) + bandwidth_limit: None + barman_home: /var/lib/barman + barman_lock_directory: /var/lib/barman + basebackup_retry_sleep: 30 + basebackup_retry_times: 3 + basebackups_directory: /var/lib/barman/hqs019/base + check_timeout: 30 + compression: gzip + config_file: /etc/postgresql/10/main/postgresql.conf + connection_error: None + conninfo: host=192.168.11.19 user=barman dbname=database_name password=barmanpassword + current_size: 142740768562 + current_xlog: 0000000100000264000000BA + custom_compression_filter: None + custom_decompression_filter: None + data_directory: /database + description: hqs019 + disabled: False + errors_directory: /var/lib/barman/hqs019/errors + hba_file: /etc/postgresql/10/main/pg_hba.conf + ident_file: /etc/postgresql/10/main/pg_ident.conf + immediate_checkpoint: True + incoming_wals_directory: /var/lib/barman/hqs019/incoming + is_in_recovery: False + is_superuser: True + last_backup_maximum_age: 1 day (WARNING! latest backup is No available backups old) + max_incoming_wals_queue: None + minimum_redundancy: 0 + msg_list: [] + name: hqs019 + network_compression: False + parallel_jobs: 1 + path_prefix: None + pg_basebackup_bwlimit: True + pg_basebackup_compatible: True + pg_basebackup_installed: True + pg_basebackup_path: /usr/bin/pg_basebackup + pg_basebackup_tbls_mapping: True + pg_basebackup_version: 10.10-0ubuntu0.18.04.1) + pg_receivexlog_compatible: True + pg_receivexlog_installed: True + pg_receivexlog_path: /usr/bin/pg_receivewal + pg_receivexlog_supports_slots: True + pg_receivexlog_synchronous: False + pg_receivexlog_version: 10.10-0ubuntu0.18.04.1) + pgespresso_installed: False + post_archive_retry_script: None + post_archive_script: None + post_backup_retry_script: None + post_backup_script: None + pre_archive_retry_script: None + pre_archive_script: None + pre_backup_retry_script: None + pre_backup_script: None + recovery_options: RecoveryOptions([]) + replication_slot: Record(slot_name='barman', active=True, restart_lsn='264/BA000000') + replication_slot_support: True + retention_policy: None + retention_policy_mode: auto + reuse_backup: None + server_txt_version: 10.10 + slot_name: barman + ssh_command: None + streaming: True + streaming_archiver: True + streaming_archiver_batch_size: 0 + streaming_archiver_name: barman_receive_wal + streaming_backup_name: barman_streaming_backup + streaming_conninfo: host=192.168.11.19 user=stream_barman dbname=database_name password=password + streaming_supported: True + streaming_wals_directory: /var/lib/barman/hqs019/streaming + synchronous_standby_names: [''] + systemid: 6688476041000599317 + tablespace_bandwidth_limit: None + timeline: 1 + wal_level: replica + wal_retention_policy: main + wals_directory: /var/lib/barman/hqs019/wals + xlogpos: 264/BA000F08 +``` + +然後用 barman check hqs019 來檢查config 有沒有問題 + +``` +barman@barman:~$ barman check hqs019 +Server hqs019: + PostgreSQL: OK + is_superuser: OK + PostgreSQL streaming: OK + wal_level: OK + replication slot: OK + directories: OK + retention policy settings: OK + backup maximum age: FAILED (interval provided: 1 day, latest backup age: No available backups) + compression settings: OK + failed backups: OK (there are 0 failed backups) + minimum redundancy requirements: OK (have 0 backups, expected at least 0) + pg_basebackup: OK + pg_basebackup compatible: OK + pg_basebackup supports tablespaces mapping: OK + pg_receivexlog: OK + pg_receivexlog compatible: OK + receive-wal running: OK + archiver errors: OK +barman@barman:~$ +``` +那個backup maximum age FAILED 不用管他，因為都還沒跑過備份，這邊錯誤是正常的 + +其他都OK 的話，就可以開始備份了 + +barman backup hqs019 + +``` +barman@ubuntu:~$ barman backup hqs019 +Starting backup using postgres method for server hqs019 in /var/lib/barman/hqs019/base/20190823T082258 +Backup start at LSN: 264/A10001A8 (0000000100000264000000A1, 000001A8) +Starting backup copy via pg_basebackup for 20190823T082258 +WARNING: pg_basebackup does not copy the PostgreSQL configuration files that reside outside PGDATA. Please manually backup the following files: + /etc/postgresql/10/main/postgresql.conf + /etc/postgresql/10/main/pg_hba.conf + /etc/postgresql/10/main/pg_ident.conf + +Copy done (time: 1 hour, 6 minutes, 28 seconds) +Finalising the backup. +Backup size: 133.0 GiB +Backup end at LSN: 264/A3000060 (0000000100000264000000A3, 00000060) +Backup completed (start time: 2019-08-23 08:22:58.116372, elapsed time: 1 hour, 6 minutes, 28 seconds) +Processing xlog segments from streaming for hqs019 + 0000000100000264000000A2 + +barman@ubuntu:~$ +``` + +跑完可以用 barman list-backup hqs019 檢查 +``` +barman@ubuntu:~$ barman list-backup hqs019 +hqs019 20190823T082258 - Thu Aug 22 17:29:26 2019 - Size: 133.0 GiB - WAL Size: 0 B (tablespaces: tablespace_a:/var/lib/postgresql/10/main/tablespace_A, tablespace_b:/var/lib/postgresql/10/main/tablespace_B) +``` + +要刪除的話，要加入 backupID + +``` +barman@ubuntu:~$ barman delete hqs019 20190822T171355 +Deleting backup 20190822T171355 for server hqs019 +Delete associated WAL segments: + 00000001000002640000009F + 0000000100000264000000A0 + 0000000100000264000000A1 +Deleted backup 20190822T171355 (start time: Fri Aug 23 09:36:43 2019, elapsed time: 3 seconds) + +``` + +restore 的部份，暫時沒有測試，我想應該是要找時間測試看看怎麼還原才對 + +不過呢，前面有提到，用streaming backup ，每一次備份都是完整備份，非常的佔用空間、時間、頻寬 + +所以還是要來試試看用rsync/SSH 備份的機制 + + diff --git a/public/404.html b/public/404.html index 5b4a9122..191b2cad 100644 --- a/public/404.html +++ b/public/404.html @@ -117,31 +117,31 @@ @@ -154,7 +154,7 @@