add auto fetch ssl certs
This commit is contained in:
@@ -13,7 +13,7 @@
|
||||
"articleSection" : "post",
|
||||
"name" : "[筆記] 在Ubuntu 18.04 下 透過 pgbarman streaming backup 備份 postgresql 10\/ backup postgresql 10 with pgbarman straming backup in ubuntu 18.04",
|
||||
"headline" : "[筆記] 在Ubuntu 18.04 下 透過 pgbarman streaming backup 備份 postgresql 10\/ backup postgresql 10 with pgbarman straming backup in ubuntu 18.04",
|
||||
"description" : "\x3cp\x3e很久以前就有看到這個用來備份postgresql 的 pgbarman\x3c\/p\x3e\n\n\x3cp\x3e\x3ca href=\x22https:\/\/www.pgbarman.org\/\x22\x3ehttps:\/\/www.pgbarman.org\/\x3c\/a\x3e\x3c\/p\x3e\n\n\x3cp\x3e前幾天老闆在slack 上面又提到，所以這次就花了點時間來玩玩看\x3c\/p\x3e\n\n\x3cp\x3e不過呢，雖然有弄起來，但是還真不知道有些問題是怎麼解決的\x26hellip;\x3c\/p\x3e",
|
||||
"description" : "\x3cp\x3e很久以前就有看到這個用來備份postgresql 的 pgbarman\x3c\/p\x3e\n\x3cp\x3e\x3ca href=\x22https:\/\/www.pgbarman.org\/\x22\x3ehttps:\/\/www.pgbarman.org\/\x3c\/a\x3e\x3c\/p\x3e\n\x3cp\x3e前幾天老闆在slack 上面又提到，所以這次就花了點時間來玩玩看\x3c\/p\x3e\n\x3cp\x3e不過呢，雖然有弄起來，但是還真不知道有些問題是怎麼解決的\x26hellip;\x3c\/p\x3e",
|
||||
"inLanguage" : "en",
|
||||
"author" : "Eric Chang",
|
||||
"creator" : "Eric Chang",
|
||||
@@ -45,9 +45,9 @@
|
||||
|
||||
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.3.1/css/all.css" integrity="sha384-mzrmE5qonljUremFsqc01SB46JvROS7bZs3IO2EmfFsd15uHvIt+Y8vEf7N7fWAU" crossorigin="anonymous">
|
||||
|
||||
<link href="https://h.cowbay.org/css/style.css?v=1626744134" rel="stylesheet" id="theme-stylesheet" type='text/css' media='all'>
|
||||
<link href="https://h.cowbay.org/css/style.css?v=1629951055" rel="stylesheet" id="theme-stylesheet" type='text/css' media='all'>
|
||||
|
||||
<link href="https://h.cowbay.org/css/custom.css?v=1626744134" rel="stylesheet" type='text/css' media='all'>
|
||||
<link href="https://h.cowbay.org/css/custom.css?v=1629951055" rel="stylesheet" type='text/css' media='all'>
|
||||
<link rel="shortcut icon" href="https://h.cowbay.org/img/favicon.ico" type="image/x-icon">
|
||||
<link rel="icon" href="https://h.cowbay.org/img/favicon.ico" type="image/x-icon">
|
||||
|
||||
@@ -81,10 +81,6 @@ if (!doNotTrack) {
|
||||
|
||||
<ul id="menu-secondary-items" class="menu-secondary-items">
|
||||
|
||||
<li class="menu-item menu-item-type-taxonomy menu-item-object-category">
|
||||
<a href="/categories/"></a>
|
||||
</li>
|
||||
|
||||
<li class="menu-item menu-item-type-taxonomy menu-item-object-category">
|
||||
<a href="/categories/ansible">ansible</a>
|
||||
</li>
|
||||
@@ -309,121 +305,66 @@ if (!doNotTrack) {
|
||||
<div class="entry-content">
|
||||
<article>
|
||||
<p>很久以前就有看到這個用來備份postgresql 的 pgbarman</p>
|
||||
|
||||
<p><a href="https://www.pgbarman.org/">https://www.pgbarman.org/</a></p>
|
||||
|
||||
<p>前幾天老闆在slack 上面又提到，所以這次就花了點時間來玩玩看</p>
|
||||
|
||||
<p>不過呢，雖然有弄起來，但是還真不知道有些問題是怎麼解決的…</p>
|
||||
|
||||
<p>pgbarman 的備份有分兩種</p>
|
||||
|
||||
<p>streaming && rsync/SSH</p>
|
||||
|
||||
<p>原理就不說了，我一直沒搞懂 postgresql 的 streaming ..</p>
|
||||
|
||||
<p>依照官方網站上的說法，比較推薦 streaming 備份的方式</p>
|
||||
|
||||
<p>原因是，設定相對簡單，WTF !</p>
|
||||
|
||||
<pre><code>On a general basis, starting from Barman 2.0, backup over streaming replication is the recommended setup for PostgreSQL 9.4 or higher
|
||||
The reason why we recommend streaming backup is that, based on our experience, it is easier to setup than the traditional one
|
||||
</code></pre>
|
||||
|
||||
<p>事實上呢，設定的確是很簡單，可是有個致命的缺點</p>
|
||||
|
||||
</code></pre><p>事實上呢，設定的確是很簡單，可是有個致命的缺點</p>
|
||||
<pre><code>Because Barman transparently makes use of pg_basebackup, features such as incremental backup, parallel backup, deduplication, and network compression are currently not available. In this case, bandwidth limitation has some restrictions - compared to the traditional method via rsync.
|
||||
</code></pre>
|
||||
|
||||
<p>如果要做差異/增量備份， streaming backup 不能做</p>
|
||||
|
||||
</code></pre><p>如果要做差異/增量備份， streaming backup 不能做</p>
|
||||
<p>所以每次備份都是完整備份，也因此，barman server 需要準備很大的硬碟空間</p>
|
||||
|
||||
<p>以我測試的資料庫來說，一次備份，目前是133G ，如果一天四次，保留七天</p>
|
||||
|
||||
<p>就需要 133 x 4 x 7 = 3724G</p>
|
||||
|
||||
<p>咦，這樣看起來，其實也還好啦 XDD</p>
|
||||
|
||||
<hr />
|
||||
|
||||
<hr>
|
||||
<p>現在開始設定的部份</p>
|
||||
|
||||
<h4 id="設定-postgresql-server">設定 postgresql server</h4>
|
||||
|
||||
<p>IP: 192.168.11.19
|
||||
hostname: hqs019</p>
|
||||
|
||||
<h5 id="在postgresql-建立相關帳號">在postgresql 建立相關帳號</h5>
|
||||
|
||||
<p>streaming backup 需要先在postgresql Server 上建立一個具有 superuser 權限的帳號</p>
|
||||
|
||||
<p>以及一個用來做replication 的資料庫帳號</p>
|
||||
|
||||
<p>這裡就簡單帶過</p>
|
||||
|
||||
<pre><code>sudo su - postgres
|
||||
psql
|
||||
|
||||
create user barman with login superuser login password 'barmanpassword';
|
||||
CREATE ROLE stream_barman WITH REPLICATION PASSWORD 'password' LOGIN;
|
||||
</code></pre>
|
||||
|
||||
<h5 id="鄉改-pg-hba-conf">鄉改 pg_hba.conf</h5>
|
||||
|
||||
</code></pre><h5 id="鄉改-pg_hbaconf">鄉改 pg_hba.conf</h5>
|
||||
<p>然後修改 pg_hba.conf，加入底下兩行</p>
|
||||
|
||||
<pre><code># for barman test
|
||||
host database_name barman 192.168.11.192/32 md5
|
||||
host replication stream_barman 192.168.11.192/32 md5
|
||||
</code></pre>
|
||||
|
||||
<p>當然，如果不考慮安全性問題， md5 直接改成用 trust ，可以省去一些麻煩。</p>
|
||||
|
||||
<h5 id="修改-postgresql-conf">修改 postgresql.conf</h5>
|
||||
|
||||
</code></pre><p>當然，如果不考慮安全性問題， md5 直接改成用 trust ，可以省去一些麻煩。</p>
|
||||
<h5 id="修改-postgresqlconf">修改 postgresql.conf</h5>
|
||||
<p>接著修改postgresql.conf</p>
|
||||
|
||||
<pre><code>### for barman test
|
||||
max_wal_senders = 5
|
||||
max_replication_slots = 3
|
||||
wal_level = 'archive'
|
||||
archive_mode = on
|
||||
</code></pre>
|
||||
|
||||
<p>重起 postgresql service</p>
|
||||
|
||||
</code></pre><p>重起 postgresql service</p>
|
||||
<h4 id="設定barman-server">設定barman server</h4>
|
||||
|
||||
<p>IP: 192.168.11.192
|
||||
hostname: barman</p>
|
||||
|
||||
<h5 id="安裝-barman">安裝 barman</h5>
|
||||
|
||||
<p>barman 在18.04 中，已經被放到標準repository 中</p>
|
||||
|
||||
<p>所以只要直接</p>
|
||||
|
||||
<pre><code>sudo apt install barman
|
||||
</code></pre>
|
||||
|
||||
<p>就可以了</p>
|
||||
|
||||
<h5 id="設定-barman-conf">設定 barman.conf</h5>
|
||||
|
||||
</code></pre><p>就可以了</p>
|
||||
<h5 id="設定-barmanconf">設定 barman.conf</h5>
|
||||
<p>安裝完成後，在/etc/barman.d/ 底下會有兩個範例檔案</p>
|
||||
|
||||
<pre><code>streaming-server.conf-template
|
||||
ssh-server.conf-template
|
||||
</code></pre>
|
||||
|
||||
<p>複製 streaming-server 檔案</p>
|
||||
|
||||
</code></pre><p>複製 streaming-server 檔案</p>
|
||||
<pre><code>sudo cp /etc/barman.d/streaming-server.conf-template /etc/barman.d/hqs019.conf
|
||||
</code></pre>
|
||||
|
||||
<p>內容如下</p>
|
||||
|
||||
</code></pre><p>內容如下</p>
|
||||
<pre><code>[hqs019]
|
||||
description = "hqs019 "
|
||||
conninfo = host=192.168.11.19 user=barman dbname=database_name password=barmanpassword
|
||||
@@ -432,10 +373,7 @@ backup_method = postgres
|
||||
retention_policy_mode = auto
|
||||
streaming_archiver = on
|
||||
slot_name = barman
|
||||
</code></pre>
|
||||
|
||||
<p>接著修改 /etc/barman.conf</p>
|
||||
|
||||
</code></pre><p>接著修改 /etc/barman.conf</p>
|
||||
<pre><code>[barman]
|
||||
barman_user = barman
|
||||
configuration_files_directory = /etc/barman.d
|
||||
@@ -447,16 +385,10 @@ immediate_checkpoint = true
|
||||
basebackup_retry_times = 3
|
||||
basebackup_retry_sleep = 30
|
||||
last_backup_maximum_age = 1 DAYS
|
||||
</code></pre>
|
||||
|
||||
<p>基本上這樣就設定完成了</p>
|
||||
|
||||
</code></pre><p>基本上這樣就設定完成了</p>
|
||||
<h5 id="檢查設定">檢查設定</h5>
|
||||
|
||||
<p>barman 有一些指令可以用來檢查目前的設定</p>
|
||||
|
||||
<p>barman show-server hqs019 可以看到所有的設定，這裡的 hqs019 跟 barman.d/hqs019.conf 裡面用”[ ]” 包起來的名稱要一致</p>
|
||||
|
||||
<pre><code>barman@barman:~$ barman show-server hqs019
|
||||
Server hqs019:
|
||||
active: True
|
||||
@@ -544,10 +476,7 @@ Server hqs019:
|
||||
wal_retention_policy: main
|
||||
wals_directory: /var/lib/barman/hqs019/wals
|
||||
xlogpos: 264/BA000F08
|
||||
</code></pre>
|
||||
|
||||
<p>然後用 barman check hqs019 來檢查config 有沒有問題</p>
|
||||
|
||||
</code></pre><p>然後用 barman check hqs019 來檢查config 有沒有問題</p>
|
||||
<pre><code>barman@barman:~$ barman check hqs019
|
||||
Server hqs019:
|
||||
PostgreSQL: OK
|
||||
@@ -569,14 +498,9 @@ Server hqs019:
|
||||
receive-wal running: OK
|
||||
archiver errors: OK
|
||||
barman@barman:~$
|
||||
</code></pre>
|
||||
|
||||
<p>那個backup maximum age FAILED 不用管他，因為都還沒跑過備份，這邊錯誤是正常的</p>
|
||||
|
||||
</code></pre><p>那個backup maximum age FAILED 不用管他，因為都還沒跑過備份，這邊錯誤是正常的</p>
|
||||
<p>其他都OK 的話，就可以開始備份了</p>
|
||||
|
||||
<p>barman backup hqs019</p>
|
||||
|
||||
<pre><code>barman@ubuntu:~$ barman backup hqs019
|
||||
Starting backup using postgres method for server hqs019 in /var/lib/barman/hqs019/base/20190823T082258
|
||||
Backup start at LSN: 264/A10001A8 (0000000100000264000000A1, 000001A8)
|
||||
@@ -595,16 +519,10 @@ Processing xlog segments from streaming for hqs019
|
||||
0000000100000264000000A2
|
||||
|
||||
barman@ubuntu:~$
|
||||
</code></pre>
|
||||
|
||||
<p>跑完可以用 barman list-backup hqs019 檢查</p>
|
||||
|
||||
</code></pre><p>跑完可以用 barman list-backup hqs019 檢查</p>
|
||||
<pre><code>barman@ubuntu:~$ barman list-backup hqs019
|
||||
hqs019 20190823T082258 - Thu Aug 22 17:29:26 2019 - Size: 133.0 GiB - WAL Size: 0 B (tablespaces: tablespace_a:/var/lib/postgresql/10/main/tablespace_A, tablespace_b:/var/lib/postgresql/10/main/tablespace_B)
|
||||
</code></pre>
|
||||
|
||||
<p>要刪除的話，要加入 backupID</p>
|
||||
|
||||
</code></pre><p>要刪除的話，要加入 backupID</p>
|
||||
<pre><code>barman@ubuntu:~$ barman delete hqs019 20190822T171355
|
||||
Deleting backup 20190822T171355 for server hqs019
|
||||
Delete associated WAL segments:
|
||||
@@ -613,12 +531,8 @@ Delete associated WAL segments:
|
||||
0000000100000264000000A1
|
||||
Deleted backup 20190822T171355 (start time: Fri Aug 23 09:36:43 2019, elapsed time: 3 seconds)
|
||||
|
||||
</code></pre>
|
||||
|
||||
<p>restore 的部份，暫時沒有測試，我想應該是要找時間測試看看怎麼還原才對</p>
|
||||
|
||||
</code></pre><p>restore 的部份，暫時沒有測試，我想應該是要找時間測試看看怎麼還原才對</p>
|
||||
<p>不過呢，前面有提到，用streaming backup ，每一次備份都是完整備份，非常的佔用空間、時間、頻寬</p>
|
||||
|
||||
<p>所以還是要來試試看用rsync/SSH 備份的機制</p>
|
||||
</article>
|
||||
</div>
|
||||
@@ -896,7 +810,7 @@ title="pinterest icon"></i>
|
||||
|
||||
</ul> <div class="design-credit">
|
||||
|
||||
<p>© 2018 Göran Svensson</p>
|
||||
<p>© 2018 Göran Svensson</p>
|
||||
|
||||
<p>Nederburg Hugo Theme by <a href="https://appernetic.io">Appernetic</a>.</p>
|
||||
|
||||
@@ -908,7 +822,7 @@ title="pinterest icon"></i>
|
||||
</div>
|
||||
<script src="https://h.cowbay.org/js/jquery.min.js"></script>
|
||||
<script src="https://h.cowbay.org/js/jquerymigrate.js"></script>
|
||||
<script src="https://h.cowbay.org/js/production.min.js?v=1626744134"></script>
|
||||
<script src="https://h.cowbay.org/js/production.min.js?v=1629951055"></script>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
||||
Reference in New Issue
Block a user