Chef ile Consul Cluster Kurulumu

21/11/202121/11/2021 tarihinde yayınlandı.

Merhaba, bu yazımda Chef ile Consul Cluster yapısının kurulumundan bahsediyor olacağım.

Öncelikle Chef’in kurulumu ve konfigürasyonu ile ilgili detaylı yazıma buradan ulaşabilirsiniz.

Service Discovery Nedir?

Bildiğiniz gibi microservice mimarisi artık birçok kurumsal şirketin yazılımsal altyapısını oluşturan bir noktaya geldi. Durum böyleyken microservice sayısı arttıkça, bu servislerin management ve monitoring kısmında ihtiyaçlar ortaya çıktı. Service Discovery, bu ihtiyaçlara aranan çözümlerdir diyebiliriz. Bu çözümü sağlayan sade, kullanışlı ve popüler bir tool olan Consul Cluster yapısını kuruyor olacağız.

Ubuntu 20.04 LTS Üzerinde Consul Cluster Kurulumu

İlk olarak consul_setups adında bir cookbook oluşturup, içerisine node1.rb, node2.rb ve node3.rb recipe’lerimizi ekleyelim. Daha sonra templates adında bir folder oluşturup subfolder olarak node1, node2 ve node3 klasörlerini oluşturalım. Subfolder’ların her birine, config.json.erb ve consul.service.erb dosyalarını ekleyelim. 3 node için consul config ve systemd config dosyalarını bu klasörlerden çekiyor olacağız.

Cookbook’un son hali aşağıdaki gibi olmalıdır.

Kurulum ile birlikte dikkat etmemiz gereken 2 case var:

İlk node’un kurulumundan sonra makineye login olup ‘consul keygen’ komutu ile guid alınıp, config.json.erb içindeki encrypt parametresinin yanına eklenmesi. (3 node içinde aynı guid eklenmesi gerekiyor. Tek bir makinede oluşturmanız yeterli.)
3 node’un /etc/hosts dosyasına aşağıdaki kayıtların eklenmesi.

192.168.1.10 cnsl1
192.168.1.20 cnsl2
192.168.1.30 cnsl3

192.168.1.10 cnsl1

192.168.1.20 cnsl2

192.168.1.30 cnsl3

Son olarak aşağıdaki komutlar ile recipe’leri makinelere bastıktan sonra Consul Cluster ortamımız hazır durumu geliyor.

knife bootstrap 192.168.1.10 -U username --sudo --use-sudo-password --node-name cnsl1 --run-list 'recipe[consul_setups::node1]'
knife bootstrap 192.168.1.20 -U username --sudo --use-sudo-password --node-name cnsl2 --run-list 'recipe[consul_setups::node2]'
knife bootstrap 192.168.1.30 -U username --sudo --use-sudo-password --node-name cnsl3 --run-list 'recipe[consul_setups::node3]'

knife bootstrap 192.168.1.10 -U username --sudo --use-sudo-password --node-name cnsl1 --run-list 'recipe[consul_setups::node1]'

knife bootstrap 192.168.1.20 -U username --sudo --use-sudo-password --node-name cnsl2 --run-list 'recipe[consul_setups::node2]'

knife bootstrap 192.168.1.30 -U username --sudo --use-sudo-password --node-name cnsl3 --run-list 'recipe[consul_setups::node3]'

Cookbook dosyalarının içeriklerini aşağıda görebilirsiniz.

#
# Cookbook:: consul_setups
# Recipe:: node1
#
# Copyright:: 2021, The Authors, All Rights Reserved.

service 'ufw' do
    action [ :disable, :stop ]
end

execute 'Update' do
    command 'sudo apt update'
    action :run
end

package 'wget'
package 'gnupg2'
package 'unzip'

execute 'Download & Install Consul Server' do
    command '
    cd /tmp && \
    wget https://releases.hashicorp.com/consul/1.10.0/consul_1.10.0_linux_amd64.zip && \
    unzip consul_1.10.0_linux_amd64.zip && \
    mv consul /usr/local/bin/ && \
    rm -rf /tmp/consul*
    '
    action :run
end

execute 'Create Consul User' do
    command '
    sudo groupadd --system consul && \
    sudo useradd -s /sbin/nologin --system -g consul consul
    '
    action :run
end

directory '/var/lib/consul' do
    owner 'consul'
    group 'consul'
    mode '0775'
    action :create
end

directory '/etc/consul.d' do
    owner 'consul'
    group 'consul'
    mode '0775'
    action :create
end

template '/etc/systemd/system/consul.service' do
    source 'node1/consul.service.erb'
    owner 'consul'
    group 'consul'
    mode '0775'
    action :create
end

#NOTE
#'consul keygen' komutu ile guid alınıp, config.json içindeki encrypt parametresinin yanına eklenir.

template '/etc/consul.d/config.json' do
    source 'node1/config.json.erb'
    owner 'consul'
    group 'consul'
    mode '0775'
    action :create
end

execute 'Run Deamon-Reload Command' do
    command 'systemctl daemon-reload'
    action :run
end

service 'consul' do
    action [ :enable, :restart ]
end

# Cookbook:: consul_setups

# Recipe:: node1

service 'ufw' do

action [ :disable, :stop ]

end

execute 'Update' do

command 'sudo apt update'

action :run

end

package 'wget'

package 'gnupg2'

package 'unzip'

execute 'Download & Install Consul Server' do

command '

cd /tmp && \

wget https://releases.hashicorp.com/consul/1.10.0/consul_1.10.0_linux_amd64.zip && \

unzip consul_1.10.0_linux_amd64.zip && \

mv consul /usr/local/bin/ && \

rm -rf /tmp/consul*

action :run

end

execute 'Create Consul User' do

command '

sudo groupadd --system consul && \

sudo useradd -s /sbin/nologin --system -g consul consul

action :run

end

directory '/var/lib/consul' do

owner 'consul'

group 'consul'

mode '0775'

action :create

end

directory '/etc/consul.d' do

owner 'consul'

group 'consul'

mode '0775'

action :create

end

template '/etc/systemd/system/consul.service' do

source 'node1/consul.service.erb'

owner 'consul'

group 'consul'

mode '0775'

action :create

end

#NOTE

#'consul keygen' komutu ile guid alınıp, config.json içindeki encrypt parametresinin yanına eklenir.

template '/etc/consul.d/config.json' do

source 'node1/config.json.erb'

owner 'consul'

group 'consul'

mode '0775'

action :create

end

execute 'Run Deamon-Reload Command' do

command 'systemctl daemon-reload'

action :run

end

service 'consul' do

action [ :enable, :restart ]

end

#
# Cookbook:: consul_setups
# Recipe:: node2
#
# Copyright:: 2021, The Authors, All Rights Reserved.

service 'ufw' do
    action [ :disable, :stop ]
end

execute 'Update' do
    command 'sudo apt update'
    action :run
end

package 'wget'
package 'gnupg2'
package 'unzip'

execute 'Download & Install Consul Server' do
    command '
    cd /tmp && \
    wget https://releases.hashicorp.com/consul/1.10.0/consul_1.10.0_linux_amd64.zip && \
    unzip consul_1.10.0_linux_amd64.zip && \
    mv consul /usr/local/bin/ && \
    rm -rf /tmp/consul*
    '
    action :run
end

execute 'Create Consul User' do
    command '
    sudo groupadd --system consul && \
    sudo useradd -s /sbin/nologin --system -g consul consul
    '
    action :run
end

directory '/var/lib/consul' do
    owner 'consul'
    group 'consul'
    mode '0775'
    action :create
end

directory '/etc/consul.d' do
    owner 'consul'
    group 'consul'
    mode '0775'
    action :create
end

template '/etc/systemd/system/consul.service' do
    source 'node2/consul.service.erb'
    owner 'consul'
    group 'consul'
    mode '0775'
    action :create
end

template '/etc/consul.d/config.json' do
    source 'node2/config.json.erb'
    owner 'consul'
    group 'consul'
    mode '0775'
    action :create
end

execute 'Run Deamon-Reload Command' do
    command 'systemctl daemon-reload'
    action :run
end

service 'consul' do
    action [ :enable, :restart ]
end

# Cookbook:: consul_setups

# Recipe:: node2

service 'ufw' do

action [ :disable, :stop ]

end

execute 'Update' do

command 'sudo apt update'

action :run

end

package 'wget'

package 'gnupg2'

package 'unzip'

execute 'Download & Install Consul Server' do

command '

cd /tmp && \

wget https://releases.hashicorp.com/consul/1.10.0/consul_1.10.0_linux_amd64.zip && \

unzip consul_1.10.0_linux_amd64.zip && \

mv consul /usr/local/bin/ && \

rm -rf /tmp/consul*

action :run

end

execute 'Create Consul User' do

command '

sudo groupadd --system consul && \

sudo useradd -s /sbin/nologin --system -g consul consul

action :run

end

directory '/var/lib/consul' do

owner 'consul'

group 'consul'

mode '0775'

action :create

end

directory '/etc/consul.d' do

owner 'consul'

group 'consul'

mode '0775'

action :create

end

template '/etc/systemd/system/consul.service' do

source 'node2/consul.service.erb'

owner 'consul'

group 'consul'

mode '0775'

action :create

end

template '/etc/consul.d/config.json' do

source 'node2/config.json.erb'

owner 'consul'

group 'consul'

mode '0775'

action :create

end

execute 'Run Deamon-Reload Command' do

command 'systemctl daemon-reload'

action :run

end

service 'consul' do

action [ :enable, :restart ]

end

#
# Cookbook:: consul_setups
# Recipe:: node3
#
# Copyright:: 2021, The Authors, All Rights Reserved.

service 'ufw' do
    action [ :disable, :stop ]
end

execute 'Update' do
    command 'sudo apt update'
    action :run
end

package 'wget'
package 'gnupg2'
package 'unzip'

execute 'Download & Install Consul Server' do
    command '
    cd /tmp && \
    wget https://releases.hashicorp.com/consul/1.10.0/consul_1.10.0_linux_amd64.zip && \
    unzip consul_1.10.0_linux_amd64.zip && \
    mv consul /usr/local/bin/ && \
    rm -rf /tmp/consul*
    '
    action :run
end

execute 'Create Consul User' do
    command '
    sudo groupadd --system consul && \
    sudo useradd -s /sbin/nologin --system -g consul consul
    '
    action :run
end

directory '/var/lib/consul' do
    owner 'consul'
    group 'consul'
    mode '0775'
    action :create
end

directory '/etc/consul.d' do
    owner 'consul'
    group 'consul'
    mode '0775'
    action :create
end

template '/etc/systemd/system/consul.service' do
    source 'node3/consul.service.erb'
    owner 'consul'
    group 'consul'
    mode '0775'
    action :create
end

template '/etc/consul.d/config.json' do
    source 'node3/config.json.erb'
    owner 'consul'
    group 'consul'
    mode '0775'
    action :create
end

execute 'Run Deamon-Reload Command' do
    command 'systemctl daemon-reload'
    action :run
end

service 'consul' do
    action [ :enable, :restart ]
end

# Cookbook:: consul_setups

# Recipe:: node3

service 'ufw' do

action [ :disable, :stop ]

end

execute 'Update' do

command 'sudo apt update'

action :run

end

package 'wget'

package 'gnupg2'

package 'unzip'

execute 'Download & Install Consul Server' do

command '

cd /tmp && \

wget https://releases.hashicorp.com/consul/1.10.0/consul_1.10.0_linux_amd64.zip && \

unzip consul_1.10.0_linux_amd64.zip && \

mv consul /usr/local/bin/ && \

rm -rf /tmp/consul*

action :run

end

execute 'Create Consul User' do

command '

sudo groupadd --system consul && \

sudo useradd -s /sbin/nologin --system -g consul consul

action :run

end

directory '/var/lib/consul' do

owner 'consul'

group 'consul'

mode '0775'

action :create

end

directory '/etc/consul.d' do

owner 'consul'

group 'consul'

mode '0775'

action :create

end

template '/etc/systemd/system/consul.service' do

source 'node3/consul.service.erb'

owner 'consul'

group 'consul'

mode '0775'

action :create

end

template '/etc/consul.d/config.json' do

source 'node3/config.json.erb'

owner 'consul'

group 'consul'

mode '0775'

action :create

end

execute 'Run Deamon-Reload Command' do

command 'systemctl daemon-reload'

action :run

end

service 'consul' do

action [ :enable, :restart ]

end

{
    "advertise_addr": "192.168.1.10",
    "bind_addr": "192.168.1.10",
    "bootstrap_expect": 3,
    "client_addr": "0.0.0.0",
    "datacenter": "DC1",
    "data_dir": "/var/lib/consul",
    "domain": "consul",
    "enable_script_checks": true,
    "dns_config": {
        "enable_truncate": true,
        "only_passing": true
    },
    "enable_syslog": true,
    "encrypt": "b34OCY2gJ+UwA+5DKpGtXB1y8+L1uojMUlFaorZKvJQ=",
    "leave_on_terminate": true,
    "log_level": "INFO",
    "rejoin_after_leave": true,
    "retry_join": [
        "cnsl1",
        "cnsl2",
        "cnsl3"
    ],
    "server": true,
    "start_join": [
        "cnsl1",
        "cnsl2",
        "cnsl3"
    ],
    "ui": true
}

{

"advertise_addr": "192.168.1.10",

"bind_addr": "192.168.1.10",

"bootstrap_expect": 3,

"client_addr": "0.0.0.0",

"datacenter": "DC1",

"data_dir": "/var/lib/consul",

"domain": "consul",

"enable_script_checks": true,

"dns_config": {

"enable_truncate": true,

"only_passing": true

"enable_syslog": true,

"encrypt": "b34OCY2gJ+UwA+5DKpGtXB1y8+L1uojMUlFaorZKvJQ=",

"leave_on_terminate": true,

"log_level": "INFO",

"rejoin_after_leave": true,

"retry_join": [

"cnsl1",

"cnsl2",

"cnsl3"

"server": true,

"start_join": [

"cnsl1",

"cnsl2",

"cnsl3"

"ui": true

}

[Unit]
Description=Consul Service Discovery Agent
Documentation=https://www.consul.io/
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=consul
Group=consul
ExecStart=/usr/local/bin/consul agent \
    -node=cnsl1 \
    -config-dir=/etc/consul.d

ExecReload=/bin/kill -HUP $MAINPID
KillSignal=SIGINT
TimeoutStopSec=5
Restart=on-failure
SyslogIdentifier=consul

[Install]
WantedBy=multi-user.target

[Unit]

Description=Consul Service Discovery Agent

Documentation=https://www.consul.io/

After=network-online.target

Wants=network-online.target

[Service]

Type=simple

User=consul

Group=consul

ExecStart=/usr/local/bin/consul agent \

-node=cnsl1 \

-config-dir=/etc/consul.d

ExecReload=/bin/kill -HUP $MAINPID

KillSignal=SIGINT

TimeoutStopSec=5

Restart=on-failure

SyslogIdentifier=consul

[Install]

WantedBy=multi-user.target

{
    "advertise_addr": "192.168.1.20",
    "bind_addr": "192.168.1.20",
    "bootstrap_expect": 3,
    "client_addr": "0.0.0.0",
    "datacenter": "DC1",
    "data_dir": "/var/lib/consul",
    "domain": "consul",
    "enable_script_checks": true,
    "dns_config": {
        "enable_truncate": true,
        "only_passing": true
    },
    "enable_syslog": true,
    "encrypt": "b34OCY2gJ+UwA+5DKpGtXB1y8+L1uojMUlFaorZKvJQ=",
    "leave_on_terminate": true,
    "log_level": "INFO",
    "rejoin_after_leave": true,
    "retry_join": [
        "cnsl1",
        "cnsl2",
        "cnsl3"
    ],
    "server": true,
    "start_join": [
        "cnsl1",
        "cnsl2",
        "cnsl3"
    ],
    "ui": true
}

{

"advertise_addr": "192.168.1.20",

"bind_addr": "192.168.1.20",

"bootstrap_expect": 3,

"client_addr": "0.0.0.0",

"datacenter": "DC1",

"data_dir": "/var/lib/consul",

"domain": "consul",

"enable_script_checks": true,

"dns_config": {

"enable_truncate": true,

"only_passing": true

"enable_syslog": true,

"encrypt": "b34OCY2gJ+UwA+5DKpGtXB1y8+L1uojMUlFaorZKvJQ=",

"leave_on_terminate": true,

"log_level": "INFO",

"rejoin_after_leave": true,

"retry_join": [

"cnsl1",

"cnsl2",

"cnsl3"

"server": true,

"start_join": [

"cnsl1",

"cnsl2",

"cnsl3"

"ui": true

}

[Unit]
Description=Consul Service Discovery Agent
Documentation=https://www.consul.io/
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=consul
Group=consul
ExecStart=/usr/local/bin/consul agent \
    -node=cnsl2 \
    -config-dir=/etc/consul.d

ExecReload=/bin/kill -HUP $MAINPID
KillSignal=SIGINT
TimeoutStopSec=5
Restart=on-failure
SyslogIdentifier=consul

[Install]
WantedBy=multi-user.target

[Unit]

Description=Consul Service Discovery Agent

Documentation=https://www.consul.io/

After=network-online.target

Wants=network-online.target

[Service]

Type=simple

User=consul

Group=consul

ExecStart=/usr/local/bin/consul agent \

-node=cnsl2 \

-config-dir=/etc/consul.d

ExecReload=/bin/kill -HUP $MAINPID

KillSignal=SIGINT

TimeoutStopSec=5

Restart=on-failure

SyslogIdentifier=consul

[Install]

WantedBy=multi-user.target

{
    "advertise_addr": "192.168.1.30",
    "bind_addr": "192.168.1.30",
    "bootstrap_expect": 3,
    "client_addr": "0.0.0.0",
    "datacenter": "DC1",
    "data_dir": "/var/lib/consul",
    "domain": "consul",
    "enable_script_checks": true,
    "dns_config": {
        "enable_truncate": true,
        "only_passing": true
    },
    "enable_syslog": true,
    "encrypt": "b34OCY2gJ+UwA+5DKpGtXB1y8+L1uojMUlFaorZKvJQ=",
    "leave_on_terminate": true,
    "log_level": "INFO",
    "rejoin_after_leave": true,
    "retry_join": [
        "cnsl1",
        "cnsl2",
        "cnsl3"
    ],
    "server": true,
    "start_join": [
        "cnsl1",
        "cnsl2",
        "cnsl3"
    ],
    "ui": true
}

{

"advertise_addr": "192.168.1.30",

"bind_addr": "192.168.1.30",

"bootstrap_expect": 3,

"client_addr": "0.0.0.0",

"datacenter": "DC1",

"data_dir": "/var/lib/consul",

"domain": "consul",

"enable_script_checks": true,

"dns_config": {

"enable_truncate": true,

"only_passing": true

"enable_syslog": true,

"encrypt": "b34OCY2gJ+UwA+5DKpGtXB1y8+L1uojMUlFaorZKvJQ=",

"leave_on_terminate": true,

"log_level": "INFO",

"rejoin_after_leave": true,

"retry_join": [

"cnsl1",

"cnsl2",

"cnsl3"

"server": true,

"start_join": [

"cnsl1",

"cnsl2",

"cnsl3"

"ui": true

}

[Unit]
Description=Consul Service Discovery Agent
Documentation=https://www.consul.io/
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=consul
Group=consul
ExecStart=/usr/local/bin/consul agent \
    -node=cnsl3 \
    -config-dir=/etc/consul.d

ExecReload=/bin/kill -HUP $MAINPID
KillSignal=SIGINT
TimeoutStopSec=5
Restart=on-failure
SyslogIdentifier=consul

[Install]
WantedBy=multi-user.target

[Unit]

Description=Consul Service Discovery Agent

Documentation=https://www.consul.io/

After=network-online.target

Wants=network-online.target

[Service]

Type=simple

User=consul

Group=consul

ExecStart=/usr/local/bin/consul agent \

-node=cnsl3 \

-config-dir=/etc/consul.d

ExecReload=/bin/kill -HUP $MAINPID

KillSignal=SIGINT

TimeoutStopSec=5

Restart=on-failure

SyslogIdentifier=consul

[Install]

WantedBy=multi-user.target

Sosyal Medyada Paylaşın