Azure와 함께 하는 DevOps
20 Terraform을 사용하여 Packer 사용자 지정 이미지에서 Azure 가상 머신 확장 집합 만들기
zerobig-k8s
2020. 2. 24. 07:57
<참조>
|
<참고> 2020년 2월 24일 19시 추가 사항
본 자습서의 원래 내용대로 실습을 진행하면 최종 SSH 접속 테스트 단계에서 vmss 내부 IP를 사용하여 SSH접속이 불가함을 확인하였다. 이에 잘못된 부분을 바로 잡아 수정하여 다시 정리한다.
|
이 자습서에서는 HCL(HashiCorp Configuration Language)을 사용하는 관리 디스크에서 Packer를 사용하여 생성된 사용자 지정 이미지로 만든 Azure 가상 머신 확장 집합을 만들고 배포하는 데 Terraform을 사용한다.
이 자습서에서는 다음 작업 방법을 알아본다.
- Terraform 배포 설정
- Terraform 배포용 변수 및 출력 사용
- 네트워크 인프라 만들기 및 배포
- Packer를 사용하여 사용자 지정 가상 머신 이미지 만들기
- 사용자 지정 이미지를 사용하여 가상 머신 확장 집합 만들기 및 배포
- jumpbox 만들기 및 배포
Azure 구독이 아직 없는 경우 시작하기 전에 체험 계정을 만듭니다.
사전 요구 사항
- Terraform 설치: Terraform 및 Azure에 액세스 구성 문서의 지침을 따른다.
-
SSH 키 쌍 만들기: 자세한 내용은 Azure에서 Linux VM용 SSH 공개 및 프라이빗 키 쌍을 만들고 사용하는 방법을 참조한다.
- Packer: Packer 설치
파일 구조 만들기
먼저 실습을 위한 디렉토리를 생성하고 VS Code로 열어 다음과 같은 이름으로 3개의 새 파일을 만든다.
- variables.tf: 이 파일은 템플릿에 사용되는 변수 값을 포함된다.
- output.tf: 이 파일은 배포 후에 표시될 설정을 설명한다.
- vmss.tf: 이 파일에는 배포 중인 인프라의 코드가 포함되어 있다.
변수 만들기
이 단계에서는 Terraform에서 만든 리소스를 사용자 지정하는 변수를 정의한다.
variables.tf 파일을 편집하고, 다음 코드를 복사한 다음, 변경 내용을 저장한다.
variable "location" {
description = "The location where resources are created"
default = "Korea Central"
}
variable "resource_group_name" {
description = "The name of the resource group in which the resources are created"
default = "handson0224"
}
Terraform 템플릿을 배포하는 경우 애플리케이션에 액세스하는 데 사용되는 정규화된 도메인 이름을 가져오려고 한다. Terraform의 output 리소스 형식을 사용하고 리소스의 fqdn 속성을 가져온다.
output.tf 파일을 편집하고 다음 코드를 복사하여 가상 머신의 정규화된 도메인 이름을 노출한다.
output "vmss_public_ip" {
value = azurerm_public_ip.vmss.fqdn
}
템플릿에 네트워크 인프라 정의
이 단계에서는 새 Azure 리소스 그룹에 다음과 같은 네트워크 인프라를 만든다.
- 주소 공간이 10.0.0.0/16인 가상 네트워크 1개
- 주소 공간이 10.0.2.0/24인 서브넷 1개
- 2개의 공용 IP 주소. 하나는 가상 머신 확장 집합 부하 분산 장치에서 사용된다. 다른 하나는 SSH jumpbox에 연결하는 데 사용된다.
모든 리소스를 만드는 리소스 그룹이 필요하다.
vmss.tf 파일에서 다음 코드를 편집하고 복사한다.
resource "azurerm_resource_group" "vmss" {
name = var.resource_group_name
location = var.location
tags = {
environment = "codelab"
}
}
resource "azurerm_virtual_network" "vmss" {
name = "vmss-vnet"
address_space = ["10.0.0.0/16"]
location = var.location
resource_group_name = azurerm_resource_group.vmss.name
tags = {
environment = "codelab"
}
}
resource "azurerm_subnet" "vmss" {
name = "vmss-subnet"
resource_group_name = azurerm_resource_group.vmss.name
virtual_network_name = azurerm_virtual_network.vmss.name
address_prefix = "10.0.2.0/24"
}
resource "azurerm_public_ip" "vmss" {
name = "vmss-public-ip"
location = var.location
resource_group_name = azurerm_resource_group.vmss.name
allocation_method = "Static"
domain_name_label = azurerm_resource_group.vmss.name
tags = {
environment = "codelab"
}
}
네트워크 인프라 만들기
.tf 파일을 만든 디렉터리에서 다음 명령을 실행하여 Terraform 환경을 초기화한다.
| terraform init |
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ terraform init
Initializing the backend...
Initializing provider plugins...
The following providers do not have any version constraints in configuration,
so the latest version was installed.
To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.
* provider.azurerm: version = "~> 1.44"
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary
공급자 플러그 인은 Terraform 레지스트리에서 명령을 실행하는 디렉터리의 .terraform 폴더로 다운로드한다.
다음 명령을 실행하여 Azure에 인프라를 배포한다.
| terraform apply |
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ terraform apply
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# azurerm_public_ip.vmss will be created
+ resource "azurerm_public_ip" "vmss" {
+ allocation_method = "Static"
+ domain_name_label = "handson0224"
+ fqdn = (known after apply)
+ id = (known after apply)
+ idle_timeout_in_minutes = 4
+ ip_address = (known after apply)
+ ip_version = "IPv4"
+ location = "koreacentral"
+ name = "vmss-public-ip"
+ public_ip_address_allocation = (known after apply)
+ resource_group_name = "handson0224"
+ sku = "Basic"
+ tags = {
+ "environment" = "codelab"
}
}
# azurerm_resource_group.vmss will be created
+ resource "azurerm_resource_group" "vmss" {
+ id = (known after apply)
+ location = "koreacentral"
+ name = "handson0224"
+ tags = {
+ "environment" = "codelab"
}
}
# azurerm_subnet.vmss will be created
+ resource "azurerm_subnet" "vmss" {
+ address_prefix = "10.0.2.0/24"
+ enforce_private_link_endpoint_network_policies = false
+ enforce_private_link_service_network_policies = false
+ id = (known after apply)
+ ip_configurations = (known after apply)
+ name = "vmss-subnet"
+ resource_group_name = "handson0224"
+ virtual_network_name = "vmss-vnet"
}
# azurerm_virtual_network.vmss will be created
+ resource "azurerm_virtual_network" "vmss" {
+ address_space = [
+ "10.0.0.0/16",
]
+ id = (known after apply)
+ location = "koreacentral"
+ name = "vmss-vnet"
+ resource_group_name = "handson0224"
+ tags = {
+ "environment" = "codelab"
}
+ subnet {
+ address_prefix = (known after apply)
+ id = (known after apply)
+ name = (known after apply)
+ security_group = (known after apply)
}
}
Plan: 4 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
azurerm_resource_group.vmss: Creating...
azurerm_resource_group.vmss: Creation complete after 0s [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224]
azurerm_virtual_network.vmss: Creating...
azurerm_public_ip.vmss: Creating...
azurerm_public_ip.vmss: Creation complete after 6s [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/publicIPAddresses/vmss-public-ip]
azurerm_virtual_network.vmss: Still creating... [10s elapsed]
azurerm_virtual_network.vmss: Creation complete after 11s [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/virtualNetworks/vmss-vnet]
azurerm_subnet.vmss: Creating...
azurerm_subnet.vmss: Creation complete after 0s [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/virtualNetworks/vmss-vnet/subnets/vmss-subnet]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
Outputs:
vmss_public_ip = handson0224.koreacentral.cloudapp.azure.com
공용 IP 주소의 정규화된 도메인 이름이 구성에 해당하는지 확인한다.
리소스 그룹에는 다음 리소스가 포함된다.
Packer를 사용하여 Azure 이미지 만들기
Packer를 사용하여 Azure에서 Linux 가상 머신 이미지를 만드는 방법 자습서의 단계를 따라 사용자 지정 Linux 이미지를 만든다.
자습서를 수행하여 Nginx를 설치한 프로비저닝 해제된 Ubuntu 이미지를 만든다.
Packer는 서비스 사용자를 사용하여 Azure를 인증한다. Azure 서비스 사용자는 앱, 서비스 및 Packer와 같은 자동화 도구를 사용할 수 있는 보안 ID다. 서비스 사용자가 Azure에서 수행할 수 있는 작업에 대한 사용 권한은 사용자가 제어하고 정의한다.
az ad sp create-for-rbac를 사용하여 서비스 사용자를 만들고 Packer가 필요로 하는 자격 증명을 출력한다.
| az ad sp create-for-rbac --query "{ client_id: appId, client_secret: password, tenant_id: tenant }" |
이전 명령에서 출력의 예는 다음과 같다.
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ az ad sp create-for-rbac --query "{ client_id: appId, client_secret: password, tenant_id: tenant }"
Creating a role assignment under the scope of "/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b"
{
"client_id": "46f7b331-1265-4287-bf72-380b793c189c",
"client_secret": "811294bb-xxxx-xxxx-xxxx-9a7a9d3067d0",
"tenant_id": "917bfe84-0ca6-488d-ad3a-236e41ceafe9"
}
Azure를 인증하기 위해 az account show를 사용하여 Azure 구독 ID를 가져와야 한다.
| az account show --query "{ subscription_id: id }" |
이전 명령에서 출력의 예는 다음과 같다.
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ az account show --query "{ subscription_id: id }"
{
"subscription_id": "f8764f39-xxxx-xxxx-xxxx-77b286ed971b"
}
각자 상황에 맞는 정보와 위에서 생생한 자격증명 정보를 토대로 다음 템플릿을 생성하여 zero_packer_ubuntu.json 라는 이름으로 저장한다.
{
"builders": [{
"type": "azure-arm",
"client_id": "46f7b331-1265-4287-bf72-380b793c189c",
"client_secret": "811294bb-xxxx-xxxx-xxxx-9a7a9d3067d0",
"tenant_id": "917bfe84-0ca6-488d-ad3a-236e41ceafe9",
"subscription_id": "f8764f39-xxxx-xxxx-xxxx-77b286ed971b",
"managed_image_resource_group_name": "handson0223",
"managed_image_name": "zeroPackerImage",
"os_type": "Linux",
"image_publisher": "Canonical",
"image_offer": "UbuntuServer",
"image_sku": "16.04-LTS",
"azure_tags": {
"dept": "Engineering",
"task": "Image deployment"
},
"location": "Korea Central",
"vm_size": "Standard_DS2_v2"
}],
"provisioners": [{
"execute_command": "chmod +x {{ .Path }}; {{ .Vars }} sudo -E sh '{{ .Path }}'",
"inline": [
"apt-get update",
"apt-get upgrade -y",
"apt-get -y install nginx",
"/usr/sbin/waagent -force -deprovision+user && export HISTSIZE=0 && sync"
],
"inline_shebang": "/bin/sh -x",
"type": "shell"
}]
}
packer를 설치하고 이미지 작성을 실행한다.
|
# packer 설치 |
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ export VER="1.5.1"
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ wget https://releases.hashicorp.com/packer/${VER}/packer_${VER}_linux_amd64.zip
Will not apply HSTS. The HSTS database must be a regular and non-world-writable file.
ERROR: could not open HSTS store at '/home/zerobig/.wget-hsts'. HSTS will be disabled.
--2020-02-23 13:32:18-- https://releases.hashicorp.com/packer/1.5.1/packer_1.5.1_linux_amd64.zip
Resolving releases.hashicorp.com (releases.hashicorp.com)... 151.101.197.183, 2a04:4e42:2e::439
Connecting to releases.hashicorp.com (releases.hashicorp.com)|151.101.197.183|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 47183647 (45M) [application/zip]
Saving to: ‘packer_1.5.1_linux_amd64.zip’
packer_1.5.1_linux_amd64.zip 100%[=========================================================================================================================>] 45.00M 450KB/s in 1m 45s
2020-02-23 13:34:04 (439 KB/s) - ‘packer_1.5.1_linux_amd64.zip’ saved [47183647/47183647]
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ unzip packer_${VER}_linux_amd64.zip
Archive: packer_1.5.1_linux_amd64.zip
inflating: packer
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ sudo mv packer /usr/local/bin
[sudo] password for zerobig:
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ packer version
Packer v1.5.1
Your version of Packer is out of date! The latest version
is 1.5.4. You can update by downloading from www.packer.io/downloads.html
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ packer build zero_packer_ubuntu.json
azure-arm: output will be in this color.
==> azure-arm: Running builder ...
==> azure-arm: Getting tokens using client secret
==> azure-arm: Getting tokens using client secret
azure-arm: Creating Azure Resource Manager (ARM) client ...
==> azure-arm: WARNING: Zone resiliency may not be supported in Korea Central, checkout the docs at https://docs.microsoft.com/en-us/azure/availability-zones/
==> azure-arm: Creating resource group ...
==> azure-arm: -> ResourceGroupName : 'packer-Resource-Group-hw8vw3g0z9'
==> azure-arm: -> Location : 'Korea Central'
==> azure-arm: -> Tags :
==> azure-arm: ->> dept : Engineering
==> azure-arm: ->> task : Image deployment
==> azure-arm: Validating deployment template ...
==> azure-arm: -> ResourceGroupName : 'packer-Resource-Group-hw8vw3g0z9'
==> azure-arm: -> DeploymentName : 'pkrdphw8vw3g0z9'
==> azure-arm: Deploying deployment template ...
==> azure-arm: -> ResourceGroupName : 'packer-Resource-Group-hw8vw3g0z9'
==> azure-arm: -> DeploymentName : 'pkrdphw8vw3g0z9'
==> azure-arm: Getting the VM's IP address ...
==> azure-arm: -> ResourceGroupName : 'packer-Resource-Group-hw8vw3g0z9'
==> azure-arm: -> PublicIPAddressName : 'pkriphw8vw3g0z9'
==> azure-arm: -> NicName : 'pkrnihw8vw3g0z9'
==> azure-arm: -> Network Connection : 'PublicEndpoint'
==> azure-arm: -> IP Address : '52.231.52.82'
==> azure-arm: Waiting for SSH to become available...
==> azure-arm: Connected to SSH!
==> azure-arm: Provisioning with shell script: /tmp/packer-shell290793532
azure-arm: Hit:1 http://azure.archive.ubuntu.com/ubuntu xenial InRelease
azure-arm: Get:2 http://azure.archive.ubuntu.com/ubuntu xenial-updates InRelease [109 kB]
azure-arm: Get:3 http://azure.archive.ubuntu.com/ubuntu xenial-backports InRelease [107 kB]
azure-arm: Get:4 http://security.ubuntu.com/ubuntu xenial-security InRelease [109 kB]
azure-arm: Get:5 http://azure.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [7,532 kB]
azure-arm: Get:6 http://azure.archive.ubuntu.com/ubuntu xenial/universe Translation-en [4,354 kB]
azure-arm: Get:7 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [831 kB]
azure-arm: Get:8 http://azure.archive.ubuntu.com/ubuntu xenial/multiverse amd64 Packages [144 kB]
azure-arm: Get:9 http://azure.archive.ubuntu.com/ubuntu xenial/multiverse Translation-en [106 kB]
azure-arm: Get:10 http://azure.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [1,110 kB]
azure-arm: Get:11 http://azure.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [785 kB]
azure-arm: Get:12 http://azure.archive.ubuntu.com/ubuntu xenial-updates/universe Translation-en [329 kB]
azure-arm: Get:13 http://azure.archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 Packages [16.8 kB]
azure-arm: Get:14 http://azure.archive.ubuntu.com/ubuntu xenial-updates/multiverse Translation-en [8,468 B]
azure-arm: Get:15 http://azure.archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages [7,280 B]
azure-arm: Get:16 http://azure.archive.ubuntu.com/ubuntu xenial-backports/main Translation-en [4,456 B]
azure-arm: Get:17 http://azure.archive.ubuntu.com/ubuntu xenial-backports/universe amd64 Packages [8,064 B]
azure-arm: Get:18 http://azure.archive.ubuntu.com/ubuntu xenial-backports/universe Translation-en [4,328 B]
azure-arm: Get:19 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [480 kB]
azure-arm: Get:20 http://security.ubuntu.com/ubuntu xenial-security/universe Translation-en [197 kB]
azure-arm: Get:21 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 Packages [5,728 B]
azure-arm: Get:22 http://security.ubuntu.com/ubuntu xenial-security/multiverse Translation-en [2,708 B]
azure-arm: Fetched 16.3 MB in 4s (3,750 kB/s)
azure-arm: Reading package lists...
azure-arm: Reading package lists...
azure-arm: Building dependency tree...
azure-arm: Reading state information...
azure-arm: Calculating upgrade...
azure-arm: The following package was automatically installed and is no longer required:
azure-arm: grub-pc-bin
azure-arm: Use 'sudo apt autoremove' to remove it.
azure-arm: 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
azure-arm: Reading package lists...
azure-arm: Building dependency tree...
azure-arm: Reading state information...
azure-arm: The following package was automatically installed and is no longer required:
azure-arm: grub-pc-bin
azure-arm: Use 'sudo apt autoremove' to remove it.
azure-arm: The following additional packages will be installed:
azure-arm: fontconfig-config fonts-dejavu-core libfontconfig1 libgd3 libjbig0
azure-arm: libjpeg-turbo8 libjpeg8 libtiff5 libvpx3 libxpm4 nginx-common nginx-core
azure-arm: Suggested packages:
azure-arm: libgd-tools fcgiwrap nginx-doc ssl-cert
azure-arm: The following NEW packages will be installed:
azure-arm: fontconfig-config fonts-dejavu-core libfontconfig1 libgd3 libjbig0
azure-arm: libjpeg-turbo8 libjpeg8 libtiff5 libvpx3 libxpm4 nginx nginx-common
azure-arm: nginx-core
azure-arm: 0 upgraded, 13 newly installed, 0 to remove and 0 not upgraded.
azure-arm: Need to get 2,860 kB of archives.
azure-arm: After this operation, 9,315 kB of additional disk space will be used.
azure-arm: Get:1 http://azure.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libjpeg-turbo8 amd64 1.4.2-0ubuntu3.3 [111 kB]
azure-arm: Get:2 http://azure.archive.ubuntu.com/ubuntu xenial/main amd64 libjbig0 amd64 2.1-3.1 [26.6 kB]
azure-arm: Get:3 http://azure.archive.ubuntu.com/ubuntu xenial/main amd64 fonts-dejavu-core all 2.35-1 [1,039 kB]
azure-arm: Get:4 http://azure.archive.ubuntu.com/ubuntu xenial-updates/main amd64 fontconfig-config all 2.11.94-0ubuntu1.1 [49.9 kB]
azure-arm: Get:5 http://azure.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libfontconfig1 amd64 2.11.94-0ubuntu1.1 [131 kB]
azure-arm: Get:6 http://azure.archive.ubuntu.com/ubuntu xenial/main amd64 libjpeg8 amd64 8c-2ubuntu8 [2,194 B]
azure-arm: Get:7 http://azure.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libtiff5 amd64 4.0.6-1ubuntu0.7 [149 kB]
azure-arm: Get:8 http://azure.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libvpx3 amd64 1.5.0-2ubuntu1.1 [732 kB]
azure-arm: Get:9 http://azure.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libxpm4 amd64 1:3.5.11-1ubuntu0.16.04.1 [33.8 kB]
azure-arm: Get:10 http://azure.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libgd3 amd64 2.1.1-4ubuntu0.16.04.11 [126 kB]
azure-arm: Get:11 http://azure.archive.ubuntu.com/ubuntu xenial-updates/main amd64 nginx-common all 1.10.3-0ubuntu0.16.04.5 [26.9 kB]
azure-arm: Get:12 http://azure.archive.ubuntu.com/ubuntu xenial-updates/main amd64 nginx-core amd64 1.10.3-0ubuntu0.16.04.5 [429 kB]
azure-arm: Get:13 http://azure.archive.ubuntu.com/ubuntu xenial-updates/main amd64 nginx all 1.10.3-0ubuntu0.16.04.5 [3,494 B]
==> azure-arm: debconf: unable to initialize frontend: Dialog
==> azure-arm: debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
==> azure-arm: debconf: falling back to frontend: Readline
==> azure-arm: debconf: unable to initialize frontend: Readline
==> azure-arm: debconf: (This frontend requires a controlling tty.)
==> azure-arm: debconf: falling back to frontend: Teletype
==> azure-arm: dpkg-preconfigure: unable to re-open stdin:
azure-arm: Fetched 2,860 kB in 1s (2,014 kB/s)
azure-arm: Selecting previously unselected package libjpeg-turbo8:amd64.
azure-arm: (Reading database ... 53913 files and directories currently installed.)
azure-arm: Preparing to unpack .../libjpeg-turbo8_1.4.2-0ubuntu3.3_amd64.deb ...
azure-arm: Unpacking libjpeg-turbo8:amd64 (1.4.2-0ubuntu3.3) ...
azure-arm: Selecting previously unselected package libjbig0:amd64.
azure-arm: Preparing to unpack .../libjbig0_2.1-3.1_amd64.deb ...
azure-arm: Unpacking libjbig0:amd64 (2.1-3.1) ...
azure-arm: Selecting previously unselected package fonts-dejavu-core.
azure-arm: Preparing to unpack .../fonts-dejavu-core_2.35-1_all.deb ...
azure-arm: Unpacking fonts-dejavu-core (2.35-1) ...
azure-arm: Selecting previously unselected package fontconfig-config.
azure-arm: Preparing to unpack .../fontconfig-config_2.11.94-0ubuntu1.1_all.deb ...
azure-arm: Unpacking fontconfig-config (2.11.94-0ubuntu1.1) ...
azure-arm: Selecting previously unselected package libfontconfig1:amd64.
azure-arm: Preparing to unpack .../libfontconfig1_2.11.94-0ubuntu1.1_amd64.deb ...
azure-arm: Unpacking libfontconfig1:amd64 (2.11.94-0ubuntu1.1) ...
azure-arm: Selecting previously unselected package libjpeg8:amd64.
azure-arm: Preparing to unpack .../libjpeg8_8c-2ubuntu8_amd64.deb ...
azure-arm: Unpacking libjpeg8:amd64 (8c-2ubuntu8) ...
azure-arm: Selecting previously unselected package libtiff5:amd64.
azure-arm: Preparing to unpack .../libtiff5_4.0.6-1ubuntu0.7_amd64.deb ...
azure-arm: Unpacking libtiff5:amd64 (4.0.6-1ubuntu0.7) ...
azure-arm: Selecting previously unselected package libvpx3:amd64.
azure-arm: Preparing to unpack .../libvpx3_1.5.0-2ubuntu1.1_amd64.deb ...
azure-arm: Unpacking libvpx3:amd64 (1.5.0-2ubuntu1.1) ...
azure-arm: Selecting previously unselected package libxpm4:amd64.
azure-arm: Preparing to unpack .../libxpm4_1%3a3.5.11-1ubuntu0.16.04.1_amd64.deb ...
azure-arm: Unpacking libxpm4:amd64 (1:3.5.11-1ubuntu0.16.04.1) ...
azure-arm: Selecting previously unselected package libgd3:amd64.
azure-arm: Preparing to unpack .../libgd3_2.1.1-4ubuntu0.16.04.11_amd64.deb ...
azure-arm: Unpacking libgd3:amd64 (2.1.1-4ubuntu0.16.04.11) ...
azure-arm: Selecting previously unselected package nginx-common.
azure-arm: Preparing to unpack .../nginx-common_1.10.3-0ubuntu0.16.04.5_all.deb ...
azure-arm: Unpacking nginx-common (1.10.3-0ubuntu0.16.04.5) ...
azure-arm: Selecting previously unselected package nginx-core.
azure-arm: Preparing to unpack .../nginx-core_1.10.3-0ubuntu0.16.04.5_amd64.deb ...
azure-arm: Unpacking nginx-core (1.10.3-0ubuntu0.16.04.5) ...
azure-arm: Selecting previously unselected package nginx.
azure-arm: Preparing to unpack .../nginx_1.10.3-0ubuntu0.16.04.5_all.deb ...
azure-arm: Unpacking nginx (1.10.3-0ubuntu0.16.04.5) ...
azure-arm: Processing triggers for libc-bin (2.23-0ubuntu11) ...
azure-arm: Processing triggers for man-db (2.7.5-1) ...
azure-arm: Processing triggers for ureadahead (0.100.0-19.1) ...
azure-arm: Processing triggers for systemd (229-4ubuntu21.27) ...
azure-arm: Processing triggers for ufw (0.35-0ubuntu2) ...
azure-arm: Setting up libjpeg-turbo8:amd64 (1.4.2-0ubuntu3.3) ...
azure-arm: Setting up libjbig0:amd64 (2.1-3.1) ...
azure-arm: Setting up fonts-dejavu-core (2.35-1) ...
azure-arm: Setting up fontconfig-config (2.11.94-0ubuntu1.1) ...
azure-arm: Setting up libfontconfig1:amd64 (2.11.94-0ubuntu1.1) ...
azure-arm: Setting up libjpeg8:amd64 (8c-2ubuntu8) ...
azure-arm: Setting up libtiff5:amd64 (4.0.6-1ubuntu0.7) ...
azure-arm: Setting up libvpx3:amd64 (1.5.0-2ubuntu1.1) ...
azure-arm: Setting up libxpm4:amd64 (1:3.5.11-1ubuntu0.16.04.1) ...
azure-arm: Setting up libgd3:amd64 (2.1.1-4ubuntu0.16.04.11) ...
azure-arm: Setting up nginx-common (1.10.3-0ubuntu0.16.04.5) ...
azure-arm: debconf: unable to initialize frontend: Dialog
azure-arm: debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
azure-arm: debconf: falling back to frontend: Readline
azure-arm: Setting up nginx-core (1.10.3-0ubuntu0.16.04.5) ...
azure-arm: Setting up nginx (1.10.3-0ubuntu0.16.04.5) ...
azure-arm: Processing triggers for libc-bin (2.23-0ubuntu11) ...
azure-arm: Processing triggers for ureadahead (0.100.0-19.1) ...
azure-arm: Processing triggers for systemd (229-4ubuntu21.27) ...
azure-arm: Processing triggers for ufw (0.35-0ubuntu2) ...
azure-arm: WARNING! The waagent service will be stopped.
azure-arm: WARNING! Cached DHCP leases will be deleted.
azure-arm: WARNING! root password will be disabled. You will not be able to login as root.
azure-arm: WARNING! /etc/resolvconf/resolv.conf.d/tail and /etc/resolvconf/resolv.conf.d/original will be deleted.
azure-arm: WARNING! packer account and entire home directory will be deleted.
==> azure-arm: Querying the machine's properties ...
==> azure-arm: -> ResourceGroupName : 'packer-Resource-Group-hw8vw3g0z9'
==> azure-arm: -> ComputeName : 'pkrvmhw8vw3g0z9'
==> azure-arm: -> Managed OS Disk : '/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/PACKER-RESOURCE-GROUP-HW8VW3G0Z9/providers/Microsoft.Compute/disks/pkroshw8vw3g0z9'
==> azure-arm: Querying the machine's additional disks properties ...
==> azure-arm: -> ResourceGroupName : 'packer-Resource-Group-hw8vw3g0z9'
==> azure-arm: -> ComputeName : 'pkrvmhw8vw3g0z9'
==> azure-arm: Powering off machine ...
==> azure-arm: -> ResourceGroupName : 'packer-Resource-Group-hw8vw3g0z9'
==> azure-arm: -> ComputeName : 'pkrvmhw8vw3g0z9'
==> azure-arm: Capturing image ...
==> azure-arm: -> Compute ResourceGroupName : 'packer-Resource-Group-hw8vw3g0z9'
==> azure-arm: -> Compute Name : 'pkrvmhw8vw3g0z9'
==> azure-arm: -> Compute Location : 'Korea Central'
==> azure-arm: -> Image ResourceGroupName : 'handson0223'
==> azure-arm: -> Image Name : 'zeroPackerImage'
==> azure-arm: -> Image Location : 'koreacentral'
==> azure-arm: Deleting resource group ...
==> azure-arm: -> ResourceGroupName : 'packer-Resource-Group-hw8vw3g0z9'
==> azure-arm:
==> azure-arm: The resource group was created by Packer, deleting ...
==> azure-arm: Deleting the temporary OS disk ...
==> azure-arm: -> OS Disk : skipping, managed disk was used...
==> azure-arm: Deleting the temporary Additional disk ...
==> azure-arm: -> Additional Disk : skipping, managed disk was used...
Build 'azure-arm' finished.
==> Builds finished. The artifacts of successful builds are:
--> azure-arm: Azure.ResourceManagement.VMImage:
OSType: Linux
ManagedImageResourceGroupName: handson0223
ManagedImageName: zeroPackerImage
ManagedImageId: /subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0223/providers/Microsoft.Compute/images/zeroPackerImage
ManagedImageLocation: koreacentral
기존 네트워크에 jumpbox 추가
이 선택적 단계를 사용하면 SSH에서 jumpbox를 사용하여 가상 머신 확장 집합의 인스턴스에 액세스할 수 있다.
기존 배포에 다음 리소스를 추가한다.
- 가상 머신 확장 집합과 동일한 서브넷에 연결된 네트워크 인터페이스
- 이 네트워크 인터페이스와 가상 머신
vmss.tf 파일의 끝에 다음 코드를 추가한다.
resource "azurerm_public_ip" "jumpbox" {
name = "jumpbox-public-ip"
location = var.location
resource_group_name = azurerm_resource_group.vmss.name
allocation_method = "Static"
domain_name_label = "${azurerm_resource_group.vmss.name}-ssh"
tags = {
environment = "codelab"
}
}
resource "azurerm_network_interface" "jumpbox" {
name = "jumpbox-nic"
location = var.location
resource_group_name = azurerm_resource_group.vmss.name
ip_configuration {
name = "IPConfiguration"
subnet_id = azurerm_subnet.vmss.id
private_ip_address_allocation = "dynamic"
public_ip_address_id = azurerm_public_ip.jumpbox.id
}
tags = {
environment = "codelab"
}
}
resource "azurerm_virtual_machine" "jumpbox" {
name = "jumpbox"
location = var.location
resource_group_name = azurerm_resource_group.vmss.name
network_interface_ids = [azurerm_network_interface.jumpbox.id]
vm_size = "Standard_DS1_v2"
storage_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = "16.04-LTS"
version = "latest"
}
storage_os_disk {
name = "jumpbox-osdisk"
caching = "ReadWrite"
create_option = "FromImage"
managed_disk_type = "Standard_LRS"
}
os_profile {
computer_name = "jumpbox"
admin_username = "azureuser"
admin_password = "Password1234!"
}
os_profile_linux_config {
disable_password_authentication = true
ssh_keys {
path = "/home/azureuser/.ssh/authorized_keys"
key_data = file("~/.ssh/id_rsa.pub")
}
}
tags = {
environment = "codelab"
}
}
outputs.tf를 편집하고 다음 코드를 추가하여 배포가 완료될 때 jumpbox의 호스트 이름을 표시한다.
output "jumpbox_public_ip" {
value = azurerm_public_ip.jumpbox.fqdn
}
jumpbox 배포
jumpbox를 배포한다.
| terraform apply |
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ terraform apply
azurerm_resource_group.vmss: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224]
azurerm_virtual_network.vmss: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/virtualNetworks/vmss-vnet]
azurerm_public_ip.vmss: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/publicIPAddresses/vmss-public-ip]
azurerm_subnet.vmss: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/virtualNetworks/vmss-vnet/subnets/vmss-subnet]
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# azurerm_network_interface.jumpbox will be created
+ resource "azurerm_network_interface" "jumpbox" {
+ applied_dns_servers = (known after apply)
+ dns_servers = (known after apply)
+ enable_accelerated_networking = false
+ enable_ip_forwarding = false
+ id = (known after apply)
+ internal_dns_name_label = (known after apply)
+ internal_fqdn = (known after apply)
+ location = "koreacentral"
+ mac_address = (known after apply)
+ name = "jumpbox-nic"
+ private_ip_address = (known after apply)
+ private_ip_addresses = (known after apply)
+ resource_group_name = "handson0224"
+ tags = {
+ "environment" = "codelab"
}
+ virtual_machine_id = (known after apply)
+ ip_configuration {
+ application_gateway_backend_address_pools_ids = (known after apply)
+ application_security_group_ids = (known after apply)
+ load_balancer_backend_address_pools_ids = (known after apply)
+ load_balancer_inbound_nat_rules_ids = (known after apply)
+ name = "IPConfiguration"
+ primary = (known after apply)
+ private_ip_address = (known after apply)
+ private_ip_address_allocation = "dynamic"
+ private_ip_address_version = "IPv4"
+ public_ip_address_id = (known after apply)
+ subnet_id = "/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/virtualNetworks/vmss-vnet/subnets/vmss-subnet"
}
}
# azurerm_public_ip.jumpbox will be created
+ resource "azurerm_public_ip" "jumpbox" {
+ allocation_method = "Static"
+ domain_name_label = "handson0224-ssh"
+ fqdn = (known after apply)
+ id = (known after apply)
+ idle_timeout_in_minutes = 4
+ ip_address = (known after apply)
+ ip_version = "IPv4"
+ location = "koreacentral"
+ name = "jumpbox-public-ip"
+ public_ip_address_allocation = (known after apply)
+ resource_group_name = "handson0224"
+ sku = "Basic"
+ tags = {
+ "environment" = "codelab"
}
}
# azurerm_virtual_machine.jumpbox will be created
+ resource "azurerm_virtual_machine" "jumpbox" {
+ availability_set_id = (known after apply)
+ delete_data_disks_on_termination = false
+ delete_os_disk_on_termination = false
+ id = (known after apply)
+ license_type = (known after apply)
+ location = "koreacentral"
+ name = "jumpbox"
+ network_interface_ids = (known after apply)
+ resource_group_name = "handson0224"
+ tags = {
+ "environment" = "codelab"
}
+ vm_size = "Standard_DS1_v2"
+ identity {
+ identity_ids = (known after apply)
+ principal_id = (known after apply)
+ type = (known after apply)
}
+ os_profile {
+ admin_password = (sensitive value)
+ admin_username = "azureuser"
+ computer_name = "jumpbox"
+ custom_data = (known after apply)
}
+ os_profile_linux_config {
+ disable_password_authentication = true
+ ssh_keys {
+ key_data = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCttszvheNvmlgaKGf9t2L1mT2bDMhGHCPbGvo9Ys1lSgBjK+FsOIK+v5J2so9G7JhSZ26EWHqeGqD3NpR9moDN0MzG9NsYwfyxQ4A+qQk0Z4mHvyykUz36X5cjBGBcLU/HThi063KRF3r1i2y3XRheTE6vFuZyJ7ixV8vLWprzqBvmHJpqcNf+ckkc6cidM4688RacVVctDyafWfDQwXrQPxTnHl2VW03zZH9jafzrwaCMgSYEL44kzuwBe9WusDS4ZgSfneHJy1ZgfoE6bf9NGEuEHFi80fTRkSQWvJlMU8pkwMraqJnJnwbHZgGp+ApBxuU35g0OAzXVhYR8fToT zerobig.kim@gmail.com\n"
+ path = "/home/azureuser/.ssh/authorized_keys"
}
}
+ storage_data_disk {
+ caching = (known after apply)
+ create_option = (known after apply)
+ disk_size_gb = (known after apply)
+ lun = (known after apply)
+ managed_disk_id = (known after apply)
+ managed_disk_type = (known after apply)
+ name = (known after apply)
+ vhd_uri = (known after apply)
+ write_accelerator_enabled = (known after apply)
}
+ storage_image_reference {
+ offer = "UbuntuServer"
+ publisher = "Canonical"
+ sku = "16.04-LTS"
+ version = "latest"
}
+ storage_os_disk {
+ caching = "ReadWrite"
+ create_option = "FromImage"
+ disk_size_gb = (known after apply)
+ managed_disk_id = (known after apply)
+ managed_disk_type = "Standard_LRS"
+ name = "jumpbox-osdisk"
+ os_type = (known after apply)
+ write_accelerator_enabled = false
}
}
Plan: 3 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
azurerm_public_ip.jumpbox: Creating...
azurerm_public_ip.jumpbox: Creation complete after 6s [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/publicIPAddresses/jumpbox-public-ip]
azurerm_network_interface.jumpbox: Creating...
azurerm_network_interface.jumpbox: Creation complete after 0s [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/networkInterfaces/jumpbox-nic]
azurerm_virtual_machine.jumpbox: Creating...
azurerm_virtual_machine.jumpbox: Still creating... [10s elapsed]
azurerm_virtual_machine.jumpbox: Still creating... [20s elapsed]
azurerm_virtual_machine.jumpbox: Still creating... [30s elapsed]
azurerm_virtual_machine.jumpbox: Still creating... [40s elapsed]
azurerm_virtual_machine.jumpbox: Still creating... [50s elapsed]
azurerm_virtual_machine.jumpbox: Still creating... [1m0s elapsed]
azurerm_virtual_machine.jumpbox: Still creating... [1m10s elapsed]
azurerm_virtual_machine.jumpbox: Still creating... [1m20s elapsed]
azurerm_virtual_machine.jumpbox: Still creating... [1m30s elapsed]
azurerm_virtual_machine.jumpbox: Creation complete after 1m32s [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Compute/virtualMachines/jumpbox]
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
Outputs:
jumpbox_public_ip = handson0224-ssh.koreacentral.cloudapp.azure.com
vmss_public_ip = handson0224.koreacentral.cloudapp.azure.com
배포가 완료된 후 리소스 그룹의 콘텐츠는 다음 이미지와 같다.
jumpbox SSH Key 생성
(참고로 원문에는 이 내용이 누락되어 있다.)
jumpbox로 ssh 접속하여 key를 생성한다.
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ ssh azureuser@handson0224-ssh.koreacentral.cloudapp.azure.com
The authenticity of host 'handson0224-ssh.koreacentral.cloudapp.azure.com (52.231.51.207)' can't be established.
ECDSA key fingerprint is SHA256:0vKzYGDmMKK08aZN0nBZ/zKKwULJoIetwQYJPcKWgxw.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'handson0224-ssh.koreacentral.cloudapp.azure.com,52.231.51.207' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 4.15.0-1071-azure x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
0 packages can be updated.
0 updates are security updates.
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
azureuser@jumpbox:~$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/azureuser/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/azureuser/.ssh/id_rsa.
Your public key has been saved in /home/azureuser/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:0478du2WxRcQtW6yGxfmOjCPisw5mBlBoa6h2Wu0V58 azureuser@jumpbox
The key's randomart image is:
+---[RSA 2048]----+
| .. .o. |
| .. . .|
| .. .. |
| . . . .. |
| . . . S . . *.|
|.o+ ... + o * =|
|oo.. .=.o.. =+ =.|
| o..+o.E....oO |
| ... =.oo. =o |
+----[SHA256]-----+
azureuser@jumpbox:~$ cat ~/.ssh/id_rsa.pub
ssh-rsa AAAAB3Nza... azureuser@jumpbox
생성된 key 파일을 로컬에 파일로 복사하여 위치시킨다.
azureuser@jumpbox:~$ exit
logout
ssh-rsa AAAAB3NzaC1yc2EAAAADAQ...Connection to handson0224-ssh.koreacentral.cloudapp.azure.com closed.
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ vi jumpbox_key
ssh-rsa AAAAB3Nza... azureuser@jumpbox
인프라를 편집하여 가상 머신 확장 집합 추가
이 단계에서는 이전에 배포한 네트워크에서 다음 리소스를 만든다.
- 애플리케이션을 제공하는 Azure 부하 분산 장치. 이전에 배포된 공용 IP 주소에 연결한다.
- 애플리케이션을 제공하는 하나의 Azure 부하 분산 장치 및 규칙. 이전에 구성된 공용 IP 주소에 연결한다.
- Azure 백 엔드 주소 풀. 부하 분산 장치에 할당한다.
- 애플리케이션에서 사용되고 부하 분산 장치에 구성된 상태 프로브 포트.
- 부하 분산 장치 뒤에 있고 이전에 배포된 가상 네트워크에서 실행되는 가상 머신 확장 집합.
- 사용자 지정 이미지에서 설치된 가상 머신 확장 노드의 Nginx
vmss.tf 파일의 끝에 다음 코드를 추가한다.
이 때 생성한 packer 이미지에 대한 리소스 그룹과 이미지명을 각자의 상황에 맞게 수정하여야 한다.
또한 위에서 생성한 ssh 키 파일 정보를 각자의 상황에 맞게 수정하여야 한다. (이 부분이 잘못되어 있다.)
resource "azurerm_lb" "vmss" {
name = "vmss-lb"
location = var.location
resource_group_name = azurerm_resource_group.vmss.name
frontend_ip_configuration {
name = "PublicIPAddress"
public_ip_address_id = azurerm_public_ip.vmss.id
}
tags = {
environment = "codelab"
}
}
resource "azurerm_lb_backend_address_pool" "bpepool" {
resource_group_name = azurerm_resource_group.vmss.name
loadbalancer_id = azurerm_lb.vmss.id
name = "BackEndAddressPool"
}
resource "azurerm_lb_probe" "vmss" {
resource_group_name = azurerm_resource_group.vmss.name
loadbalancer_id = azurerm_lb.vmss.id
name = "ssh-running-probe"
port = var.application_port
}
resource "azurerm_lb_rule" "lbnatrule" {
resource_group_name = azurerm_resource_group.vmss.name
loadbalancer_id = azurerm_lb.vmss.id
name = "http"
protocol = "Tcp"
frontend_port = var.application_port
backend_port = var.application_port
backend_address_pool_id = azurerm_lb_backend_address_pool.bpepool.id
frontend_ip_configuration_name = "PublicIPAddress"
probe_id = azurerm_lb_probe.vmss.id
}
data "azurerm_resource_group" "image" {
name = "handson0224"
}
data "azurerm_image" "image" {
name = "zeroPackerImage"
resource_group_name = data.azurerm_resource_group.image.name
}
resource "azurerm_virtual_machine_scale_set" "vmss" {
name = "vmscaleset"
location = var.location
resource_group_name = azurerm_resource_group.vmss.name
upgrade_policy_mode = "Manual"
sku {
name = "Standard_DS1_v2"
tier = "Standard"
capacity = 2
}
storage_profile_image_reference {
id=data.azurerm_image.image.id
}
storage_profile_os_disk {
name = ""
caching = "ReadWrite"
create_option = "FromImage"
managed_disk_type = "Standard_LRS"
}
storage_profile_data_disk {
lun = 0
caching = "ReadWrite"
create_option = "Empty"
disk_size_gb = 10
}
os_profile {
computer_name_prefix = "vmlab"
admin_username = "azureuser"
admin_password = "Passwword1234"
}
os_profile_linux_config {
disable_password_authentication = true
ssh_keys {
path = "/home/azureuser/.ssh/authorized_keys"
key_data = file("./jumpbox_key")
}
}
network_profile {
name = "terraformnetworkprofile"
primary = true
ip_configuration {
name = "IPConfiguration"
subnet_id = azurerm_subnet.vmss.id
load_balancer_backend_address_pool_ids = [azurerm_lb_backend_address_pool.bpepool.id]
primary = true
}
}
tags = {
environment = "codelab"
}
}
variables.tf에 다음 코드를 추가하여 배포를 사용자 지정한다.
variable "application_port" {
description = "The port that you want to expose to the external load balancer"
default = 80
}
variable "admin_password" {
description = "Default password for admin"
default = "Passwwoord11223344"
}
Azure에서 가상 머신 확장 집합 배포
다음 명령을 실행하여 가상 머신 확장 집합 배포를 시각화하고 Azure에서 새 리소스를 배포한다.
|
terraform plan terraform apply |
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.azurerm_resource_group.image: Refreshing state...
azurerm_resource_group.vmss: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224]
azurerm_public_ip.vmss: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/publicIPAddresses/vmss-public-ip]
azurerm_virtual_network.vmss: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/virtualNetworks/vmss-vnet]
azurerm_public_ip.jumpbox: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/publicIPAddresses/jumpbox-public-ip]
data.azurerm_image.image: Refreshing state...
azurerm_subnet.vmss: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/virtualNetworks/vmss-vnet/subnets/vmss-subnet]
azurerm_network_interface.jumpbox: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/networkInterfaces/jumpbox-nic]
azurerm_virtual_machine.jumpbox: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Compute/virtualMachines/jumpbox]
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# azurerm_lb.vmss will be created
+ resource "azurerm_lb" "vmss" {
+ id = (known after apply)
+ location = "koreacentral"
+ name = "vmss-lb"
+ private_ip_address = (known after apply)
+ private_ip_addresses = (known after apply)
+ resource_group_name = "handson0224"
+ sku = "Basic"
+ tags = {
+ "environment" = "codelab"
}
+ frontend_ip_configuration {
+ id = (known after apply)
+ inbound_nat_rules = (known after apply)
+ load_balancer_rules = (known after apply)
+ name = "PublicIPAddress"
+ outbound_rules = (known after apply)
+ private_ip_address = (known after apply)
+ private_ip_address_allocation = (known after apply)
+ public_ip_address_id = "/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/publicIPAddresses/vmss-public-ip"
+ public_ip_prefix_id = (known after apply)
+ subnet_id = (known after apply)
}
}
# azurerm_lb_backend_address_pool.bpepool will be created
+ resource "azurerm_lb_backend_address_pool" "bpepool" {
+ backend_ip_configurations = (known after apply)
+ id = (known after apply)
+ load_balancing_rules = (known after apply)
+ loadbalancer_id = (known after apply)
+ name = "BackEndAddressPool"
+ resource_group_name = "handson0224"
}
# azurerm_lb_probe.vmss will be created
+ resource "azurerm_lb_probe" "vmss" {
+ id = (known after apply)
+ interval_in_seconds = 15
+ load_balancer_rules = (known after apply)
+ loadbalancer_id = (known after apply)
+ name = "ssh-running-probe"
+ number_of_probes = 2
+ port = 80
+ protocol = (known after apply)
+ resource_group_name = "handson0224"
}
# azurerm_lb_rule.lbnatrule will be created
+ resource "azurerm_lb_rule" "lbnatrule" {
+ backend_address_pool_id = (known after apply)
+ backend_port = 80
+ disable_outbound_snat = false
+ enable_floating_ip = false
+ frontend_ip_configuration_id = (known after apply)
+ frontend_ip_configuration_name = "PublicIPAddress"
+ frontend_port = 80
+ id = (known after apply)
+ idle_timeout_in_minutes = (known after apply)
+ load_distribution = (known after apply)
+ loadbalancer_id = (known after apply)
+ name = "http"
+ probe_id = (known after apply)
+ protocol = "Tcp"
+ resource_group_name = "handson0224"
}
# azurerm_virtual_machine_scale_set.vmss will be created
+ resource "azurerm_virtual_machine_scale_set" "vmss" {
+ automatic_os_upgrade = false
+ id = (known after apply)
+ license_type = (known after apply)
+ location = "koreacentral"
+ name = "vmscaleset"
+ overprovision = true
+ resource_group_name = "handson0224"
+ single_placement_group = true
+ tags = {
+ "environment" = "codelab"
}
+ upgrade_policy_mode = "Manual"
+ identity {
+ identity_ids = (known after apply)
+ principal_id = (known after apply)
+ type = (known after apply)
}
+ network_profile {
+ ip_forwarding = false
+ name = "terraformnetworkprofile"
+ primary = true
+ ip_configuration {
+ application_gateway_backend_address_pool_ids = []
+ application_security_group_ids = []
+ load_balancer_backend_address_pool_ids = (known after apply)
+ load_balancer_inbound_nat_rules_ids = (known after apply)
+ name = "IPConfiguration"
+ primary = true
+ subnet_id = "/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/virtualNetworks/vmss-vnet/subnets/vmss-subnet"
}
}
+ os_profile {
+ admin_password = (sensitive value)
+ admin_username = "azureuser"
+ computer_name_prefix = "vmlab"
}
+ os_profile_linux_config {
+ disable_password_authentication = true
+ ssh_keys {
+ key_data = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc5XOLzlROszUmSRqD+KY37LCmK0OYN/q87U37ggCE2fVmBVQtyZYpNd6RbWgCvtNIzpgw9Rwu6/gqrDf5V/9aJElUDiD5DNVto0YbMY5vX2mUofQ+c+hKhZ8MN7bvtpkrfsLXAfMrbTmWr34tu3XkMuN/duCx/I9EbifpvGuO5L2v4r1UG9/C6zJn64+DFQhZyaX9nr6RXvpnxs7hRGJSQpC/wboEwbHjoaUNMsTYtoeF+kVk7JIQo1kUrLSdBIFNGtBJIq6gSyLVmN0fopBqoJJ/ivnEqq0KIB5CK21254umxZlnQjkBS3MTGMc5HWoBLa27a0SV3kbispH/yHyZ azureuser@jumpbox\n"
+ path = "/home/azureuser/.ssh/authorized_keys"
}
}
+ sku {
+ capacity = 2
+ name = "Standard_DS1_v2"
+ tier = "Standard"
}
+ storage_profile_data_disk {
+ caching = "ReadWrite"
+ create_option = "Empty"
+ disk_size_gb = 10
+ lun = 0
+ managed_disk_type = (known after apply)
}
+ storage_profile_image_reference {
+ id = "/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Compute/images/zeroPackerImage"
}
+ storage_profile_os_disk {
+ caching = "ReadWrite"
+ create_option = "FromImage"
+ managed_disk_type = "Standard_LRS"
+ vhd_containers = []
}
}
Plan: 5 to add, 0 to change, 0 to destroy.
------------------------------------------------------------------------
Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ terraform apply
data.azurerm_resource_group.image: Refreshing state...
azurerm_resource_group.vmss: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224]
azurerm_public_ip.jumpbox: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/publicIPAddresses/jumpbox-public-ip]
azurerm_public_ip.vmss: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/publicIPAddresses/vmss-public-ip]
azurerm_virtual_network.vmss: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/virtualNetworks/vmss-vnet]
data.azurerm_image.image: Refreshing state...
azurerm_subnet.vmss: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/virtualNetworks/vmss-vnet/subnets/vmss-subnet]
azurerm_network_interface.jumpbox: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/networkInterfaces/jumpbox-nic]
azurerm_virtual_machine.jumpbox: Refreshing state... [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Compute/virtualMachines/jumpbox]
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# azurerm_lb.vmss will be created
+ resource "azurerm_lb" "vmss" {
+ id = (known after apply)
+ location = "koreacentral"
+ name = "vmss-lb"
+ private_ip_address = (known after apply)
+ private_ip_addresses = (known after apply)
+ resource_group_name = "handson0224"
+ sku = "Basic"
+ tags = {
+ "environment" = "codelab"
}
+ frontend_ip_configuration {
+ id = (known after apply)
+ inbound_nat_rules = (known after apply)
+ load_balancer_rules = (known after apply)
+ name = "PublicIPAddress"
+ outbound_rules = (known after apply)
+ private_ip_address = (known after apply)
+ private_ip_address_allocation = (known after apply)
+ public_ip_address_id = "/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/publicIPAddresses/vmss-public-ip"
+ public_ip_prefix_id = (known after apply)
+ subnet_id = (known after apply)
}
}
# azurerm_lb_backend_address_pool.bpepool will be created
+ resource "azurerm_lb_backend_address_pool" "bpepool" {
+ backend_ip_configurations = (known after apply)
+ id = (known after apply)
+ load_balancing_rules = (known after apply)
+ loadbalancer_id = (known after apply)
+ name = "BackEndAddressPool"
+ resource_group_name = "handson0224"
}
# azurerm_lb_probe.vmss will be created
+ resource "azurerm_lb_probe" "vmss" {
+ id = (known after apply)
+ interval_in_seconds = 15
+ load_balancer_rules = (known after apply)
+ loadbalancer_id = (known after apply)
+ name = "ssh-running-probe"
+ number_of_probes = 2
+ port = 80
+ protocol = (known after apply)
+ resource_group_name = "handson0224"
}
# azurerm_lb_rule.lbnatrule will be created
+ resource "azurerm_lb_rule" "lbnatrule" {
+ backend_address_pool_id = (known after apply)
+ backend_port = 80
+ disable_outbound_snat = false
+ enable_floating_ip = false
+ frontend_ip_configuration_id = (known after apply)
+ frontend_ip_configuration_name = "PublicIPAddress"
+ frontend_port = 80
+ id = (known after apply)
+ idle_timeout_in_minutes = (known after apply)
+ load_distribution = (known after apply)
+ loadbalancer_id = (known after apply)
+ name = "http"
+ probe_id = (known after apply)
+ protocol = "Tcp"
+ resource_group_name = "handson0224"
}
# azurerm_virtual_machine_scale_set.vmss will be created
+ resource "azurerm_virtual_machine_scale_set" "vmss" {
+ automatic_os_upgrade = false
+ id = (known after apply)
+ license_type = (known after apply)
+ location = "koreacentral"
+ name = "vmscaleset"
+ overprovision = true
+ resource_group_name = "handson0224"
+ single_placement_group = true
+ tags = {
+ "environment" = "codelab"
}
+ upgrade_policy_mode = "Manual"
+ identity {
+ identity_ids = (known after apply)
+ principal_id = (known after apply)
+ type = (known after apply)
}
+ network_profile {
+ ip_forwarding = false
+ name = "terraformnetworkprofile"
+ primary = true
+ ip_configuration {
+ application_gateway_backend_address_pool_ids = []
+ application_security_group_ids = []
+ load_balancer_backend_address_pool_ids = (known after apply)
+ load_balancer_inbound_nat_rules_ids = (known after apply)
+ name = "IPConfiguration"
+ primary = true
+ subnet_id = "/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/virtualNetworks/vmss-vnet/subnets/vmss-subnet"
}
}
+ os_profile {
+ admin_password = (sensitive value)
+ admin_username = "azureuser"
+ computer_name_prefix = "vmlab"
}
+ os_profile_linux_config {
+ disable_password_authentication = true
+ ssh_keys {
+ key_data = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc5XOLzlROszUmSRqD+KY37LCmK0OYN/q87U37ggCE2fVmBVQtyZYpNd6RbWgCvtNIzpgw9Rwu6/gqrDf5V/9aJElUDiD5DNVto0YbMY5vX2mUofQ+c+hKhZ8MN7bvtpkrfsLXAfMrbTmWr34tu3XkMuN/duCx/I9EbifpvGuO5L2v4r1UG9/C6zJn64+DFQhZyaX9nr6RXvpnxs7hRGJSQpC/wboEwbHjoaUNMsTYtoeF+kVk7JIQo1kUrLSdBIFNGtBJIq6gSyLVmN0fopBqoJJ/ivnEqq0KIB5CK21254umxZlnQjkBS3MTGMc5HWoBLa27a0SV3kbispH/yHyZ azureuser@jumpbox\n"
+ path = "/home/azureuser/.ssh/authorized_keys"
}
}
+ sku {
+ capacity = 2
+ name = "Standard_DS1_v2"
+ tier = "Standard"
}
+ storage_profile_data_disk {
+ caching = "ReadWrite"
+ create_option = "Empty"
+ disk_size_gb = 10
+ lun = 0
+ managed_disk_type = (known after apply)
}
+ storage_profile_image_reference {
+ id = "/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Compute/images/zeroPackerImage"
}
+ storage_profile_os_disk {
+ caching = "ReadWrite"
+ create_option = "FromImage"
+ managed_disk_type = "Standard_LRS"
+ vhd_containers = []
}
}
Plan: 5 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
azurerm_lb.vmss: Creating...
azurerm_lb.vmss: Creation complete after 0s [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/loadBalancers/vmss-lb]
azurerm_lb_probe.vmss: Creating...
azurerm_lb_backend_address_pool.bpepool: Creating...
azurerm_lb_probe.vmss: Creation complete after 0s [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/loadBalancers/vmss-lb/probes/ssh-running-probe]
azurerm_lb_backend_address_pool.bpepool: Creation complete after 0s [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/loadBalancers/vmss-lb/backendAddressPools/BackEndAddressPool]
azurerm_lb_rule.lbnatrule: Creating...
azurerm_virtual_machine_scale_set.vmss: Creating...
azurerm_lb_rule.lbnatrule: Creation complete after 1s [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Network/loadBalancers/vmss-lb/loadBalancingRules/http]
azurerm_virtual_machine_scale_set.vmss: Still creating... [10s elapsed]
azurerm_virtual_machine_scale_set.vmss: Still creating... [20s elapsed]
azurerm_virtual_machine_scale_set.vmss: Still creating... [30s elapsed]
azurerm_virtual_machine_scale_set.vmss: Still creating... [40s elapsed]
azurerm_virtual_machine_scale_set.vmss: Still creating... [50s elapsed]
azurerm_virtual_machine_scale_set.vmss: Still creating... [1m0s elapsed]
azurerm_virtual_machine_scale_set.vmss: Creation complete after 1m2s [id=/subscriptions/f8764f39-xxxx-xxxx-xxxx-77b286ed971b/resourceGroups/handson0224/providers/Microsoft.Compute/virtualMachineScaleSets/vmscaleset]
Apply complete! Resources: 5 added, 0 changed, 0 destroyed.
Outputs:
jumpbox_public_ip = handson0224-ssh.koreacentral.cloudapp.azure.com
vmss_public_ip = handson0224.koreacentral.cloudapp.azure.com
리소스 그룹의 내용은 다음 이미지와 같다.
브라우저를 열고 명령에 의해 반환된 정규화된 도메인 이름에 연결한다.
7. 브라우저를 열고 명령에 의해 반환된 FQDN에 연결한다.
|
참고 배포한 가상 머신 확장 집합 및 jumpbox에서 암호를 사용하여 로그인하는 기능이 사용하지 않도록 설정되었다. 가상 머신에 액세스하려면 SSH를 사용하여 로그인한다.
|
이제 jumpbox로 접속한 후 vmss 인스턴스의 내부 IP를 가지고 SSH 접속을 시도하여 결과를 검증한다.
vmss의 내부 IP 정보를 확인하는 명령은 다음과 같다.
| az vmss nic list -g myResourceGroup --vmss-name vmssName | grep -w "privateIpAddress" |
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ az vmss nic list -g handson0224 --vmss-name vmscaleset | grep -w "privateIpAddress"
"privateIpAddress": "10.0.2.6",
"privateIpAddress": "10.0.2.7",
vmss의 내부 IP 정보를 얻었으니 jumpbox로 ssh 접속을 시도 후 다시 vmss로 ssh 접속을 시도한다.
zerobig@ZEROBIG-NT800:/mnt/d/Azure_DevOps_Study/20200223_terraform_handson$ ssh azureuser@handson0224-ssh.koreacentral.cloudapp.azure.com
Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 4.15.0-1071-azure x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
0 updates are security updates.
New release '18.04.4 LTS' available.
Run 'do-release-upgrade' to upgrade to it.
Last login: Mon Feb 24 09:58:56 2020 from 210.219.181.78
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
azureuser@jumpbox:~$ ssh azureuser@10.0.2.6
The authenticity of host '10.0.2.6 (10.0.2.6)' can't be established.
ECDSA key fingerprint is SHA256:/pnMxnvms4TMkiPXyQ7/gyejY58zNaRoY6bRDJ2g/m8.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.0.2.6' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 4.15.0-1071-azure x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
0 packages can be updated.
0 updates are security updates.
New release '18.04.4 LTS' available.
Run 'do-release-upgrade' to upgrade to it.
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
azureuser@vmlab000001:~$
패스워드 입력 필요없이 정상적으로 접속이 이루어짐을 확인할 수 있다.
환경 정리
결과가 확인되면 terraform destroy를 통해 배포한 리소스를 제거한다.
| terraform destroy |
리소스 삭제를 확인하는 메시지가 표시되면 yes를 입력한다. 소멸 프로세스를 완료하는 데 몇 분 정도가 걸릴 수 있다.