Technomusing

Musings of technology

Running TeamCity Server on AWS - Part 3

| Comments

In part 1 of this series, I provided a high-level explanation of the project, its requirements and technologies to be used for running TeamCity Server on Amazon Web Services.

In part 2, I created a CloudFormation template to allocate the basic AWS resources to be used for the project including the auto-scale group, launch configuration and security group.

In this post, I’ll cover the installation details of TeamCity server and automating the install with the cloud formation helper scripts.

CloudFormation helper scripts

CloudFormation provides a few helper scripts that let you automate the installation of software on EC2 instances. We’ll use two of the tools:

  • cfn-init
  • cfn-hup

Cfn-init is an initialization tool. It’s purpose is to install packages, extract zip archives and configure services based on metadata associated with the LaunchConfiguration.

Cfn-hup is a monitoring tool and periodically checks for changes to the CloudFormation template and invokes update commands on the EC2 instance when a change is detected. We’ll configure cfn-hup in a future post.

Hosting installation files

I used S3 for hosting the installation files. This is so that I’m not relying on 3rd-parties to have their download sites available, also file transfers from S3 to EC2 within the same region are free.

Using the AWS management console or your favorite S3 tool, create a new bucket. This bucket will be used for hosting the installation files and your instance backups. I chose ‘teamcityserver.patrickshafer.com’, but you can use any name that suits you.

I created a ‘pub’ folder and uploaded the latest installation file from TeamCity. Make sure you’re using the tar.gz download, not the setup executable.

TeamCity Server installation script

This is the batch file we’ll use for installing TeamCity Server. You’ll notice that it has 4 major sections:

  • Install: this copies the files and installs the windows service.
  • Uninstall: this uninstalls the windows service and removes the files
  • Start: this launches TeamCity
  • Stop: this stops the TeamCity server

You may be wondering why we have an uninstall? The reason is that we may need to change the source package for our TeamCity Server (eg, a new version is released). If we deploy an update to our existing TeamCity Server stack with a new version of teamcity, uninstall will need to be used to make sure that we have a clean install of the new version.

You may also be wondering why start and stop are split out and not part of the install/uninstall process- basically, for doing our periodic backups, we’ll have to stop the TeamCity Server before we can make a copy of the data directory. This allows us to start and stop TeamCity server without uninstalling and reinstalling.

The installation steps are pretty straight forward:

  1. Configure the teamcity_data_path environment variable. This variable tells TeamCity where to store its files and HLSQL DB.
  2. Copy the TeamCity server files to Z:\TeamCityServer
  3. Create a rule for the Windows Firewall that allows TCP 80 connections to the machine
  4. Install the windows service.

Here is the batch file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
@ECHO OFF
IF "%1"=="install" GOTO install
IF "%1"=="uninstall" GOTO uninstall
IF "%1"=="start" GOTO start
IF "%1"=="stop" GOTO stop
GOTO end

:install
SETX TEAMCITY_DATA_PATH Z:\TeamCityServerData /m
XCOPY Z:\TCServer-8.1.2 Z:\TeamCityServer /s /q /i
NETSH advfirewall firewall add rule name="TeamCity Server" dir=in action=allow protocol=TCP localport=80
CALL Z:\TeamCityServer\bin\teamcity-server service install /runAsSystem
GOTO end

:uninstall
CALL Z:\TeamCityServer\bin\teamcity-server service delete
NETSH advfirewall firewall delete rule name="TeamCity Server"
RMDIR /s /q Z:\TeamCityServer
GOTO end

:start
NET start TeamCity
GOTO end

:stop
NET stop TeamCity
GOTO end

:end

In the metadata from the below section, you’ll see that the TeamCity tar is extracted to Z:\TCServer-8.1.2. The installation XCOPY step copies the contents to Z:\TeamCityServer.

LaunchConfiguration metadata

The LaunchConfiguration will store a copy of the above batch file along with a directive for downloading and extracting the TeamCity Server files.

We encode tcsmanage.cmd (from above) by escaping quotes and adding EOL characters (\n), then use the CloudFormation Fn::Join function to combine all the lines into one file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
"Metadata" : {
  "AWS::CloudFormation::Init" : {
      "configSets" : {
          "all" : ["server"],
          "serverSet" : ["server"],
      },
      "server" : {
          "files" : {
              "Z:\\tcsmanage.cmd" : {
                  "content" : {
                      "Fn::Join" : ["", [
                          "@ECHO OFF\n",
                          "IF \"%1\"==\"install\" GOTO install\n",
                          "IF \"%1\"==\"uninstall\" GOTO uninstall\n",
                          "IF \"%1\"==\"start\" GOTO start\n",
                          "IF \"%1\"==\"stop\" GOTO stop\n",
                          "GOTO end\n",
                          "\n",
                          ":install\n",
                          "SETX TEAMCITY_DATA_PATH Z:\\TeamCityServerData /m\n",
                          "XCOPY Z:\\TCServer-8.1.2 Z:\\TeamCityServer /s /q /i\n",
                          "NETSH advfirewall firewall add rule name=\"TeamCity Server\" dir=in action=allow protocol=TCP localport=80\n",
                          "CALL Z:\\TeamCityServer\\bin\\teamcity-server service install /runAsSystem\n",
                          "GOTO end\n",
                          "\n",
                          ":uninstall\n",
                          "CALL Z:\\TeamCityServer\\bin\\teamcity-server service delete\n",
                          "NETSH advfirewall firewall delete rule name=\"TeamCity Server\"\n",
                          "RMDIR /s /q Z:\\TeamCityServer\n",
                          "GOTO end\n",
                          "\n",
                          ":start\n",
                          "NET start TeamCity\n",
                          "GOTO end\n",
                          "\n",
                          ":stop\n",
                          "NET stop TeamCity\n",
                          "GOTO end\n",
                          "\n",
                          ":end\n",
                          "\n"
                          ]]
                  }
              }
          },
          "commands" : {
              "1-stop" : {
                  "command" : "Z:\\tcsmanage.cmd stop",
                  "waitAfterCompletion" : 5,
                  "ignoreErrors" : "true"
              },
              "2-uninstall" : {
                  "command" : "Z:\\tcsmanage.cmd uninstall",
                  "waitAfterCompletion" : 5,
                  "ignoreErrors" : "true"
              },
              "3-install" : {
                  "command" : "Z:\\tcsmanage.cmd install",
                  "waitAfterCompletion" : 5
              },
              "4-start" : {
                  "command" : "Z:\\tcsmanage.cmd start",
                  "waitAfterCompletion" : 120
              }
          },
          "sources" : {
              "Z:\\TCServer-8.1.2" : "http://teamcityserver.patrickshafer.com/pub/TCServer-8.1.2.zip"
          }
      }
  }
}

Invoking cfn-init

In order for cfn-init to run the installation script, we have to invoke it. We use CloudInit to do this by passing in the following in the UserData field of the LaunchConfiguration. Here is the updated UserData field:

1
2
3
<script>
cfn-init.exe --stack [StackName] --resource LaunchConfiguration --region [Region] --configsets all -v
</script>

The script for CF

1
2
3
4
5
6
7
8
9
10
11
"UserData" : {
  "Fn::Base64" : {
      "Fn::Join" : ["",
                ["<script>\n", "cfn-init.exe --stack ", {
                  "Ref" : "AWS::StackName"
              }, " --resource LaunchConfiguration --region ", {
                  "Ref" : "AWS::Region"
              }, " --configsets all -v\n", "</script>"]
        ]
  }
}

Combined Template

Here is the combined template to publish to CloudFormation:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Description" : "TeamCity Server for Windows Server 2008 R2",
  "Resources":
  {
      "AutoScaleGroup" : {
          "Type" : "AWS::AutoScaling::AutoScalingGroup",
          "Properties" : {
              "AvailabilityZones" : { "Fn::GetAZs" : { "Ref" : "AWS::Region" }},
              "LaunchConfigurationName" : { "Ref" : "LaunchConfiguration" },
              "MaxSize" : "1",
              "MinSize" : "0",
              "DesiredCapacity" : "1"
          }
      },
      "LaunchConfiguration": {
          "Type" : "AWS::AutoScaling::LaunchConfiguration",
          "Properties" : {
              "ImageId" : "ami-d9e09ee9",
              "InstanceType" : "m1.small",
              "KeyName" : "us-west-2",
              "SecurityGroups" : [{ "Ref" : "SecurityGroup" }],
              "SpotPrice" : "0.027",
              "UserData" : {
                  "Fn::Base64" : {
                      "Fn::Join" : ["", [
                          "<script>\n",
                          "cfn-init.exe --stack ", { "Ref" : "AWS::StackName" },
                          " --resource LaunchConfiguration --region ", { "Ref" : "AWS::Region" },
                          " --configsets all -v\n",
                          "</script>"]
                      ]
                  }
              }
          },
          "Metadata" : {
              "AWS::CloudFormation::Init" : {
                  "configSets" : {
                      "all" : ["server"],
                      "serverSet" : ["server"]
                  },
                  "server" : {
                      "files" : {
                          "Z:\\tcsmanage.cmd" : {
                              "content" : {
                                  "Fn::Join" : ["", [
                                      "@ECHO OFF\n",
                                      "IF \"%1\"==\"install\" GOTO install\n",
                                      "IF \"%1\"==\"uninstall\" GOTO uninstall\n",
                                      "IF \"%1\"==\"start\" GOTO start\n",
                                      "IF \"%1\"==\"stop\" GOTO stop\n",
                                      "GOTO end\n",
                                      "\n",
                                      ":install\n",
                                      "SETX TEAMCITY_DATA_PATH Z:\\TeamCityServerData /m\n",
                                      "XCOPY Z:\\TCServer-8.1.2 Z:\\TeamCityServer /s /q /i\n",
                                      "NETSH advfirewall firewall add rule name=\"TeamCity Server\" dir=in action=allow protocol=TCP localport=80\n",
                                      "CALL Z:\\TeamCityServer\\bin\\teamcity-server service install /runAsSystem\n",
                                      "GOTO end\n",
                                      "\n",
                                      ":uninstall\n",
                                      "CALL Z:\\TeamCityServer\\bin\\teamcity-server service delete\n",
                                      "NETSH advfirewall firewall delete rule name=\"TeamCity Server\"\n",
                                      "RMDIR /s /q Z:\\TeamCityServer\n",
                                      "GOTO end\n",
                                      "\n",
                                      ":start\n",
                                      "NET start TeamCity\n",
                                      "GOTO end\n",
                                      "\n",
                                      ":stop\n",
                                      "NET stop TeamCity\n",
                                      "GOTO end\n",
                                      "\n",
                                      ":end\n",
                                      "\n"
                                      ]
                                  ]
                              }
                          }
                      },
                      "commands" : {
                          "1-stop" : {
                              "command" : "Z:\\tcsmanage.cmd stop",
                              "waitAfterCompletion" : 5,
                              "ignoreErrors" : "true"
                          },
                          "2-uninstall" : {
                              "command" : "Z:\\tcsmanage.cmd uninstall",
                              "waitAfterCompletion" : 5,
                              "ignoreErrors" : "true"
                          },
                          "3-install" : {
                              "command" : "Z:\\tcsmanage.cmd install",
                              "waitAfterCompletion" : 5
                          },
                          "4-start" : {
                              "command" : "Z:\\tcsmanage.cmd start",
                              "waitAfterCompletion" : 120
                          }
                      },
                      "sources" : {
                          "Z:\\TCServer-8.1.2" : "http://teamcityserver.patrickshafer.com/pub/TCServer-8.1.2.zip"
                      }
                  }
              }
          }
      },
      "SecurityGroup" : {
            "Type" : "AWS::EC2::SecurityGroup",
            "Properties" : {
                "GroupDescription" : "TeamCity Server Security Group",
                "SecurityGroupIngress" : [
                  { "IpProtocol" : "tcp", "CidrIp" : "**Your IP here**", "FromPort" : "3389", "ToPort" : "3389" },
                  { "IpProtocol" : "tcp", "CidrIp" : "0.0.0.0/0", "FromPort" : "80", "ToPort" : "80" }
                ]
            }
        }
  }
}

Once you create or update your stack with this template (after replacing **Your IP Here** with your RDP CDIR), you should be able to visit your instance URL and see the TeamCity Server startup screen: