03/24/2020: Using Boto3 To Create NS Hosted Zone Record Set
Acknowledgements
This work is being done at the request of the Enterprise Container Working Group (ECWG) of the Office of Information and Technology (OIT - https://www.oit.va.gov/) at the Department of Veteran Affairs.
Article
I wanted to create a qwerty.va-oit.cloud sub-domain using a set of four name servers.
nameservers = ['ns-657.awsdns-18.net', 'ns-422.awsdns-52.com', 'ns-1995.awsdns-57.co.uk', 'ns-1469.awsdns-55.org']
I could not find example code how to do this. It’s not terribly complicated but providing an example might safe someone time so I’ll take a few minutes.
The first step is to add a period to the end of each nameserver to create an fully-qualified domain name (FQDN).
nameservers = ['{}.'.format(nameserver) for nameserver in nameservers]
The example from the boto3 documentation shows the following (which I have shortened):
response = client.change_resource_record_sets(
HostedZoneId='string',
ChangeBatch={
'Changes': [{
'Action': 'UPSERT',
'ResourceRecordSet': {
'Name': 'string',
'Type': 'NS',
'TTL': 123,
'ResourceRecords': [{'Value': 'string'},],
}
}]
}
)
I tried various ways to get the nameserver list into that string associated with value. However, it wasn’t until I realized that ResourceRecords actually was an array that I understood what was needed.
The following code converts from an “array of strings” to an “array of dict”.
nameserver_resource_records = [{'Value': nameserver} for nameserver in nameservers]
With the “array of dict”, calling the upsert becomes trivial.
response = client.change_resource_record_sets(
HostedZoneId=domain_hosted_zone_id,
ChangeBatch={
'Changes': [
{
'Action': 'UPSERT',
'ResourceRecordSet': {
'Name': sub_domain_name,
'Type': 'NS',
'TTL': 60,
'ResourceRecords': nameserver_resource_records
}
}
]
}
)
10/15/2018: Provisioning SonarQube With CloudFormation
Provisioning SonarQube typically takes a few manual steps after the software itself is installed. For example, you need to change the default password and generate a token.
The process shown below uses the internal database so needs to be modified so large scale use. However, if you just have a few projects, this approach should be fine.
As always, please adjust the files to your situation. Pay special attention to the parameters in the CloudFormation template.
The project is located at https://github.com/medined/provision-sonarqube-using-cloudformation. You’ll find the CloudFormation yaml file there.
In this post, I want to show the script I use to wrap around the CloudFormation script. Some things are done first. And some afterward.
- Before
- Generate random password.
- Save password to Parameter Store.
- Execute the CloudFormation stack.
- After
- Get IP of SonarQube server.
- Wait for server to respond to API request.
- Wait until server is no longer STARTING.
- Verify server is responding with UP.
- Get password from Parameter Store.
- Change the default password.
- Get a sonar token.
- Save token in Parameter Store.
Below is the shell script that implements the above steps.
#!/bin/bash
if [ -z $REGION_NAME ]; then
echo "Set REGION_NAME"
exit 1
fi
#############################
echo "Saving random password to Parameter Store"
#############################
echo "$(date)" > floop.tmp
RPASSWORD=\$(sha256sum floop.tmp)
rm floop.tmp
aws ssm put-parameter \
--name sonar-password \
--value ${RPASSWORD} \
--type String \
--overwrite
#
#############################
echo "Sonar started."
aws cloudformation deploy \
--stack-name "sonar" \
--region $REGION_NAME \
--capabilities CAPABILITY_NAMED_IAM \
--template-file sonar.yaml
#############################
echo "Getting IP address of SonarQube server."
#############################
HOST_SONAR=$(aws cloudformation list-exports \
--query "Exports[?Name==\`sonar:PublicIp\`].Value" \
--output text)
if [ -z ${HOST_SONAR} ]; then
echo "ERROR: Missing CloudFormat export: sonar:PublicIp";
exit
fi
#############################
echo "Waiting for SonarQube to start."
#############################
# I tried using the healthcheck url but it started to return the
# empty string even when I knew the server was running. I switched
# to the system status.
STATUS=$(curl --silent -u admin:admin --connect-timeout 2 --max-time 2 http://${HOST_SONAR}:9000/api/system/status | jq -r '.status')
while [ "${STATUS}x" == "x" ]; do
echo -n "."
sleep 10
STATUS=$(curl --silent -u admin:admin --connect-timeout 2 --max-time 2 http://${HOST_SONAR}:9000/api/system/status | jq -r '.status')
done
echo ""
#############################
echo "Waiting for SonarQube to finish starting, after 10 second delay."
#############################
STATUS=$(curl --silent -u admin:admin --connect-timeout 2 --max-time 2 http://${HOST_SONAR}:9000/api/system/status | jq -r '.status')
while [ "${STATUS}x" == "STARTINGx" ]; do
echo -n "."
sleep 10
STATUS=$(curl --silent -u admin:admin --connect-timeout 2 --max-time 2 http://${HOST_SONAR}:9000/api/system/status | jq -r '.status')
done
echo ""
#############################
echo "Verifying that SonarQube is UP, after 2 second delay."
#############################
sleep 2
STATUS=$(curl --silent -u admin:admin --connect-timeout 2 --max-time 2 http://${HOST_SONAR}:9000/api/system/status | jq -r '.status')
echo "s3: $STATUS"
if [ "$STATUS" != "UP" ]; then
echo "ERROR: SonarQube is not UP. System status was ${STATUS}";
exit
fi
#############################
echo "Getting new password from Parameter Store."
#############################
RPASSWORD=$(aws ssm get-parameter \
--name sonar-password \
--query 'Parameter.Value' \
--output text 2>/dev/null)
if [ -z ${RPASSWORD} ]; then
echo "ERROR: Missing value Parameter Store: sonar-password";
exit
fi
#############################
echo "Changing Sonar default password."
#############################
curl -X POST \
-u admin:admin \
-d "login=admin&password=${RPASSWORD}&previousPassword=admin" \
http://${HOST_SONAR}:9000/api/users/change_password
#############################
echo "Getting Sonar token."
#############################
TOKEN=$(curl \
--silent \
-u admin:${RPASSWORD} \
-d "name=sonar" \
http://${HOST_SONAR}:9000/api/user_tokens/generate \
| jq -r '.token')
#############################
echo "Saving Sonar token to Parameter Store."
#############################
aws ssm put-parameter \
--name sonar-token \
--value $TOKEN \
--type String \
--overwrite > /dev/null