Custom Build Service in Linux

Introduction

In today’s microblog, I am going to focus on creating a custom build service in Linux. The idea here is NOT to re-invent the wheel, but rather to: learn how build services work and document my learning(s) publicly.

First of all, I started by writing a shell script to pull the latest source code from Github, build the artifacts and then deploy the artifacts to Apache2 server. Now, I would like to explain the script line-by-line and then I will provide the complete script at the end of this section.

Automating pull, build and deploy phases

I assume you are familiar with Shebang. It is a directive to indicate that we want our script to be treated as a shell script.

#!/bin/bash

I will be printing a lot of texts in the script and hence I want to prefix output with current date and time for better readability. I have accomplished this by implementing the following function:

log_message() {
  echo "$(date +'%Y-%m-%d %H:%M:%S') - $1"
}

You can learn about functions in shell scripts here. For this example, I have defined a function named log_message which takes a string and prints it to stdout after prepending current date and time.

Now comes the interesting part: finding the directory where my Github repository is located and execute git pull. However, I do not want to hardcode the repository location. So, I have created an environment variable in /etc/environment file. You can do so by executing the following command:

echo "PROJECT_DIR=/path/to/repository" >> /etc/environment

Now, I change the directory to project directory, pull the source code from Github and store the output in a variable.

# pull
cd $PROJECT_DIR
PULL_OUTPUT=$(git pull)

It is a good practice to only build artifacts if there is a change in the source artifacts. For this example, it means only continue if I have a new commit(s) pushed to Github repository. I achieve this by:

# Check if changes were pulled
if [[ "$PULL_OUTPUT" == *"Already up to date."* ]]; then
  SHOULD_BUILD_AND_DEPLOY=false
else
  SHOULD_BUILD_AND_DEPLOY=true
fi

Now that I have a variable SHOULD_BUILD_AND_DEPLOY which is only true if there are changes, I can use it to conditionally execute build and deploy phases.

Building my website means invoking the hugo CLI like:

# build
if [ $SHOULD_BUILD_AND_DEPLOY = true ] ; then
  log_message "Building..."
  hugo --config hugo.toml
fi

In the last step, I need to copy the public directory and all of its contents to ~/var/www/html. I have done it in the following way:

# deploy
if [ $SHOULD_BUILD_AND_DEPLOY = true ] ; then
  log_message "Deploying..."
  rm -rf /var/www/html/*
  cp -r public/* /var/www/html/
fi

Complete Script

#!/bin/bash

# Function to log messages with timestamp
log_message() {
  echo "$(date +'%Y-%m-%d %H:%M:%S') - $1"
}

# pull
cd $PROJECT_DIR
PULL_OUTPUT=$(git pull)

# Check if changes were pulled
if [[ "$PULL_OUTPUT" == *"Already up to date."* ]]; then
  SHOULD_BUILD_AND_DEPLOY=false
else
  SHOULD_BUILD_AND_DEPLOY=true
fi

if [ $SHOULD_BUILD_AND_DEPLOY = true ] ; then
  log_message "Building and deploying..."
else
  log_message "No changes to deploy."
fi

# build
if [ $SHOULD_BUILD_AND_DEPLOY = true ] ; then
  log_message "Building..."
  hugo --config hugo.toml
fi

# deploy
if [ $SHOULD_BUILD_AND_DEPLOY = true ] ; then
  log_message "Deploying..."
  rm -rf /var/www/html/*
  cp -r public/* /var/www/html/
fi

Invoking the script

Invoking the script is the trickiest part for a reason: do we use polling or do we use webhooks to invoke the build process? Checkout this article.

Anyway, for this particular example I decided to implement a cronjob which will run every minute and execute my build script. You can configure the cronjob by executing the command crontab -e and then adding the following line to crontab file.

* * * * * /path/to/script >> /path/to/output/file

Please note that I am redirecting the output to a file. If I had not done that, cronjob will send an email containing the output to admin email address.

That is all! Now as soon as I push changes to my Github repository, my website will be updated!

P.S: If you see this blog post, that means the steps documented here are working fine for me :)

Conclusion

I’ve successfully crafted a custom build service in Linux, offering detailed insights and a comprehensive script for reference. By automating crucial phases like pulling, building, and deploying, I ensure my website updates automatically whenever changes are pushed to the GitHub repository. Leveraging environment variables, conditional execution, and cronjobs, I’ve optimized the efficiency and adaptability of the build process. This post stands as a practical guide for those interested in implementing similar build automation in their Linux setup.