Custom Build Service in Linux
Introduction
In today’s microblog, I am going to focus on creating a custom build service in Linux. The idea here is NOT to re-invent the wheel, but rather to: learn how build services work and document my learning(s) publicly.
First of all, I started by writing a shell script to pull the latest source code from Github, build the artifacts and then deploy the artifacts to Apache2 server. Now, I would like to explain the script line-by-line and then I will provide the complete script at the end of this section.
Automating pull, build and deploy phases
I assume you are familiar with Shebang. It is a directive to indicate that we want our script to be treated as a shell script.
#!/bin/bash
I will be printing a lot of texts in the script and hence I want to prefix output with current date and time for better readability. I have accomplished this by implementing the following function:
log_message() {
echo "$(date +'%Y-%m-%d %H:%M:%S') - $1"
}
You can learn about functions in shell scripts here. For this example, I have defined a function named log_message
which takes a string and prints it to stdout
after prepending current date and time.
Now comes the interesting part: finding the directory where my Github repository is located and execute git pull
. However, I do not want to hardcode the repository location. So, I have created an environment variable in /etc/environment
file. You can do so by executing the following command:
echo "PROJECT_DIR=/path/to/repository" >> /etc/environment
Now, I change the directory to project directory, pull the source code from Github and store the output in a variable.
# pull
cd $PROJECT_DIR
PULL_OUTPUT=$(git pull)
It is a good practice to only build artifacts if there is a change in the source artifacts. For this example, it means only continue if I have a new commit(s) pushed to Github repository. I achieve this by:
# Check if changes were pulled
if [[ "$PULL_OUTPUT" == *"Already up to date."* ]]; then
SHOULD_BUILD_AND_DEPLOY=false
else
SHOULD_BUILD_AND_DEPLOY=true
fi
Now that I have a variable SHOULD_BUILD_AND_DEPLOY
which is only true if there are changes, I can use it to conditionally execute build and deploy phases.
Building my website means invoking the hugo CLI like:
# build
if [ $SHOULD_BUILD_AND_DEPLOY = true ] ; then
log_message "Building..."
hugo --config hugo.toml
fi
In the last step, I need to copy the public
directory and all of its contents to ~/var/www/html
. I have done it in the following way:
# deploy
if [ $SHOULD_BUILD_AND_DEPLOY = true ] ; then
log_message "Deploying..."
rm -rf /var/www/html/*
cp -r public/* /var/www/html/
fi
Complete Script
#!/bin/bash
# Function to log messages with timestamp
log_message() {
echo "$(date +'%Y-%m-%d %H:%M:%S') - $1"
}
# pull
cd $PROJECT_DIR
PULL_OUTPUT=$(git pull)
# Check if changes were pulled
if [[ "$PULL_OUTPUT" == *"Already up to date."* ]]; then
SHOULD_BUILD_AND_DEPLOY=false
else
SHOULD_BUILD_AND_DEPLOY=true
fi
if [ $SHOULD_BUILD_AND_DEPLOY = true ] ; then
log_message "Building and deploying..."
else
log_message "No changes to deploy."
fi
# build
if [ $SHOULD_BUILD_AND_DEPLOY = true ] ; then
log_message "Building..."
hugo --config hugo.toml
fi
# deploy
if [ $SHOULD_BUILD_AND_DEPLOY = true ] ; then
log_message "Deploying..."
rm -rf /var/www/html/*
cp -r public/* /var/www/html/
fi
Invoking the script
Invoking the script is the trickiest part for a reason: do we use polling or do we use webhooks to invoke the build process? Checkout this article.
Anyway, for this particular example I decided to implement a cronjob which will run every minute and execute my build script. You can configure the cronjob by executing the command crontab -e
and then adding the following line to crontab file.
* * * * * /path/to/script >> /path/to/output/file
Please note that I am redirecting the output to a file. If I had not done that, cronjob will send an email containing the output to admin email address.
That is all! Now as soon as I push changes to my Github repository, my website will be updated!
P.S: If you see this blog post, that means the steps documented here are working fine for me :)
Conclusion
I’ve successfully crafted a custom build service in Linux, offering detailed insights and a comprehensive script for reference. By automating crucial phases like pulling, building, and deploying, I ensure my website updates automatically whenever changes are pushed to the GitHub repository. Leveraging environment variables, conditional execution, and cronjobs, I’ve optimized the efficiency and adaptability of the build process. This post stands as a practical guide for those interested in implementing similar build automation in their Linux setup.