Real World Scripting

So that was a lot to learn! Now, what can we do with it? We haven't really gone through the useful stuff we can do with scripting, so this section will focus more on some cool things we can do with scripting.

To do this, we'll need to learn commands

cURL/wget

curl/wget are two commands that basically do the same thing with slight variations: they both allow us to send and receive data through common networking protocols. For our usecase, the most important thing is that it allows us to talk to an Application Programming Interface (API). Without going into too much detail, APIs are just a magical gateway for us to talk to programs other people have made.

Parsing large config (json) files

JSON is a common format for receiving and transporting data you will eventually have to work with, and it can sometimes be a pain to work with. Luckily, we have a nice program that can help us:

jqjqlang.github.io

WSL/Linux

sudo apt-get install jq

MacOS

brew install jq

Getting the latest bus stop timings

Here's a really nice API:

GitHub - cheeaun/arrivelah: Fast simple API for bus arrival times in Singapore.GitHub

curl https://arrivelah2.busrouter.sg/?id=18331

It gives us the bus arrival timings at public bus stops! Let's say I want to write a script that gives me the time till the next bus, let's say bus 95:

#!/bin/bash

# Replace this with your actual curl command that fetches the JSON response
response=$(curl -s "https://arrivelah2.busrouter.sg/?id=18331")

# Extract the 'time' field of the next bus using jq
next_bus_time=$(echo "$response" | jq -r '.services[0].next.time')

# Convert current time and the bus time to epoch for comparison
# TODO: Implement this (hint: figure out how to use date +%s)
current_time=
bus_arrival_time=

# Calculate time difference in seconds
time_diff=

# Convert time difference to minutes and seconds
minutes=$((time_diff / 60))
seconds=$((time_diff % 60))

# Display the result
if [ "$time_diff" -gt 0 ]; then
    echo "Bus number 95 will arrive in $minutes minutes and $seconds seconds."
else
    echo "Bus number 95 has already arrived or will arrive shortly."
fi

Solution

#!/bin/bash

# Replace this with your actual curl command that fetches the JSON response
response=$(curl -s "https://arrivelah2.busrouter.sg/?id=18331")

# Extract the 'time' field of the next bus using jq
next_bus_time=$(echo "$response" | jq -r '.services[0].next.time')

# Convert current time and the bus time to epoch for comparison
current_time=$(date +%s)
bus_arrival_time=$(date -d "$next_bus_time" +%s)

# Calculate time difference in seconds
time_diff=$((bus_arrival_time - current_time))

# Convert time difference to minutes and seconds
minutes=$((time_diff / 60))
seconds=$((time_diff % 60))

# Display the result
if [ "$time_diff" -gt 0 ]; then
    echo "Bus number 95 will arrive in $minutes minutes and $seconds seconds."
else
    echo "Bus number 95 has already arrived or will arrive shortly."
fi

Cool, but we don't really want to have to find the directory where our script is every time we want to run it! We can put it in special folders that allow us to run it from anywhere. To find this directory, we can do:

echo $PATH

This checks the environment variable PATH, which what the shell looks at when looking for your normal commands like ls and cd. We can then add our program to the PATH locations, most likely something like /usr/local/bin.

cp bus.sh /usr/local/bin/bus

Now we can just run the bus command from anywhere like a normal shell command!

Getting updates on the weather

data.gov.sgdata.gov.sg

Let's do something else that way more complex! We have an API that gives us the 24 hour weather forecast in Singapore. We want to:

• Check this forecast every morning

• If it's about to rain, send us an alert. For simplicity, lets send a telegram message!

Getting the weather forecast

curl https://api.data.gov.sg/v1/environment/24-hour-weather-forecast | jq

Here's the rough script we want to send:

#!/bin/bash

# Replace these with your actual Telegram bot token and chat ID
TELEGRAM_BOT_TOKEN="YOUR_BOT_TOKEN"
CHAT_ID="YOUR_CHAT_ID"
TELEGRAM_API_URL="https://api.telegram.org/bot$TELEGRAM_BOT_TOKEN/sendMessage"

# Replace this with your actual curl command that fetches the JSON weather data
response=$(curl -s "https://api.data.gov.sg/v1/environment/24-hour-weather-forecast
")

# Define an array of rain-related keywords
rain_keywords=("Rain" "Showers" "Thundery Showers" "Heavy Thundery Showers")

# Extract the general weather forecast using jq
general_forecast=$(echo "$response" | jq -r '.items[0].general.forecast')

# Initialize a flag to check if rain is found
rain_found=0

# Check for rain-related keywords in the general forecast
# TODO: Try implementing this! Your code should follow this general idea:
# for each keyword in rain_keywords check if the general forecast matches it. If
# so, set rain_found to 1

# If rain is detected, send an alert
if [ "$rain_found" -eq 1 ]; then
    message="Weather Alert: Rain expected! General forecast is '$general_forecast'. Stay prepared!"
    
    # Send the message to the Telegram bot
    curl -s -X POST $TELEGRAM_API_URL \
        -d chat_id=$CHAT_ID \
        -d text="$message" > /dev/null

    echo "Alert sent: $message"
else
    echo "No rain expected. General forecast is '$general_forecast'."
fi

• To get a telegram bot id, just go to @BotFather, and create a new bot, enter the token we receive inside

• To get your chat id, just go to @getmyid_bot and copy your chat id there

• You'll need to start a chat with your bot first, go to your bot and do '/start' before you try running the script

Solution

#!/bin/bash

# Replace these with your actual Telegram bot token and chat ID
TELEGRAM_BOT_TOKEN="YOUR_BOT_TOKEN"
CHAT_ID="YOUR_CHAT_ID"
TELEGRAM_API_URL="https://api.telegram.org/bot$TELEGRAM_BOT_TOKEN/sendMessage"

# Replace this with your actual curl command that fetches the JSON weather data
response=$(curl -s "https://api.data.gov.sg/v1/environment/24-hour-weather-forecast
")

# Define an array of rain-related keywords
rain_keywords=("Rain" "Showers" "Thundery Showers" "Heavy Thundery Showers")

# Extract the general weather forecast using jq
general_forecast=$(echo "$response" | jq -r '.items[0].general.forecast')

# Initialize a flag to check if rain is found
rain_found=0

# Check for rain-related keywords in the general forecast
for keyword in "${rain_keywords[@]}"; do
    if [[ "$general_forecast" == *"$keyword"* ]]; then
        rain_found=1
        break
    fi
done

# If rain is detected, send an alert
if [ "$rain_found" -eq 1 ]; then
    message="Weather Alert: Rain expected! General forecast is '$general_forecast'. Stay prepared!"
    
    # Send the message to the Telegram bot
    curl -s -X POST $TELEGRAM_API_URL \
        -d chat_id=$CHAT_ID \
        -d text="$message" > /dev/null

    echo "Alert sent: $message"
else
    echo "No rain expected. General forecast is '$general_forecast'."
fi

Great! We have a working weather alert! But we don't really want to have to manually run this every morning, it would defeat the purpose of the script! Assuming your laptop runs 24/7, we can make this script run on a schedule using cronjobs!

Obviously, your laptop running 24/7 is unrealistic. However, this method of scheduling scripts to be run is similar to how you would run really important scripts on a schedule: you put a cronjob on an always available server, whether it's in the cloud or somewher running in your basement

Scheduling jobs with cron

• Cron is a scheduling daemon that executes tasks at specified intervals.

• These tasks are called cron jobs and are mostly used to automate system maintenance or administration.

Essentially, we can schedule jobs on our machine to run at a specific time by putting in some magic string + the command we want to run. So what is this magic string?

Without going into too much detail about how cronjobs work, we can grab our magic string from the site below

Cron Expression Examples - Crontab.gurucrontab.guru

and pick out the crontab we want, in this case:

Crontab.guru - The cron schedule expression generatorcrontab.guru

So our expression should follow the format:

<magic string> <command>

In our case:

0 9 * * * /path/to/script/weather.sh

It is important we use absolute paths here.

And that's it! As long as our laptop or machine is running at 9am in the morning, this script should run automagically!