# NUS Hackers Wiki

Welcome to our wiki!

This is a collection of resources from past workshops that we have conducted, such as those from our [Hackers Toolbox](https://www.nushackers.org/hackerstoolbox) series. Feel free to use the information here for your own learning, experimentation and hacks.

[Reach out](mailto:active@nushackers.org) to us if you want to conduct your own workshop!


# Virtual Machines and Linux

## Slides

{% file src="/files/4WgramVm81dPItamWbQg" %}

## Introduction

### Required Software

* Download and install VirtualBox
* Download Ubuntu 24.04 ISO file

## Linux & Virtual Machines

### Brief Introduction to Linux & Unix

Linux is a Unix-like operating system kernel, known for being the most popular kernel in the world. It's widely used in various devices and systems, including Android smartphones, Chromebooks, most routers, servers, and even supercomputers.

**Unix-like systems,** including Linux, are at the heart of the most popular operating system family in the world. Their architecture and principles have influenced countless other systems, making them a staple in the world of computing.

If you're a computing student, sooner or later, you'll find yourself developing for a Unix-like platform!

### Virtual Machine: What? Why?

A virtual machine (VM) is a **simulated computer** that allows you to run an operating system and applications in a completely isolated environment. You can configure a guest virtual machine with any operating system and settings you want, and use it without affecting your host environment.

#### Why Use a Virtual Machine?

Virtual machines are incredibly useful for several reasons:

* **Experimentation**: They allow you to test out different operating systems, software, and configurations without the risk of damaging your main system.
* **Software Compatibility**: You can run software that is only compatible with a specific operating system by creating a VM with that OS.
* **Safe Testing**: If you need to experiment with potentially malicious software, a VM provides a safe, isolated environment to do so.

#### Useful Features of VMs

* **Isolation**: A VM isolates the guest environment from the host, which means you can run buggy or untrusted software with a reasonable level of safety.
* **Snapshots**: VMs can take snapshots, which capture the entire machine's state at a particular point in time. This allows you to make changes, test configurations, or install software and then easily revert to a previous state if something goes wrong.

#### Disadvantages of Virtual Machines

* **Performance**: VMs are generally slower compared to running an operating system directly on your hardware (bare metal).
* **Resource Competition**: VMs share the host system's resources, such as CPU, memory, and storage, which can impact performance.
* **Unsuitability for Certain Applications**: VMs might not be ideal for resource-intensive applications like games or high-performance computing tasks.

### Virtual Machine: Setting up

#### Why Choose VirtualBox?

We're using VirtualBox because it offers several advantages:

* **Free and Open-Source Software (FOSS)**: VirtualBox is completely free and open-source, making it accessible to everyone.
* **Graphical User Interface (GUI)**: It comes with a user-friendly graphical interface, making it easier to use, especially for beginners.
* **Cross-Platform**: VirtualBox works on multiple operating systems, including Windows and Linux. There is a developer preview for M1+ Macs, but the performance isn't great.

## Setting up your first virtual machine

**1. VirtualBox Main UI**

* Once you open VirtualBox, click on the "Add" button to start creating a new virtual machine.

<figure><img src="/files/fUeCl4PlKA4v1oGJNt6K" alt=""><figcaption></figcaption></figure>

**2. Creating a New VM**

* **Name**: Enter "Ubuntu" as the name of your new virtual machine. VirtualBox should automatically detect the type (Linux) and version (Ubuntu) based on the name.

<figure><img src="/files/XgP0BXpcXEMPTFan6uUB" alt=""><figcaption></figcaption></figure>

**3. Set the Amount of Memory (RAM)**

* **Memory Allocation**: Ubuntu requires a minimum of 512 MiB of RAM and recommends 2 GiB. However, as a general rule, do not allocate more than 1/4 of your physical RAM to the virtual machine to ensure that your host system runs smoothly.

<figure><img src="/files/j9A9E6hIEnMEcyQYg4S2" alt=""><figcaption></figcaption></figure>

**4. Create a Virtual Hard Disk**

* **Step 1**: Your virtual machine needs a virtual hard disk. Click on "Create" to begin setting it up.
*

```
<figure><img src="/files/BRl2ck3kqklCUXAz6wc8" alt=""><figcaption></figcaption></figure>
```

* **Step 2**: Use the default virtual hard disk format for the best performance.

<figure><img src="/files/r4hgLRv0MF1O3XSSrEMF" alt=""><figcaption></figcaption></figure>

* **Step 3**: Opt for a dynamically allocated disk. This means the virtual hard disk will only use as much space as it currently needs, rather than reserving a large chunk of your storage upfront.

<figure><img src="/files/V0evb6BBJmtXryAirVXL" alt=""><figcaption></figcaption></figure>

* **Step 4**: Ubuntu recommends a minimum of 10 GiB of storage and 25 GiB for a full installation. For this guide, we'll be using the minimum installation, which will require about 6 GiB.

<figure><img src="/files/kuaddcHPRZpgpT1Qgg0X" alt=""><figcaption></figcaption></figure>

#### Back to the Main UI

* After setting up the virtual hard disk, return to the main VirtualBox interface and click on "Settings".

<figure><img src="/files/1Am7rNTXV1xlSsMsFNAJ" alt=""><figcaption></figcaption></figure>

#### Configuring Settings

* **Storage Settings**: Navigate to "Storage", then select the "Empty" slot under "Controller: IDE". Click on the disc icon beside "IDE Secondary Master", and then choose "Choose Virtual Optical Disk File".

<figure><img src="/files/JepXafnU8TbYQr26HV3z" alt=""><figcaption></figcaption></figure>

#### Choose Your Ubuntu ISO File

* Select the Ubuntu ISO file that you have downloaded earlier.

<figure><img src="/files/OD1wrn7jalQ7asNOL2jc" alt=""><figcaption></figcaption></figure>

#### Final Steps

You're all set up with VirtualBox! You can further customize your settings later if needed. For now, return to the main UI and click "Start" to launch your new virtual machine.

#### For power users&#x20;

> Skip this part if you're not familiar with working in a command line environment

We can easily do exactly what we just did in a matter of seconds, using the command line interface (cli).

```
VBoxManage createvm --name "Ubuntu" --ostype Ubuntu24_LTS_64 --register 
VBoxManage modifyvm "Ubuntu" --memory 2048 --acpi on --boot1 dvd --nic1 nat
VBoxManage createhd --filename "Ubuntu.vdi" --size 40960
VBoxManage storagectl "Ubuntu" --name "IDE Controller" --add ide --controller PIIX4
VBoxManage storageattach "Ubuntu" --storagectl "IDE Controller"  --port 0 --device 0 --type hdd --medium "Ubuntu.vdi"
VBoxManage storageattach "Ubuntu" --storagectl "IDE Controller" --port 0 --device 1 --type dvddrive --medium /full/path/to/iso.iso
VBoxManage modifyvm "Ubuntu" --cpus 4
VBoxManage modifyvm "Ubuntu" --vrde on
VBoxHeadless --startvm "Ubuntu"
```

## Installing Ubuntu

#### Booting Ubuntu

Once your virtual machine starts and boots up, you should see a screen like this:

<figure><img src="/files/BREf2qDWRwTWdKriZkt4" alt=""><figcaption></figcaption></figure>

* **Step 1**: Choose "Install Ubuntu" to begin the installation process.

#### Choose Keyboard Layout

<figure><img src="/files/cCHAkQHWojMTPDJU7UtV" alt=""><figcaption></figcaption></figure>

* **Step 2**: Select your keyboard layout. If you're using a computer purchased in Singapore, the default layout should be "English (US)."

#### Updates and Other Software

<figure><img src="/files/2rcPhaP4Bun820DRu8sG" alt=""><figcaption></figcaption></figure>

* **Step 3**: Opt for a "Minimal Installation" to save time during the installation process. You can leave the checkboxes unticked for a quicker setup.

#### Installation Type

<figure><img src="/files/oHgme0GKFNZSlSuHHEjx" alt=""><figcaption></figcaption></figure>

* **Step 4**: Choose "Erase disk and install Ubuntu" to proceed with the installation. When the dialogue box appears, click "Continue" to confirm.

#### Location

<figure><img src="/files/yW2uaGy9PJLGzYlkANs6" alt=""><figcaption></figcaption></figure>

* **Step 5**: Ubuntu should automatically detect your location as Singapore. If it doesn't, you can manually select your location.

#### Setting Up Your Username

<figure><img src="/files/wQdStXp0gdLvwtBfiqnT" alt=""><figcaption></figcaption></figure>

* **Step 6**: Enter your name and create a password for your Ubuntu installation.

#### Sit Back and Relax

<figure><img src="/files/YelURuVAo0mAX7zvrc0i" alt=""><figcaption></figcaption></figure>

* **Step 7**: Now, just sit back and relax while Ubuntu installs. This may take some time, so feel free to take a break while the installation completes.

## Guest Additions

Many Linux distributions offer Guest Additions through their package repositories, making the installation process straightforward. To install them:

1. Open a terminal in your Ubuntu virtual machine.
2. Run the following commands:

   ```bash
   sudo -i
   apt update
   apt install virtualbox-guest-utils virtualbox-guest-x11
   ```

This will install the necessary utilities and drivers to enable Guest Additions features.

#### Installing Guest Additions Using the ISO

If you're using an operating system like Windows or another OS that doesn't have Guest Additions available in its package repository, you can install them using a CD image:

1. In VirtualBox, go to the "Devices" menu.
2. Select "Insert Guest Additions CD image...".
3. This will mount the Guest Additions as a virtual CD in your VM.
4. Follow the on-screen instructions to install the software.

### Shared Folders

One cool feature of Guest Additions is shared folders. This provides a useful interface for sharing files between the host machine and virtual machine.&#x20;

* Click on Settings > Shared Folders

<figure><img src="/files/l8f71G7vm9datqiliHSy" alt=""><figcaption></figcaption></figure>

* Click on the folder icon on the right

<figure><img src="/files/HtBIcpCCwYJBVUE6P03b" alt=""><figcaption></figcaption></figure>

* The folder path should point to the location on the host machine
* Give the folder a folder name, let's say "SharedFolder"

Now in your virtual machine, run the following command

```
mkdir ~/Shared
sudo mount -t vboxsf SharedFolder ~/Shared/ -o uid=1000,gid=1000
```

Now if you drag a folder into your shared folder in your host, you should see it appear in your virtual machine as well!

## Nifty Tricks with VM

#### Force Shutdown the VM

Sometimes, you might need to run unstable software on your virtual machine, which could cause the VM to hang or become unresponsive. If this happens, you can force a shutdown:

* **Step 1**: Close the VirtualBox window for your VM.
* **Step 2**: When prompted, choose "Power off the machine." This will force the VM to shut down immediately.
* **Step 3**: You can quickly bring the VM back up by starting it again from the main VirtualBox interface.

#### Saving Machine State

You don't always need to completely shut down the operating system inside your VM when you're done working:

* **Step 1**: To pause and save the VM's current state, simply close the VM's window.
* **Step 2**: Choose "Save the machine state" from the options. This will pause the VM, allowing you to resume exactly where you left off the next time you start it.

#### What is a Snapshot?

A snapshot is like a time machine for your VM. It captures the state of your virtual machine at a specific point in time. You can return to this state later if needed, making snapshots very useful for experimentation or testing.

#### Taking a Snapshot in VirtualBox

To take a snapshot:

* **Step 1**: In the VirtualBox main interface, click on the list icon beside your VM name (e.g., "Ubuntu").
* **Step 2**: Select "Snapshots" to view, take, or manage snapshots.

#### Restoring a Snapshot in VirtualBox

If you want to restore your VM to a previous snapshot:

* **Step 1**: Ensure your VM is shut down. If necessary, either shut down the OS within the VM or close the VM and select "Power off the machine."
* **Step 2**: In the list of snapshots, select the one you want to restore.
* **Step 3**: Click "Restore" to revert your VM to the selected snapshot.

## Use Cases

* Running risky programs or commands
* SSH with VSCode
* Experimenting with low level programs that might break your computer


# Beginners' Guide to the Terminal

> This workshop is renamed and heavily lifted from the original workshop: Hacker Tools - Shell and Scripting, which itself is lifted from the [MIT's Missing Semester](https://missing.csail.mit.edu/2020/shell-tools/)

## Introduction

There are tons of ways for us to interface with modern computers, from beautiful GUIs and web applications. But that can only really get us so far. To fully utilize your computer to it's maximum potential and efficiency, learning how to use your a textual interface like your terminal is necessary.

Many applications and use cases are terminal first, as it is extremely easy to develop for and work with. No fancy framework or dependencies needed, just good ol' text input and outputs. In this specific case, we'll be going through Bash, a unix-like shell (or a POSIX-compatible shell).

### What is Unix?

Without going into too much detail about the history of [Unix](https://en.wikipedia.org/wiki/Unix), we just need to understand that unix was a really great idea for how an operating system should be like, and it is what some modern operating systems today are born from. The important thing for us to know is that it gave us the rise of the **Unix Philosophy**, which guides a lot of how a terminal should work and function.

### The Unix Philosophy

The Unix Philosophy can be summarized as such:

* Write programs that do one thing and do it well.
* Write programs to work together.
* Write programs to handle text streams, because that is a universal interface.

and these principals, still guide a lot of applications today!


# Introduction to the Terminal

## Introduction to the terminal/shell

> Nearly all platforms you can get your hands on have a shell in one form or another, and many of them have several shells for you to choose from. While they may vary in the details, at their core they are all roughly the same: they allow you to run programs, give them input, and inspect their output in a semi-structured way.
>
> \-- Excerpt from the Missing Semester

For the most part, when people talk about the terminal, we normally talk about Bash, or POSIX-compliant shells. POSIX is just a fancy name for a set of rules that a shell should abide by, so that different shells can have similar behaviour. Other shells include:

* fish
* zsh
* nushell
* powershell

In modern operating systems, to open a shell prompt, you often need a terminal. Think of it as a nice GUI wrapping the textual interface (the shell). Your device should probably be shipped with one, or you should be able to install one easily.

### The Shell Prompt

When you first launch a terminal, you will see a *prompt, similar to or a slight variation of*:

```
chun@legion:~$
```

This prompt tells you for example, that your username is `chun` on the machine named `legion`, and that you 'working directory' is `~` (short for home, we'll get to that). You should also see a blinking cursor, which you can type anything to, and when you hit `Enter`, it should execute the command you've typed out.

Here are some really simple commands you should be able to run (but some may not be installed depending on your machine):

* `date` - shows the date

```shell-session
Sun Sep  8 08:23:13 PM +08 2024`
```

* `cal` - a tiny calendar

```shell-session
   September 2024   
Su Mo Tu We Th Fr Sa
 1  2  3  4  5  6  7
 8  9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30               
              
```

* `uptime` - shows how long your computer has been powered on

```shell-session
 20:26:04  up   1:01,  2 users,  load average: 0.45, 0.62, 0.62
```

* `echo` - echoes what you typed

```shell-session
chun@legion:~$ echo hello
hello
```

### Folders and directories

These commands are neat, but we can't really do much in the command line until we understand the concept of folders and directories.&#x20;

All your files and directories on your system are stored in a structure known as a tree. This tree starts from a 'root' directory, this will be `/` on Linux and MacOS, and something like `C:\` on Windows.&#x20;

A **path** on the shell is just a list of directories, seperated by `/` on Linux and macOS and `\` on Windows. For example:

```
C:\User\user\Documents - For Windows
~/Downloads - For Linux or MacOS
```

There are two types of paths:

* The absolute path is a path that starts from the root directory
* Relative paths are relative to your *current working directory,* or where you shell currently is

To see where you current directory is, use **print working directory,** or `pwd` for short in the terminal.

In a path, `.` refers to the current directory, and `..` refers to the parent directory.

### Common Commands

* to get the **man**ual pages of a command
* to **c**hange **d**irectory
* to **l**i**s**t files and directories
* to **r**e**m**ove files and directories
* to **c**o**p**y file
* to **m**o**v**e file
* to **p**rint **w**orking **d**irectory

### Navigating the shell

`bash` has shortcuts that are based on `emacs` keybindings:

* `Ctrl` + `a` - beginning of line
* `Ctrl` + `e` - end of line
* `Alt` + `b` - move back one word
* `Alt` + `f` - move forward one word
* `Ctrl` + `k` - delete from cursor to end of line
* `Ctrl` + `_` - undo

And some special keybindings:

* `Ctrl` + `u` - delete from cursor to the start of line
* `Ctrl` + `w` - delete from cursor to the start of word
* `Ctrl` + `c` - terminates the command
* `Ctrl` + `z` - suspends the command
* `Ctrl` + `l` - clears the screen
* `Ctrl` + `s` - stops the output to the screen
* `Ctrl` + `q` - allows output to the screen

You can find even more by doing `man readline`

{% embed url="<https://www.man7.org/linux/man-pages/man3/readline.3.html>" %}

### Editing in the terminal

There are a few ways you can make changes to your filesystem, be it editing files or directories/folders:

* `mkdir` to make a new empty directory
* `touch <filename>` to make a new empty file
* `nano <filename>` to open a editor to edit the file
  * `Ctrl` + `o` - to save
  * `Ctrl` + `x` - to exit

### Finding commands

What if we want to find our previously used commands? If you haven't already realized, you should be able to use the up arrow to scroll through your previous commands, but it's not very efficient. This is where we can use a command called `history`.

```shell-session
chun@legion:~$ history
```

### Composability

Remember what we said about the Unix Philosophy? A big part about it is the idea of programs working well together. The terminal allows this by allow the output of one program to be the input of another program. This is known as **piping.** To pipe we can do something like:

```bash
history | grep "echo"
```

* `history` prints out the entire command history as the output
* `grep` takes in an input and tries to filter for the keyword "echo"

Some other ways you could use pipes:

```bash
history | head ## Grabs the first 10 lines of the output
history | tail ## Grabs the last 10 lines of the output
## Opens output in a scrollable format, use you arrow keys to navigate
## and press q to quit
history | less 
```

There are more ways we can compose programs, which we'll go through in the scripting section!


# Modern Shell Tools

## Shell Tools

All these are quite nice if you're working with a bare shell, or an uncustomized shell. However, in this day and age, there's a lot of new commands and tools to enhance your terminal experiences.

{% embed url="<https://github.com/ibraheemdev/modern-unix>" %}

### Finding out how to use commands and installing them

Before we get started with tools, we need to know how to install them and also how to learn what they do. For commands, we can often pass in **flags** to tell the program how we want it to run. One of the universal flags is the `--help` flag.

<pre class="language-shell-session"><code class="lang-shell-session"><strong>chun@legion:~$ cat --help
</strong></code></pre>

To install a program, we often times use what is known as a **package manager.** This allows us to search and install packages without having to googling the tool and trying to find the correct downloadable file

If you're on Linux or WSL, you should have a package manager installed. If you're on Ubuntu/Debian-based distros, this should be `apt`. If you're on anything else, you should try and figure out what the package manager is based on your distro.

If you're on MacOS, you'll need to install brew:

{% embed url="<https://brew.sh>" %}

Now to install a program, you can just do:

{% tabs %}
{% tab title="WSL/Linux" %}
You will need to update the package lists before installing a package!

```shell-session
sudo apt-get update
sudo apt-get install <package-name>
```

{% endtab %}

{% tab title="MacOS" %}

```ruby
brew install <package-name>
```

{% endtab %}
{% endtabs %}

### Finding Files

{% embed url="<https://github.com/junegunn/fzf>" %}

{% tabs %}
{% tab title="WSL/Linux" %}

```bash
sudo apt-get install fzf
```

{% endtab %}

{% tab title="MacOS" %}

```shell-session
brew install fzf
```

{% endtab %}
{% endtabs %}

Once installed, do:

`fzf` stands for fuzzy finder. It allows you to find anything with a fuzzy search (you can make spelling errors). If you just run `fzf`, it will do a fuzzy find on your current directory

```bash
fzf
```

But we can do so much more than that! To do so, we need to add some keybindings by running this command:

```bash
eval "$(fzf --bash)"
```

Now try out these new keybindings:

* `CTRL-T` - Paste the selected files and directories onto the command-line
* `CTRL-R` - Paste the selected command from history onto the command-line
* `ALT-C` - cd into the selected directory

Now notice that if you quit the terminal or start a new terminal, these keybindings won't be available. To make the change permanent, we need to save it into a **config file.** For bash, this config file is is at `~/.bashrc`

```bash
nano ~/.bashrc

## Inside the editor, add this line
eval "$(fzf --bash)"
```

### Finding code or text

So we can find specific files and directories, but what about finding specific contents within a file? `ripgrep` is a program that aims to solve this.

{% embed url="<https://github.com/BurntSushi/ripgrep>" %}

{% tabs %}
{% tab title="WSL/Linux" %}

```bash
sudo apt-get install ripgrep
```

{% endtab %}

{% tab title="MacOS" %}

```bash
brew install ripgrep
```

{% endtab %}
{% endtabs %}

```bash
# Find all python files where I used the requests library
rg -t py 'import requests'
# Find all files (including hidden files) without a shebang line
rg -u --files-without-match "^#\!"
# Find all matches of foo and print the following 5 lines
rg foo -A 5
# Print statistics of matches (# of matched lines and files )
rg --stats PATTERN
```

### Fast directory navigation

It is quite troublesome so jump around directories, especially if you're copying something, or the path is really, really, really long. A good way around this is to have a program guess what directory you want to jump to based on keywords and frequency of which directory you jump to! That is exactly what zoxide does.

{% embed url="<https://github.com/ajeetdsouza/zoxide>" %}

{% tabs %}
{% tab title="WSL/Linux" %}

```bash
curl -sSfL https://raw.githubusercontent.com/ajeetdsouza/zoxide/main/install.sh | sh
```

{% endtab %}

{% tab title="MacOS" %}

```bash
brew install zoxide
```

{% endtab %}
{% endtabs %}

You can use `z` to jump to directories, similar to how you use `cd`. For example, if you have a directory you frequently go to, like:

```
 /home/user/downloads/temp_dir/funny_project
```

You could do something like `z funny` to jump into it.

{% hint style="info" %}
If all you want to do is learn how to use the terminal, you can just stop here! The next section goes above an beyond and talks about how you can build your own scripts and commands which you can then run.
{% endhint %}


# Shell Scripting

## Scripting

So we know how to run commands from an interactive prompt, but what if we want to save the commands we run so that we can reuse in the future? That's where scripting comes into play

## Basics of Scripting

You can write programs directly at the prompt, or write into a file (writing scripts)

```{.bash
#!/bin/sh
echo something
```

* Open an editor (for beginner, `nano` is recommended), save the script as `example-script`
* On your shell, run `chmod +x example-script`
* You can run your script as `./example-script`
* `#!/bin/sh` is also known as the **shebang**, specifies the interpreter
* `echo` is a command that prints its arguments to the standard output.&#x20;

### More on Flags

Most command line utilities take parameters using **flags**. They come in short form (`-h`) and long form (`–help`). Usually, running `COMMAND -h` or `man COMMAND` will give you a list of the flags the program takes.

* Short flags can be combined: `rm -r -f` is equivalent to `rm -rf` or `rm -fr`
* A double dash `–` is used in to signify the end of command options, after which only positional parameters are accepted.
  * For example, to create a file called `-v`, Use `touch -- -v` instead of `touch -v`
  * For example, to grep a file called `-v`, `grep pattern -- -v` will work while `grep pattern -v` will not.

#### Common Flags

There are a few flags that are widely accepted and have similar meanings throughout many programs

* `-a` commonly refers to all files (i.e. also including those that start with a period\[^4])
* `-f` usually refers to forcing something, e.g. `rm -f`
* `-h` displays the help for most commands
* `-v` usually enables a verbose output
* `-V` usually prints the version of the command

## Unix Directory Structure

The Unix Directory Structure Unix has a different directory structure from Windows.

There is no concept of drives.

Everything is files and directories. The root directory is `/`

We use forward slash `/` instead of backward slash `\`

Specifically for Linux, there is FHS

{% embed url="<https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard>" %}

### Important Unix Directories

* `/bin`, `/sbin`, `/usr/bin`, `/usr/local/bin`, `/opt` = executables
* On Linux: `/home` = user home directories
* On macOS: `/Users` = user home directories
* `/var/log` = log files
* `/tmp` = temporary files
* `/dev/urandom` = random number generator

## Shell Syntax

```bash
echo Hello
```

We've seen this command before, but we've never assigned it the proper terminology. Whenever we type something out, we can split the input into **COMMANDs** and **ARGs** (short for arguments)

* `COMMAND ARG1 ARG2 ARG3`

## Variables

<pre class="language-bash"><code class="lang-bash"><strong>echo location
</strong>name=COM3
echo $name
</code></pre>

* Used to store text
* `name=value` to set variable
* `$name` to access variable

:There are also a bunch of special variables we can use in our scripts:

* `$?`: get exit code of the previous command
* `$1` to `$9`: arguments to a script
* `$0`: name of the script itself
* `$#`: number of arguments
* `$$`: process ID of current shell&#x20;

### Environment Variables

On top of variables you can declare, there are a bunch of global variables that are declared in order for your system to run. We call these **Environment Variables.** You can see the full list of environment variables using the command:

```bash
env
```

### Quick Exercise

Create a script `variable-example` containing the code below, then try running it with various arguments.

```{.bash
#!/bin/sh
echo $0
echo $1
echo $2
echo $#
```

## Loops

Loop is used to run a command a bunch of times.

For example:

```bash
for i in $(seq 1 5); do echo hello; done
```

Let's unpack this!

```
`for x in list; do BODY; done`
```

`for x in list; do BODY; done`

* `;` terminates a command -- equivalent to newline
* Split `list`, assign each to `x`, and run `BODY`
* Split by "whitespace" -- we will get into it later
* Compared to C, no curly braces, instead `do` and `done`

So, knowing the above,

```bash
for i in $(seq 1 5); do echo hello; done
```

* `$(seq 1 5)`
  * Run the program `seq` with arguments `1` and `5`
  * Substitute the `$(...)` block with the output of the program
  * Equivalent to

    ```bash
    for i in 1 2 3 4 5; do echo hello; done
    ```
* `echo hello`
  * Everything in a shell script is a command
  * Here, it means run the `echo` command, with argument `hello`.
  * All commands are searched in `$PATH` (colon-separated)
  * Find out where a command is located by running `which COMMAND`, e.g. `which ls`&#x20;

## Conditionals

```bash
if test -d /bin; then echo true; else echo false; fi;
```

Let's unpack this!

```
if CONDITION; then BODY; fi
```

* `CONDITION` is a command.
* If its exit code is `0` (success), then `BODY` is run.
* Optionally, you can also hook in an `else` or `elif`

So, knowing the above,

```bash
if test -d /bin; then echo true; else echo false; fi;
```

* `test -d /bin`
  * `test` is a program that provides various checks and comparison which exits with exit code `0` if the condition is true.
* Alternate syntax: `[ condition ]`, e.g. `[ -d /bin ]`

Let's create a command that only prints directories

{% hint style="danger" %}
Bug! Hold on! What if the directory is called "`My Documents`"?
{% endhint %}

* `for f in $(ls)` expands to\
  `for f in My Documents`
* Will first perform the test on `My`, then on `Documents`

## &#x20;Argument Splitting

* Bash splits arguments by whitespace (tab, newline, space)
* Same problem somewhere else: `test -d $f`
* If `$f` contains whitespace, `test` will error!
* Need to use quote to handle spaces in arguments `for f in "My Documents"`
* How do we fix our script?
* What do you think `for f in "$(ls)"` does?

## Globbing&#x20;

`bash` knows how to look for files using patterns:

* Thus, `for f in *` means all files in this directory
* When globbing, each matching file becomes its own argument
* However, still need to make sure to quote, e.g.\
  `test -d "$f"`

You can make advanced patterns

* `for f in a*`: all files starting with `a` in the current directory
* `for f in foo/*.txt`: all `.txt` files in `foo`
* `for f in foo/*/p??.txt`: all three-letter text files, starting with p, in subdirectories of `foo`

## Whitespace issues

* `if [ $foo = "bar" ]; then`: What's the issue?
* What if `$foo` is empty? arguments to `[` are `=` and `bar`
* Possible workaround: `[ x$foo = "xbar" ]`, but very hacky
* Instead, use `[[ CONDITION ]]`: `bash` built-in comparator that has special parsing
* Good news: it also allows `&&` instead of `-a`, `||` instead of `-o`, etc.

## Shellcheck

* The mentioned problems are the most common bugs in shell scripts.
* A good tool to check for these kinds of possible bugs in your shell script: <https://www.shellcheck.net/>&#x20;

## Composability

* Shell is powerful, in part because of **Composability**
* You can chain multiple programs together, rather than one program that does everything
* Remember **The Unix Philosophy**:
  1. Write programs that do one thing and do it well.
  2. Write programs to work together.
  3. Write programs to handle text streams, because that is a universal interface.&#x20;

### More Pipes

`cat /var/log/sys*log | grep "Sep 10" | tail`

* `cat /var/log/sys*log` prints the system log
* This output is fed into `grep Sep 10`, which looks for all entries from today.
* This output is then further fed into `tail`, which prints only the last 10 lines.&#x20;

## Streams

* All programs launched have 3 streams:
  * `STDIN`: the program reads input from here
  * `STDOUT`: the program prints to here
  * `STDERR`: a second output that the program can choose to use.
* By default, `STDIN` is your keyboard, `STDOUT` and `STDERR` are both your terminal&#x20;

### Stream Redirection

* However, this can be changed!
* `a | b`: makes `STDOUT` of `a` the `STDIN` of `b`.
* `a > foo`: `STDOUT` of `a` goes to the file `foo`
* `a 2> foo`: `STDERR` of `a` goes to the file `foo`
* `a < foo`: `STDIN` of `a` is read from the file `foo`
* `a <<< some text`: `STDIN` of `a` is read from what comes after `<<<`
* You can also pipe to `tee` (look up in `man` what `tee` does)&#x20;

**So why is this useful?**

It lets you manipulate output of a program!

* `ls | grep foo`: all files that contain the word `foo`
* `ps | grep foo`: all processes that contain the word `foo`
* On Linux: `journalctl | grep -i intel | tail -n 5`: last 5 system log messages with the word `intel` (case-insensitive)
* Note that this forms the basis for **data-wrangling**, which will be covered later.&#x20;

### Grouping Commands `(a; b) | tac`

* Run `a`, then `b`, and send all their output to `tac`\[^7]
* For example: `(echo qwe; echo asd; echo zxc) | tac`

### Process Substitution `b <(a)`

* Run `a`, generate a temporary file name for its output stream, and pass that filename to `b`
* To demonstrate: `echo <(echo a) <(echo b)`
* On Linux: `diff <(journalctl -b -1 | head -n20) <(journalctl -b -2 | head -n20)`
* This shows the difference between the first 20 lines of the last boot log and the one before that.&#x20;

### Jobs

Used to run longer-term things in the background.

* Use the `&` suffix
  * It will give back your prompt immediately.
  * For example: `(for i in $(seq 1 100); do echo hi; sleep 1; done) &`
  * Note that the running program still has your terminal as `STDOUT`. Instead, can redirect `STDOUT` to file.
  * Handy especially to run 2 programs at the same time like a server and client: `server & client`
  * For example: `nc -l 1234 & nc localhost 1234 <<< test`
* `jobs`: see all jobs
* `fg %JOBS`: bring the job corresponding to the id to the foreground (with no argument, bring the latest job to foreground)
* You can also background the current program: `^Z`, then run `bg`
  * `^Z` stops the current process and makes it a job.
  * `bg` runs the last job in the background.
* `$!` is the PID of the last background process.&#x20;

### Some Exercises

* Sometimes piping doesn't quite work because the command being piped into does not expect the newline separated format.
* For example, `file` command tells you properties of the file.
* Try running `ls | file` and `ls | xargs file`
* What is `xargs` doing?&#x20;


# Real World Scripting

So that was a lot to learn! Now, what can we do with it? We haven't really gone through the useful stuff we can do with scripting, so this section will focus more on some cool things we can do with scripting.

To do this, we'll need to learn  commands

### cURL/wget

curl/wget are two commands that basically do the same thing with slight variations: they both allow us to send and receive data through common networking protocols. For our usecase, the most important thing is that it allows us to talk to an Application Programming Interface (API). Without going into too much detail, APIs are just a magical gateway for us to talk to programs other people have made.

### Parsing large config (json) files

JSON is a common format for receiving and transporting data you will eventually have to work with, and it can sometimes be a pain to work with. Luckily, we have a nice program that can help us:

{% embed url="<https://jqlang.github.io/jq/>" %}

{% tabs %}
{% tab title="WSL/Linux" %}

```bash
sudo apt-get install jq
```

{% endtab %}

{% tab title="MacOS" %}

```bash
brew install jq
```

{% endtab %}
{% endtabs %}

## Getting the latest bus stop timings

Here's a really nice API:

{% embed url="<https://github.com/cheeaun/arrivelah>" %}

```bash
curl https://arrivelah2.busrouter.sg/?id=18331
```

It gives us the bus arrival timings at public bus stops! Let's say I want to write a script that gives me the time till the next bus, let's say bus 95:

```bash
#!/bin/bash

# Replace this with your actual curl command that fetches the JSON response
response=$(curl -s "https://arrivelah2.busrouter.sg/?id=18331")

# Extract the 'time' field of the next bus using jq
next_bus_time=$(echo "$response" | jq -r '.services[0].next.time')

# Convert current time and the bus time to epoch for comparison
# TODO: Implement this (hint: figure out how to use date +%s)
current_time=
bus_arrival_time=

# Calculate time difference in seconds
time_diff=

# Convert time difference to minutes and seconds
minutes=$((time_diff / 60))
seconds=$((time_diff % 60))

# Display the result
if [ "$time_diff" -gt 0 ]; then
    echo "Bus number 95 will arrive in $minutes minutes and $seconds seconds."
else
    echo "Bus number 95 has already arrived or will arrive shortly."
fi
```

<details>

<summary>Solution</summary>

```bash
#!/bin/bash

# Replace this with your actual curl command that fetches the JSON response
response=$(curl -s "https://arrivelah2.busrouter.sg/?id=18331")

# Extract the 'time' field of the next bus using jq
next_bus_time=$(echo "$response" | jq -r '.services[0].next.time')

# Convert current time and the bus time to epoch for comparison
current_time=$(date +%s)
bus_arrival_time=$(date -d "$next_bus_time" +%s)

# Calculate time difference in seconds
time_diff=$((bus_arrival_time - current_time))

# Convert time difference to minutes and seconds
minutes=$((time_diff / 60))
seconds=$((time_diff % 60))

# Display the result
if [ "$time_diff" -gt 0 ]; then
    echo "Bus number 95 will arrive in $minutes minutes and $seconds seconds."
else
    echo "Bus number 95 has already arrived or will arrive shortly."
fi

```

</details>

Cool, but we don't really want to have to find the directory where our script is every time we want to run it! We can put it in special folders that allow us to run it from anywhere. To find this directory, we can do:

```bash
echo $PATH
```

This checks the environment variable PATH, which what the shell looks at when looking for your normal commands like `ls` and `cd`. We can then add our program to the PATH locations, most likely something like `/usr/local/bin`.&#x20;

```bash
cp bus.sh /usr/local/bin/bus
```

Now we can just run the `bus` command from anywhere like a normal shell command!

## Getting updates on the weather

{% embed url="<https://data.gov.sg/datasets?formats=API&page=2&resultId=d_50d2bbe678607d78d74a0fe6e8b5b6dd#tag/default/GET/environment/24-hour-weather-forecast>" %}

Let's do something else that way more complex! We have an API that gives us the 24 hour weather forecast in Singapore. We want to:

* Check this forecast every morning
* If it's about to rain, send us an alert. For simplicity, lets send a telegram message!

### Getting the weather forecast

```bash
curl https://api.data.gov.sg/v1/environment/24-hour-weather-forecast | jq
```

Here's the rough script we want to send:

<pre class="language-bash"><code class="lang-bash">#!/bin/bash

# Replace these with your actual Telegram bot token and chat ID
TELEGRAM_BOT_TOKEN="YOUR_BOT_TOKEN"
CHAT_ID="YOUR_CHAT_ID"
TELEGRAM_API_URL="https://api.telegram.org/bot$TELEGRAM_BOT_TOKEN/sendMessage"

# Replace this with your actual curl command that fetches the JSON weather data
response=$(curl -s "https://api.data.gov.sg/v1/environment/24-hour-weather-forecast
")

# Define an array of rain-related keywords
rain_keywords=("Rain" "Showers" "Thundery Showers" "Heavy Thundery Showers")

# Extract the general weather forecast using jq
general_forecast=$(echo "$response" | jq -r '.items[0].general.forecast')

<strong># Initialize a flag to check if rain is found
</strong>rain_found=0

# Check for rain-related keywords in the general forecast
# TODO: Try implementing this! Your code should follow this general idea:
# for each keyword in rain_keywords check if the general forecast matches it. If
# so, set rain_found to 1

# If rain is detected, send an alert
if [ "$rain_found" -eq 1 ]; then
    message="Weather Alert: Rain expected! General forecast is '$general_forecast'. Stay prepared!"
    
    # Send the message to the Telegram bot
    curl -s -X POST $TELEGRAM_API_URL \
        -d chat_id=$CHAT_ID \
        -d text="$message" > /dev/null

    echo "Alert sent: $message"
else
    echo "No rain expected. General forecast is '$general_forecast'."
fi
</code></pre>

* To get a telegram bot id, just go to @BotFather, and create a new bot, enter the token we receive inside
* To get your chat id, just go to @getmyid\_bot and copy your chat id there
* You'll need to start a chat with your bot first, go to your bot and do '/start' before you try running the script

<details>

<summary>Solution</summary>

```bash
#!/bin/bash

# Replace these with your actual Telegram bot token and chat ID
TELEGRAM_BOT_TOKEN="YOUR_BOT_TOKEN"
CHAT_ID="YOUR_CHAT_ID"
TELEGRAM_API_URL="https://api.telegram.org/bot$TELEGRAM_BOT_TOKEN/sendMessage"

# Replace this with your actual curl command that fetches the JSON weather data
response=$(curl -s "https://api.data.gov.sg/v1/environment/24-hour-weather-forecast
")

# Define an array of rain-related keywords
rain_keywords=("Rain" "Showers" "Thundery Showers" "Heavy Thundery Showers")

# Extract the general weather forecast using jq
general_forecast=$(echo "$response" | jq -r '.items[0].general.forecast')

# Initialize a flag to check if rain is found
rain_found=0

# Check for rain-related keywords in the general forecast
for keyword in "${rain_keywords[@]}"; do
    if [[ "$general_forecast" == *"$keyword"* ]]; then
        rain_found=1
        break
    fi
done

# If rain is detected, send an alert
if [ "$rain_found" -eq 1 ]; then
    message="Weather Alert: Rain expected! General forecast is '$general_forecast'. Stay prepared!"
    
    # Send the message to the Telegram bot
    curl -s -X POST $TELEGRAM_API_URL \
        -d chat_id=$CHAT_ID \
        -d text="$message" > /dev/null

    echo "Alert sent: $message"
else
    echo "No rain expected. General forecast is '$general_forecast'."
fi
```

</details>

Great! We have a working weather alert! But we don't really want to have to manually run this every morning, it would defeat the purpose of the script! Assuming your laptop runs 24/7, we can make this script run on a schedule using cronjobs!

{% hint style="info" %}
Obviously, your laptop running 24/7 is unrealistic. However, this method of scheduling scripts to be run is similar to how you would run really important scripts on a schedule: you put a cronjob on an always available server, whether it's in the cloud or somewher running in your basement
{% endhint %}

### Scheduling jobs with `cron`

* Cron is a scheduling daemon that executes tasks at specified intervals.
* These tasks are called cron jobs and are mostly used to automate system maintenance or administration.&#x20;

Essentially, we can schedule jobs on our machine to run at a specific time by putting in some magic string + the command we want to run. So what is this magic string?

Without going into too much detail about how cronjobs work, we can grab our magic string from the site below

{% embed url="<https://crontab.guru/examples.html>" %}

and pick out the crontab we want, in this case:

{% embed url="<https://crontab.guru/every-morning>" %}

So our expression should follow the format:

```bash
<magic string> <command>
```

In our case:

```bash
0 9 * * * /path/to/script/weather.sh
```

{% hint style="danger" %}
It is important we use absolute paths here.
{% endhint %}

And that's it! As long as our laptop or machine is running at 9am in the morning, this script should run automagically!


# Resources

## What's next

Some really cool resources to take your terminal skills to a different level:

* Hackers' Toolbox on working with data quickly in the terminal

{% embed url="<https://youtu.be/gB-xbpvVpBM?si=Ymkca60l7sD0ZQ6z>" %}

* Even more on the command-line environment

{% embed url="<https://missing.csail.mit.edu/2020/command-line/>" %}

* A bit more on modern, standard tools we use on the command line

{% embed url="<https://nushackers.github.io/hackertools_materials/unix-utils/>" %}

{% embed url="<https://github.com/johnalanwoods/maintained-modern-unix>" %}


# Hands On Self-Hosting

## Note: This wiki was made for a workshop in Mar 2026.&#x20;

*The recording is still being processed, and will be uploaded soon*

<figure><img src="/files/TBjG42QgTA6lJCOY5Gvx" alt=""><figcaption></figcaption></figure>

Thanks to [DigitalOcean](https://digitalocean.com/) for sponsoring the credits for the workshop!

## Prologue: What is Self Hosting?&#x20;

> **The act of providing or serving digital content or an online service typically delivered by a business.**&#x20;
>
> The service or content is generally served locally from your own hardware. Often "self-hosters" use older Enterprise-grade hardware from their home internet connections however they also use other hosting providers hardware. This is still considered self-hosting.

While people who get into self-hosting often use their own hardware, using hosting providers help abstract away the hardware difficulties while getting you 90% of the way there.

### What are some things we can create by self-hosting?

* Hosting a website is one of the lowest barrier of entry. (Nginx)
* Media-streaming services (Jellyfin, Plex)
* File synchronization services (Gitlab, Syncthing, Garage, etc)
* Photo galleries (Immich)
* Databases (Supabase, Convex, Postgres, etc)
* Game-servers (Minecraft, Rust, Factorio)
* You should checkout: [Awesome-Selfhosted](https://github.com/awesome-selfhosted/awesome-selfhosted)!

### Why not just pay/get someone to do these for you?

* It’s great for learning! (and fun!)
* Self-custody of your own data (this is limited with VPS)
* Run servers/services that aren’t available as a SaaS
* Get fine-grained control over your service (great for debugging personal projects)

You can learn more here:

{% embed url="<https://wiki.r-selfhosted.com/>" %}


# Server Setup

## Servers

> Servers are machines whose purpose is to provide a service or content over a network. They are typically administered remotely and only connect physically to power and a network. They "serve" content or services using software daemons. Their natural habitat is the datacenter, where they live in racks to survive off electricity and network data. While they are not able to reproduce, they have no natural predators, so their population is stable. Some breeds of server can be found in network/data closets where they live in a business. Fewer are still kept in captivity in private homes. Virtual servers are servers that are run under an emulator or hypervisor to provide a server-like environment using a software envelope which may be augmented with hardware support.

## Finding a server to use

For small projects and little experiments, there are some no cost options you can try!

{% embed url="<https://github.com/cloudcommunity/Cloud-Free-Tier-Comparison>" %}

### Renting Virtual Private Servers

A virtual private server, also known as a VPS, acts as an isolated, virtual environment on a physical server, which is owned and operated by a cloud or web hosting provider.

### Procuring your own hardware

* You can use anything as your server!
  * Old Laptops
  * Cheap NAS
  * Single Board Computers (Raspberry Pi)
  * Second-hand mini-pcs

We'll assume you have somehow gotten a server to work with, or if you're following the workshop, you should have something to work with already! The following guide will assuming your server is running **Ubuntu**, an operating system commonly found in servers.

***

## Setting up your server \[Run on your local computer]

* Get a username and password to your server, and SSH with a client:
  * Terminal on MacOS
  * Windows Terminal or PuTTy on Windows
  * Any shell on Linux

```
ssh root:<ip-address>
```

You will be prompted for a password.

If you are using Ghostty, run this command (on your local computer) as well:&#x20;

`infocmp -x xterm-ghostty | ssh <username>@<ip-address> tic -x -`

***

## Updating Packages

```shellscript
sudo apt update && sudo apt upgrade
```

***

### Creating a new user, and disabling root

This is important if you are logged into your server as **root.** As root is a common username, there will be people enumerating through common usernames on every possible IP address just to try their luck and compromise servers.&#x20;

```shellscript
useradd -m -d /home/<username> -s /bin/bash <username> # Add user
usermod -a -G sudo,adm <username> # Give permissions
sudo passwd <username> # To create a password for the user
```

{% hint style="info" %}
You might need to add the keyword `sudo` before all these commands. `sudo` basically allows us to run with superuser privileges.
{% endhint %}

***

## Setting up SSH Keys

#### What exactly are keys?

Keys are a secure way to log into remote computers without using passwords. Here's a simple explanation:

* SSH keys come in pairs: a public key and a private key
* The public key is like a padlock that you put on the remote server
* The private key is like the key to that padlock, which you keep on your local computer
* When you try to log in, your computer uses the private key to prove it can "unlock" the padlock
* If successful, the server lets you in without asking for a password

This method is more secure than passwords because:

* The private key never leaves your computer
* It's extremely difficult for someone to guess or crack your key
* Even if someone intercepts your login attempt, they can't see your private key

By using SSH keys, you can log in quickly and securely without typing a password each time.

#### Setting up your own SSH Keys

While some service providers have a webshell, it’s much nicer (and less laggy) to work in your own terminal, and it familarises you for other services too!

#### 1) On your local terminal (NOT in SSH)

* `ssh-keygen -t ed25519`
  * Press enter for the defaults, and note the file that you saved the key in.&#x20;
  * Take note of the `path/to/ssh/key` , we'll need it later!
* `ssh-copy-id <username>@<ip-address>`
  * If you configured your own path to install the key, run: \
    `ssh-copy-id -i path/to/ssh/key <username>@<ip-address>`

You should see a confirmation message: `Number of key(s) added: 1`

Now, you should be able to get a shell in your server, without any passwords!

#### 2) On the Server

You should now be able to go to `vim /home/<username>/.ssh/authorized_keys`, to see the authorized keys&#x20;

### SSH Hardening (Restricting Root + PasswordAuthentication)

* `sudo vim /etc/ssh/sshd_config`
* Set `PermitRootLogin` to  `prohibit-password`
* Set `PasswordAuthentication` to `no`, so that we disable password login
* Save and exit (`:wq` in vim)
* Some servers will create pre-populated SSH configurations. We should check if it exists, and remove it with:&#x20;
  * &#x20;`rm -r /etc/ssh/sshd_config.d/`
* 🚨⚠️ **Note that after this, you will have no way of recovering into this server without your key.**
* Then, restart ssh `sudo systemctl restart ssh`&#x20;

### Quality of Life: Disable Sudo Password

* `sudo visudo`&#x20;
* At the end of the file, add: `<username> ALL=(ALL) NOPASSWD:ALL`&#x20;

{% hint style="info" icon="triangle-exclamation" %}
Note the security implication: anything with user-level access, can make system-wide changes without your password.
{% endhint %}

### Quality of Life: SSH Config \[Edit on your local computer]

* `sudo vim ~/.ssh/config`&#x20;
* In the config file, add the following:

  ```
  Host <hostname>
      HostName <ip-address>
      User <username>
      IdentityFile <path-to-ssh-key>
  ```

{% hint style="info" %}
Not familiar with the terminal/shell? Check out a quick introduction here:

<https://wiki.nushackers.org/hackers-toolbox/beginners-guide-to-the-terminal/introduction-to-the-terminal#the-shell-prompt>
{% endhint %}


# Firewalls and Security

We want to make sure that we can expose services and services to the internet, but we want to ensure that bad actors cannot enter our system.&#x20;

## UFW (Uncomplicated Firewall)

### What is UFW?&#x20;

**UFW (uncomplicated firewall)** is a command-line tool designed to simplify firewall management on Linux systems, particularly those based on Ubuntu. Built on top of [`iptables`](https://www.digitalocean.com/community/tutorials/iptables-essentials-common-firewall-rules-and-commands), it provides a user-friendly way to define rules for controlling network traffic, such as allowing or blocking specific ports, IP addresses, or services.

### Let's set up UFW&#x20;

```shellscript
sudo su                        # You must be root to configure UFW
ufw status                     # Check current firewall status (should be inactive)

ufw app list                   # List available application profiles
ufw allow OpenSSH              # Allow SSH connections
ufw limit ssh                  # Rate-limit SSH to prevent brute force

ufw default deny incoming      # Block all INcoming traffic by default
ufw default allow outgoing     # Allow all OUTgoing traffic by default

ufw enable                     # Activate the firewall 

ufw status                     # See the current ufw rules!
exit                           # go back to your own user
```

{% embed url="<https://www.digitalocean.com/community/tutorials/how-to-set-up-a-firewall-with-ufw-on-ubuntu>" %}

#### Note that there are some implications of UFW and Docker:

{% embed url="<https://github.com/chaifeng/ufw-docker>" %}

But no worries, we will also cover how we can tackle this on Docker's side - spoiler, its to always define the service hostnames as  `127.0.0.1` !


# Domains & DNS

This section will guide you on how to register for a domain, but we will not be explaining in depth how DNS works.

## Domain Names

So, now we’ve got a server up. But you notice an IP address is kind of ugly and hard to remember… that’s where **domain names** come into play! If you’ve ever typed a website URL, you’ve effectively typed a domain name.

Well, how do IP Addresses turn into domain names? All you need to know is there are a lot of servers out there maintaining a large table of IP addresses to domain name mappings. These are known as **DNS servers**.

### Getting your own domains

If you have the Github Student Developer Pack, you should be able to get a domain name from .tech for free for a year.&#x20;

{% embed url="<https://www.name.com/partner/github-students>" %}

If you want to get a cool domain name, you can use this to compare prices from different registrars:

{% embed url="<https://get.tech/github-student-developer-pack>" %}

{% embed url="<https://tld-list.com>" %}

***

Every domain will have the following structure

```
<domain name>:<tld>
nushackers.org
```

For every domain, you can also have a bunch of records for **subdomains**

```
www.nushackers.org (www subdomain)
hacknroll.nushackers.org (hack&roll subdomain for hack&roll site)
wiki.nushackers.org (wiki subdomain, hosted on gitbook!)
```

***

## Shifting DNS Nameservers to Cloudflare

#### Why is this necessary?

* Performance - Fast global DNS resolution
* Free DDoS protection - Protect your websites against attacks
* CDN & caching - Faster worldwide content delivery
* Free SSL - HTTPS Websites

#### How do I do this?

{% stepper %}
{% step %}

### In the domain registar > "Manage Nameservers"

<figure><img src="/files/6xv0YTjKxPUs6plottWC" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Delete Existing Nameservers&#x20;

Delete all of them, as we will be migrating to Cloudflare's

<figure><img src="/files/BHeQIHeo4wnNnZYI1xh2" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Cloudflare > Domain Management > "Onboard a domain"

<figure><img src="/files/1K2cZvnnryoPO59GCONV" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Enter the domain name

<figure><img src="/files/cdEtLXfWA1oBV4JRdydh" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Cloudflare will give you the nameservers to update

<figure><img src="/files/GKAdPNo1oHKtm1Ppod0N" alt=""><figcaption></figcaption></figure>

In this case, it is `dean.ns.cloudflare.com` , and `sue.ns.cloudflare.com`&#x20;
{% endstep %}

{% step %}

### Update registar with Cloudflare DNS

<figure><img src="/files/lgdFbtXYkjpbuSRfaqqj" alt=""><figcaption></figcaption></figure>
{% endstep %}
{% endstepper %}

Congrats, your website will be part of Cloudflare's next outage! 🤞

This will likely take a 5-10 minutes to update.

<details>

<summary>Additional: Configuring your own DNS Records</summary>

We’ll start by creating a bunch of A records:

* Leaving the hostname blank will just lead to the main domain
* Add \* as the hostname will route all empty subdomains to a single address
* Time to Live (TTL) is a field on DNS records that controls how long each record is valid and — as a result — how long it takes for record updates to reach your end users.

</details>

### Additional Reading

{% embed url="<https://www.cloudflare.com/learning/dns/what-is-dns/>" %}

Fun reading behind the names of Cloudflare Name Servers:

{% embed url="<https://blog.cloudflare.com/whats-the-story-behind-the-names-of-cloudflares-name-servers/>" %}


# Docker

> Docker is an OS‑level virtualization (or containerization) platform, which allows applications to share the host OS kernel instead of running a separate guest OS like in traditional virtualization. This design makes Docker containers lightweight, fast, and portable, while keeping them isolated from one another.

### Why use Docker?

* **Portability**: Runs anywhere in local machine, cloud, on‑prem servers.
* **Consistency**: Same behavior in development, testing, and production.
* **Lightweight**: No full OS per app; containers share the host kernel.
* **Scalability**: Ideal for microservices and orchestrators like Kubernetes and Docker Swarm.
* **Efficiency**: Starts in seconds, uses fewer system resources.

### Docker Set Up

We can set up using apt. First, we set up Docker's `apt` repository.

```shellscript
# Add Docker's official GPG key:
sudo apt update
sudo apt install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
sudo tee /etc/apt/sources.list.d/docker.sources <<EOF
Types: deb
URIs: https://download.docker.com/linux/ubuntu
Suites: $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}")
Components: stable
Signed-By: /etc/apt/keyrings/docker.asc
EOF

sudo apt update
```

*Generally, best practices dictate that you should copy and paste commands from the internet but you trust me right ;)*

Next, lets install Docker:

```bash
 sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
```

We can then check that Docker is running:

```bash
sudo systemctl status docker
```

We can now add the user to the docker group, so that you can run docker commands without sudo:&#x20;

```bash
sudo usermod -aG docker <username>
exit
```

ssh back into the server and you should be able to run:&#x20;

```bash
docker ps
```

\
Full documentation of setting up in Ubuntu:

{% embed url="<https://docs.docker.com/engine/install/ubuntu/>" %}

***

## Brief Introduction to Docker

{% hint style="info" %}
The **following subsections 1-8 are not necessary for set up.** This is for your learning!
{% endhint %}

{% stepper %}
{% step %}

### How does Docker work?

Docker is a **containerization** platform, that allows applications to run in **containers**.&#x20;

Think of containers like micro virtual machines, where each container lives in its own isolated namespace, where it has its own processes, networking users, etc.&#x20;

This makes containers very lightweight (*and fast)* compared to virtual machines.&#x20;
{% endstep %}

{% step %}

### Images & Docker Hub

Docker images are templates to create containers. The image specifies things like what files and applications(binaries) should exist in the container, and how to start the container.&#x20;

Images are normally stored in **registries.** The most common registry is dockerhub.&#x20;

<https://hub.docker.com/>

```docker
FROM python:3.14.3-alpine

# equiv to mkdir app; cd app/
WORKDIR /app 

# copy files from the host machine into image
COPY . . 
# tell docker that port 8000 is exposed
EXPOSE 8000
# command to run
CMD ["python3", "-m", "http.server", "8000"]
```

* `python:3.14.3`:&#x20;
  * `python` is the image it is going to pull from the registry. for docker, it pulls from dockerhub by default.&#x20;
  * `3.14.3` is the image tag, a label assigned to a particular version of an image.&#x20;

This builds the above Dockerfile with the name mypython.&#x20;

```bash
docker build -f Dockerfile . -t mypython
docker image ls
```

{% endstep %}

{% step %}

### Containers

Containers are running instances of an image. From the image, it runs the `cmd` block specified on startup.&#x20;

{% hint style="warning" %}
Containers are ephemeral by default! All data is lost when container is removed.&#x20;
{% endhint %}

* ```bash
  docker run hello-world
  ```
  * finds the image hello-world, and runs it until it completes
* ```bash
  docker container ls -a
  docker container rm <image_name>
  ```
  * Lists all containers, running and stopped.&#x20;
  * Deletes a given container
* ```bash
  docker run -i -t --rm ubuntu
  ```
  * finds the image ubuntu, attaches your input and terminal and runs in it. when you exit, it deletes the container.&#x20;
  * `-i` : Run in an interactive process (you can type and it will be passed into the container)
  * `-t` : it attaches a terminal&#x20;
  * `--rm` : it deletes the container its running automatically when stopped
* ```bash
  docker run -d --name mynginx nginx

  ```
  * finds the image nginx (a web server), creates a container with the name mynginx and runs it detached, in the background
  * `-d` : run detached, running in the background instead of attached to your terminal
* ```bash
  docker ps
  docker stats
  ```
  * shows you information about the current running containers in your system. &#x20;
* ```bash
  docker stop mynginx
  ```
  * stops the running container (doesn't delete it!)
    {% endstep %}

{% step %}

### Volumes

Volumes are how you persist data outside of the container's lifecycle.&#x20;

Volumes are normally used for databases, sharing files between host and container, etc.&#x20;

```bash
docker volume create mydata 
docker run -v mydata:/data postgres
docker run -v $(pwd):/app node
```

<table><thead><tr><th width="90">Type</th><th>Description</th></tr></thead><tbody><tr><td>Volume</td><td>Docker managed persistent storage</td></tr><tr><td>Bind Mount</td><td>Mount your host directory into the container</td></tr></tbody></table>
{% endstep %}

{% step %}

### Ports

Containers run in their own isolated networks, so their ports are not accessible by default. We need to specify which ports to *publish* when we run the container

```bash
docker run -d -p 127.0.0.1:8080:80 nginx
```

* `8080:80` : This follows a `host`: `container` structure. You local machine's port `8080` is bound to the containers port `80` , so all traffic between those 2 ports are piped.

{% hint style="danger" %}
Docker by default ignores the rules set by ufw. Read more here: <https://github.com/chaifeng/ufw-docker>

\
So you can run&#x20;

```
docker run -d -p 8080:80 nginx
```

but this will be exposed to the internet.
{% endhint %}

<details>

<summary>You can access a container's ports without publishing it, but it isn't really recommended</summary>

```bash
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' <containername>
# get the ip address from that
curl http://<ip>:<port>
```

</details>
{% endstep %}

{% step %}

### Environment variables

Similar to environment variables in your host system, we can pass environment variables to containers as well. A lot of applications use environment variables for configuration.&#x20;

```bash
docker run -e POSTGRES_PASSWORD=boo -e POSTGRES_USER=someone postgres
```

{% endstep %}

{% step %}

### Networking

Docker provides builtin networking. Networking is a very complex topic, that we do not have the time or the capabilities to explain...

Containers in the same network can see and talk to each

```bash
docker network create -d bridge mynetwork
docker run --network mynetwork nginx
```

{% endstep %}

{% step %}

### Docker Compose

Docker compose is a tool for running **multi-container applications**

Multi-container applications? Example:&#x20;

* your application, written in python
* `postgres` which your application talks to
* `redis` which your application uses as a cache

All of these can be bundled and *orchestrated* together using docker compose!

Another side effect: All the commands above? We don't need to run any of them! WAHOOOO
{% endstep %}
{% endstepper %}

## Let's Run Docker Compose for Portainer&#x20;

```bash
cd /home/<username> &&
mkdir -p docker-compose/portainer &&
cd docker-compose/portainer &&
vim docker-compose.yml
```

{% code title="docker-compose.yml" %}

```yaml
services: # each item under services would be a container
  portainer:
    container_name: portainer
    image: portainer/portainer-ce:sts
    restart: always # what to do when the container crashes / exits
    
    volumes: # where to mount the volumes into the container
      - /var/run/docker.sock:/var/run/docker.sock
      - portainer_data:/data
    ports: # what ports we want to publish
      - 127.0.0.1:9443:9443

volumes: # docker volume create ...
  portainer_data:
    name: portainer_data

networks: # docker network create ...
  default:
    name: portainer_network
```

{% endcode %}

```bash
docker compose up ## deploy the docker-compose 

## we can run compose down to shutdown the docker containers defined in the docker-compose 
## docker compose down 
```

## Useful Docker Commands

<table><thead><tr><th width="130"></th><th></th></tr></thead><tbody><tr><td><code>docker run</code> </td><td>It used for launching the containers from images, with specifying the runtime options and commands</td></tr><tr><td><code>docker pull</code></td><td>Fetch the container images from the container registry like Docker Hub to the local machine</td></tr><tr><td><code>docker ps</code></td><td>Display the running containers along with their important information like container ID, image used and status</td></tr><tr><td><code>docker stats</code></td><td>Show the stats and compute usage of the current running containers</td></tr><tr><td><code>docker stop</code></td><td>Halt running containers gracefully shutting down the processes within them</td></tr><tr><td><code>docker start</code></td><td>Restart the stopped containers, resuming their operations from the previous state.</td></tr></tbody></table>

## Additional Readings

Some content here was based off:

{% embed url="<https://www.geeksforgeeks.org/devops/introduction-to-docker/>" %}

{% embed url="<https://docs.docker.com/reference/compose-file/>" %}


# Portainer

> [Portainer](https://github.com/portainer/portainer) is a lightweight, web-based graphical user interface (GUI) designed to simplify container management for Docker, Docker Swarm, Kubernetes, and Azure ACI

## Alternatives to Portainer

There are many alternatives to portainer. In fact, you don't even need to use it at all! It just makes configuring docker set ups easier.

There are many other alternatives, such as:

* [Dockge](https://github.com/louislam/dockge)
* [Komodo](https://komo.do/)

However, we won't be going into detail for this session - we shall use portainer mainly just because I'm familiar with it :D

## Portainer Set Up

{% stepper %}
{% step %}

### Write Docker Compose File

First, let's navigate to a good place to write our docker files. Typically, I just write it in my home directory

```bash
cd /home/<username> &&
mkdir -p docker-compose/portainer &&
cd docker-compose/portainer &&
vim docker-compose.yml
```

{% code title="docker-compose.yml" %}

```yaml
services: # each item under services would be a container
  portainer:
    container_name: portainer
    image: portainer/portainer-ce:sts
    restart: always # what to do when the container crashes / exits
    
    volumes: # where to mount the volumes into the container
      - /var/run/docker.sock:/var/run/docker.sock
      - portainer_data:/data
    ports: # what ports we want to publish
      - 127.0.0.1:9443:9443

volumes: # docker volume create ...
  portainer_data:
    name: portainer_data

networks: # docker network create ...
  default:
    name: portainer_network
```

{% endcode %}
{% endstep %}

{% step %}

### Running portainer

In the same directory that you wrote the docker-compose, you should now start the docker container:&#x20;

```bash
docker compose up
```

After it starts up, you can always see check the running process by running:

```bash
docker ps
```

<figure><img src="/files/UvPvuX4QSfn27sm19UF9" alt=""><figcaption></figcaption></figure>

Note how `0.0.0.0:9443->9443/tcp`, this means that later when we expose to the internet, we need to connect to the host port `9443` to be able to connect to this portainer container!
{% endstep %}

{% step %}

### Make Portainer Accessible from the Internet

We can use Cloudflare Tunnels for this. Read ahead to [Exposing to the Internet](/hackerschool/hands-on-self-hosting/exposing-to-the-internet), then come back!
{% endstep %}

{% step %}

### Configuring Portainer

You should now be seeing a screen like this. Create a username and password, and store it safely!

<figure><img src="/files/YsnkF8fGUxZfFYoF0wNM" alt=""><figcaption></figcaption></figure>
{% endstep %}
{% endstepper %}

## How to deploy Docker Compose on Portainer?

Portainer calls Docker Compose as Stacks. So, we can head to `Dashboards > Stacks > Add Stacks` in order to add new docker compose containers.

Now, we can go ahead with [Let's Host Things!](/hackerschool/hands-on-self-hosting/lets-host-things), where we will walk through using Portainer by hosting a service.

## Portainer UI

Home Page:

<figure><img src="/files/X4f5nWt4aieuy8InEoGa" alt=""><figcaption></figcaption></figure>

Dashboard:

<figure><img src="/files/ormehRmgqHOxHzV9Fr6N" alt=""><figcaption></figcaption></figure>

Managing Containers:

<figure><img src="/files/HuJ8QyOAYFIwfpzjQxmE" alt=""><figcaption></figcaption></figure>

Container Details:

<figure><img src="/files/LGKDLpOg6nwEp5g71f76" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/ViNxFxFRkAQlYwBgL7IR" alt=""><figcaption></figcaption></figure>

For more details, please refer to Portainer's Guide:

{% embed url="<https://docs.portainer.io/user/home>" %}


# Exposing to the Internet

## Why do we want to expose things?

We want to expose services to the internet for people to use! For example, a gitlab instance such as <https://git.yadunut.dev/> should be publicly available!

However, Yadunand might not want to expose his `TopSecretService` running on port `8080` to the internet, so we have to be careful of how we are exposing these services.

## Cloudflare Tunnels

### How does it work?

<figure><img src="/files/ObQ4lJpSsPibPnS4zeX8" alt=""><figcaption></figcaption></figure>

Cloudflare Tunnel uses an outbound-only connection model to enable bidirectional communication. When you install and run `cloudflared`, `cloudflared` initiates an outbound connection through your firewall from the origin to the Cloudflare global network.

Most firewalls (and ours!) allow outbound traffic. `cloudflared` takes advantage of this standard by connecting out to the Cloudflare network from the server you installed `cloudflared` on. You can then configure your firewall to allow only these outbound connections and block all inbound traffic, effectively blocking access to your origin from anything other than Cloudflare.&#x20;

All traffic will then be securely routed through the tunnel.

***

#### We will not be setting up via the Cloudflare Dashboard

For some reason, it requires us to add a card, even though it is a free service :<&#x20;

But no fear, we can do it on the cli for free! WAHOOo

{% stepper %}
{% step %}

### Installation On Ubuntu 24.04&#x20;

For other Linux Distros: <https://pkg.cloudflare.com/index.html>

```zsh
# Add cloudflare gpg key
sudo mkdir -p --mode=0755 /usr/share/keyrings
curl -fsSL https://pkg.cloudflare.com/cloudflare-main.gpg | sudo tee /usr/share/keyrings/cloudflare-main.gpg >/dev/null

# Add this repo to your apt repositories
# Stable
echo 'deb [signed-by=/usr/share/keyrings/cloudflare-main.gpg] https://pkg.cloudflare.com/cloudflared noble main' | sudo tee /etc/apt/sources.list.d/cloudflared.list
# Nightly
echo 'deb [signed-by=/usr/share/keyrings/cloudflare-main.gpg] https://next.pkg.cloudflare.com/cloudflared noble main' | sudo tee /etc/apt/sources.list.d/cloudflared.list

# install cloudflared
sudo apt-get update && sudo apt-get install cloudflared
```

{% endstep %}

{% step %}

### For the rest of the setup, we are following:

{% embed url="<https://developers.cloudflare.com/cloudflare-one/networks/connectors/cloudflare-tunnel/do-more-with-tunnels/local-management/create-local-tunnel/>" %}

If you did not manage to register a domain, you can skip the next few steps, instead just expose your services now with [Cloudflare Temporary Tunnels](#quickly-exposing-services-to-the-internet)
{% endstep %}

{% step %}

### Login

```bash
sudo su 
cloudflared tunnel login
```

It will then launch a window on cloudflare dashboard, where you can select a domain to authorize.

<figure><img src="/files/awYlcb3ZNiTQFeDaiSzi" alt="" width="375"><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Create a Tunnel

```bash
cloudflared tunnel create <NAME>
```

This will create a tunnel with a unique UUID, and generate a tunnel credential file.

**Take note of this UUID, as we will be using it later.**

You can view your existing tunnels using: `cloudflared tunnel list`&#x20;

<details>

<summary>If you need to delete a tunnel</summary>

```bash
cloudflared tunnel cleanup <UUID>
cloudflared tunnel delete <UUID>
```

If the tunnel is still running, you need to run these to kill the process:

```bash
systemctl stop cloudflared
pkill -f "cloudflared tunnel"
```

</details>
{% endstep %}

{% step %}

### Creating Tunnel Configuration file

The configuration file should be created at `/etc/cloudflared/config.yml`

This is what the file can look like:

```bash
sudo mkdir -p /etc/cloudflared
sudo vim /etc/cloudflared/config.yml
```

{% code title="config.yml" %}

```yaml
### THIS IS AN EXAMPLE CONFIG FILE ###

tunnel: 6ff42ae2-765d-4adf-8112-31c55c1551ef
credentials-file: /root/.cloudflared/6ff42ae2-765d-4adf-8112-31c55c1551ef.json
origincert: /root/.cloudflared/cert.pem

ingress:
  # Rules map traffic from a hostname to a local service:
  - hostname: example.com
    service: http://localhost:8000
  # Rules can match the request's path to a regular expression:
  - hostname: static.example.com
    path: \.(jpg|png|css|js)$
    service: http://localhost:8001
  # Rules can match the request's hostname to a wildcard character:
  - hostname: "*.example.com"
    service: http://localhost:8002
  # An example of a catch-all rule:
  - service: http://localhost:8003
```

{% endcode %}

In our case, let's set up a config which points the `docker` subdomain to `https://localhost:9443`

{% code title="config.yml" %}

```yaml
tunnel: TUNNEL-UUID
credentials-file: /root/.cloudflared/<TUNNEL-UUID>.json
origincert: /root/.cloudflared/cert.pem

ingress:
  # Let's map the subdomain docker here.
  - hostname: docker.<domain-name>.<tld>
    service: https://localhost:9443
    originRequest:
      noTLSVerify: true
  
    
  # Catch all everything else to 404
  - service: http_status:404
```

{% endcode %}

Note that we had to add an additional config of noTLSVerify. This is because&#x20;

&#x20;[Full Configuration File Docs (Cloudflare)](https://developers.cloudflare.com/cloudflare-one/networks/connectors/cloudflare-tunnel/do-more-with-tunnels/local-management/configuration-file/)
{% endstep %}

{% step %}

### Configure DNS

In our example, since we have configured cloudflared to tunnel traffic from: `docker.<domain-name>.tld -> https://localhost:9443` , we need to configure out cloudflare DNS to also match this

{% code title="" %}

```bash
sudo cloudflared tunnel route dns <TUNNEL-NAME OR TUNNEL-UUID> docker.<domain-name>.<tld>
```

{% endcode %}
{% endstep %}

{% step %}

### Start running Cloudflared

#### Install

{% code title="" %}

```bash
sudo cloudflared service install
```

{% endcode %}

Start service

{% code title="" %}

```bash
sudo systemctl start cloudflared
```

{% endcode %}

Check service

{% code title="" %}

```bash
sudo systemctl status cloudflared
```

{% endcode %}
{% endstep %}

{% step %}

### How to add new routes&#x20;

1\) Edit the config and add the new domain and which localhost its pointed to

{% code title="" %}

```bash
sudo vim /etc/cloudflared/config.yml
```

{% endcode %}

2\) Add new DNS route

{% code title="" %}

```bash
sudo cloudflared tunnel route dns <TUNNEL-UUID or TUNNEL-NAME> docker.<domain-name>.<tld>
```

{% endcode %}

3\) Reload cloudflared

{% code title="" %}

```bash
sudo systemctl restart cloudflared
```

{% endcode %}

4\) Check the status after

{% code title="" %}

```bash
sudo systemctl status cloudflared
```

{% endcode %}
{% endstep %}
{% endstepper %}

### We should be able to view portainer now!

Nothing shows up? This might be the Portainer instance timed out for security purposes, to re-enable your Portainer instance, you will need to restart Portainer.

<details>

<summary>To do this, we can run: </summary>

{% code title="" %}

```bash
docker restart portainer
```

{% endcode %}

This might not work if the container is not named portainer. In that case, you should run:

{% code title="" %}

```bash
docker ps
docker logs <container-id>
docker restart <container-id>
```

{% endcode %}

</details>

We can also troubleshoot other problems on the Cloudflare Dashboard. Common culprits are DNS as well. (Are you sure you set that up?)

***

### Quickly exposing services to the internet

Cloudflare provides a service called [Quick Tunnels](https://developers.cloudflare.com/cloudflare-one/networks/connectors/cloudflare-tunnel/do-more-with-tunnels/trycloudflare/). This is usually meant for development environments, so it is not a good permanent solution!

It will generate a domain that you can use, which is pointed towards your ip address:

You will however, probably need to run these in separate tmux windows.. (It really isn't ideal!)&#x20;

```bash
sudo cloudflared tunnel  --config /dev/null --url http://127.0.0.1:<port-number>
```

To tunnel to portainer specifically, you can run:

```bash
sudo cloudflared tunnel --config /dev/null --no-tls-verify --url https://127.0.0.1:9443 
```

:warning:: The `--config /dev/null`  flag is needed because cloudflared will sometimes use the signed in credentials rather than creating a quick tunnel.

### Additional Docs:

{% embed url="<https://developers.cloudflare.com/cloudflare-one/networks/connectors/cloudflare-tunnel/do-more-with-tunnels/local-management/create-local-tunnel/>" %}

{% embed url="<https://developers.cloudflare.com/cloudflare-one/networks/connectors/cloudflare-tunnel/do-more-with-tunnels/local-management/configuration-file/>" %}

{% embed url="<https://developers.cloudflare.com/cloudflare-one/networks/connectors/cloudflare-tunnel/do-more-with-tunnels/local-management/tunnel-useful-commands/>" %}

## What are some other ways to do this?

This will not be covered in the scope of this workshop, but there are many other ways to do this as well.&#x20;

### VPNs: Wireguard/ Tailscale

Usually, VPNs are used when we only want our services exposed to authenticated users with a VPN client. This is the same principle that SoC uses, for us to connect into SoC internal network via SoC VPN.&#x20;

<figure><img src="/files/rv5u9WpPPcpuyUJ8ZuVF" alt=""><figcaption></figcaption></figure>

{% embed url="<https://tailscale.com/blog/how-tailscale-works>" %}

### Reverse Proxies: Traefik/ Caddy

<figure><img src="/files/j93M4doLQmZbkt4k3PKA" alt=""><figcaption></figcaption></figure>

{% embed url="<https://doc.traefik.io/traefik/>" %}

## Additional Readings


# Let's Host Things!

Now finally, we can easily host new containers and services, securely!&#x20;

## Let's try hosting ConvertX

{% embed url="<https://github.com/C4illin/ConvertX>" %}

A self-hosted online file converter. Supports over a thousand different formats. Written with TypeScript, Bun and Elysia.

<figure><img src="/files/1sRxTAWxXqEDVK8DxsbV" alt=""><figcaption></figcaption></figure>

{% stepper %}
{% step %}

### Let's use Docker Compose

From the Github:

```yaml
# docker-compose.yml
services:
  convertx:
    image: ghcr.io/c4illin/convertx
    container_name: convertx
    restart: unless-stopped
    ports:
      - "127.0.0.1:3000:3000"
    environment:
      - JWT_SECRET=aLongAndSecretStringUsedToSignTheJSONWebToken1234 # will use randomUUID() if unset
      # - HTTP_ALLOWED=true # uncomment this if accessing it over a non-https connection
    volumes:
      - ./data:/app/data
```

{% endstep %}

{% step %}

### Open Portainer

Go to Stacks > Click on +Add Stacks, then put in the Docker Compose.

<figure><img src="/files/vlhIn0cg7GW7CiSmLUYz" alt=""><figcaption></figcaption></figure>

Then, we can click on Deploy the Stack!
{% endstep %}

{% step %}

### Configure Cloudflare Tunnels

Remember, now we need to configure Cloudflare to be able to expose this to the internet, and we can follow the guide from the previous page on [how to add new routes](https://app.gitbook.com/o/DCgnBetC4YnbJAmkVblZ/s/TUqAJOgHs57S8lmqdxRV/~/edit/~/changes/57/hackerschool/hands-on-self-hosting/exposing-to-the-internet#how-to-add-new-routes)! We reccommend you trying it yourself.

<details>

<summary>Instructions</summary>

```bash
sudo vim /etc/cloudflared/config.yml
```

Add the following lines to config.yml (remember to edit the domain name and tld):

```
  - hostname: convert.<domain-name>.<tld>
    service: http://localhost:3000
```

Add the new subdomain record to DNS:

```bash
sudo cloudflared tunnel route dns <TUNNEL-UUID or TUNNEL-NAME> convert.<domain-name>.<tld>
```

Restart and update Cloudflard:

```bash
sudo systemctl restart cloudflared
```

</details>
{% endstep %}

{% step %}

### Congrats, WE'RE DONEEE

{% endstep %}
{% endstepper %}

## A Non-Exhaustive List of Cool Things to Self Host:

Now that you know how to self-host things, it's also good to know where to find other cool things.&#x20;

Of course, you can also just run your own dev projects and expose the port to the internet the same!

### Some cool services we use:

* [Jellyfin](https://github.com/jellyfin/jellyfin): Open-source media server for streaming and managing your movies, shows, music, and photos.
* [Uptime Kuma](https://github.com/louislam/uptime-kuma): Self-hosted uptime monitoring tool with a clean dashboard and alerting support.
* [Shlink](https://github.com/shlinkio/shlink): Self-hosted URL shortener with analytics and API support.
* [Copyparty](https://github.com/9001/copyparty): Lightweight file server for sharing, uploading, and managing files over the web.
* [Beszel](https://github.com/henrygd/beszel): Lightweight server monitoring platform for tracking system stats and health.
* [CodiMD](https://github.com/hackmdio/codimd): Collaborative markdown editor for real-time note-taking and documentation.
* [Grafana](https://github.com/grafana/grafana): Visualization platform for dashboards, metrics, logs, and observability data.
* [Prometheus](https://github.com/prometheus/prometheus): Open-source monitoring and alerting toolkit for collecting time-series metrics.
* [Home Assistant](https://github.com/home-assistant/core): Open-source home automation platform for controlling and automating smart devices.
* [Stirling PDF](https://github.com/Stirling-Tools/Stirling-PDF): Self-hosted PDF toolkit for merging, splitting, converting, and editing PDFs.
* [MeTube](https://github.com/alexta69/metube): Web UI for downloading videos and audio from YouTube and many other sites.
* [TinyAuth](https://github.com/steveiliop56/tinyauth): Lightweight self-hosted authentication service for protecting internal apps.
* [Pocket ID](https://github.com/pocket-id/pocket-id): Simple self-hosted identity provider with passkey support for secure logins.
* [Karakeep](https://github.com/karakeep-app/karakeep): Self-hosted bookmark manager for saving, organizing, and searching links.
* [Forgejo](https://github.com/forgejo/forgejo): Self-hosted lightweight software forge for Git hosting, issues, and collaboration.
* [Immich](https://github.com/immich-app/immich): Self-hosted photo and video backup platform with search and mobile sync.

{% embed url="<https://awesome-selfhosted.net/>" %}

This is based off: <https://github.com/awesome-selfhosted/awesome-selfhosted>

{% embed url="<https://www.reddit.com/r/selfhosted/>" %}


# Running Other Services

This was initially made for an older version of this workshop, and hence will not be used today, but it is still super relevant!

## Running Services

### Hosting a web page with nginx

* `nginx` is a great web server for these things, but you can pick whatever you want, or are more familiar with
* On Ubuntu, this is pretty simple: apt install nginx
* Enable it and start it: `sudo systemctl enable nginx`, `sudo systemctl start nginx`
* Open up your browser and head to <http://host/> — you should see a page there
* Edit files in /var/www/html/..., etc: try replacing index.html? with "hello world"
* check <http://host/> again

Here's a pretty cool, simple website. Let's try and deploy it:

{% @github-files/github-code-block url="<https://github.com/PaulineLabaisse/Motherfuckingwebsite/blob/master/Motherfuckerwebsite.html>" %}

### Running a cron job that sends us a message via telegram

One of the best things about a server is that it's meant to run 24/7, something that our laptops or desktops aren't great at doing. This means it's really good at running things constantly at a scheduled interval. To do such a thing, we'll use something known as cron jobs.

* Cron is a scheduling daemon that executes tasks at specified intervals.
* These tasks are called cron jobs and are mostly used to automate system maintenance or administration.

#### Cron jobs

The crontab command allows you to install, view , or open a crontab file for editing:

* `crontab -e` - Edit crontab file, or create one if it doesn’t already exist.
* `crontab -l` - Display crontab file contents.
* `crontab -r` - Remove your current crontab file.
* `crontab -i` - Remove your current crontab file with a prompt before removal.
* `crontab -u` - Edit other user crontab file. This option requires system administrator privileges.

#### Cron Syntax

* <https://crontab.guru/>

  ```
  Syntax:
  * * * * * command(s)
  - - - - -
  | | | | |
  | | | | ----- Day of week (0 - 7) (Sun=0 or 7)
  | | | ------- Month (1 - 12)
  | | --------- Day of month (1 - 31)
  | ----------- Hour (0 - 23)
  ------------- Minute (0 - 59)

  Example:
  */5 * * * * /path/to/script.sh # Run every 5 minutes
  ```

#### Creating a telegram bot with cronjob

We have an API that gives us the 24 hour weather forecast in Singapore. We want to:

* Check this forecast every morning
* If it's about to rain, send us an alert. For simplicity, lets send a telegram message!

Here's a bash script that sends a telegram message if it's going to rain that day:

<pre class="language-bash"><code class="lang-bash"><strong>#!/bin/bash
</strong>
# Replace these with your actual Telegram bot token and chat ID
TELEGRAM_BOT_TOKEN="YOUR_BOT_TOKEN"
CHAT_ID="YOUR_CHAT_ID"
TELEGRAM_API_URL="https://api.telegram.org/bot$TELEGRAM_BOT_TOKEN/sendMessage"

# Replace this with your actual curl command that fetches the JSON weather data
response=$(curl -s "https://api.data.gov.sg/v1/environment/24-hour-weather-forecast")

# Define an array of rain-related keywords
rain_keywords=("Rain" "Showers" "Thundery Showers" "Heavy Thundery Showers")

# Extract the general weather forecast without jq
general_forecast=$(echo "$response" | grep -o '"forecast":"[^"]*' | sed 's/"forecast":"//')

# Initialize a flag to check if rain is found
rain_found=0

# Check for rain-related keywords in the general forecast
for keyword in "${rain_keywords[@]}"; do
    if [[ "$general_forecast" == *"$keyword"* ]]; then
        rain_found=1
        break
    fi
done

# If rain is detected, send an alert
if [ "$rain_found" -eq 1 ]; then
    message="Weather Alert: Rain expected! General forecast is '$general_forecast'. Stay prepared!"
    
    # Send the message to the Telegram bot
    curl -s -X POST $TELEGRAM_API_URL \
        -d chat_id=$CHAT_ID \
        -d text="$message" > /dev/null

    echo "Alert sent: $message"
else
    echo "No rain expected. General forecast is '$general_forecast'."
fi

</code></pre>

* To get a telegram bot id, just go to @BotFather, and create a new bot, enter the token we receive inside
* To get your chat id, just go to @getmyid\_bot and copy your chat id there
* You'll need to start a chat with your bot first, go to your bot and do '/start' before you try running the script

Here are some more local APIs you can try to automate/script!

{% embed url="<https://github.com/jackveiga/singapore-apis>" %}

These should give you a very simple idea of hosting some services on our own servers. There's so much more we can't cover in a 2 hour workshop but here are some resources if you're fast/want to learn more!

### Syncing your files with Syncthing

Syncthing is an open-source file synchronization tool that allows you to effortlessly synchronize files across multiple devices. It runs on various platforms, providing a seamless experience for syncing files securely and privately over local networks or the internet.

{% embed url="<https://www.atlantic.net/dedicated-server-hosting/how-to-install-syncthing-on-ubuntu-22-04/>" %}

### Hosting your own VPN over your own server

There are plenty of reasons why you should [never use or trust public VPN providers](https://gist.github.com/joepie91/5a9909939e6ce7d09e29). We can set up our own to ensure that we can browse the internet securely or hide our IP addresses by routing our network traffic through our servers instead.

{% embed url="<https://www.cyberciti.biz/faq/ubuntu-20-04-set-up-wireguard-vpn-server/>" %}


# Monitoring

This was initially made for an older version of this workshop, and hence will not be used today, but it is still super relevant! Personally, I also use Beszel for monitoring

### Understanding your system

So now we want to be able to navigate, operate and monitor our system. To do this, most servers have a relatively homogenous system: systemd!

systemd is a software suite that provides an array of system components for Linux operating systems. The main aim is to unify service configuration and behavior across Linux distributions.

`systemctl` is the command-line tool that manages the systemd system and service manager in Linux.

* How to work with systemd units, etc
  * Many modern Linux distros go with systemd — it can handle services for you in a convenient manner
  * It comes with systemctl, along with a lot of other things — DNS caching resolvers, time sync, a bootloader, etc, But we'll want to focus on systemctl
  * Start/stop/restart services: systemctl start unit.service, etc
  * Check service status: systemctl status unit.service

`journalctl` is a utility for querying and displaying logs from journald, systemd’s logging service.

To see live logs:

```bash
journalctl -f
```

To see the first 20 lines

```bash
journalctl -n 20
```

To check a specific service:

```bash
journalctl -u sshd
```

To get all logs from last boot

```bash
journalctl -b
```

To filter by time (last 15 minutes for example)

```bash
journalctl --since "15 minutes ago"
```

### Monitoring with top

Aside from reading warnings, and making sure our services are running, we often times also want to make sure our system as a whole is running as expected. This is where `top` and `htop` come into play

#### What is top?

* top command is used to show the Linux processes. It provides a dynamic real-time view of the running system.
* Think of it as a super powerful task manager for Linux.

#### Basic Usage

* Just type `top` to start the program.
* Pressing q will simply exit the command mode.
* Pressing h will show you the help menu.

#### What does everything mean?

* PID: Shows task’s unique process id.
* PR: The process’s priority. The lower the number, the higher the priority.
* VIRT: Total virtual memory used by the task.
* USER: User name of owner of task.
* %CPU: Represents the CPU usage.
* TIME+: CPU Time, the same as ‘TIME’, but reflecting more granularity through hundredths of a second.
* SHR: Represents the Shared Memory size (kb) used by a task.
* NI: Represents a Nice Value of task. A Negative nice value implies higher priority, and positive Nice value means lower priority.
* %MEM: Shows the Memory usage of task.
* RES: How much physical RAM the process is using, measured in kilobytes.
* COMMAND: The name of the command that started the process.

#### `top` cheatsheet

{% embed url="<https://gist.github.com/ericandrewlewis/4983670c508b2f6b181703df43438c3>" %}

#### Some exercises!

* Try running `top` and see what you can find out about your system!
* What are the top 5 processes using the most CPU?
* What would I press if I want to kill the processes using the most memory?
* I want to see what processes start running when I start my computer. How would I do that?

<details>

<summary>Solution</summary>

* What are the top 5 processes using the most CPU?
  * `top` -> `Shift + P`
* What would I press if I want to kill the processes using the most memory?
  * `top -> Shift + M` -> `k`
* I want to see what processes start running when I start my computer.
  * `top` -> `f` -> `PID` -> `s` -> `q` -> `Shift + R`

</details>


# Vim

### Slides

You can get the slides for this workshop here:

{% embed url="<https://hckr.cc/ht2425s2-w4-slides>" %}

### Motivation

Writing code or editing files on a computer has a lot of moving parts: you spend a lot of time switching between files, navigating and editing code compared to writing a long stream of words sometimes.

As programmers, or just general power users, we will spend a lot of time doing these things, so it is extremely beneficial to invest some time into learning an editor.

### Learning a new editor

People tend to have extremely strong opinions on editors to learn. See:

{% embed url="<https://en.wikipedia.org/wiki/Editor_war>" %}

For purposes of this workshop, we'll be trying to learn vim, a command line editor. Feel free to bring your experience of learning a new editor to any other editor

### Introduction to vim

Within NUS, vim is the main editor you'll learn and use (you'll probably use it if you've done CS2030 within SOC). Vim is an editor with an extremely rich history. Vim is actually an acronym for VI iMproved, created by the late Bram Moolenaar in 1991, who unfortunately passed on in 2023.

{% embed url="<https://en.wikipedia.org/wiki/Bram_Moolenaar>" %}

As the name suggests, it was based of another text editor, vi, created by Bill Joy in 1976.

{% embed url="<https://en.wikipedia.org/wiki/Bill_Joy>" %}

Vim is built around a bunch of cool ideas, and a lot of tools support vim emulation. It is probably really useful to learn the neat ideas of vim even if you end up using a different editor for your day to day use.

### Origins of Vi(m)

Bill Joy was trying to create an editor that was usable with a 300 baud modem. To put this into context, this is approximately 0.3 kbits/second connection, and transmitting text was often much slower than the time it took to read it. This is why vim has somewhat *unintuitive* commands when you first start out. Because you could type only about one letter a second, the commands had to be **really, really short.**

As it turns out, if you build an editor with the purpose of **minimizing keystrokes,** you have a really efficient editor.

### Philosophy of Vim

The key idea behind vim is that **as a power user, you spend most of the time reading, not writing.** As such, vim is built to be a **modal editor:** it has different modes for inserting text and manipulating text. Vim itself is a programmable with Vimscript and other languages. As such, there is also a huge thriving plugin community around vim. Vim itself is also a programmable interface, keystrokes are commands and we can compose them to do complex actions.  While vim does have mouse support, to efficiently use vim, we very much avoid the use of the mouse, simply because it's too slow; Vim even avoids using the arrow keys because it would take too much time to move your hands to the arrow keys.

#### Summary

1. Vim is built to be as efficient as possible
2. Vim itself is a programmable interface: individual keystrokes become our commands and we can combine them to do some cool stuff.
3. Vim tends to avoid anything that requires the movement of the hand off the homerow of the keyboard. This means less emphasis on arrow keys and the mouse, even though they still work in vim.

### Using Vim

There are a few primary modes of vim:

* Normal Mode - For moving around a file and making small edits
* Insert Mode - For inserting text
* Visual Mode - For selecting blocks of text
* Command Mode - For entering commands

Keystrokes have very different meanings in different operating modes. For example, `x`in insert mode will just insert the 'x' character, but in normal mode it would delete a character.

In its default configuration, Vim shows the current mode in the bottom left. The initial/default mode is Normal mode. You’ll generally spend most of your time between Normal mode and Insert mode.

#### Opening vim and quitting vim

There's a common joke that if you give a web designer a computer with vim loaded up, and ask them to quit vim, you will get a random string generator. To open vim, just type `vim`in the terminal. To quit, just type `:q`. This might seem unintuitive at first, but we'll explain it as we go along.

If you're confused why vim is built this way, remember that the whole philosophy of vim is doing things with as little keystrokes as possible.

### Normal to Command mode

Command mode is where we run commands similar to a command line in vim. To go to this mode, simple press : . A text bar should appear at the bottom of the screen. From there, we can execute several vim commands.

* : - go to command mode&#x20;
* q (in command mode) - quits the file&#x20;
* w (in command mode) - saves the file&#x20;
* ! - force an action&#x20;
* :wq - save the file then quit&#x20;
* :q! - force quit file without saving&#x20;

Once you are done, vim should automatically put you back in normal mode

### Any mode to Normal mode

Normal mode is your default mode where you should spend most of your time. In vim, if you ever get lost or are not sure what is happening, always reset to normal mode Esc will bring you to normal mode from any of the other modes. You will be using this a lot.

#### But why escape?

It might seem quite counterintuitive to use escape since it’s quite out of place on your keyboard. However, vi was created using an ADM-3A terminal. It looks like this:

<figure><img src="/files/gFtHzyDwNRBIu7mI3ZUO" alt=""><figcaption></figcaption></figure>

Notice where the `Esc`key is?

You will use the `Esc`key a lot when using vim. Consider remapping your Caps Lock:

* <https://vim.fandom.com/wiki/Map_caps_lock_to_escape_in_macOS>
* <https://vim.fandom.com/wiki/Map_caps_lock_to_escape_in_Windows#Keyboard_Manager_(Microsoft_PowerToys)>
* If you're using Linux, you'll figure it out ;)

### Navigation in Normal mode

First let’s open up a file using vim by using vim (filename). We can navigate the file by using `hjkl` (left, down, up, right respectively) Why not arrow keys (or a mouse)? Historically, it’s because the old keyboards did not have arrow keys or a mouse. However, in practice, it is extremely efficient as you don’t need to move you hands away from the alphanumeric keys to do anything.

Here are more movement options in Normal mode:

| Type          | Description                                                                                                     |
| ------------- | --------------------------------------------------------------------------------------------------------------- |
| Basic         | `hjkl`: left, down, up, right                                                                                   |
| Word          | `w`: next word, `b`: back a word                                                                                |
| File          | `gg`: go to top of file, `G`: go to bottom of file                                                              |
| Line          | `0`: beginning of line, `$`: end of line, `^`: first non-whitespace of line                                     |
| Line Numbers  | `34G`: Go to line 34                                                                                            |
| Screen        | `H`: igh part of screen, `M`:iddle of screen, `L`:ow part of screen                                             |
| Braces        | `%`: go to corresponding braces                                                                                 |
| Repeating     | `10j`: to go down 10 times                                                                                      |
| Scroll        | `Ctrl + d`: scroll down, `Ctrl + u`: scroll up                                                                  |
| Find (inline) | `f`: to find further up the line, `F`: to find everything before the cursor, `,/;`: to navigate between results |
| Search (file) | `/ + query`: to search forward from the cursor, `?`to search backwards from the cursor                          |

### Practice

Open a text file using the command `vim file`. It could be any piece of text. Try navigating around the file with the above commands!

### Editing Text

Make sure you are in normal mode. `Esc`&#x20;

1. `i` to insert before cursor&#x20;
2. `I` to insert at the start of line&#x20;
3. `a` to insert after cursor
4. `A` to insert at the end of line&#x20;
5. `o` to start a next line and insert&#x20;
6. `O` to start a line above the current selection and insert&#x20;

Get out of insert once done. `Esc`

#### Deleting Text

In normal word, you can quickly delete a portion of text:

* `d` + *modifier*, deletes a certain portion based on the modifier
  * `dw` – delete word
  * `6dw` – delete 6 words
  * `dd` – delete the entire line
  * `d$` – delete till end of line
  * `dt` + *char* – delete till character

#### Making small edits

Similarly, **change** allows you to quickly delete and change a certain portion of text:

* `c` + *modifier* – deletes then puts you into insert mode
  * `cw` – change word
  * `7cw` – change 7 words
  * `c$` – change till end of line
  * `ct` + *char* – change till certain character

#### Copy and pasting

In normal mode, **yank** (`y`) copies text into a buffer (think of *Ctrl + C*). The following commands are variations:

* `yy` – yank the entire line
* `yw` – yank a single word
* `6yw` – yank 6 words
* `yt` + *char* – yank till (but not including) a certain character
* `p` – put/paste whatever is in the buffer (below the current line)
* `P` – put/paste whatever is in the buffer (above the current line)

#### Misc

* `x` – delete a certain character
* `r` – replace a character
* `.` – repeat the last action
* `u` – undo the previous action
* <kbd>Ctrl</kbd> + `r` – redo the last undone action

#### Counts

You can combine nouns and verbs with a count, which will perform a given action a number of times.

* `3w` move 3 words forward
* `5j` move 5 lines down
* `7dw` delete 7 words

#### Modifiers

You can use modifiers to change the meaning of a noun. Some modifiers are `i`, which means “inner” or “inside”, and `a`, which means “around”.

* `ci(` change the contents inside the current pair of parentheses
* `ci[` change the contents inside the current pair of square brackets
* `da'` delete a single-quoted string, including the surrounding single quotes

### Practice

Try and fix the typos and small erros here!

{% embed url="<https://hckr.cc/ht-vim-p1>" %}

### Visual Mode

There are a few kinds of visual modes:

* **Visual** – `v`
* **Visual Line** – `V`
* **Visual Block** – <kbd>Ctrl</kbd> + `v`

You can use these selections along with the commands covered above:

* **y** (yank)
* **d** (delete)
* **c** (change)

### Practice

It seems like the data from the first two lines are corrupted, lets remove them from our data!

{% embed url="<https://hckr.cc/ht-vim-p2>" %}

### Opening Files and other commands

Aside from saving and quitting, here are a few more important commands to know:

* `:enew` – opens a new file
* `:e <filepath>` – open the file at the specified path
* `:sp` – open a new horizontal split
* `:vsp` – open a new vertical split
* `:sort`- sort selected text

### Macros

Macros are one of the really powerful features in Vim that can significantly speed up your workflow:

* `q` + *register* to start recording a macro, then `q` again to stop recording
* `@<register>` to apply (play back) the macro

### Practice

Let’s use a macro to extract the names from these emails!

{% embed url="<https://hckr.cc/ht-vim-p3>" %}

### Extending Vim

There are tons of plugins for extending Vim. Contrary to outdated advice that you might find on the internet, you do *not* need to use a plugin manager for Vim (since Vim 8.0).

* <https://github.com/amix/vimrc&#x20>;
* <https://vimconfig.com&#x20>;
* <https://missing.csail.mit.edu/2020/files/vimrc>

### Modern Vim/Neovim Workflows

Without giving a long, non-exhaustive list of plugins, here are some really cools ones you should try out!

{% @github-files/github-code-block url="<https://github.com/preservim/nerdtree>" %}

{% @github-files/github-code-block url="<https://github.com/nvim-telescope/telescope.nvim>" %}

{% @github-files/github-code-block url="<https://github.com/itchyny/calendar.vim>" %}

These plugins not only try to speed up your vim workflows, but also add additional functionality to vim.

### Vim Emulation

Many tools support Vim emulation. The quality varies from good to great; depending on the tool, it may not support the fancier Vim features, but most cover the basics pretty well.

{% embed url="<https://chromewebstore.google.com/detail/vimium/dbepggeogbaibhgnhhndojpepiihcmeb?hl=en&pli=1>" %}

{% @github-files/github-code-block url="<https://github.com/VSCodeVim/Vim>" %}

### Help

Vim is a really powerful editor if you are able to master it.\
Don’t worry, there are plenty of resources!

* `:help <command>` – get the manual for a specific command
* `vimtutor` – built-in Vim tutor. Give it a read; it shouldn’t take more than 30 minutes
* **VimGolf** – a really good practice site: edit the file in the fewest keystrokes
* There are tons of Vim wikis, guides, and cheat sheets out there—just search for them!

### Resources

This workshop was loosely based of MIT's Missing Semester of your CS Education:

{% embed url="<https://missing.csail.mit.edu/2020/editors/>" %}


# Introduction to Zig


# Language Basics

The basics of the Zig language are quite straightforward. Given here are examples of each basic concept, which should be picked up and experimented upon.

{% hint style="info" %}
For code examples that don't define a `main`function, please define your own `main`function and paste the code inside in order to run it.
{% endhint %}

## Hello, world!

We have to import the standard library here using `@import("std")`, which we then store in the `std`variable. We can access functions (and types) from the standard library using the `.`syntax.

```zig
const std = @import("std");

pub fn main() !void {
    std.debug.print("Hello, world!\n", .{});
}

```

## Primitives

```zig
// ints
const my_32_bit_int: i32 = -42;
const my_64_bit_int: i64 = -323;
const my_32_bit_unsigned_int: u32 = 3424;
const my_64_bit_unsigned_int: u64 = 34;

// more ints ...?
const my_17_bit_int: i17 = 17;
const my_38_bit_unsigned_int: i38 = 38;

// floats
const my_32_bit_float: f32 = 3.14;
const my_64_bit_float: f64 = 3.14159;

// bool
const my_bool: bool = true;

// string
const my_string: []const u8 = "Hello, world!";

std.debug.print("32-bit int: {}\n", .{my_32_bit_int});
std.debug.print("64-bit int: {}\n", .{my_64_bit_int});
std.debug.print("32-bit unsigned int: {}\n", .{my_32_bit_unsigned_int});
std.debug.print("64-bit unsigned int: {}\n", .{my_64_bit_unsigned_int});
std.debug.print("17-bit int: {}\n", .{my_17_bit_int});
std.debug.print("38-bit unsigned int: {}\n", .{my_38_bit_unsigned_int});
std.debug.print("32-bit float: {}\n", .{my_32_bit_float});
std.debug.print("64-bit float: {}\n", .{my_64_bit_float});
std.debug.print("bool: {}\n", .{my_bool});
std.debug.print("string: {s}\n", .{my_string});
```

## Arrays, Pointers & Slices

### Arrays

Arrays in Zig have a fixed size (defined in the type of the array). There aren't many differences between Zig arrays and those found in C, C++, Java or Go besides syntax.

```zig
// Arrays have a fixed size.
var my_int_array = [5]i32{ 1, 2, 3, 4, 5 };

// You can use the `_` character to have the compiler infer the size.
const my_other_int_array = [_]i32{ 1, 2, 3, 4, 5 };

std.debug.print("my_int_array: {any}\n", .{my_int_array});
std.debug.print("my_other_int_array: {any}\n", .{my_other_int_array});

// `len` is the only field of an array.
std.debug.print("length of my_int_array: {any}\n", .{my_int_array.len});

// Access and modify items within an array using the `[]` syntax.
std.debug.print("my_int_array[1]: {}\n", .{my_int_array[1]});

// Arrays are copied by default.
const my_copied_int_array = my_int_array;
my_int_array[2] = 33;
std.debug.print("my_copied_int_array: {any}\n", .{my_copied_int_array});

// Arrays can have sentinel values.
const my_int_sentinel_array = [5:0]i32{ 1, 2, 3, 4, 5 };

std.debug.print("my_int_sentinel_array: {any}\n", .{my_int_sentinel_array});
std.debug.print("my_int_sentinel_array.len: {any}\n", .{my_int_sentinel_array.len});
std.debug.print("my_int_sentinel_array (with sentinel): {any}\n", .{@as([6]i32, @bitCast(my_int_sentinel_array))});
```

### Pointers

Zig defines two kind of pointers: pointers to a single value, and pointers to multiple values. This is a departure from C and C++, where a pointer to an array with 1000000 integer looks the same as a pointer to a single integer (`int*`).

```zig
// Pointers work similarly to C or C++.
var some_int: i32 = 42;

const some_int_pointer: *i32 = &some_int;

std.debug.print("some_int_pointer: {*}\n", .{some_int_pointer});

// Zig distinguishes between pointers to single values and pointers to multiple values (C-style arrays).
var some_int_array = [3]i32{ 1, 2, 3 };

const single_int_pointer: *i32 = &some_int_array[0];
const many_int_pointer: [*]i32 = &some_int_array;

std.debug.print("single_int_pointer: {*}\n", .{single_int_pointer});
std.debug.print("many_int_pointer: {*}\n", .{many_int_pointer});
std.debug.print("single_int_pointer == many_int_pointer: {}\n", .{@intFromPtr(single_int_pointer) == @intFromPtr(many_int_pointer)});

// Just like arrays, pointers to multiple values can have sentinel values.
var some_int_sentinel_array = [3:0]i32{ 1, 2, 3 };

const many_int_sentinel_pointer: [*:0]i32 = &some_int_sentinel_array;

std.debug.print("many_int_sentinel_pointer[3]: {}\n", .{many_int_sentinel_pointer[3]});
```

{% hint style="info" %}
Notice that the `[*:0]u8`type is perfect for representing strings in C, since they terminate in `\0`and we keep track of a pointer to their first character.
{% endhint %}

### Slices

```zig
var some_array = [_]i32{ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };

// Slices are a view into an array.
const some_slice: []i32 = some_array[2..6];
std.debug.print("some_slice: {any}\n", .{some_slice});

// Slices are represented by a pointer to the first element and a length.
std.debug.print("some_slice.ptr: {*}\n", .{some_slice.ptr});
std.debug.print("some_slice.len: {}\n", .{some_slice.len});
std.debug.print("some_slice.ptr == &some_array[2]: {}\n", .{@intFromPtr(some_slice.ptr) == @intFromPtr(&some_array[2])});

// Slices should be treated as pointers to arrays. Modifying the slice modifies the original array.
some_slice[2] = 33;
std.debug.print("some_slice: {any}\n", .{some_slice});
std.debug.print("some_array: {any}\n", .{some_array});

// Slices can be sliced further.
const some_subslice = some_slice[1..3];
std.debug.print("some_subslice: {any}\n", .{some_subslice});

// Just like arrays, slices can have sentinel values.
var some_sentinel_array = [10:0]i32{ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };

const some_sentinel_slice: [:0]i32 = some_sentinel_array[2..10];
std.debug.print("some_sentinel_slice: {any}\n", .{some_sentinel_slice});
std.debug.print("some_sentinel_slice[8]: {}\n", .{some_sentinel_slice[8]});
```

## Control Flow

### If/else

```zig
const x = 5;
if (x > 7) {
    std.debug.print("x is greater than 7!\n", .{});
} else {
    std.debug.print("x is smaller than or equal to 7...\n", .{});
}

// If-else can also be used as expressions rather than statements.
const y = if (x > 4) 10 else 20;
std.debug.print("y: {}\n", .{y});
```

### Switch

```zig
const x = 34;
switch (x) {
    1...5 => {
        std.debug.print("x is between 1 and 5!\n", .{});
    },
    6...10 => {
        std.debug.print("x is between 6 and 10!\n", .{});
    },
    else => {
        std.debug.print("x is not between 1 and 10...\n", .{});
    },
}

// Switch can also be used as an expression rather than a statement.
const y = switch (x) {
    1...5 => 10,
    6...10 => 20,
    else => 30,
};
std.debug.print("y: {}\n", .{y});
```

### While

```zig
var x: i32 = 0;
while (x < 32) {
    std.debug.print("x: {}\n", .{x});
    x += 1;
}

// You can also pass an expression to perform each iteration.
var y: i32 = 0;
while (y < 32) : (y += 1) {
    std.debug.print("y: {}\n", .{y});
}
```

### For

```zig
var some_array = [_]i32{ 1, 2, 3, 4, 5 };
for (some_array) |item| {
    std.debug.print("array item: {}\n", .{item});
}

// Slices work too!
const some_slice = some_array[1..4];
for (some_slice) |item| {
    std.debug.print("slice item: {}\n", .{item});
}

// You can also iterate over pointers to each element rather than the value.
for (some_slice) |*item| {
    std.debug.print("slice item pointer: {*}\n", .{item});
}

std.debug.print("{any}\n", .{some_slice});
```

## Structs

```zig
const std = @import("std");

const Point = struct {
    x: i32,
    y: i32,
};

const Rect = struct {
    top_left: Point,
    bottom_right: Point,

    // You can define methods in structs.
    fn area(self: Rect) i32 {
        return (self.bottom_right.x - self.top_left.x) * (self.bottom_right.y - self.top_left.y);
    }
};

pub fn main() !void {
    var point1 = Point{ .x = 32, .y = 32 };
    const point2 = Point{ .x = 99, .y = 44 };
    const rect = Rect{ .top_left = point1, .bottom_right = point2 };
    std.debug.print("point1: {any}\n", .{point1});
    std.debug.print("point2: {any}\n", .{point2});
    std.debug.print("rect: {any}\n", .{rect});

    // You can access struct members using `.`. Works for nesting too.
    std.debug.print("point1.x: {}\n", .{point1.x});
    std.debug.print("rect.bottom_right.y: {}\n", .{rect.bottom_right.y});

    // Methods are accessed in a similar way.
    std.debug.print("rect.area(): {}\n", .{rect.area()});

    // Structs are copied.
    point1.x = 99;
    std.debug.print("point1.x: {}\n", .{point1.x});
    std.debug.print("rect.top_left.x: {}\n", .{rect.top_left.x});
}

```

## Enums

```zig
const std = @import("std");

const Color = enum {
    red,
    green,
    blue,
    yellow,
    brown,
    // ...
};

// The integer representation of enums can be overrided.
const Operation = enum(u8) {
    add = 0,
    sub = 1,
    mul = 2,
    div = 3,
    rem = 4,
    shift_left = 5,
    shift_right = 6,

    // Enums can have methods too!
    fn name(self: Operation) []const u8 {
        switch (self) {
            .add => return "add",
            .sub => return "sub",
            .mul => return "mul",
            .div => return "div",
            .rem => return "rem",
            .shift_left => return "shift_left",
            .shift_right => return "shift_right",
        }
    }
};

pub fn main() !void {
    std.debug.print("red: {any}\n", .{Color.red});
    std.debug.print("blue: {any}\n", .{Color.blue});

    std.debug.print("mul: {any}\n", .{Operation.mul});
    std.debug.print("mul (tag value): {}\n", .{@intFromEnum(Operation.mul)});
    std.debug.print("shift_left: {any}\n", .{Operation.shift_left});
    std.debug.print("shift_left (tag value): {}\n", .{@intFromEnum(Operation.shift_left)});

    // You can also use enums in switch statements.
    const some_color = Color.red;
    switch (some_color) {
        .red => std.debug.print("some_color is red\n", .{}),
        .green => std.debug.print("some_color is green\n", .{}),
        .blue => std.debug.print("some_color is blue\n", .{}),
        .yellow => std.debug.print("some_color is yellow\n", .{}),
        .brown => std.debug.print("some_color is brown\n", .{}), // try removing this and compiling
    }
}
```

## Unions

Unions allow you to store one of their members at a time, instead of all at once like in a struct. Zig unions can be treated similarly to C unions, except that they do throw a runtime error if you access the incorrect member (at the cost of a larger runtime size due to storing extra info).

```zig
const std = @import("std");

// We could use an enum to represent a shape, but we can't store any
// shape data within it.
const ShapeEnum = enum {
    circle,
    rectangle,
    square,
};

// We can use a union instead. A union is like a struct, but it can only
// store one of its members at a time, rather than all at once.
const ShapeUnion = union {
    circle: struct { radius: f32 },
    rectangle: struct { width: f32, height: f32 },
    square: struct { size: f32 },
};

const TaggedShapeUnion = union(ShapeEnum) {
    circle: struct { radius: f32 },
    rectangle: struct { width: f32, height: f32 },
    square: struct { size: f32 },
};

// We can also use an automatic enum to tag the union.
const TaggedShapeUnionAutomatic = union(enum) {
    circle: struct { radius: f32 },
    rectangle: struct { width: f32, height: f32 },
    square: struct { size: f32 },
};

pub fn main() !void {
    // We can access members of a union directly.
    const some_rectangle = ShapeUnion{ .rectangle = .{ .width = 3.14, .height = 2.71 } };
    std.debug.print("some_rectangle has width {} and height {}\n", .{ some_rectangle.rectangle.width, some_rectangle.rectangle.height });

    // But what if we do the following???
    //                ------------ try to uncomment the code below -------------
    // std.debug.print("some_rectangle has radius {}\n", .{some_rectangle.circle.radius});

    // Notice how given a shape of type ShapeUnion, we don't know which kind
    // shape it is? We can't switch on it...
    //                ------------ try to uncomment the code below -------------
    // const some_shape = ShapeUnion{ .circle = .{ .radius = 3.14 } };
    // switch (some_shape) {
    //     .circle => std.debug.print("some_shape is a circle of radius {}\n", .{some_shape.circle.radius}),
    //     .rectangle => std.debug.print("some_shape is a rectangle of width {} and height {}\n", .{ some_shape.rectangle.width, some_shape.rectangle.height }),
    //     .square => std.debug.print("some_shape is a square of size {}\n", .{some_shape.square.size}),
    // }

    // We must "tag" the union with an enum to know which kind of shape it is.
    const some_shape = TaggedShapeUnion{ .circle = .{ .radius = 3.14 } };
    switch (some_shape) {
        .circle => std.debug.print("some_shape is a circle of radius {}\n", .{some_shape.circle.radius}),
        .rectangle => std.debug.print("some_shape is a rectangle of width {} and height {}\n", .{ some_shape.rectangle.width, some_shape.rectangle.height }),
        .square => std.debug.print("some_shape is a square of size {}\n", .{some_shape.square.size}),
    }
}
```

## Functions

```zig
const std = @import("std");

pub fn foo(x: i32, y: f32) f32 {
    std.debug.print("inside the foo function... x: {}, y: {}\n", .{ x, y });
    return @as(f32, @floatFromInt(x + 2)) * y;
}

pub fn main() !void {
    std.debug.print("foo(3, 5.34) returned {}\n", .{foo(3, 5.34)});
}
```

## Optionals

```zig
// Zig values can never be `null`, unless they are explicitly marked as optional.
const some_int: i32 = 34;
const some_optional_int: ?i32 = 34;
const some_optional_int_null: ?i32 = null;

std.debug.print("some_int: {}\n", .{some_int});
std.debug.print("some_optional_int: {any}\n", .{some_optional_int});
std.debug.print("some_optional_int_null: {any}\n", .{some_optional_int_null});

// You can check if an optional is null using an if-else, and unwrap it at the same time.
if (some_optional_int) |an_int| {
    std.debug.print("some_optional_int is not null: {}\n", .{an_int});
} else {
    std.debug.print("some_optional_int is null\n", .{});
}

// You can also unwrap it with a default value.
const an_int = some_optional_int orelse 0;
std.debug.print("an_int: {}\n", .{an_int});

// If you know an optional is definitely not null, you can unwrap it using `.?`.
std.debug.print("some_optional_int is definitely not null: {}\n", .{some_optional_int.?});

// But this will crash if you attempt to unwrap a null.
//                ------------ try to uncomment the code below -------------
// std.debug.print("some_optional_int_null is definitely not null: {}\n", .{some_optional_int_null.?});

// Optionals aren't for free. They take up more space.
std.debug.print("size of i32: {}\n", .{@sizeOf(i32)});
std.debug.print("size of ?i32: {}\n", .{@sizeOf(?i32)});

// However, they're free if the underlying value is a pointer! Any guesses why?
std.debug.print("size of *i32: {}\n", .{@sizeOf(*i32)});
std.debug.print("size of ?*i32: {}\n", .{@sizeOf(?*i32)});
```


# Error Handling

Did you see the `!void`return type of the `main`function from earlier? That is Zig's way of indicating that the `main`function could possibly return an error. Error handling is a feature that was designed carefully in Zig, and worth exploring before we go further.

## Defining an error

An error is defined in a similar way to an enum. You can also use `||`to combine different error types together. Here are some errors from the standard library.

```zig
// in std/mem/Allocator.zig
pub const Error = error{
    OutOfMemory,
};

// in std/io.zig
pub const NoEofError = ReadError || error{
    EndOfStream,
};

// in std/dynamic_library.zig
const ElfDynLibError = error{
    FileTooBig,
    NotElfFile,
    NotDynamicLibrary,
    MissingDynamicLinkingInformation,
    ElfStringSectionNotFound,
    ElfSymSectionNotFound,
    ElfHashTableNotFound,
} || posix.OpenError || posix.MMapError;
```

## Returning errors

After defining your error, you have to indicate that a function you've written can possibly return an error. This is done by placing the error name, then a `!`, then the actual return type.

In the following example, the function `myFunction`can *only* return `MyError`. There is no other possible error type. On the other hand, `myOtherFunction`could possibly return other error types — its error type is *inferred* from the function body.

```zig
const MyError = error{
    FooReason,
    BarReason,
    AnotherReason,
};

pub fn myFunction(x: i32) MyError!i32 {
    if (x < 43) {
        // Simply return the error if you encounter an error condition, instead
        // of returning the result.
        return MyError.FooReason;
    } else if (x > 43) {
        return MyError.BarReason;
    } else {
        return 33;
    }
}

// Here the error type is inferred, instead of being explicitly defined.
pub fn myOtherFunction(x: i32) !i32 {
    if (x < 99) {
        return MyError.AnotherReason;
    } else if (x > 99) {
        return MyError.BarReason;
    } else {
        return 22;
    }
}
```

{% hint style="info" %}
Zig also has an `anyerror`type, which represents the union of all the error types across the entire program. So pretty much *any error in the program*.
{% endhint %}

## Handling errors

You can either use `try`to propagate errors from functions that you call, or `catch`to handle the errors at the call site. In Zig, errors *must* be handled (eventually), otherwise you'll get a compiler **error**.

```zig
pub fn main() !void {
    // This will propagate the error up to the caller of this function.
    // In the case of the `main` function, it would end the program.
    const my_function_result = try myFunction(43);
    std.debug.print("The result of myFunction is: {}\n", .{my_function_result});

    // Instead of propagating the error, you can also handle it.
    const my_other_function_result = myOtherFunction(43) catch |err| {
        switch (err) {
            MyError.FooReason => std.debug.print("FooReason\n", .{}),
            MyError.BarReason => std.debug.print("BarReason\n", .{}),
            MyError.AnotherReason => std.debug.print("AnotherReason\n", .{}),
        }
        // Return early from the main function.
        return;
    };
    std.debug.print("The result of myOtherFunction is: {}\n", .{my_other_function_result});
}
```


# Memory Management

Memory management in Zig is handled *manually*, similar to C and C++. By default, variables are allocated on the stack. However, should we need some dynamic memory, we can use *allocators* to allocate memory on the heap for use.

Allocation is handled by the `std.mem.Allocator`struct, which defines methods to allocate and free memory based on some underlying allocation strategy. The Zig standard library provides several different allocators with different strategies.

## Allocators

Let's first go through a simple example using the general-purpose allocator.

### General-Purpose Allocator

The general purpose allocator in Zig can be used for most purposes. Here we use the `alloc`function of the allocator to allocate memory for 16 `u8`s (16 bytes of memory).

```zig
// We first create the general-purpose allocator.
var gpa = std.heap.GeneralPurposeAllocator(.{}){};

// Then we get the general `std.mem.Allocator` struct from it.
// This is what we'll call to (de)allocate memory.
const allocator = gpa.allocator();

// Let's allocate 16 bytes of memory.
const some_bytes: []u8 = try allocator.alloc(u8, 16);

// Maybe put a string into it.
std.mem.copyForwards(u8, some_bytes, "Hello, my world!");

// What's in the memory?
std.debug.print("{s}\n", .{some_bytes});

// Wait, don't we need to free the memory???
```

#### defer

Here's a brief digression to introduce the `defer`keyword. This keyword can be used to execute an expression at the *end* of the current scope. If there are multiple `defer`s in the same scope, they wil be executed in the reverse order from which they were introduced.

```zig
std.debug.print("normal 1\n", .{});
defer std.debug.print("defer 1\n", .{});
std.debug.print("normal 2\n", .{});
defer std.debug.print("defer 2\n", .{});
std.debug.print("normal 3\n", .{});
defer std.debug.print("defer 3\n", .{});
```

The following statements will produce the following console output.

```
normal 1
normal 2
normal 3
defer 3
defer 2
defer 1
```

#### Checking for leaks

The example we gave earlier wasn't complete. We didn't actually *free* the memory we allocated. Should the program have been more long-running (e.g., web server), we would have leaked memory. Luckily, the general-purpose allocator comes with a built-in way to check for leaks, which composes nicely with the `defer` keyword that we just learnt.

```zig
// We first create the general-purpose allocator.
var gpa = std.heap.GeneralPurposeAllocator(.{}){};
defer _ = gpa.deinit();

// Then we get the general `std.mem.Allocator` struct from it.
// This is what we'll call to (de)allocate memory.
const allocator = gpa.allocator();

// Let's allocate 16 bytes of memory.
const some_bytes: []u8 = try allocator.alloc(u8, 16);

// Maybe put a string into it.
std.mem.copyForwards(u8, some_bytes, "Hello, my world!");

// What's in the memory?
std.debug.print("{s}\n", .{some_bytes});

// LEAKKKKKKKK
```

Now, the program should crash indicating where the memory leak took place. To fix this, we can use `defer` once more to free the memory at the end of the scope.

```zig
// We first create the general-purpose allocator.
var gpa = std.heap.GeneralPurposeAllocator(.{}){};
defer _ = gpa.deinit();

// Then we get the general `std.mem.Allocator` struct from it.
// This is what we'll call to (de)allocate memory.
const allocator = gpa.allocator();

// Let's allocate 16 bytes of memory.
const some_bytes: []u8 = try allocator.alloc(u8, 16);
defer allocator.free(some_bytes);

// Maybe put a string into it.
std.mem.copyForwards(u8, some_bytes, "Hello, my world!");

// What's in the memory?
std.debug.print("{s}\n", .{some_bytes});

// Phew, no more leaks!
```

### Fixed Buffer Allocator

This allocator takes in a slice of bytes and performs allocations on it. The example should make this clear.

```zig
var buf: [16]u8 = undefined;

// We first create the fixed buffer allocator.
var fba = std.heap.FixedBufferAllocator.init(&buf);

// Then we get the general `std.mem.Allocator` struct from it.
// This is what we'll call to (de)allocate memory.
const allocator = fba.allocator();

// Let's allocate 16 bytes of memory.
const some_bytes: []u8 = try allocator.alloc(u8, 16);
defer allocator.free(some_bytes);

// Maybe put a string into it.
std.mem.copyForwards(u8, some_bytes, "Hello, my world!");

// What's in the memory?
std.debug.print("{s}\n", .{some_bytes});
```

### Arena Allocator

The arena allocator wraps an existing allocator, using it the perform allocations. However, it doesn't perform any frees, instead free-ing all the memory it allocated at once upon `deinit`.

### page\_allocator

This is the most basic allocator. When you make an allocation, it will ask the OS for an entire page of memory, which makes this extremely space inefficient, and also not performant.

## ArrayList

Let's explore memory management further by looking at a common data structure used in Zig programs: the humble `ArrayList`. This is Zig's implementation of a dynamically-sized array in the standard library.

```zig
var gpa = std.heap.GeneralPurposeAllocator(.{}){};
defer _ = gpa.deinit();

const allocator = gpa.allocator();

// Initialise the ArrayList, passing in the allocator it will use to dynamically
// allocate memory for its items.
var some_array_list = std.ArrayList(i32).init(allocator);
defer some_array_list.deinit(); // REMEMBER TO DEINIT WHAT YOU INIT!!

// Append some items to the list. Notice how we need to use `try` here, since the
// memory allocation can fail.
try some_array_list.append(3);
try some_array_list.append(8);
try some_array_list.append(4);
try some_array_list.append(39);

// Remove some items in the list. Notice how we don't allocate memory here, so we
// don't need to use `try`. But we need to assign the result to something.
_ = some_array_list.orderedRemove(1);

// Iterate through the array list.
for (some_array_list.items) |item| {
    std.debug.print("array list item: {}\n", .{item});
}
```

Notice how we pass `allocator` into the constructor of the `ArrayList`. It will store the allocator and use it whenever it needs to allocate memory internally. This is quite different from C, where the allocator is assumed to be a global construct (e.g., `malloc`and `free`). In this way, and since `std.mem.Allocator` represents any allocator, we can separate the concerns of *how to allocate memory* from *how to implement a dynamic list*.

However, storing the allocator means that the `ArrayList`struct takes up more space. There is another version called `ArrayListUnmanaged`that doesn't require passing the `allocator` in the constructor. Instead, you pass the `allocator` each time you need to allocate memory.

```zig
var gpa = std.heap.GeneralPurposeAllocator(.{}){};
defer _ = gpa.deinit();

const allocator = gpa.allocator();

// Initialise the ArrayListUnmanaged. Notice we don't need to pass any allocator here,
// instead we just need to pass it in `deinit`.
var some_array_list = std.ArrayListUnmanaged(i32){};
defer some_array_list.deinit(allocator); // REMEMBER TO DEINIT WHAT YOU INIT!!

// Append some items to the list. Notice how we need to pass the allocator here, and also
// use `try` here, since the memory allocation can fail.
try some_array_list.append(allocator, 3);
try some_array_list.append(allocator, 8);
try some_array_list.append(allocator, 4);
try some_array_list.append(allocator, 39);

// Remove some items in the list. Notice how we don't allocate memory here, so we
// don't need to use `try` or pass any allocator. But we need to assign the result
// to something.
_ = some_array_list.orderedRemove(1);

// Iterate through the array list.
for (some_array_list.items) |item| {
    std.debug.print("array list item: {}\n", .{item});
}
```

And this highlight a common choice in Zig, whether to store the allocator or pass it in each memory-allocating operation. There isn't a correct answer, and really depends on the use-case. Hash maps in Zig also follow a similar principle, with the standard library providing both managed and unmanaged versions.


# Working with C

One of the great features of Zig is that it is fully compatible with C. Let's return to the *Hello, world!* example, this time using C to print instead of Zig.

```zig
const c = @cImport({
    @cInclude("stdio.h");
});

pub fn main() !void {
    _ = c.printf("Hello, world!\n");
}
```

That's it! Using the `cImport` directive, we can directly use C code inside of Zig. Of course, please compile this to verify that it works as expected.

{% hint style="info" %}
For the rest of this page, we'll be working across multiple files. The filenames will be given in a comment that precedes the source code. Creating a new directory to store all these source files is encouraged, so that it doesn't get too messy.
{% endhint %}

## Custom C code

Let's write some of our own code instead of using the C standard library. Our custom C program will have just a single `greet` function, that says hello to the name passed as a parameter.&#x20;

```c
// In file /c-src/greeter.h

void greet(const char *name);

// In file /c-src/greeter.c

#include "greeter.h"
#include <stdio.h>

void greet(const char *name) {
  if (name == NULL) {
    printf("Hello, world!\n");
  } else {
    printf("Hello, %s!\n", name);
  }
}
```

Let's verify that this works as a C program first. We can create a temporary main file for the C program.

```c
// In file /c-src/main.c

#include "greeter.h"
#include <stdio.h>

int main() {
  greet("Alice");
  greet("Bob");
  greet(NULL);
  return 0;
}
```

And then compile it using `gcc` as with any other C program.

```bash
$ gcc -o greet c-src/greeter.c c-src/main.c  
$ ./greet
Hello, Alice!
Hello, Bob!
Hello, world!
```

It works! So how do we get `gcc` to work with Zig? Ah! We unveil the secret behind Zig's ability to handle C code so well. It *comes with a C compiler :O*

```bash
$ zig cc -o greet c-src/greeter.c c-src/main.c  
$ ./greet
Hello, Alice!
Hello, Bob!
Hello, world!
```

{% hint style="info" %}
Okay, if you're cheeky and try doing `zig cc --help`, you might discover that it's actually Clang (on a Macbook at least). But the creator of Zig actually [wrote about this](https://andrewkelley.me/post/zig-cc-powerful-drop-in-replacement-gcc-clang.html), and TL;DR it is Clang but smaller and does more out-of-the-box.
{% endhint %}

Insane! Let's continue by trying using this C code in Zig instead of just C. We can start by modifying our build file to look for `greeter.h` in the correct place.

```zig
// In file /build.zig

const std = @import("std");

pub fn build(b: *std.Build) void {
    const exe = b.addExecutable(.{
        .name = "greet",
        .root_source_file = b.path("src/main.zig"),
        .target = b.standardTargetOptions(.{}),
        .optimize = b.standardOptimizeOption(.{}),
    });
    exe.addIncludePath(b.path("c-src"));

    b.installArtifact(exe);

    const run_cmd = b.addRunArtifact(exe);
    run_cmd.step.dependOn(b.getInstallStep());

    const run_step = b.step("run", "Run the app");
    run_step.dependOn(&run_cmd.step);
}
```

Now, we can configure our `main.zig` file to import our custom `greeter.c`.

```zig
// In file /src/main.zig

const c = @cImport({
    @cInclude("greeter.c");
});

pub fn main() !void {
    c.greet("Alice");
    c.greet("Bob");
    c.greet(null);
}
```

We use the `zig build run` command instead of `zig run`, and we can see that everything works well!

```bash
$ zig build run
Hello, Alice!
Hello, Bob!
Hello, world!
```


# Exploring comptime

Let's create a `sum` function that takes in a slice of numbers (any type of number!!) and returns the sum of all the numbers! How can we do that?

{% hint style="info" %}
*heheh i see you Java/C++ programmers reaching for your < and > hehehe*
{% endhint %}

```zig
const std = @import("std");

fn sum(comptime T: type, values: []const T) T {
    var result: T = 0;
    for (values) |value| {
        result += value;
    }
    return result;
}

pub fn main() void {
    const some_i32s = [_]i32{ 1, 2, 3, 4, 5 };
    std.debug.print("sum of i32s: {}\n", .{sum(i32, &some_i32s)});

    const some_f32s = [_]f32{ 1.0, 2.0, 3.0, 4.0, 5.0 };
    std.debug.print("sum of f32s: {}\n", .{sum(f32, &some_f32s)});

    const some_u64s = [_]u64{ 1, 2, 3, 4, 5 };
    std.debug.print("sum of u64s: {}\n", .{sum(u64, &some_u64s)});
}
```

Woah, what is `comptime`?  And why is the type of `T` like `type` itself?

## What is comptime?

I'm sure many of us have heard of macros. They exist in languages like C, Rust or even Lisp (in quite a different form), and they serve as a way of executing some code at compile-time instead of runtime.

We might have also heard of generics in Java and Rust, or even templates in C++ (not sure if I'm committing a sin to lump these together), in order to write code that works across all types with certain constraints.

If you're a user of Go, you might've also used `go generate` to write repeated code for you.

Well, Zig has a solution that encompasses all three use-cases mentioned above, and that is comptime! What the comptime feature in Zig allows you to do, is simply write Zig code (not any other special language *ala C++ template metaprogramming*) that is executed at compile-time, instead of runtime!

## Pre-computing values

Looking at our earlier example, our `sum` function is pure, and can be pre-computed. Let's try to get Zig to precompute the results instead of computing the results at runtime.

{% hint style="info" %}
Of course, the Zig compiler might have also realised this and done the optimisation already, but for the sake of this being an example of pre-computing values, let's treat this as an optimisation!
{% endhint %}

```zig
const std = @import("std");

fn sum(comptime T: type, values: []const T) T {
    var result: T = 0;
    for (values) |value| {
        result += value;
    }
    return result;
}

pub fn main() void {
    const some_i32s = [_]i32{ 1, 2, 3, 4, 5 };
    std.debug.print("sum of i32s: {}\n", .{comptime sum(i32, &some_i32s)});

    const some_f32s = [_]f32{ 1.0, 2.0, 3.0, 4.0, 5.0 };
    std.debug.print("sum of f32s: {}\n", .{comptime sum(f32, &some_f32s)});

    const some_u64s = [_]u64{ 1, 2, 3, 4, 5 };
    std.debug.print("sum of u64s: {}\n", .{comptime sum(u64, &some_u64s)});
}
```

Wow! Do you notice what changed? By simply adding the `comptime` keyword in front of our call to `sum`, the function was called at compile-time instead of runtime.

## Generics

Our `sum` function is already a pretty good example of using comptime to create the effect of generics. Let's go one step further and create a data structure that supports generics.

```zig
const std = @import("std");

fn Vector2D(comptime T: type) type {
    return struct {
        x: T,
        y: T,
    };
}

pub fn main() void {
    const vector_of_i32s = Vector2D(i32){ .x = 1, .y = 2 };
    const vector_of_f32s = Vector2D(f32){ .x = 1.0, .y = 2.0 };

    std.debug.print("vector_of_i32s: {any}\n", .{vector_of_i32s});
    std.debug.print("vector_of_f32s: {any}\n", .{vector_of_f32s});
}
```

Notice how we can return `struct`s from functions in Zig! This definitely wouldn't work at runtime, since types don't have a representation at runtime (unless we use runtime reflection). So Zig is actually running the `Vector2D` function at comptime here, and treating the resulting structs as types to be constructed.

Remember that this is all Zig code, which means we can go one step further and make the length generic as well!

```zig
const std = @import("std");

fn Vector(comptime T: type, comptime len: usize) type {
    return struct {
        values: [len]T,
    };
}

const Vector_i32_2D = Vector(i32, 2);
const Vector_f32_4D = Vector(f32, 4);

pub fn main() void {
    const vector_of_2_i32s = Vector_i32_2D{ .values = [_]i32{ 1, 2 } };
    const vector_of_4_f32s = Vector_f32_4D{ .values = [_]f32{ 1.0, 2.0, 3.0, 4.0 } };

    std.debug.print("vector_of_2_i32s: {any}\n", .{vector_of_2_i32s});
    std.debug.print("vector_of_4_f32s: {any}\n", .{vector_of_4_f32s});
}
```

Here, both parameters to `Vector` play a part in defining what kind of `struct` the resulting type will be!


# CI/CD with Github Actions

## Introduction

Continuous Integration and Continuous Deployment (CI/CD) is the cornerstone to many modern day software projects. Automating the process the building, testing, and deploying code tightens the software release cycle, improving the software delivery times and increasing the reliability and consistency of software.

## Goals of this guide

In this workshop, we will be covering the following:

1. What is CI/CD?
2. What is Github Actions?
3. Anatomy of Github Actions
4. Implementing a CI/CD pipeline on an example application
5. Advanced use cases for Github Actions

As we discuss the types of workflows you can achieve with Github Actions, we will start unpacking the various concepts in Github Actions.

## Prerequisites

To start using Github Actions for CI/CD, ensure that the following are properly setup:

1. Create a Github account. You can refer to [this guide](https://docs.github.com/en/get-started/start-your-journey/creating-an-account-on-github).
2. Download Git to your local machine and set it up. You may refer to our installation guide [here](/orbital/git/setup).
3. Optionally, download the [`act` CLI](https://github.com/nektos/act)

This guide will assume that you have some basic understanding of what Github is and some of its core behavior such as repositories, issues, pull requests, etc. We will also be using Javascript as the basic language for all examples for its accessibility. If you have never used Javascript before, you can refer to [our guide](/orbital/readme-1) for a quick refresher. However, this guide does not require in-depth knowledge of Javascript as we will be focusing on writing Github Actions instead.


# Background

## What is CI/CD?

According to [Red Hat](https://www.redhat.com/en/topics/devops/what-is-ci-cd), CI/CD is defined as:

> [Continuous integration](https://www.redhat.com/en/topics/integration) (CI) refers to the practice of [**automatically**](https://www.redhat.com/en/topics/automation) **and frequently integrating code changes into a shared source code repository.** [**Continuous delivery**](https://www.redhat.com/en/topics/devops/what-is-continuous-delivery) **and/or deployment (CD) is a 2 part process that refers to the integration, testing, and delivery of code changes.** Continuous delivery stops short of automatic production deployment, while continuous deployment automatically releases the updates into the production environment.

<figure><img src="/files/CaOigRUQuVhAXmbH5r8w" alt=""><figcaption></figcaption></figure>

Essentially, after writing code, you can think of CI/CD as the pipeline in which brings your code from local to production in an (almost) automated fashion.

As seen in the diagram above, it is primarily responsible for the following:

1. Building your project
2. Running unit and (maybe) integration tests
3. Deployments to production

A CI/CD pipeline may not include every step. For instance, you might only want the CI/CD pipeline to run unit tests, or perform linting for a pull request. It is not a one-size-fits-all mechanism, but rather a "pick as you go" approach.

CI/CD pipelines are often built as part of the version control systems. This means that when you push your code onto a repository, the CI/CD pipeline will start.

Some common CI/CD software include:

1. TravisCI
2. CircleCI
3. Jenkins
4. Github Actions

## What is Github Actions?

One of the most common CI/CD tools is Github Actions due to Github's pervasiveness in personal, open-source, and commercial software.

{% @github-files/github-code-block url="<https://github.com/features/actions>" %}

Github Actions was first released in 2018, and it aims to be a tightly integrated CI/CD tool that works out-of-the-box with Github repositories.

{% @github-files/github-code-block url="<https://github.com/resources/whitepapers/actions>" %}

It is designed to integrate with existing Github flows, reducing the overhead involved in setting up a CI/CD pipeline.

Github Actions also goes beyond simple CI/CD pipelines as it can integrate with other Github events, such as running on a fork, issue created, or release.

The rest of this guide will cover the core syntax and concepts of Github Actions, common workflows you can achieve with Github Actions, and some other slightly unconventional workflows that you can achieve with Github Actions.


# Basics of Github Actions

The core of Github Actions are workflow files found in a repository. These workflow files are stored in the folder `.github/workflows` and are automatically read by Github.

Before we dive into the common workflows in Github Actions, let's first understand the high-level anatomy of Github Actions.

## Anatomy of Github Actions

**Workflows** are configurable automated processes designed to run when an **event** occurs. These [events](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows) may include things creating a pull request or when an issue is created. A single workflow may be triggered by different events, and it may have certain restrictions being placed on it (for example, a workflow triggered by a pull request will only run when the target branch is the `main` branch).&#x20;

Each workflow is comprised of one or more **jobs**. Jobs are essentially units of work within the workflow. Jobs can either run in sequential order or in parallel (if one depends on the other). Every job runs within its own **runner** which is a virtual machine or a container.

{% hint style="success" %}
You are able to pick between using a virtual machine to execute a job (default) or a [Docker container](https://docs.github.com/en/actions/sharing-automations/creating-actions/creating-a-docker-container-action). This guide will not focus on the nuances behind choosing one over the other.
{% endhint %}

Each job is comprised of several sequential **steps** that either execute some script defined or an **action**, which are reusable extensions.

<figure><img src="/files/kmojxiaZncmFn2hF8gXb" alt=""><figcaption></figcaption></figure>

Because steps execute within the same job (and thus the same runner), they can share data between one another through the shared virtual machine/container filesystem. However, because jobs run in different runners, they do not have direct access to the same virtual machine/container filesystem. There are other ways to share data between jobs that we will discuss in one of the common workflows.

{% hint style="warning" %}
The above content is taken from the official Github Actions documentation on the components of Github Actions.\
\
<https://docs.github.com/en/actions/about-github-actions/understanding-github-actions>
{% endhint %}

## Example application

To allow you to get a glimpse of what it is like working with Github Actions and setting up various pipelines in Github Actions, we have prepared a simple example React application for you.

### End goal

The goal of this guide would be for you to add various Github Actions workflows to this example application and extend off of it.

So, throughout a few hands-on activities, you will get the opportunity to build a common CI/CD pipeline to automatically test, lint, and deploy the application.

### Structure

It is a simple React app created with Vite, built using Typescript, and styled with Tailwind CSS. It is a very simple calculator application that allows you to add/subtract/divide/multiply two numbers — `x` and `y` — and display the output:

<figure><img src="/files/9RdTWoQjqg8TLYcILtJY" alt=""><figcaption></figcaption></figure>

The calculations are performed using a utility class `calculator.ts`  and there is a unit test suite `calculator.test.ts` that we have provided as well.

Everything else is not that relevant and you are free to gloss over them if you want.

### Setup

Fork the repository

{% @github-files/github-code-block url="<https://github.com/woojiahao/cicd-calculator>" %}

Clone the repository

```
git clone https://github.com/<your Github username>/cicd-calculator
```

(Optional) Run the repository

```
yarn
yarn dev
```

{% hint style="warning" %}
You do not need to run the project locally since we will be focusing on writing Github Actions workflows, which will not require running the project locally.
{% endhint %}

As you go through this section, we will be building on top of this existing project, adding Github Actions and exploring the concepts discussed above.

## (Optional) Setting up <kbd>act</kbd>

As you work through this section, you may want to test your Github Action workflows locally before pushing them to Github (to conserve the minutes). You may use the `act` tool to do so.

The installation instructions for `act` can be found here: <https://nektosact.com/installation/index.html>

{% hint style="danger" %}
There are several limitations to using `act` , such as not having direct access to an actual Github environment, and it does not also simulate/work for every event type. So use it just for understanding the basics of Github Actions.&#x20;
{% endhint %}


# Target workflow

It is very common for many Github workflows to involve:

1. (Optional) Creating a fork of a repository
2. Working on a feature/bug fix on a feature branch
3. Pushing the feature branch to Github
4. Creating a pull request of the feature branch to the main branch of the repository
5. Have some set of tests and automated checks start to verify the state of the pull request

We are going to replicate this workflow on the example application.

To avoid being overwhelmed with tasks, let's break down the "expected set of tests and automated checks" to be the following:

1. Run unit tests (`calculator.test.ts`)
2. Lint

Then, once the pull request is merged in `main`, we also want to (3) deploy it to Github Pages.


# Running unit tests

This section will tackle our very first task: running unit tests.

## Constructing workflows

The most fundamental questions that Github Actions workflows answers are:

1. When will it run?
2. What will it do?

"When will it run?" is answered by specifying the events that the workflow responds to. In our scenario, we want this workflow to run when a pull request is created on the repository.

"What will it do?" can be then broken down into several more guiding questions?

1. Which OS should this run in?
   1. Are there key differences between OSes that should be accounted for?
2. What programming language is this project written in?
3. What steps should be run to achieve the given workflow?
4. Can certain tasks be split into separate jobs and run in parallel?
5. What are jobs that depend on others?

We will take a look at how we can answer questions (4) and (5) in a following section. Let's first tackle questions (1), (2), and (3) in our scenario.

1. In the case of `calculator.test.ts`, the OS we run it on does/should not matter as there are no OS specific test cases. For simplicity, we will pick Ubuntu.
2. The project is written in React, so it depends on having Node.js available.
3. We first need to get the repository, install all necessary dependencies, and run the `test` script that is provided (see `package.json`)

{% hint style="success" %}
You might be asking yourself, "What if I have unit tests that are specific to an OS? Or version of Node.js? Or Ubuntu version?"\
\
Github Actions supports something it calls [matrix strategies](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/running-variations-of-jobs-in-a-workflow), that run a given workflow across a matrix of variables. We will briefly dive into it in the [Cookbook](/hackerschool/ci-cd-with-github-actions/cookbook)!
{% endhint %}

## Designing the workflow

Now that we have clearly outlined the key details of this workflow, let's get down to writing your very first workflow file. Remember, all Github Actions workflows must reside in `.github/workflows`, so create a new file called `ci.yml`!

{% hint style="info" %}
Github Actions uses a [YAML](https://www.redhat.com/en/topics/automation/what-is-yaml)-based specification language to define its workflows. It might seem a little strange at first, but it becomes more intuitive as you work with it.\
\
For a crash course on YAML syntax, refer to [this guide.](https://www.cloudbees.com/blog/yaml-tutorial-everything-you-need-get-started)
{% endhint %}

<pre class="language-yaml" data-full-width="false"><code class="lang-yaml"><strong># .github/workflows/ci.yml
</strong><strong>name: CI/CD Pipeline
</strong>on: [pull_request, workflow_dispatch]
jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Fetch repository
        uses: actions/checkout@v4
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'yarn'
      - name: Install dependencies
        run: |
          yarn
      - name: Run unit tests
        run: |
          NODE_ENV=production yarn test
</code></pre>

## Breaking it down

Let's break down each section and explain what we're doing.

### Workflow name

```yaml
name: CI/CD Pipeline
```

We are giving the workflow a name that Github Actions will display. If the name is omitted, GitHub displays the workflow file path relative to the root of the repository.

### Workflow triggers

```yaml
on: [pull_request, workflow_dispatch]
```

Then, we specify the times where this workflow runs, aka "when will it run?". They are specified with the `on` key and the values correspond to the [list of events that trigger a workflow](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows).&#x20;

{% hint style="info" %}
In this case, we specify the list of events as an array in YAML. However, there are instances where you might want to specify additional conditions on when the workflow may run (such as only executing when the pull request's target branch is the `main` branch). In that case, you will need to use the YAML dictionary definition. We will cover this in a bit.
{% endhint %}

In our case, we know that we want to run the workflow during a pull request, so we have included the `pull_request`  event. We have also included the `workflow_dispatch` event, so that we are able to manually trigger this workflow from Github without requiring a pull request (more about it [here](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#workflow_dispatch)). This is particularly useful if we want to verify that the workflow works without going through the hassle of creating a pull request.

### Defining jobs

```yaml
jobs:
  unit-tests:
```

We then start to specify the jobs that comprise the workflow. We give the job a job ID of `unit-tests`.

```yaml
    runs-on: ubuntu-latest
```

Recall that we said that every job runs in its own runner, which is a virtual machine by default. Therefore, we need to specify the OS that our `unit-tests` job will execute in, which we have decided earlier to be Ubuntu. As we are not particular about the version of Ubuntu we will use, we can use the `ubuntu-latest` , which is one of the [many available Github-hosted runners](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#choosing-github-hosted-runners).&#x20;

This means that we can think of every step of this job executing within an Ubuntu virtual machine (because they really do!). So we will be using `bash` commands, and we will have access to things like the [`apt` package manager](https://documentation.ubuntu.com/server/how-to/software/package-management/index.html) that is available in Ubuntu.

### Fetching the current repository

```yaml
    steps:
      - name: Fetch repository
        uses: actions/checkout@v4
```

We declare the steps of a job under the `steps` array as a list of dictionaries. As we are running the job in a virtual machine, we will basically have an empty machine at the start of the job, and it is our responsibility to start populating and interacting with this empty machine to create the intended workflow.&#x20;

The very first step we need to do is fetch the current repository, so that the virtual machine runner has access to the project structure, and more importantly, the unit tests. We give the step a human readable name (that is also displayed on Github) using the `name` key. If there is no name provided, Github will display the script or action that is being used.&#x20;

Here, we use an action — which are reusable extensions that perform some set of operations — called `actions/checkout@v4`. You can read more about what the action is comprised of [here](https://github.com/actions/checkout), but essentially, we are using it to perform a checkout of the current repository, retrieving all of its contents (from the latest commit) onto the virtual machine runner.

{% hint style="info" %}
The virtual machine runner has a root directory defined as `$GITHUB_WORKSPACE` . Without any other inputs for the `actions/checkout@v4` action, we default to fetching the contents of the repository directly into the `$GITHUB_WORKSPACE` directory.\
\
If you are wondering how you can verify this information, you can refer to the appendix's [debugging Github Actions](/hackerschool/ci-cd-with-github-actions/cookbook#debugging-github-actions) which covers how to do so!
{% endhint %}

### Setting up Node.js

```yaml
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'yarn'
```

Then, we need to setup Node.js on the virtual machine runner. We can use the action `actions/setup-node@v4` to setup Node.js automatically for us. You can read more about the action [here](https://github.com/actions/setup-node).

We can specify inputs for the action using the `with` key, providing the various inputs as a dictionary. In this case, we want to use Node.js version 20, and we want to use the `yarn` package manager — instead of `npm` — as that is what we have used for the example application.

### Installing dependencies

```yaml
      - name: Install dependencies
        run: |
          yarn
```

Before we can run the unit tests, we need to ensure that all of the necessary project dependencies are retrieved. This is where we can use scripts in steps.

Recall in [#defining-jobs](#defining-jobs "mention"), we mention that all steps in the job are effectively running on Ubuntu, so we will specify scripts that use Ubuntu's built-in shell: `bash`. You might want to specify a different shell for certain use cases, which [you can do so here](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsshell).

So, we declare our script through the `run` string. The pipe operator after the `run` key indicates that we are specifying a multi-line string in YAML.

The script we will run this time is `yarn`, which effectively installs the project's dependencies. You can think of this as running `yarn` directly in an Ubuntu terminal, where the current folder (`$GITHUB_WORKSPACE`) is the root of the Node.js project.

{% hint style="info" %}
Again, if you are wondering how you can verify this information, you can refer to the cookbook's recipe on [debugging Github Actions](/hackerschool/ci-cd-with-github-actions/cookbook#debugging-github-actions) which covers how to do so!
{% endhint %}

### Running unit tests

```yaml
      - name: Run unit tests
        run: |
          NODE_ENV=production yarn test
```

Finally, we can start to execute the unit tests of the project. Again, we use a script, but this time, the script will be `NODE_ENV=production yarn test` which effectively sets the environment variable `NODE_ENV` to be of value `production`.

`test` is a script that we have provided in the example application, and it essentially runs `vitest` , executing `calculator.test.ts`.

Voilà! You have successfully written your very first Github Actions workflow! Simple isn't it? Let's recap what we did.

## Visualizing the workflow

<figure><img src="/files/pO2UjWTObPtw4D0cDlvj" alt=""><figcaption></figcaption></figure>

At a glance, this is the high-level overview of the new CI/CD pipeline you have written.

To properly visualize and understand how the filesystem of the virtual machine runner changes throughout the job, we have also created this visualization (bolded text are the changes between steps):

<figure><img src="/files/ZVGzN0MOw93u5YFxDrPx" alt=""><figcaption></figcaption></figure>

## Verifying the workflow

### Manually running workflows

Once you have added the workflow, you need to commit and push it!

```
git add .github/workflows/ci.yml
git commit -m "Add CI workflow"
git push -u origin main
```

Then, we can start to verify that the Github Action works as intended! This is where the `workflow_dispatch` event comes in handy, where we are able to manually trigger Github Actions. If you navigate to your fork of the example repository, you can visit the Actions tab. You will see the following:

<figure><img src="/files/C4LrucASgWKBPMP39CdN" alt=""><figcaption></figcaption></figure>

It lists the workflows available, and what has run/are running. We are interested in our new pipeline `CI/CD Pipeline`, so select it and you should see the following:

<figure><img src="/files/FGWNkoI6G3LtXRYBE2qD" alt=""><figcaption></figcaption></figure>

It looks almost the same, but this time, there is a banner that tells you that "This workflow has a `workflow_dispatch` event trigger". Then, there is a dropdown to "Run workflow", select it and stick with `main` and click the "Run workflow" button:

<figure><img src="/files/7NPUbBny19irAm5siL85" alt=""><figcaption></figcaption></figure>

Refresh the page, and you will now see a new entry in the table:

<figure><img src="/files/KzTBzb18GF8BVJ2O0UGZ" alt=""><figcaption></figcaption></figure>

Give it a few seconds and then click into the action. You will see that the unit tests have failed:

<figure><img src="/files/MQqPC5PwBuxInGYGRQxF" alt=""><figcaption></figcaption></figure>

This is because we have intentionally made one of the unit tests to fail (divide ½ is not 0.4!). Let's try to fix this unit test while exploring the the `pull_request` event!

### <kbd>pull\_request</kbd> event

{% hint style="warning" %}
The following steps expects some level of understanding of Git. You can refer to our [Git guide for more information](/orbital/git)!
{% endhint %}

This guide is not a software engineering exercise, so we will tell you exactly where the error is and we will focus on examplifying the `pull_request` event.

We had intentionally set one of the unit test assertions to be incorrect, specifically along [these lines](https://github.com/woojiahao/cicd-calculator/blob/main/src/calculator.test.ts#L12-L14):

```javascript
test('divide 1 / 2 to equal 0.5', () => {
  expect(divide(1, 2)).toBe(0.4);
});
```

We have set the expected value to be `0.4` when it should clearly be `0.5` ! Let's fix this as a pull request to your own repository to see the `pull_request` event in action.

Create a new branch, called `fix-unit-test`:

```
git checkout -b fix-unit-test
```

Then, go to the file `calculator.test.ts` and fix line 13 to be the following:

```javascript
  expect(divide(1, 2)).toBe(0.5);
```

Then, add the file and create a commit. The commit message can be anything you want:

```
git add src/calculator.test.ts && git commit -m "Fix unit test"
```

Finally, push the branch to your fork:

```
git push -u origin fix-unit-test
```

Then, go to your fork on Github and create a new pull request. Pull request > New pull request. Give the pull request any title and you can leave the description blank.&#x20;

{% hint style="warning" %}
The base branch should be YOUR `main` branch, not the original repository's branch!
{% endhint %}

<figure><img src="/files/4iHQto98ag9lwQjl8xCa" alt=""><figcaption></figcaption></figure>

Create the pull request and wait a while. You will notice that the component towards the bottom changes to this:

<figure><img src="/files/eEEAyntC5vjn3cmqVCUg" alt=""><figcaption></figcaption></figure>

This is how you know that your workflow is running and it was triggered by the `pull_request`!

Now, if you select the CI/CD pipeline running, you will be brought back to the same page as earlier, instead, this time, you will notice that the workflow passes!

<figure><img src="/files/crV07X17epuaUAZnFUWw" alt=""><figcaption></figcaption></figure>

In fact, you will even see the individual steps of the job `unit-tests` that you defined! Wonderful! You have successfully:

1. Created a new Github Actions workflow
2. Observed how a failing unit test might look like
3. Fixed and verified that the `pull_request` event is working

Go ahead and merge the pull request into `main` and update your local repository to receive the latest changes:

```
git checkout main
git fetch origin main && git merge origin/main
```

Now that we have implemented the very first step, let's take a look at implementing step 2: linting!


# Linting code

With your new workflow `ci.yml`, you are able to run unit tests. But another key operation in most CI/CD workflows is linting the project, ensuring that the code follows a certain standard and set of conventions.

Using Github Actions, we want a pull request to fail if the branch contains poorly linted code.&#x20;

{% hint style="info" %}
Given that we have already demonstrated how `pull_request` events trigger the workflow and have verified that our workflow works correctly with unit testing, we will be demonstrating the aforementioned using the `workflow_dispatch` event instead for the sake of simplicity.
{% endhint %}

## Constructing the workflow

As described in the previous section, we think of answering two key questions when constructing the workflow:

1. "When will it run?" — established to be during `pull_request` (and additionally `workflow_dispatch` for testing)
2. "What will it do?" — execute the `yarn lint` script given in the `package.json`

However, there is an additional question we will want to answer, given that we already have an existing workflow:

1. "Is this going to be a separate workflow? A separate job in the same workflow? Or just another step in the existing job?"

There is no right or wrong answer for the above. But it is worth considering the following factors:

* Is this a part of the CI workflow? — yes, so we might not want to separate it out
* Is the task a part of unit testing? — no, so we might want to split it out to avoid cluttering a single job

So in this case, we choose to create a separate job within the same workflow `ci.yml`. By default, jobs will run in parallel, but can be designed to run sequentially. So, we get the added benefit of having both linting and unit tests running in parallel, saving time (arguable since we need to reinstall the project dependencies in each job, but as jobs get more complex, running them in parallel will allow simpler ones to complete first), and preventing the results of one job from affecting the other (one can fail while the others pass).&#x20;

{% hint style="success" %}
Before we dive into the code required, maybe take some time to think about and attempt to implement the above job! It is not very different from the previous implementation!
{% endhint %}

## Designing the workflow

```yaml
# .github/workflows/ci.yml
name: CI/CD Pipeline
on: [pull_request, workflow_dispatch]
jobs:
  linting:
    runs-on: ubuntu-latest
    steps:
      - name: Fetch repository
        uses: actions/checkout@v4
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'yarn'
      - name: Install dependencies
        run: |
          yarn
      - name: Lint code
        run: |
          yarn lint

  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Fetch repository
        uses: actions/checkout@v4
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'yarn'
      - name: Install dependencies
        run: |
          yarn
      - name: Run unit tests
        run: |
          NODE_ENV=production yarn test
```

## Breaking it down

You will notice that every step except the last is the same as the `unit-tests`  job. That is because the initial setup of the virtual machine runner does not change! We still need to&#x20;

1. Fetch the repository
2. Setup Node.js
3. Install project dependencies

And this is all because all jobs run in separate virtual machine runners! So `linting` does not share these steps with `unit-tests` .

### Linting code

```yaml
      - name: Lint code
        run: |
          yarn lint
```

The only step that differs between `linting` and `unit-tests` is the linting step, which we rely on the provided `lint` script in `package.json`, which calls `eslint .`.

## Visualizing the workflow

<figure><img src="/files/F9uxNDdt64c5izY2ozUE" alt=""><figcaption></figcaption></figure>

Now, the single workflow has evolved to include two parallel jobs!

## Verifying the workflow

As mentioned at the start of this section, we will be verifying that the linting works by using the `workflow_dispatch` event. So, once again, push the latest changes to `ci.yml` to your fork and manually run the workflow:

```
git add .github/workflows/ci.yml
git commit -m "Add linting step"
git push -u origin main
```

This time, you will see that there are now two separate jobs running within the same workflow:

<figure><img src="/files/LT9uUnVf84CKEho1OyxC" alt=""><figcaption></figcaption></figure>

Both of them will also complete at around the same time since both linting and unit tests are relatively small at this time:

<figure><img src="/files/9V3TAHavrxy73dXhg28G" alt=""><figcaption></figcaption></figure>

Try playing around with this new job. Create a branch and purposely commit and PR poorly linted code and see what happens! The `linting` job should fail while the `unit-tests` job will continue to work.

<details>

<summary>Poorly linted code</summary>

Add the following to the top of the `App.tsx` file:

```javascript
  if (true) {
    useEffect(() => console.log("hi"), [])
  }
```

</details>

<figure><img src="/files/t3PglO4FNAoc88hIwRq7" alt=""><figcaption></figcaption></figure>

Amazing! We have not only setup a CI/CD workflow that runs unit tests, but also linting, and both run in parallel and don't affect each other's outcomes!

Let's tackle the the final piece of the puzzle: deploying the application to Github Pages!


# Deploying to Github Pages

The final step of our [Target workflow](/hackerschool/ci-cd-with-github-actions/basics-of-github-actions/target-workflow) is automatically deploying our application to Github Pages when it is merged into `main`. This is the CD of CI/CD!

However, as Github Actions does not express "merging into `main`" as an event, we will instead be thinking in terms of deploying when changes are pushed to `main` instead.

{% hint style="info" %}
Github Pages is a free static site hosting option that integrates directly with Github repositories. You can read more about them [here](https://pages.github.com/).
{% endhint %}

## Constructing the workflow

We first go through the same questions as [Linting code](/hackerschool/ci-cd-with-github-actions/basics-of-github-actions/linting-code):

1. "When will it run?" — when changes are pushed to `main`&#x20;
2. "What will it do?" — compile and build the React project and publish the generated build files to Github Pages
3. "Is this going to be a separate workflow? A separate job in the same workflow? Or just another step in the existing job?" — this will be a separate workflow because (a) the events that trigger it are different from `ci.yml`, and (b) it is not logically a part of the `ci` workflow

If we look at our answer for (2), you may notice that we are essentially describing two separate tasks:

1. Compiling and building the React project
2. Publishing the generated build files to Github Pages

While we can represent them as a single job, we would like to explore what it's like to design jobs that are dependent on one another and passing around artifacts in Github Actions.

We also realize that (3) reveals that we are no longer treating `ci.yml` as the full CI/CD pipeline, so you are free to rename the workflow!

## Designing the workflow

```yaml
# .github/workflows/cd.yml
name: Deploying to Github Pages
on:
  push:
    branches: [main]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Fetch repository
        uses: actions/checkout@v4
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'yarn'
      - name: Install dependencies
        run: |
          yarn
      - name: Building
        run: |
          NODE_ENV=production yarn build
      - name: Uploading production artifacts
        uses: actions/upload-pages-artifact@v3
        with:
          name: github-pages
          path: dist

  deploy:
    needs: build
    runs-on: ubuntu-latest
    permissions:
      pages: write
      id-token: write
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    steps:
      - name: Publishing production artifact
        id: deployment
        uses: actions/deploy-pages@v4
        with:
          artifact_name: github-pages
```

## Breaking it down

What a mouthful! That's quite a lot of new steps and concepts. Fret not, we will be explaining each step individually.

### Conditional workflow trigger

```yaml
on:
  push:
    branches: [main]
```

Recall in [Running unit tests](/hackerschool/ci-cd-with-github-actions/basics-of-github-actions/running-unit-tests#workflow-triggers) previously, we mentioned that the events may be dictionaries instead when there may be more properties/conditions. In our scenario, we only want the workflow to deploy to Github Pages when we push to `main`. So, we can express this using the `push` event, and specify that it should only run when one of the `branches` (which includes `main`) is pushed to.

### <kbd>build</kbd> job

```yaml
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Fetch repository
        uses: actions/checkout@v4
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'yarn'
      - name: Install dependencies
        run: |
          yarn
      - name: Building
        run: |
          NODE_ENV=production yarn build
      - name: Uploading production artifacts
        uses: actions/upload-pages-artifact@v3
        with:
          name: github-pages
          path: dist
```

Similar to the jobs written in [Running unit tests](/hackerschool/ci-cd-with-github-actions/basics-of-github-actions/running-unit-tests) and [Linting code](/hackerschool/ci-cd-with-github-actions/basics-of-github-actions/linting-code), we will declare a job to

1. Fetch the repository
2. Setup Node.js
3. Install project dependencies
4. Build the production distribution

Doing so, we should now have a `dist/` folder in our virtual machine runner for the `build` job.

Let's talk more about the final step of the `build` job.

### Publishing artifacts

```yaml
      - name: Uploading production artifacts
        uses: actions/upload-pages-artifact@v3
        with:
          name: github-pages
          path: dist
```

**Artifacts** are files or collections of files produced during a workflow run. These artifacts are stored on Github. You may wish to use artifacts for things like storing build logs, test results, binary or compressed files, etc.&#x20;

They are also a way to share data between jobs in a workflow. Recall in [Basics of Github Actions](/hackerschool/ci-cd-with-github-actions/basics-of-github-actions#anatomy-of-github-actions) where we mentioned that steps in the same job share the same filesystem as they belong to the same job's virtual machine runner, but steps in different jobs do not share the same filesystem as they have completely separate filesystems, belonging to separate virtual machine runners. Artifacts are the way to bridge this gap.

For our use case, we want to publish the generated `dist/` folder from the previous step in `build` as an artifact, so that our next job, `deploy` has access to the files and can publish them accordingly.

Thankfully, there is an existing action `actions/upload-pages-artifact@v3` that handles this process, as we specify the name of the artifact generated `github-pages` and the path to the directory containing the static assets, i.e. `dist/`.&#x20;

So, after `build` runs, we would have an artifact called `github-pages` uploaded to Github and accessible to subsequent jobs. You can read more about artifacts on the [Github Actions documentation](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/storing-and-sharing-data-from-a-workflow).

### <kbd>deploy</kbd> job

```yaml
  deploy:
    needs: build
    runs-on: ubuntu-latest
```

We then declare a new job, `deploy` that `needs` the `build` job. This is how we construct the dependency graph between jobs, requiring one to complete before the other can execute.

### Permissions & secrets

```yaml
    permissions:
      pages: write
      id-token: write
```

In order to ensure that we can successfully publish to Github Pages, we also need to modify the default permissions of `GITHUB_TOKEN`.

`GITHUB_TOKEN` is a [special access token](https://dev.to/github/the-githubtoken-in-github-actions-how-it-works-change-permissions-customizations-3cgp) that is automatically created as a secret in all workflows. It has access to the current repository, and it expires after the workflow completes. [Secrets](https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions) are a way of storing sensitive information in an organization, repository, or repository environment.

{% hint style="info" %}
We talk about how secrets are stored and used in the [Github script](/hackerschool/ci-cd-with-github-actions/advanced-use-cases/github-script) section!
{% endhint %}

Essentially, `GITHUB_TOKEN` allows steps in the job to have some access to the current repository. So, in order for the job to publish to Github Actions, we want to grant the token `write` access to both `pages` and `id-token` . More information about the various permissions of `GITHUB_TOKEN` can be found [here](https://docs.github.com/en/actions/security-for-github-actions/security-guides/automatic-token-authentication#permissions-for-the-github_token).

### Environments

```
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
```

An **environment** in Github refers to a general deployment target that can be configured with protection rules and secrets. Essentially, they allow you to handle different stages of your project, like `development`, `production`, and in our case, `github-pages`.

These environments are displayed on the repository page.

{% hint style="info" %}
If you are interested to learn more about Github environments, read this [guide](https://runs-on.com/github-actions/environments/)!
{% endhint %}

For our scenario, we want to set the `url` of the environment to point to the output (`page_url`) of one of the job steps with step id `deployment`.&#x20;

### Deploying the artifact

```yaml
    steps:
      - name: Publishing production artifact
        id: deployment
        uses: actions/deploy-pages@v4
        with:
          artifact_name: github-pages
```

Finally, we can start deploying the artifact we published earlier. We use the action `actions/deploy-pages@v4` , targeting the artifact named `github-pages`, which we named earlier.

Notice that we also give an additional `id` to the step, `deployment`. This allows the step to be accessible via `${{ steps.deployment }}` and allows the outputs of the action to be accessible to the environment (seen above). `${{ ... }}` is a way of declaring expressions in a workflow file. More information about expressions in Github Actions can be found [here](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/evaluate-expressions-in-workflows-and-actions).

## Visualizing the workflow

<figure><img src="/files/nETi4L1MQqBTkvZfEAjI" alt=""><figcaption></figcaption></figure>

We now have a workflow with sequential jobs, with `build` generating the production build as an artifact, and `deploy` consuming that artifact and publishing it to Github Pages.

We can also visualize the process of uploading an artifact as such:

<figure><img src="/files/HvGboHNtzrTlerPE8IBY" alt=""><figcaption></figcaption></figure>

## Verifying the workflow

As per usual, add `cd.yml`, create a commit, and push to `main`.&#x20;

```
git add .github/workflows/cd.yml
git commit -m "Add CD workflow"
git push
```

This should already trigger the workflow to run. However, if you navigate to Actions and select the "Deploying the Github Pages" workflow, you will notice that it fails:

<figure><img src="/files/LUGUeqKmhazT9366cm2B" alt=""><figcaption></figcaption></figure>

If you select the specific workflow run, go to the `deploy` job, and select the "Publishing production artifact" step, you will see the following error:

<figure><img src="/files/SGb9gsBGRVZrbeeSs1aX" alt=""><figcaption></figcaption></figure>

The last message tells us what went wrong:

{% code overflow="wrap" %}

```
Error: Failed to create deployment (status: 404) with build version 6dbc2327e55394cd2690908b1b23d14eddb4a3cb. Request ID 1481:16FC0C:39A4ED5:737FC5E:67EA91C2 Ensure GitHub Pages has been enabled: https://github.com/woojiahao-git-mastery/cicd-calculator/settings/pages
```

{% endcode %}

So, visit the URL provided (it is different from the one above!) and select "Github Actions" instead:

<figure><img src="/files/vPlnCmA13VmTEetrvFGO" alt=""><figcaption></figcaption></figure>

Finally, to re-run the workflow, go back to Actions and return to the failed workflow run. At the top right corner, you should see a button to "Re-run worflows", choose to re-run all jobs:

<figure><img src="/files/CDYxkh50WSdBvt0R5u9z" alt=""><figcaption></figcaption></figure>

This time, you should see the following:

<figure><img src="/files/aNf0lTTwovcrY5IToT3K" alt=""><figcaption></figcaption></figure>

You can select the URL in the `deploy` job component and you should be greeted with the following UI:

<figure><img src="/files/sXptFXHgvwTKOpPm7gq0" alt=""><figcaption></figcaption></figure>

:confetti\_ball: Congratulations! You have successfully setup a traditional CI/CD pipeline using Github Actions! Play around with the workflows we have created!

Next up: we will be exploring some unique workflows in Github Actions!


# Advanced use cases

After going through the guided tour from [Basics of Github Actions](/hackerschool/ci-cd-with-github-actions/basics-of-github-actions), you should have a general grasp of how Github Actions works and how workflows are constructed.

Now, you might be wondering how Github Actions can be used beyond the basics of building CI/CD pipelines.

In this section, we will focus on discussing several advanced use cases of Github Actions. This is by no means an exhaustive list of what Github Actions can achieve, but it hopes to broaden how you think of Github Actions and use Github Actions.

Rather than having a central example application like [Basics of Github Actions](/hackerschool/ci-cd-with-github-actions/basics-of-github-actions), we will be presenting each use case as individual examples and linked to some live repository to better illustrate how it has been used. We trust that you have the understanding and mental models developed to combine these use cases with what we have previously covered to create infinitely many powerful workflows!

We will cover the following use cases:

1. Creating pollers with Github Actions
2. Using Github script inside workflows
3. Executing third-party scripts
4. Reusing workflows


# Pollers

There may be times where you wish to have a workflow run at a fixed duration. For instance, using a workflow to fetch and update a set of data everyday. Github Actions supports such workflows by offering the `schedule` event type that triggers a workflow.

To declare such a workflow, use the `schedule` event type along with the `cron` key, specify a [cron schedule format](https://crontab.guru/):

```yaml
on:
  schedule:
    # * is a special character in YAML so you have to quote this string
    - cron:  '30 5,17 * * *'
```

From the official Github Actions documentation on the [`schedule` event](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#schedule), you would specify the cron schedule and this will cause the workflow to be triggered at the given timing. You could even schedule it multiple times a day or across different times.

```yaml
on:
  schedule:
    - cron: '30 5 * * 1,3'
    - cron: '30 5 * * 2,4'

jobs:
  test_schedule:
    runs-on: ubuntu-latest
    steps:
      - name: Not on Monday or Wednesday
        if: github.event.schedule != '30 5 * * 1,3'
        run: echo "This step will be skipped on Monday and Wednesday"
      - name: Every time
        run: echo "This step will always run"
```

Once again, taken from the official Github Actions documentation.

You can use this "poller" pattern in conjunction with some of the next use cases to really power up your workflows. We will discuss them as we go.

{% hint style="danger" %}
There are several restrictions to this event type:

1. There may be delays to when exactly the workflow runs due to an increase in workload
2. This only works for workflows located on the [default branch](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-branches-in-your-repository/changing-the-default-branch) (this may change, so it's not always `main`)
3. These workflows can only run on the default branch
4. These workflows are disabled in repositories with no activity in 60 days
   {% endhint %}


# Github script

You might also want to interact with the Github API during your workflows. Some of the common use cases we've noticed include:

1. Fetching information about the repository/pull request/user
2. Creating issues
3. Creating issue/pull request comments
4. Updating issues/pull requests
5. Retrieving information about another repository
6. Automatically running jobs and creating commit

You can use the Github API via Github script, an action that allows you to write Javascript scripts using the Github API — `actions/github-script@v7`.

{% @github-files/github-code-block url="<https://github.com/actions/github-script>" %}

The `README.md` of the action contains a lot of examples of use cases with the Github script. However, we will just cover a very simple script to illustrate a few points:

```yaml
on:
  issues:
    types: [opened]

jobs:
  comment:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/github-script@v7
        with:
          script: |
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: '👋 Thanks for reporting!'
            })
```

In the above example, we are using Github script to create a new comment on a newly created issue. We can see that the event that triggers this workflow is the `issues`  event, when one is `opened`.&#x20;

The various API calls are based on the Octokit documentation: <https://octokit.github.io/rest.js/v21/> where you replace `octokit` with `github` !

You might be wondering, "How does the Github script have access to the Github API when some APIs require an API token?"

Amazing question! This is where the `GITHUB_TOKEN` we talked about in [Deploying to Github Pages](/hackerschool/ci-cd-with-github-actions/basics-of-github-actions/deploying-to-github-pages#permissions-and-secrets) come into play. By default, Github script uses the `GITHUB_TOKEN` to access these APIs, which means that it is restricted to only accessing the current repository. Therefore, if you wish to access other repositories or data that does not belong to the current repository and requires authentication, you will need to:

1. Create a Personal Access Token: <https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic>
2. Add it to the repository's secrets: <https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions>
3. Set the `github-token` input for the action: see below

```yaml
on:
  issues:
    types: [opened]

jobs:
  apply-label:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/github-script@v7
        with:
          github-token: ${{ secrets.MY_PAT }}
          script: |
            github.rest.issues.addLabels({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              labels: ['Triage']
            })
```

Another thing you can use Github script for is combining it with [Pollers](/hackerschool/ci-cd-with-github-actions/advanced-use-cases/pollers) to automatically perform some actions to the current repository at set intervals. An example of this might be to poll for new information across multiple repositories and updating a set of files on the current repository as commits:

```yaml
on:
  workflow_dispatch:
  schedule:
    - cron: "0 12 * * *"
jobs:
  poll:
    runs-on: ubuntu-latest
    steps:
      - name: Fetch all
        uses: actions/github-script@v7
        with:
          github-token: ${{ secrets.ORG_PAT }}
          script: |
            const repos = await github.paginate(github.rest.search.repos, {
              q: "<query string>",
            })

            let newReadme = `
            <updating the README>
            `
            const existingReadme = await github.rest.repos.getContent({
              owner: context.repo.owner,
              repo: "<repo>",
              path: "README.md",
            })

            await github.rest.repos.createOrUpdateFileContents({
              owner: context.repo.owner,
              repo: "<repo>",
              path: "README.md",
              message: "Update README",
              content: btoa(newReadme),
              sha: existingReadme.data.sha,
            })
```

This poller runs every day at midnight UTC, fetching all repositories that satisfy some query string and updating the README of the current repository as a commit.


# Executing third-party scripts

However, sometimes what you want goes beyond accessing the Github API. You might want to run a Python script as part of the workflow, calling other APIs. To do so, you can simply treat them as regular files in a filesystem (think back to how jobs run in virtual machine runners) and call these scripts.

The caveat is that you have to setup the job's virtual machine runner to support the third-party script's language. So, if you're using Python, you need to ensure that Python and all of the scripts' dependencies are installed. If you're using Javascript, ensure that Node.js and all of the project dependencies are installed.

Since we've covered how to use Node.js in Github Actions in [Basics of Github Actions](/hackerschool/ci-cd-with-github-actions/basics-of-github-actions) already, we will focus on setting up the job virtual machine runner to work with Python scripts this time.

```
on: [push]

jobs:
  autograding:
    permissions: write-all
    runs-on: ubuntu-22.04
    steps:
      - name: Fetch repository
        uses: actions/checkout@v4
      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.13"
      - name: Installing Python dependencies
        run: |
          pip install -r requirements.txt
      - name: Run Python
        run: |
          python3 script.py
        env:
          base_repository: ${{ inputs.repository }}
          is_local: ${{ inputs.is_local }}
          repository_name: ${{ env.REPO_NAME }}
```

Essentially, what you need to do is to:

1. Fetch the current repository
2. Setup Python using the `actions/setup-python@v5` action
3. Install all of the Python dependencies from a `requirements.txt` in the current repository (or individual dependencies)
4. Execute the Python script

It's that simple! Now, the `script.py` Python script will start to execute and now it will have full filesystem access to the job's virtual machine runner. You can additionally set environment variables for the script to access via `env`.

{% hint style="warning" %}
If you are attempting to run a third-party script every time a `pull_request` event occurs and want to read any repository secrets or access the `GITHUB_TOKEN` token, make sure you use the `pull_request_target` event instead. The `pull_request` event is susceptible to having untrusted scripts accessing this secure information, so for security reasons, Github has disabled its access to these values. However, `pull_request_target` does not suffer from such limitations!\
\
Read more about it here: <https://stackoverflow.com/questions/74957218/what-is-the-difference-between-pull-request-and-pull-request-target-event-in-git>
{% endhint %}

You may combine this use case with the previous two to create scheduled scripts that run and interact with the Github API!


# Reusable workflows

Our final advanced use case we would like to cover is the use of reusable workflows.

Suppose you're in a [monorepo](https://monorepo.tools/) and have the following sub-projects:

1. `web`
2. `admin`
3. `api`

All of them require the exact same CI pipeline of running unit tests and linting that we introduced in [Basics of Github Actions](/hackerschool/ci-cd-with-github-actions/basics-of-github-actions). If you copy-pasted the same workflow file three times, it might work, but this means that if one changes, everything needs to change. While some might argue that as the application expands, this flexibility is required to avoid a tight coupling to one type of workflow. However, for the sake of simplicity, let's suppose that this duplication is fundamentally bad for this use case. How do we go about reconciling this?

Well, this is where reusable workflows come in. They allow you to effectively define a "common workflow" that can be shared and reused by other workflows as steps. Essentially, what you're creating is custom **actions** that have not been properly published.

The [official documentation](https://docs.github.com/en/actions/sharing-automations/reusing-workflows) goes into the nitty gritty of the limitations and access of reusable workflows, so we will not cover it in this section. Instead, we will focus on setting up a very rudimentary reusable workflow for the above scenario.

So let's suppose that the original workflow looks like this:&#x20;

```yaml
# .github/workflows/web_ci.yml
name: CI/CD Pipeline
on: [pull_request, workflow_dispatch]
jobs:
  linting:
    runs-on: ubuntu-latest
    steps:
      - name: Fetch repository
        uses: actions/checkout@v4
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'yarn'
      - name: Install dependencies
        run: |
          yarn
      - name: Lint code
        run: |
          yarn lint

  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Fetch repository
        uses: actions/checkout@v4
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'yarn'
      - name: Install dependencies
        run: |
          yarn
      - name: Run unit tests
        run: |
          NODE_ENV=production yarn test
```

You realize that the job steps are exactly identical, apart from the folder that these commands are being run in. We can generalize these as inputs to the reusable workflow!

```yaml
# .github/workflows/reusable-ci.yml
name: Reusable CI Workflow
on:
  workflow_call:
    inputs:
      workdir:
        description: 'Working directory'
        required: true
        type: string
jobs:
  linting:
    runs-on: ubuntu-latest
    steps:
      - name: Fetch repository
        uses: actions/checkout@v4
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: ${{ inputs.node-version }}
          cache: 'yarn'
          cache-dependency-path: "${{ inputs.workdir }}/yarn.lock"
      - name: Install dependencies
        working-directory: ${{ inputs.workdir }}
        run: |
          yarn
      - name: Lint code
        working-directory: ${{ inputs.workdir }}
        run: |
          yarn lint

  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Fetch repository
        uses: actions/checkout@v4
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: ${{ inputs.node-version }}
          cache: 'yarn'
          cache-dependency-path: "${{ inputs.workdir }}/yarn.lock"
      - name: Install dependencies
        working-directory: ${{ inputs.workdir }}
        run: |
          yarn
      - name: Run unit tests
        working-directory: ${{ inputs.workdir }}
        run: |
          NODE_ENV=production yarn test
```

Essentially, the key things that had to alter were:

1. Changing the trigger event type to `workflow_call`, indicating it's a reusable workflow
2. Specifying the inputs that the reusable workflow requires, such as the `workdir` since that is the only thing that changes across variations of this CI workflow
3. Specifying the `cache-dependency-path` in the `actions/setup-node@v4` action as we need to use the `yarn.lock` files specific to each sub-project
4. Specifying the `working-directory` of each step to point to the given sub-project directory

This is all we really need to create the reusable workflow. Then, we can update our original `ci.yml` with the following:

```yaml
# .github/workflows/web_ci.yml
name: CI/CD Pipeline
on: [pull_request, workflow_dispatch]
jobs:
  ci:
    uses: <org name/username>/<repo name>/.github/workflows/reusable-ci.yml@main
    with:
      workdir: web
```

In fact, we can even inline every sub-project's CI into the same workflow:

```yaml
# .github/workflows/ci.yml
name: CI/CD Pipeline
on: [pull_request, workflow_dispatch]
jobs:
  web_ci:
    uses: <org name/username>/<repo name>/.github/workflows/reusable-ci.yml@main
    with:
      workdir: web
  admin_ci:
    uses: <org name/username>/<repo name>/.github/workflows/reusable-ci.yml@main
    with:
      workdir: admin
  api_ci:
    uses: <org name/username>/<repo name>/.github/workflows/reusable-ci.yml@main
    with:
      workdir: api
```

Incredible, we've managed to greatly simplify our CI workflow by using reusable workflows!


# Cookbook

These are some additional recipes you can implement in Github Actions!

## Debugging Github Actions

Given that steps in a job can be a script, you can actually perform logging as steps in a job:

```yaml
- name: Logging
  run: |
    echo ${{ github.repository }}
    echo 'Hello!'
```

These will add a step to the job that logs these echos.

## Conditional steps

You can also run steps based on certain conditions (using [expressions](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/evaluate-expressions-in-workflows-and-actions)). This is particularly useful when you want to only run a step when certain conditions are met.

```yaml
- name: Only run on Windows
  if: ${{ runner.os == 'windows' }}
  run: |
    Write-Output "test output"
```

The above step only runs when the runner OS is Windows.

{% hint style="info" %}
It is very important to note that there are several limitations to what expressions might work in the `if` key. These are dependent on what contexts are available to each part of a step. More information about contexts can be found [here](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/accessing-contextual-information-about-workflow-runs) and the contexts available to `if` are: `github`, `needs`, `vars`, `inputs`&#x20;
{% endhint %}

## Adding environment variables

Given that virtual machine runners run an OS, you will have access to environment variables from within the job through the `env` context. To add to the `env` context, you can use a step:

```yaml
- name: Export environment variables
  run: |
    echo "START=$(date +'%Y-%m-%dT%H:%M:%S')" >> $GITHUB_ENV
```

The above exports a new environment variable `START` into `env` . This can be then accessed via `${{ env.START }}`.

## Matrix strategies

Suppose that you want to verify that a set of changes are not susceptible to backward compatibility issues in a Node.js environment (version `20`), while ensuring that the latest Node.js version is supported as well (version `23`).

You can actually use matrix strategies to verify this information by running the same job across different parameters.

```yaml
jobs:
  example_matrix:
    strategy:
      matrix:
        version: [20, 23]
    steps:
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.version }}
```

So using the above, we are able to then run the same job `example_matrix` twice with two different node versions: `20` and `23`.


# Lightning Git

Learn Git, as fast as possible

### Overview

This will aim to get you introduced and using as many tools Git provides you, as quickly as possible. If you want to learn more about Git in detail, here are some resources:

{% embed url="<https://git-scm.com/book/en/v2>" %}

{% embed url="<https://wiki.nushackers.org/orbital/git>" %}

{% embed url="<https://youtu.be/pGAorBdZ6Y8?si=9p7jBxfKlnEFxkpc>" %}

We'll be using lazygit, a nice terminal wrapper around git.&#x20;

<details>

<summary>Why are we teaching a git wrapper instead of just git?</summary>

Traditionally, we have never really thought any flavour of git wrappers, because they tend to abstract out complexity and control which the git CLI offers. Personally, lazygit does the opposite job, exposing a lot of the more advanced commands and features in a manner that is easy-to-use and visualize

</details>

###


# Git Concepts

### What is Version Control? What is Git?

We use version control all the time in our lives. Think of something like Google Docs, where you can revert changes and "rewind" your changes if you make a mistakes and also see how the document has changed over time.

#### Why don't we just "Google Docify" everything?

Well, the key thing is that Google Docs, as the name suggests, is very much confined to a single document. Well, that can be easily solved, we can just Google Docify our folders right? Here's an example of how the Google Docs method of version control starts to fall apart.

Google Docs takes a snapshot of your document every n amount of time, and tries to "blame" each change on someone (every character/line changed has to be attributed to someone).

<figure><img src="/files/NIU3mHDZdPyvlgdsAn0t" alt=""><figcaption></figcaption></figure>

Imagine I'm helping modify a cake recipe in Google Docs:

1. At x point in time, a snapshot is taken. There is a line that says "Add 5g of sugar"
2. At x + 1 point in time, I decide that's too much sugar, I make a change to the line to "Add 3g of sugar"
3. At x + 2 point in time, Person Y accidentally sits on his keyboard while the docs is open and replaces the line with "Add 2348g of sugar"
4. At x + 3 point in time, a snapshot is taken. Google Docs versioning now shows that Person Y has changed 5g to 2348g of sugar, and my changes are lost to time.

Now imagine this problem on a large codebase of millions of lines, with hundreds of engineers contributing to different parts of this file. How can one prevent something like this? We want each and every change to be well documented, justified, and more importantly, reversible. These are the guarantee version control like Git provides.&#x20;

## Local and remote repositories

Git relies on the core concept of a repository, which is essentially a parent folder that Git is added to to monitor the changes of the folder and its contents (including sub-folders).

These repositories can exist on both your local machines or remotely on an external server (or [self-hosted](https://about.gitea.com/)). This guide will look at both instances.

Github is an example of a hosted remote Git server where you can create remote repositories and work on them locally (while pushing changes remotely, hence the "decentralized" nature of Git).

Think of it like having two versions of a Google Docs. When you are editing your document in a train for example, you might lose connectivity, and you'll have an offline copy which is different from the online copy (the source of truth)

<figure><img src="/files/KstKKtEJby0cSbWS0kRd" alt=""><figcaption></figcaption></figure>

### Commits

A commit is a snapshot of the entire **repository** at a point in time, plus some metadata. More specifically, it contains:

* A hash, or a unique (kinda) identifier for a commit, sort of like your student ID
* The author of the commit, the email of the author, and the time of the commitds
* Each commit also has a parent commit (except the first commit)
* We can "chain" commits by following the parent commit till we hit the first commit. If we do this for every commit, we get a **directed acyclic graph**
  * Directed: Commits point to their parents
  * Acyclic: There cannot be commit cycles.&#x20;

<figure><img src="/files/gQgwPn07Z9tvMw8Qo2Iw" alt="" width="373"><figcaption></figcaption></figure>

<figure><img src="/files/5CoKqe93E3jcp5XU0fnY" alt=""><figcaption></figcaption></figure>


# Getting Started with Git

## Installing Git

You will need to [install Git on your local machine](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git).

## Configuring Git

{% hint style="info" %}
This guide assumes that you are using Git Bash (for Windows) or the terminal (for macOS and Linux), as we will be using Bash commands.\
\
If you are facing issues running certain commands (especially on Windows), please consider using Git Bash instead.
{% endhint %}

To ensure that Git installed correctly, run the following command:

```sh
git version
```

You should see a line of output like this:

```
git version 2.54.0
```

Once Git is installed on your local machine, tell Git who you are:

```sh
git config --global user.name '<your full name>'
git config --global user.email '<your email address>'
```

Then, tell Git about your preferred default code editor:

```sh
git config --global core.editor '<editor executable>'
```

For example, to use Visual Studio Code as your default editor for Git, replace `<editor executable>` from above with `code --wait` :

```sh
git config --global core.editor 'code --wait'
```

For this workshop, you'll also need lazygit.[ Follow the instructions here](https://github.com/jesseduffield/lazygit?tab=readme-ov-file#installation)


# Making your first commit

Great, now we have Git set up! You might have some questions about the Git Setup process:

<details>

<summary>Why do I need to indicate my name and email?</summary>

If you remember, each commit contains metadata about the author's name and email. When you set this up in your git config, this will ensure all your commits are properly attributed to the correct author (you!)

</details>

<details>

<summary>What is this ssh key stuff?</summary>

To put it simply, it is a good way for Github to "authenticate" you. You wouldn't want unauthorized people trying to change your repositories.

</details>

## Adding files to a snapshot

By default, Git does not know what files it should be including in a snapshot (and this is a good thing because we don't want Git to just add every file as they may contain sensitive information).

This is where the "three areas" concept comes into play. It is often good to think of your projects with Git as three separate concepts:

<figure><img src="/files/448Dx4PtnN0L1bdnM7EQ" alt=""><figcaption></figcaption></figure>

1. Working directory: where your codebase actually resides
2. Staging area: set of files that you want to include in a snapshot
3. Repository: local/remote repository storing metadata about the project and Git

By default, all of your files reside in the working directory and are not yet added to the staging area. If you want a file included in the staging area, then you must first add it to the staging area (we will cover how this happens later on).

{% hint style="info" %}
There are also ways to remove files from the staging area!
{% endhint %}

<figure><img src="/files/M8bRjWgkPBYokp2AYwOX" alt=""><figcaption></figcaption></figure>

The traditional way is to use the commands in [Command Glossary](/orbital/git/command-glossary) to add files to the staged area, then using `git commit`. Let's try using lazygit to speed up this workflow.

To start, let's first initialize a repository somewhere.

```bash
mkdir recipe_repo
cd recipe_repo
git init
```

Make a new file, `recipe.txt` and modifying it a little.

{% tabs %}
{% tab title="On lazygit" %}
Now, run lazygit

```bash
lazygit
```

* Hit 2 to go to the files submenu.
* Hit a to stage all commits (this is the same as `git commit -A`)
* Hit spacebar to stage commits by individual files
* Hit Enter to enter into a file and use spacebar to select line by line which files to stage (This is known as interactive staging)
* Once you've selected what you want to commit, press c, and enter a message, then hit Enter to commit
  {% endtab %}

{% tab title="On Git CLI" %}

## Adding a new file

Create a new file in the folder and add some text to it.

```
echo 'Hello world' >> hello.txt
```

{% hint style="info" %}
The command above essentially redirects the output of the `echo` (Hello world) into a new file `hello.txt`
{% endhint %}

If you don't want to use bash commands, you can just create the file using your preferred method as well.

## Making your first commit

### Getting the status of a repository

Now, run the following command to view the status of your repository:

```
git status
```

You should see the following:

```
On branch main

No commits yet

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	hello.txt

nothing added to commit but untracked files present (use "git add" to track)
```

Recall that in [Fundamental Concepts](/orbital/git/fundamental-concepts#adding-files-to-a-snapshot), Git does not automatically add files to a snapshot as it does not know exactly what you want. So we want to tell Git that we want `hello.txt` in the snapshot.

{% hint style="info" %}
`git status` is to view the state of your repository in Git's eyes. Use it to view things like the current files in the snapshot.
{% endhint %}

### Tracking files

You may notice that the `git status` message states that `hello.txt` is untracked. Untracked files are those that have never been registered with Git before. They are often new files that have been added to the repository and have not existed in any snapshots.

Files that have been added to a snapshot before are considered "tracked" and Git knows to look out for changes between snapshots.

### Adding files to the staging area

As discussed in [Fundamental Concepts](/orbital/git/fundamental-concepts#introducing-the-commit), a file from the working directory needs to be explicitly added to the staging area for a snapshot to include it. By default, an untracked file that is added to a snapshot becomes tracked for future snapshots.

To add `hello.txt` to the staging area, use the following command:

```
git add hello.txt
```

Then, use `git status` to view the status of your repository again:

```
On branch main

No commits yet

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)
	new file:   hello.txt
```

Notice that now, instead of stating that your file is untracked, Git is indicating that the changes have not committed. This is a sign that the file(s) have been tracked and added to the snapshot.

{% hint style="info" %}
You can use `.` to add all files in the current folder as well.
{% endhint %}

### Taking the snapshot

Now, to take the snapshot (make the commit), you can use the following:

```
git commit -m "First commit"
```

The `-m` flag is used to specify the commit message. Every commit has an accompanying message that you can use to indicate what the commit contains/entails.

{% hint style="info" %}
If you do not use `-m`, your favorite terminal/GUI editor will be launched and you can compose the commit message in that editor, save it, and close the editor
{% endhint %}

There you have it! You have made a local repository and created a snapshot! We will now look at how we can integrate Github with your local repository!
{% endtab %}
{% endtabs %}

### Ignoring files

See: [Ignoring Files](/orbital/git/advanced-concepts/ignoring-files)


# Branching

On top of commits, we like to label different "lines of work" as branches. By default, your branch will be something like "master" or "main". We can create new branches to group together a bunch of changes and commit.

<figure><img src="/files/qQV7R65RLQRpgqIg8pTN" alt=""><figcaption></figcaption></figure>

## HEAD

Notice that all commits come with a unique ID. This can make it difficult to reference the exact commit you are on. This is where `HEAD`  label comes in.

`HEAD` is a special label given to the current commit you are looking at.

<figure><img src="/files/aHFqH6tMjOJhdvNHK9sc" alt=""><figcaption></figcaption></figure>

To move you HEAD around in lazygit, go to the Commits submenu (hit 4), then navigate with arrow keys/mouse to the commit you want and hit spacebar to switch to that commit.

Without lazygit, you would do something like:

```
git checkout <commit-hash/branch name>
```

## Creating a new branch

By default, a new branch is always created from the point of `HEAD` of your current branch (usually `main`) onwards. This means that the branch will have all the snapshots that precede (and include) `HEAD` but any new snapshots made on the branch are not reflected (yet) on `main.`

{% tabs %}
{% tab title="On Lazygit" %}

* Hit 3 to go to the branches submenu
* Hit n to create a new branch
* Hit space to switch branches
* Hit d to delete a branch
* Hit R to rename a branch
  {% endtab %}

{% tab title="On Git CLI" %}
There are two ways to create a new branch:

```
git branch <branch name>
```

Then, you can switch to the branch by using:

```
git checkout <branch name>
```

Alternatively, you can use the `git checkout` command for both:

```
git checkout -b <branch name>
```

## Changing branches

As mentioned earlier, you can switch to a branch via:

```
git checkout <branch name>
```

## Viewing all branches created

To view all branches, you can use the following:

```
git branch -v
```

## Deleting a branch

To delete a branch, you add the `-d` flag:

```
git branch -d <branch name>
```

## Renaming a branch

You may have misspelled the branch name or parts of it. You can rectify it using the `-m` flag:

```
git branch -m <new branch name>
```

{% hint style="info" %}
This renames the current branch that you are on.
{% endhint %}
{% endtab %}
{% endtabs %}

## Combining changes of branches

Recall that we mentioned that the changes of a branch are not reflected across any other branch UNTIL otherwise specified? How exactly do we specify this?

An easy way to do so is by merging the branches into one another.

Suppose we have two branches: `main` and `feature-A` and we want all the changes from `feature-A` to be present in `main` so that we can demo it to the executives. We first need to clearly denote which is the source branch (where the changes exist) and the target branch (where we want the changes to appear in). In this scenario, `feature-A` is the source branch and `main` is the target branch.

Then, a simple procedure to perform the merge would be:

1. Switch to the target branch
2. Merge source branch into target branch

This can be done via:

{% tabs %}
{% tab title="On lazygit" %}
Hit M while to merge a branch to your current branch
{% endtab %}

{% tab title="On Git CLI" %}

```
git checkout main
git merge feature-A
```

{% endtab %}
{% endtabs %}

However, this process is not always so straightforward. As you will see in the coming chapter, merging has its own set of "problems" that may arise.


# Commit Manipulation and Reflog

### Commit Manipulation

In many projects, commits are more than a snapshot. It should represent a working state of the repository. That means, some projects don't really like it if you have a typo, then you have to create a separate commit to fix said typo.

So then, how do we rewrite the commit to fix the typo? That's where commit manipulation comes in.

<figure><img src="/files/Fehy8lySdUdQUKDRtCIj" alt=""><figcaption></figcaption></figure>

{% hint style="warning" %}
Notice that the master branch changes to the new commit. That is, there will be no way to checkout the commit A unless you know the exact hash you are looking for.
{% endhint %}

Amending a commit adds some changes to an additional commit and "destructively" replaces the previous commit from it. This could be changes to the file, the author, or commit message, or even a combination of everything.

To do this is lazygit:

* Move your head to the commit to amend
* Make some changes and stage it
* Hit A to amend the current commit with the changes

### Reflog

That's pretty cool, but what if I screw up, and want to look at the commit before I amended it? There are no pointers or labels now pointing at my old commit? That's where reflog comes into play.

Reflog is essentially a history of all the commits your HEAD has touched. It also keeps track of the hashes so that you can easily navigate between commits that might have been lost otherwise.

To use reflog:

{% tabs %}
{% tab title="In lazygit" %}

* Click on the reflog tab
* Navigate the the commit you want, hit space to checkout
* You can also hit C to cherry pick the commit if you want to apply it to your current branch
  {% endtab %}

{% tab title="In Git CLI" %}

* Do `git reflog`.
* Find the hash of the commit you want to checkout
* `git checkout <hash>`
  {% endtab %}
  {% endtabs %}


# Interactive rebasing

So what if I want to manipulate multiple commits at once?

This is where interactive rebasing comes in. Given a bunch of commits, you can change the order, drop certain commits or amend them.

### Exercise

Go to <https://github.com/nushackers/git-exercise>

There's a type in `1dab2c`. Let's amend it and propagate it up.

In lazygit:

* Use v to select all the commits you want in your interactive rebase
* Hit e to start the interactive rebase
* While hovering over a commit
  * p to pick
  * d to drop
  * e to edit
  * s to squash
  * \<C-j> or \<C-k> to rearrange your commits


# filter-repo

Oh no! I've committed a secret to my repository, and pushed the changes!&#x20;

{% hint style="warning" %}
If you alreay pushed the changes, you might want to revoke the secret.
{% endhint %}

<details>

<summary>Can't I just amend the commit and force push?</summary>

Even if you force push, the commit will still exist somewhere in the git in remote

</details>

git-filter-repo is an add-on to git that allows you to remove secrets from your git completely. Remember that, even if we amend a commit, the commit that contains our secrets still remain in the tree. We'll need a way to remove that

{% @github-files/github-code-block url="<https://github.com/newren/git-filter-repo>" %}

Once you've installed git-filter-repo, run:

```bash
git-filter-repo --force --invert-paths --path <path-to-secret>
```


# Introduction to C++


# Welcome!

C++ can feel overwhelming!

Maybe you first encountered it in CS2040C algorithms & data structures module. Maybe you heard horror stories about pointers and seemingly mysterious segmentation faults that span 300 lines!

Despite its pitfalls, C++ remains the language of choice behind the many of the world's fastest systems. This includes high performance databases like [Clickhouse](https://clickhouse.com/) and [RocksDB](https://rocksdb.org/), game engines like [Unreal Engine](https://www.unrealengine.com/en-US/unreal-engine-5) and browsers like [Google Chrome](https://www.google.com/intl/en_sg/chrome/).

### Why C++?

Unlike purely managed languages, C++ does not shy away from the messy details. Instead, it forces one to understand what your program is actually doing. This includes understanding how memory layout, object lifetimes and resource management actually works. While this might appear initially uncomfortable, it helps builds deep intuition about how computers actually work.

### Using this guide

If you are new to C++, you are highly encouraged to read through the content in order. The guide walks you through some of C++ fundmwnrtasl, common memory pitfalls and how modern C++ features are designed to help you tackle and avoid them.

If however, you are familiar with C++ constructs (i.e variables, loops, functions), feel free to skip ahead towards the [Memory & Ownership](/hackerschool/introduction-to-cpp/memory-and-ownership) section. This introductory workshop is tailored towards begineers so you might find the content a bit brief.

### Who Am I?

I'm Benn Tan, a computer science undergraduate from the National University of Singapore and a core member of NUS Hackers.

I've previously used C++ in my own free time and I hope to share some of the knowledge gained through [learncpp.benntan.com](https://learncpp.benntan.com)


# C++ Fundamentals

This guide aims to provide you with the fundamentals of C++. It's **not** meant to be an exhaustive guide. For more in-depth details, you should refer to the [official C++ reference](https://en.cppreference.com/w/).

We do assume that you have basic programming knowledge and are cognizant of common programming constructs (i.e variables, functions, loops).

### Getting started

To run C++ code, you will need a **compiler**. Alternatively, you can also use an online compiler like [JDoodle](https://www.jdoodle.com/online-compiler-c++17) to follow along.

#### Mac

To check if you have the `clang++` compiler installed, run:

```bash
clang++ --version
```

If not, you can install it via:

```bash
xcode-select --install
```

A software update window will pop up. You will need to agree to the licensing agreement before you can commence installing.

If all goes well, you should be greeted with:

<figure><img src="/files/ehSvW1Y4ZoTq3NMFiLnB" alt=""><figcaption></figcaption></figure>

#### Windows

On Windows, the setup process varies. The simplest option is to install an Integrated Development Environment (IDE) like [Dev-C++](https://www.dev-cpp.com/) that bundles with a C++ compiler.

#### Ubuntu / Debian

To check if you have the `g++` compiler installed, you can check via:

```bash
g++ --version
```

If it is not installed, you can install it via your favourite package manager.

```bash
sudo apt update
sudo apt update g++
```


# Program Execution Model

When one develops a C++ program, it does not run directly. Instead, it goes through a fixed sequence of steps before your computer actually understands and executes it.

### Running Your Source Code

Let's start with a simple single-file C++ program. The following *prints **Hello World*** to the console.

{% code title="main.cpp" %}

```cpp
#include <iostream>

int main() {
    std::cout << "Hello, world!";
    return 0;
}
```

{% endcode %}

This file is called the **source code.**

To run it, we need to compile the code:

{% code title="" %}

```sh
clang++ main.cpp -o hello-world
```

{% endcode %}

This will produce an executable file called `hello-world` which your computer can finally run:

{% code title="" %}

```sh
./hello-world
```

{% endcode %}

Each time you make changes to the source code, you will need to compile again.

<figure><img src="/files/znhZtV1SEiJxo5aSGAjH" alt=""><figcaption></figcaption></figure>

### What Just Happened ?!

Think of a compiler as a black box that simply ***converts*** source code to machine code.

<figure><img src="/files/qZahkv3ZEbJGKQhEPEIy" alt=""><figcaption><p>src: <a href="https://www.sitesbay.com/cpp/cpp-compiler">https://www.sitesbay.com/cpp/cpp-compiler</a></p></figcaption></figure>

<details>

<summary>What does a compiler exactly do?</summary>

This is beyond the scope of this workshop.

If you happen to be curious:

<figure><img src="/files/P11RzQeCr6A8Q8MT1rxl" alt=""><figcaption><p>src: <a href="https://www3.ntu.edu.sg/home/ehchua/programming/cpp/gcc_make.html">https://www3.ntu.edu.sg/home/ehchua/programming/cpp/gcc_make.html</a></p></figcaption></figure>

Compiling a C++ source code is usually a 4 step process. It composes of

**a) Preprocessing** where we substitute macros and header files with their actual content

**b) Compilation** from C++ to platform-specific assembly

**c) Assemble** where the resultant assembly code is assembled into actual object code

**d) Linking** to link external library functions that the executable needs

You can even pause the compilation process at each stage to inspect the immediate outputs using varying CLI flags.

</details>


# Basic Syntax & I/O

Let's kick things off by familiarising ourselves with the basic structure of a C++ program.

### Anatomy of a C++ Program

Recall the source code you previously saw:

{% code title="main.cpp" %}

```cpp
#include <iostream>

int main() {
    std::cout << "Hello, world!";
    return 0;
}
```

{% endcode %}

Breaking it down:

<table><thead><tr><th width="190.94439697265625">Code</th><th>What it does</th></tr></thead><tbody><tr><td><pre><code>#include &#x3C;iostream>
</code></pre></td><td>Brings in the standard I/O library so that we can use things like <code>std::cout</code></td></tr><tr><td><pre><code>int main() {
</code></pre></td><td><p>The entry point of every C++ program.</p><ul><li><code>int</code> indicates the return type of the function</li><li><code>main</code> is the function name.</li></ul></td></tr><tr><td><pre><code>std::cout &#x3C;&#x3C; "Hel...
</code></pre></td><td><p>Prints the text to the console.</p><p>👉 You can remember <code>&#x3C;&#x3C;</code> as <em>pushing</em> the text into the output stream (i.e <code>std::cout</code> in this case)</p></td></tr><tr><td><pre><code>return 0;
</code></pre></td><td><p>Recall that <code>int</code> is the return type of the <code>main</code> function.</p><p>Hence, at the end of our <code>main</code> function, we should<sup>[1]</sup> return an int.</p><p>You can return anything, really. But usually returning 0 indicates our program has successfully executed.</p></td></tr></tbody></table>

{% hint style="info" %}
Try augmenting the `Hello, world!` string to something else like your name!

<img src="/files/WE8WyhpjSPhl7yY0aXL0" alt="" data-size="original">
{% endhint %}

<sub>\[1] Your code will still compile even if you don't return anything from the main function. This is because C++ implicilty returns 0 at the end of</sub> <sub>`main()`</sub><sub>. For more info, refer</sub> [<sub>here</sub>](https://stackoverflow.com/questions/19293642/why-does-the-main-function-work-with-no-return-value)<sub>.</sub>

### Newlines

If you try printing multiple times, you will notice that the output gets concatenated on the same line. You can fix this by adding a newline using `std::endl` .

<table><thead><tr><th>Code</th><th>Output</th></tr></thead><tbody><tr><td><pre class="language-cpp" data-title="main.cpp"><code class="lang-cpp">#include &#x3C;iostream>
int main() {
    std::cout &#x3C;&#x3C; "benn";
    std::cout &#x3C;&#x3C; "tan";
    std::cout &#x3C;&#x3C; "jia";
    return 0;
}
</code></pre></td><td><img src="/files/uTUq0Fx7LjY5E32PeVLy" alt="" data-size="original"></td></tr><tr><td><pre class="language-cpp" data-title="main.cpp"><code class="lang-cpp">#include &#x3C;iostream>
int main() {
    std::cout &#x3C;&#x3C; "benn" &#x3C;&#x3C; std::endl;
    std::cout &#x3C;&#x3C; "tan" &#x3C;&#x3C; std::endl;
    std::cout &#x3C;&#x3C; "jia";
    return 0;
}
</code></pre></td><td><img src="/files/y9bIxBrt6PdYtoxEbczd" alt="" data-size="original"></td></tr></tbody></table>

### Input

It's not quite fun if we hardcode everything into program. Let's augment our program to accept user input externally.

To read input from the user, we can use `std::cin >>`

{% code title="main.cpp" %}

```cpp
#include <iostream>

int main() {
    int x;
    std::cin >> x;
    std::cout << "You entered: " << x << std::endl;
}
```

{% endcode %}

{% hint style="info" %}
Note that the direction of the arrows (`>>`) for `std::cin` is now reversed compared to `<<`.

👉 You can remember as **data flowing into `x`**.
{% endhint %}

### Namespaces

If you find prefixing `std::` before the function calls cumbersome, you can omit it with `using namespace std`

{% code title="main.cpp" %}

```cpp
#include <iostream>
using namespace std;
int main() {
    int x;
    cin >> x;
    cout << "You entered: " << x << endl;
}
```

{% endcode %}

However, this is generally discouraged in larger projects as it does lead to naming conflicts.

👉 It's usually best practice to be explicit and use `std::`

### Exercise

Build a program that takes in and prints the result of the summation of two numbers.


# Variables

Now that we are able to accept input and print output externally, we need a mechanism to **store data** inside our program.

### What is a Variable?

A variable is simply an named container that stores some value.

{% code title="main.cpp" %}

```cpp
int x = 5;
```

{% endcode %}

In the example above,

* `int` indicates the **type** of the variable
* `x` is the **name** of the variable
* `5` is the **value** of the variable

### Common Data Types

|               |                             |                         |
| ------------- | --------------------------- | ----------------------- |
| `int`         | `int age = 24`              | Integer (whole numbers) |
| `double`      | `double pi = 3.14`          | Floating point numbers  |
| `char`        | `char x = 'c'`              | Singular character      |
| `bool`        | `bool isGood = true`        | True / False            |
| `std::string` | `std::string name = "Benn"` | Sequence of characters  |

{% hint style="info" %}
⚠️ Common Pitfall

Use double quotes (`" "`) for strings and single quotes `' '` for characters
{% endhint %}

### Declaration vs Initialisation

Declaration is when we define a variable's **type** and **name** without assigning it a **value** yet. Initialisation is when we give it a **value**.

{% code title="main.cpp" %}

```cpp
int x;      // declaration
x = 10;     // assignment

int y = 10; // declaration + initialisation
```

{% endcode %}

### Arithmetic Operators

You can also perform arithmetic operations just like how you would do in normal math:

{% code title="main.cpp" %}

```cpp
int x = 10;
int y = 12;

int sum = x + y;
int difference = x - y;
int product = x * y;
double quotient = x / y; // note that we use the double here
int remainder = x % y;

int expression = ((x + y) * (x - y));
```

{% endcode %}

{% hint style="info" %}
⚠️ Common Pitfall

Integer division **truncates** the division

So `10 / 12 = 0` instead of `0.83`
{% endhint %}

### Exercise

Extending from our previous exercise, we now want to support different operations beyond addition. Our mini-calculator should now be able to add, subtract, multiply and divide two numbers!


# Control Flow

So far, our programs have executed line-by-line, step-by-step from top to bottom. In the real world however, we might want to

* make decisons based on a condition
* repeat certain actions
* organise logic into reusable pieces

This is where **control flow** comes in

### Conditionals

Conditionals let our program make decisions.

{% code title="main.cpp" %}

```cpp
int score = 85;

if (score == 100) {
    std::cout << "Perfect" << std::endl;
} else if (score >= 90) {
    std::cout << "Grade A" << std::endl;
} else if (score >= 80) {
    std::cout << "Grade B" << std::endl;
} else if (score >= 70) {
    std::cout << "Grade C" << std::endl;
} else {
    std::cout << "Needs improvement" << std::endl;
}
```

{% endcode %}

{% hint style="info" %}
⚠️ Common Pitfall

`=` means assignment while `==` checks for equality

{% code title="" %}

```cpp
int x = 5;      // assignment
x == 5;         // comparison
```

{% endcode %}
{% endhint %}

### Loops

Loops lets us repeat certain blocks of code

#### while loop

A `while` loop keeps running as long as the condition is fufilled.

<table><thead><tr><th width="354.14404296875">Code</th><th>Output</th></tr></thead><tbody><tr><td><pre class="language-cpp" data-title=""><code class="lang-cpp">int x = 1;
while (x &#x3C;= 5) {
    std::cout &#x3C;&#x3C; x &#x3C;&#x3C; std::endl;
    x++;
}
</code></pre></td><td><img src="/files/4CWI8Nv0xuUwQYyjQOYy" alt="" data-size="original"></td></tr></tbody></table>

#### for loop

A `for` loop is useful when we know how many times we want to repeat something.

<table><thead><tr><th width="353.73614501953125"></th><th></th></tr></thead><tbody><tr><td><pre class="language-cpp" data-title="main.cpp"><code class="lang-cpp">for (int i = 1; i &#x3C;= 5; i++) {
    std::cout &#x3C;&#x3C; i &#x3C;&#x3C; std::endl;
}
</code></pre></td><td><img src="/files/k0Out8ab7fDt2ro1r4cy" alt="" data-size="original"></td></tr></tbody></table>

#### break / continue

`break` forces the loop to terminate prematurely.

`continue` skips the rest of the current iteration.

<table><thead><tr><th width="354.77777099609375"></th><th></th></tr></thead><tbody><tr><td><pre class="language-cpp" data-title="main.cpp"><code class="lang-cpp">for (int i = 1; i &#x3C;= 5; i++) {
    if (i == 3) {
        break;
    }
    std::cout &#x3C;&#x3C; i &#x3C;&#x3C; std::endl;
}
</code></pre></td><td><img src="/files/cD3KvON07aLaCv7gPfKr" alt="" data-size="original"></td></tr><tr><td><pre class="language-cpp" data-title="main.cpp"><code class="lang-cpp">for (int i = 1; i &#x3C;= 5; i++) {
    if (i == 3) {
        continue;
    }
    std::cout &#x3C;&#x3C; i &#x3C;&#x3C; std::endl;
}
</code></pre></td><td><img src="/files/UjWeamwbRydM8sAKRsKf" alt="" data-size="original"></td></tr></tbody></table>

### Functions

Functions let us group code into reusable blocks.

#### Basic Usage

{% code title="main.cpp" %}

```cpp
#include <iostream>

void greet() {
    std::cout << "Hello!" << std::endl;
}

int main() {
    greet();
}
```

{% endcode %}

Here:

* `void` means the function does not return anything
* `greet` is the function name
* `greet()` *calls* the function, causing **Hello!** to be printed.

#### Function Parameters

Our current `greet` function always print the same message (which isn't very flexible). Let's augment it by adding a parameter so that our greeting message can change depending on the person.

{% code title="main.cpp" %}

```cpp
#include <iostream>

void greet(std::string name) {
    std::cout << "Hello, " << name << "!" << std::endl;
}

int main() {
    greet("Benn");
    greet("Bryan");
    greet("Anton Tim");
}
```

{% endcode %}

#### Functions Return Value

A return value is the value the function sends back to the caller after it's done executing. At the call site, you can use the result however you wish.

{% code title="main.cpp" %}

```cpp
#include <iostream>

int add(int x, int y) {
    return x + y;
}

int main() {
    int result = add(3, 4);
    std::cout << result << std::endl;
}
```

{% endcode %}

### Exercise(s)

⭐ Write a function `countdown(int n)` that takes in a number `n` and prints from `n` down to 1.

{% code title="" %}

```shellscript
countdown(5)
5
4
3
2
1
```

{% endcode %}

⭐⭐ Write a program that hardcodes a secret number. It will repeatedly ask the user to guess. Depending on the user's guess, the program will feedback **Too low, Too high** or **Correct.**

{% code title="" %}

```
Guess the number: 50
Too high

Guess the number: 25
Too low

Guess the number: 30
Too low

Guess the number: 42
Correct!
```

{% endcode %}


# Memory & Ownership

So far, we've been using variables without really thinking **where they live** / **who owns them.**

In C++, understanding memory is extremely important as it helps you write

* faster programs
* safer code
* avoid nasty bugs like memory leaks


# Pointers

### Address of Variables

So far, we've been working with variables like:

{% code title="main.cpp" %}

```cpp
int x = 10;
```

{% endcode %}

But sometimes, we don't want the value itself, rather we will want to know **where is this value stored in memory?**

The variable `x` could have some address like `0x100..`.

<figure><img src="/files/NEOblVAP8VnSkTyj3TzT" alt=""><figcaption></figcaption></figure>

We don't know so lets find out using the ampersand `&` operator.

{% code title="main.cpp" %}

```cpp
int main() {
    int x = 10;
    std::cout << "location of x: " << &x << std::endl;
}
```

{% endcode %}

<figure><img src="/files/9H019mt8omNqkhpOhsdI" alt=""><figcaption></figcaption></figure>

### Type of Pointers

We know the type of `x` is an `integer` , but what's the type of `&x` ?

Instead of second guessing ourselves, let's rely on the C++ compiler to tell us. C++ offers a [`typeid` function ](https://en.cppreference.com/w/cpp/language/typeid.html)that seems useful:

{% code title="main.cpp" %}

```cpp
int main() {
    int x = 10;
    std::cout << typeid(x).name() << std::endl;
    std::cout << typeid(&x).name() << std::endl;
}
```

{% endcode %}

<figure><img src="/files/MtSlHQ88CAE32FiwzVoE" alt=""><figcaption></figcaption></figure>

The output may look cryptic but:

* `i` (as you might have guessed) represents **int**
* `Pi` (which you also might have guessed) represents **pointer to int** (`int*`)

Indeed, we can write it like so:

<table><thead><tr><th>Code</th><th>Visual Aid</th></tr></thead><tbody><tr><td><pre class="language-cpp"><code class="lang-cpp">int main() {
    int x = 10;
    int* p = &#x26;x;
}
</code></pre></td><td><img src="/files/1uSaVDBqt7kdTEdfl4q2" alt=""></td></tr></tbody></table>

### De-referencing

Given a pointer of type (`int *`), is there a way I can get the *actual* *value (*&#x61;nd not the address) it points to?

In other words,

{% code title="main.cpp" %}

```cpp
int main() {
    int x = 10;
    int* p = &x;
    
    // can i get 10 using variable p alone?
}
```

{% endcode %}

Yes we can! Since the pointer `p` stores an address, we can dereference the pointer using `*` to get the value! Think of dereferencing as follow the address to get the value stored there.

<table><thead><tr><th>Code</th><th>Visual Aid</th></tr></thead><tbody><tr><td><pre class="language-cpp"><code class="lang-cpp">#include &#x3C;iostream>

int main() {
int x = 10;
int\* p = \&x;
int y = \*p;

```
std::cout &#x3C;&#x3C; "value of x: " &#x3C;&#x3C; x &#x3C;&#x3C; std::endl;
std::cout &#x3C;&#x3C; "value of y: " &#x3C;&#x3C; y &#x3C;&#x3C; std::endl;

return 0;
```

} </code></pre></td><td><img src="/files/8vvUWds9IzltNb8G9Cwy" alt=""></td></tr></tbody></table>

Realise while both values of x and y are the same, they are **not** pointing to the same 10. They each contain their own version of 10 (and possess their own memory addresses).

<details>

<summary>How do you make <code>x</code> and <code>y</code> to point to the same 10 then?</summary>

Simple: we don't make two variables. We declare 1 variable and multiple pointers to the same variable.

<table><thead><tr><th></th><th></th></tr></thead><tbody><tr><td><pre class="language-cpp" data-title="main.cpp"><code class="lang-cpp">#include &#x3C;iostream>

int main() {
int x = 10;

```
int* y = &#x26;x;   // y points to x
int* z = &#x26;x;   // z also points to x

std::cout &#x3C;&#x3C; "x: " &#x3C;&#x3C; x &#x3C;&#x3C; std::endl;        // 10
std::cout &#x3C;&#x3C; "*y: " &#x3C;&#x3C; *y &#x3C;&#x3C; std::endl;      // 10
std::cout &#x3C;&#x3C; "*z: " &#x3C;&#x3C; *z &#x3C;&#x3C; std::endl;      // 10

*y = 20;  // modify x via pointer y

std::cout &#x3C;&#x3C; "x: " &#x3C;&#x3C; x &#x3C;&#x3C; std::endl;   // 20
std::cout &#x3C;&#x3C; "*z: " &#x3C;&#x3C; *z &#x3C;&#x3C; std::endl; // 20

return 0;
```

} </code></pre></td><td><img src="/files/2vFBfnia8IwMJmXBDkbX" alt=""></td></tr></tbody></table>

</details>

### Pointer-Ception

And yes because pointers also have addresses, we can have *pointers that point to a pointer that points to a value.*

<figure><img src="/files/mc3lrLaMpGZUWwUfbxJO" alt=""><figcaption></figcaption></figure>

<table><thead><tr><th>Code</th><th>Output</th></tr></thead><tbody><tr><td><pre class="language-cpp" data-title="main.cpp"><code class="lang-cpp">#include &#x3C;iostream>

int main() {
int x = 10;            // normal variable
int\* p = \&x;           // pointer to x
int\*\* pp = \&p;         // pointer to pointer

```
std::cout &#x3C;&#x3C; "x value: " &#x3C;&#x3C; x &#x3C;&#x3C; std::endl;
std::cout &#x3C;&#x3C; "Address of x (&#x26;x): " &#x3C;&#x3C; &#x26;x &#x3C;&#x3C; std::endl;

std::cout &#x3C;&#x3C; "p (points to x): " &#x3C;&#x3C; p &#x3C;&#x3C; std::endl;
std::cout &#x3C;&#x3C; "Value at p (*p): " &#x3C;&#x3C; *p &#x3C;&#x3C; std::endl;

std::cout &#x3C;&#x3C; "pp (points to p): " &#x3C;&#x3C; pp &#x3C;&#x3C; std::endl;
std::cout &#x3C;&#x3C; "Value at pp (*pp): " &#x3C;&#x3C; *pp &#x3C;&#x3C; std::endl;
std::cout &#x3C;&#x3C; "Value at *pp (**pp): " &#x3C;&#x3C; **pp &#x3C;&#x3C; std::endl;

return 0;
```

} </code></pre></td><td><img src="/files/aU94CX1g018HJSq61Vck" alt="" data-size="original"></td></tr></tbody></table>


# Arrays & Strings

Thus far, pointers pointed to a singular variable. But what if we have multiple values stored together (exactly, like an array!)

### Arrays

To declare an array:

{% code title="" %}

```cpp
int arr[3] = {10, 20, 30}; // array
int* p_arr = arr;          // pointer to array
```

{% endcode %}

Visually, this is how an array is represented:

<figure><img src="/files/eJLPZthI6fbRO5USQHB8" alt=""><figcaption></figcaption></figure>

Note that:

* All 3 elements sit **side-by-side** (i.e there exists no gaps between them). This is what we mean by **contiguous memory.**
* You may also have noticed that the difference between each memory address is 4 bytes. This because (usually) `int` occupies 4 bytes.

### Pointer Arithmetic

Since `p_arr` stores the address of the first element, you can access `arr[0]` via dereferencing `*p_arr`.

What about `arr[1]`? The cool thing about pointer arithmetic is that it's **type-aware,** you can simply perform `(p_arr + 1)` to move forward by 1 element of type `int` (i.e 4 bytes).

Functionally,

<table><thead><tr><th width="243.72222900390625">Operation</th><th>Result</th></tr></thead><tbody><tr><td><code>*p_arr</code></td><td>Move <code>p_arr</code> by 0 bytes, dereference to get <code>arr[0]</code></td></tr><tr><td><code>*(p_arr + 1)</code></td><td>Move <code>p_arr</code> by 4 bytes, dereference to get <code>arr[1]</code></td></tr><tr><td><code>*(p_arr + 2)</code></td><td>Move <code>p_arr</code> by 8 bytes, dereference to get <code>arr[2]</code></td></tr></tbody></table>

Note that `arr` also decays into a pointer to the first element that shares the same memory address as `p_arr`, hence operations that can be done with `arr` can also be done with `p_arr`

{% code title="main.cp" %}

```cpp
#include <iostream>

int main() {
    int arr[3] = {10, 20, 30};
    std::cout << arr[0] << std::endl; // 10
    std::cout << arr[1] << std::endl; // 20
    std::cout << arr[2] << std::endl; // 30
    
    std::cout << *(arr + 0) << std::endl; // 10
    std::cout << *(arr + 1) << std::endl; // 20
    std::cout << *(arr + 2) << std::endl; // 30
    
    int* p_arr = arr;
    std::cout << *(p_arr + 0) << std::endl; // 10
    std::cout << *(p_arr + 1) << std::endl; // 20
    std::cout << *(p_arr + 2) << std::endl; // 30
}
```

{% endcode %}

<details>

<summary>😎 Cool Fun Fact</summary>

We've learnt that we can access `arr[0]` via `*(arr + 0)`. Since addition is commutative, `*(arr + 0) == *(0 + arr)`, hence `0[arr]` works too!

{% code title="main.cpp" %}

```cpp
#include <iostream>
int main()
{
    int arr[3] = {10, 20, 30};
    std::cout << 1 [arr] << std::endl;
    std::cout << arr[1] << std::endl;
}
```

{% endcode %}

</details>

### Strings

Fundamentally, a string is simply an *array* of characters ending with a special character `\0` (i.e null terminator)

The usual pointer arithmetic rules that we've learnt above apply too.

{% code title="main.cpp" %}

```cpp
#include <iostream>

int main() {
    char str[] = "hello";
    std::cout << str[0] << std::endl;        // 'h'
    std::cout << *(str + 1) << std::endl;    // 'e'
    
    char* str2 = "world";
    std::cout << str2[0] << std::endl;        // 'w'
    std::cout << *(str2 + 1) << std::endl;    // 'o'
}
```

{% endcode %}

Notice when we compile, we get a warning:

<figure><img src="/files/EgqQw5eSqqqZozino1pr" alt=""><figcaption></figcaption></figure>

<details>

<summary>😕 Why the warning for <code>char* str2</code> but not <code>char str[]</code> ?</summary>

`"hello"` is a string literal stored in read-only memory.

`char[]` has no issues since it copies the string literal onto its own array on the stack.

`char *` is pointing to the same string literal in read-only memory. Writing to it causes undefined behaviour hence the warning.

In fact, if you try compiling the program with stricter flags: `clang++ -std=c++17 -Wall -Wextra -pedantic-errors main.cpp -o hello-world`, the compiler rejects the code entirely.

<figure><img src="/files/G5gAC58wsSVOmeQ4dbUj" alt=""><figcaption></figcaption></figure>

To squash this, we can use `const char *` to assert that we are not going to modify the string.

</details>


# Memory Model

### Memory Layout

Variables can either be *created* or *destroyed.* They can also exists on different locations (i.e the **stack**, **heap** or **static storage)**

<figure><img src="/files/BSBmXX1ZhR80lcmWSFYz" alt=""><figcaption></figcaption></figure>

### Stack vs Heap

#### Stack

Variables created (i.e allocated) on the stack possess automatic storage duration. This means they are:

* allocated when execution enters their scope
* deallocated when out of scope

<details>

<summary>Why is stack allocation fast?</summary>

<figure><img src="/files/Ma2Gi8vMJ01mZ8oExuSI" alt=""><figcaption><p>src: <a href="https://chessman7.substack.com/p/how-your-code-executes-a-guide-to">https://chessman7.substack.com/p/how-your-code-executes-a-guide-to</a></p></figcaption></figure>

Allocating (and deallocating) variables on the stack is fast as it mainly involves incrementing (and decrementing) the stack pointer.

This is also the reason why uninitialized variables on the stack possess garbage values because the memory they occupy is not automatically zeroed.

</details>

To jog your memory, all the previous variable declarations in [Variables](/hackerschool/introduction-to-cpp/cpp-fundamentals/variables) and [Pointers](/hackerschool/introduction-to-cpp/memory-and-ownership/pointers) were all allocated on the stack.

{% code title="" %}

```cpp
int main () {
    int x = 5;
    int* p = &x;
    int arr[3] = {10, 20, 30};     
    // this copies the string literal onto the array on the stack
    char str[] = "hello";
    
    // the pointer str2 is on stack
    // the string literal world is stored in ROM
    const char* str2 = "world"; 
}
```

{% endcode %}

Consider the following example (please ignore the implementation details of `Person` class for now)

{% code title="main.cpp" %}

```cpp
#include <iostream>

class Person
{
private:
    int age_;

public:
    Person(int age) : age_(age)
    {
        std::cout << "creating a person with " << this->age_ << std::endl;
    }
    int get_age()
    {
        return this->age_;
    }
    ~Person()
    {
        std::cout << "destroying a person with " << this->age_ << std::endl;
    }
};

int main()
{
    Person p1(10);
    Person p2(15);
    Person p3(20);
}
```

{% endcode %}

The following output is as shown:

<figure><img src="/files/eDrfusiYSjgPmekkg9gp" alt=""><figcaption></figcaption></figure>

Observe that similar to the Last-In-First-Out (LIFO) behaviour of literal stack data structure, objects are constructed in order of declaration. They are also destroyed in order of reverse order.

<figure><img src="/files/cmpDEp9uqmsq5v44zFyY" alt=""><figcaption></figcaption></figure>

Note that the behaviour is tied to scope, objects are destroyed as soon as it exits the scope.

<table><thead><tr><th>Code</th><th>Output</th></tr></thead><tbody><tr><td><pre class="language-cpp" data-title="main.cpp"><code class="lang-cpp">int main()
{
    Person p1(10);
    {
        Person p2(15);
    }
    Person p3(20);
}
</code></pre></td><td><img src="/files/9bqjIT0BvQvXCNLuTFxp" alt="" data-size="original"></td></tr></tbody></table>

### Heap

While the stack is fast, it does suffer from a few limitations:

* Size of the objects must be known / determinable at compile time
* The lifetime of objects is tied to scope

<details>

<summary>⚠️ More about the limitation</summary>

Consider the following code snippet:

```cpp
int main()
{
    int n;
    std::cin >> n;
    int arr[n];
}
```

It *appears* to work when we try running with `clang++ main.cpp -o hello-world`

But look what happens when I try to run with stricter flags: `clang++ -std=c++17 -Wall -Wextra -pedantic-errors main.cpp`

![](/files/xsR36NMqCNJp64FEfmIp)

This is because VLA (Variable-Length Arrays) are actually not part of standard C++. Indeed, the size of arrays **must** be known at compile-time.

Following the above school of thought, why does the below code still not work? 🤔

```cpp
int main()
{
    int n = 5;
    int arr[n];
}
```

<figure><img src="/files/2UDfgJedU746zoJKSVM3" alt=""><figcaption></figcaption></figure>

</details>

You can use a `new` keyword to allocate memory on the heap. Unlike the stack, the heap memory is not tied to scope. It persists until it is explictly deallocated with `delete.`

Let's take a look at how heap tackles these limitations:

{% code title="main.cpp" %}

```cpp
#include <iostream>

int main()
{
    int n;
    std::cin >> n;
    int* arr = new int[n];
    for (int i = 0; i < n; i++) 
        arr[i] = i;
    for (int i = 0; i < n; ++i)
        std::cout << arr[i] << " ";
}
```

{% endcode %}

<figure><img src="/files/9vFn1Pwa6ExoqvjFGUZB" alt=""><figcaption></figcaption></figure>

<details>

<summary>⚠️ <strong>Spot</strong> the error in the above code snippet</summary>

We've just encountered a common pitfall of allocating variables on the heap. The above code snippet is plagued by a **memory leak.**

The memory allocated using `new` is never `deallocated` using `delete` . This means the allocated memory remains reserved after after it's no longer needed.

We can fix the code by appending `delete[] arr;` to the back of the program like so:

{% code title="main.cpp" %}

```cpp
#include <iostream>

int main()
{
    int n;
    std::cin >> n;
    int* arr = new int[n];
    for (int i = 0; i < n; i++) 
        arr[i] = i;
    for (int i = 0; i < n; ++i)
        std::cout << arr[i] << " ";
    delete[] arr;
}
```

{% endcode %}

</details>

#### Dangling Pointers

Another common pitfall that programmers often stumble upon is the issue of dangling pointers. This refers to the situation when we try to access memory that already has been deallocated.

{% code title="main.cpp" %}

```cpp
#include <iostream>
int main() {
    int *p = new int(5);
    std::cout << *p << std::endl;
    delete p;
    std::cout << *p << std::endl; 
}
```

{% endcode %}

The above is an example of [**undefined behavior**](https://stackoverflow.com/questions/28727439/is-it-undefined-behavior-to-dereference-a-dangling-pointer)**.** For me, it prints 0 but the C++ standard makes no guarantees about what happens.

<figure><img src="/files/6lq4lapF0fcrAyHpXQLc" alt=""><figcaption></figcaption></figure>

It could also potentially print 5, print garbage values or crash through a segmentation fault.


# Memory Painpoints

Fundamentally, ownership answers the question:

> Who is responsible for cleaning up this piece of memory?

As we've seen earlier, memory can be allocated on two distinct regions (i.e the stack and the heap)

### Stack

When variables are allocated on the stack, there's no burden on us to deallocate the memory. C++ handles it for us automatically!

{% code title="main.cpp" %}

```cpp
int main() {
    int x = 10;
} // x is automatically destroyed by end of this scope
```

{% endcode %}

### Heap

#### Exercise

I tried my best to write a code snippet to simulate a game I had in mind. Unfortunately, the code doesn't seem to work for some reason. Do you mind helping me find out where the error(s) are ?

{% code title="main.cpp" %}

```cpp
#include <iostream>

int* spawnScoreBonus() {
    return new int(100); 
}

int* generatePlayerHealth() {
    int* health = new int(50);
    delete health;    
    return health;
}

int main() {
    int* score = spawnScoreBonus();
    std::cout << "Player gained score: " << *score << std::endl;

    int* health = generatePlayerHealth();
    std::cout << "Player health: " << *health << std::endl; 

    int* enemyHp = new int(200);
    int* archerTarget = enemyHp;
    int* knightTarget = enemyHp;
    delete knightTarget;
    delete archerTarget;
    
    return 0;
}
```

{% endcode %}

<details>

<summary><span data-gb-custom-inline data-tag="emoji" data-code="26a0">⚠️</span> Memory Issue 1</summary>

The `spawnScoreBonus()` function allocates a new block of memory containing `100` . Ownership of this object is transferred from `spawnScoreBonus()` to `main()` .

This means the onus is on `main()` to `delete` the object but `score` is never deallocated, leading to a **memory leak**!

</details>

<details>

<summary><span data-gb-custom-inline data-tag="emoji" data-code="26a0">⚠️</span> Memory Issue 2</summary>

In the `generatePlayerHealth()` function, `health` is deleted before being returned. This means the returned pointer is pointed to free'ed memory.

However, we are still trying to access the value `health` is pointing to in the `main()` function, leading a **dangling-pointer /** **use-after-free.**

</details>

<details>

<summary><span data-gb-custom-inline data-tag="emoji" data-code="26a0">⚠️</span> Memory Issue 3</summary>

Both `archerTarget` and `knightTarget` both point to the same integer of 200. So both pointers behave like owners and both delete the same blokc of memory.

This leads to a **double-free** scenario where both owners think they own the shared resource.

</details>

As we have seen earlier, even in a small piece of code, there can be many subtle memory issues. Imagine how many of such bugs exist in larger codebases!

To tackle this, modern C++ (since C++11) introduced something a little smarter...


# Smarter Pointers

As we've seen, we humans are pretty dumb and inept at managing memory.

Here's where smart pointers come in! They are smarter abstractions introduced in C++11 to help us manage memory automatically.

### Unique Pointers

#### Basics

{% code title="main.cpp" %}

```cpp
#include <iostream>
#include <memory>

int main() {
    std::unique_ptr<int> score = std::make_unique<int>(100);
    std::cout << *score << std::endl;
} // automatically deleted here
```

{% endcode %}

Semantically, unique pointers convey the idea there is **exactly only one owner** of the resource being pointed to.

Unique pointers:

* cannot be copied
* can be moved
* are automatically deleted when it goes out of scope

#### Transferring Ownership

{% code title="main.cpp" %}

```cpp
#include <memory>

void f(std::unique_ptr<int> p) {
    // owns the pointer
}

int main() {
    auto x = std::make_unique<int>(50);
    f(std::move(x));  // transfer ownership
}
```

{% endcode %}

### Shared Pointers

{% code title="main.cpp" %}

```cpp
#include <iostream>
#include <memory>

int main() {
    auto p1 = std::make_shared<int>(10);
    auto p2 = p1;  // shared ownership

    std::cout << *p1 << std::endl; // 10
    std::cout << *p2 << std::endl; // 10
}
```

{% endcode %}

Both `p1` and `p2` point to the same object. Memory for the integer 10 is only free'ed when both `p1` and `p2` goes out o scope.

Internally, a `shared _ptr` uses a reference count to track when it shoud deallocate. We can see the live reference count by invoking `use_count().`

{% code title="main.cpp" %}

```cpp
#include <iostream>
#include <memory>

int main() {
    auto p1 = std::make_shared<int>(10);
    std::cout << p1.use_count() << std::endl; // 1

    auto p2 = p1;
    std::cout << p1.use_count() << std::endl; // 2
}
```

{% endcode %}

### Tackling Our Original Code

#### Exercise

Given the original code in [Memory Painpoints](/hackerschool/introduction-to-cpp/memory-and-ownership/memory-painpoints), can you try to use some of these abstractions to create better, memory-safe code?

<details>

<summary>Answer</summary>

{% code title="main.cpp" %}

```cpp
#include <iostream>
#include <memory>

std::unique_ptr<int> spawnScoreBonus() {
    return std::make_unique<int>(100);
}

std::unique_ptr<int> generatePlayerHealth() {
    return std::make_unique<int>(50);
}

int main() {
    std::unique_ptr<int> score = spawnScoreBonus();
    std::cout << "Player gained score: " << *score << std::endl;

    std::unique_ptr<int> health = generatePlayerHealth();
    std::cout << "Player health: " << *health << std::endl;

    std::shared_ptr<int> enemyHp = std::make_shared<int>(200);
    std::shared_ptr<int> archerTarget = enemyHp;
    std::shared_ptr<int> knightTarget = enemyHp;

    return 0;
}
```

{% endcode %}

</details>


# JavaScript

## About the Guide

This guide was created as an effort by [NUS Hackers](https://hckr.cc/) to make knowledge easily available for various technical topics!

## About this Section

This section of the guide will cover JavaScript, from its syntax and constructs to its use in frontend web development. Below are the slides containing an abridged version of the guide (originally intended to accompany a live workshop), along with a link to a repository that holds some sample code as well as sample solutions to exercises posed throughout this guide.

### Slides

[JavaScript\_HTML\_CSS Notes for Orbital 2024.pdf](https://github.com/nushackers/wiki/files/15285413/JavaScript_HTML_CSS.Notes.for.Orbital.2024.pdf)

### Code

[Code](https://github.com/Dinoman44/Orbital2024-JS-example-problems-code)

## What is JavaScript?

JavaScript[^1] is a high-level programming language that is often combined with HTML and CSS to enhance the frontend of a browser-based application. Its main uses are interactive elements, frontend input validation and fetch requests.

JavaScript as a language has influenced web development (and programming as a whole) a lot since its inception, with other languages being developed to add functionality to it (such as TypeScript, which is a typed version of JavaScript).

What makes JavaScript rather unique is that it does not have its own compiler interpreter. Java has the JVM, Python has its installable interpreter and C has the clang compiler. But JavaScript instead relies on a web browser for execution.

Another feature is non-mandatory semi-colons at the end of each line: you can either omit or leave in a semi-colon at the end of each line in JavaScript. The browser will still be able to understand it either way. However by convention it is recommended to place semi-colons at the end of every line, as this guide will follow.

## Getting started

Do you have a web browser that is not Internet Explorer? Do you have an IDE (or other text/code editor) installed on your computer? If you answered yes to both questions, then you are ready to started coding! If not, then install a browser as well as a text editor or IDE of your choice. The browser of choice is Mozilla Firefox, but feel free to use any browser you like.

### Next steps

Next, we will look at how the Browser Developer Tools will aid us throughout the development cycle when using JavaScript, HTML and CSS (to be covered in brief in this guide).

[^1]: No relation to Java


# Browser Developer Tools

For this guide, Mozilla Firefox will be our browser of choice because of its powerful, wide-ranging and easy-to-use developer tools. Feel free to use any browser of your choice, keeping in mind that not all features may be available in the same form as with Firefox.

## Removing distractions

When first learning JavaScript, it is customary to open a blank page on your browser to give your Browser Console a clean slate. To do so, open a new tab in your browser and type the following:

```
about:blank
```

and press enter. You should see a completely blank white (or black, based on your settings) page.

<figure><img src="/files/19vxmg63Q8RF9O7Yvdd1" alt=""><figcaption><p><em>Blank page in Firefox (dark mode)</em></p></figcaption></figure>

## Opening the Developer Tools

To open up the Browser Developer Tools, navigate to a blank page and just use the keyboard shortcut `f12` or `fn + f12` (based on your keybinds). This should open up the Dev Tools window, which is resizable and can be placed on any side of the screen you want (except the top).

<figure><img src="/files/TKpSfTAZhdT7I8dNWksy" alt=""><figcaption><p><em>Dev Tools in Firefox</em></p></figcaption></figure>

## Navigating the Tools Menu

You'll see a few tabs at the top of the window labelled with names like `Inspector`, `Console` and `Style Editor` (depending on the browser). Navigate to the `Console` tab by clicking on it. You should see a, well, console with your typing cursor targetted on the first line. This is where you will write all your JavaScript code for the first part of this guide.

<figure><img src="/files/Gn8QN7UjUX5fHX0nZWbh" alt=""><figcaption><p><em>The Console in Firefox</em></p></figcaption></figure>

## Next steps

Next you'll get started with using the console to execute some JavaScript code.


# Getting Started

## Input and Output

It's convention to write a program to print out "hello world" when first learning a language, so let's do exactly that. In the browser console that you opened up earlier, type in the following:

```js
console.log("hello world");
```

And hit enter. You should see "hello world" printed (without the quotation marks) in the line below, and `undefined` just below it (on Firefox). Here, `undefined` is the return value of the expression that you just evaluated. In general, assignment and output expressions return `undefined`, as do functions that have no return value.

<figure><img src="/files/Xwe5OjYNDPcq8JC0dsfc" alt=""><figcaption></figcaption></figure>

This is one way to give output. The second way is to use the `alert` function. Type in the following in the console:

```js
alert("hello world");
```

And hit enter again. Now instead of some output in the console, you should see a popup window with the message "hello world". Clicking "ok" will result in the function evaluation completing, which results in the function returning `undefined` as before.

<figure><img src="/files/rtixezPdmiGkMm689Qq1" alt=""><figcaption></figcaption></figure>

Now let's try to take some user input. JavaScript provides the `prompt` function, which takes in a string to prompt the user with, and returns the user's input. It works like `alert`: a pop-up appears on the page, except this time the user can also type in some text into an input field.

```js
let x = prompt("Enter a number:"); // 'let' declares a variable, see below
console.log(x); // prints out the number the user typed in
```

<figure><img src="/files/SmNe3c6U66MtLK3AMZDI" alt=""><figcaption></figcaption></figure>

## Variable declaration

To declare a variable in JavaScript, you can either use the `let` keyword or the `var` keyword. The key difference between the two is that declaring a variable with the `let` keyword will only allow it to be used within the code block it was declared in, whereas a variable declared with the `var` keyword will allow it to be used within its parent function block and overrides any previous declaration. This is explained in detail [here](https://sentry.io/answers/difference-between-let-and-var-in-javascript/).

To get started, let's declare two variables, `x` and `y` and assign them values of `5` and `10`.

```js
let x = 5;
let y = 10;
```

Now we can check what their values are by printing them out as before, or by just entering the name of the variable:

```js
x; // will return 5
y; // will return 10
```

As you can guess, `//` is used to begin a comment. To write comments spanning multiple lines, start a comment with `/*` and end it with `*/`.

## Next steps

Next, we'll look at some data types and values in JavaScript.


# Datatypes

## Types in JavaScript

JavaScript is dynamically-typed and weakly-typed language, meaning you do not need to declare a variable's type in advance, you can reassign a variable to any type, and implicit type conversion occurs automatically when possible.

```js
let x; // no value assigned; by default undefined
x = 10; // x is now 10, a number
x = "hello"; // x is now "hello", a string
x = true; // x is now true, a boolean
```

The following page has some of the datatypes in JavaScript.

### Number

The `Number` datatype in JavaScript is a primitive datatype that represents, surprise surprise, a number. Unlike many other languages, JavaScript does not explicitly differentiate between integers and floats and any such value just has the type `Number`.

Some examples of Numbers are `0`, `1`, `-1.32`, and `Infinity`. `Infinity` is a special value used by JavaScript to indicate an infinite value, such as the result of dividing anything by 0.

JavaScript numbers can be formatted in scientific notation as follows:

```js
let x = 23e3; // x is 23 times 10^3
let y = 4e-5; // y is 4 times 10^-5
```

### Boolean

The `boolean` datatype in JavaScript is another primitive datatype that can take on one of two values: `true` or `false`. These values are useful in any code involving predicates.

### String

The `String` datatype in JavaScript is a compound datatype that is essentially any text enclosed in quotation marks (double or single are both fine, as long as the start and end quotation marks match).

```js
let str = "Hello world!";
```

Strings are immutable, which means that any function that takes in a string does not change its value; it instead returns a new string.

A particular character in a string can be accessed using square brackets (`[]`) wrapped around a number representing the position of the character, called the index. The first character is said to be at index 0. There are other ways to access a particular character in a string which will be discussed later.

```js
let str = "H0LA";
console.log(str[1]); // will print out "0"
```

The length of a string can be found using the `.length` property:

```js
let str = "orbital";
str.length; // returns 7
```

### Array

The `Array` datatype in JavaScript is compound mutable datatype that represents an ordered collection of values. The values can be of any type, and values of different types can be assigned to the same array without an issue. An array is declared by separating the values with commas (`,`) and enclosing the list with square brackets `[]`. As with strings, you can get the length of the array with the `.length` property, and access a particular value with square bracket indices. Arrays can also be nested if need be.

```js
let arr = [1, -2, 3, true, "random string here", ["another array", 4]];
arr[0]; // returns 1
arr[4]; // returns "random string here"
arr[5]; // returns ["another array", 3]
arr[5][1]; // returns 4
arr.length; // returns 6
```

As you saw above, indexes can be chained in the case of nested arrays.

### Objects

An `Object` in JavaScript refers to a container variable that contains many values. By this definition, an array is an object. In JavaScript, objects are declared and assigned using JavaScript Object Notation, or JSON. You can imagine an object as a set of key-value pairs.

```js
let myObj = {firstName: "Prakamya", lastName: "Singh", year: 1, isComputingStudent: true};
```

As you can see, the key-value pairs are separated by commas (`,`), enclosed by curly braces (`{}`) and the pairs are formatted as `key: value`. Note that the key does not have to be enclosed in quotation marks as long as it has no spaces, operators(`+`, `=` etc.) and other reserved characters or keywords.

To access a value assigned to a key (formally called a **property** in JS) you can either use square brackets (`[]`) or dot notation:

```js
myObj["firstName"]; // returns "Prakamya"
myObj.firstName; // also returns "Prakamya"
```

Note that with dot notation, you do not need quotation marks whereas with square bracket notation you do need quotation marks.

## Some other primitive values

### undefined

In JavaScript, `undefined` is a primitive value that is returned by functions or expressions that have no return value as mentioned in the previous section. It is also a default value that is assigned by JavaScript to a variable that has been declared without assignment. For instance,

```js
let x;
```

will assign `undefined` to `x` until a different value is assigned later on.

{% hint style="warning" %}
`undefined` can also be assigned to a variable, strangely enough.
{% endhint %}

### null

This is another primitive value that can be assigned to variables or returned by functions or expressions. It is used to represent an empty, non-existent value rather than as a flag to indicate no return value.

{% hint style="info" %}
Treat `null` like a "nothing here" symbol and `undefined` as a "nothing here *yet*" symbol.
{% endhint %}

### NaN

This primitive value stands for "Not a Number" and is used by JavaScript to indicate that the result of an expression or function is not a number. One way to get this is to divide a number by a string that does not have numeric digits. While most languages would throw an error or exception of some kind in such situations, JavaScript instead handles it by returning `NaN` to indicate that the expression was erroneous.

## How to determine the type of a value/variable

There are two methods to determine the type of a variable in JavaScript. The first method is more common, and involves using the `typeof` function which takes in any value as an argument and returns a string with the data type.

```js
// Let's declare some variables first
let num = 10;
let otherNum = 9.9;
let str = "some string";
let bool = true;
let arr = [1, false, "hello"];
let obj = {car: "Puma", licensePlate: "SLG2034B", topSpeed: 120};

// Now let's check their data types using the typeof function.
typeof(num); // returns "number"
typeof(otherNum); // still returns "number"
typeof(str); // returns "string"
typeof(bool); // returns "boolean"
typeof(obj); // returns "object"
typeof(arr); // returns "object", not "array"
```

As you'll notice, `typeof` returns `"object"` when you pass in an array, even though they are essentially different. So the second way is to use the `instanceof` keyword. Here's how it works:

```js
// assume we have our same variables as before
arr instanceof Array; // returns true
arr instanceof Object; // still returns true
obj instanceof Object; // returns true
obj instanceof Array; // returns false
```

Now we can differentiate between objects and arrays. But the `instanceof` keyword has its own issues:

```js
// assume we have our same variables as before
num instanceof Number; // returns false
str instanceof String; // returns false
bool instanceof Boolean; // returns false
```

Whelp, 10 isn't a number apparently. Thanks `instanceof`.

Moral of the story: Based on what you are trying to achieve, make sure you use the right technique to determine a variable's type. For instance, if I had to validate some input to make sure it is a number, I would use the `typeof` operator and check if it outputs "number". But if I had to instead check if my input is an array specifically, I would need to use the `instanceof` keyword as above.

{% hint style="info" %}
`typeof(undefined)` returns "undefined"
{% endhint %}

{% hint style="warning" %}
`typeof(null)` returns "object"
{% endhint %}

{% hint style="danger" %}
`typeof(NaN)` returns "number"
{% endhint %}

## Next steps

Next, we'll take a look at operations and some operators in JavaScript, along with what datatypes can be used with them.


# Operators and Operations

Operators in JavaScript are symbols that can perform operations on certain values and data types. The most basic example are the arithmetic operators: `+`, `-`, `*` and `/`. Operators each have a corresponding operation, can be binary or unary, and can accept a fixed set of values

{% hint style="info" %}
A binary operator is one that can operate on two values, and a unary operator can operate only on one value
{% endhint %}

{% hint style="info" %}
A value that an operator performs an operation on is called an **operand**
{% endhint %}

The following section has a few tables of operators, their corresponding operations and their accepted operand(s).

## Arithmetic operators

These are the classic addition, subtraction, multiplication and division operators that are nearly standard across languages:

* `+` for addition
* `-` for subtraction
* `*` for multiplication
* and `/` for division

These are all binary operators that take two numbers in as operands.

As an aside, the `+` operator also allows for string concatenation and array concatenation:

```js
let str1 = "hello";
let str2 = "world";
str1 + str2; // evaluates to "helloworld"

let arr1 = [1, 2];
let arr2 = [3, 4];
arr1 + arr2; // evaluates to [1, 2, 3, 4]
```

### Other math operators

* The `**` operator allows for exponents:

```js
let x = 2;
let y = 3;
x ** y; // 8
```

* The `%` operator gets the remainder after division

```js
let x = 5;
let y = 3;
x % y; // 2
```

## Comparison operators

These operators allow for, as you can guess, comparing two values. They are binary operators that return a boolean value. The symbols are as follows, with their usage with numbers being as expected:

* `a > b` returns `true` if `a` is greater than `b`
* `a >= b` returns `true` if `a` is greater than *or equal to* `b`
* `a < b` returns `true` if `a` is lesser than `b`
* `a <= b` returns `true` if `a` is lesser than *or equal to* `b`
* `a !== b` returns `true` if `a` is not equal to `b`

The comparison operators can also be used to compare two strings; they return `true` or `false` based on a character-by-character comparison of the two strings. An example is below:

```js
let str1 = "abcd";
let str2 = "abdc";
str1 > str2; // returns false
```

The way the above works is that the first character in each string are compared. If they are equal, then the next character in each string are compared. Eventually, when two characters in a string are unequal, then they are compared according to the operator (after internally converting the characters to their numeric ASCII values) and a boolean value returned accordingly.

If one string ends before the other and all characters up till that point are the same, the longer string is deemed the "greater" value:

```js
"abcd" > "abc"; // returns true
```

### Equality operators

The equality operation has two possible operators, each of which function slightly differently.

The first is the "triple-equals" or the "type-strict equals" operator: `===`. This operator works as you would expect:

```js
2 === 2; // returns true
2 === 3; // returns false
2 === "abcd"; // returns false
2 === "2"; // returns false
2 === [2]; // returns false
```

The second is the "double-equals" or the "type-lax equals" operator: `==`. This operator works similar to the type-strict equals operator, except that it implicity converts the two operands to the same datatype:

```js
2 == 2; // returns true
2 == 3; // returns false
2 == "abcd"; // returns false
2 == "2"; // returns true, because "2" is converted to 2
2 == [2]; // returns true, because [2] is converted to 2
2 == [2, 2]; // returns false, because there is more than one value in the array now
```

{% hint style="info" %}
It is good practice to stick to more strict typing in your program, to prevent the chance of errors propagating. On the other hand, sometimes it may be prefered to use the type-lax equals, such as in cases where a value has to store the number 2 but it is irrelevant whether it is in a string, array, or as a primitive number. But such cases are rare.
{% endhint %}

## Logical operators

These operators can take one or two boolean expressions and returns a boolean value. They are often combined with comparison operators.

### AND

The AND operator, `&&` (double ampersand symbol), is a binary operator that compares two boolean values expressions and returns `true` **only if both the expressions** evaluate to `true`. If either one of the expressions evaluate to `false`, then `&&` returns `false`.

```js
2 === 2 && 3 < 4; // true because 2 is equal to 2 AND 3 is less than 4
2 === 2 && 3 > 4; // false because 3 is not greater than 4
2 === 3 && 3 > 4; // false
```

### OR

The OR operator, `||` (double bars), is a binary operator that compares two boolean expressions and returns `true` if either one of them evaluates to `true`. If both the expressions evaluate to `false`, then the operator returns `false`.

```js
2 === 2 || 3 < 4; // true
2 === 2 || 3 > 4; // true because 2 is equal to 2
2 === 3 || 3 > 4; // false because 2 is not equal to 3 and 3 is not greater than 4
```

### NOT

The NOT operator, `!`, is a unary operator that reverses a boolean expression's value. If the expression evaluates to `true`, it returns `false`; if the expression evalutes to `false` then it returns `true`. It does not change the original expression's value

```js
let value = 2 === 2 || 3 < 4; // true
!value; // false
value; // still true
```

## Assignment operators

### Value assignment operator

As seen before, this operator has the symbol `=` and allows you to assign a value to a variable or constant. Example:

```js
let x = 10; // assigning 10 to x
```

### Operation assignment operators

This class of operators are formed by combining a binary logical or mathematical operator with the value assignment operator: `op=`. They can then be used as an assignment operator, assigning a value to a variable while performing the binary operation on both. Essentially: `a op= b` is the same as `a = a op b`. Some examples are below:

```js
let x = 10;
x += 5; // same as x = x + 5; x is now 15
x -= 2; // same as x = x - 2; x is now 13
x *= 10; // same as x = x * 10; x is now 130
x **= 2; // same as x = x ** 2; x is now 16900
x; // 16900

let y = x > 1000; // 16900 is greater than 1000, so y is true
y &&= (x !== 4); // same as y = y && (x !== 4)
// x is not equal to 4, so the expression in brackets evaluates to true
y; // true && true gives true
```

### Increment and decrement operators

These two unary operators allow to increment and decrement numeric variable values by 1. The increment operator is a double plus (`++`) and the decrement operator is a double minus (`--`). They can be placed either behind or in front of a variable name, and are accordingly called pre- or post-increment or decrement.

```js
let m = 4;
m++; // post-increment, m is now 5
m--; // post-decrement, m is back to 4
--m; // pre-decrement, m is now 3
++m; // pre-increment, m is back to 4 again
```

{% hint style="info" %}
The difference between the pre- and post- version of these operators is to do with their return values:

```js
let x = 10;
let y;
y = ++x; // x is 11, and y is also 11
y = x++; // x is 12, but y remains 11 because the increment happens after assignment
```

{% endhint %}

## Bitwise operators

These operators allow you to perform bitwise operations like bitwise AND, OR and NOT on values. To see how they work in detail, visit [this site](https://www.w3schools.com/js/js_bitwise.asp).

## Type conversion

Recall that JavaScript is a weakly-typed language. This means that you can end up with situations like this:

```js
"10" + 1; // "101" because 1 gets converted to a string and JS performs string concatenation
"10" - 1; // 9 because "10" gets converted to a number and JS performs subtraction
4 / "2"; // 2 because "2" gets converted to a number
"3" * "2"; // 6 because JS converts both operands to numbers
[3] / "10"; // 0.3, same reason
```

This is one of the reasons it is important to maintain type consistency and stronger typing in your code. It is also wacky issues like this that contributed to the popularity of languages like TypeScript, a strongly-typed version of JavaScript.

## Next steps

Now that we have covered data types and operations concerning these datatypes, the next section will cover a few more coding constructs of JavaScript.


# Loops and Conditions

Now that we have understood the basic datatypes and operations of JavaScript, we can start learning about other constructs. This section will cover loops and conditions.

{% hint style="info" %}
To clear the code in the console, type and enter `clear();` or hit `ctrl` + `L`. This will still keep any variables you declared earlier.
{% endhint %}

{% hint style="info" %}
To write multiple lines without executing them one at a time, hit `ctrl` + `enter` (or `cmd` + `return` for Mac) after every line instead of just `enter`. This will be useful for the this and the next few sections.
{% endhint %}

## Conditionals

Conditionals are a set of statements that execute different blocks of code based on the evaluation of one or more boolean expressions. In simpler words, if something is true then one thing happens otherwise something else happens.

The syntax for conditional statements in JavaScript is as follows:

```
if (booleanExpression) {
    // do thing 1
} else if (otherBooleanExpression) {
    // do thing 2
} else {
    // do something else
}
```

Example:

```js
let num = prompt("Enter a number"); // recall that this allows the user to input a value
if (num === null) { // prompt() returns null if the user gives no input
    alert("You did not enter a number");
} else if (num % 2 === 0) {
    alert("The number you entered was even");
} else {
    alert("The number you entered was odd");
}
```

<figure><img src="/files/3voOcNFJgZNGUUCmlirA" alt=""><figcaption></figcaption></figure>

### Ternary operation

This is an operation that operates on 3 operands, specifically to allow the evaluation of a conditional statement in a single line. Like any other operation, ternary conditions can be nested. The syntax is as follows:

```
predicate ? returnIfTrue : returnIfFalse;
```

Example:

```js
// assume num is guaranteed to be a number
num % 2 === 0 ? "Even" : "Odd"; // if num is even then "Even" is returned, else "Odd" is returned
```

## Loops

A loop is a block of code that gets executed repeatedly until an end condition is satisfied. There are two types of loops in JavaScript, the `for` loop and the `while` loop. The syntax for each is as follows:

```
for (counter; limit; step) {
    // do something
}

while (condition) {
    // do something
}
```

An example of a for loop is as follows:

```js
for (let i = 0; i < 11; i++) {
    console.log(i);
}
```

The loop above will print out numbers from 0 to 10 one by one. As you can see, first the counter is declared (`let i = 0`), then the limit is defined (loop keeps executing as long as `i < 11`), and then the step is defined (`i` is incremented by 1 each time; `i++`). The eventual output will look like this:

<figure><img src="/files/dXLE5QDIdq5p9QCSjfGS" alt=""><figcaption></figcaption></figure>

An while loop that does the same thing as the for loop above would be as below:

```js
let i = 0;
while (i < 11) {
    console.log(i);
    i++;
}
```

Note that here, the counter variable declaration needs to be done outside the loop, and the step is defined inside the loop. This is because the while loop only has a boolean expression in its keyword call. Omitting to declare the counter variable will lead to a `ReferenceError: i is not defined`. On the other hand, omitting the step will cause an infinite loop, since the value of `i` will never change and is always less than 11.

<figure><img src="/files/4jp6FbpOpuZqxCaTUOPn" alt=""><figcaption><p>Firefox's browser console is able to catch errors and pre-empt the results of executing a block of code before you actually run it. In the screenshot above, you can see <code>ReferenceError: i is not defined</code> in grey text, because the program has not yet run but the console knows what the result is going to be. This tells me as the programmer to look for the problem and fix it before running the code.</p></figcaption></figure>

In a for loop, the step can be of any size and can involve valid operation:

```js
for (let k = 1024; k > 1; k /= 2) {
    console.log(k);
}
```

The above code prints out numbers backwards from 1024, each time dividing `k` by 2, as long as `k` is greater than 1.

The counter variable can also be of any type, and you can declare multiple counter variables:

```js
for (let i = 0, j = 15; i < 5; i++, j -= 3) {
    console.log(i, j); // prints the values of i and j separated by a space
}
```

Here `i` increases by 1 each time from 0 to 4 (stopping at 5) while `j` decreases by 3 each time starting from 15 until the loop ends.

<figure><img src="/files/hcHX7OM5feP52B5cD1mc" alt=""><figcaption></figcaption></figure>

A loop involving strings:

```js
let str = "eowjpfi45ofr;kl143";
let i = 0;
while (i < str.length) {
    console.log(str[i]);
    i += 2;
}
```

This loop prints every 3rd character of `str`, starting from the 1st one. The same result can be achieved with a `for` loop:

```js
for (let i = 0; i < str.length; i += 2) {
    console.log(str[i]);
}
```

Similar results can be achieved with an array.

## Next steps

In the next section, we'll look at functions, their types, and how to declare and use them.


# Functions

Functions can be thought of as blocks of code that perform particular tasks. In JavaScript, like in most other programming languages, functions can take in a number of arguments and return a value.

The syntax to declare a function is as follows:

```
function functionName(arg1, arg2, ...) {
    // do something
    return something; // optional
}
```

A function can take any number of arguments, and the [arity](https://en.wikipedia.org/wiki/Arity) of a function is the number of arguments it can take

* A nullary function takes in no arguments,
* A unary function takes in 1 argument,
* A binary function takes in 2 arguments,
* A ternary function takes in 3 arguments, and so on

A function can return any value, or can return no value at all. In this case, the return type of the function is `undefined`. An example is the `console.log` function which logs something to the console and has no return value.

An example of a unary function is below:

```js
function sumToN(n) {
    let sum = 0;
    for (let i = 1; i <= n; i++) {
        sum += i;
    }
    return sum;
}
```

This function uses a `for` loop to calculate the sum of numbers from 1 to its input `n`. You could instead use the formula:

```js
function sumToN(n) {
    return n * (n + 1) / 2;
}
```

To call a function on a value, use the name of the function and pass in the argument(s) (if any) in brackets:

```js
sumToN(10); // returns 55
```

Here are some more functions:

```js
function nullary() {
    console.log("hello");
    return 2;
}

function binary(x, y) {
    let k = x ** y;
    let m = k % y;
    return m + y;
}

function ternary(x, y, z) {
    let a = x + y + z;
    let b = x - y + z;
    return a / b;
}

function noReturnUnary(x) {
    console.log(x);
}
```

{% hint style="info" %}
When calling a function in JS, if you pass in more arguments than is accepted by the function, JS ignores the extra arguments. If you pass in less arguments that is required by the function, JS replaces the missing values with the value `undefined`. This is different from other languages which throw an error if you pass in the wrong number of arguments.
{% endhint %}

## Variadic functions

Sometimes you may want to create a function that can take in a variable number of arguments. These functions are called variadic functions. The syntax to declare a variadic function is as follows:

```js
function variadic(...args) {
    // do something
}
```

In the function body, `args` can be treated as an array with a `length` property and indexing.

Python has a `sum` function that can take in any number of numbers and returns their sum. Let's recreate this in JavaScript:

```js
function sum(...nums) {
    let total = 0;
    for (let i = 0; i < nums.length; i++) {
        total += nums[i];
    }
    return total;
}
```

To use a variadic function, simply call the function and pass in any number of arguments:

```js
sum(3, 43, -4, 7.6, 9); // returns 58.6
```

<figure><img src="/files/HJud2wAF0ydjhOQ59phx" alt=""><figcaption><p>The variadic function <code>sum</code>. Note how the console is able to follow along as I type the function call and shows me the return value before I execute the line.</p></figcaption></figure>

## Lambda functions

Let's go back to our `sumToN` function:

```js
function sumToN(n) {
    return n * (n + 1) / 2;
}
```

Note that we only have one line in the function body, a return statement that performs a relatively simple math operation. Yet just for this one line return statement we need 2 extra lines to declare the function. Fortunately, JavaScript provides a way to shorten such one-line functions using lambda functions. The syntax to declare a lambda function is as follows:

```js
let lambdaFunction = (arg1, arg2, ...) => something;
```

This is equivalent to:

```js
function lambdaFunction(arg1, arg2, ...) {
    return something;
}
```

Since the `let` keyword creates a variable, this means that `lambdaFunction` can be reassigned to a different value. This is bad - what if we accidentally reassign our function to a different value? Instead, we use the `const` keyword, which declares a constant. As the name suggests, constants cannot have their values reassigned once assigned.

Now with all this in mind, let's rewrite out `sumToN` function:

```js
const sumToN = n => n * (n + 1) / 2;
```

Calling a lambda function is same as calling a regular function:

```js
sumToN(4); // returns 10
sumToN(10); // returns 55
sumToN(100); // returns 50500
```

Lambda expressions can have any arity (i.e. can take any number of arguments) and can be used to write functions that span multiple lines by using curly braces `{}`:

```js
const sum = (...nums) => {
    let total = 0;
    for (let i = 0; i < nums.length; i++) {
        total += nums[i];
    }
    return total;
}
```

We have now rewritten the `sum` function from before as a lambda expression. Note that when writing multiline lambda expression, the `return` keyword needs to be present.

{% hint style="info" %}
Lambda expressions work best when there is just one return statement in the function body; if there are multiple lines then it is better to use a regular function declaration.
{% endhint %}

### Alternate syntax for lambda expressions

Lambda expressions can also be declared as below:

```js
// the sum function from earlier
const sum = function(...nums) {
    let total = 0;
    for (let i = 0; i < nums.length; i++) {
        total += nums[i];
    }
    return total;
};

// the sum to n function from earlier
const sumToN = function(n) {
    return n * (n + 1) / 2;
};
```

This is more useful for complex anonymous functions with long bodies.

## Higher order functions

Higher order functions are functions that take in function(s) as argument(s) and/or return a function. A simple example of a higher order function is a mapper function:

```js
function map(func, val) {
    return func(val);
}
```

Here, the `map` function takes in two arguments: a function `func` and a value `val`. It then applies the function `func` to the value `val` and returns the result.

Let's try using this function:

```js
// this function adds 5 to the input
function add5(x) {
    return x + 5
}

map(add5, 4); // returns 9
```

<figure><img src="/files/iErgXUDENlaGxGfNDKLn" alt=""><figcaption></figcaption></figure>

### Using lambda functions

Lambda functions are quite useful when it comes to higher-order functions because they allow to shorten expressions. Let's rewrite the `map` function example above with lambda functions:

```js
const map = (func, val) => func(val);

const add5 = x => x + 5;

map(add5, 4); // returns 9
```

If there is a particular function that is only ever going to be called once during the program execution, then there is no need to assign the function a name. These are called anonymous functions and can use lambda function notation.

Rewriting the above example using an anonymous function:

```js
// assume we have map, but add5 is only going to be used once
map(x => x + 5, 4); // returns 9
```

{% hint style="info" %}
Lambda expressions work very well when it comes to declaring anonymous functions (functions that are only used once in the entire code) and using them for higher-order function evaluation. When an anonymous function needs to have multiple lines, then anonymous lambda expressions can still be used but may look cluttered.
{% endhint %}

## Next steps

Next, we'll take a look at strings in JavaScript and some string methods.


# Strings

Strings, as mentioned [before](https://github.com/nushackers/wiki/blob/main/orbital/javascript/datatypes.md) are any ordered set of characters enclosed in quotation marks.

```js
let str = "hello world";
let spaceship = "artemis";
let strongPassword = "c<Db39)(-2?^#_=11MD.{[a"; // not my password
let emoji = "🦾🦵"; // yes this is valid, try it
```

## Indexing

To access a particular character in a string at a particular position, there are 3 ways:

* Square bracket indexes (`[]`)
* The `charAt` method
* The `at` method

{% hint style="info" %}
Strings are 0-indexed, so the first character in a string is considered index 0 and the last character is considered index `str.length - 1`
{% endhint %}

Let's create a string, and use the 3 methods to compare outputs with different cases:

```js
let name = "dino";
```

#### Square brackets

Usage: `str[i]` where `str` is the string and `i` is a nonnegative integer

```js
name[0]; // returns "d"
name[3]; // returns "o"
// Testing with other indices:
name[]; // no input - Uncaught SyntaxError: expected expression, got ']'
name[-2]; // negative number - returns undefined
name[2.7]; // decimal value - returns undefined
name[32]; // index out of range - returns undefined
```

We can see from the above that passing in any value for `i` that is not a nonnegative integer less than the length of the string results in `undefined`.

#### The `charAt` method

Usage: `str.charAt(i)` where `str` is the string `i` is a number

```js
name.charAt(0); // returns "d"
name.charAt(3); // returns "o"
// Testing with other indices:
name.charAt(); // no input - returns "d"
name.charAt(-2); // negative number - returns "d"
name.charAt(2.7); // decimal value - returns "n"
name.charAt(32); // index out of range - returns ""
```

We can see from the above that:

1. passing in a negative value for `i` returns the first character in the string
2. passing in a decimal value for `i` returns the index at position `floor(i)`
3. passing in a value greater than the length for `i` returns an empty string `""`

#### The `at` method

Usage: `str.at(i)` where `str` is the string `i` is a number

```js
name.at(0); // returns "d"
name.at(3); // returns "o"
// Testing with other indices:
name.at(); // no input - returns "d"
name.at(-2); // negative number - returns "n"
name.at(2.7); // decimal value - returns "n"
name.at(32); // index out of range - returns ""
```

We can see from the above that the `at` method behaves similarly to the `charAt` method except for negative indices. In this case, an index of -1 corresponds to the **last** character in a string, and every negative number after that is counting backwards. Thus an index of -2 is the second-to-last character in the string, -3 is the third-to-last character, and so on.

{% hint style="info" %}
Try passing a negative index that is beyond the valid range for negative indices (like `name.at(-10)` for the above example). What happens?
{% endhint %}

## String methods

Strings are provided with a whole host of functions that can operate on strings. These functions are called string methods, and W3Schools has a an [extensive list](https://www.w3schools.com/jsref/jsref_obj_string.asp) of these functions. For now, we'll look at a few in detail, and how they work.

We've already seen 2 string methods that help to get a character in the string at a particular position: `charAt` and `at`.

Let's look at some string formatting methods.

#### Converting cases

To convert a string to all uppercase, we have the `toUpperCase` method. You can probably guess from this that the method to convert a string to all lowercase is `toLowerCase`.

```js
let name = "Dino";
let nameUpper = name.toUpperCase();
let nameLower = name.toLowerCase();
nameUpper; // "DINO"
nameLower; // "dino"
name; // "Dino"
```

Note that the value of `name` has not changed after applying the string methods to it. This is because strings are immutable, so the methods just create new strings.

### Getting subsections

To get a subsection of a string, called a substring, there are a few methods.

```js
let name = "rocket sloth";
```

#### `slice`

The first is the `slice` method. This takes in two arguments, a `start` and `end`, and returns the characters in the string from the `start` index to just before the `end` index:

```js
name.slice(0, 3); // "roc"
name.slice(4, 10); // "et slo"
```

Ommitting the `end` value will return all the characters from the `start` to the end of the string:

```js
name.slice(3); // "ket sloth"
```

If the `end` value is greater than or equal to the `start` value, an empty string is returned:

```js
name.slice(10, 3); // ""
```

If only one argument `k` is passed in, and the argument is negative, then the last `|k|` characters are returned:

```js
name.slice(-5); // "sloth"
```

#### `substr`

The next is the `substr` method. This takes in two arguments, a `start` and `length`, and returns the first `length` characters from the `start` index inclusive.

```js
name.substr(0, 3); // "roc"
name.substr(4, 10); // "et sloth"
```

Ommitting the `length` value will return all the characters from the `start` to the end of the string:

```js
name.substr(3); // "ket sloth"
```

If the `start` is negative, then the characters are counted backwards:

```js
name.substr(-3, 5); // "t slo"
```

#### `substring`

The last method is the `substring` method. This works like the `slice` method but with one change. If `end` is greater than `start`, then the two are swapped:

```js
name.slice(4, 10); // "et slo"
name.substring(4, 10); // "et slo"
name.slice(10, 4); // ""
name.substring(10, 4); // "et slo"
```

### Checking for substrings

There are a few methods to help check if one string is a substring of another. These are `includes`, `startsWith` and `endsWith`. They are case-sensitive and their names are pretty self-explanatory.

```js
let name = "Rocket Sloth";
name.startsWith("Rock"); // true
name.startsWith("rock"); // false
name.endsWith("th"); // true
name.includes(" "); // true
name.includes(""); // true
name.includes("hijse"); // false
```

## String templating

Let's say we want a simple program that will add two numbers and output their sum in the form "x + y = sum".

```js
let x = 10;
let y = 5;
```

### Method 1:

```js
console.log(x, "+", y, "=", (x + y));
```

Passing in multiple arguments `console.log` causes them to be joined into one string with each argument separated by a space. However this method can only be used when a function is able to take in multiple arguments and knows to concatenate them into a string, like `console.log`. The same code will not work with `alert` (see below).

<figure><img src="/files/IqTFjfza4PJF5ns461Zp" alt=""><figcaption></figcaption></figure>

### Method 2: String concatenation

```js
console.log(x + " + " + y + " = " + (x + y));
```

As you hopefully recall, the [`+` operator](/orbital/readme-1/operations) when applied to two strings (or to a number and a string) performs string concatenation. This results in a single string being passed in to the function to print out. The advantage of this is that string concatenation results in a string value being formed, so it can be passed in to any function, and can be assigned to a variable/constant as needed. However, if I was not careful and omitted the brackets around `x + y` at the end, I would run into a problem (see below).

<figure><img src="/files/1U1Cftk9e2rMa6NKBpWq" alt=""><figcaption></figcaption></figure>

### Method 3: String templating

This method works similarly to string concatenation, but when substituting multiple values into a string it looks more clean and is easier to read. To form a string template, wrap the string in backticks (\`\`) instead of quotes. Then, at the positions of the variables, use curly braces and the `$` sign before the braces, and place the variable/expression between the braces. See below:

```js
console.log(`${x} + ${y} = ${x + y}`);
```

While this has the same effect as before, it looks a lot cleaner and its clearer to anyone reading the code as to what is going on. There is also less chance of operations getting confused.

<figure><img src="/files/2zdy9C7M1U03IEvBi1l2" alt=""><figcaption></figcaption></figure>

## Next steps

Next, we'll look at arrays and some useful array methods.


# Arrays

Arrays, as mentioned [before](https://github.com/nushackers/wiki/blob/main/orbital/javascript/datatypes.md) are any ordered set of values separated by commas and enclosed in square brackets (`[]`).

```js
let arr = [1, 2, 3, 4];
let emptyArr = [];
let nestedArrs = [[1, 2, 3], [4, 5], [6]];
let jumbledArr = [2, true, false, "2.43", [4, ["hello"], []]];
```

{% hint style="info" %}
When creating an array, it is possible to split it into multiple lines. Type the opening square bracket and hit `enter`; your cursor will go to the next line. Then, enter a value, put a comma, and hit `ctrl` + `enter` (or `cmd` + `return` on mac). The console will go to the next line. Keep doing this until you have entered all the values needed, then navigate to the closing square bracket and hit `enter` to execute the line.
{% endhint %}

As with strings, you can access a particular element in an array using square bracket indexes:

```js
let arr = [1, 2, 3, 4];
arr[0]; // 1
arr[2]; // 3
```

These indexes can be chained to access particular elements in nested arrays:

```js
let nestedArr = [
    [1, 2, 3, 4],
    [5, 6, 7],
    [8, 9]
]
nestedArr[0]; // [1, 2, 3, 4]
nestedArr[0][1]; // 2
nestedArr[2][1]; // 9
```

You can also reassign values in an array in this way:

```js
let nums = [1, 2, 3, 4];
nums[2] = 10;
nums; // [1, 2, 10, 4]
```

The length of an array is accessible through its `length` property:

```js
arr.length; // 4
nestedArr.length; // 3
nestedArr[2].length; // 2
```

## Adding values to an array

To add values to the end of an array, there are 2 ways. The first way is to assign values using indexing.

```js
let arr = []; // empty array
arr[0] = 1
arr; // [1]
arr[1] = 2;
arr; // [1, 2]
```

In general, `arr[arr.length] = x` will add `x` to the end of the array.

It is possible to use the wrong index an accidentally reassign a value. It is also possible to pass in an index beyond the length of an array. In this case, JavaScript will create "gaps" in the array filled with the value `undefined`:

```js
let arr = [1, 2, 3];
arr[4] = 4;
arr; // [1, 2, 3, undefined, 4]
```

The second way to add arrays prevents any chance of using the wrong index, and involves the `push` method.

```js
let arr = [1, 2, 3];
let l = arr.push(4);
arr; // [1, 2, 3, 4]
l; // 4
```

The method takes in one argument and pushes it to the back of the array. It also returns the new length of the array after insertion. `push` is a [variadic function](/orbital/readme-1/functions), so you can push multiple values at once:

```js
let arr = [];
let l = arr.push(1, 2, 3, 4, 5, 6);
arr; // [1, 2, 3, 4, 5, 6]
l; // 6
```

{% hint style="warning" %}
There is no easy way to insert elements into a specific position in the array. One way is to copy over the elements into a new array, making sure to insert the new value you want to at the right index, and then use the new array. Another way is to use the [`splice` method](https://www.w3schools.com/jsref/jsref_splice.asp), as detailed in [this StackOverflow post](https://stackoverflow.com/questions/586182/how-to-insert-an-item-into-an-array-at-a-specific-index).
{% endhint %}

## Removing values from an array

The best way to remove elements from an array is to use the `pop` method. It is a nullary function that removes the last element in the array and returns the element it removed.

```js
let nums = [1, 2, 3, 4];
let x = nums.pop();
nums; // [1, 2, 3]
x; // 4
```

{% hint style="warning" %}
Again, it is not easy to remove a specific element; the `splice` and `indexOf` methods will be needed.
{% endhint %}

## Searching for an element

To check if an element exists in an array, the `indexOf` method is useful. It returns -1 if the element is not present, otherwise it returns the index of the element.

```js
let arr = [1, 2, 3, 4];
arr.indexOf(3); // 2
arr.indexOf(23); // -1
```

{% hint style="warning" %}
`indexOf` will not work when passing in another array as an argument:

```js
let arr = [[1, 2, 3], [4, 5, 6]]; // 2D nested array
arr.indexOf([1, 2, 3]); // returns -1 even though [1, 2, 3] is an element
```

This is because `indexOf` uses the `===` operator, which returns `false` for two arrays even if they have the same elements unless they refer to the same object:

```js
let arr1 = [1, 2, 3];
let arr2 = [arr1, [4, 5, 6]];
arr2.indexOf(arr1); // returns 0 now
```

{% endhint %}

## Array sliceing

To get a particular portion of an array, you can use the `slice` method. It takes in a `start` and `end` and returns all the elements from the `start` index to just before the `end` index:

```js
let arr = [1, 2, 3, 4, 5, 6, 7, 8, 9];
arr.slice(2, 5); // [3, 4, 5]
```

Omitting the `end` value will slice till the end of the array:

```js
arr.slice(4); // [5, 6, 7, 8, 9]
```

If the `end` value is greater than or equal to the `start` value then it returns an empty array:

```js
arr.slice(6, 2); // []
```

If only one argument `k` is provided and the argument is negative, then the *last* `|k|` elements are returned:

```js
arr.slice(-4); // [6, 7, 8, 9]
```

## Array sorting

There are two ways to sort an array. The first is the `sort` method. It takes no arguments, has no return value and sorts the array it is called on:

```js
let nums = [3, 4, 6, 1, 2, 5];
nums[3]; // 1
nums.sort();
nums; // sorted to [1, 2, 3, 4, 5, 6]
nums[3]; // 4
```

The second way is to use the `toSorted` method. This does not change the original array and instead returns a new sorted array:

```js
let nums = [3, 4, 6, 1, 2, 5];
let sortedNums = nums.toSorted();
nums; // still [3, 4, 6, 1, 2, 5]
sortedNums; // [1, 2, 3, 4, 5, 6]
```

Both these methods sort the elements of the array according to their natural ascending order as defined by JavaScript. To sort an array in descending order, you can call one of the `sort` or `toSorted` methods and then reverse the array.

## Reversing an array

Like with sorting, there are two ways to reverse the ordering of elements in an array. The first is the `reverse` method, which works like the `sort` method (i.e. it changes the original array):

```js
let nums = [1, 2, 3, 4, 5, 6];
nums[3]; // 4
nums.reverse();
nums; // [6, 5, 4, 3, 2, 1]
nums[3]; // 3
```

The second way is to use the `toReversed` method which is similar to the `toSorted` method (i.e. it does not modify the original array and returns a new one instead):

```js
let nums = [1, 2, 3, 4, 5, 6]
let reversedNums = nums.toReversed();
nums; // still [1, 2, 3, 4, 5, 6]
reversedNums; // [6, 5, 4, 3, 2, 1]
```

To sort an array in descending order, combine the sorting and reversing methods:

```js
let nums = [3, 5, 1, 2, 6, 4];
nums.sort()
nums.reverse();
nums; // now [6, 5, 4, 3, 2, 1]
```

## Iterating through an array

A `for` or `while` loop can be used to iterate through the elements of an array and perform an action. For this next bit let's try to go through the elements of an array `nums` and add the squares of each element to a new array. Below is the code for this:

```js
let nums = [1, 2, 3, 4, 5];
let squaredNums = []
for (let i = 0; i < nums.length; i++) {
    squaredNums.push(nums[i] ** 2);
}
squareNums; // [1, 4, 9, 16, 25]
```

But there is a better way of doing this. Arrays have a `forEach` method that takes in a unary function and applies it to every element in the array. It is common to use lambda expressions and anonymous functions for this. The above code can be shortened to:

```js
let nums = [1, 2, 3, 4, 5];
let squaredNums = [];
nums.forEach(x => squaredNums.push(x ** 2));
squaredNums; // [1, 4, 9, 16, 25]
```

{% hint style="warning" %}
If the unary function passed in to `forEach` returns any value, the value is ignored and "lost" i.e. cannot be used by any code you write
{% endhint %}

## More methods

There is a full list of array methods [here](https://www.w3schools.com/jsref/jsref_obj_array.asp); this section would be far too long if I tried covering them here.

## Next steps

Now we're done covering the basics of JavaScript. The next section will move on to HTML, followed by CSS, and then back to JavaScript where we'll combine the 3 to make a functional frontend.


# HTML

This section will introduce you to HTML, a markup language that is used to define and structure webpages. We'll start off by making a page and adding some elements, then adding more elements and giving those element properties (or attributes), add an HTML form to submit queries to Google Search, and wrap up with a look into Firefox's Browser Inspector tool.


# Getting Started

## What is HTML?

HTML, which stands for HyperText Markup Language, is a language used to construct web pages by defining their structure and content. The structure is defined by tags, which are keywords that define elements on a page, and are surrounded by angle brackets (`<>`).

## Tags

Tags, as mentioned before, define the structure of a page by creating elements. There are two types of tags: paired and unpaired.

Paired tags have an opening tag, which is the name of element surrounded by angle brackets, and a closing tag, which is the name of the element prefixed with a `/` and surrounded by angle brackets (`<element>...</element>`). The content goes in between the opening and closing tags.

Example of paired tags include the `<html>` tag, the `<head>` tag and the `<body>` tag.

Unpaired tags only have a single tag for the element (`<element>`).

Examples of unpaired tags are the `<br>` tag and the `<hr>` tag.

## Creating an HTML file

Open up a text/code editor or IDE of your choice, and type in the following code:

```html
<!DOCTYPE html>
<html>
   <head>
       <title>My web page</title>
   </head>
   <body>
       <h1>Welcome</h1>
       <p>This is my first page</p>
   </body>
</html>
```

Then save the file as `index.html`. Congratulations, that is your first HTML file ready. To view it in the browser, naviate to the file in your finder/library and open it with a browser of your choice (see below).

<figure><img src="/files/ebjeCoLvUbMVwqTBmZOu" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
On most operating systems `.html` files are set to open in your default browser by default; you can change this if you want.
{% endhint %}

In your browser, you should see something like this:

<figure><img src="/files/by5m5BHpK4puEhh40QCl" alt=""><figcaption></figcaption></figure>

Now lets see what these tags do.

* `<!DOCTYPE html>` essentially declares to the browser that the document is an HTML file. It's not necessary, but helpful.
* The `<html>` tag contains the entire contents of the page between its opening and closing tags.
* The `<head>` tag contains meta information about the page, as well as imported scripts and stylesheets. Here it also contains the `<title>` tag:
  * The `<title>` tag defines the title of the page, which is the text that appears on the tab in the browser.
* The `<body>` tag contains the body of the page, including text, forms, images and more.
  * The `<h1>` tag defines a heading. It can range from 1 to 6, with 1 being the largest size and 6 being the smallest size heading
  * The `<p>` tag defines a paragraph of normal-sized text. Note that it is possible to place the text on its own without a `<p>` tag and it will still render as a paragraph.

## More tags

Let's take a look at some more tags:

* The `<div>` tag is a paired tag that is often used to create a generic element that may or may not contain text.
* The `<a>` tag is a paired tag that is used for hyperlinks (a stands for anchor).
* The `<button>` tag is a paired tag that is used to create a clickable button.
* The `<form>` tag is a paired tag that is used to create an HTML form. We'll see more of these later.
* The `<hr>` tag is an unpaired tag that is used to create a horizontal line on the page
* The `<br>` tag is an unpaired tag used to denote a line break.
* The `<img>` tag is an unpaired tag used to embed images on a page.
* The `<input>` tag is an unpaired tag used to create an input field inside the form.
* The `<link>` tag is an unpaired tag used to import files (usually stylesheets).
* The `<script>` tag is a paired tag that can be used to embed/import JavaScript code into the page.

We'll end up using all of these tags throughout the guide, so make sure to keep them in mind. A full list of HTML tags can be found [here](https://www.w3schools.com/tags/).

{% hint style="info" %}
Comments in HTML files can be denoted by `<!-- put comment here -->`. It can be extended to multiple lines as needed.
{% endhint %}

{% hint style="info" %}
Spaces between tags and empty lines are ignored by the browser when rendering HTML.
{% endhint %}

{% hint style="info" %}
As you have seen, tags can be nested within one another.
{% endhint %}

## Next steps

Next, we'll take a look at tag attributes and how they can be used to give properties to elements.


# Tag Attributes

Tag attributes are essentially properties that can be assigned, or *attributed* to a tag. The way to assign attributes is as follows:

```
<pairedTag attribute1="value1" attribute2="value2" ...>some content</pairedTag>

<unpairedTag attribute1="value1" attribute2="value2" ...>
```

Let's enhance our previous `index.html` file with some more elements and attributes.

## Adding some divs

Going back to our `index.html` file, let's add a 4 divs below the paragraph tag. Copy the following code into the body of your HTML code: (or write your own divs if you want):

```html
<div>This is a div, a generic html element</div>
<div>This is also a generic html div element</div>
<div>Also a div</div>
<div>Still a div</div>
```

Below these divs, add a line break element, `<br>`. This will force the next element to go onto a new line, instead of being stacked next to the div. Go the browser and reload the HTML page (or reopen if it you closed it earlier). You should see the above text rendered below the earlier paragraph we wrote, in 4 separate lines (see below).

<figure><img src="/files/2qMZfuASyX5khUIiTj0l" alt=""><figcaption></figcaption></figure>

### Adding a `class` attribute

You'll notice how two of the divs above are quite long, while the other two are quite short. Hmmmm.

When writing HTML code there may be certain elements that you may want to format/style similarly because they are intended to have a similar appearance and behaviour. Here, say you want to group the longer divs with each other, and the shorter ones with each other.

This is where the `class` attribute comes in. Elements are assigned a class based on what you, as the developer, want the element(s) to look/behave like. Elements that look/behave similarly can be given the same class.

Here, assign the longer divs the class "long-div", and the class "short-div" (or any class name of your choice, as long as there are no spaces). It should look like this:

```html
<div class="long-div">This is a div, a generic html element</div>
<div class="long-div">This is also a generic html div element</div>
<div class="short-div">Also a div</div>
<div class="short-div">Still a div</div>
```

Now, once we add CSS to the page we can use these classes. Reloading the page won't change anything yet though.

## Adding some links

Recall that hyperlinks are created using the `a` tag. Let's add two links to our page: one for the orbital webpage and one for the NUS Hackers webpage. Inside the body, below the divs and the line break, add the following code (note that the anchor tags are nested within paragraph tags):

```html
<p><a>Orbital webpage</a></p>
<p><a>NUS Hackers webpage</a></p>
```

Lets add another line break using `<br>` below these links, and a horizontal line using the `<hr>` unpaired tag.

```html
<p><a>Orbital webpage</a></p>
<p><a>NUS Hackers webpage</a></p>
<br>
<hr>
```

Now, when you go back and reload the page, you should see the above text rendered in two separate lines. But you'll realise that you haven't really added any hyperlink yet because clicking on the text takes us nowhere. This is where the `href` attribute comes in.

### The `href` attribute

"href" stands for "HyperText Reference" and is used to link to images, links or files on the web (or on your local storage). It can be assigned to the `<a>` tag (and a few others) to give it a hyperlink. You can give a link to a page as a value to the `href` attribute to create the hyperlink:

```html
<p><a href="https://orbital.comp.nus.edu.sg/">Orbital webpage</a></p>
<p><a href="https://hckr.cc">NUS Hackers webpage</a></p>
```

Now if you go back and reload the page, you'll see the links, and you'll see that clicking on them sends you to the respective pages (see below).

<figure><img src="/files/7nWfbprQOCZz6TTsjyYW" alt=""><figcaption></figcaption></figure>

## Adding a form

Now let's add a form to our page. This form will, eventually allow us to conduct a Google Search query. Below the links we added before, add the following code:

```html
<form>
    <input>
    <button>Submit</button>
</form>
```

If you reload the page and try to enter something and submit the form, nothing will happen. This is because we haven't defined any attributes for the form yet. We'll do that in the next section, but for now lets add a couple of attributes to the other elements.

### The `required` attribute

You can imagine having a form where some or all of the fields are necessary (i.e. you don't want the form to get submitted if they are empty). How do you control this? Simple: add the attribute `required` to the input field.

In the html file, add the `required` attribute to the `<input>` tag:

```html
<input required>
```

The `required` attribute takes in either "true" or "false" (as strings), but such attributes (called boolean attributes) can instead be omitted to set their value to "false", or place without assignment to set their value to "true".

Another example of a boolean attribute is the `hidden` attribute that can be added to any content-defining tag (paragraphs, buttons, forms, input fields, divs, tables etc.) to specify whether it should be hidden or not (big surprise there).

### The `type` attribute

Generally, input fields take in only text. But what if you want to specify the kind of input it takes? That's where the `type` attribute comes into play. The `<input>` tag can take on a `type` attribute to define what type it is. The values could be one of the following:

* "text" - this is the default value, no need to define a `type` attribute in this case
* "email" - this checks for a valid email format (<sometext@somedomain.com>)
* "password" - masks the input so the user cannot see what they are typing (now you know how to get those black dots in a password field)
* "number" - only allows numbers
* "checkbox" - makes the input a checkbox that can be checked/unchecked. Multiple checkboxes can be checked at once.
* "radio" - makes the input a radio option (like checkboxes, but only one radio option can be selected at a time, and cannot be unselected once clicked)
* "file" - allows the user to upload a file

In our case, we do not need a `type` attribute for the `<input>` tag since it is already of type "text". But the `type` attribute can assigned to the button instead, with the value "submit". This means that, when the button is clicked, the form gets submitted. Go ahead and add the type attribute to the button:

```html
<button type="submit">Submit</button>
```

### The `placeholder` attribute

The `placeholder` attribute can be added to the `<input>` tag to define some placeholder text. Let's add some:

```html
<input placeholder="query" required>
```

Now if you reload the page, you'll see "query" in greyed out text inside the input field. This placeholder will disppear immediately once you start typing. If you try to submit the form without any input, you will get a small message from the browser requesting you to fill in the input field, thanks to the `required` attribute.

<figure><img src="/files/63NEWGbjxKZZf0dUzpEp" alt=""><figcaption></figcaption></figure>

## Adding a (useless) button

Below the form, add a line break and a button below the break. This is done using the paired `<button>` tag:

```html
<button>This button does nothing</button>
```

Now this button is unique: unlike the previous one, which submits the form, this one does nothing. It's useless. Hence, it is unique on our page. We can use the `id` attribute to denote this uniqueness:

```html
<button id="useless-button">This button does nothing</button>
```

Like the `class` attribute, the `id` attribute has no visible effect on the page. However, it denotes that the element is unique on the page, and hence you can later add special, unique behaviour and styling to the element.

{% hint style="info" %}
Despite its name, the `id` attribute of multiple elements can be assigned the same value without an error. However, this defeates the purpose of using `id` to denote uniqueness, and it is better to use the `class` attribute in this case.
{% endhint %}

## Adding an image

{% hint style="info" %}
Feel free to add line breaks to pad out elements as you wish
{% endhint %}

Finally, let's add an image to our page. Since we're learning HTML, why not an HTML logo? Visit [this link](https://github.com/Dinoman44/Orbital2024-JS-example-problems-code/blob/main/html-logo.png) and download the file. Then move the file to the same folder as your `index.html` file. Feel free to get your own HTML logo from elsewhere, but this is the same logo used in the [sample code](https://github.com/Dinoman44/Orbital2024-JS-example-problems-code/) for this guide, and its background color becomes relevant later.

The image should be present alongside `index.html` in the same folder, and should have the name `html-logo.png`.

Now let's add it to our page. Use the `<img>` tag for this, along with the `src` attribute (src stands for source):

```html
<img id="html-logo" src="html-logo.png">
<!-- i also gave it an id since it is again a unique element and i want it to have some unique styling later -->
```

You should now see the image on the page once you reload it. If you don't, check for spelling mistakes ("scr" instead of "src", or "hmtl-logo", etc.), or it may be in a different folder than the `index.html` file.

### Adding an alt

In case the image doesn't load, you would see a small icon in place of the image. Instead of this icon, you can display some text using the `alt` attribute:

```html
<img id="html-logo" src="html-logo.png" alt="html logo">
```

This means that if the image doesn't load for whatever reason, the text "html logo" will be displayed instead. Try this out by purposely mispelling the filename and reloading the page.

<figure><img src="/files/5QRpxSuB9miAOCTMRyxI" alt=""><figcaption><p>You can see the alt text "html logo" when the image doesn't load</p></figcaption></figure>

### Adding mouseover text

Some webpages have feature called mouseover text, which is small text that appears when a user hovers their cursor above an element on the page for a few seconds. This is defined using the `title` attribute, and can be used to either give users a little more information about the element, or in a [more creative way](https://xkcd.com).

Add some mouseover text using the `title` attribute to the picture:

```html
<img id="html-logo" src="./html-logo.png" alt="html logo" title="html logo">
```

Now you'll see "html logo" when you hover your cursor over the image:

<figure><img src="/files/fTYTKgZTqKU2oH3pwOdI" alt=""><figcaption></figcaption></figure>

## Putting it all together

At the end of all this, your page should render like this in the browser:

<figure><img src="/files/fSipuwwdc9GR1cTqIPfI" alt=""><figcaption></figcaption></figure>

The body of your html file should look something like this: (final code available [here](https://github.com/Dinoman44/Orbital2024-JS-example-problems-code/)):

```html
<h1>Welcome</h1>
<p>This is my first page</p>

<div class="long-div">This is a div, a generic html element</div>
<div class="long-div">This is also a generic html div element</div>
<div class="short-div">Also a div</div>
<div class="short-div">Still a div</div>
<br>

<br>
<p><a href="https://orbital.comp.nus.edu.sg/">Orbital webpage</a></p>
<p><a href="https://hckr.cc">NUS Hackers webpage</a></p>
<br>
<hr>

<form>
    <input placeholder="query" required>
    <button type="submit">Submit</button>
</form>

<br>
<button id="useless-button">This button does nothing</button>
<br><br>
<img id="html-logo" src="./html-logo.png" alt="html logo" title="html logo">
```

## Next steps

Next, we'll add some functionality to the form to allow it to submit queries to Google Search.


# HTML Forms

An HTML form allows users to submit data to a server for processing. Let's add some functionality to our previous HTML form to allow queries to Google Search.

## The necessary attributes

First, we need to add some attributes to the `<form>` tag. But what attributes does it accept? Let's look at two of them.

### The `method` attribute

This specifies the HTTP request method to use. By default, this method is "GET", but there are 4 other possibilities: "POST", "DELETE", "PUT" and "PATCH". In our case, Google Search expects a GET request, so the "GET" value is fine. This means we do not need to specify a `method` this time.

### The `action` attribute

This attribute specifies the URL to submit the form data to. Let's figure out what the action URL should be for our form.

Go to the [Google homepage](https://www.google.com) and enter some search query. Then look at the URL bar and see what address it went to.

{% hint style="info" %}
Ignore everything after the `?` in the URL bar; these are the search parameters and will be used later.
{% endhint %}

<figure><img src="/files/fSVevMSHkNcMWOE4dua9" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/iRil4B1nhwuXl03zOhyC" alt=""><figcaption></figcaption></figure>

As we see above, the URL it went to was [`https://www.google.com/search`](https://www.google.com/search). Hence this is the URL we need to submit our form data to. Let's add this attribute to our `<form>` tag:

```html
<form action="https://www.google.com/search">...</form>
```

### The `name` attribute

This attribute is given to an input field, and specifies the name of the input parameter. In this case, what name do we give our input?

Consider the URL bar again. Right after the `?`, you should see something along the lines of `q=whateveryoutyped`.

<figure><img src="/files/IZVDTYddwq10cZl6xoVd" alt=""><figcaption></figcaption></figure>

This is how form data is arranged in a GET request. The name of the parameter, following by an equals sign `=`, followed by the value of the parameter itself. For multiple parameters, they are separated by ampersand symbols `&`. In general, in a GET request, the URL bar looks like this:

```
https://www.domain.com/action-url?name1=value1&name2=value2&name3=value3
```

For Google Search, most of the parameters are for Google's internal service: type of browser, operating system, cookies, google account signed in, etc. But the one we want is the first one: `q=`

Here `q` is the name of the parameter, and it stands for (presumably) query. Whatever you searched up would be the value. So now we know what name to give our `<input>` field:

```html
<input name="q" placeholder="query" required>
```

## Putting it together

Your form by now should look like this:

```html
<form action="https://www.google.com/search">
    <input name="q" placeholder="query" required>
    <button type="submit">Submit</button>
</form>
```

Once you reload the page, you won't notice any visible changes. But when you enter a query into the form field, you'll be redirected to a Google Search page for the query as below.

<figure><img src="/files/35q0QZy2xv444jVwUUpR" alt=""><figcaption></figcaption></figure>

## Next steps

We're pretty much done with HTML for now. The next section of the guide will explain a little about the Browser Inspector in Firefox, a tool that will be very useful when we move on to CSS right after.


# Browser Inspector

[Remember how to open the Browser Developer Tools?](/orbital/readme-1/browser-dev-tools) Good, you'll need it.

Press `f12` or `fn` + `f12` to open up the Dev Tools. On Firefox, this by default opens the Inspector tab, which is where the page's HTML can be seen. Let's use Google's homepage as an example for now.

<figure><img src="/files/FUbfAigWU6jtBkwi7fNN" alt=""><figcaption></figcaption></figure>

You'll see 3 windows in the tab (on Firefox at least).

1. The leftmost window (in the picture above) contains the HTML of the page. It allows to read the HTML, see elements, look at what tags they are using and what attributes they are assigned, and more. It also allows you to look for hidden elements - elements that aren't visible on the page but are present all the same. You can even edit the HTML by double-clicking at a certain part of the code to change its value temporarily and see (in real time) how it affects the contents of the page.
2. The middle window allows you to see the CSS properties of a selected element on the page. We'll get into CSS in the next section, and there this will be handy since you can inspect the CSS of any element on any webpage, as well as edit styles to see what effect the changes have on the page in real time.
3. The rightmost window allows you to see the Box model of a selected element, Flexbox of a selected element and Grid layout of the page. When we get into margin and padding later on, the Box model will become a useful visualising tool, but for now we have nothing to do with it.

If you want to select an element on a page, there are 2 ways. The first is to read through the HTML and try and figure out which element is the one you want. This however, is going to be difficult when pages get long and complicated. Just look at this excerpt from the Google homepage:

<figure><img src="/files/fKyP3BC7imx5oX3nD2I1" alt=""><figcaption></figcaption></figure>

The second way, and better, is to use the element picker. You'll notice a cursor icon next to the "Inspector" tab title. This is the element picker.

<figure><img src="/files/9c4OuUqnUS40jvFV3qzr" alt=""><figcaption></figcaption></figure>

Click on it, and when you move your cursor over the webpage you'll be able to highlight an element. Click on the element or part of the page you wish to look at in detail, and the element will get selected, showing its style and HTML in the Inspector.

<figure><img src="/files/QeFmZG8vCxcC31NY8CKK" alt=""><figcaption></figcaption></figure>

Remember how you can temporarily edit the HTML and CSS of the page? Combine that with the element picker, and you have a powerful set of tools to test out CSS styles and HTML elements in real time for development. Here's an example of me playing around on the Google homepage using these tools:

<figure><img src="/files/v4rPX28Npy4KfhU00Rnm" alt=""><figcaption><p><em>Ignore the slow framerate</em></p></figcaption></figure>

Once you reload the page, all the temporary changes you make are removed and the page is restored to its original, unaltered state.

## Next steps

We've made a pretty nice webpage so far, but it seems... plain. It needs some style, some color, a different font maybe.

In the next section, we'll learn some CSS to add some styling to our page.


# CSS

This section will explore CSS to help us add some style to our pages.

## What is CSS?

CSS, which stands for Cascading StyleSheets, is a stylesheet language that is used to specify the styling of an HTML document. This is done by adding properties and values to elements by the way of selectors. Some CSS would look like this:

```
element {
    property1: value;
    property2: value;
}
```

As you can see, the element that you want to style is at the top, with the properties enclosed in curly braces (`{}`). The properties are assigned values by the way of `property-name: value;` and each line ends with a semi-colon.

CSS can be used to assign styles to elements that have the same tag, class, and/or id, and there are ways to give styling for particular events (for example, what a button temporarily looks like when the user hovers over it).

## How to add CSS to HTML

There are 2 ways to add CSS styling to an HTML page. The first is to use in-line styling by way of the `<style>` tag. In the head of the HTML file, add the opening and closing `<style>` tags. Then you can place the CSS inside of tag.

{% hint style="info" %}
The `<style>` tags can be added anywhere in the html document (as long as it before the elements you want to style), but it is convention to put it in the head of the html document.
{% endhint %}

The second way is to create a new file, add the CSS there, and save it with the `.css` file extension. Then it can be imported into the html document using the unpaired `<link>` tag:

```html
<link rel="stylesheet" href="filepath/nameOfFile.css">
```

{% hint style="info" %}
While the first method may seem easier, it is recommended to use the second method to separate out CSS and HTML for easier debugging and as good practice. However sometimes you may just want to add, say 3-4 lines of styling will just 1 or 2 rules to a page, in which case it is okay to use inline styling.
{% endhint %}

There is a third way, which involves using the `style` attribute to assign styling to a specific element but this is only used in rare cases when just 1 or 2 properties need to be defined for a very specific element, and even then it is recommended to instead use method 2 and give the element an `id` attribute.

## Next steps

Get the `index.html` file ready, as we're now going to add some styling to that. Note that we will stick to method 2 of adding CSS (create a new file and import) throughout this guide. First, create a file called `styles.css` and place it in the same folder as the `index.html` file. The add the following line to the head of the `index.html` document:

```html
<link rel="stylesheet" href="styles.css">
```

Now, you are ready to add some styling. Let's get started with some core concepts.


# Selectors

Selectors are, as you can guess, ways to select elements. These selectors apply to both CSS and JavaScript later on.

## By name

The first way to select a group of elements is to select them by name. An example use in CSS would be this:

```css
p {
    /* some styling */
}
```

Here, the styling will be applied to all `<p>` elements in the html file. This works with any valid html tag, and the general syntax is:

```css
tagName {
    /* styling goes here */
}
```

## By class

The next way is to select elements by class. As mentioned before, elements in the same class would be expected to have similar styling and behaviour, so it makes sense that you would want to select a class of elements instead of by name. An example is as follows:

```css
.long-div {
    /* some styling */
}
```

Here, the styling will be applied to all elements that have the class "long-div". Note that to specify that a class is being selected, you need to prefix the class name with a dot (`.`)

```css
.class-name {
    /* styling goes here */
}
```

## By id

Lastly, you may want to select an element by its id. This involves using the `#` symbol as a prefix to the id name:

```css
#useless-button {
    /* some styling */
}
```

In this case, the element with the id "useless-button" will be assigned the styling. Generally:

```css
#id-name {
    /* styling goes here */
}
```

## Multiple elements

If multiple elements of different classes/tags are being assigned the same styling, it is possible to combine them using commas:

```css
a, p, div {
    /* some styling */
}
```

Here, the same styling is being assigned to the `<a>` tag, the `<p>` tag and the `<div>` tag. This reduces the length of the file and makes it easier to control shared styling.

If I wanted to add a few unique rules to the `<div>` elements while keeping the rest of the styles constant, I could do this:

```css
a, p, div {
    /* common styles */
}

div {
    /* extra styling only for div */
}
```

This is because styles are *cascading* (hence *Cascading* StyleSheets). This means that if style rules for an element are defined multiple times, all the properties are combined and applied to the element.

{% hint style="warning" %}
If the same property is redefined multiple times, then the last definition of the property is applied.
{% endhint %}

## Redefining properties

When the same property is redefined for a group of elements, the browser decides which definition of the property to use based on this simple algorithm:

* Check the selector and choose the most specific selector's property: `id` selectors are most specifc, and tag name selectors are the least specific
* If there are multiple most-specific selectors, then pick the one that appears last and before the element appears (a style rule that appears after the element will not be applied to the element)

## Next steps

Next we'll look at colors in CSS and what values they can take on.


# Colors in CSS

In CSS, there are many ways to define colors. We'll look at 4 popular ways to do so in this section.

## Method 1: by name

There is a long list of colors that are known to all browsers by name, and these colors can be directly assigned as color values for properties like `color` (color of the text) or `background-color`. Of course there are the more "common" colors like `red`, `orange`, `black`, `white` or `green`, but there are also many variations and shades like `coral`, `cornflowerblue`, `limegreen` and `whitesmoke`. The full list of these colors can be found [here](https://www.w3schools.com/tags/ref_colornames.asp), and there are nearly 150 of them.

{% hint style="info" %}
Note that quotation marks must not be present around the color values when assigning them: `color: red;` is correct, but `color: "red";` is invalid.
{% endhint %}

## Method 2: by hexadecimal code

To understand this, we need to understand how colors are represented.

### Representation of colors in browsers

A color on the screen is actually a combination of 3 main colors: red, green and blue. Combining different "strengths" or "intensities" of these colors leads to a different color that is rendered on screen. The lower the intensity, the darker the shade of the color and the less prominent it is.

The intensity is a decimal integer that ranges from 0 to 255 (and no higher or lower). This is then converted into [hexadecimal](https://en.wikipedia.org/wiki/Hexadecimal) (a number system with 16 digits: 0 to 9, then a-f for 10-15 respectively) which reduces it to two digits.

{% hint style="info" %}
0 as a decimal integer is 00 in hexadecimal, and 255 as a decimal integer represented by ff. There are online conversion calculators to convert decimal to hexadecimal and vice-versa, like [this one](https://www.rapidtables.com/convert/number/hex-to-decimal.html)
{% endhint %}

The 3 intensities are then combined into one 6-digit hexadecimal value, which is used to represent the color. The red intensity comes first, followed by green, then lastly blue.

When assigning hexadecimal code colors as color values, a `#` symbol is prefixed before the 6-digit hex value. Note that again quotation marks need to be omitted.

Format of color: `#rrggbb`

### Examples:

If you want pure red, then you max out the red value and minimize the green and blue. This means that the red value is 255 and the green and blue are both 0. Since 255 in decimal is ff in hex, the color code for pure red becomes `#ff0000`

The same can be done for pure blue and pure green.

In HTML, if all the values (red, green and blue) are maxed out, then you get the color white: `#ffffff`

On the other hand, removing all color leaves black: `#000000`

More generally, if the intensities of red, green and blue are equal to each other you end up with a shade of grey. The lower the intensities, the darker the shade.

Here are some more examples:

* Yellow is max red, max green, no blue: `#ffff00`
* Orange is max red, little less green, no blue: `#ffa500`
* Violet is medium red, no green and max blue: `#7f00ff`

### Back to CSS

Coming back to CSS color values, you can assign the 6-digit hexadecimal value to a property the same way to assign a color by name. For example: `color: #7f00ff;` (violet, as above).

There are online HTML color pickers, and some IDEs (like Visual Studio Code) provide ways to visualise a color using its hex code while you write CSS, so there is no need to memorize color codes. Additionally, if there is a font color or some such color on a website that you like, you can use the [Browser Inspector](/orbital/readme-1/html/browser-inspector) to select the element and read its styling rules to see what its color code is.

The end of the page will show another way to pick a color from a page using the Eyedropper.

## Method 3: the rgb function

While hex codes provide a lot of flexibilty and control over the precise shade of color that you want to use, they are rather unintuitive. CSS provides the `rgb` function, which takes in 3 arguments and returns a color value. The arguments are integers from 0 to 255 and are the red, green and blue values (left to right). They are slightly easier to use than the direct hex codes, and are supported by all browsers.

To represent the color violet as above, I would write: `color: rgb(127, 0, 255);`

Again, there are color pickers online that can be used to find the RGB values for particular colors and color palettes.

## Method 4: the rgba function

This performs like the `rgb` function but it takes an additional argument. This argument is a decimal number between 0 and 1 inclusive, and indicates the opacity of the color. 1 means the color is fully opaque and 0 means the color is fully transparent.

Usage: `color: rgba(127, 0, 255, 0.6);`

## Full list of CSS color values

Here's a [full list of color values accepted by CSS](https://www.w3schools.com/cssref/css_colors_legal.php).

## The Eyedropper

Firefox has a tool called the Eyedropper that, when active, can pick out the exact html color code of anything present on a webpage. To use this, open up the Browser Inspector and find the dropper icon at the top right corner of the html window (see below).

<figure><img src="/files/OKdDSyhxCS9603XKs81X" alt=""><figcaption><p>The Eyedropper is highlighted by a white circle in this picture</p></figcaption></figure>

Alternatively, you can customize the Firefox toolbar to add the Dev Tools menu with the Eyedropper.

Now click on the icon and you'll have the Eyedropper active. It takes control of your cursor and shows you the exact hexadecimal color code of the pixel your cursor is on. If you click while it is active, it copies the color code to your clipboard and deactivates (returning your cursor to normal). To deactivate it without copying the color code, press the `esc` key.

Here's how it works (note that I have customized my toolbar to have the Eyedropper and Developer tools accessible from there without using the keyboard shortcut):

<figure><img src="/files/JsWlJfx6rT8nRSu5sTIJ" alt=""><figcaption></figcaption></figure>

Later we'll see use the color picker to choose a suitable background color for our html document.

## Next steps

Next, we'll see how measurements in CSS work and what valid measurement values can be assigned to elements.


# Measurements in CSS

In CSS, you may want to define measurements for elements such as font size, height, width, curvature, thickness etc. There are multiple ways to define measurements, so let's look at a few.

## Notation

CSS measurement values are written as a number followed by the unit (with no spaces in-between). Example: `20px`.

Negative numbers are ignored (except for `padding` and `margin` properties) and are considered 0. Decimal values are accepted.

## Absolute measurements

The measurement units have absolute values, meaning they do not change no matter the system, browser or screen size. There are a few that we use in our daily lives:

* `mm` (millimeters)
* `cm` (centimeters)
* `in` (inches)

They can be converted between as follows:

* `1cm = 1000mm`
* `1in = 2.54cm`

There are a few more absolute measurements:

* `px` (pixels) - width of a pixel on the screen
* `pt` (points) - often used for font size; similar to font size values on MS Word
* `pc` (picas) - a typographic unit, similar to `pt`

They can be converted to other units as follows:

* `1in = 96px`
* `1in = 72pt`
* `1pc = 12pt`

Note that `pt` and `pc`, while supported, are not very commonly used as a unit of measurement in CSS. `px` is the most commonly used, followed by `mm` and then `cm` and `in` to a much lesser extent.

The nature of being absolute measurements means that regardless of the computer specs, screen size, browser width/type, resolution, size of other elements, etc. the values correspond to constant sizes. This means that `100px` measures the same physical distance whether its on Safari or Chrome, whether its on an old iPhone 3 or an 80 inch 4K ultra TV screen, whether the browser is max size or resized to half.

## Relative measurements

Far more common (and in some ways better) than absolute measurements are the relative measurement units. These units are relative to some or the other size, and they will differ based on these dimensions. Let's take a look a few of them:

* `%` - this is relative to the size of the parent element as a percentage (i.e. 45% means 45% the width of the parent element)
* `em` - this is relative to the font size of the current element; `2em` means 2 times the font size
* `rem` - this is relative to the font size of the root element; `2rem` means 3 times the font size of the root
* `vw` - this is relative to the width of the viewport; `1vw` means 1% the width of the viewport\[
* `vh` - this is relative to the height of the viewport; `1vh` means 1% the height of the viewport

{% hint style="info" %}
The viewport is the browser window; resizing the browser window changes the dimensions of the viewport
{% endhint %}

## Difference between absolute and relative units

The following two figures demonstrate the difference between absolute and relative measurements. Note how the element size changes as the browser window is resized. Note that

1. The screen is 1440px wide
2. The browser window initially takes up the entire screen
3. 50vw is 50% the width of the browser, or 1440/2 = 720px. This means the two orange boxes initially have the same width.

<figure><img src="/files/JvmM3saLWonnRkWc2mJA" alt=""><figcaption><p>Absolute measurement (720px)</p></figcaption></figure>

<figure><img src="/files/KvJNRpcBE4nEirqy1hgE" alt=""><figcaption><p>Relative measurement (50vw)</p></figcaption></figure>

## Next steps

We'll look at the box model for elements to introduce `padding` and `margin`, before we finally begin adding CSS to the page.


# The Box Model

The box model is a way to visualize the space taken up by an element on a page. It involves 4 main properties:

1. The content: This is content of the element (text, other elements, images, etc.)
2. The padding: This is the space between the content and each border
3. The border: This is the, uh, border around the element
4. The margin: This is the space outside the border of the element separating it from other elements

The padding, border and margin can have individual properties on each side (top, left, bottom and right).

The Browser Inspector allows you to visualise the Box Model of a selected element on the page. An example is below:

<figure><img src="/files/GNjafVdFZR3d5JUFQ6CL" alt=""><figcaption></figcaption></figure>

We can see the "Google Search" button element selected, and its box model shows its properties. They can be listed as below:

* The button's content is ≈94 pixels wide and 34 pixels high
* The button has a 1 pixel thick border all around
* The button has no padding on its top and bottom but has 16 pixels of padding on its left and right
* The button has a 11 pixel margin above and below it, and a 4 pixel margin to its left and right

In CSS, `border`, `margin` and `padding` are all properties that can be assigned to any element and given a [valid measurement value](/orbital/readme-1/css/measurements). There are additional directional properties for each of these to give specific measurements for a specific side.

Below you can see how each of these are used in CSS.

## Border

Property usage: `border: type thickness color;`

* `type` can take on one of the following values: `solid`, `none`, `dashed`, `dotted`, `double`, `groove`, `ridge`, `inset`, `outset` or `hidden`
* `thickness` can take on any valid measurement as we saw in the previous section
* `color` can take on any [valid color value](/orbital/readme-1/css/colors)

To style different sides differently, you can use one of the following properties:

* `border-top` - the top border
* `border-bottom` - the bottom border
* `border-right` - the right border
* `border-left` - you can figure this one out

The type, thickness and color can also be separated out by using different properties:

* `border-style` - the type of border
* `border-width` - thickness of the border
* `border-color` - color of the border

These can again be separated into the directions:

* `border-top-style`, `border-bottom-style`, `border-right-style`, `border-left-style`
* `border-top-width`, `border-bottom-width`, `border-right-width`, `border-left-width`
* `border-top-color`, `border-bottom-color`, `border-right-color`, `border-left-color`

## Padding

Property usage 1: `padding: value;`

This assigns the same padding on all 4 sides.

Property usage 2: `padding: top right bottom left;`

This takes 4 measurements, separated by spaces, and assigns them in clockwise order.

Individual sides can get different paddings in by using either `padding-top`, `padding-bottom`, `padding-right` or `padding-left`.

## Margin

Property usage 1: `margin: value;`

This assigns the same margin on all 4 sides.

Property usage 2: `margin: top right bottom left;`

This takes 4 measurements, separated by spaces, and assigns them in clockwise order.

Individual sides can get different margins in by using either `margin-top`, `margin-bottom`, `margin-right` or `margin-left`.

## Next steps

Next, we'll finally begin to add some CSS to our html document using all the knowledge from these past 4 articles.


# Adding Styles - Part 1

Let's get started with some styling. But what styles do we add?

Why not start with the background color of the page?

### Body background color

You'll notice how the white colored page doesn't match up with the grey background of the html-logo picture. There's two ways to fix this: either edit the image so it has a transparent background, or change the background color of the document to match the image. The second one is easier, so let's do that for now.

First, use the [Eyedropper](/orbital/readme-1/css/colors) to get the hex code of the color of the image by clicking anywhere on the background of the image with the eyedropper active. You'll find that it is `#25272a`.

<figure><img src="/files/17B9dgYWHgMzjdyqZiCB" alt=""><figcaption></figcaption></figure>

Now, go to the `styles.css` file and add this snippet:

```css
body {
    background-color: #25272a;
}
```

Go back and refresh the page. You should see the page background color change to match that of the image.

{% hint style="info" %}
If reloading the page doesn't change the style, open up the Browser inspector, select the element for which you changed the style, and check if the CSS rule has been applied. If it shows up in the inspector, then there is something wrong with the CSS (syntax, invalid value, wrong file name in the import etc.). Otherwise, reload the page with cache by hitting `ctrl` + `shift` + `r`
{% endhint %}

### Text color

We have a new problem now. You can't see the text! So let's change the text color. This is done by assigning a value to the `color` property.

```css
body {
    background-color: #25272a;
    color: honeydew;
}
```

`honeydew` is one of the valid colors recognized by browsers, and is a very light shade of creamy-white-yellow. Feel free to choose a different text color though, as long as it makes the text visible.

### Font family

Now I want to change the font. There's nothing wrong with it, I just want a different one. I like `Trebuchet MS`, but you can pick any font you want. To assign a font, the `font-family` property has to be used:

```css
body {
    background-color: #25272a;
    color: honeydew;
    font-family: 'Trebuchet MS', sans-serif;
}
```

This changes the font to Trebuchet sans-serif. Reload the page to see the changes.

{% hint style="info" %}
You can look up fonts online to choose one that is suitable for your page. You can also import a font if it is not natively supported by a browser.
{% endhint %}

### Inheritance

You'll notice that when a property is added to the `<body>` tag, it changes the values for all tags inside it (i.e. I did not have to add styling specifically for the `<p>` tags and the divs and headings etc.). This is because, when styling is applied to an element, the same properties are **inherited** by all elements nested within it. This is why, if I want to change a property for every element on the page, I just need to assign it to the `<body>` tag.

I can later overrule a property by giving a more specific selector (i.e. I can add a new `color: red` to the `<p>` tag, which makes it so that the `<p>` tags have red text but the rest of the page remains as-is).

### Hyperlinks

You'll notice that the hyperlinks have not changed color, even though we specified a different color for the text. This is because the `<a>` tag has a different default color that overrules any color specified earlier. We need to define a new rule to change the color specifically for the `<a>` tag

*Note that hyperlinks are often colored differently from the rest of the text to highlight them.*

In the CSS file, add this new rule below the body tag rule:

```css
a {
    color: yellowgreen;
}
```

Now reload the page, and you'll see the links are colored yellowish-green.

### Image dimensions

Let's go to the html-logo image. You'll see that it is quite large, and we want to resize it to make it smaller. One way is to give it a static/absolute height and width:

```css
#html-logo {
    width: 300px;
    height: 176px
}
```

{% hint style="info" %}
Note that we use the id as a selector here to specify the styling only for this image.
{% endhint %}

You'll see the image get smaller, but it also means you need to "hardcode" the values. You could use relative measurements so that the image gets bigger/smaller according the screen size, but that is left as an exercise.

### Don't forget the divs

Remember our "long-div" and "short-div" classes? Let's use those to assign some properties.

For the "long-div", I might want the text to be italicised, *like this*. I can use the `font-style` property for this:

```css
.long-div {
    font-style: italic;
}
```

As for the "short-div", I want something called "small-caps", which is when the font appears in all capital letters but sized according to the smaller letters. See for yourself with the `font-variant` property:

```css
.short-div {
    font-variant: small-caps;
}
```

## The page so far

So far, these are the styles we have added:

```css
body {
    background-color: #25272a;
    color: honeydew;
    font-family: 'Trebuchet MS', sans-serif;
}

a {
    color: yellowgreen;
}

#html-logo {
    width: 300px;
    height: 176px
}

.long-div {
    font-style: italic;
}

.short-div {
    font-variant: small-caps;
}
```

The page should look like this:

<figure><img src="/files/UqcXY4708qFaEhFiVHgT" alt=""><figcaption></figcaption></figure>

## Next steps

Next, we'll continue adding some style to the buttons, and introduce pseudoelements and pseudoclasses.


# Adding Styles - Part 2

## Adding styles - part 2

Continuing from the previous section, let's add some more styles to our page.

#### The buttons

The buttons look rather plain for now, but that won't be true for long. Let's change some of the properties.

```css
button {
    background-color: black;
    color: white;
    border: solid rgb(160, 78, 146) 3px;
    border-radius: 15px;
    margin-left: 5px;
    padding: 5px 8px 5px 8px;
}
```

So what's happened here? Let's take a look:

* `background-color: black;` - this just changes the background-color of the button to black
* `color: white` - this changes the color of the text in the button to white
* `border: solid rgb(160, 78, 146) 3px` - this gives the button a solid 3-pixel-thick border that is colored in with a shade of purple.
* `border-radius: 15px` - this allows me to curve the corners of the border of an element. In this case, it has a radius of curvature of 15 pixels.
* `margin-left: 3px` - this adds 3 pixels worth of space on the left of the element (remember what a [margin](/orbital/readme-1/css/box-model) is?)
* `padding: 5px 8px 5px 8px;` - this adds [padding](/orbital/readme-1/css/box-model) on the inside of the button. Going clockwise, it gives 5px padding at the top, 8px on the right, 5px at the bottom and 8 px on the left.

#### Remember the useless button?

Let's change the styling of the "useless-button" a little. This button, as you may recall, does nothing (hence its id). Recall also that we gave it the id "useless-button" so we can select the element by its id and assign some styling to it. I'm just going to add 1 change, but feel free to add as many as you'd like.

```css
#useless-button {
    border-radius: 0;
}
```

Here, I have reduced the curvature of the border to 0, i.e. the corners are no longer curved.

#### Exercise

Try adding some styling to the form and/or the input field, and maybe to the other elements in other ways.

#### Pseudoelements and pseudoclasses

You may have noticed some sites have buttons or other elements that seem to change their appearance/style when you, for example, hover over them, or click them. This is achieved by pseudoelements and pseudoclasses, which are essentially used as styling rules for elements and are applied when an action is done on the element.

In general, styling can be added for the pseudoclasses/pseudoelements as follows:

```css
element:pseudoclass { /* single colon */
    /* styles */
}

element::psuedoelement { /* double colon */
    /* styles */
}
```

A full list of these pseudoelements/classes can be found [here](https://www.w3schools.com/css/css_pseudo_elements.asp) but for now let's use one pseudoclass in particular, the `:hover` pseudoclass.

This pseudoclass allows us to define styling that will be applied when the user hovers over an element. Let's apply it to our button:

```css
button:hover {
    font-weight: bold;
    border-radius: 5px;
}
```

Nothing much going on here except two things:

* `font-weight: bold;` - this makes the text bold
* `border-radius: 5px;` - this changes the border radius to 5px

Reloading the page shows no obvious changes. But when you hover your cursor over the element, you'll see the changes come into effect:

<figure><img src="/files/i0rqLKEIr091TgHqhFWR" alt=""><figcaption></figcaption></figure>

There are many more possible pseudoelements and pseudoclasses that can help make a complex webpage more interesting and dynamic, but for now let's leave it at this. Feel free to look through the resource above to see a list of pseudoelements; assigning style to them is the same as assigning style to any regular element.

## Final result

Now that we're done adding some style to the document, this is the end result:

<figure><img src="/files/wTeIfNfx3xMgB4XuwsKe" alt=""><figcaption></figcaption></figure>

The CSS file should look like this:

```css
body {
    background-color: #25272a;
    color: honeydew;
    font-family: 'Trebuchet MS', sans-serif;
}

a {
    color: yellowgreen;
}

#html-logo {
    width: 300px;
    height: 176px
}

.long-div {
    font-style: italic;
}

.short-div {
    font-variant: small-caps;
}

button {
    background-color: black;
    color: white;
    border: solid rgb(160, 78, 146) 3px;
    border-radius: 15px;
    margin-left: 5px;
    padding: 5px 8px 5px 8px;
}

#useless-button {
    border-radius: 0;
}

button:hover {
    font-weight: bold;
    border-radius: 5px;
}
```

Our page now has some structure, thanks to HTML, and a little style thanks to CSS. In the next section, we'll look at adding JavaScript to the page to add some interactivity.


# Working with the DOM

This section of the guide will cover how JavaScript can be integrated with HTML and CSS to add functionality and interactivity in a page. We'll take a look at the DOM(Document Object Model), events, how to query and manipulate the DOM, and end with fetch requests.

## The DOM

DOM stands for Document Object Model and is a way of structuring an html document as a tree. This makes it easier for the browser to update the document's styling and when JS is applied.

Take the following html document:

```html
<!DOCTYPE html>
<html>
   <head>
       <link rel="stylesheet" href="styles.css">
       <title>My web page</title>
   </head>
   <body>
       <h1>Hello world</h1>
       <button onclick="alert('Hello!')">Say hello</button>
   </body>
</html>
```

This html document is converted into a tree like the one below:

<figure><img src="/files/THz7V0o98kyzKkku1RGm" alt=""><figcaption></figcaption></figure>

It is not important to know how this is done, but it is important to understand that every html document is converted into this model by the browser to allow for easy querying and updating of the elements.

## Events

Events are, quite literally, "things that happen". In this context, they are "things that happen to element(s) on the page". Examples of events are "click", "hover" and "keyup" (when the user presses a key and lets go, letting the key come up).

Events attributes can be assigned to html elements to execute some JS code when the event occurs. Take an example from the above html:

```html
<button onclick="alert('Hello!')">Say hello</button>
```

This sets an `onclick` attribute for the button; this means when the button is clicked, the JS code is executed. In this case, the code is `alert('Hello!')`, so when the button is clicked, the user gets a popup with the text "Hello!".

There is a full list of JS events [here](https://www.w3schools.com/jsref/dom_obj_event.asp).

## Next steps

Copy the above html into a separate html file and save it with a name of your choice. Then add some CSS of your choice, or reuse the `styles.css` file from the previous section. You can even go in with a combination of the two, and define some additonal styles in a separate CSS file, then import both files into the html file.

For the next few sections we'll look at querying the DOM from the Browser Console.


# Querying the DOM - Selectors

Querying the DOM involves searching for elements in the html document. This is done by the way of [selectors](/orbital/readme-1/css/selectors) (remember them?)

There are a few ways to search for elements in an html document using JS. We'll take a look at some of them here.

{% hint style="info" %}
`document` in JS is a global reference to the html document that is the target of the script.
{% endhint %}

## `document.querySelector`

Usage: `document.querySelector(query);`

This function allows us to select elements based on a selector:

```js
document.querySelector("button"); // searches for the first <button> tag
document.querySelector(".vintage"); // searches for the first tag with the "vintage" class
document.querySelector("#nexus"); // searches for the first tag with the id "nexus"
```

As before, selectors can search for tag names, classes or ids. It is important to note that `querySelector` returns the *first* match it finds.

The function takes in a string with the selector as an argument, and returns the element as a `Node` object. If there is no element that matches the query, it returns `null`.

For the html page we've opened up, navigate to the Browser console and lets run some queries.

### `document.querySelectorAll`

This functions similarly to `document.querySelector` but this time returns an array of all the elements that match the query. If there are no matches, it returns an empty array.

Usage: `document.querySelectorAll(query);`

### `document.getElementById`

This is a specific way to query the DOM for an element with a given id. It takes in a string with the expected id (omit the `#` prefix) and return the element if it exists (otherwise it returns `null`).

Usage: `document.getElementById(id);`

### `document.getElementsbyClassName`

This returns an array of all the elements that belong to a given class.

Usage: `document.getElementsByClassName(className);`

### `document.getElementsByTagName`

This returns an array of all the elements in the document that have a given tag.

Usage: `document.getElementsByTagName(tagName);`

## Next steps

We'll look at how to access the attributes and other properties of elements in the next section, as well as how we can change them using JS.


# Querying the DOM - Element Attributes

Open up the html page you made [earlier](/orbital/readme-1/dom). Open it in the browser, and open the browser console. We're going to query the document for elements and change their attributes, then see the changes in real-time.

## Pick an element

Our page currently only has two things: a heading and a button. So lets pick one of them. Run the following in the console:

```js
let button = document.querySelector("button");
```

Now, `button` has the button element assigned to it. Test it by typing `button` and hitting enter in the console; you should see something like below:

<figure><img src="/files/RIkppJqCVobcWDWK8uCG" alt=""><figcaption></figcaption></figure>

## Check its attributes

To get an element's properties or attributes, we can use dot notation:

```js
button.onclick; // function onclick(event)
```

The above line returns the value of the `onclick` attribute, which is a function. To make things more interesing, let's go to the html file and add an id attribute:

```html
<button onclick="alert('Hello!')" id="hello">Say hello</button>
```

Reload the page, reopen the console and rerun the above lines to select the button (this time try using the id as a selector instead).

Then run:

```js
button.id; // "hello"
```

This should return "hello", since that is the id we assigned the button.

### In general...

Given that you have a variable `element` that has been assigned an element after querying the DOM, you can access any of its attributes using dot notation:

```js
element.attributeName; // returns attribute value
```

If the element has not been assigned a given attribute it returns an empty string, or if the attribute is invalid it returns `undefined`

Example: running the following should return `""` since the button has no class:

```js
// the class attribute uses .className, not .class
button.className; // ""
button.class; // undefined
```

Some attributes have default values, such as the `hidden` attribute which is `false` by default:

```js
button.hidden; // false
```

<figure><img src="/files/FRGnskQMX9rBUWFcysqI" alt=""><figcaption></figcaption></figure>

## Changing its attributes

Just as an element's attributes can be accessed by JS code, they can also be changed by reassigning them values. Let's try this on our button.

### Changing the class

Run the following (after querying for the button if needed):

```js
button.className = "random-class"; // feel free to plug in your class name
```

You won't see any visual change on the page (since the class doesn't affect its appearance) but if you navigate to the Browser Inspector and look for the button, you'll see that it has been assigned a `class` attribute. You can even remove the attribute if you wish:

```js
button.className = "";
```

<figure><img src="/files/NiYmxSMiVM7E2YS5Ol11" alt=""><figcaption></figcaption></figure>

To visualise the change in a better way, try these steps:

1. Go to your CSS file and add some rules for a class "random-class" (or whatever you want to call it)
2. Refresh the page, reopen the console, and query for the button
3. Assign the button a class attribute as above, with the class being "random-class" (or whatever you called it)
4. You'll see the CSS being applied to the button
5. Reset the class attribute back to blank. You'll see that the styling no longer applies.

### Hiding the button

To hide the button (or any element really) just set its `hidden` attribute to `true`:

```js
button.hidden = true;
```

You'll see the button disappear from the page. To make it reappear, set the `hidden` attribute back to `false`:

```js
button.hidden = false;
```

### In general...

Any element's (valid) attributes can be assigned or reassigned to any (valid) value by accessing them using dot notation after querying for the element. If an invalid attribute or invalid value is assigned, then it is simply ignored by the page.

{% hint style="warning" %}
If there is no match for the element query, `null` is returned by the `querySelector` and `getElementById` functions. this means that trying to get/reassign any attributes will cause an `Uncaught TypeError: x is null`.

<img src="/files/CS5Dglvc3fZcwYbJKv38" alt="" data-size="original">
{% endhint %}

## Contents of an element

There are two ways to get the content of an element. One is to use to `.innerHTML` property:

```js
button.innerHTML; // "Say Hello"
```

And the other is to use the `.innerText` property

```js
button.innerText; // "Say Hello"
```

The difference between the two is that `innerHTML` refers to *everything* inside the element, whereas `innerText` refers to just the text inside the element. In our example there is no difference, but let's try editing our html file temporarily:

```html
<button onclick="alert('Hello!')" id="hello">
    <span>Say hello</span>
</button>
```

Now we have a `<span>` element inside the button; a `<span>` is just an easy way of encapsulating part of a line so specific styling can be added to it.

Now try getting the contents:

```js
let button = document.getElementById("hello");
button.innerHTML; // "<span>Say Hello</span>" with some whitespace around it
button.innerText; // "Say Hello"
```

You can change the contents as well by reassigning the properties.

## Next steps

Next, we'll look at how you can modify the style of an element using JS.


# Querying the DOM - Element Styles

Let's go back to our button and try to access and change its style. To get its style, we can use the `style` property:

```js
let button = document.getElementById("hello");
button.style;
```

You should get a `CSSProperties` object.

To access a particular property, use dot notation again to access the property. Here are some common ones:

```js
let element = document.querySelector(query);
element.style.color; // returns the color of the element
element.style.border; // returns the border specification of the element
element.style.backgroundColor; // returns the background color
```

In general, `element.style.propertyName` should return the value for that property.

## One small problem

If you actually run the above code to get a property of the element (say, the background-color), you instead get an empty string `""` rather than the value you specified in the CSS file. This is because JS, when searching for an element's style, looks at the element's `style` attribute. This attribute allows you to define inline CSS for an element, but it is not good practice to use it since it mixes HTML and CSS.

However, this doesn't stop us from changing the style as we see fit. Select the button from the console as before, and run the following:

```js
button.style.backgroundColor = "red"; // or any other color value you like
```

You'll see the button color changes to red.

If you navigate to the Inspector and select the button, you'll also see that it now has a new `style` attribute that wasn't there before:

```html
<button id="hello" onclick="alert('Hello!')" style="background-color: red;">...</button>
```

You can do the same with any style properties by getting/setting their values to any (valid) values as you like.

To reset the value back to normal, simply assign the empty string:

```js
button.style.backgroundColor = "";
```

This removes the inline styling for that property of the button which sets it back to its original style.

{% hint style="info" %}
Note that for the style values to change, they must be valid CSS values and must be assigned as strings.
{% endhint %}

Here's how this looks on the page:

<figure><img src="/files/CD5KjIukwmgbu12FKbNe" alt=""><figcaption></figcaption></figure>

## Next steps

Next, we'll look at adding JS to our html pages to make it more interactive.


# Events with JS and HTML

## Adding JS to HTML pages

There are two ways to add JS code into HTML pages. The first way is to make use of the `<script>` tag:

```html
<script>
    // code goes here
</script>
```

The second way is to write the code in a separate file, save it as with the `.js` file extension, and import it using the `<script>` tag:

```html
<script src="script.js"></script>
```

This is the recommended method which we will stick to for this guide, but note that for shorter programs (3-4 lines long) it is sometimes easier to use the first method.

## Event Listeners

Event Listeners in JavaScript are functions that wait and "listen" for events (like clicks) on the page. Once the event happens, they can execute a listener function. These are nullary functions that have no return value.

An event listener can be added either to an element on the page (like a button, a paragraph or a form), or the page itself. The method to add an event listener is `addEventListener` as is used as follows:

```js
document.addEventListener(event, listener); // add a listener to the page itself
element.addEventListner(event, listener); // add a listener to a particular element
```

### Adding a click event listener to the button

Go back to the html file from before, and remove the button's onclick attribute. Then, open a new file and save it as `script.js`. Here, write the following code:

```js
let button = document.getElementById("hello");
button.addEventListener("click", () => alert("Hello!"));
```

The first line will query for the button, and the second line adds an event listener. The event it listens for is "click", and the listener is a lambda expression for an anonymous nullary function.

Save the code, and import it into the html document by adding the following line into the head of the document:

```html
<script src="script.js"></script>
```

Now save the file, reload the page, click the button and...

<figure><img src="/files/RI5pqzYnCbfRBO3CybnM" alt=""><figcaption><p>Its hard to tell, but I am clicking the button I promise</p></figcaption></figure>

Hmmm. Nothing is happening. Go back, check the syntax and spelling. No issues there. Check the browser console, and aha. There's an error: `Uncaught TypeError: button is null`

Well, we're running the same query that we were running in the Browser console in the previous sections, and that worked fine. What's the issue here?

### The problem

Let's take a look our html file:

```html
<!DOCTYPE html>
<html>
    <head>
        <link rel="stylesheet" href="../styles.css">
        <script src="script.js"></script>
        <title>My web page</title>
    </head>
    <body>
        <h1>Hello world</h1>
        <button id="hello">Say hello</button>
    </body>
</html>
```

The browser reads the file from top to bottom, rendering elements and styles as it reads them line by line. This creates the issue that the JavaScript code is read and executed *before* the rest of the document is rendered. So the button we're querying for does not exist by the time the code is read, which means that querying the document for the button returns `null`, hence the `TypeError` we see.

#### Fix 1

The easiest way to fix it is to execute the code *after* the document is rendered. This involves moving it to the bottom of the file:

```html
<!DOCTYPE html>
<html>
    <head>
        <link rel="stylesheet" href="../styles.css">
        <title>My web page</title>
    </head>
    <body>
        <h1>Hello world</h1>
        <button id="hello">Say hello</button>
        <script src="script.js"></script>
    </body>
</html>
```

Save the file, reload the page, and try clicking the button. It should display a popup with "Hello!" on it. If it doesn't, check for syntax errors or spelling mistakes in the code and in your HTML file, and make sure the file is saved and the page is reloaded.

This is an easy way to fix the issue, and is quite convenient. However, it means that the code is not executed until the entire page is loaded and rendered, including images, stylesheets and other large files. So if there is just one single file or image that is taking time to load, the JS code will not be executed until it is loaded, which could cause some inconvenience as event listeners won't be added in time.

#### Fix 2 (recommended)

The second option is to wait till the elements of the page are loaded into the DOM (which is done before rendering the elements on the page itself). This will happen much earlier and is faster than waiting for the elements to render. To do this, there is a special event only applicable to the `document` itself. This event is called `"DOMContentLoaded"` (case-sensitive). It can be used as below:

```js
document.addEventListener("DOMContentLoaded", function() {
    // add event listeners to elements on the page
});
```

This will allow the JS code to be executed much before the page renders, so event listeners and other JS-dependent elements or features are ready as soon as the page is rendered and displayed to the user. Let's apply this to our code:

```js
// In the script.js file:
document.addEventListener("DOMContentLoaded", function() {
    let button = document.getElementById("hello");
    button.addEventListener("click", () => alert("Hello!"));
});
/*
Note that function() {...} is an alternate way of
 creating anonymous functions spanning multiple lines,
 as explained much earlier in the guide
*/
```

Now move the `<script>` tag back into the head of the document, save the file, reload the page on the browser, and the button should work:

<figure><img src="/files/XVWHOoSxbhytOPepph1v" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
You can remove a listener from an element using the `removeEventListener` function:

```js
element.removeEventListener(event, listener); // to remove a specific listener
element.removeEventListener(event); // to remove all listeners for an event
```

{% endhint %}

### Next steps

Next, we'll make a simple example of querying and updating the DOM using a click counter.


# Exercise: Click Counter

*Note: We will use the same HTML page we have been using. I'll refer to it as `clicker.html`.*

In this section, we'll make a simple page that has a button and a counter. Each time the button is clicked, the value of the counter is incremented. We'll use the same page we have been using before.

## Set up the html file

Set up the html file such its body contains at least the following two items:

* a button with an id
* a text element of your choice also with an id

An example is below:

```html
<body>
    <p id="counter">0</p>
    <button id="trigger">Click me</button>
</body>
```

You can have other elements and other styling to make it nicer, but that is up to you.

## The script

### Adding an event listener

Add an event listener to the `script.js` file that listens for the DOM content to be loaded, upon which it adds an event listener to the button. Here's how that looks:

```js
document.addEventListener("DOMContentLoaded", function() {
    let button = document.getElementById("trigger");
    button.addEventListener("click", updateCounter);
});
```

`updateCounter` is the function that we will use to update the counter on the page.

### `updateCounter` function

Inside this function we need to do the following:

1. Get the counter element from the page
2. Get its text value
3. Increment it by 1
4. Set the counter element's inner value to the new value

Try doing this yourself before looking at the code below.

{% hint style="info" %}
Note that an element's inner text is always returned as a string, so you need to convert it to an integer. This can be done with the `parseInt` function:

```js
parseInt("4324"); // returns 4324
```

{% endhint %}

```js
function updateCounter() {
    // query for the element
    let counterElement = document.getElementById("counter");
    // get the counter value
    let count = parseInt(counterElement.innerText);
    // increment the counter value
    count++;
    // set the counter value
    counterElement.innerText = count;
}
```

### Test it out

That's all we had to do. Now, test out the code by saving all files, reloading the page, and then clicking the button. The counter value should get incrememented:

<figure><img src="/files/kEAK7VTY1hp2rxwZsR8b" alt=""><figcaption></figcaption></figure>

## Bonus (try it yourself)

Try adding the following features:

1. When the counter value reaches a non-zero multiple of 10, its font color should change to gold, and set back to original color otherwise
2. Add a second button that resets the counter variable to 0
3. Add a third button that decrements the counter, but no longer decreases after reaching 0.

<figure><img src="/files/aVRKkQafCKrn1eFkZ3Zr" alt=""><figcaption></figcaption></figure>

A sample solution can be found [here](https://github.com/Dinoman44/Orbital2024-JS-example-problems-code/tree/main/click-counter).

## Next steps

Next we'll look at how elements can be created or removed in a page.


# Editing the DOM

We've so far seen how to query and update the DOM, but JS allows us to create and delete elements as well.

## Creating elements

To create an element, simply use the `document.createElement` method, passing in the tag name:

```js
let newElement = document.addElement("a");
```

The above line creates a new `<a>` element.

### Setting attributes

When a new element is made, it is a raw element with no attributes. However, we can use the same methods as earlier to set attribute values for the element:

```js
newElement.className = "purple-link";
newElement.href = "https://orbital.comp.nus.edu.sg/";
```

The second way is to use the `setAttribute` method:

```js
newElement.setAttribute("className", "purple-link");
newElement.setAttribute("href", "https://orbital.comp.nus.edu.sg/");
```

Both methods have the same result, but it is convention to use the second method when new attributes are being added to an element.

An element's styling can be changed the same way as before, using the `style` attribute.

### Adding text

To add some text inside a newly created method, you can use one of two methods. The first involves the `innerText` property:

```js
let newElement = document.addElement("a");
newElement.setAttribute("className", "purple-link");
newElement.setAttribute("href", "https://orbital.comp.nus.edu.sg/");
newElement.innerText = "Link to Orbital page";
```

The second way is to create a special element, called a Text Node, and add that to the element:

```js
let newElement = document.addElement("a");
newElement.setAttribute("className", "purple-link");
newElement.setAttribute("href", "https://orbital.comp.nus.edu.sg/");
let text = document.createTextNode("Link to Orbital Page");
newElement.appendChild(text); // see below on how to add elements to the page
```

## Adding elements to the page

Once the element has been created, it is time to add it to the page so it renders. There are a number of ways to do this, depending on where on the page the element has to go.

### Insert before

If the new element has to be inserted before an existing one, then the `insertBefore` method is useful:

```js
let tag1 = document.createElement("div"); // new element we make
let tag2 = document.getElementById("tag2"); // already present on page
document.body.insertBefore(tag1, tag2); // tag1 inserted before tag2 in the body
```

If the new element has to be inserted before an existing one *inside* some parent element that is not the body, the following code will work:

```js
let newChild = document.createElement("div"); // new element to make
let otherChild = document.getElementById("other-child"); // element to insert before
let parent = document.getElementById("parent"); // element to insert inside of
parent.insertBefore(newChild, otherChild);
```

The above code will turn this:

```html
<div id="parent">
    <div>Thing</div>
    <div>Thing again</div>
    <div id="other-child">Something</div>
    <div>Something again</div>
</div>
```

Into this:

```html
<div id="parent">
    <div>Thing</div>
    <div>Thing again</div>
    <div></div> <!-- the new element we made -->
    <div id="other-child">Something</div>
    <div>Something again</div>
</div>
```

### Insert at the end

To insert an element at the end of a parent element, using the `appendChild` method:

```js
let newChild = document.createElement("div"); // new element to make
newChild.innerText = "Child 5"; // add some text
let parent = document.getElementById("parent"); // element to insert inside of
parent.appendChild(newChild);
```

The above code will turn this:

```html
<div id="parent">
    <div>Child 1</div>
    <div>Child 2</div>
    <div>Child 3</div>
    <div>Child 4</div>
</div>
```

Into this:

```html
<div id="parent">
    <div>Child 1</div>
    <div>Child 2</div>
    <div>Child 3</div>
    <div>Child 4</div>
    <div>Child 5</div>
</div>
```

## Removing elements from the page

Let's say we select an element that we want to remove from a page:

```js
let elementToRemove = document.getElementById("remove-this");
```

First, we need its parent element:

```js
let parentElement = document.getElementById("parent");
```

Then we can use the `removeChild` method:

```js
parent.removeChild(elementToRemove);
```

### Replace one element with another

To replace one element with another, use the `replaceChild` method:

```js
let newChild = document.createElement("div");
let oldChild = document.getElementById("old-child");
let parent = document.getElementById("parent");
parent.replaceChild(newChild, oldChild);
```

## Next steps

Next, we'll look at fetch requests and how they can retrieve information without having to reload the page. We'll do so with an example of the NUSMods API.


# Fetch Requests

Fetch requests are a way to send HTTP requests to a server directly from a page using JavaScript. The advantage of using fetch requests over, say, reloading the page are that

1. The contents of the page do not need to be reloaded each time the request is made
2. The request takes a little less time to get processed compared to sending server-to-server requests from the backend.
3. Allows content to be loaded "optionally" - content is not loaded on the page until it is specifically requested by the user, so pages have less content and are rendered faster.

## Syntax

Fetch requests follow this syntax:

```js
fetch(resource, options)
```

* `resource` is the server (or **resource**) that the request is being sent to. It is a string with the URL.
* `options` contain some custom settings for the request. More details later

They return a ["Promise"](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) - in other words, they return an asynchronous wrapper for the response from the server.

{% hint style="info" %}
For those of you who have taken CS2030/S, Promises in JavaScript are like CompletableFutures in Java.
{% endhint %}

This promise can be processed using the `.then` method, which takes in a unary function. This function is applied on the response recieved from the server.

```js
fetch(resource, options)
.then(processResponse);
```

## The response

The response from a fetch request is a JSON object with parameters like `status` and `protocol` and `body`. Of these, the `status` and `body` parameters are the most useful for now.

`response.status` is an integer representing the HTTP Response Code returned by the server. The full list of response code can be found [here](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status), but a few common ones are below:

* `200 OK` - all good
* `404 NOT FOUND` - the url/resource was not found on the server side
* `500 Internal Server Error` - the server ran into an error when trying to process the request

`response.body` is an encoded object that contains the body of the response. Note that failed requests have `null` as the value for the `body` property for their response objects. To decode the body into readable JSON that can be processed by your JS code, use the method `response.json()`.

You could imagine using the `status` property in this way:

```js
fetch("somedomain.com/some-route/", someData)
.then(response => {
    if (response.status === 200) {
        let data = response.json(); // get the data
        // do something with the data
    } else {
        // show the user some error message
        alert(`Error: ${response.status}`);
    }
})
```

This status-checking and body-retrieval is often chained into 2 different promises using two sequential `then`s:

```js
fetch("somedomain.com/some-route/", someData)
.then(response => response.status === 200 ? response.json() : response.status)
.then(data => {
    if (typeof(data) === "number") {
        // show the user some error message
        alert(`Error: ${response.status}`);
    } else {
        // process the data
    }
})
```

Let's walk through it step by step:

1. `fetch("somedomain.com/some-route/", someData)` - this line performs the fetch request and returns the response wrapped in a promise
2. `.then(response => response.status === 200 ? response.json() : response.status)` - here, we check the response status code to see if the request was successful.
   * If the response was successful (i.e. `response.status === 200`) then we return the body of the response using `response.json()`
   * If the response was not successful, we return the status code of the response so it can be shown to the user
3. `.then(data => {...})` - this section of the code takes the data returned after processing the response
   * If the data returned is a number (i.e. `typeof(data) === "number"`) then this means that the response status code was returned by the previous function, so there was an error, so we tell the user as such
   * Otherwise process the data as normal

Note that you may want to add more intermediary conditions and process different response codes differently, but the general form of a fetch request is as above.

## The options

Note that the `fetch` function takes in two parameters: the `resource`, and the `options`.

`options` is a JSON object that contains any custom settings that you may want to apply to the request. This includes header data, type of content being sent (if any), the content itself, and others. If any settings are ommitted, then the browser plugs in the default values for each setting. Below is an example of the options object, filled with default values for each setting (not all possible settings are shown).

```js
let options = {
    method: "GET",
    mode: "cors",
    cache: "default",
    credentials: "same-origin",
    redirect: "follow",
    headers: {
        "Content-Type": "application/json",
        "Access-Control-Allow-Origin": "*"
    },
    body: JSON.stringify(someJSONData)
}
```

Let's look at each setting:

1. `method` - this setting specifies the request method. There are 4 commonly used methods(`GET`, `POST`, `PUT` and `DELETE`), and each are used in different situations. More details [here](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods)
2. `mode` - what resource sharing mode should be allowed. `"cors"` stands for "Cross-Origin Resource Sharing", and this mode allows sharing resources across origins. The other options are `"no-cors"` and `"same-origin"`. More details [here](https://developer.mozilla.org/en-US/docs/Web/API/Request/mode)
3. `cache` - how the request interacts with the browser's cache. Some different options are "no-cache", "no-store", "reload", and "only-if-cached". These options are explained [here](https://developer.mozilla.org/en-US/docs/Web/API/Request/cache)
4. `credentials` - this specifies whether or not the user should send/receive cookies from the resource. The two other options are `"omit"` (never send/receive cookies) and `"include"` (always send/receive cookies). More details [here](https://developer.mozilla.org/en-US/docs/Web/API/Request/credentials)
5. `redirect` - this specifies how to handle if the resource redirects our request elsewhere. Some other options are `"error"` and `"manual"`. More details [here](https://developer.mozilla.org/en-US/docs/Web/API/Request/redirect)
6. `headers` - this is a JSON object that contains the headers for the request. The sample shows two header settings:
   * `"Content-Type"` - this header specifies the type of content being requested. It is in the format `"type/subtype"` and some examples are `"image/png"` (a png image), `"text/plain"` (plain text), or `"multipart/form-data"` (multiple parts of form data). The full list of options can be found [here](https://www.geeksforgeeks.org/http-headers-content-type/).
   * `"Access-Control-Allow-Origin"` - this header indicates whether the response can be shared with requesting code from the given origin. This could be `"*"`, so all origins can have access to the response, or `"<origin>"` where `<origin>` is a domain/IP address. More details [here](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Origin)

* There are many more headers that can be assigned, and they are all listed [here](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers).

7. `body` - this setting specifies the body of the request (i.e. some data that needs to be processed by the resource server). Note that this must be a string; `JSON.stringify` allows us to represent a JSON object with the request body as a string. Note also that a `body` CANNOT be present for a `GET` request.

The [Mozilla Developer Network web docs](https://developer.mozilla.org/en-US/https://developer.mozilla.org/en-US/) are a very very useful resource for understanding HTTP requests and responses, as well as frontend web development with JavaScript.

## Next steps

Next, we'll write our own fetch request to the NUSMods API.


# Exercise: The NUSMods API

I'm sure you know what [NUSMods](https://nusmods.com/) is, its a website that has details about every course offered at NUS, as well as a degree planner, a timetable builder and a map of the campus. It also has its own API ([documentation here](https://api.nusmods.com/v2/)) which we are going to use.

### The task

The task is to build a simple page that will have a form. The form will allow a user to type in a course code, and once submitted the form will submit a fetch request to the NUSMods API, request the course data, and then display it to the user. If the course code they enter is invalid, then show them an error message.

It is recommended to try the first steps on your own, until you get to the fetch request, to practice writing HTML and JS.

## Some info

The API we will use is the one to get the course info given a course code and year. The general URL format is below:

```http
https://api.nusmods.com/v2/{acadYear}/modules/{courseCode}.json
```

Example:

```http
https://api.nusmods.com/v2/2023-2024/modules/CS2030S.json
```

Copy paste the above URL into your browser address bar to see the response. You'll see the response body in JSON format, like below.

<figure><img src="/files/pQTYD6CjXOmm3zPJSELS" alt=""><figcaption></figcaption></figure>

Note: The course code needs to be in all capital letters for the request to succeed.

For this exercise, we'll stick to the year 2023-2024.

## Step 1 - Get the form set up

So the first step is to ready the form. It will have the following:

* A text input field that must have a value before the form is submitted
* A submit button

Both of these, along with the form, must be uniquely identifiable as well.

Here's how that will look:

```html
<form id="nusmods-form">
    <input id="course-code" placeholder="Course code (ex: MA1301)" required>
    <button type="submit">Find course</button>
</form>
```

Note that we do not define an `action` or `method` attribute because we do not want the form to actually submit, we just want it to trigger a fetch request.

## Step 2 - Ready a container

Once the request completes, we'll need to put the course details somewhere, so it's a good idea to have a container element ready to accomodate the data. You could also directly place the details in the body, but a container helps to structure the page better. The container should not initially be visible to the user.

```html
<div id="course-container" hidden></div>
```

(Optional) You may also want to have a separate element ready to show an error message to the user, unless you intend to display an alert instead.

## Step 3 - Add an event listener to the form

Now we need to add a listener to the form that will wait till it is submitted. Then it should trigger a fetch request.

```js
document.addEventListener("DOMContentLoaded", function() {
    document.getElementById("nusmods-form").addEventListener(
        "submit",
        () => {
            event.preventDefault(); // prevent the form from submitting to the default path "/" using this line
            // note: event is deprecated, but still works
            // try instead to look at form validation functions
            // or try using this: arguments[0].preventDefault();
            getCourseData(); // this function will have the fetch request
        }
    )
})
```

{% hint style="info" %}
When a form is submitted, it automatically submits data to its action URL. In this case since there is no action URL defined, the form attempts to submit the input to the page itself, which reloads the page and prevents the rest of the code from being executed. To prevent this, we have the line `event.preventDefault()` in the listener function.
{% endhint %}

{% hint style="warning" %}
`event` is a deprecated global variable that refers to the event in question. It is better to use form validation functions to prevent forms from submitting. In this case I am using `event` because it is easier, and form validation is out of the scope of this guide.
{% endhint %}

## Step 3.5 - Prepping the data for the fetch request

In our `getCourseData` function from above (or whatever you decided to name the function), we need to ready the course code. This is nothing much, just get the input field value and make it all caps:

```js
function getCourseData() {
    let inputField = document.getElementById("course-code");
    let courseCode = inputField.value.toUpperCase();
    // fetch request here
}
```

Note that the `value` attribute of an input field returns the value of the field, which in this case is whatever the user has typed into the text field.

## Step 4 - The fetch request

Now comes the fetch request. Recall that to get course details based on course code, we submit a request to this URL:

```http
https://api.nusmods.com/v2/2023-2024/modules/{courseCode}.json
```

So the `resource` parameter for the fetch request will be the above URL with the user's input plugged in. As for the `options` parameter, luckily we do not need to specify any because the default options suffice!

So the fetch request looks like this:

```js
fetch(`https://api.nusmods.com/v2/2023-2024/modules/${courseCode}.json`, {});
```

Next, we need to check the response status. The API will return `200` if the course code was valid, `404` if it wasn't, and other codes indicate other unforeseen errors. You can decide how you want to process it, but in this case since we don't care what the response code is, we can return `null` if there is an error.

```js
fetch(`https://api.nusmods.com/v2/2023-2024/modules/${courseCode}.json`, {})
.then(response => response.status === 200 ? response.json() : null);
```

Lastly, we need to process the data and present it to the user. For starters, try displaying the course code, course title, description, and how many units it is.

Here's the basic skeleton:

```js
fetch(`https://api.nusmods.com/v2/2023-2024/modules/${courseCode}.json`, {})
.then(response => response.status === 200 ? response.json() : null)
.then(data => {
    if (data === null) {
        // course code was invalid, display an error
    } else {
        // data contains the course details in a JSON format
        // read the documentation to see what properties are relevant to your needs
        // OR copy paste the example URL from the start of this section into your browser
        // to see the JSON for yourself
    }
});
```

That's the fetch request completed, now in the body of `then` we need to display the course details

## Step 5 - Display the course details

There are 3 main ways to do this, and it's up to you to choose which one. The implementation is left to you as an exercise.

### Approach 1 - The builder method

This approach involves creating elements and adding the text inside of them, then appending them inside the course-container element we made earlier. Make use of the `document.createElement` method, the `element.appendChild` method and (optionally) the `document.createTextNode` method.

The advantage of this is that it is quite flexible to changes in specification. For instance, if I decide to also show the prerequisites and corequisites of every course, its easier to just edit the JS file to create a couple new elements.

The disadvantages of this approach is that every time the fetch request is run, the elements are re-created and re-added to the page.

{% hint style="info" %}
This approach is fine for instances where requests are not very often (or happen only once or at most twice after the page loads), but in our case it is slower than the other approaches.
{% endhint %}

### Approach 2 - The select-and-change method

For this approach, you'll need to edit the html to add elements for the course details to be contained within, and give them all ids. An example is below:

```html
<div id="course-container" hidden>
    <h1 id="cc"></h1>
    <h2 id="course-title"></h2>
    <p id="course-description"></p>
    <p id="credits"></p>
</div>
```

You'll then need to query for these elements and edit their `innerText` properties to contain the data needed.

The advantage of this approach is that the elements are only created once, and you just need to change their values.

The disadvantage of this approach is that the code is not that flexible to changes in specification, since you need to edit both the HTML AND the JS script to implement any changes.

{% hint style="info" %}
This approach is useful for instances where multiple fetch requests could be performed by the user, like this one.
{% endhint %}

### Approach 3 - The lazy method

This approach is the quickest way to do it. Remember the `innerHTML` property? It holds *everything* inside the element in question, including nested elements. This means that it is possible to add elements inside of another one by way of the `innerHTML` attribute:

```js
let element = document.querySelector("div");
element.innerHTML = "<h1>Heading</h1>";
```

The above code will insert an `<h1>` into the div that was selected.

This means we can just format the entire contents of the course-container div into a string, and set the element's `innerHTML` to that string.

The advantage of this approach is that it is fast and easy to write, and it is easy to implement changes to the specification.

The disadvantage is that this approach allows for something called [HTML Injection](https://www.imperva.com/learn/application-security/html-injection/). This is when malicious code can be injected into a webpage, and one of the ways to do this is to use the `innerHTML` property. Look at the code below:

```js
let element = document.querySelector("div");
element.innerHTML = "<script>alert('HTML Injection Successful 😈');</script>";
```

This adds a `<script>` tag inside the div, which causes the code inside it to be executed. Here, the code is just showing a simple alert, but it is possible to write code which behaves much more maliciously, such as creating and submitting invisible forms, reading cookies set by the page, messing with the page content, or worse.

{% hint style="warning" %}
This method, due to the vulnerability it creates, is not recommended for use at all, except in cases where the developer has complete control over the content being added to the `innerHTML` or it is extremely certain that the content being read in is safe, and when the contents are not very long.
{% endhint %}

## The end result

<figure><img src="/files/lJaYKaKm7JutMFD32J2C" alt=""><figcaption></figcaption></figure>

The code for this exercise can be found [here](https://github.com/Dinoman44/Orbital2024-JS-example-problems-code/tree/main/fake-nusmods). Approach 1 has been used because we have not yet done a real example of creating+adding elements. Approach 2 has been left as an exercise, and approach 3 has been demonstrated just to show how it works.

## Next steps

This is the end of the JavaScript guide! If you would like to practice more fetch requests, I suggest testing out some of the API endpoints of [Reqres](https://reqres.in/). There is also an example [here](https://github.com/Dinoman44/Orbital2024-JS-example-problems-code/tree/main/reqres-demos) that shows a couple of the endpoints, and a two fetch request examples (one of them using a `PUT` request, so take a look at the `options` JSON for that request).

The next guide will be on React, a framework of JavaScript that allows to combine HTML, CSS and JS into a single abstraction to make frontend development slightly easier.


# React

## Introduction

This guide is a step-by-step introduction to **React.js**, one of the most powerful libraries for building modern, interactive web applications. With its **component-based architecture**, **virtual DOM**, and **declarative approach**, React has changed how developers build for the web.

Throughout this guide, you’ll learn core concepts through hands-on examples and progressively build your understanding of React’s capabilities. Here’s what we’ll cover:

### **Part 1**

* State and JSX interaction
* Component structure and file organization
* Props and dynamic rendering
* Events, spread operator, and preventing defaults
* Conditional rendering
* Callbacks, destructuring, and immutability
* Lifting state up

### Part 2

* API calls with `fetch` and async handling
* Side effects with `useEffect`
* Third-party libraries and custom components
* Reading docs and using external tools
* Persisting data with a backend
* Deploying your app

## Prerequisites

To get the most out of this guide, you should have a basic understanding of:

* [JavaScript](/orbital/readme-1) and [TypeScript](/orbital/typescript)
* HTML and CSS

If you’re not comfortable with these yet, we recommend brushing up on the syntax before diving in.

## Setup

Please refer to this [setup guide](/orbital/react/setup).


# Setup

## Step 1: Install Node.js

Download the latest **LTS version** of Node.js from the [official website](https://nodejs.org/).

Once installed, open your terminal (macOS/Linux) or Command Prompt (Windows), and verify the installation with the following commands:

```bash
$ node --version
v22.14.0

$ npm --version
v11.3.0
```

> Your versions might differ slightly — that’s perfectly fine!

## Step 2: Create a New React App

Navigate to the folder where you’d like to create your project:

```bash
$ cd path/to/your/folder
```

Run the following command to create a new React project with TypeScript:

```bash
$ npx create-react-app my-app --template typescript <your-project-name>
```

This will take a few minutes to install dependencies. Once complete, navigate into your project folder:

```bash
$ cd <your-project-name>
```

To start the development server, run:

```bash
$ npm run start
```

Your React app should now be running on <http://localhost:3000>.


# State

### What is State?

State serves as a component's memory, allowing it to store and track information between renders.

### The `useState` Hook

The `useState`  hook enables state management in functional components. It accepts an initial value and returns an array containing:

1. The current state value
2. A function to update that value

```tsx
const [currentValue, setValue] = useState(initialVal);

// example:
const [color, setColor] = useState(defaultColor);
```

Note: `useState(initialVal)`  returns an array, `["val", "func"]` destructures the array to access the elements and assign it to a specific name.

### React's Rendering Process

When a component's state or props change, React performs a rerender by:

1. Destroying the current component instance (including all variables and functions)
2. Recreating it with the updated state values

During this process, React maintains state consistency by providing the latest values to the recreated component. The initial value is only used on the first render.

### Implementing State Variables

To add state:

1. Import useState:

```tsx
import { useState } from "React";
```

2. Replace regular variables with state:

```tsx
// Before
let counter = 0;

// After
const [counter, setCounter] = useState(0);
```

3. Update state using a setter function:

```tsx
function increment() {
    setCounter(counter + 1);
}

// or you can declare an arrow/anonymous function, which is more modern
const increment = () => setCounter(counter + 1);
```

### Additional resources:

{% embed url="<https://react.dev/learn/managing-state>" %}

{% embed url="<https://www.w3schools.com/react/react_state.asp>" %}


# React Native

This guide aims to progressively introduce various concepts necessary to start using Git to manage your project.

## Using this guide

Before starting the guide, it is highly recommended that you read up on javascript and CSS. It will be good to have some React knowledge though this guide will provide enough guidance even without React knowledge.


# Setup

### Why React Native?

* It is cross platform (Android and iOS phones can use my app)
* It is very similar to React (React is arguably the most popular framework for web development right now, skills can be easily transferrable between them)

### Setting up React Native (Expo)

> Expo is a set of tools and services built around React Native and, while it has many features, the most relevant feature for us right now is that it can get you writing a React Native app within minutes. You will only need a recent version of Node.js and a phone or emulator.

In other words, Expo abstracts away a lot of the nitty gritty details of dealing with React Native. You can focus on building the app instead of spending time on configurations.

#### Setting up Expo Go

Follow the instructions [here](https://reactnative.dev/docs/environment-setup).

#### Setting up Android emulator

Follow the instructions [here](https://docs.expo.dev/workflow/android-studio-emulator/).

### Getting started

1. Open the project in the IDE of your choice (eg. VSCode)
2. Open a new terminal window
3. Run `npx expo start`
4. Press `i` to bring up your iOS emulator (only for MacOS) or press `a` to bring up your Android emulator
5. Try editing some text and see the changes in real time!


# Intro to JSX

## Basics

Consider the following:

```jsx
const element = <div>Hello World</div>;
```

It looks like a mix of javascript and HTML, and it is called JSX, which you will see a lot in React based applications. It contains an opening tag and a closing tag.

In React Native, the idea behind it the same, except the syntax might be slightly different:

<pre class="language-jsx"><code class="lang-jsx"><strong>import { Text } from 'react-native';
</strong>
const element = &#x3C;Text>Hello World&#x3C;/Text>;
</code></pre>

In this example, we declare a variable name and embed it inside the JSX element with curly brackets:

```jsx
import { Text } from 'react-native';

const name = "Justin";
const element = <Text>Hello {name}</Text>;
```

JSX can contain children:

```jsx
import { Text, View } from 'react-native';

const name = "Justin";

const element = (
    <View>
        <Text>Hello {name}</Text>
        <Text>Nice to meet you!</Text>
    <View>
);
```

## Components

Components are independent and reusable bits of code.

<pre class="language-jsx"><code class="lang-jsx">import { Text } from 'react-native';

const TestComponent = () => {
<strong>    const name = "Justin";
</strong><strong>
</strong>    return (
        &#x3C;View>
            &#x3C;Text>Hello {name}&#x3C;/Text>
        &#x3C;/View>
    );
}
</code></pre>


# Basic Syntax

## These two are equivalent

**Using const syntax for javascript**

```javascript
const logHelloWorld = () => {
    console.log("Hello World");
}
```

**Using function syntax for javascript**

```javascript
function logHelloWorld() {
    console.log("Hello World");
}
```

## These two are equivalent

**Using const syntax for JSX**

```jsx
const App = () => {
    return (
        <View>
            <Text>This is an app</Text>
        </View>
    );
}
```

**Using function syntax for JSX**

```jsx
function App() {
    return (
        <View>
            <Text>This is an app</Text>
        </View>
    );
}
```

## These two are equivalent

**Closing tag for components that do not encapsulate anything**

```jsx
function App() {
    return (
        <Button title="Press Me"></Button>
    );
}
```

**Self-closing tag for components that do not encapsulate anything**

```jsx
function App() {
    return (
        <Button title="Press Me" />
    );
}
```

## These three are equivalent

**Nesting components together**

```jsx
const MainComponent = () => {
    return (
        <View>
            <View>
                <Text>Subcomponent one</Text>
            </View>
            <View>
                <Text>Subcomponent two</Text>
            </View>
        </View>
    );
}
```

**Extracting the components into their own components within the same page**

```jsx
const MainComponent = () => {
    return (
        <View>
            <SubComponentOne />
            <SubComponentTwo />
        </View>
    );
}

const SubComponentOne = () => {
    return (
        <View>
            <Text>Subcomponent one</Text>
        </View>
    );
}

const SubComponentTwo = () => {
    return (
        <View>
            <Text>Subcomponent two</Text>
        </View>
    );
}
```

**Extracting the components into their own components into other files, export them and importing them for use**

```jsx
// index.jsx
import SubComponentOne from "./component-one.jsx"
import SubComponentTwo from "./component-two.jsx"

const MainComponent = () => {
    return (
        <View>
            <SubComponentOne />
            <SubComponentTwo />
        </View>
    );
}

// component-one.jsx
const SubComponentOne = () => {
    return (
        <View>
            <Text>Subcomponent one</Text>
        </View>
    );
}
export default SubComponentOne;

// component-two.jsx
const SubComponentTwo = () => {
    return (
        <View>
            <Text>Subcomponent two</Text>
        </View>
    );
}
export default SubComponentTwo;
```


# Handling UI

Think of it as a tool that allows you to to create stuff with logic and UI. Since CS1101S (or any of the CS1010 variants) focuses mainly on logic, this might be the first time you are dealing with UI.

UI is generally handled with the JSX syntax in the previous section. For React Native, we have some basic out-of-the-box components that we can use.

## Common React Native Components

These components are enough to solve 90% of your needs.

* `View`: A container like `div`
* `ScrollView`: Like `View` but Scrollable
* `Text`: Displays texts
* `Button`: Supports touches
* `TouchableOpacity`: Like `Button` but can encapsulate Button
* `TextInput`: Supports inputting texts
* `Image`: Display images

You can read up on other components [here](https://reactnative.dev/docs/components-and-apis).

## Organizing Components

Components can be organised using [flex box](https://css-tricks.com/snippets/css/a-guide-to-flexbox/). Understanding the 4 concepts below can meet 90% of your needs. Later on we will try to create the mockup for NUS NextBUS using these concepts alone.

### **Flex direction**

<figure><img src="/files/zDD8aenrKaPRbfByrhiU" alt=""><figcaption><p>flex direction</p></figcaption></figure>

* `row`: organise components horizontally
* `column`: organise components vertically

### **Justify content**

<figure><img src="/files/tZ3XMvyidVy1caZZydQZ" alt=""><figcaption><p>justify content</p></figcaption></figure>

* `space-between`: Spread out components evenly. First component at the left end, last component at the end.
* `center`: Components are centered.

### Styling UI components

You can style components using the `StyleSheet`:

```jsx
import { StyleSheet, Text, View } from 'react-native';

const App = () => (
  <View style={styles.container}>
    <Text style={styles.title}>React Native</Text>
  </View>
);

const styles = StyleSheet.create({
  container: {
    flex: 1,
    backgroundColor: '#eaeaea',
  },
  title: {
    marginTop: 16,
    backgroundColor: '#61dafb',
    color: '#20232a',
    fontSize: 30,
    fontWeight: 'bold',
  },
});
```

Alternatively, you can style components using CSS libraries such as [Tailwind](https://tailwindcss.com/). There are different paradigms to approach styling components but the general idea is the same - they all uses concepts from CSS. You can visit the [CSS](/orbital/readme-1/css) section in the wiki for more info!

{% hint style="info" %}
You can set the background colour to black to visualise the space that the component takes up.
{% endhint %}


# Props

Props is how react components communicate with each other. Every parent component can pass information to its child components by giving them props.

We've already seen props (short for properties) before, like how we pass in `onPress` is passed to the `Button` component, or how the `styles` is passed to the `View` component. Just like how functions can take in arguments, components can take in properties.

```jsx
import React from 'react';
import { View, Text, Button } from 'react-native';

// Child component
const Greeting = (props) => {
  return (
    <View>
      <Text>Hello, {props.name}!</Text>
    </View>
  );
};

// Parent component
const App = () => {
  return (
    <View style={styles.container}>
      <Greeting name="Alice" />
      <Greeting name="Bob" />
      <Greeting name="Charlie" />
    </View>
  );
};
```

You can also pass functions as props!

```jsx
import React from 'react';
import { View, Button } from 'react-native';

// Child component
const MyButton = (props) => {
  return (
    <Button title={props.title} onPress={props.onPress} />
  );
};

// Parent component
const App = () => {
  const handlePress = () => {
    alert('Button was pressed!');
  };

  return (
    <View>
      <MyButton title="Press Me" onPress={handlePress} />
    </View>
  );
};
```

In this example, the MyButton component receives a title and an onPress function as props from the parent App component. When the button is pressed, it triggers the handlePress function in the parent component.

Instead of using props, you can destructure the arguments directly in the function signature. This makes it clear which props the component expects and avoids the need to repeatedly write `props`.

```jsx
import React from 'react';
import { View, Button } from 'react-native';

// Child component using destructuring
const MyButton = ({ title, onPress }) => {
  return (
    <Button title={title} onPress={onPress} />
  );
};

// Parent component
const App = () => {
  const handlePress = () => {
    alert('Button was pressed!');
  };

  return (
    <View style={styles.container}>
      <MyButton title="Press Me" onPress={handlePress} />
    </View>
  );
};
```


# State Management

## Incrementing Number

Suppose we want to build an an app with the following requirements:

1. A Text component displaying the counter value, with an initial value of 0.
2. A Button to increment the counter.
3. Another Button to decrement the counter.

### Wrong Example

Without knowledge on React state management, our knowledge of javascript would tell us to do something like this:

```jsx
function Component() {
  let counter = 0;
  
  // DON'T DO THIS - mutating counter doesn't tell React to re-render
  return <Button onPress={() => counter += 1} />;
}
```

React Native doesn’t know when to re-render a component if we simply use let counter = 0 and update it. This breaks the rules for props since it involves mutating the value directly.

### Correct Example

Instead, React provides the useState hook to manage state changes and trigger re-renders appropriately.

```jsx
function Component() {
  const [counter, setCounter] = useState(0);
  
  // This is correct - the counter value is updated in the next render
  return <Button onPress={() => setCounter(counter + 1)} />;
}
```

## Managing Text

State is used in various components, such as TextInput, where the text changes every time the user types something. We use useState to track and update the text.

```jsx
import React, { useState } from 'react';
import { View, TextInput } from 'react-native';

const TextInputComponent = () => {
  const [text, setText] = useState('');

  return (
    <View>
      <TextInput 
        style={styles.input} 
        value={text} 
        onChangeText={setText} 
        placeholder="Type here"
      />
    </View>
  );
};

```

## Conditional Rendering

Conditional rendering allows components to render dynamically based on the state. Here’s an example with a modal:

```jsx
import React, { useState } from 'react';
import { View, Button, Modal, Text } from 'react-native';

const ModalComponent = () => (
  <View>
    <Text>This is a modal!</Text>
  </View>
);

const App = () => {
  const [isModalOpen, setIsModalOpen] = useState(false);

  const toggleModal = () => {
    setIsModalOpen(!isModalOpen);
  };

  return (
    <View >
      <Button title="Toggle Modal" onPress={toggleModal} />
      {isModalOpen && (
        <Modal
          transparent={true}
          visible={isModalOpen}
          onRequestClose={toggleModal}
        >
          <View>
            <ModalComponent />
            <Button title="Close Modal" onPress={toggleModal} />
          </View>
        </Modal>
      )}
    </View>
  );
};
```

In this example, the toggleModal function toggles the isModalOpen state, which conditionally renders the ModalComponent.


# Git

This guide aims to progressively introduce various concepts necessary to start using Git to manage your project.&#x20;

## Using this guide

Before starting the guide, it is highly recommended that you give [Setting up Git and GitHub](/orbital/git/setup) a read to ensure that everything you need for this guide is correctly setup.

We also highly recommend that you read [Fundamental Concepts](/orbital/git/fundamental-concepts) to understand the terminologies that this guide uses.

The guide is broken up into three parts:

1. Fundamental concepts
2. Collaborative workflows
3. Advanced concepts

The first two parts are the most important and we highly recommend that you, at the very least, finish them. The last part is good to know but not crucial if you are in a rush.

{% hint style="info" %}
If you encounter a command with `<text>` format, you are to replace `<text>` with your own.
{% endhint %}

## Why this guide?

This guide was created as an effort by [NUS Hackers](http://nushackers.org/) to make knowledge easily available for various technical topics!

## Where can I find the slides?

The slides are published here:&#x20;

{% @github-files/github-code-block url="<https://github.com/nushackers/hackertools_materials/blob/main/git/slides.pdf>" %}

## Changelog

1. 11 May 2024: First draft for first 2 sections
2. 18 May 2024: Conducting the workshop for Orbital 2024
3. 17 May 2025: Conducting the workshop for Orbital 2025




---

[Next Page](/llms-full.txt/1)

