AI (LLMs)

Deep dive into LLMs and how they work.

Algorithms & Data Structures

Time Complexity

The amount of steps required for an algorithm to execute.

Big O Notation

Maximum time for n input size. (Upper bound - worst case)

Omega Notation

Minimum time for n input size. (Lower bound)

• Ω(n²)
• Ω(n log n)
• Ω(n)
• Ω(log n)
• Ω(1)

If both are the same, use θ

Searching Algorithms

Linear Search

Check every element until n is found.

Building a Computer From Scratch

From diodes and transistors, to a Turing complete computer.

C Compiling

A compiler converts C code into machine code in 4 steps:

1. Preprocessing (Convert all preprocessor instructions (#…) to C code.)
2. Compiling (Convert C code to assembly.)
3. Assembling (Compile the necessary libraries.)
4. Linking (Merge the compiled code with the compiled libraries.)

Libraries

Libraries are pre-written collections of code that can be reused in other programs. On UNIX systems, they are usually located in the /lib/ and /usr/include directories.

Math.h

For example, math.h is very useful to implement complex arithmetic operations.

C Language

The C Programming Language

C Snippets

Cast Strings to Numbers

The atoi() function in stdlib is implemented similarly to the one below.

ASCII encodes numbers in order, after special characters.

The encoded value for '0' is 48, so subtracting any numeric char by 48 outputs its real numerical value.

char number = '7';
int result = number - 48;
int same_result = number - '0';

Algorithm to convert strings to numbers:

int str_to_int(char *str) {
	int result = 0;
	for (int i = 0; str[i] != '\0'; i++) {
		if (str[i] < '0' && str[1] > '9') return -1; // Error if NaN
		result = (result * 10) + (str[i] - '0');
	return result;
}

(result * 10) is shifting the previous number to the left, as it is an order of magnitude above the following digit.

Databases [SQL]

Flat-File Databases

A “table” written into a single file. The most common file type for this purpose is CSV.
The CSV format reads each line as a row, and each comma-separated value as a column.
The first row on a CSV file is used to describe the data in each column. If a , is present on the dataset, surround that entry with " to ’escape’ it.
These files can be read and written to using languages like c-language and python.

Debugging [GDB]

GDB Debugging

Compile with Debug Information

To allow gdb access to the source code (Provides debug symbols - Do not share publicly as it contains the source code).

gcc -g <file>

Look for / Fix Bugs

First, initialize gdb with the executable to debug.

gdb ./<executable> --tui

After gdb is ready, we can use the following commands:

Check for Memory Leaks

Use valgrind to check for lost memory.

Encryption [GPG]

Symmetric Encryption

• User A sends a password to user B.
• The password is used to encrypt the messages.
• A secure way to share the password is required.

Asymmetric Encryption

• Users A and B have a public key and a private key.
• The public keys are shared, and they are used to encrypt the messages.
• The users can use their private keys to decrypt the messages.

GPG

GPG uses asymmetric encryption to sign, encrypt and decrypt files.

Firewall [UFW]

A firewall monitors and controls all incoming and outgoing network traffic, and can be implemented at the hardware or software level.

See All Open Ports

All ports opened by a program, including those blocked by a firewall.

sudo ss -tupln

Software Firewall (UFW)

# Enable the SSH port if it is being used
sudo ufw limit 22/tcp # `limit` is used to prevent bruteforce

# Set default connection settings
sudo ufw default deny incoming
sudo ufw default allow outgoing

# Open ports for the services running
sudo ufw allow 80/tcp   # Web server 1
sudo ufw allow 443/tcp  # Web server 2

# Check if UFW is running and is configurations
sudo ufw status numbered

# Delete an entry
sudo ufw delete 2 # Number given by `status numbered`

# Enable UFW
sudo ufw enable

Block Pings

Blocking pings prevents hackers from quickly discovering the server. It is still possible to scan all ports on the server and eventually find any open one, but it adds another layer of security.

Flask

Flask is a web framework for Python. It facilitates the creation of web apps (dynamic web pages).

Run Flask

Flask comes with its own server for debugging purposes, which can be started with:

flask run

Folder Structure

app.py               # main code
requirements.txt     # required libraries
static/              # files that never change
templates/           # dynamic files

“Hello, name” — Example App

templates/index.html

<!DOCTYPE html>
<html lang="en">
	<head>
		<meta name = "viewport" content="initial-scale=1, width=device-width">
		<title>Hello</title>
	</head>
	<body>
		hello, {{ name_placeholder }} <!-- Jinja template -->
	</body>
</html>

app.py
http://website.domain/?name=Trude

HTML

HTML is a markup language: The foundation of every website, as it structures content and provides information such as text, images and other media to the browser.

Hello World

HTML is not a programming language, only formatting to write a document as. The following ‘code’ is valid HTML.

Hello, world.

A more complete solution, however, would be to define a structure for the document.

 <!DOCTYPE html>
 <html lang="en">
  <head>
   <meta charset="UTF-8" />
   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
   <meta http-equiv="X-UA-Compatible" content="ie=edge" />
   <title>Document</title>
  </head>
  <body>
   <p>Hello, world.</p>
  </body>
 </html>

HTML uses tags to define a hierarchy. The <head> tag defines metadata for the site, such as the page’s title, encoding types, and any external resources. The <body> tag is the content itself: any paragraphs (<p>), images, forms and scripts, to name a few.

HTTP [CURL]

HTTP

HTTP (Hypertext Transfer Protocol) is a communication protocol used to send messages between the client and server, mainly used for the web. It’s stateless, meaning each request is independent, which is why web browsers often use cookies to save state.

Request Structure

1. Request Line: Method, URI, HTTP version (e.g., GET /index.html HTTP/1.1).
2. Headers: Metadata about the request.
3. Body (Optional): Data for POST, PUT, PATCH requests.

Response Structure

1. Status Line: HTTP version, status code, reason phrase (e.g., HTTP/1.1 200 OK).
2. Headers: Metadata about the response.
3. Body (Optional): Response data (HTML, JSON, etc.).

Methods

• GET: Retrieve a resource. Should only retrieve data and not have side effects.
• POST: Submit data to be processed.
• PUT: Replace a resource.
• DELETE: Delete a resource.
• PATCH: Partially modify a resource.
• HEAD: Retrieve headers only.
• OPTIONS: Describe communication options.

HTTP Headers

HTTP headers are key-value pairs providing additional information about requests and responses.

HTTPS and SSL Certificates

HTTP(s)

The http protocol sends data as plaintext, which is an issue when sharing sensitive data such as messages and passwords.
https uses TLS to encrypt sensitive traffic between the client and server, creating a secure connection between the two.

TLS

TLS is a form of encryption, used to secure HTTPS connections.
TLS replaces SSL (a deprecated protocol), however, the term SSL is still used often.

Handshake

To establish a secure connection, a ‘handshake’ is performed between the client and the server.

IRC

The IRC Protocol

IRC is a very simple communication protocol that allows users to chat in real time. IRC is very lightweight, but does not encrypt messages by default.

Using IRC

Server

To be able to communicate, users must connect to a server. Each server has its own rules, bots and commands. The IRC protocol itself does not implement encryption, however, SSL Certificates can be used to establish a secure connection with the server.
Every message sent can be read by the server, including private messages between users.
Separate IRC instances can communicate. This concept is often called server federation. This allows for users in different servers to send messages to each other.

Linux

Kernel

Linux is a kernel: the core of an operative system. Operating systems that use the Linux kernel are called Linux Distros (Distributions).
A Kernel:

• Executes first when the computer finishes booting up and has full access to the hardware.
• Implements drivers to control peripherals, network devices and other resources.
• Runs other programs (userland software) and allows them to communicate with each other and with the hardware through system calls.

Compiling

First, install the necessary dependencies. For example, on Debian:

Password Manager [PASS]

Password Managers

A password manager is a program responsible for saving all your passwords.
You could have a single password and use it for everything, but if an attacker gets a hold of your password on just one service, they would have access to all of your accounts. Different services may have different requirements for passwords, too. To mitigate these issues, it’s recommended to use a password manager, and a unique password for each service.
There are many options available, however, pass is one of the simplest ones.

Python

Python3 Documentation

• number: Replace with a number.
• *object: Any number of objects.
• [, ndigits]: Anything between [] is optional.

Run Python Code

Python is an interpreted language. It is not compiled like C.

python hello.py

Skipping the compiling step makes the language easier to use but at the cost of performance.

Print

Print information on the console.

print("hello, world")
answer = "Some Text"
print(f"Your answer is: {answer}") # formatted string (any type)
print("Your answer is: " + answer) # answer must be string
print("Your answer is:", answer) # answer can be any type
print("Same line", end="")
print("!" * 4) # "!!!!
# '' and "" do the same.

Input

Prompt the user for information.

SSH

SSH is a protocol for accessing a terminal remotely.
For SSH to work, the remote machine needs to have an OpenSSH instance running and listening for connections, and port 22 must be allowed through any VPNs or firewalls between the client and host machines. An SSH client is also needed, be it any UNIX system (using the ssh command), or a Windows server using a client such as PuTTY.

OpenSSH Client

Connect to a Remote Server

Connect Using a Password

ssh <user>@<ip>
> <password>
ssh [email protected]
> Ctrl+D # Disconnect

After connecting to a server for the first time, the fingerprint of that server is stored in the ~/.ssh/known-hosts file.

Terminal Multiplexer [TMUX]

Overview

tmux is a multiplexer that, among other things, is able to:

• Preserve a terminal session if it is closed or lost.
• Access an old session from a new terminal window.
• Connect to a remote session and save its state once disconnected.
• Split the terminal into multiple tabs and panes.

Components

• Sessions (Only one tmux session can be used at a time. Similar to workspaces.)
• Windows (A session contains windows. These behave like tabs in other programs.)
• Panes (A split in the window, each with its own terminal instance.)

Version Control [GIT]

Git is a version control system first developed by Linus Torvalds. It facilitates collaboration on large projects, keeps track of changes, and allows mistakes to be rolled back into a previous state.

Configure Git

Git uses a hierarchy of configuration files:

• System: (/etc/gitconfig) Configuration for all users in a system.
• Global: (~/.gitconfig) Configure Git for all project of the current user.
• Local: (.git/config) Configure Git for the current project.
• Worktree: (.git/config.worktree) Configure part of a project.

# Check if user name and email are set.
git config --get user.name
git config --get user.email
# If not, set those values.
git config --add --global user.name "username"
git config --add --global user.email "[email protected]"
git config --add --global init.defaultBranch main  # GitHub's default
git config --unset example.key      # Remove a configuration value
git config --unset-all example.key  # Remove all instances of a configuration key
git config --remove-section section # Remove an entire section
# Rebase on pull by default to keep a linear history
git config --global pull.rebase true

Create a Repository

Git stores all project information in the .git directory. This includes branches, commits, and metadata.

WSL2

Windows’ WSL allows for installing a Linux subsystem on a Windows host. WSL2 enhances WSL with support for launching GUI apps as if they were regular Windows programs (or any X.org window).

Install WSL2

On recent WSL versions, this process is quite simple:

wsl --list --online           # See available distros
wsl --install -d <distro>     # Select & Install distro
wsl --set-default-version 2   # Set WSL 2 as the default version

If the previous steps fail, WSL can still be installed manually: