content/wiki/pkg/packages.md

On this page, you will find information about MatterLinux packages. 

# Format 
MatterLinux packages uses the **M**atterLinux **P**ackaging **F**ormat, `MPF`. Don't let fancy name 
mislead you, a basic `MPF` file is just a renamed Gunziped TAR archive. The reason that packages use the 
`.mpf` extension and not the `.tar.gz` extension is to make it easier to recognize and easier to work 
with in the scripts and the tools. 

### Naming
A package is named after the software and the version of that software that it provides. For 
example package containing `bash` version `5.2.15` is named `bash_5.2.15.mpf`.

### Structure 
File structure of a package follows this format:
```
package_1.0.mpf
├── DATA
├── CHANGES
├── INSTALL
├── HASHES 
└── files.tar.gz
```
Let's break this down:
- `DATA`: An `.ini` formatted file, contains information about package name, version, description,
dependencies, files to keep (save) during update/removal and SHA256 hash of the `MPF` file.
- `CHANGES`: A changelog file, each version's changelog is split with 3 newlines (`\n\n\n`)
- `INSTALL`: An optional shell script. If it exists, then should be ran after the installation. This file
is also named the "install script".
- `HASHES`: Contains MD5 hashes of every file in the `files.tar.gz` archive. Summaries are
followed by a white space (` `) and the full file path. These paths do not start with `/`,
also they do not end with a `/`.
- `files.tar.gz`: The actual files that the package contains, which should be installed.

File structure of the `files.tar.gz` archive matches with the MatterLinux root file structure. This is important
as this archive will most likely be extracted in a MatterLinux root file system. 

For example, we can take a look at the `which` package, to do this you can download the `MPF` file,
extract it in a temporary folder, and list the contents of `files.tar.gz` with the `tar tf files.tar.gz`
command:
```
usr/
usr/share/
usr/share/man/
usr/share/man/man1/
usr/share/man/man1/which.1
usr/share/info/
usr/share/info/dir
usr/share/info/which.info
usr/bin/
usr/bin/which
```

### Install scripts
Some packages may contain an install script, `INSTALL`, this shell script is ran by the 
[MatterLinux Package Manager (`mp`)](/wiki/package_man) using the bash shell, right after the 
extraction of `files.tar.gz`.

This script is used to do post-install actions, such as adding users, groups etc.

# Working with packages
While installing a package using the [MatterLinux Package Manager (`mp`)](/wiki/package_man), `mp` 
downloads the target package(s) from the pools, these packages are in the format discussed above.
After downloading and verifying the target package(s), `mp` extracts the packages using `libarchive`. 
To learn more about this process see the [page for package management](/wiki/package_man).

### Building packages
Package are built with the `mp-build` tool. In order to build a package, you will need the source 
of the package, which can be found in the source tree of the pool which contains the package.

After obtaining the source, package can be simply built by running:
```
$ mp-build <package dir>
```

### Package scripts
Each package source contains a `pkg.sh` shell script. This is the source script that is used to 
build the package. In the build process, this shell script gets sourced by the `mp-build` tool 
using the `source` command.

Let's take a closer look at a `pkg.sh` file:
```
NAME="which"
DESC="Shows the full path of (shell) commands"
VERSION="2.21"
FILES=("https://ftp.gnu.org/gnu/which/which-$VERSION.tar.gz")
HASHES=("097ff1a324ae02e0a3b0369f07a7544a")
DEPENDS=()

build() {
  tar xf $NAME-$VERSION.tar.gz
  cd $NAME-$VERSION

  ./configure --prefix=/usr && make
  make DESTDIR=$ROOTDIR install

  cd .. && rm -rf $NAME-$VERSION 
}
```
This `pkg.sh` file is for the `which` package (version `2.21`). Let's start by breaking down the 
variables:

- `NAME`: Specifies the package name. A package should be named after the software, tool or library 
it provides. Preferably the name should not contain `_` to avoid confusion with naming. 
- `DESC`: A short description about the software, tool or the library that package provides. 
Explain what it does, what it contains etc.
- `VERSION`: Version of the software, tool or library the package provides. If you are using a 
git commit version, you can name the version `LAST_VERSION+FIRST_7_CHARS_OF_COMMIT_ID`
- `FILES`: Upstream files and patches needed to build this package, you can use `http`, `https` 
or `ftp` protocols.
You can also specify multiple files. These files will be downloaded by `mp-build` in the build 
process.
- `HASHES`: Hashes for the files you specify. You can use `MD5`, `SHA1`, `SHA256` or `SHA512` hashes.
And yes, you need to specify hashes for all the files, using the same order with the `FILES` variable.
You can also specify `NOHASH` instead of a hash if you want to skip hash checking for a file.
- `DEPENDS`: Package(s) that this package depends on. 

There are also other options that are not present in this example:
- `KEEP`: A list of files to keep during the update/removal of the package
- `BUILD`: A list packages required for building this package

Now let's take a look at the `build` function. Each package needs a `build` function, this 
function is called by `mp-build` after downloading and verifying all the packages. It will be 
called in the `$ROOTDIR`. This directory will contain all the downloaded files, any files 
in this directory will be included into the build, so don't forget to cleanup. 

> **Note** 
> 
> You don't need to cleanup the downloaded files in the package script, 
> they will be cleaned by the `mp-build`.

- `tar xf $NAME-$VERSION.tar.gz`: Extract the downloaded archive file.
- `cd $NAME-$VERSION`: Change directory into the extracted directory.
- `./configure --prefix=/usr && make`: Builds the `which` tool, different packages may have 
different build instructions. These instruction are usually provided by the upstream. You can 
also check out [LFS](https://www.linuxfromscratch.org/lfs/view/12.0-systemd/) and [BLFS](https://www.linuxfromscratch.org/blfs/view/12.0-systemd/) for instructions.
- `make DESTDIR=$ROOTDIR install`: Install the package. Make sure that you are installing the package 
into the `$ROOTDIR` and not the root file system.
- `cd .. && rm -rf $NAME-$VERSION`: Change directory back into `$ROOTDIR` and clean the extracted archive.

Check out [`base`](https://git.matterlinux.xyz/Matter/base) and [`desktop`](https://git.matterlinux.xyz/Matter/desktop)
pool sources for more example `pkg.sh` scripts.