From 0ff4366363f8f739c171979e081d2a549fe3d8d9 Mon Sep 17 00:00:00 2001 From: ngn Date: Wed, 14 Aug 2024 01:30:42 +0300 Subject: [PATCH] update: new package system docs and reformatting --- wiki/main.md | 26 ++--- wiki/pkg/confer.json | 6 + wiki/pkg/{mc.md => confer.md} | 40 ++++--- wiki/pkg/create.json | 4 +- wiki/pkg/create.md | 156 +++++++++++++++---------- wiki/pkg/libmp.json | 6 + wiki/pkg/libmp.md | 1 + wiki/pkg/matt.json | 6 + wiki/pkg/matt.md | 145 +++++++++++++++++++++++ wiki/pkg/mc.json | 6 - wiki/pkg/mirrors.json | 2 +- wiki/pkg/mirrors.md | 29 ++--- wiki/pkg/mptp.json | 6 + wiki/pkg/mptp.md | 1 + wiki/pkg/packages.json | 2 +- wiki/pkg/packages.md | 50 ++++---- wiki/pkg/pkg_man.json | 6 - wiki/pkg/pkg_man.md | 209 ---------------------------------- wiki/pkg/pools.json | 2 +- wiki/pkg/pools.md | 44 +++---- 20 files changed, 371 insertions(+), 376 deletions(-) create mode 100644 wiki/pkg/confer.json rename wiki/pkg/{mc.md => confer.md} (62%) create mode 100644 wiki/pkg/libmp.json create mode 100644 wiki/pkg/libmp.md create mode 100644 wiki/pkg/matt.json create mode 100644 wiki/pkg/matt.md delete mode 100644 wiki/pkg/mc.json create mode 100644 wiki/pkg/mptp.json create mode 100644 wiki/pkg/mptp.md delete mode 100644 wiki/pkg/pkg_man.json delete mode 100644 wiki/pkg/pkg_man.md diff --git a/wiki/main.md b/wiki/main.md index 68cae6d..c638cc3 100644 --- a/wiki/main.md +++ b/wiki/main.md @@ -12,21 +12,21 @@ edit a page or create a new page, make a [pull request](/wiki/contribute) to [co | --------------------------------------- | ----------------------------------------------- | | [Introduction](/wiki/intro) | Learn about MatterLinux and other general stuff | | [Installation](/wiki/install) | Learn how to install a base MatterLinux system | -| [Post-Installation](/wiki/post_install) | Make your system (somewhat) usable | +| [Post-installation](/wiki/post_install) | Make your system (somewhat) usable | ## Packaging -| Name | Description | -| --------------------------------------------- | -------------------------------------------------------------------------- | -| [Releases](/wiki/releases) | Learn about MatterLinux release cycle | -| [Packages](/wiki/packages) | Learn about MatterLinux packages | -| [Pools](/wiki/pools) | Learn about MatterLinux package pools | -| [Mirrors](/wiki/mirrors) | Discover pool mirros and learn how you can setup your own | -| [Pool Guidelines](/wiki/pool_guides) | Package guidelines for the official pools | -| [Create a packages](/wiki/create_packages) | Learn how you can create and add a package to the official repos | -| [Working with mp](/wiki/mp) | Learn how to manage packages with MatterLinux package manager (`mp`) | -| [Working with libmp](/wiki/libmp) | Learn how to use the MatterLinux package library (`libmp`) | -| [Understanding MPTP](/wiki/mptp) | Learn about MatterLinux package transfer protocol (MPTP) | -| [Working with mc](/wiki/mc) | Learn how to manage your configs MatterLinux configuration manager (`mc`) | +| Name | Description | +| --------------------------------------------- | ------------------------------------------------------------------------------ | +| [Releases](/wiki/releases) | Learn about MatterLinux release cycle | +| [Packages](/wiki/packages) | Learn about MatterLinux packages | +| [Pools](/wiki/pools) | Learn about MatterLinux package pools | +| [Mirrors](/wiki/mirrors) | Discover pool mirros and learn how you can setup your own | +| [Pool guidelines](/wiki/pool_guides) | Package guidelines for the official pools | +| [Creating packages](/wiki/create) | Learn how you can create and add a package to the official pools | +| [Working with matt](/wiki/matt) | Learn how to manage packages with MatterLinux package manager (`matt`) | +| [Working with libmp](/wiki/libmp) | Learn how to use the MatterLinux package library (`libmp`) | +| [Understanding MPTP](/wiki/mptp) | Learn about MatterLinux package transfer protocol (MPTP) | +| [Working with confer](/wiki/confer) | Learn how to manage your configs MatterLinux configuration manager (`confer`) | ## Other | Name | Description | diff --git a/wiki/pkg/confer.json b/wiki/pkg/confer.json new file mode 100644 index 0000000..7d97f71 --- /dev/null +++ b/wiki/pkg/confer.json @@ -0,0 +1,6 @@ +{ + "id": "confer", + "title": "Working with confer", + "author": "ngn", + "date": "14/08/24" +} diff --git a/wiki/pkg/mc.md b/wiki/pkg/confer.md similarity index 62% rename from wiki/pkg/mc.md rename to wiki/pkg/confer.md index 84ac2e4..9e62a6b 100644 --- a/wiki/pkg/mc.md +++ b/wiki/pkg/confer.md @@ -1,16 +1,16 @@ On this page you will learn how you can use MatterLinux configuration manager. -# MatterLinux Configuration Manager -`mc` or the MatterLinux configuration manager, is a simple program that allows you to -easily install and package your configurations. +# MatterLinux configuration manager +`confer` is a simple program that allows you to easily install and package your configurations, also +known as "dot files". -### Installing `mc` -`mc` can be installed with `mp` (MatterLinux package manager), it's located in the desktop -repo: +### Installation +`confer` can be installed with `matt` (MatterLinux package manager), it's located in the desktop +pool: ``` -# mp-install mc +# matt install --yes mc ``` -In order to run `mc` as a non-root user, you will need to [install and configure `doas` or `sudo`](/wiki/post_install). +In order to run `confer` as a non-root user, you will need to [install and configure `doas` or `sudo`](/wiki/post_install). ### Packaging your configuration First of all you should create and empty git repository: @@ -19,7 +19,7 @@ $ mkdir my-config && cd my-config $ git init ``` After adding a readme and a license (or any other files you might like), create an -`mc.cfg` file. This file should follow the simple `INI` syntax. First section you should +`confer.ini` file. This file should follow the simple `INI` syntax. First section you should create in this file, is the details section: ``` [details] @@ -46,30 +46,34 @@ src = src/rofi/config.rasi for (in this case the target is for `rofi` configuration, so its named `rofi`) - `require`: A comma-separated list of packages that this configuration requires - `desc`: A simple description explaining what the target is for -- `dst`: Where should the configuration should be copied during the installation, `mc` will +- `dst`: Where should the configuration should be copied during the installation, `confer` will append the current user's home directory to this path - `src`: Where should the configuration should be copied during the packaging, this path is relative to the repository's path -After adding all the targets, run the `mc-gen` command, specifying the local path to the git repository -that contain the `mc.gen` file. `mc` will copy all the target files/directories from the `dst` to the +After adding all the targets, run the `gen` command, specifying the local path to the git repository +that contain the `confer.ini` file: +``` +$ confer gen /path/to/your/repo +``` +`confer` will copy all the target files/directories from the `dst` to the `src`. When you are done, feel free to publish the git repository and share it. ### Installing configurations -To install a configuration you need to use the `mc-pull` command. After the command, +To install a configuration you need to use the `pull` command. After the command, specify a directory, a HTTP(S) git URL, or just specify a name: ``` -$ mc-pull messier87 +$ confer pull messier87 ``` -If you just specify a name, `mc` will look for that configuration in the [config hub](/hub). -In this case, `mc` will pull down the `messier87` configuration, and walk you through the installation +If you just specify a name, `confer` will look for that configuration in the [config hub](/hub). +In this case, `confer` will pull down the `messier87` configuration, and walk you through the installation by asking yes/no questions. -After the installation `mc` will install all the requirements using `mp`, and you will be ready to go! +After the installation `confer` will install all the requirements using `matt`, and you will be ready to go! > **Important** > -> Please check the `mc.cfg` file before installing third party configuration packages +> Please check the `confer.ini` file before installing third party configuration packages > (packages that are not in the config hub), as they may include malicious configuration files > that may harm your system. diff --git a/wiki/pkg/create.json b/wiki/pkg/create.json index e8524fc..3d9ebfe 100644 --- a/wiki/pkg/create.json +++ b/wiki/pkg/create.json @@ -1,6 +1,6 @@ { - "id": "create_pkg", + "id": "create", "title": "Create a package", "author": "ngn", - "date": "23/02/24" + "date": "04/08/24" } diff --git a/wiki/pkg/create.md b/wiki/pkg/create.md index 4293a32..b951758 100644 --- a/wiki/pkg/create.md +++ b/wiki/pkg/create.md @@ -1,5 +1,5 @@ On this page, you will learn how you can create your first -MatterLinux package for the official repos. +MatterLinux package for the official pools. ## Pre-requirements - You should have an up-to-date MatterLinux installation @@ -9,75 +9,90 @@ MatterLinux package for the official repos. - You should know the basics of git version control system ## Getting started -If you want to add your package to the official repos, then please -check the [official repo guidelines](/wiki/repo_guide). Make sure +If you want to add your package to the official pools, then please +check the [official pool guidelines](/wiki/pool_guides). Make sure that your package follows the guidelines or your pull request maybe rejected. Also create an account on Gitea server and configure your editor as specified in the [contribution guide](/wiki/contribute). -## Selecting a repo -If you want to add a new package to the official repos, you should -first choose the repo you want to add the package to. If you are +## Selecting a pool +If you want to add a new package to the official pools, you should +first choose the pool you want to add the package to. If you are packaging a software that is fundamental to the OS, such as a kernel, -or a C library, you should add your package to the `base` repo. -If this is not the case then you should add the package to the `desktop` -repo. +or a C library, you should add your package to the `base` pool. -## Fork the repo -After selecting the repo, fork it on Gitea and clone it to MatterLinux +If the software is mainly targeted for server use, you should add the package +to the `server` pool. + +If none of these is the case then you should add the package to the `desktop` +pool. + +## Fork the pool +After selecting the pool, fork it on Gitea and clone it to MatterLinux system: ``` -$ git clone +$ git clone ``` -## Install `mp-repo` -MatterLinux project has few different tools used for building and creating packages, -these tools are part of the `mp-repo` tool. To install it, follow the [instructions on -the README](https://git.matterlinux.xyz/matter/mp-repo). +## Install requirements +To build and create packages, you should install the `build-essential` package from the +`base` pool. This package will provide you with all the scripts and the tools required. ## Create the package template -Change directory into the `src` folder of the repo you cloned, and use the `mp-temp` -command (a script from `mp-repo`) to create an empty package script: +Change directory into the `src` folder of the pool you cloned, and use the `mp-new` +command (a script from `mp-build`) to create an empty package script: ``` $ cd src -$ mp-temp +$ mp-new _ ``` Package name should be **full lowercase**, it should not contain any non-English characters, and it should not contain underscore (`_`). If there is an underscore in the package name, replace it with `-`. Package name also **should NOT contain a version number**. -For this example we will be packaging the `LXTerminal` from LXDE: +Version name ideally should contain a version number separated with dots. However this may heavily +change based on how the upstream actually versions the software. Version should also be updated +after every modification. If the modification does not change the upstream version, then you should +suffix the version with `m`, for example `1.2m1`, `8.7.3m7` etc. Version name should +also not contain underscores (`_`), you should replace them with `-`. + +For this example we will be packaging the `LXTerminal` (version `0.4.0`) from LXDE: ``` $ cd src -$ mp-temp lxterminal +$ mp-new lxterminal_0.4.0 ``` ## Modify the package script `mp-temp` will create a directory for the new package, which will contain a `pkg.sh` script. This script is called the **package script**. Open this script with the configured editor: ``` +# general info NAME="lxterminal" DESC="" -VERSION="" +VERSION="0.4.0" + +# required files FILES=() HASHES=() + +# install and build depends DEPENDS=() +BUILD=() -build() { - tar xf $NAME-$VERSION.tar.gz - cd $NAME-$VERSION +PACKAGE() { + tar xf "${NAME}-${VERSION}.tar.gz" + cd "${NAME}-${VERSION}" - cd .. && rm -r $NAME-$VERSION + cd .. && rm -r "${NAME}-${VERSION}" } ``` ### Description First of all you should add a description. Description should be short (less than 200 characters), and it should explain details about the package. Ideally, description should not contain words such as "*contains*" -and "*includes*". Here are **bad examples**: +and "*includes*". Here are some **bad examples**: - *LXTerminal package contains a VTE-based terminal emulator* - *LXTerminal includes the LXDE terminal emulator* @@ -86,8 +101,8 @@ And here are some **good examples**: - *LXDE terminal emulator* ### Version -You should package a stable version of the software, do not package the latest git release or -something like that. **Package an official release version.**. For this example we will be +You should package a stable version of the software, do not package the latest git commit or +something like that. **Package an official released version.**. For this example we will be packaging the version `0.4.0`. ### Files @@ -101,37 +116,49 @@ Specify hashes for the files you specified, you can use the `NOHASH` as the hash hash verification for a specific file. ### Depends -Dependencies for the package, **do not** specify the make or the build dependencies that are only used -for compiling the software. These dependencies should be specified in the `repo.sh` script which is located -at the root of the repo. +Run-time dependencies for the package, **do not** specify the make or the build dependencies that are only used +for compiling the software. + +### Build +Build-time dependencies for the package, **dot not** specify any run-time dependencies again if they are also needed +for the build process. Also **do not** specify any package that is part of `build-essential`. With the addition of the details above, here is how our example package script looks like: ``` +# general info NAME="lxterminal" -DESC="VTE-based terminal emulator for LXDE" +DESC="Commands for manipulating POSIX Access Control Lists" VERSION="0.4.0" + +# required files FILES=("https://downloads.sourceforge.net/lxde/lxterminal-$VERSION.tar.xz") HASHES=("7938dbd50e3826c11f4735a742b278d3") + +# install and build depends DEPENDS=("vte") +BUILD=() -build() { - tar xf $NAME-$VERSION.tar.gz - cd $NAME-$VERSION +PACKAGE() { + tar xf "${NAME}-${VERSION}.tar.gz" + cd "${NAME}-${VERSION}" - cd .. && rm -r $NAME-$VERSION + cd .. && rm -r "${NAME}-${VERSION}" } ``` -### Build function -The build function contains the instructions for building the package itself. When building the package, `mp-repo` -will download all the files and remove them after the build, so you do not have to deal with that. +### Package function +The package function contains the instructions for building the package itself. When building the package, `mp-build` +will download all the files and remove them after the build, so you do not have to deal with that. + +To make your life simpler, `mp-new` creates a boilerplate `PACKAGE()` function template. Now will modify it +to make it work our package. First, we should do is to extract the downloaded file, and change directory into the newly extracted folder: ``` -tar xf $NAME-$VERSION.tar.xz -cd $NAME-$VERSION +tar xf "${NAME}-${VERSION}.tar.gz" +cd "${NAME}-${VERSION}" ``` -Then we can change directory into the repo source directory and run the build and install instructions: +Then we can change directory into the pool source directory and run the build and install instructions: ``` ./configure --enable-man --prefix=/usr make && make install @@ -140,35 +167,42 @@ With this configuration, `make install` will try to install the package to the a we want to install the package to the package build directory, so we will use the `DESTDIR` flag for that: ``` ./configure --enable-man --prefix=/usr -make && make DESTDIR="$ROOTDIR" install +make +make DESTDIR="${ROOTDIR}" install ``` `$ROOTDIR` is a custom variable that is set during the package build. During the build it will point to the package build directory. After building and installing the package, we can change directory back up, and remove the extracted folder: ``` -cd .. && rm -r $NAME-$VERSION +cd .. && rm -r "${NAME}-${VERSION}" ``` So the final script should look like this: ``` +# general info NAME="lxterminal" -DESC="VTE-based terminal emulator for LXDE" +DESC="Commands for manipulating POSIX Access Control Lists" VERSION="0.4.0" + +# required files FILES=("https://downloads.sourceforge.net/lxde/lxterminal-$VERSION.tar.xz") HASHES=("7938dbd50e3826c11f4735a742b278d3") -DEPENDS=("vte") -build() { - tar xf $NAME-$VERSION.tar.xz - cd $NAME-$VERSION +# install and build depends +DEPENDS=("vte") +BUILD=() + +PACKAGE() { + tar xf "${NAME}-${VERSION}.tar.xz" + cd "${NAME}-${VERSION}" ./configure --enable-man --prefix=/usr - make && make DESTDIR="$ROOTDIR" install + make + make DESTDIR="${ROOTDIR}" install - cd .. && rm -r $NAME-$VERSION + cd .. && rm -r "${NAME}-${VERSION}" } - ``` ## Formatting the package script @@ -176,32 +210,30 @@ When you are done, format the package script following these rules: - If you have multiple files, put each one of them to a new line, same for the hashes. - If you have more than 5 depends, split them into new lines, grouping 3 depends together. - If you have long commands in the build function, split them to new lines using a backslash (`\`) -- If you have really short and related commands, put them into the same line using `&&` +- Prevent using `&&` unless it's absolutely necessary, it usually ends up messing the error checking - Split unrelated instructions with a newline - ## Building the package To build the newly created package, run the following in the `src` directory: ``` -$ mp-repo --no-sign .. +$ mp-build --no-sign ``` This will install all the required dependencies, and only build the specified package. ## Testing the package -When the package is built, inspect its contents using the `tar tf` command, and extract it to -test it: +When the package is built, check out the newly created root directory inside the package directory +to make sure that everything is placed where it should be: ``` -$ tar tf ../dist/_.mpf -# tar xf ../dist/_.mpf -C / +$ tree /root ``` ## Staging and commit your changes -After making sure that the package is working as intended, go back to the root of the repo source directory, +After making sure that the package is working as intended, go back to the root of the pool source directory, and use `git add .` to stage all the changes. Then use `git commit -m ` to commit them. For the commit message, follow the guidelines on the [contribution page](/wiki/contribute). For our example, the following message should be fine: ``` -$ git commit -m "new: Add lxterminal package" +$ git commit -m "new: add lxterminal package" ``` Then push your changes: ``` diff --git a/wiki/pkg/libmp.json b/wiki/pkg/libmp.json new file mode 100644 index 0000000..8b91d4d --- /dev/null +++ b/wiki/pkg/libmp.json @@ -0,0 +1,6 @@ +{ + "id": "libmp", + "title": "Working with libmp", + "author": "ngn", + "date": "14/08/24" +} diff --git a/wiki/pkg/libmp.md b/wiki/pkg/libmp.md new file mode 100644 index 0000000..5c1562d --- /dev/null +++ b/wiki/pkg/libmp.md @@ -0,0 +1 @@ +This page is currently under development. diff --git a/wiki/pkg/matt.json b/wiki/pkg/matt.json new file mode 100644 index 0000000..e309d1d --- /dev/null +++ b/wiki/pkg/matt.json @@ -0,0 +1,6 @@ +{ + "id": "matt", + "title": "Working with matt", + "author": "ngn", + "date": "14/08/24" +} diff --git a/wiki/pkg/matt.md b/wiki/pkg/matt.md new file mode 100644 index 0000000..ae76418 --- /dev/null +++ b/wiki/pkg/matt.md @@ -0,0 +1,145 @@ +Package management is the process of installing, updating and removing software, tools and libraries. +On GNU/Linux systems, package management is usually done with a package manager. + +# MatterLinux Package Manager +In a Matter system, package management is generally done with the MatterLinux Package Manager (`matt`). +`matt` lets you install, remove, update and search different MatterLinux packages across different pools. + +### Installation +`matt` should be already installed on a MatterLinux system. However if you want to install `matt` from +the source, then you should follow [this section](https://git.matterlinux.xyz/matter/matt#installation) +from the README. + +> **Note** +> +> You should run all the `matt` and related commands as the root user. + +### Configuration +Configuration file for `matt` can be found at `/etc/matt/config.ini`. This file is used specify pools and +configure general settings. + +Here is the configuration shipped with the MatterLinux 24 releases: +``` +# configuration file for matt, please see the wiki page for more information +# https://matterlinux.xyz/wiki/matt + +tmpdir = /var/lib/matt/temp +datadir = /var/lib/matt/data + +[base] +url = mptp://stable.matterlinux.xyz/base +# url = mptp://next.matterlinux.xyz/base + +[desktop] +url = mptp://stable.matterlinux.xyz/desktop +# url = mptp://next.matterlinux.xyz/base + +# [server] +# url = mptp://stable.matterlinux.xyz/server +# url = mptp://next.matterlinux.xyz/server +``` +Lets start by breaking down the first section: + +- `tmpdir (path)`: Temporary storage directory. Used for extracting temporary files and usually +this directory will be cleaned after an operation. + +- `datadir (path)`: Persistent storage directory. Used for storing package databases and pool +information. + +Next sections specify the pools. Section name is the pool name, and it **should match with the +actual pool name**. Pool options are: + +- `url (URL)`: Pool URL, only `mptp` protocol is supported. +- `signing (yes/no)`: Enable/disable PGP signature verification for the pool, by default it's enabled. + +> **Important** +> +> Do NOT disable signature verification if you don't know what are you doing! There is no +> way to confirm the legitimacy of the packages without signature verification! + +### Options +`matt` also has a few other options that you specify as an arguments/parameters. You can list these +options using the `--help` option: +``` +# matt --help +``` +You can also list options for specific commands, for example: +``` +# matt info --help +# matt list --help +``` + +### Syncing pools +To sync remote pool information and package lists, you can use the `sync` command: +``` +# matt sync +``` +After syncing remote pools, `matt` will provide public keys for the available pools. +You can add these keys to your keyring with `gpg`: +``` +# gpg --receive-keys [fingerprint] +``` + +### Installing packages +To install a package, you can use the `install` command: +``` +# matt install which +``` +After running this command `matt` will attempt to find, download and verify the `which` +package, and then install it. + +You can also install packages from local archives: +``` +# matt install which_2.21.mpf +``` +And you can install multiple packages at once: +``` +# matt install grub systemd linux +``` + +### Removing/Uninstalling packages +To remove an installed package, you can use the `remove` command: +``` +# matt remove which +``` +You can also remove multiple packages: +``` +# mp-remove git curl +``` + +### Updating packages +You can update all of the installed packages using the `update` command: +``` +# matt update +``` + +### Querying packages +You can get more information about a package by querying it. For example to get +more information about the `which` package: +``` +# matt info which +``` +This will provide the following information about the package: + +- Name of the package +- Version +- Description +- Size +- Dependencies +- If the package is installed +- If the package is up-to-date + +### List packages +You can list all the installed packages with the `list` command: +``` +# matt list +``` + +# Other package managers +It's a really bad idea to use multiple package managers as they may conflict and may result in an +unstable system. However package managers such as flatpak and snap offer containerized packages, +making them possible to use with other package managers. + +- Flatpak will be added to `desktop` pool in the future. Currently the only way to use flatpak is to +install it from the source. +- Snap will never be added to any official pool, so only way to use it is to install it from the source. diff --git a/wiki/pkg/mc.json b/wiki/pkg/mc.json deleted file mode 100644 index 6f28f17..0000000 --- a/wiki/pkg/mc.json +++ /dev/null @@ -1,6 +0,0 @@ -{ - "id": "mc", - "title": "Working with `mc`", - "author": "ngn", - "date": "05/05/24" -} diff --git a/wiki/pkg/mirrors.json b/wiki/pkg/mirrors.json index 254cecf..a35d421 100644 --- a/wiki/pkg/mirrors.json +++ b/wiki/pkg/mirrors.json @@ -2,5 +2,5 @@ "id": "mirrors", "title": "Mirrors", "author": "ngn", - "date": "19/06/24" + "date": "04/08/24" } diff --git a/wiki/pkg/mirrors.md b/wiki/pkg/mirrors.md index 4a68723..68456ac 100644 --- a/wiki/pkg/mirrors.md +++ b/wiki/pkg/mirrors.md @@ -1,20 +1,19 @@ On this page, you will find information about MatterLinux package mirrors. # Using different mirrors -To use a mirror with `mp`, change the pool URL for the target pool in your `mp` -configuration file (`/etc/mp/cfg`). +To use a mirror with `matt`, change the pool URL for the target pool in your `matt` +configuration file (`/etc/matt/config.ini`). ### Creating a mirror -To create a mirror you can use the `matter-mirror` tool. This tool is a part of the [MatterLinux `tools` -project](https://git.matterlinux.xyz/matter/tools). These tools can be installed with the `tools` -package in the desktop pool. +To create a mirror you can use the [`mirp` tool](https://git.matterlinux.xyz/matter/mirp). +This package can be installed from the `server` pool. -After installing `matter-mirror`, specify an URL and an output directory (files will be downloaded +After installing `mirp`, specify an URL and an output directory (files will be downloaded into the output directory). For example to mirror the `base` pool from `https://pkgs.matterlinux.xyz/base` to `base_mirror` directory: ``` $ mkdir base_mirror -$ matter-mirror -u https://pkgs.matterlinux.xyz/base -o base_mirror +$ mirp mptp://pkgs.matterlinux.xyz/base base_mirror ``` To serve the mirror, you can setup [`pooler`](https://git.matterlinux.xyz/matter/pooler). @@ -24,7 +23,7 @@ You will mostly likely want to update your mirror every once in a while. To do s create a cron job. For example to update your mirror every day at 00:00, you can add the following entry to your `/etc/crontab`: ``` -0 0 * * * matter-mirror -u -o +0 0 * * * mirp ``` # Mirror lists @@ -39,12 +38,14 @@ that you can use. To add your mirror to this list get in contact with the mainta > from the list. ### Official pools -| Pool | MPTP | HTTP(S) | Location | -| ------- | -------------------------------------- | ----------------------------------------- | ----------- | -| base | `mptp://pkgs.matterlinux.xyz/base` | `https://pkgs.matterlinux.xyz/base` | 🇹🇷 Turkey | -| desktop | `mptp://pkgs.matterlinux.xyz/desktop` | `https://pkgs.matterlinux.xyz/desktop` | 🇹🇷 Turkey | -| base | `mptp://next.matterlinux.xyz/base` | `https://next.matterlinux.xyz/base` | 🇹🇷 Turkey | -| desktop | `mptp://next.matterlinux.xyz/desktop` | `https://next.matterlinux.xyz/desktop` | 🇹🇷 Turkey | +| Pool | MPTP | HTTP(S) | Location | +| -------------- | ---------------------------------------- | ----------------------------------------- | ----------- | +| base | `mptp://stable.matterlinux.xyz/base` | `https://stable.matterlinux.xyz/base` | 🇹🇷 Turkey | +| desktop | `mptp://stable.matterlinux.xyz/desktop` | `https://stable.matterlinux.xyz/desktop` | 🇹🇷 Turkey | +| server | `mptp://stable.matterlinux.xyz/server` | `https://stable.matterlinux.xyz/server` | 🇹🇷 Turkey | +| base (next) | `mptp://next.matterlinux.xyz/base` | `https://next.matterlinux.xyz/base` | 🇹🇷 Turkey | +| desktop (next) | `mptp://next.matterlinux.xyz/desktop` | `https://next.matterlinux.xyz/desktop` | 🇹🇷 Turkey | +| server (next) | `mptp://next.matterlinux.xyz/server` | `https://next.matterlinux.xyz/server` | 🇹🇷 Turkey | ### Mirrors Currently there are no available mirrors. diff --git a/wiki/pkg/mptp.json b/wiki/pkg/mptp.json new file mode 100644 index 0000000..c3b9f13 --- /dev/null +++ b/wiki/pkg/mptp.json @@ -0,0 +1,6 @@ +{ + "id": "mptp", + "title": "Working with mptp", + "author": "ngn", + "date": "14/08/24" +} diff --git a/wiki/pkg/mptp.md b/wiki/pkg/mptp.md new file mode 100644 index 0000000..5c1562d --- /dev/null +++ b/wiki/pkg/mptp.md @@ -0,0 +1 @@ +This page is currently under development. diff --git a/wiki/pkg/packages.json b/wiki/pkg/packages.json index 5287072..5baf390 100644 --- a/wiki/pkg/packages.json +++ b/wiki/pkg/packages.json @@ -2,5 +2,5 @@ "id": "packages", "title": "Packages", "author": "ngn", - "date": "19/06/24" + "date": "04/08/24" } diff --git a/wiki/pkg/packages.md b/wiki/pkg/packages.md index 7e9cf5f..578c4e7 100644 --- a/wiki/pkg/packages.md +++ b/wiki/pkg/packages.md @@ -23,8 +23,8 @@ package_1.0.mpf Let's break this down: - `DATA`: An `.ini` formatted file, contains information about package name, version, description, dependencies, files to keep (save) during update/removal and SHA256 hash of the `MPF` file. -- `CHANGES`: A changelog file, each version's changelog is split with 3 newlines (`\n\n\n`) -- `INSTALL`: An optional shell script. If it exists, then should be ran after the installation. This file +- `CHANGES`: A changelog file, containg information about the chnages accross different package versions +- `INSTALL`: An optional shell script. If it's not empty, then should be ran after the installation. This file is also named the "install script". - `HASHES`: Contains MD5 hashes of every file in the `files.tar.gz` archive. Summaries are followed by a white space (` `) and the full file path. These paths do not start with `/`, @@ -51,26 +51,28 @@ usr/bin/which ``` ### Install scripts -Some packages may contain an install script, `INSTALL`, this shell script is ran by the -[MatterLinux Package Manager (`mp`)](/wiki/package_man) using the bash shell, right after the -extraction of `files.tar.gz`. +Some packages may contain an non-empty install script, `INSTALL`, this shell script is ran by the +[MatterLinux Package Manager (`matt`)](/wiki/matt) using the bash shell, right after the +extraction of `files.tar.gz`, in the package installation directory (usually `/`). This script is used to do post-install actions, such as adding users, groups etc. # Working with packages -While installing a package using the [MatterLinux Package Manager (`mp`)](/wiki/package_man), `mp` +While installing a package using the [MatterLinux Package Manager (`matt`)](/wiki/matt), `matt` downloads the target package(s) from the pools, these packages are in the format discussed above. -After downloading and verifying the target package(s), `mp` extracts the packages using `libarchive`. -To learn more about this process see the [page for package management](/wiki/package_man). +After downloading and verifying the target package(s), `matt` extracts the packages using `libarchive`. +To learn more about this process see the [page for package management](/wiki/matt). ### Building packages -Package are built with the `mp-build` tool. In order to build a package, you will need the source -of the package, which can be found in the source tree of the pool which contains the package. +Package are built with the [`mp-build` tool](https://git.matterlinux.xyz/matter/mp-build). In order to +build a package, you will need the source of the package, which can be found in the source tree of the +pool which contains the package. After obtaining the source, package can be simply built by running: ``` $ mp-build ``` +To learn more about building and creating packages, please [see this page](/wiki/create_packages). ### Package scripts Each package source contains a `pkg.sh` shell script. This is the source script that is used to @@ -82,18 +84,20 @@ Let's take a closer look at a `pkg.sh` file: NAME="which" DESC="Shows the full path of (shell) commands" VERSION="2.21" -FILES=("https://ftp.gnu.org/gnu/which/which-$VERSION.tar.gz") + +FILES=("https://ftp.gnu.org/gnu/which/which-${VERSION}.tar.gz") HASHES=("097ff1a324ae02e0a3b0369f07a7544a") + DEPENDS=() -build() { - tar xf $NAME-$VERSION.tar.gz - cd $NAME-$VERSION +PACKAGE() { + tar xf "${NAME}-${VERSION}.tar.gz" + cd "${NAME}-${VERSION}" ./configure --prefix=/usr && make - make DESTDIR=$ROOTDIR install + make DESTDIR="${ROOTDIR}" install - cd .. && rm -rf $NAME-$VERSION + cd .. && rm -r "${NAME}-${VERSION}" } ``` This `pkg.sh` file is for the `which` package (version `2.21`). Let's start by breaking down the @@ -118,8 +122,8 @@ There are also other options that are not present in this example: - `KEEP`: A list of files to keep during the update/removal of the package - `BUILD`: A list packages required for building this package -Now let's take a look at the `build` function. Each package needs a `build` function, this -function is called by `mp-build` after downloading and verifying all the packages. It will be +Now let's take a look at the `PACKAGE` function. Each package needs a `PACKAGE` function, this +function is called by `mp-build` after downloading and verifying all the files. It will be called in the `$ROOTDIR`. This directory will contain all the downloaded files, any files in this directory will be included into the build, so don't forget to cleanup. @@ -128,14 +132,14 @@ in this directory will be included into the build, so don't forget to cleanup. > You don't need to cleanup the downloaded files in the package script, > they will be cleaned by the `mp-build`. -- `tar xf $NAME-$VERSION.tar.gz`: Extract the downloaded archive file. -- `cd $NAME-$VERSION`: Change directory into the extracted directory. +- `tar xf "${NAME}-${VERSION}.tar.gz"`: Extract the downloaded archive file. +- `cd "${NAME}-${VERSION}"`: Change directory into the extracted directory. - `./configure --prefix=/usr && make`: Builds the `which` tool, different packages may have different build instructions. These instruction are usually provided by the upstream. You can also check out [LFS](https://www.linuxfromscratch.org/lfs/view/12.0-systemd/) and [BLFS](https://www.linuxfromscratch.org/blfs/view/12.0-systemd/) for instructions. -- `make DESTDIR=$ROOTDIR install`: Install the package. Make sure that you are installing the package +- `make DESTDIR="${ROOTDIR}" install`: Install the package. Make sure that you are installing the package into the `$ROOTDIR` and not the root file system. -- `cd .. && rm -rf $NAME-$VERSION`: Change directory back into `$ROOTDIR` and clean the extracted archive. +- `cd .. && rm -r "${NAME}-${VERSION}"`: Change directory back into `$ROOTDIR` and clean the extracted archive. Check out [`base`](https://git.matterlinux.xyz/Matter/base) and [`desktop`](https://git.matterlinux.xyz/Matter/desktop) -pool sources for more example `pkg.sh` scripts. +pool sources for more example package scripts. diff --git a/wiki/pkg/pkg_man.json b/wiki/pkg/pkg_man.json deleted file mode 100644 index 6a0221e..0000000 --- a/wiki/pkg/pkg_man.json +++ /dev/null @@ -1,6 +0,0 @@ -{ - "id": "package_man", - "title": "Package Management", - "author": "ngn", - "date": "21/02/24" -} diff --git a/wiki/pkg/pkg_man.md b/wiki/pkg/pkg_man.md deleted file mode 100644 index d3c1835..0000000 --- a/wiki/pkg/pkg_man.md +++ /dev/null @@ -1,209 +0,0 @@ -Package management is the process of installing, updating and removing software, tools and libraries. -On GNU/Linux systems, package management is usually done with a package manager. - -# MatterLinux Package Manager -In a Matter system, package management is generally done with the MatterLinux Package Manager (`mp`). -`mp` lets you install, remove, update and search different MatterLinux packages across different repos. - -### Installation -`mp` should be already installed on a MatterLinux system. However if you want to install `mp` from -the source, then you should follow [this section](https://git.matterlinux.xyz/matter/mp#installation) -from the README. - -> **Note** -> -> You should run all the `mp` and related commands as the root user. - -### Configuration -Configuration file for `mp` can be found at `/etc/mp/cfg`. This file is used specify repos and -configure general settings. - -Here is the configuration shipped with the MatterLinux 24 releases: -``` -[general] -tmpdir = /tmp/mp - -[base] -uri = https://pkgs.matterlinux.xyz/base - -[desktop] -uri = https://pkgs.matterlinux.xyz/desktop -``` -Lets start by breaking down the `[general]` section: - -- `tmpdir (path)`: Specifies the temporary directory used by the `mp`. This directory will be used to save -packages and signatures, most of the files in this directory will be removed after operation. - -Other general configuration options are: - -- `debug (yes/no)`: This setting is used to enable the debug logging, however `mp` don't have any -debug logging (it will be implemented in the future) so there is no point at enabling it. - -Next sections specify the repos. Section name is the repo name, and it **should match with the -actual repo name**. Repo options are: - -- `uri (URI)`: Repo URI, you can use `http`, `https` or `ftp` protocols. -- `signing (yes/no)`: Enable/disable PGP signature verification for the repo, by default it's enabled. - -> **Important** -> -> Do NOT disable signature verification if you don't know what are you doing! There is no -> way to confirm the legitimacy of the packages without signature verification! - -### Environment options -`mp` also has a few other options that you specify as an environment variable. - -One of these is the `MP_ROOT` option. This option lets you change the target root directory. -By default `mp` install all the packages to `/`, stores all the information under the `/` etc. -By changing the target root directory you can change this. - -For example lets say you want to install all the packages to `/tmp/testroot`. To do so, -first you will need to setup a root file system structure inside `/tmp/testroot`: -``` -# mkdir /tmp/testroot -# pushd /tmp/testroot -# mkdir -p etc boot dev home proc root run srv sys tmp usr/lib usr/bin usr/sbin var/lib -# ln -sf bin usr/bin -# ln -sf sbin usr/sbin -# ln -sf lib usr/lib -# ln -sf lib64 usr/lib -# popd -``` -Now you can use `mp` with the `MP_ROOT` variable, for example: -``` -# MP_ROOT=/tmp/testroot mp-sync -# MP_ROOT=/tmp/testroot mp-install which -``` - -Another one of these environment options is the `MP_YES` option. By default operations such as -installing packages, updating packages and removing packages will ask you to confirm the operation -using an interactive yes/no prompt. To disable this prompt and confirm all operations by default, -you can set the `MP_YES` environment variable. You can set it to whatever you want, just make sure -it's set. - -`MP_NORET` is another option that you can set. By default `mp` will print the return status for each -command, you can disable this by setting `MP_NORET` to whatever you want. - -The last one of these options is the `MP_GREP` option. This option is only useful if you are using -the query or the list command. If set, `mp` will print grep-able output for these commands. Again, you can set it -to whatever you want. - -### Syncing Repos -To sync remote repo information and package lists, you can use the `mp-sync` command. After running it -`mp` will: - -1. Download the repo metadata for all the repos. Repo metadata is located at `REPO_URI/repo` -2. Parse the repo metadata and download all the repo package lists. These lists are located at -`REPO_URI/REPO_NAME.tar.gz` -3. Move and extract the repo lists to `$MP_ROOT/var/lib/mp/repos/REPO_NAME` and parse the plain package lists (`pkgs`) -4. Tell you the repo signatures - -You can add these signatures to your public key ring with `gpg`: -``` -# gpg --receive-keys [fingerprint] -``` - -### Installing Packages -To install a package, you can use the `mp-install` command: -``` -# mp-install which -``` -After running this command, `mp` will: - -1. Check all the synced repos for the `which` package -2. Check if the package is already installed -3. Resolve all the dependencies for the `which` package -4. Ask you if you want to continue with the installation -5. Download the `which` package and it's signature from the repo to the `tmpdir` -6. Verify the package using the signature -7. Extract the package to `$MP_ROOT` -8. Run the package install script if it contains one -9. Add an entry for the `which` package to `$MP_ROOT/var/lib/mp/db` and `$MP_ROOT/var/lib/mp/files` - -You can also install multiple packages using the `mp-install` command: -``` -# mp-install grub systemd linux -``` - -### Removing/Uninstalling Packages -To remove an installed package, you can use the `mp-remove` command: -``` -# mp-remove which -``` - -After running this command, `mp` will: - -1. Check all the installed packages to see if `which` is installed or not -2. Find all the files that `which` package extracted during the installation -3. Ask you if you want to continue with the removal -4. Delete all the files found in the step 2 -5. Remove the package entry for the `which` package from `$MP_ROOT/var/lib/mp/db` and `$MP_ROOT/var/lib/mp/files` - -You can also remove multiple packages: -``` -# mp-remove git curl -``` - -### Updating Packages -You can update all of the installed packages using the `mp-update` command. -After running this command, `mp` will: - -1. Check if all the installed packages are up-to-date -2. Find all the packages that are not up-to-date -3. Ask if you want to continue with the update -4. Act like the `mp-install` command, it will download, verify and install all -the packages, overwriting old files -5. Save new up-to-date package entries to the `$MP_ROOT/var/lib/mp/db` and `$MP_ROOT/var/lib/mp/files`, -overwriting the old entries. - -### Searching Packages -You can search for packages in the repos using the `mp-search` command. -For example, this command will return a list of all the packages that contain `lib` -in their name: -``` -# mp-search lib -``` -After running this command, `mp` will: - -1. Check all the repos for any packages that contain `lib` in the name -2. Print the package -3. Print the count of the results - -### Querying packages -You can get more information about a package by querying it. For example to get -more information about the `which` package: -``` -# mp-query which -``` -This will provide the following information about the package: - -- Name of the package -- Repo that the package is in -- Description -- Hash (sum) -- Version -- If the package is installed -- If the package is up-to-date - -After running this command, `mp` will: - -1. Check all the repos for the `which` package -2. Fetch information about the package from the `$MP_ROOT/var/lib/mp/repos/REPO_NAME` and `$MP_ROOT/var/lib/mp/db`. - -### List packages -You can list all the installed packages with the `mp-list` command. - -After running this command, `mp` will: - -1. Load the database (`$MP_ROOT/var/lib/mp/db`) -2. Loop over all the entries and print the package names and the versions - - -# Other Package Managers -It's a really bad idea to use multiple package managers as they may conflict and may result in an -unstable system. However package managers such as flatpak and snap offer containerized packages, -making them possible to use with other package managers. - -- Flatpak will be added to `desktop` repo in the future. Currently the only way to use flatpak is to -install it from the source. -- Snap will never be added to any official repo, so only way to use it is to install it from the source. diff --git a/wiki/pkg/pools.json b/wiki/pkg/pools.json index f2bb008..a97e521 100644 --- a/wiki/pkg/pools.json +++ b/wiki/pkg/pools.json @@ -2,5 +2,5 @@ "id": "pools", "title": "Pools", "author": "ngn", - "date": "19/06/24" + "date": "04/08/24" } diff --git a/wiki/pkg/pools.md b/wiki/pkg/pools.md index fde5b42..12786fe 100644 --- a/wiki/pkg/pools.md +++ b/wiki/pkg/pools.md @@ -9,9 +9,9 @@ A pool contains: - Package list file (`LIST`) ### Package files and signatures -MatterLinux pools contain at least two package. Each package also has a PGP signature +MatterLinux pools contain at least two packages. Each package also has a PGP signature that can be used to verify the package. These signatures belong to public key fingerprint -specified in the pool metadata. +specified in the pool info file. ### Pool info file Pool info file contains information about the pool itself. This information includes @@ -22,7 +22,7 @@ Pool info file contains information about the pool itself. This information incl This info file can be found in the at the root of the pool. For example the pool info file for `https://pkgs.matterlinux.xyz/base` is located at `https://pkgs.matterlinux.xyz/base/INFO`. -`INI` format is used for this file. +`.ini` format is used for this file. ### Package list file A list containing all the package information can be found in an archive file named `LIST`. @@ -93,7 +93,7 @@ MatterLinux releases offer 2 different pools, `base` and `desktop`. There are al `base` and `desktop` pools (named "next"). URLs for these pools and the mirrors can be found at the [mirrors page](/wiki/mirrors). These pools can provide packages for users, these packages can be managed using the -[MatterLinux package manager (`mp`)](/wiki/mp). +[MatterLinux package manager (`matt`)](/wiki/matt). ### Building an already existing pool To build an already existing pool, such as the `base` or the `desktop` pool, first you will need to @@ -117,26 +117,31 @@ $ mp-pool --no-sign Unlike most of the package managers, `mp` does not use HTTP, FTP or RSYNC to communicate with the pools. It has it's own custom protocol named [MPTP](/wiki/mptp). Pools are hosted over this protocol using [pooler](https://git.matterlinux.xyz/matter/pooler). -To get host a pool using `pooler`, download and install the `pooler` by following the instructions on the [README](https://git.matterlinux.xyz/matter/pooler). It's also in the official pools so you can install it with `mp` as well. -After installing it, you can simply host the pool by running: -``` -$ pooler -``` -This will start the MPTP server on port 0.0.0.0:5858. You can also host multiple pools by specifying them: -``` -$ pooler -``` -Note that each pool needs to have a unique name for in order to host multiple pools using the same -pooler instance. +To get host a pool using `pooler`, download and install the `pooler` by following the instructions on the [README](https://git.matterlinux.xyz/matter/pooler). It's also in the official pools so you can install it with `matt` as well. -If you want to change the service address, you can do so by using the `-h` and the `-p` flag: +After installing it, you'll need to create/edit the configuration file (`/etc/pooler/config.ini`). +Here is an example configuration file: ``` -$ pooler -h 0.0.0.0 -p 5566 +# location for the pooler log file +log = /var/log/pooler.log + +# address to host the MPTP server on +addr = 0.0.0.0:5858 + +# pool name +[my-pool] +# pool directory +dir = /srv/my-pool + +# pool hostname +host = my-pool.example.com ``` -You can also specify host names for your pools: +After editing this configuration for your needs, you can run pooler with the configuration file: ``` -$ pooler ::p1.example.com ::p2.example.com +$ pooler /etc/pooler/config.ini ``` +After running this command, pooler should start hosting your pool over MPTP on port 5858. If you experience +any issues, you should check the log file for any errors. ### Creating a new pool If you want to create an unofficial pool, setup the directory structure for the source pool as @@ -145,6 +150,5 @@ edit it. After adding your packages and building them with `mp-pool`, you can host it using pooler. I suggest you also host them on a HTTP or a FTP server just to make them easier to access. -Doing so will also allow other people to use [`matter-mirror` tool to mirror your pool](/wiki/mirrors). Also you can add your unofficial pool to this wiki page [by creating an issue](https://git.matterlinux.xyz/Matter/content/issues).