PyRPM
=====

// This document contains asciidoc (http://www.methods.co.nz/asciidoc/) data
// and you should probably read the corresponding .html page.

The current location of the PyRPM project is at
link:http://people.redhat.com/laroche/pyrpm/[],
link:http://sf.net/projects/pyrpm/[] and
link:http://freshmeat.net/projects/pyrpm/[].

The newest version can be downloaded from
link:http://people.redhat.com/laroche/pyrpm/download/[].

Please send patches to link:mailto:laroche@redhat.com[Florian La Roche],
link:mailto:pknirsch@redhat.com[Phil Knirsch],
link:mailto:twoerner@redhat.com[Thomas Wörner] and
link:mailto:ffesti@redhat.com[Florian Festi].

The cvs source code repository is automatically converted to a
link:http://git.or.cz/[git] repository at
link:http://www.jur-linux.org/git/?p=cvs-pyrpm.git;a=summary[].
(This is experimental only, automatically converted from a
cvs repository.)


Overview
--------

PyRPM is an experimental project to look at rpm package management. It is
a python module and a collection of scripts that provide similar functionality
as rpm, yum and co. It is written from ground up and is implemented only in
python. It mainly deals with rpm packages, the rpm database `rpmdb` in
`/var/lib/rpm`, comps.xml files and yum repositories (called repo data).

The reason for this project was to have a codebase with which we can test and
verify rpm, rpmdb and yum itself and experiment with new features and
implementations very quickly. Python is Red Hat's in-house defacto standard,
so using python to do rapid prototyping was only logical.

The code itself is still very small and modular. We're constantly looking
to improve the code, make it more production ready and remove duplicate
functionality. Results from this project back to rpm and co are very important
for us to improve the current available Linux tools and not rush with only
adding another project for this. Next to working on good algorithms and sane
implementations, most code is backed up with verify and testing scripts.


Warning
-------

The PyRPM project is by no means meant to be a full replacement of rpm or yum.
In its current state we feel confident that in general it won't break
anything, but the code isn't production quality yet.

The check scripts can be safely used as they won't modify anything on your
system, so feel free to use them at your convenience.

We personally have been using the yum replacement very extensively and even
used it to update our own systems, but that's not even remotely a guarantee
that it will work on your machine. It has only been run by a few people until
now.

Using it to set up buildroots on the other hand should be very safe as this
all happens in a very contained environment (sandbox principle).

We do have run lots of installation tests into buildroots and then run update
tests. We have also updated between releases like going from Fedora Core 1
over to release 2, 3, 4, ... Same with enterprise releases. At least 32bit
x86 and 64bit x86_64 should be extensively tested, most other archs should
also be ok.

Please let us know of any problems you encounter, if possible with some
reproducible test cases so we can figure out what might have happened.


Warning about Locking
---------------------

An additional warning about locks when using the pyrpm tools: When using
pyrpmyum you will get the yum lock, but unfortunately due to the nature of
how rpm locks the rpmdb we can't do those locks from within python. So while
using yum and pyrpmyum concurrently works nicely, don't ever use rpm while
running pyrpmyum, it's basically guaranteed to break your rpmdb and/or your
system.


Features/Highlights
-------------------

Over time quite a few nice things have come out of the project:

- Stable and reliable dependency resolver with efficient algorithms: +
  Due to the easy nature of python being a rapid prototyping language and the
  modular design of pyrpm we were able to test and write several generations
  of dependency resolvers and over time optimize and improve the reliability
  of it to a point where we are very confident that the results are what a
  user would expect.
- Handling of multilib systems: +
  This is connected to the previous feature. Multilib systems are generally
  a lot trickier to handle than single arch systems. The algorithms used in
  pyrpm try to make intelligent choices as to which packages should be
  installed resp. how packages should be updated.
- Ordering of all packages in a transaction: +
  In order for a whole transaction to work reliable it is not enough to just
  order the installs and updates but also the erase operations properly. In
  pyrpm we are handling all operations and order them in a way that
  honors all requirements correctly, even for erases.
- Smart like package selection: +
  In pyrpm we are able to make package selections similar to the SMART
  package manager by trying to downgrade packages or even remove them if
  dependencies can't be resolved at all. The semi automatic removal is of
  course optional and not enabled by default.
- Very easy to use scripts to setup changeroots: +
  Using pyrpmkickstart you can easily use a typical anaconda kickstart file
  to do an automated install either on a disk, partition, diskimage or even
  simple changeroots.
- SELinux support is only working with newer versions of the python bindings
  (v 2.0.13+). If the libselinux version is older, then SELinux support is 
  disabled, because there are missing functions and a bug in rpm_execcon.

Requirements for PyRPM
----------------------

This is required for PyRPM to work:

- link:http://python.org/[python] 2.3 or newer. PyRPM is also using the
  gzip, zlib, bz2, bsddb python modules.
- `python-elementtree` for a fast xml parser (This used to be an extra
  python module, more recent versions of python have this as standard module.)
- link:http://linux.duke.edu/projects/urlgrabber/[python-urlgrabber] to read
  data via http and ftp.

There are also optional dependencies like:

- pyparted if you want to use pyrpmkickstart with the `--no-stage2` option
  where the apps from your machine are used instead of the
  `Fedora/base/*.img` files.
- link:http://www.methods.co.nz/asciidoc/[asciidoc] if you want to change the
  documentation and need to re-generate the html files. The releases already
  contain pre-generated html files.
- link:http://pysqlite.org/[python-sqlite] if you want to experiment with the
  optional sqlite database backend.
- `libselinux-python` for proper selinux support
- link:http://xmlsoft.org/[libxml2] with the python bindings in
  `libxml2-python`

PyRPM is currently spending most of its CPU and memory on the list of files
that are stored within rpm packages. Those are needed for dependency
resolution. Re-writing all in C could improve this again and also allow
production usage of it. Another possibility would be to change the repodata
to allow for reduced data sets and better algorithms. Currently we move
most data lookups to sqlite and try to remove unnecessary data lookups.


Other PyRPM Project Pages
-------------------------

- The available tools are described in link:pyrpm-tools.html[].
- A collection of related projects are listed in link:pyrpm-links.html[].
- Background information and more development related docu is
  available in link:pyrpm-devel.html[].

