About
================

PackageMaker is a tool to automate process to package files, dirs and symlinks.

It helps building packages of existing files on your system by automating
almost all of the steps needed for packaing: arrange source tree, create
makefiles and rpm specs or debina packaging files, etc.


Basic Usage
======================

see the output of `pmaker --help` or pmaker(8) or examples/*.log.



How it works
=======================

When packaging files, dirs and symlinks which user gave in the path list,
PackageMaker will try gathering the information of these targets  and then:

1. arrange src tree contains these files, dirs and symlinks with these
   relative path kept, and build files (Makefile.am, configure.ac, etc.)
   to install these.

2. generate packaging metadata like RPM SPEC, debian/rules, etc.

3. build package such as rpm, src.rpm, deb, etc.


NOTE: The permissions of the files might be lost during packaging process. If
you want to ensure these are saved or force set permissions as you wanted,
specify these explicitly in Makefile.am or rpm spec, etc.



"Package-based system construction" - the concept behind packagemaker
==============================================================================

Usually, systems are constructed through the following steps:

1. kitting: setup hardware
2. install os
3. configure os
4. install middleware and apps
5. configure middleware and apps


It is possible to automate some of steps 2..4, above but it takes much time and
hard sometime because most steps consist of procedual steps with side effects.

However, if those procedual steps with side effects are wrapped and "lift"-ed
to a kind of "monad", it should be easy to manage and helps making these
processes less procedual and more functional a lot. For example, if all of the
changes (file/dir/symlink additions and modifications) after the step 2 could
be packaged into some RPMs, we can re-play these steps with a kickstart file
with basic os installatio and package list which these additional RPMs are
added.

Along with that, most important point is that all we want are just *what*
files, dirs and symlinks makes that system and we don't care *how* these were
made actaully in most cases.

PackageMaker is one of the important pieces to actualize and establish "package
based system construction" methodology because it should enable reducing the
cost of packaging files, dirs and sysmlinks (captureing what objects making
that system), and may enables re-playing the system construction process.


Comparison with configuration management system
-------------------------------------------------------

Some configuration management systems such like puppet, chef also can
accomplish the same goal packagemaker focusing but:

* These add some more extra software layers to os-native software management
  systems. Puppet conflicts with and overrides package management system for
  example.

* Most of all are based on server-client architecture and client and central
  management systems are tightly-coupled as usual.

* Custom scripts are required: kickstart does not support puppet natively and
  some kinds of post scripts are needed to make use of puppet, for example.


Packages built with packagemaker do not need any runtime system or libraries
and should be able to work well with os-native package management systems in
standalone mode.


Architecture
===========================

Simply put, PackageMaker can be divided into four components:

a. Models
b. Collectors
c. Backends
d. Utility modules


a. Models
---------------------------

Classes in pmaker.models.FileObjects implement the basic model of target
objects: files, dirs and symlinks.  Because many instance of these objects
should be created as user requested, operations (copy, move, etc.) for these
models are implemented in in functions in another module,
pmaker.models.FileObjectOperations.

Model classes are instantiated from dedicated factory functions in
pmaker.models.FileObjectFactory and never instantiated directly.

The code of models are placed in pmaker/models directory.


b. Collectors
---------------------------

This component process user input (files list) and to collect FIleObject
instances to package.

The code of collectors are placed in pmaker/collectors directory.


c. Backends (Drivers)
---------------------------

The modules underr pmaker/backend is the core componenet to manage and drive
packaging process.

All backend classes are children of pmaker.backend.base.Base class and may
override methods {setup, preconfigure, configure, sbuild, build} represents
each build steps.


d. Utility modules
---------------------------

Most of python code in module's top directory (pmaker/\*.py) are utility modules.


How to build
================


Build w/ mock
------------------------

It takes some time to make a rpm but should be better, I think.

1. python setup.py srpm
2. mock -r <target_build_dist> dist/SRPMS/packagemaker-*.src.rpm


Build w/o mock
------------------------

It's easier than the above but only possible to make a rpm for build host.

1. python setup.py rpm


How to test
================

* Unit tests: `python setup.py test`
* Unit tests + System tests: `python setup.py test --full`

If you want to test specific python code:

a. source code: ./runtest.sh <path_to_python_source>
b. a class in source code: ./runtest.sh <path_to_python_source>:<class_name>
c. a method of a class in source code:./runtest.sh <path_to_python_source>:<class_name>.<method_name>


SEE ALSO: nosetests(1)


Here are some examples:

::

  $ ./runtest.sh pmaker/tests/rpmutils.py
  FIXME: Implement tests for this function ... ok
  test_info_by_path (pmaker.tests.rpmutils.TestFunctions) ... ok
  test_rpm_attr (pmaker.tests.rpmutils.TestFunctions) ... ok
  test_rpm_search_provides_by_path (pmaker.tests.rpmutils.TestFunctions) ... ok
  test_rpmh2nvrae (pmaker.tests.rpmutils.TestFunctions) ... ok
  test_rpmh2nvrae__no_rpmdb (pmaker.tests.rpmutils.TestFunctions) ... ok
  FIXME: Implement tests for this function ... ok
  FIXME: Implement tests for this function ... ok

  ----------------------------------------------------------------------
  Ran 8 tests in 0.517s

  OK
  $ ./runtest.sh pmaker/models/tests/FileInfo.py:TestFileInfo
  test__init__ (pmaker.models.tests.FileInfo.TestFileInfo) ... ok

  ----------------------------------------------------------------------
  Ran 1 test in 0.019s

  OK
  $ ./runtest.sh tests/07_multi_files_filelist_json.py:Test_00_multi_files_filelist_json.test_01_system_files__tgz
  test_01_system_files__tgz (tests.07_multi_files_filelist_json.Test_00_multi_files_filelist_json) ... configure.ac:2: installing `./install-sh'
  configure.ac:2: installing `./missing'
  ok

  ----------------------------------------------------------------------
  Ran 1 test in 21.479s

  OK
  $


HACKING
================

This is my usual way for enhancements:

1. create a branch: git branch foo
2. modify or add code to archive objective enhancements in that branch: git checkout foo; vim ...
3. add (unit) tests for enhancements to verify the correctness of changes
4. commit and run full test (unit + system tests)

if all looks ok, merge the branch to main.


And here is my usual way for bug fixes:

1. Write tests for the bug
2. Modify / add code for the fix
3. Run the tests and confirm if the fix was right


TODO
================

* resolve package name collisions due to overriding packages; there is
  'man-pages-overrides' package exist. How about using '-overlay' suffix
  instead of '-overrides' ?
* correct wrong English expressions
* define schema for input (JSON, YAML?, XML?, ...)

  * perhaps, the contents of files will be gotten from external site pointed by
    URL reference in JSON data

* more complete tests
* eliminate the strong dependency to rpm and make it runnable on debian based
  systems (w/o rpm-python)
* find causes of warnings during deb build and fix them all
* plugin system: posponed
* keep permissions of targets in tar archives


Finished TODO items
---------------------------

* refactor its architecture: Done

  * make collector (generator) and packagemaker classes loosely-coupled: Done
  * separate packaging strategy (PackageMaker._scheme) and packaging format
    (PackageMaker._format): Done

* sort out command line options: Done
* Run w/o python-cheetah: Done (now it uses pytenjin instead)


References
================

In random order:

* http://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM_Guide/ch-creating-rpms.html
* http://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM_Guide/ch-rpm-programming-python.html
* http://cdbs-doc.duckcorp.org
* https://wiki.duckcorp.org/DebianPackagingTutorial/CDBS
* http://kitenet.net/~joey/talks/debhelper/debhelper-slides.pdf
* http://wiki.debian.org/IntroDebianPackaging
* http://www.debian.org/doc/maint-guide/ch-dother.ja.html


Alternatives
================

Basic idea and implementation design of PackageMaker was arised from offhand
talk with my very talented co-worker, Masatake Yamato (yamato at redhat.com).

Around the same time I started working on PackageMaker, Magnus-san developed
buildrpm and I was very impressed with it. Implementation was completely
different but PackageMaker and buildrpm do the same thing basically.

I believe PackageMaker is useful and helps you but if you want features
PackageMaker lacks or will not have or you don't like it, take a look at
buildrpm:

* buildrpm: http://magnusg.fedorapeople.org/buildrpm/


License
================

Copyright (C) 2011 Satoru SATOH <satoru.satoh @ gmail.com>
Copyright (C) 2011 Satoru SATOH <ssato @ redhat.com>

This program is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation, either version 3 of the License, or (at your option) any later
version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.  See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with
this program.  If not, see <http://www.gnu.org/licenses/>.


Exceptions
-----------------------

Files under pmaker/imported/ were imported from external projects and the above
license is not applied. 


Author
================

Satoru SATOH <ssato at redhat.com>


.. vim:sw=2 ts=2 et: