Developer Guide
###############

Do not hang around in Pythran code base without your developer guide! It is
the compass that will guide you in the code jungle!

Disclaimer
----------

This document is a never ending work-in-progress draft. Please contribute!

Configuration
-------------

Pythran can be configured with a rc file. An example is found in `pythran/pythran.cfg`.
Look at it! To customize it::

    $> cp pythran/pythran.cfg ~/.pythranrc

In particular, you may want to add ``-g -O0`` to the ``cxxflags``.

Coding Style
------------

All Python code must be conform to the PEP 8, and the ``flake8`` command must not
yield any message when run on our database. Additionally, avoid backslashes,
and try to make your code as concise as possible.

    $> flake8 pythran/*.py pythran/*/*.py --exclude="pythran/tests/test*.py,__init__.py"

C++ code use spaces (no tabs) and a tab width of 4.

File Hierarchy
--------------

Listing the top level directory yields the following entries:

setup.py
    The files that describes what gets installed, that holds ``PyPI`` entries
    and such.

docs/
    If you're reading this document, you know what it's all about!  ``MANUAL``
    is the user documentation and ``DEVGUIDE`` is the developer documentation.

    Use ``make`` from this directory to produce the static website.

LICENSE
    Boring but important stuff.

MANIFEST.in
    Describe additional stuff to package there.

README.rst
    Quick introduction and description of _pythran_.

pythran/
    The source of all things.

pythran/tests/
    The source of all issues.

pythran/pythonic/
    Where C++ back-end lies.


Validation
----------

``pythran`` uses the ``unittest`` module and the `pytest
<http://pytest.org/latest/>`_ package to manage test cases. The whole
validation suite is run through the command::

    $> python -m pytest pythran/tests

To run it faster we use the ``pytest`` extension `xdist
<https://pypi.org/project/pytest-xdist/>`_, the test suite will run using all
available cores. Otherwise it might run **very** slowly, something like four
hours on a decent laptop :'(.

Note that it is possible to use the ``pytest`` module to pass a subset of the
test suite::

    $> pytest -n 8 pythran/tests/test_list.py

runs all the tests found in ``pythran/tests/test_list.py``.

Only compiler tests can be check using test filtering::

   $> pytest -n 8 pythran/tests -m "not module"

There are two kinds of tests in ``pythran``:

1. unit tests that test a specific feature of the implementation. Such tests
   are listed as method of a class deriving from ``test_env.TestEnv`` and must
   call the ``run_test(function_to_translate, *effective_parameters,
   **name_to_signature)`` method [1]_.  It translates ``function_to_translate``
   into a native function using the type annotations given in the
   ``name_to_signature`` dictionary, runs both the python and the native
   version with ``effective_parameters`` as arguments and asserts the results
   are the same.

   .. [1] See examples in ``pythran/tests/test_base.py`` for more details.

2. test cases that are just plain python modules to be converted in native
   module by ``pythran``. It is used to test complex situations, codes or
   benchmarks found on the web etc. They are just translated, not run. These
   test cases lie in ``pythran/tests/cases/`` and are listed in
   ``pythran/tests/test_cases.py``.


C++ runtime
-----------

The C++ code generated by ``pythran`` relies on a specific back-end,
``pythonic``. It is a set of headers that mimics Python's intrinsics and
collections behavior in C++. It lies in ``pythran/pythonic/``. There is one
directory per module, e.g. ``pythran/pythonic/numpy`` for the ``numpy`` module,
and one file per function, e.g. ``pythran/pythonic/numpy/ones.hpp`` for the
``numpy.ones`` function. Type definitions are stored in the seperate
``pythran/pythonic/types`` directory, one header per type. Each function header
must be ``#includ``-able independently, i.e. it itself includes all the type
and function definition it needs. This helps keeping compilation time low.

All Pythran functions and types live in the ``pythonic`` namespace. Each extra
module defines a new namespace, like ``pythonic::math`` or
``pythonic::random``, and each type is defined in the ``pythonic::types``
namespace. The ``DECLARE_FUNCTOR`` and ``DEFINE_FUNCTOR`` macros from
``pythonic/utils/functor.hpp`` is commonly used to convert functions into
functors and put them into the mandatory ``functor`` namespace.

The pythonic runtime can be used without Python support, so it is important to
protect all Python-specific stuff inside ``ENABLE_PYTHON_MODULE`` guard.

All methods are represented by functions in Pythran. The associated
pseudo-modules are prefixed and suffixed by a double underscore ``__``, as in
``pythran/pythonic/__list__``.


Benchmarking and Testing
------------------------

Stand-alone algorithms are put into ``pythran/tests/cases``. They must be valid
Pythran input (including spec annotations). To be taken into account by the
validation suite, they must be listed in ``pythran/tests/test_cases.py``. To be
taken into account by the benchmarking suite, they must have a line starting
with the ``#runas`` directive. Check ``pythran/tests/matmul.py`` for a complete
example.

To run the benchmark suite, one can rely on::

    $> python setup.py bench --mode=<mode>

where *<mode>* is one among:

python
    Uses the interpreter used to run ``setup.py``.

pythran
    Uses the Pythran compiler.

pythran+omp
    Uses the Pythran compiler in OpenMP mode.

All measurements are made using the ``timeit`` module. The number of iterations
is customizable through the ``--nb-iter`` switch.

How to
------

:Add support for a new module:
    1. Provide its C++ implementation in ``pythran/pythonic++/<mymodule>``.
           ``pythran/pythonic++/math/*.hpp`` and
           ``pythran/pythonic++/__list__/*.hpp`` are good example to referer to.
    2. Provide its description in ``pythran/tables.py``. Each function, method
           or variable must be listed there with the appropriate description.
    3. Provide its test suite in ``pythran/tests/`` under the name
           ``test_my_module.py``. One test case per function, method or variable
           is great.

:Add a new analysis:
    1. Subclass one of ``ModuleAnalysis``, ``FunctionAnalysis`` or ``NodeAnalysis``.
    2. List analysis required by yours in the parent constructor, they will be built automatically and stored in the attribute with the corresponding uncameled name.
    3. Write your analysis as a regular ``ast.NodeVisitor``. The analysis result must be stored in ``self.result``.
    4. Use it either from another pass's constructor, or through the ``passmanager.gather`` function.


:Push changes into the holy trunk:
    1. Use the ``github`` interface and the pull/push requests features
    2. Make your dev available on the web and asks for a merge on the IRC
           channel ``#pythran`` on ``irc.oftc.net``
           (via your browser: https://webchat.oftc.net)
