nohrio
======

nohrio is a project designed to facilitate OHRRPGCE data manipulation utilities and
human inspection of data, by providing:

a) numpy.memmap based interface for directly manipulating OHRRPGCE fixed-length-record based data files.

b) a human readable and editable YAML serialization of all records, upon which standard tools like 'grep' and 'diff' are applicable

c) sane, readily updateable binary serialization of the insane-in-spots OHRRPGCE formats

It is based upon the dtype system of NumPy, so it's strongly recommended to have a good understanding
of NumPy to get the most out of nohrio. In the future less knowledge will be required -- you will
still need such an understanding to get best performance.

OHRRPGCE: https://rpg.hamsterrepublic.com/ohrrpgce

Requires NumPy; requires PyYAML if you want to serialize to/ read from YAML.

Setuptools or Distribute are required; if you don't have one of them, setuptools will automatically
be installed when you run ``python setup.py build``

For building docs, requires Sphinx, SCons, and PyLit
(http://pylit.berlios.de/)

Completeness:
 a) is mostly complete.
 b) is mostly complete.
 c) is just begun

Mission statement
-----------------

The ultimate goal of the nohrio project is to inspire the creation
of a system which renders the nohrio project obsolete.

Perhaps you find that confusing and recursive.
The point is: as per systemantics_ , as systems increase in complexity, age,
and usage, their implicit goal moves towards self-preservation -- until
eventually 100% of resources are dedicated to that task.

So I'm defining the goal of this system in a way to directly thwart that.

Some of the non-obvious ways this goal could be fulfilled:

 * migrating OHRRPGCE fileformats to be self-describing


Quickstart
----------

You can install the package using either ``python setup.py install``
or ``python setup.py develop`` (the second is mainly useful when you
want to make changes to nohrio and you want the installed nohrio module
to always be up-to-date with your changes, no need to keep running
``python setup.py install``.
On linux you will need to prepend ``sudo`` to those commands
(or otherwise be root when running them)


Start up an ipython session (or well, just a Python session -- but ipython is
vastly better, believe me.) (http://ipython.scipy.org)

>>> from nohrio.ohrrpgce import *

.. load an array from hexdump
   poke it around a bit
   convert it to dict/list format
   write some YAML from it
   calculate a diff between it and another record
   handle multiple records -- overwrite one with another.

.. read lumps inline from a packed RPG file (can even be writable -- however, is fixed in size
   (you can't write past the array end to add more records)

   arr = mmap (filename, dtype, offset = offset, shape = shape)

   The key here is specifying the shape. it's quite easy -- if you know you have 20 records (either by
   reading GEN data or calculations based on lump size), just specify 20.

   if you are in the tests/data directory, the following will mmap records 4-9 of EFS
   (there are 17 records total)

   >>> import numpy as np
   >>> arr = mmap ('viking.efs', dtype = dtypes['efs'], np.dtype (dtypes['efs']).itemsize * 4)
   >>> arr.shape
   (5,)

   When you want to read inline from an rpg file, the only difference is,
   you must also add the starting offset of the lump in the RPG file
   to the 'offset' value passed to mmap.

   (we currently have a wrapper class to virtualize RPG files /RPGdirs, it is very work-in-progress)

   You should also note that we don't currently deal directly with dt0 and attack.bin, since they are in
   a quite bizarre 'needs-stapling-together' format. So you cannot entirely avoid unlumping if you want
   to read attack data -- we need to do the joining, then memmap the resulting file.

.. filter an image (eg. skew)

::

 def xspin (array, amount):
     offset = 0
     step = float (amount) / array.shape[-2]
     for line in array:
     line[:] = line[int (offset):].tolist() + line[:int (offset)].tolist()
     offset += step

.. since:: 

   We now have a higher-level interface to RPG files and RPGdirs,
   which is accessed through the nohrio.rpg module.


.. you can extract multiple fields like this

>>> array[['maxmap','maxtileset','maxnpcpic']]
(48, 153, 20)

Building docs
-------------

Make sure you have the specified dependencies installed (Sphinx, SCons, PyLit).
Sphinx and SCons can be installed using ``easy_install Sphinx`` and ``easy_install SCons``

PyLit must be downloaded and installed manually
(place 'pylit.py' and 'pylit' files somewhere in your PATH,
``chmod +x pylit`` if you're using Linux or MacOSX)

Changing directory to doc/ and
running ``scons rst html`` will ensure you have fully up-to-date documentation in
doc/build/html/index.html


Building html from this file
----------------------------

Yes, this is written in ReStructured Text too :) If you have docutils installed
(hint: if Sphinx is working, you do have docutils installed :),
you can generate an html page from this file using the command
'rst2html.py README > readme.html' (and presumably open the resulting file in your web browser)


Improving docs
--------------

Do not edit the .rst files in doc/ -- generally they are autogenerated from a corresponding source
file (eg. doc/ohrrpgce.rst is generated from nohrio/ohrrpgce.py,)

Documentation is literate, meaning:

a) it's in the code and
b) the code has a meaningful narrative organization, just as a book does.

ReStructured text is the format of the documentation. Have a look at some of the comments in ohrrpgce.py
and you should then readily understand how to add documentation.

Note that documentation comments must be separated from the code by a blank line on either side,
as in::

 from ohrrpgce import mmap

 # This function does something too awesome to describe with words.
 # It's documentation is, therefore, in the form of interpretive dance.
 #
 # sillymovielink_
 #

 def myfunc (wow):




Credits
-------

David Gowers :
  Author.
  Wrote quite a few utilities for OHRRPGCE, and made the new master palette for it
  with a good deal of research and experimentation. Without those I would never have made these,
  this is a total paradigm shift (my old utilities were written when OHR ran on DOS :)

James Paige :
  Created the OHRRPGCE, through which I have learnt a surprising amount about fileformats.
  James would tell you not to use it as an example! And I would agree --
  it's a better counterexample. James is also a really nice guy.

Ralph Versteegen :
  Ongoing maintenance, initial testing and feedback, Script and other classes.
  It's easy, in your own intimate knowledge of a software  system you created, to be completely
  out of touch with 'what things people need to know'. He helped remedy that :)
  Also wrote several programs that use nohrio -- '3rdparty/colouruse.py', '3rdparty/tabulate.py',
  and [rpgbatch](https://github.com/ohrrpgce/tools/tree/tools/rpgbatch).

Fenrir-Lunaris :
  Created Vikings of Midgard, for which part of it's aim is to show off features.
  Gave me permission to use VoM lumps for my test suite, for which they are ideal for aforementioned
  reasons (And well, I like the game).
  Also an artistic inspiration, as both an example and counter-example.

A vast array of XML-abusers:
  Tremendously effective counter-examples. XML is not a Data Language, it's a Markup Language!

.. _systemantics: https://en.wikipedia.org/wiki/Systemantics