Projects/Cereale

edit
revisions

what's new
search
help

kiwi


Cereale - A proposal for portable agent/object state persistence

Objects usually have a short lifetime : they are created at application start and destroyed at application end. Sometimes, data is extracted from the objects and saved in a file using a given file format (such as XML), and objects are also created when reading data from such a file.

Bringing persistence to objects means that there is a uniform way to make objects last longer than an application runtime, in other words, to make sure that the object will be able to both live in memory (while the application is active) and on the dist (when the application is not running).

The purpose of the Cereale project is to write a library that allows to easily implement object persistence, for purposes such as program checkpointing, remote procedure call and object bus. Cereale is implemented in C to provide portability and availability to many different languages.

Related pages:

1. Hypotheses

This document is based on the following hypotheses:

  1. Data can be represented by a combination of primitive values: numbers (signed, unsigned, integers, floats), strings(characters, unicode), and tuples. Tuples can be considered as the primitive that allows to structure the data.
  2. Primitive values can be represented as character strings or byte arrays
  3. Objects state is data.

2. Requirements

  • Store an object state at a given moment
  • Recursively store an object state when it is referenced by another object state
  • Allow to specify whether parts of an object state should be stored or not
  • Provide ways to specify how the object gets serialised (stored) and unserialised (unstored).
  • Allow to easily clone an object
  • Allow to separate state from object (externalisation)
  • Allow to exchange states among objects (mutations)
  • Allow to unstore old object states, event when the object definition (class, proto, etc) has changed.
  • Provide a mechanism to allow two objects to exchange object state, and negociate a proper storage model.
  • Language, system and platform neutral implementation
  • Compact and simple

3. Terminology

In order to keep things clear, we need to define the following terms:

State
the state of an object is the data encapuslated by the object, in other words the object attributes. According to some definition of objects, an object can be considered as an encapsulation of a state and of the operations that manipulate this state.
Persistence
a persistent program is a program that is able to save its whole state upon request, and restore it at any moment. The same apply to persistent objects. Note that I do not consider that a persistent program in the context of fault/tolerant systems, even if there are obvious benefits.
Serialisation
serialisation is the mechanism that converts a state to its representation, under the form of a binary array or a character string. The equivalent term for serialisation is marshalling.
Deserialisation
deserialisation is the mechanism of recreating a state from its representation. The equivalent term is unmarshalling.
Transient
transient data is data that does not need, or should not be serialiased.

4. Proposition

3.1. Primitive values

  1. Nil: represents an empty, undefined value
  2. Integers: represents integer values, they are always signed.
  3. Floats: represents real numbers, also always signed.
  4. Strings: unicode strings.
  5. Tuples: ordered list of primitives
  6. Reference: a reference to an object (see below)

3.2. Object value

An object is identified by:

  • A number: the number references a particular object instance within the stored state. This number can be used by references to represent a reference/pointer to another object.
  • A classifier: the object belonging class/prototype is described by a unicode string.
  • A version: the object version is used by the serialisation system to determine how or if a stored state can be loaded.

3.3. API

Objects which state can be stored must be declared as "Storable" objects. Storable objects define the following properties: 

   interface Storable
   {
         // Instance methods
         // ----------------

         // Captures the object state, storing it in a StoredState? instance
         public StoredState? store();
         // Changes the object state with the given state
         public void mutate( StoredState? aState ) throws MutationException?;

         // Class methods
         // -------------

         // Restores an object from the given state
         public static restore( StoredState? aState ) throws RestoreException?;
         // Gets the version of this storable class
         public static Version getVersion();
         // Geths the classifier of this storable class
         public static String  getClassifier();
   }

Storable objects basically allow to generate a StoredState? object, which represents a snapshot of the current object state, and also allows to restore the object state given a particular StoredState? object. Details of StoredStates? will be given later. Storables also define accessors for version and classifier, which enable the serialisation system to manage information about available storable objects.

Version objects are defined as followed: 

   interface Version
   {
         public unsigned int getMajorNumber();
         public unsigned int getMinorNumber();
   }

RestoreExceptions? and MutationExceptions? can be subclassed to the developer's needs.

Storable object are at first registered into the StorageManager?, which takes care of referencing every available storable class: 

   interface StorageManager?
   {
         // Registers a storable class within the storage manager
         public void registerStorable( Class aStorable );
         // Tries to restore the object given the state.
         public Object restore( StoredState? aState ) throws RestoreException?;
   }
Storing an object state

Here are examples of how the API can be used to store an object state: 

   StoredState? state = new StoredState?( this.getClass() );

   //We store an integer
   state.storeInteger(1);
   //We store a flot
   state.storeFloat(0.1);
   //We store a string encoded as UTF-8
   state.storeString("Hello");
   //We store a string encoded as ISO-8859-1
   state.storeString(" World!", ISO_8859_1);
StoredState? signatures

So as to identify if stored states can be restored (fully or partially), storage manager also manage StoredState? signatures. A stored state signature is a string representing stored values types. For instance a state as follows: 

   Object (id=0)
   +-- Integer (val=1)
   +-- Tuple
         +-- Integer (val=10)
         +-- String  (val="Hello")

Will have the signature "1(12)". In case an object is refereced, the object classifier is not stored in the signature.

4. Adding automatic serialisers/deserialisers

It may be useful to provide serialisation/deserialisation mechanisms for objects that you do not control. For instance, you may want to provide store/restore mechanisms for existing API objects (arrays, maps, and so on). In this respect, the storage manage allows to register specific serialisers and deserialisers for a particular class: 

  storageManager.makeStorable( Class objectClass, Store store )

This can be used as follows: 

  array = new Array( new int[] { 1, 2, 3, 4, 5 } );
  storageManager.makeStorable( array.getClass(), new ArrayStore?( )

  try
  {
         state = storageManager.store(array);
  }
  catch ( StoreException? )
  {
         ...
  }

The Store interface is defined as follows: 

  interface Store
  {
         public StoredState? store( Object o ) throws StoreException?;
         public Object restore( StoredState? s ) throws RestoreException?;
  }

5. References

last modified on September 27, 2005, at 03:35 PM

© type-z.org and its contributing authors, 2001-2004.
Content is licensed under a Creative Commons License.