Koala XML Store

XML Store is a peer-to-peer mobile persistence layer with a value-oriented interface for storing and retrieving semi-structured data. Koala XML Store is loosely built upon the Bøgebjerg XML Store implementation. Its features include asynchronous sharing and garbage collection.

Please send any comments, requests, or bug reports to Thomas Ambus.

Downloads

Koala XML Store is implemented using J2SE 5.0 and also requires this to run. You can browse the API JavaDocs online.

Binary jar:	koala-xmlstore-0.5.4.jar
Binary distribution:	koala-xmlstore-0.5.4.tar.gz
Binary distribution:	koala-xmlstore-0.5.4.zip
Source distribution:	koala-xmlstore-0.5.4-src.tar.gz
Source distribution:	koala-xmlstore-0.5.4-src.zip

Dependencies

Koala XML Store requires the following components:

http://www.plan-x.org/projects/msd/	msd-1.3.jar
http://www.megginson.com/Software/	xml-writer-0.2.jar

Optional components that you may wish to use, but are not required, include:

http://www.plan-x.org/projects/xpath/

xpath-1.2.jar

Maven Repository

A Maven repository containing all Plan-X components has been setup at www.plan-x.org/maven/. To enable your Maven project to download files from this repository, include the following line in the project.properties file:

maven.repo.remote = http://www.ibiblio.org/maven/,http://www.plan-x.org/maven/

Application Development

If you are doing a Plan-X related project, take a quick look at the project development page. It describes setting up a Plan-X project and suggests good practices for project development.

Usage Examples

The following is a small example of using a local XML Store to store an XML file and extracting the first child element of the root. The source is also available here.

import org.planx.xmlstore.*;
import org.planx.xmlstore.stores.*;
import org.planx.xmlstore.input.*;
import java.io.*;

public class XMLStoreExample {
    public static void main(String[] args) throws XMLException,
                                     UnknownReferenceException,
                                                   IOException {
        if (args.length < 1) {
            System.out.println("Usage: XMLStoreExample <xml file>");
            return;
        }

        // Create XML Store
        XMLStore xmlstore = new LocalXMLStore("mystore");

        // Use SAX parser to construct a representation of the file
        Node node = SAXBuilder.build(args[0]);

        // ...and save it in XML Store
        Reference ref = xmlstore.save(node);

        // Load the first child
        Node c = xmlstore.load(ref).getChildren().get(0);

        // ...and print its tag
        System.out.println(c.getNodeValue());

        xmlstore.close();
    }
}

The following illustrates how to created a distributed XML Store. The first peer, peer1, constructs a DistributedXMLStore from a LocalXMLStore as follows:

        XMLStore store = 
          new DistributedXMLStore(
            new TranslatorXMLStore(new LocalXMLStore("mystore")),
            9000 /* UDP port for routing */,
            9000 /* TCP port for data */,
            null /* no bootstrap peer */
          );

Subsequent peers, can "join" the XML Store on peer1 by creating a DistributedXMLStore (again from a LocalXMLStore) this time explicitly giving peer1 as bootstrap as follows:

        XMLStore store = 
          new DistributedXMLStore(
            new TranslatorXMLStore(new LocalXMLStore("mystore")),
            9000 /* UDP port for routing */,
            9000 /* TCP port for data */,
            new InetSocketAddress(peer1, 9000 /* route UDP port of peer1 */)
          );

replacing peer1 with the IP address or host name of peer1. (This only works if the two peers are on different hosts; if they are on the same host use a different set of ports for the second peer.) All subsequent peers use this fragment for joining the network (but may use any of the already joined peers as bootstrap peers).

The following illustrates how to use the XPath engine (developed by Thomas Ambus) with XML Store.

import org.planx.xmlstore.*;
import org.planx.xmlstore.input.*;
import org.planx.xpath.*;
import org.planx.xpath.object.*;
import java.io.*;
import java.util.*;

public class XPathExample {
    public static void main(String[] args) throws XMLException,
        if (args.length < 2) {
            System.out.println("Usage: XMLStoreExample <xml file> <xpath expr>");
            return;
        }

        Node node = SAXBuilder.build(args[0]);

	XMLStoreNavigator nav = new XMLStoreNavigator();
	XPath xp = new XPath(args[1], nav);
	XObject obj = xp.evaluate(new DocNode(n));
	if (obj instanceof XNodeSet) {
	    XNodeSet ns = (XNodeSet) obj;
	    Iterator it = ns.iterator();
	    while(it.hasNext()) {
	       Node c = (Node) it.next();
	       System.out.println(c);
	    }
        } else {
            System.out.println(obj);
        }
    }
}

(As can be seen, an XML Store is not really needed at all, just a Node. However, typically, one performs an XPath expression on a node stored in an XML Store.)

Hints

Storing large amounts of data: It is no good to build an in-memory graph of all Nodes in your document since the JVM will run out of memory. Saving does not release the saved nodes from memory. To achieve this, split your graph into smaller parts and create a proxy Node out of the returned Reference. Use this new node as the only handle to the data and release all pointers to the original data by setting them to null. Here is an example:
```
    XMLStore<LocalLocator> xmlstore = new LocalXMLStore("mystore");
    NodeFactory factory = NodeFactory.instance();
    Node original = ...(create data)...;
    Node proxy = factory.createProxy(xmlstore, xmlstore.save(original));
    original = null;
    
```
Note, that SAXBuilder will create proxy nodes automatically while saving data. The frequency with which this is done is determined by the parameter flushDepth.

Changes

Release	Date	Feature	Description
0.5.4	10-May-2005	Network protocol extensions	Added support for nameserver operations in the network protocol. Added interface ExecXMLStore to allow XML Stores to execute code stored in an XML Store (also available via the network protocol).
0.5.3	31-Jan-2005	Interface additions	Added NodeFactory.createProxy to handle saving of large documents. Added DistributedMap.getLocalIdentifier().
0.5.2	26-Jan-2005	Binary node bugfix	Fixed a serious bug that caused the sharer to destroy binary nodes when these were not in memory.
0.5.1	07-Jan-2005	XMLStore parameterization	XMLStore and NameServer interfaces are now parameterized by Reference allowing type checking when combining XMLStore decorators.
		XMLStore name server	Added getNameServer() method to XMLStore interface.
		DVMBuilder obsolete	Removed DVMBuilder - use SAXBuilder instead (possibly calling XMLStore.save(Node) afterwards to obtain a Reference instead of a Node).
0.5.0	06-Jan-2005	Internal rewrite	Complete rewrite of internal storage system. `XMLSTORE_DATA` environment variable no longer supported.
0.4.18	27-Oct-2004	Replication	Simple replication to N <= K peers with `DistributedXMLStore`. Here, K is the neighbourhood/bucket size in the Kademlia routing layer.
		Binary node type	Added new node type `BINARY` for storage of raw data. Use `NodeFactory` to create.
		Storage location	Added environment variable `XMLSTORE_DATA` specifying path to local storage location. If not set, current directory will be used.
0.4.17	21-Oct-2004	Koala XML Store	First release.

Resources

Last updated: 10-May-2005 (Henning Niss)