You are viewing the documentation for Pilosa v0.4. View the latest documentation for Pilosa v0.8.

PDK

The Pilosa Dev Kit contains Go libraries to help you use Pilosa effectively. From importing data quickly, to managing the mappings from contiguous integer ids to values of other types, the PDK should help you get off the ground quickly.

The PDK also contains some fully worked examples which make use of its tools. These are available in the usecase subdirectory and can be run as subcommands of the pdk binary.

Library

Mapping

Importing data into Pilosa is dependent on mapping it to integer IDs. PDK provides some predefined functions for inline mapping to simplify this process, supported by a framework for linking these mappings with the associated fields in a source CSV file. If no custom mapping is necessary, the entire import process can be described by an import definition file. The file is composed of four main parts:

This definition file can quickly get long, and defining it manually would be quite tedious. That’s why we have a tool to generate a definition file by looking at a data set. This will handle most of the legwork, but since it can only guess at the application, it uses the simplest mappings - each column gets mapped to one frame in an appropriate way. This is intended as a starting point, to be updated to suit your use of the PDK.

With this definition available, the PDK tool can run the import, which consists of these steps:

The process is summarized in this flowchart:

Bitmapper flowchart

Some of the simple mapper functions available with PDK include:

Examples

Run make install to build and install the pdk binary which contains all the examples. Just running pdk will bring up a list of all the examples, with a brief description of each. pdk help <example> will bring up a more detailed description of that example along with all arguments that it accepts to configure its functionality.