Skip to content

Querying Nessie Data as a Graph

Summary

In this tutorial, you will:

  • Start a PuppyGraph container alongside a Nessie-backed Iceberg lakehouse stack (Spark + Nessie REST catalog + MinIO) and load example data.
  • Connect Nessie to PuppyGraph and define a graph schema.
  • Run Cypher and Gremlin queries against the Nessie-managed Iceberg data as a graph.

Self-contained Nessie Data

This tutorial bundles a complete Nessie + Iceberg stack (Spark for writes, Nessie for catalog and version management, MinIO for object storage) and seeds it with the TinkerPop modern graph sample data.

In real deployments, PuppyGraph queries your existing Nessie catalog and storage directly. PuppyGraph treats Nessie as an Iceberg REST catalog; see Connecting to Iceberg for the connection reference.

Prerequisites

Please ensure that docker compose is available. The installation can be verified by running:

docker compose version

See https://docs.docker.com/compose/install/ for Docker Compose installation instructions and https://www.docker.com/get-started/ for more details on Docker.

Accessing the PuppyGraph Web UI requires a browser. The schema upload and query steps also have CLI alternatives via curl and the bundled Gremlin console.

Setup

Deployment

▶ Create a file docker-compose.yaml with the following content:

docker-compose.yaml 
version: "3"
services:
  spark-iceberg:
    image: tabulario/spark-iceberg
    container_name: spark-iceberg
    networks:
      - nessie_net
    depends_on:
      - minio
      - nessie
    environment:
      - AWS_ACCESS_KEY_ID=admin
      - AWS_SECRET_ACCESS_KEY=password
      - AWS_REGION=us-east-1
    ports:
      - "8888:8888"
      - "8180:8080"
      - "10000:10000"
      - "10001:10001"
  minio:
    image: quay.io/minio/minio
    container_name: minio
    networks:
      - nessie_net
    ports:
      - "9000:9000"
      - "9001:9001"
    environment:
      - MINIO_ROOT_USER=admin
      - MINIO_ROOT_PASSWORD=password
      - MINIO_REGION=us-east-1
    entrypoint: >
      /bin/sh -c "
      minio server /data --console-address ':9001' &
      sleep 5;
      mc alias set myminio http://localhost:9000 admin password;
      mc mb myminio/my-bucket --ignore-existing;
      tail -f /dev/null"
  nessie:
    image: ghcr.io/projectnessie/nessie
    container_name: nessie
    networks:
      - nessie_net
    ports:
      - "19120:19120"
    environment:
      - nessie.version.store.type=IN_MEMORY
      - nessie.catalog.default-warehouse=warehouse
      - nessie.catalog.warehouses.warehouse.location=s3://my-bucket
      - nessie.catalog.service.s3.default-options.region=us-east-1
      - nessie.catalog.service.s3.default-options.endpoint=http://minio:9000/
      - nessie.catalog.service.s3.default-options.path-style-access=true
      - nessie.catalog.service.s3.default-options.access-key=urn:nessie-secret:quarkus:nessie.catalog.secrets.access-key
      - nessie.catalog.secrets.access-key.name=admin
      - nessie.catalog.secrets.access-key.secret=password
      - nessie.server.authentication.enabled=false
  puppygraph:
    image: puppygraph/puppygraph:latest
    pull_policy: always
    container_name: puppygraph
    networks:
      - nessie_net
    environment:
      - PUPPYGRAPH_USERNAME=puppygraph
      - PUPPYGRAPH_PASSWORD=puppygraph123
    ports:
      - "8081:8081"
      - "8182:8182"
      - "7687:7687"
    depends_on:
      - spark-iceberg
networks:
  nessie_net:
    name: puppy-nessie

Default credentials

The compose file ships with default MinIO credentials (admin / password) and disables Nessie authentication for convenience. Change them before running on a publicly accessible machine.

▶ Start the stack:

docker compose up -d

[+] Running 5/5
 ✔ Network puppy-nessie           Created
 ✔ Container minio                Started
 ✔ Container nessie               Started
 ✔ Container spark-iceberg        Started
 ✔ Container puppygraph           Started

Data Preparation

▶ Open a Spark-SQL shell connected to Nessie's Iceberg REST endpoint:

docker exec -it spark-iceberg spark-sql \
  --conf spark.sql.catalog.demo=org.apache.iceberg.spark.SparkCatalog \
  --conf spark.sql.catalog.demo.uri=http://nessie:19120/iceberg/ \
  --conf spark.sql.catalog.demo.warehouse=s3://my-bucket/ \
  --conf spark.sql.catalog.demo.type=rest

▶ Paste the following SQL into the spark-sql ()> prompt to create the schema and insert data:

modern.sql 
CREATE DATABASE demo.modern;

CREATE EXTERNAL TABLE demo.modern.software (
  id   string,
  name string,
  lang string
) USING iceberg;
INSERT INTO demo.modern.software VALUES
  ('v3', 'lop',    'java'),
  ('v5', 'ripple', 'java');

CREATE EXTERNAL TABLE demo.modern.person (
  id   string,
  name string,
  age  int
) USING iceberg;
INSERT INTO demo.modern.person VALUES
  ('v1', 'marko', 29),
  ('v2', 'vadas', 27),
  ('v4', 'josh',  32),
  ('v6', 'peter', 35);

CREATE EXTERNAL TABLE demo.modern.created (
  id      string,
  from_id string,
  to_id   string,
  weight  double
) USING iceberg;
INSERT INTO demo.modern.created VALUES
  ('e9',  'v1', 'v3', 0.4),
  ('e10', 'v4', 'v5', 1.0),
  ('e11', 'v4', 'v3', 0.4),
  ('e12', 'v6', 'v3', 0.2);

CREATE EXTERNAL TABLE demo.modern.knows (
  id      string,
  from_id string,
  to_id   string,
  weight  double
) USING iceberg;
INSERT INTO demo.modern.knows VALUES
  ('e7', 'v1', 'v2', 0.5),
  ('e8', 'v1', 'v4', 1.0);

The above creates four Iceberg tables under demo.modern.*, managed by Nessie.

id name age
v1 marko 29
v2 vadas 27
v4 josh 32
v6 peter 35
id name lang
v3 lop java
v5 ripple java
id from_id to_id weight
e9 v1 v3 0.4
e10 v4 v5 1.0
e11 v4 v3 0.4
e12 v6 v3 0.2
id from_id to_id weight
e7 v1 v2 0.5
e8 v1 v4 1.0

▶ Exit the shell with Ctrl+D or quit;.

Modeling a Graph

We model the data as the TinkerPop modern graph: two node types (person, software) and two edge types (knows, created).

Modern Graph
Modern Graph

▶ First, log into the PuppyGraph Web UI at http://localhost:8081 with the credentials configured above:

Field Value
Username puppygraph
Password puppygraph123

There are two ways to define the schema in PuppyGraph: build it interactively in the Schema Builder, or upload a JSON file directly. Pick whichever you prefer; both produce the same graph.

Build the graph in the Schema Builder

The Schema Builder is the visual editor in the PuppyGraph Web UI for adding catalogs, nodes, and edges step by step. It's the recommended path when you're modeling a graph for the first time or want to inspect what each click produces. For a deeper visual walkthrough of every dialog and field, see Modeling a Graph through the Schema Builder. The summary below covers what's needed to build the modern graph against this tutorial's Nessie-managed Iceberg tables.

PuppyGraph treats Nessie as an Iceberg REST catalog, so the catalog type below is Apache Iceberg.

Connecting to Nessie

▶ Click Create Catalog, then expand Data Lakes and pick Apache Iceberg.

▶ Fill in the connection form:

Field Value
Catalog name nessie_data
Metastore type Iceberg REST
Endpoint URL http://nessie:19120/iceberg
REST Warehouse s3://my-bucket/
S3 Endpoint http://minio:9000
Access Key admin
Secret Key password
Path-style access enabled
SSL disabled
Nessie Iceberg REST catalog form
Nessie Iceberg REST catalog form

▶ Click Create Catalog.

Adding nodes

▶ Click Add Node in the toolbar. The Select Table for Node dialog opens. Expand nessie_data then modern, pick software, then click Next.

Select the software table for a node
Select the software table for a node

▶ In the Add Node wizard, click Add to ID and select id from the dropdown. The wizard moves id into ID Columns, leaving name and lang as attributes. Click Next, leave Enable Local Replication off, then click Add Node.

Configure the software node
Configure the software node

▶ Repeat for person. The flow is the same: click Add Node, pick the table, click Next, assign id to ID Columns, leave replication off, click Add Node.

Adding edges

▶ Click Add Edge in the toolbar, pick created from the catalog tree, then click Next.

▶ In the Add Edge wizard, set:

Field Value
From Node person
To Node software
FROM Select Column from_id
TO Select Column to_id
Configure the knows edge
Configure the knows edge

▶ Click Add to ID and select id to set the edge identifier. Click Next, leave Enable Local Replication off, then click Add Edge.

▶ Repeat for knows with both From Node and To Node set to person. The other settings are identical to created.

Completed modern graph schema
Completed modern graph schema

Upload a schema file

If you've already built the graph in the Schema Builder above, you can skip this section. The resulting schema is the same.

This method writes the full schema to a JSON file and uploads it directly. It's useful when you already have a schema for an environment and want to recreate it elsewhere (e.g. for CI, scripted setup, or copy-pasting between PuppyGraph instances).

▶ Create a file schema.json with the following content:

schema.json 
{
  "catalog": [
    {
      "name": "nessie_data",
      "type": "iceberg",
      "metastore": {
        "type": "rest",
        "uri": "http://nessie:19120/iceberg",
        "warehouse": "s3://my-bucket/"
      },
      "storage": {
        "useInstanceProfile": "false",
        "accessKey": "admin",
        "secretKey": "password",
        "enableSsl": "false",
        "endpoint": "http://minio:9000",
        "enablePathStyleAccess": "true"
      }
    }
  ],
  "node": [
    {
      "label": "software",
      "dataSourceGroup": {
        "externalDataSource": {
          "enabled": true,
          "catalog": "nessie_data",
          "schema": "modern",
          "table": "software",
          "mappedField": [
            { "sourceFieldName": "id",   "targetFieldName": "id"   },
            { "sourceFieldName": "name", "targetFieldName": "name" },
            { "sourceFieldName": "lang", "targetFieldName": "lang" }
          ]
        }
      },
      "id":        [{ "name": "id",   "type": "STRING" }],
      "attribute": [
        { "name": "name", "type": "STRING" },
        { "name": "lang", "type": "STRING" }
      ]
    },
    {
      "label": "person",
      "dataSourceGroup": {
        "externalDataSource": {
          "enabled": true,
          "catalog": "nessie_data",
          "schema": "modern",
          "table": "person",
          "mappedField": [
            { "sourceFieldName": "id",   "targetFieldName": "id"   },
            { "sourceFieldName": "name", "targetFieldName": "name" },
            { "sourceFieldName": "age",  "targetFieldName": "age"  }
          ]
        }
      },
      "id":        [{ "name": "id",   "type": "STRING" }],
      "attribute": [
        { "name": "name", "type": "STRING" },
        { "name": "age",  "type": "INT"    }
      ]
    }
  ],
  "edge": [
    {
      "label":         "created",
      "fromNodeLabel": "person",
      "toNodeLabel":   "software",
      "dataSourceGroup": {
        "externalDataSource": {
          "enabled": true,
          "catalog": "nessie_data",
          "schema": "modern",
          "table": "created",
          "mappedField": [
            { "sourceFieldName": "id",      "targetFieldName": "id"      },
            { "sourceFieldName": "from_id", "targetFieldName": "from_id" },
            { "sourceFieldName": "to_id",   "targetFieldName": "to_id"   },
            { "sourceFieldName": "weight",  "targetFieldName": "weight"  }
          ]
        }
      },
      "id":        [{ "name": "id",      "type": "STRING" }],
      "fromKey":   [{ "name": "from_id", "type": "STRING" }],
      "toKey":     [{ "name": "to_id",   "type": "STRING" }],
      "attribute": [
        { "name": "from_id", "type": "STRING" },
        { "name": "to_id",   "type": "STRING" },
        { "name": "weight",  "type": "DOUBLE" }
      ]
    },
    {
      "label":         "knows",
      "fromNodeLabel": "person",
      "toNodeLabel":   "person",
      "dataSourceGroup": {
        "externalDataSource": {
          "enabled": true,
          "catalog": "nessie_data",
          "schema": "modern",
          "table": "knows",
          "mappedField": [
            { "sourceFieldName": "id",      "targetFieldName": "id"      },
            { "sourceFieldName": "from_id", "targetFieldName": "from_id" },
            { "sourceFieldName": "to_id",   "targetFieldName": "to_id"   },
            { "sourceFieldName": "weight",  "targetFieldName": "weight"  }
          ]
        }
      },
      "id":        [{ "name": "id",      "type": "STRING" }],
      "fromKey":   [{ "name": "from_id", "type": "STRING" }],
      "toKey":     [{ "name": "to_id",   "type": "STRING" }],
      "attribute": [
        { "name": "from_id", "type": "STRING" },
        { "name": "to_id",   "type": "STRING" },
        { "name": "weight",  "type": "DOUBLE" }
      ]
    }
  ]
}

▶ In the Web UI, click Graph in the sidebar, then Upload Schema, and select schema.json.

Upload via CLI

You can also POST the schema directly:

curl -X POST -H "content-type: application/json" \
  --data-binary @./schema.json \
  --user "puppygraph:puppygraph123" \
  http://localhost:8081/schema

Querying the Graph

In the PuppyGraph Web UI, click Query in the sidebar. You can run graph queries in either Cypher or Gremlin.

The following query answers "What software was created by people that marko knows?"

MATCH path = (p:person)-[:knows]->()-[:created]->()
WHERE p.name = 'marko'
RETURN path;

g.V().hasLabel('person').has('name', 'marko')
  .out('knows').out('created').path()

There are two paths in the result: marko knows josh, who created lop and ripple.

Cleanup

▶ Shut down the stack:

docker compose down