Understanding Query Performance

Welcome to Indexes and Performance! This is the first lesson in our journey together, and we're starting with something that will prove to be valuable throughout your work with Neo4j: understanding query performance.

When we write queries against a graph database, we often focus on getting the correct results. However, as our data grows, we need to understand not only what our queries return but also how efficiently they execute. In this lesson, we'll explore Neo4j's built-in tools for analyzing query performance, which will help us write faster and more efficient queries.

Before diving into the tools, let's establish why performance analysis is crucial. When working with graph databases, the same result can often be achieved through different query patterns. Consider finding a user named Alice: we could scan all users and filter by name, or we could use an index to jump directly to the matching record.

With a handful of users, both approaches work fine. However, as our database grows to thousands or millions of nodes, the difference becomes significant. One query might take milliseconds, while another takes seconds or even minutes. Understanding why this difference exists helps us make informed decisions about how to structure our queries, what indexes to create, and how to optimize our database design.

Neo4j provides two keywords that help us peek under the hood of query execution: PROFILE and EXPLAIN. While they serve similar purposes, they work differently.

EXPLAIN shows us the execution plan without actually running the query. It's safe to use with any query, including those that modify data, because nothing actually happens. We see what Neo4j plans to do, along with estimated metrics.

PROFILE, on the other hand, actually executes the query and returns real performance metrics. This gives us accurate information about what happened during execution, including actual row counts, database operations, and resource usage. For this lesson, we'll focus on PROFILE, since it provides the most concrete performance insights.

Let's see PROFILE in action. The keyword is simple to use: we just place it at the beginning of any Cypher query:

PROFILE MATCH (user:User {name: 'Alice'})
RETURN user

This query does two things. First, it executes normally and returns any users named Alice. Second, it provides a detailed execution plan showing exactly how Neo4j processed the query. Here's what the output might look like:

user
(:User {name: "Alice", city: "New York", age: 28, email: "alice@example.com"})

Plan: "PROFILE"
Statement: "READ_ONLY"
Version: "5"
Planner: "COST"
Runtime: "SLOTTED"
Time: 45
DbHits: 46
Rows: 1
Memory (Bytes): 64

When we run a PROFILE query, Neo4j returns several key pieces of information. The execution plan shows us a tree of operations, where each operation represents a step in processing our query. The plan reveals which operations were performed, in what order, and how much work each operation required. Notice the key metrics: Time shows execution time in milliseconds, DbHits shows the number of database accesses, Rows indicates results produced, and Memory shows bytes used during execution.

For each operation, we see metrics like:

• The number of rows produced
• database hits performed
• memory used
• time taken

These metrics help us identify bottlenecks. If we see an operation with millions of database hits but only a few rows produced, that is a sign that the query is doing unnecessary work. Similarly, operations that produce huge numbers of intermediate rows might indicate that we need to filter earlier in the query.

The most important metric to understand is database hits. This represents the number of times Neo4j accessed the underlying data store to read nodes, relationships, or properties.

Think of database hits as the amount of "work" the database had to do. Even if a query returns just one result, it might require thousands or millions of database hits to find that result. For example, scanning through every user in the database and checking their name property would generate one database hit per user, plus additional hits for reading properties.

Lower database hits generally mean faster queries, especially when combined with good cache utilization. When we optimize queries, we are often trying to reduce database hits by helping Neo4j find the data more directly, usually through indexes or by restructuring the query pattern.

Now let's look at a slightly different way to write the same query. This comparison will show us how Neo4j processes different query patterns:

PROFILE MATCH (user:User)
WHERE user.name = 'Alice'
RETURN user

At first glance, this looks very similar to our previous query. Both find users named Alice, but the structure differs. The first query included the name filter directly in the pattern match, while this version matches all User nodes first and then filters by name in a separate WHERE clause.

Running this query produces output like:

user
(:User {name: "Alice", city: "New York", age: 28, email: "alice@example.com"})

Plan: "PROFILE"
Statement: "READ_ONLY"
Version: "5"
Planner: "COST"
Runtime: "SLOTTED"
Time: 26
DbHits: 46
Rows: 1
Memory (Bytes): 64

When we compare the execution plans of these two queries, we can examine the key metrics side by side:

Notice that both queries perform the same number of database hits (46), produce the same number of rows (1), and use the same amount of memory (64 bytes). The execution time varies slightly, but this is often due to factors like caching, system load, or other background processes rather than fundamental differences in the query execution.

This demonstrates an important insight: Neo4j's query planner is sophisticated enough to recognize that these patterns are equivalent and generates very similar execution plans. The query with the inline property match {name: 'Alice'} and the query with a separate WHERE clause both access the data in essentially the same way.

By comparing different approaches with PROFILE, we learn to recognize which query patterns lead to efficient execution and, importantly, when different syntax produces equivalent performance. This knowledge helps us write better queries from the start, rather than having to optimize them later when performance problems emerge.

In this lesson, we explored the foundation of query performance analysis in Neo4j. We learned about the PROFILE keyword, which executes queries and returns detailed performance metrics, including database hits, row counts, and operation details. We also saw how the same logical query can be written in different ways and why it is important to verify actual performance rather than making assumptions.

Understanding execution plans is a skill that develops with practice. As we progress through this course, we will build on these concepts, learning to interpret more complex plans and recognize common performance patterns. Now it's time to put this knowledge into action and start profiling queries yourself!