Why use HDP?

HDP Provides Powerful Use Cases

HDP has the power to process and compute massive amounts of on-chain data in fully sound way.

Efficiency Through Batching

HDP can batch multiple tasks together, significantly reducing proving and verification costs for all users.

Batching is crucial because, in Cairo, the efficiency of proving remains constant for (2^n) clock cycles. For example, proving 6 clock cycles takes the same time as proving 8 clock cycles. Therefore, batching tasks maximizes efficiency.

Moreover, when batching, data can be reused between runs if there is overlap in requirements. If multiple users request similar computations or access the same data, HDP can optimize the process by reusing data, further reducing costs and improving performance.

Define Any Arbitrary Computation Using On-Chain Data

HDP encapsulates computations into modules. These modules can be either predefined or user-defined functions that perform calculations on authenticated data.

For common computations like calculating average balances, you can use a predefined data lake with appropriate parameters and a predefined module, such as the average function. This approach simplifies the process for standard computations.

However, developers may need to execute custom logic within HDP that doesn't fit into predefined functions like min, max, or average. Instead, it may involve custom conditions and specific data access patterns.

To address these needs, HDP allows developers to define their own modules. These user-defined modules can be Turing complete, meaning they can access any on-chain data and perform any required computations.

Private Input, Private Computation

Additionally, HDP supports private modules, where only the prover, registry maintainer, or deployer knows the module's function. This feature enables use cases such as trustless yet non-gameable credit scoring, where computation details remain private while ensuring result integrity.

Structured vs. Unstructured Data

Many use cases require specific data access patterns, such as retrieving all ETH or USDC balances of an account within a given timeframe. These structured data access patterns are commonly referred to as data lakes.

Structured data access patterns provide a clear and organized structure for the resulting data set. However, some scenarios require more complex approaches.

In certain use cases, you might need to define logic where, if the average balance exceeds a certain value, you then check the number and frequency of transactions for that address. This involves accessing various unpredictable data points from the blockchain, such as the account's nonce, all transactions associated with that nonce, their timestamps, and more.

These scenarios require handling unstructured data, which is inherently more complex and less predictable than structured data.

Accessing Various Data Types in HDP

HDP enables access to various types of on-chain data, making it a powerful tool for developers. Here's a breakdown of the data types you can access:

• Block Headers: Summaries of each block on the blockchain, containing metadata such as the previous block hash, timestamp, and more.

• Account Balances: The amount of ETH held by an account at any given time.

• Account Nonces: A counter that tracks the number of transactions sent from an account. It helps in preventing double-spending and replay attacks.

• Account Code: The smart contract code associated with an account. Smart contracts are self-executing contracts with the terms directly written into code.

• Account Storage: The storage space where smart contracts can save data on the blockchain.

• Transaction Receipts: Records that contain the outcome of transactions, including logs of events and the status of the transaction (success or failure).

• Transactions: The individual operations sent from one account to another, including the transfer of funds or execution of smart contracts.

• Blobs: Large, arbitrary chunks of data stored on the blockchain, often used in advanced applications such as rollups. They have been introduced with EIP-4844.

• Beacon Chain State: The state of the Beacon Chain, which is part of Ethereum and responsible for managing the proof-of-stake protocol.

HDP provides developers with the ability to access and utilize these diverse data types, enabling a wide range of applications and computations on the Ethereum blockchain.

HDP Modules

More about Module checkout jargon page

HDP modules enable developers to define custom computations they want to perform on the provided data scope. Modules receive authenticated on-chain data as input, process it according to their defined logic, and produce a result that is verified using STARK proofs. HDP offers both predefined modules (aggregation functions) for common computations and allows developers to create custom modules for specific or more complex tasks. Currently, modules can only be written in Cairo 1.

All HDP modules are identified by their corresponding code hash. A registry manages these modules, and when a developer uploads a module to the registry, it becomes callable on-chain. This system allows for easy utilization and sharing of modules within the HDP ecosystem.

Data Processor Features

Feature	Today	Future Releases
On-chain data access	✅	✅
Storage proofs	✅	✅
Aggregating functions	✅	✅
Multi-chain deployment	✅	✅
Unrestricted access patterns	✅	✅
Parallelization	❌	✅
Caching	❌	✅
Composition	❌	✅
Ability to pull data from multiple chains at once	❌	✅
Custom modules	✅	✅
Private modules	❌	✅