tayauc.blogg.se - Firehose chat

FIREHOSE CHAT ARCHIVE

I would be keen to help with testing and have the compute and storage to do so, let me know. On the subject of clients, OE is on the path for retirement - any thoughts on building instrumentation for Erigon? This is where Indexers are likely going en masse in the near future.Ĭan an instrumented client continue to be used by traditional methods in parallel with serving functions for the firehose process? I expect most Indexers will transition gradually to Erigon where we are growing more and more confident of using that client in production (plus it brings a huge number of performance and resource efficiency benefits). On client instrumentation: Indexers are mostly running Openethereum 3.3.0-rc4 with tracing enabled. Where is the bulk of the time spent in preparing a firehose for a new protocol, assuming an existing client can be used? Sounds like the design of the protobuff schema is a significant and critical task? This is a really good read and very clearly communicates the rationale behind the design of The Firehose. I believe that, by introducing new ideas and having them collide with those of this community, we together will be able to create the greatest open data platform of all time. I’d ask you to concentrate all feedback and comments in this thread. It is a rather deep technical document, but I hope we can have some of the best brains meditate on it, provide suggestions and comments. reduces the need for RPC calls to nodes.

FIREHOSE CHAT ARCHIVE

simplifies the operations of an indexer by relying on flat data files instead of live processes like an archive node.

improves testability and developer experience when iterating on subgraphs.

reduces risks of non-deterministic output.

increases backfilling performance & maximize data agility by enabling parallel processing.

is capable of handling high throughput chains (network bound).

The goal of the Firehose is to provide a way to index blockchain data which:

to provide the necessary context to understand StreamingFast’s current contributions to the graph-node implementation.

to contribute knowledge and insights gathered by StreamingFast in the last 3+ years to The Graph’s ecosystem.This proposition is a building block to an even larger vision of performance optimizations of the indexing stack, leading the path to parallelization. It is called the Firehose.įirehose: a files-based and streaming-first approach to processing blockchain data. Today I want to present one of the ideas that helped us achieve great indexing performance in the past. I’m Alex from StreamingFast and you are reading my very first post on these forums.įollowing the recent announcement made by the Foundation, our team has been hard at work towards improving indexing performance on The Graph Network.