Elastic.Channels 0.2.1

Elastic.Channels

Provides an specialized System.Threading.Channels.ChannelWriter implementation that makes it easy to consume data pushed to that thread in batches.

The batches will emit either when a certain maximum is hit or when a batch's lifecycle exceeds a certain age.

This allows data of various rates to pushed in the same manner while different implementations to send the batched data to receivers can be implemented.

This package serves mainly as a core library with abstract classes and does not ship any useful implementations.

It ships with a NoopBufferedChannel implementation that does nothing in its Send implementation for unit test and benchmark purposes.

BufferedChannelBase<>

An abstract class that requires implementers to implement:

protected abstract Task<TResponse> Send(IReadOnlyCollection<TEvent> buffer);

Any implementation allows data to pushed to it through:

var e = new TEvent();
if (await channel.WaitToWriteAsync(e))
	written++;

ChannelOptionsBase<>

Implementers of BufferedChannelBase<> must also create their own implementation of ChannelOptionsBase<>. This to ensure each channel implementation creates an appropriately named options class.

Quick minimal implementation


public class Event { }
public class Response { }

public class NoopChannelOptions 
  : ChannelOptionsBase<Event, Response> { }

public class NoopBufferedChannel 
  : BufferedChannelBase<NoopChannelOptions, Event, Response>
{

  public NoopBufferedChannel(NoopChannelOptions options) 
    : base(options) { }

  protected override Task<Response> Send(IReadOnlyCollection<NoopEvent> buffer)
  {
    return Task.FromResult(new Response());
  }
}

Now once we instantiate an NoopBufferedChannel we can use it push data to it.

var e = new Event();
if (await noopChannel.WaitToWriteAsync(e))
	written++;

BufferOptions

Each ChannelOptionsBase<> implementation takes and exposes a BufferOptions instance. This controls the buffering behavior of BufferedChannelBase<>.

Option Description
MaxInFlightMessages The maximum number of in flight instances that can be queued in memory. If this threshold is reached, events will be dropped
MaxConsumerBufferSize The number of events a local buffer should reach before sending the events in a single call to Elasticsearch.
MaxRetries The maximum number of retries over Send
MaxConsumerBufferLifetime The maximum age of buffer before its flushed
ConcurrentConsumers Controls how many concurrent Send operations may occur
BackOfPeriod Func that calculates an appropriate backoff time for a retry
BufferFlushCallback Called once whenever a buffer is flushed, excluding retries
WaitHandle Inject a waithandle that will be signalled after each flush, excluding retries.

Showing the top 20 packages that depend on Elastic.Channels.

Packages Downloads
Elastic.Ingest.Transport
Provides components to build a buffer-backed channel for publishing events to distributed systems over HTTP through Elastic.Transport
29
Elastic.Ingest.Transport
Provides components to build a buffer-backed channel for publishing events to distributed systems over HTTP through Elastic.Transport
14
Elastic.Ingest.Transport
Provides components to build a buffer-backed channel for publishing events to distributed systems over HTTP through Elastic.Transport
13
Elastic.Ingest.Transport
Provides components to build a buffer-backed channel for publishing events to distributed systems over HTTP through Elastic.Transport
12
Elastic.Ingest.Transport
Provides components to build a buffer-backed channel for publishing events to distributed systems over HTTP through Elastic.Transport
11
Elastic.Ingest.Transport
Provides components to build a buffer-backed channel for publishing events to distributed systems over HTTP through Elastic.Transport
10

https://github.com/elastic/elastic-ingest-dotnet/releases

.NET Standard 2.1

Version Downloads Last updated
0.8.0 9 02/18/2025
0.7.5 8 12/06/2024
0.7.4 7 12/01/2024
0.7.3 28 11/04/2024
0.7.2 25 11/04/2024
0.7.1 26 11/04/2024
0.7.0 17 04/12/2024
0.6.0 9 04/03/2024
0.5.7 15 03/17/2024
0.5.6 19 03/17/2024
0.5.5 9 03/20/2024
0.5.4 13 12/17/2023
0.5.3 10 03/20/2024
0.5.2 12 12/17/2023
0.5.1 11 06/13/2023
0.5.0 12 03/20/2024
0.4.3 9 03/20/2024
0.4.2 10 03/11/2024
0.4.1 12 03/19/2024
0.4.0 11 03/20/2024
0.3.2 9 03/20/2024
0.3.1 12 03/20/2024
0.3.0 10 03/19/2024
0.2.2 12 03/20/2024
0.2.1 10 03/20/2024
0.2.0 13 03/20/2024
0.1.0 13 02/11/2024