Dowser.Client.NDJSON (Dowser.Client v0.3.0)

Encode and decode NDJSON (newline-delimited JSON), the payload format of bulk-indexing and multi-search style APIs.

Every line goes through Dowser.Client.JSON, so NDJSON decodes to the same string-keyed maps as a plain JSON body.

encode/2 joins the encoded entries with "\n" and appends a trailing newline (required by those APIs). decode/2 splits on newlines and skips blank lines, so a trailing newline — or the odd empty line — is handled transparently.

Both short-circuit, returning a Dowser.Client.JSON.Error as soon as a single line fails.

The casting options

A line, not the whole payload, is the unit here: each one is its own document, action or query. So :encoder/:encode and :keys/:decoder are applied per entry rather than once over the list, which is what lets an encoder pattern-match the action lines of a bulk request and leave them alone:

def encode(%{"index" => _action} = line, _opts), do: line
def encode(document, opts), do: cast(document, opts[:index])

A path works the same way, within each entry — encode: ["doc"] over a bulk update payload encodes the "doc" of every line that has one, and skips the action lines that don't.

See Dowser.Client.Encoder and Dowser.Client.Decoder for the options themselves; Dowser.Client.Request/Dowser.Client.Response pass them here.

Examples

iex> {:ok, iodata} = Dowser.Client.NDJSON.encode([%{"a" => 1}, %{"b" => 2}])
iex> IO.iodata_to_binary(iodata)
~s({"a":1}\n{"b":2}\n)
iex> Dowser.Client.NDJSON.decode(~s({"a":1}\n{"b":2}\n))
{:ok, [%{"a" => 1}, %{"b" => 2}]}

Summary

Functions

decode(binary, opts \\ [])

Decodes an NDJSON binary into a list of terms, one per non-blank line.

encode(entries, opts \\ [])

Encodes a list of terms into NDJSON iodata, one JSON document per line with a trailing newline.

Functions

decode(binary, opts \\ [])

@spec decode(binary(), keyword()) ::
  {:ok, [term()]} | {:error, Dowser.Client.JSON.Error.t()}

Decodes an NDJSON binary into a list of terms, one per non-blank line.

opts may carry :key_fn and :decoder (see Dowser.Client.Decoder), which are applied to each decoded entry.

encode(entries, opts \\ [])

@spec encode([term()], keyword()) ::
  {:ok, iodata()}
  | {:error, Dowser.Client.Error.t() | Dowser.Client.JSON.Error.t()}

Encodes a list of terms into NDJSON iodata, one JSON document per line with a trailing newline.

opts may carry :encoder and :encode (see Dowser.Client.Encoder), which are applied to each entry before it is encoded.