Dowser.
View Source
Per-node cache of index mappings, shared by the backend packages.
A backend package defines its own cacher with use, which gives it its own
process, its own ETS table and its own application environment:
defmodule Dowser.OpenSearch.MappingCacher do
use Dowser.MappingCacher, otp_app: :dowser_opensearch
endThe backend package doesn't start it: an application that casts against the cluster's mappings adds it to its own supervision tree, with its options:
children = [MyApp.Search, {Dowser.OpenSearch.MappingCacher, ttl: :timer.minutes(10)}]Asked without having been started, it reports {:not_started, cacher} —
which Dowser.Codec treats as a mapping it couldn't fetch.
The functions below take that module as their first argument (server);
the module use defines exposes them without it — start_link/1,
child_spec/1, key/2, get/2, lookup/2, fetch/2, put/3,
invalidate/2 and clear/0 — and implements Dowser.Mapping with
fetch_mapping/2, which makes it a backend's default mapping source.
A GenServer owns an ETS table; reads go straight to ETS (concurrent, lock-free), and only misses/expired entries go through the GenServer — which de-duplicates concurrent fetches for the same key (single-flight).
Entries are keyed by {endpoint, scope, index}, so the same index reached
through different contexts is cached separately, and each fetch honours its
context. The scope is what distinguishes two contexts pointing at the same
endpoint — see key/2.
Options (start_link/1)
:ttl— entry lifetime in ms (default 5 min).:sweep_interval— how often expired entries are actively purged, in ms (default 1 min).nil/0disables active sweeping; lazy expiration on read still applies.:fetch—(context, index -> {:ok, value} | {:error, reason}), how a mapping is loaded on a miss. Defaults to aDowser.ClientGET /{index}/_mappingcall.:eager— preload at startup viahandle_continue/2:false(default, lazy), a list of{context, index}, or a 0-arity fun returning one.
Mappings that are never fetched
A mapping given in the backend's application environment answers for its index without any request at all — and so can never fail to be fetched:
config :dowser_opensearch,
mappings: %{
"posts" => %{
"properties" => %{"published_at" => %{"type" => "date", "format" => "strict_date_time"}}
}
}It wins over the cache. Use it for an index whose mapping is pinned, and in
test suites, where it makes casting behave as it does in production without
a cluster to ask (put/4 does the same through a running cacher).
When a fetch fails
lookup/4 distinguishes "there is no mapping" from "the mapping could not be
fetched"; fetch/4 is the lenient read that folds both into nil. The
difference matters because the second silently changes the type of what a
caller gets back — see Dowser.Codec's :mapping_failure.
Summary
Functions
Returns a specification to start this module under a supervisor.
A child spec starting the cacher named server.
Clears the whole cache of server.
Like lookup/4, but returns the mapping directly, or nil when there is
none to be had or the fetch failed.
Returns the mapping for {context, index} from server, fetching lazily on a miss.
Invalidates a single {context, index} entry of server.
The cache key for {context, index}: {endpoint, scope, index}.
Caches mapping for index directly in server, without fetching anything.
Starts the cacher named server, which also names its ETS table.
Types
Functions
Returns a specification to start this module under a supervisor.
See Supervisor.
@spec child_spec(atom(), keyword()) :: Supervisor.child_spec()
A child spec starting the cacher named server.
@spec clear(atom()) :: :ok
Clears the whole cache of server.
@spec fetch(atom(), atom(), Dowser.Repo.t(), term()) :: map() | nil
Like lookup/4, but returns the mapping directly, or nil when there is
none to be had or the fetch failed.
The lenient read: a mapping that can't be resolved degrades a cast to identity rather than failing the request.
@spec get(atom(), Dowser.Repo.t(), term()) :: {:ok, term()} | {:error, term()}
Returns the mapping for {context, index} from server, fetching lazily on a miss.
@spec invalidate(atom(), Dowser.Repo.t(), term()) :: :ok
Invalidates a single {context, index} entry of server.
@spec key(Dowser.Repo.Config.t(), term()) :: key()
The cache key for {context, index}: {endpoint, scope, index}.
Two contexts pointing at the same endpoint do not necessarily see the same mapping — different credentials can mean field-level security hiding fields, or an alias resolving to a different concrete index — so the credentials are part of what identifies an entry.
They are hashed into it rather than stored. A cache key lives in an ETS
table for the lifetime of the entry, where any process can read it and a
crash dump would carry it; Dowser.Repo.Config goes as far as redacting
:auth from its own Inspect, and putting it in a table here would undo
that.
The hash is SHA-256, truncated to 128 bits. An application that builds
contexts from end-user credentials makes the hashed value attacker-
influenced, and a collision there would serve one tenant another's mapping —
so the cheaper :erlang.phash2/1 is the wrong tool: its default range is
2^27, which a targeted collision search exhausts in seconds.
:http_opts is hashed alongside :auth, since a credential can also arrive
as a header and a proxy or TLS setting can change which cluster answers. The
purely client-side fields (:profile, :keys, :decoder, :encoder) are
not: they can't change what the backend returns. A plain unauthenticated
context hashes to nil, so the common key stays readable.
Like get/3, but tells a mapping that cannot exist apart from one that
could not be fetched — which get/3 reports the same way, and which the
caller must not: casting against no mapping is correct for the first and
silently wrong for the second.
{:ok, mapping}— a static mapping (fromotp_app's environment), a cached one, or a freshly fetched one.{:ok, nil}— there is no mapping to be had: no index named, or no repo to ask.{:error, {:not_started, server}}— the cacher isn't running. A backend package doesn't start it: an application that casts against the cluster's mappings starts it in its supervision tree. A static mapping still answers without it.{:error, reason}— the fetch failed. The mapping may well exist; this request just couldn't see it.Dowser.Codecturns this into aDowser.MappingErrorby default, rather than casting nothing and handing back raw JSON values.
A static mapping wins over the cache and is never fetched, so it cannot fail.
Caches mapping for index directly in server, without fetching anything.
The seam tests use, so that casting behaves the way it does in production instead of degrading to identity for want of a cluster to ask:
setup do
Dowser.OpenSearch.MappingCacher.put("posts", %{"properties" => %{...}}, repo: MyApp.Search)
endOptions
:repo(required) — the repo the entry belongs to, as inkey/2: aDowser.Repomodule or[endpoint: ...]. Raises anArgumentErrorwhen it can't be resolved.:ttl— entry lifetime in ms (default:infinity, unlike a fetched entry: an entry put by hand is not a cached answer that can go stale).
@spec start_link(atom(), keyword()) :: GenServer.on_start()
Starts the cacher named server, which also names its ETS table.