Components

Three common patterns for LLM application development:

1. Direct chat: handle user input and generate replies.
2. Knowledge processing: semantic processing, storage, and retrieval of text documents.
3. Tool calling: decide based on context and invoke the right tools.

Eino abstracts common capabilities into reusable components.

Mapping between component abstractions and these patterns:

Chat-oriented components

1. Modular handling of parameters for LLM interaction: ChatTemplate
2. Direct LLM interaction: ChatModel

Text semantics components

1. Loading and processing text documents: Document.Loader, Document.Transformer
2. Semantic processing of documents: Embedding
3. Storing indices after embedding: Indexer
4. Indexing and recalling semantically related documents: Retriever

Decision and execution components

Component for the model to decide and call tools: ToolsNode

Custom components

User-defined logic: Lambda

Eino’s component design follows these principles:

1. Modularity and standardization: capabilities with the same role are abstracted into uniform modules; components have clear roles and boundaries and compose flexibly.
2. Extensibility: interfaces constrain modules as little as possible so component authors can implement custom components easily.
3. Reusability: the most common capabilities and implementations are packaged for out-of-the-box use.

Chain & graph orchestration

Orchestration: compose and chain atomic component capabilities.

• Business logic must not leak into orchestration.
• The core of LLM apps is composing and chaining components that provide atomic abilities; components are first-class citizens in orchestration.
• From an abstraction standpoint, orchestration builds a network through which data flows; each node expects certain shapes/content for that data. A network that flows smoothly hinges on whether upstream and downstream data formats align.
• Complexity of scenarios shows up in the complexity of the orchestration artifact; only horizontal governance keeps complex setups under control.
• LLMs and LLM apps keep evolving fast; only extensible applications stay viable.

Eino provides a graph-based (edge + node) orchestration solution that:

• Uses components as atomic nodes
• Uses type alignment between upstream and downstream as the foundation
• Centers on components and standardizes how business functionality is encapsulated
• Keeps business complexity inside components so the orchestration layer has a clearer global view
• Offers aspect-like capabilities; callbacks support unified governance per node (what “aspect capability” means)
• Provides call options—extensibility is a basic need for fast iteration
• Emphasizes a type-aligned development style to reduce cognitive load and leverage Go’s type safety
• Provides automatic stream conversion so “streams” drop off the list of complexity drivers (Eino stream programming)

---

Graph downside: the point-and-edge graph model requires graph.AddXXXNode() and graph.AddEdge() to wire a data path—powerful but a bit heavy.

Eino wraps this with the easier Chain. Chain is a wrapper over Graph and exposes almost all Graph capabilities except cycles.