/add-memory-kind — Add a new business memory kind

When to invoke

Adding a new persisted business entity (Episode, Case, Skill, AtomicFact, Foresight, Profile, or something custom). Multiple storage layers may be involved; this skill walks the decision then the wiring.

1. Decide the storage combination

A memory kind does not have to use all three layers. Pick by what the kind actually needs:

Common combinations seen in EverOS:

Rule of thumb: markdown is the truth; sqlite and lancedb are derived indexes that can be rebuilt from md. Drop md only when the kind has no human-readable form (pure system state).

2. Pick the markdown storage strategy (if md is in your combo)

Three strategies — declared in the EverOS Markdown First spec:

This skill currently has a complete recipe for daily-log append. Skill-named and single-file recipes are sketched at the bottom — their base writers (BaseSkillWriter / BaseProfileWriter) land later in the project; until then build a thin wrapper over MarkdownWriter directly.

3. Markdown daily-log: 4 steps

3.1 Frontmatter schema — infra/persistence/markdown/_mds/<name>.py

"""Episode daily-log frontmatter."""

from __future__ import annotations

import datetime as _dt
from typing import ClassVar, Literal

from everos.infra.persistence.markdown import UserScopedFrontmatter


class UserEpisodeDailyFrontmatter(UserScopedFrontmatter):
    """``users/<u>/episodes/episode-<YYYY-MM-DD>.md``."""

    ENTRY_ID_PREFIX: ClassVar[str] = "ep"
    DIR_NAME: ClassVar[str] = "episodes"
    FILE_PREFIX: ClassVar[str] = "episode"

    type: Literal["user_episode_daily"] = "user_episode_daily"
    date: _dt.date
    entry_count: int = 0
    last_appended_at: _dt.datetime | None = None

For agent-track kinds subclass AgentScopedFrontmatter instead. If user-track and agent-track share a kind name (e.g. memcell), give each a distinct ENTRY_ID_PREFIX (e.g. umc vs amc) so reverse lookup is unambiguous.

3.2 Re-export — mds/__init__.py

from .episode import UserEpisodeDailyFrontmatter as UserEpisodeDailyFrontmatter

3.3 Business writer — infra/persistence/markdown/_writers/<name>.py

"""Episode appender."""

from __future__ import annotations

from pathlib import Path

from everos.infra.persistence.markdown import MarkdownReader

from ..mds import UserEpisodeDailyFrontmatter
from .base import BaseDailyWriter


class UserEpisodeAppender(BaseDailyWriter):
    schema = UserEpisodeDailyFrontmatter

    # OPTIONAL: override the count strategy. Default is len(entries);
    # override to trust the frontmatter field instead.
    def _current_count(self, path: Path) -> int:
        if not path.exists():
            return 0
        return MarkdownReader.read(path).frontmatter.get("entry_count", 0)

3.4 Re-export — writers/__init__.py

from .episode import UserEpisodeAppender as UserEpisodeAppender

Done — usage

from everos.infra.persistence.markdown._writers import UserEpisodeAppender

appender = UserEpisodeAppender(memory_root)
eid = appender.append("u_jason", "I went to the doctor today.")
# → users/u_jason/episodes/episode-<today>.md
# → entry markers carry an auto-generated EntryId (e.g. ep_20260507_001)

4. (Optional) SQLite table — 4 steps

Skip this section if the kind doesn't need structured state beyond markdown.

4.1 Schema — infra/persistence/sqlite/_tables/<name>.py

from everos.infra.persistence.sqlite import BaseTable, Field


class EpisodeState(BaseTable, table=True):
    __tablename__ = "episode_state"  # type: ignore[assignment]

    id: int | None = Field(default=None, primary_key=True)
    entry_id: str = Field(index=True, unique=True)
    cluster_id: str | None = Field(default=None, index=True)
    status: str = Field(default="active")

BaseTable already provides created_at / updated_at (auto-bumped).

4.2 Re-export — tables/__init__.py

from .episode import EpisodeState as EpisodeState

4.3 Repo — infra/persistence/sqlite/_repos/<name>.py

from everos.infra.persistence.sqlite._base.repository import RepoBase

from ..tables import EpisodeState


class EpisodeStateRepo(RepoBase[EpisodeState]):
    """Rows of ``episode_state``."""

    model = EpisodeState

The session factory comes in through RepoBase.__init__. Do not create a module-level instance: make check-singletons rejects one, and the repository would be bound to no database.

4.4 Re-export + wire — repos/__init__.py, repositories.py

# repos/__init__.py
from .episode import EpisodeStateRepo as EpisodeStateRepo

Add a field to SqliteRepos (sqlite/repositories.py) and build it in SqliteRepos.over(database); the bootstrap hands that bundle to whatever needs the table.

5. (Optional) Derived index — 4 steps

Skip this section if the kind doesn't need vector / BM25 / hybrid retrieval.

5.1 Record declaration — infra/persistence/records/<name>.py

from typing import ClassVar

from everos.infra.persistence.records import BaseLanceTable, Vector


class EpisodeIndex(BaseLanceTable):
    TABLE_NAME: ClassVar[str] = "episode_index"
    PRIMARY_KEY: ClassVar[tuple[str, ...]] = ("app_id", "project_id", "id")

    id: str
    app_id: str = "default"
    project_id: str = "default"
    entry_id: str
    text: str
    tags: list[str]
    vector: Vector(1024) | None = None  # type: ignore[valid-type]

TABLE_NAME / BM25_FIELDS / PRIMARY_KEY are the backend-neutral declarations every index backend reads (IndexRecordMeta). PRIMARY_KEY has no default: check_primary_keys refuses to start, and LanceDB upsert refuses to write, a table that does not declare one. id is unique only within (app_id, project_id), so every business table declares the three-column key — LanceDB merges on it, Milvus keys the collection on a composed __pk and rejects key components containing /. Vector(N) must match your embedding dimension.

5.2 Re-export — records/__init__.py

from .episode import EpisodeIndex as EpisodeIndex

5.3 Repo — infra/persistence/lancedb/_repos/<name>.py

from everos.infra.persistence.lancedb._base.repository import LanceRepoBase

from ...records.episode import EpisodeIndex


class _EpisodeIndexRepo(LanceRepoBase[EpisodeIndex]):
    schema = EpisodeIndex

The table comes from the LanceDbConnection passed as source=; no module-level instance (make check-singletons rejects one).

upsert(records) merges on the declared key and is the only write; look a row up with find_one_where(primary_key(app_id, project_id, row_id)).

5.4 Wire — index/repositories.py + each backend

Add the record to BUSINESS_SCHEMAS and a field to IndexRepositories (infra/persistence/index/repositories.py), then build the repository in LanceIndexBackend.__init__ (_EpisodeIndexRepo(source=connection)) and in MilvusIndexBackend.__init__. Business code reaches it as index.repositories.<field>, through the DerivedIndex the bootstrap builds.

6. (Future) Skill-named & single-file markdown strategies

When the new memory kind needs:

• skill-named files (one file per named skill, in-place rewrite) — wait for BaseSkillWriter, or use MarkdownWriter.write_markdown directly with a thin wrapper.
• single-file rewrite (one fixed file like user.md) — wait for BaseProfileWriter, same fallback.

These strategies do not use entry markers; their frontmatter schema does not need ENTRY_ID_PREFIX (only id / type / schema_version plus the scope mixin fields).

7. Verification checklist

• make lint — ruff + import-linter clean
• make test — existing manager / lifespan / writer tests still pass
• When the kind crosses multiple layers:
  • markdown entry id is the join key for the sqlite / lancedb rows
  • business code reads only via the injected repository (no raw engine access in service / memory)
  • cascade daemon (when it lands) can rebuild sqlite / lancedb from md alone — keep md as the truth

Tests by layer:

8. Common pitfalls

Background

• Architecture: ../../rules/architecture.md
• __init__.py re-export rules: ../../rules/init-py-and-reexport.md
• Async programming: ../../rules/async-programming.md
• Datetime handling: ../../rules/datetime-handling.md