@lorecord/site-engine

Astro-based site engine with a content and theme convention.

Create a site:

npm exec --package=@lorecord/site-engine@latest -- create-site my-site
cd my-site
npm install
npm run dev

Generated projects use this structure:

site/
  assets/   source assets processed by Astro
  config/   system, site, and application data
  pages/    page content
  themes/   theme schemas, content resolvers, components, and templates
  public/   files copied directly to the output root
src/        optional project-specific source extensions

Generated projects also include systemd.service.example as a starting point for running the built static site with site start under systemd. Adjust its deployment path, Node.js PATH, and service user before installing it.

The project entrypoint is site.config.mjs, not astro.config.mjs:

import { defineSiteConfig } from "@lorecord/site-engine/config";

export default defineSiteConfig();

defineSiteConfig() supplies Astro's source entrypoints, content collections, middleware, localized routes, text routes, sitemap, CSP, public directory, and /_assets build path. A project only adds config when it needs a real extension.

Use the site command for the complete lifecycle:

site dev
site build
site preview
site start
site prepare
site routes
site search:capture
site search:notify
site interactions:sync [--since ISO_DATE]
site interactions:register
site interactions:send [--since ISO_DATE]
site astro <command>

Sites can expose interaction endpoints in site/config/system.yml:

interaction:
  enabled: true
  endpoint: https://interact.example/v1/interaction
  # Optional; otherwise interactions:register derives /v1/site/sync.
  metadataEndpoint: https://interact.example/v1/site/sync
webmention:
  enabled: true
  accept: true
  endpoint: https://interact.example/v1/webmention
pingback:
  enabled: true
  accept: true
  endpoint: https://interact.example/v1/pingback
mention:
  send:
    enabled: true
    since: 2015-01-01
websub:
  enabled: true
  endpoint: https://pubsubhubbub.appspot.com/

interactions:sync uses the site-scoped interaction query to fetch a site's results once, then stores them beside each item source in the native and webmention channels. Query APIs are public; if a deployment still expects authentication, INTERACT_TOKEN or --token can be sent. --since consumes the incremental upsert/delete events. interactions:send discovers external webmention/pingback endpoints and records delivery status in each channel's target.yml. search:notify --targets indexnow,websub submits IndexNow and WebSub notifications after deployment. Configure indexNowKey in site/config/system.yml (or use INDEXNOW_KEY for both build and notification).

interactions:register sends the complete route, comment-policy, feature, and author metadata to interact in one authenticated request. It scans all Markdown page templates, including non-item pages such as a guestbook, and does not send page bodies. Use --token or INTERACT_TOKEN for the site's interact token; --endpoint can point at either /v1/interaction or the explicit /v1/site/sync endpoint. Registration is optional for a first native comment: interact can create an unregistered root with its default interaction policy; registration should be used when the site needs explicit policy, authors, titles, or language metadata. IndexNow checks that the deployed /indexnow.txt matches the submission key before sending URLs. A 403 SiteVerificationNotCompleted response is retried after 10, 30, and 60 seconds; other errors are reported immediately. HTTP 202 means the submission was accepted with key validation still pending. If all verification retries fail, rerun site search:notify --targets indexnow later; SEARCH_NOTIFY_FAIL_ON_ERROR=1 makes notification failures exit with an error.

For legacy Markdown content that may contain missing local images, enable a warning-only fallback:

export default defineSiteConfig({
  engine: {
    markdown: {
      missingImages: "warn",
    },
  },
});

Missing images are printed to the server log and rendered as ordinary <img> elements, while existing images continue to use Astro's image pipeline.

Markdown extensions

The engine can provide the shared readMore and Mermaid behavior used by themes. readMore removes a marker-only paragraph from the rendered content; the default marker is ===. Mermaid code blocks are rendered by the browser and, by default, are also pre-rendered into static HTML during a production build:

export default defineSiteConfig({
  engine: {
    markdown: {
      readMore: true,
      // Or: readMore: { marker: "<!-- more -->" }
      mermaid: true,
      // Or: mermaid: { prerender: false }
    },
  },
});

Set mermaid: { prerender: false } when a site needs client-only Mermaid rendering. Sites and themes can also supply mermaid.render(source, context) to replace the engine's static renderer for build-time output.

alerts, footnotes, and externalLinks are also engine features. Themes can provide localized footnote labels and explicit Markdown options, while the engine supplies the shared AST transformations and reads the site's base HTML link settings.

Affiliate alternatives are an explicit Markdown extension. The normal link destination is the affiliate URL; non-aff supplies its ordinary alternative:

Compare [IPRoyal](https://iproyal.com/?r=dallaslu){non-aff="https://iproyal.com/"} plans.

Reference links support the same suffix. Existing .class, #id, and quoted attributes can appear in the same block. Code spans, fenced code and escaped braces are not interpreted as attributes. Both destinations must be absolute HTTP(S) URLs without credentials; invalid values fail the build with the source file location. The engine never infers affiliate links from query parameters.

With non-aff, the engine preserves the standard Markdown anchor's href, adds rel="sponsored nofollow" and a visible benefit disclosure followed by a real original anchor. Both remain available without a theme, JavaScript, or in feeds. The static fallback also includes a localized affiliate marker (〔推广〕-style for CJK locales and [Aff]-style for Latin-script locales) and an ordinary-link marker. The stable data-affiliate-link, data-affiliate-primary, data-affiliate-original, data-affiliate-mark, data-affiliate-fallback, data-affiliate-disclosure, data-affiliate-ordinary and data-affiliate-original-mark attributes let themes enhance the presentation without guessing destinations or rewriting links on click. engine.markdown.affiliateLinks accepts defaultLocale, labels (disclosure, ordinary, affiliate, original, markerOpen, markerClose) or getLabels(locale, file) to localize the fallback. Pill styles, favicon display and interactive affiliate details belong to themes, not to this content relationship.

The aff form remains supported, with its original meaning: the standard Markdown destination is ordinary and the attribute supplies its affiliate variant:

Compare [IPRoyal](https://iproyal.com/){aff="https://iproyal.com/?r=dallaslu"} plans.

This form explicitly selects the affiliate variant as the rendered primary link. Both forms generate the same disclosure and original-link fallback. They can coexist in one article; neither is deprecated. Both forms automatically add sponsored and nofollow, preserving other rel tokens without duplicates; ordinary alternatives do not inherit these affiliate markers. If one link specifies both attributes, they define the two destinations explicitly, and its Markdown destination must match one of them. A third, conflicting destination is an error. An unrelated Markdown renderer follows the standard Markdown destination and may display the attribute suffix as text: the paired-link fallback requires the engine's extension, or an explicit ordinary Markdown link or footnote.

Themes may provide static files under site/themes/<theme>/public/. The engine serves these files during development and copies them to the site output during a production build. The standard sitemap.xsl and rss.xsl files are optional; their XML stylesheet declarations are emitted only when the active theme provides the corresponding file.

The engine also generates /sitemap.txt from the common page metadata. A theme can extend the page model, but sitemap visibility, localization, and taxonomy URLs are handled by the engine.

Feeds

The engine provides the default RSS feed structure through @lorecord/site-engine/feeds. It excludes drafts, scheduled future pages and deleted pages, selects pages by locale and the template configured by the theme, and maps standard page metadata to RSS items. A theme only declares its feed paths and template mapping:

export const feed = {
  feed: "item",
  "note/feed": "note",
};

The engine automatically creates both unlocalized and localized feed routes. Themes can still use the lower-level renderRssFeed() and FeedItem helpers for genuinely non-page feeds such as comments or external news.

Interactions and content envelopes

The engine owns interaction and envelope data processing. It reads comments, citations, and content envelopes stored beside a content source, filters and normalizes them, renders comment Markdown, and builds reply trees. Themes only render the resulting data:

import { getCommentsForSource } from "@lorecord/site-engine/interactions";
import { getEnvelopesForSource } from "@lorecord/site-engine/envelopes";

const comments = getCommentsForSource(sourcePath);
const envelopes = await getEnvelopesForSource(sourcePath);

Settings model

The engine provides an extensible settings model. Its base shape covers site identity and localized metadata such as title, homeTitle, and description. Themes extend that shape only for values they consume, while site instances provide the values under site/config/settings/.

import { defineSettingsModel, loadSettings } from "@lorecord/site-engine/settings";
import { z } from "zod";

const model = defineSettingsModel({
	settings: { companyFounded: z.string().default("") },
});

export const { settings, localeSettings } = loadSettings(model);

Settings models are Zod schemas. The default locale file is validated directly; other locale files are deep-merged over it and then validated, so translations only need to provide changed values. Settings schemas reject unknown fields to surface typos instead of silently dropping them. Use Zod defaults for optional values and model intentional extensions explicitly.

Theme implementation configuration remains separate under site/themes/<theme>/config/; it is not merged into the site settings model.

The engine contains no theme-specific content-kind aliases, locale metadata, or site copy.

Pages

The engine defines the common ContentPageBaseData fields used for routing, localization, publication state, authors, summaries, taxonomy, identifiers, licenses, and CFF metadata. Themes can add their own page fields without reimplementing page grouping or locale fallback:

import {
  createContentPageResolver,
  type ContentPageBaseData,
  type ContentPageEntry,
} from "@lorecord/site-engine/pages";

type ThemePageExtension = { toc?: boolean; menu?: { header?: boolean } };
type ThemePage = ContentPageBaseData<"en"> & ThemePageExtension;

const resolver = createContentPageResolver<"en", ContentPageEntry, ThemePageExtension>({
  // ...the normal page resolver options
  pageDataShape: {},
});

For each content path, the resolver materializes a route for every configured locale when at least one routable source variant exists. A default file such as template.md is the fallback source for missing locales; when no default file exists, the first available localized variant is used as the fallback source. The requested route locale remains distinct from contentLocale, and availableContentLocales contains only actual routable source variants.

A localized variant marked routable: false, draft: true, scheduled for the future, or deleted (deleted: true or a deleted date) blocks that locale instead of falling back to another language. visible is not a generic route switch; it remains a theme/content-type concern.

Content dates and publication

date: '2023-05-10'       # Creation date, not the filesystem creation time
published: '2023-05-18'  # Optional first publication date; otherwise date is used
updated: '2026-09-05'    # Optional content update date, not a build timestamp
living: true           # Optional: a continuously maintained work

draft: true prevents publication regardless of the dates; omission means false. Dates are date strings (optionally with time and timezone), not booleans. Use published only when creation and first publication differ. An effective first publication date in the future schedules the content: routes, locale fallbacks, feeds and sitemaps exclude it until that time. Static sites still need a new build and deployment at or after the scheduled time. Drafts, deletion records and undated ordinary pages do not need invented timestamps.

All consumers use resolvePageDates(data) from @lorecord/site-engine/pages: created reads date, published reads published ?? date, and updated reads updated. isPagePublished(data, now?) centralizes draft, schedule and deletion checks; routing and feed consumers additionally honor routable: false. living describes editorial intent, not a date override. Themes may highlight an update for living works, but first publication, archive chronology, feed publication and citation defaults remain unchanged. Schema.org output maps the three date facts to dateCreated, datePublished and dateModified.

A localized source with its own body has its own dates and living value; it does not inherit a different language's update or publication time. Actual fallback content and metadata-only overrides retain the underlying timeline.

The read boundary accepts legacy published: true/false, modified, and wrapped { date: ... } values. It normalizes them in memory to the canonical fields; conflicting updated/modified or published: false/draft: false fail with a source-specific error. New content should use only the canonical fields. Migration removes published: true, replaces published: false with draft: true, and renames modified to updated; existing date and explicit publication values are retained. Preserve source date precision and the site's existing timezone convention. Never rewrite historical revisions or envelope bytes to migrate metadata, and never use revision creation time as content update time.

Accounts and profiles

The engine loads account profiles from site/data/accounts/. A base file such as username.yml can be supplemented by locale files such as username.zh-CN.yml. Profile lookup removes credential fields before returning data to a site or theme:

import { getProfile, resolveProfiles } from "@lorecord/site-engine/profiles";

const author = getProfile("username", "zh-CN");
const authors = resolveProfiles(["username"], "zh-CN");

Themes can extend the common profile shape without taking ownership of account loading or locale merging:

import { createProfileResolver, type Profile } from "@lorecord/site-engine/profiles";

type ThemeProfile = Profile & { mastodon?: string };
export const { getProfile, resolveProfiles } = createProfileResolver<ThemeProfile>({
  extendProfile: (profile) => profile as ThemeProfile,
});

Deployment adapters

site build writes language configuration, the HTML route manifest, and ready entrypoints under:

.site-engine/generated/adapters/cloudflare-worker.mjs
.site-engine/generated/adapters/esa-function.mjs

Adapter factories are also exported from @lorecord/site-engine/adapters/cloudflare and @lorecord/site-engine/adapters/esa. Generated adapters embed their route data and do not import sibling files through relative .site-engine paths.

Portable site state

Commit site/data/seo/previous-sitemap.xml, the last captured online sitemap, so deleted URL detection also works in a fresh checkout. site search:capture updates it before deployment; failed capture preserves the previous snapshot. Both capture and notify honor PREVIOUS_SITEMAP_PATH and previousSitemapPath; the default follows a custom siteDir. Migrate an existing .search-engine-cache/previous-sitemap.xml to the new location when upgrading. Review and commit snapshot updates after deployment; a CI working-directory change alone is not shared with the next run. A retained snapshot may be stale and should be refreshed before deployment when the site is reachable.

Also retain article .data/interactions/ delivery/source records and site data/webhub.json state. Generated dist/, .astro/, .site-engine/, Vite caches and Wrangler local state remain disposable. Mermaid's current cache is process-local; there is no existing portable render-cache file to migrate.

Engine and theme responsibilities

The engine owns remote link metadata, HTTP/DNS transport, persistent caches and refresh scheduling. Import createLinkCardResolver from @lorecord/site-engine/link-cards, and createLinkIconResolver from @lorecord/site-engine/link-icons. Cache paths and formats are unchanged. Default rendering remains offline; only explicit refresh performs requests.

refreshRenderedLinks(distDirectory, options) from @lorecord/site-engine/link-refresh requires a theme-supplied collectLinkRequests(html) returning { cards: [{ url, language }], icons: [url] }. It owns source-directory mapping, deduplication, bounded concurrency and progress. It accepts the usual site path options, including rootDir, siteDir, pagesDir. Unavailable metadata is represented by undefined; unexpected resolver errors propagate instead of silently becoming cache misses. HTML selectors and card/icon markup belong to the theme and are not dependencies of the engine.

evaluateFreshness(value, now) from @lorecord/site-engine/freshness returns expired or stale state with normalized dates. Themes choose the warning text, CSS classes and placement. Likewise, presentation choices such as header-date selection, menus and rendered-content feature detection stay in the theme.

Queries, metadata and cache lifetime

selectArticles (content-query) centralizes published/deleted/locale eligibility. Themes choose includeHidden, recommendation weights and result counts. Both boolean and dated deletion markers exclude articles from lists.

Metadata composition belongs to the theme. metadata offers string/date/media helpers and warning-only display URL handling; components/JsonLd.astro only serializes the supplied object, escaping HTML delimiters. Neither chooses a Schema.org type or dictates the site's content model.

Display settings use warnings and defaults in development and production alike. Malformed freshness notices are warned about and omitted. Metadata strings are retained when a URL cannot be resolved. Conflicting routes, unsafe network targets and invalid integrity evidence remain errors at their respective boundaries.

createCacheScope owns memoized queries for one site/build, supports clear(), and evicts rejected promises. Press disables query reuse in development and clears scopes on HMR disposal. createLinkMetadata provides matching offline card/icon readers with explicit site paths and a clearable lifetime; development readers observe refreshed disk metadata without restarting the server.

Resource and taxonomy services

The Astro integration exports virtual:site-engine/media: its resolvePageMedia uses a source-file index built from the configured pages directory. Vite still owns asset URLs and small-file inlining. Relative parent paths and query/hash suffixes are supported; missing resources retain their value with a source warning.

taxonomy owns slug/label resolution and alias indexes. Themes retain templates, page titles and layout choices. Undefined terms remain usable; ambiguous aliases remain errors because silently picking one would change the destination.

Use createLinkMetadata for site-scoped rendering and the resolver factories for explicit refresh. The previous module-level resolveLinkCard / resolveLinkIcon singletons have been removed; no process-global reader state is created on import.