ENG-1983 Define intermediate schemas for node, relations and schemas

[maparent]

https://linear.app/discourse-graphs/issue/ENG-1983/define-intermediate-schemas-for-node-relations-and-schemas

[linear-code]

ENG-1983

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
discourse-graph	Ready	Preview, Comment	Jul 6, 2026 7:26pm

[supabase]

This pull request has been ignored for the connected project zytfjzqyijgagqxrzbmz because there are no changes detected in packages/database/supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[mdroidian]

@maparent
Let's only export types that are being used elsewhere. This adds positive signal and really helps navigate and grok the file.

When we export types that are not used, it adds a false positive and makes it harder to understand intent.

[sid597]

Some questions and suggestions, I read CrossApp... so many time that now I am confused what it even means lol

[sid597]

snake case -> camelCase

What do you think about naming, like appending source vs direct e.g

• sourceLocalId, vs localId
• space.sourceApp vs space.app

[maparent]

I have tended to remove source here, because everything was named sourceSomething, and it did not seem to add specific information (as you were remarking about CrossApp!) Maybe I missed something? Fully negotiable.

[mdroidian]

+1 to camelCase for TypeScript-facing contract fields

[maparent]

Agreed. Did the rename in the type and forgot to update the example.

[maparent]

As for source: With the same reasoning as account, happy to reintroduce it where it fits database names. It was quite broad in the original file, but that argument does not hold anymore. Would you prefer that?

[sid597]

Is this for future? Can space be absent in Obsidian <-> roam sync?

[maparent]

This is for the case where we have a reference to a node in another space. It currently happens in Obsidian; the source or destination of a relation may be an imported node, in which case we give the reference to the original node (imported nodes are not materialized again.)
But yes, it can be absent and inferred from context in most cases.

[mdroidian]

I'm assuming that localId/space can be used for more use cases? If not, then having it in a union this high up will just complicate things by making the downstream types just more complex for a single use case.

[mdroidian]

When would author use Ref (specifically LocalRef)?

[mdroidian]

@maparent

[maparent]

This allows a minimal reference to the author, i.e. enough for a reference, not enough for an upsert. I did not check how we currently include authors in the sync, but I think it's closer to Ref usage than to an InlineAccount. Reusing the Ref type is a bit of overloading here, maybe we should define a separate AccountRef. But it would end up having exactly the same parameters.

[mdroidian]

I’m still unclear on when author would actually use Ref, specifically LocalRef. Could you give a real world example?

[maparent]

In convertRoamNodeToFullContent, we use author_local_id which would be the equivalent of a LocalRef.

[devin-ai-integration]

Devin Review found 1 potential issue.

[devin-ai-integration]

🚩 Breaking change to exported contract type

This PR completely replaces the shape of CrossAppNode — from a flat structure with fields like sourceApp, sourceSpaceId, sourceNodeId, sourceNodeRid, sourceModifiedAt, and content.full.format to a deeply nested composable structure using LocalRef, BaseAttributes, Ref, etc. While the grep confirms no other files currently import from this module besides the two example files, any external consumers (other repos, deployed services) that depend on @repo/database/crossAppNodeContract will break at both the import path level (renamed to crossAppContracts) and the type shape level. This is worth noting in release notes or migration documentation.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[mdroidian]

@maparent could you clarify the intended scope of this PR relative to ENG-1983 and its child tickets? This is fairly urgent because it is currently blocking @sid597.

The PR is attached to ENG-1983, but the changes seem to overlap with ENG-1984, ENG-1987, ENG-1863, and ENG-1864 as well. That may be the right approach, but as discussed in the ticket and in our 1:1, it would help to make explicit which tickets this PR is meant to close and which remain follow-ups. I could not find that clarification.

Could you also add a short walkthrough, ideally a video, explaining the contract choices and intended usage? There are several decisions that are hard to evaluate from the patch alone, for example: Inline vs Standalone, multiple unused exports, and several types/fields marked as future.

Re: future

As part of our commitment to "Stop moving forward on technical proposals before the user need is clear", could you clarify the user need for any future types or fields, ideally tied to existing in-scope projects or tickets? If they are speculative, let's remove them and focus on the immediate need for this PR.

Right now I cannot tell which exported types are intended as stable API versus future scaffolding. My preference is to only export types that have a concrete producer/consumer or example in this PR, and keep future-facing shapes internal until the usage is clear.

Ideally, let’s focus this PR on the immediate problem that needs to be solved, and defer anything else that is not required for that scope.

[mdroidian]

What is the purpose of this? Where will it be used?

[maparent]

This is a utility type used in eng-1985.

[mdroidian]

Great. Then lets remove it from this PR and add to the PR for ENG-1985. 👍

[mdroidian]

What is "Inline" referring to?

[mdroidian]

And where is this intended to be used?

[maparent]

The db upsert function has always allowed to put some objects inline inside related objects instead of passing a reference to the inline object; here, we can put documents in the content objects. The current usage is mostly to auto-generate the document, providing neither reference nor inline object, but this is only because our nodes are all pages. It will be false when we upload candidate nodes, for example.
When we provide an inline object, some shared attributes can be specified in the enclosing object, and hence there are less required attributes for the inline object than if the same object is upserted standalone. Hence the Inline/Standalone distinction.

[mdroidian]

If the only current use cases auto-generate documents, then let's remove this and add it back when it is required for use. Adding it now adds confusion for the dev who is trying to do the action.

When we have defined user need, then we add the scaffolding, but not before.

[maparent]

I will comment it out.

[mdroidian]

I'd prefer to remove it, as the example provided (candidate nodes) is not yet in our immediate timeline.

[maparent]

On second thought: Document has a useful semantic, and should be in the contract. Removing it because we accidentally did not use it yet in the way we wrote the function so far actually makes some of the shared contract harder to understand. Actually, I should not comment it out at all, but add a comment "Not used in the current code paths yet" or something to that effect to direct attention.

[mdroidian]

I think this is the distinction I’m trying to keep clear for ENG-1983: this PR should define the client-side cross-app contract for nodes, relations, and schemas. It should describe what Roam/Obsidian produce and consume before conversion, not the shape the database upsert API happens to accept.

From that perspective, I’m not convinced the client contract should expose `InlineCrossAppDocument` or an inline/standalone distinction. Whether a document is passed inline to `upsert_content` or upserted separately feels like converter/persistence strategy, which doesn't belong in ENG-1983. A client producing a node/relation/schema payload should not need to know which database write shape the converter chooses.

If `document` itself is meaningful in the cross-app payload, let’s define the minimal client-facing document shape and show where it appears in a concrete node/content example. But I’d avoid exporting `Inline...` / `Standalone...` variants in this PR unless a current client-side producer/consumer needs to choose between them.

A comment saying “not used in current code paths yet” does not really solve the concern for me; it still makes future scaffolding part of the public contract.

[mdroidian]

What is a standalone document? And where is it intended to be used?

If it's not used now, why is it included in this PR?

[maparent]

Standalone: See above. Here, I'm using "future" in a very restricted sense: Not yet used in ENG-1985. This is the type you would use if you were to call upsert_document on its own. It is not speculative in that way, and I think that having both the inline and standalone type clarifies the inline usage.

[mdroidian]

What is the difference between inline vs standalone?

Where is standalone going to be used? What user need is it solving?

[maparent]

It solves the user need of understanding the full type.

[mdroidian]

I’m still not following what “the full type” means here. Is that the full database-backed content shape, the full cross-app payload, or something else?

Assuming the user here is the developer, when do they need to understand or use this full type directly? For example, is this needed when calling upsert_content, writing converters, validating a received payload, or something else?

If the only current use case is internal type composition, I think we should avoid exposing it as part of the public contract until there is a concrete producer/consumer that needs it.

[maparent]

It is used when calling upsert_content, yes.
I think that having both inline and standalone type helps understand how to use the inline type.

[mdroidian]

I think we need to realign on what the goal of ENG-1983 was. And that was elaborating on the crossAppNode example that Sid provided that gave a shape for client side code to adhere to. In that sense, the distinction between Inline and Standalone should not be something that the client side shape should need to be concerned about. If Standalone provides some value at a future point, let's introduce it at that future point where the need is. But debating it in the comments of a PR is not the place for this.

To help move forward, let's defer any code that is not solving a user need immediately related to what problem the ticket is meant to solve (in this case, namely: "We will propose specific platform-neutral schemas, following the example of CrossAppNode, but making it hew a bit closer to the database format.").

In this case I see that InlineCrossAppContent is used in CrossAppNode and StandaloneCrossAppContentis not used anywhere meaningful yet.

So let's please remove StandaloneCrossAppContent so that we can move forward with this PR with the minimal requirements, and plan to discuss StandaloneCrossAppContent at our next dev or 1:1 meeting.

[maparent]

The problem with that approach is that we're in the middle of a code transition, where we're currently using upsert_content, which would require StandaloneCrossAppContent after refactoring our sync conversions to use the CrossApp contract.
We want to transition to mostly using upsert_concepts with inline content. So both are needed across the transition. This is not a far future speculative change, it's on my plate now (ENG-1823/1824).
Even as I complete those two tasks, I expect it will make sense to still use upsert_content when making changes to content that do not affect the concepts, such as in the ENG-1852 PR I just reviewed for Sid.
I promise if I find out I can eliminate all uses of upsert_content, I'll add a comment to this file that StandaloneCrossAppContent is not used. But that may change due to future optimizations, and I am -10 on removing it altogether.
(Edit: Grammar)

[maparent]

I'll put it another way: You want to minimize the cognitive surface. I understand that. But the question is: Is this something the user of these structures have to know about to use them? And I think in this case the answer is a resounding YES.

[mdroidian]

I understand we disagree. To move forward, please remove it so we push forward with this PR and can continue the conversation elsewhere about where it should be added, knowing that this will be added shortly if need be.

[maparent]

Apologies I forgot to add a video here. I'm not working from home now, and it's not easy for me to do so until later today. I hope we can resolve the main issues asynchronously.

[maparent]

As for tickets: This currently covers all sub-tasks of ENG-1983.

[maparent]

Decision: Will be split in sub-tasks.