feat(plot): addjgd graphics device integration

[grantmcdermott]

Closes #1679

Embeds the jgd VS Code extension code directly into vscode-R. Users only need the jgd R package installed on their system---and there's a graceful fallback otherwise---with no separate extension required.

Details

Adds a new r.plot.backend enum setting (auto | standard | httpgd | jgd). Default is auto, which tries the best available backend at runtime:

1. If r.plot.useHttpgd is true → httpgd (ensures backwards compatability)
2. Otherwise R-side hooks follow this logic: jgd (if installed) → httpgd (if installed) → standard

This means users who install.packages("jgd") get interactive plots with zero configuration. (Note: I didn't want to presume that the jgd R package becomes a hard dependency, but am happy to adjust if others feel differently.)

Whenjgd is active:

• Extension starts a Unix socket / named pipe server on activation
• JGD_SOCKET and SESS_PLOT_BACKEND env vars are passed to R terminals
• R's sess::register_hooks() sets jgd::jgd() as the default device
• Plots stream as JSONL frames over the socket and render via Canvas2D in a dedicated webview panel
• Plot history, navigation (prev/next/first/last), delete, resize, and PNG/SVG export all work

Main files

I've added three new files to src/plotViewer/ (ported from from the currently standalone jgd VSC extension):

• jgdPlotHistory.ts — multi-session plot history with eviction and resize-after-delete protection
• jgdSocketServer.ts — socket server managing concurrent R sessions and JSONL routing
• jgdViewer.ts — JgdManager + JgdViewer (Canvas2D webview with font metrics pipeline)

UX

JGD-specific settings:

• r.plot.jgd.historyLimit
• r.plot.jgd.exportWidth
• r.plot.jgd.exportHeight
• r.plot.jgd.exportDpi.

R-side changes: sess::connect() and sess::register_hooks() gain a use_jgd parameter. The hook priority is jgd > httpgd > standard, gated on package availability and JGD_SOCKET being set.

Backward compatibility: r.plot.useHttpgd is deprecated but still respected. When r.plot.backend is auto (default) and r.plot.useHttpgd is true, httpgd is forced. Existing Docker images and devcontainer configs will continue to work without changes.

Screenshots

[randy3k]

I haven't had a chance to use jgd, will let @renkun-ken review it. Anyway, great work!

[benz0li]

Adds a new r.plot.backend enum setting (auto | standard | httpgd | jgd). Default is auto, which tries the best available backend at runtime:

👍

1. If r.plot.useHttpgd is true → httpgd (ensures backwards compatability)
2. Otherwise R-side hooks try jgd (if installed) → httpgd (if installed) → standard

The r.plot.backend enum setting (alone) is sufficient.
👉 I simply set "r.plot.backend": "httpgd" if I want to continue using httpgd.

IMHO the handling of setting r.plot.useHttpgd can be removed.

[benz0li]

No handling = no tinkering (touching, deleting, etc.) of setting "r.plot.useHttpgd: true" in ~/.local/share/code-server/User/settings.json

If r.plot.useHttpgd is true → httpgd (ensures backwards compatability)

This actually breaks forward compatibility.

---

What I intend to do: My default ~/.local/share/code-server/User/settings.json for the R extension will be

{
  "r.bracketedPaste": true,
  "r.plot.useHttpgd": true,
  "r.plot.backend": "auto",
  "r.rterm.linux": "/usr/local/bin/radian",
  "r.rterm.option": [
    "--no-save",
    "--no-restore"
  ],
  "r.workspaceViewer.showObjectSize": true,
}

This will lead to the following behaviour:

• Newer versions of the R extension will interpret r.plot.backend (only) and use jgd (if installed).
  • If jgd is not installed: Fallback to httpgd (if installed); otherwise fallback to standard.
• Older versions (≤ 2.8.8) of the R extension will interpret "r.plot.useHttpgd": true (only) and use httpgd.

[benz0li]

@grantmcdermott I am just realising that my detailed comments at #1679 have caused more confusion than clarity.

Sorry about that.

[eitsupi]

I haven't looked at the details, but based on the description, I think it would be better to do it this way.

• Delete old setting r.plot.useHttpgd
• In auto mode, the priority is set to httpgd, then jgd.
  This means that in older environments where httpgd is installed, httpgd will continue to be used automatically.

I'm also wondering if we could remove the svglite backend for simplification.
At the very least, I don't think it should be a hard dependency of the sess package.
I believe jgd is lighter than svglite.

[grantmcdermott]

Thanks for the quick comments. I don't want to respond to everything until @renkun-ken has had a chance to review. But just quickly:

• RE: removing the r.plot.useHttpgd setting. I think we should follow the follow the mode (exemplified by data.table, etc. ) of soft deprecating functions for at least one release cycle, before fully removing them. Also, I think that this solution is more elegant i.t.o. of fallback than switching straight (only) to the auto enum setting. Remember: we're active and up to date users of vscode-R. I think most normies would be caught out if their settings suddenly stopped working altogether.
• RE: auto default. I personally think jgd makes the most sense as the lighter (and now native) alternative, but will defer to the maintainers.
• Similarly, yes, jgd is definitely a lighter dep than svglite (incl. no recursive sysdeps), but I'm not sure if they're a like-for-like swap in this context.

[benz0li]

I think most normies would be caught out if their settings suddenly stopped working altogether.

If it is implemented as @eitsupi suggests, IMHO everything will continue to work as usual.

[eitsupi]

My understanding is that only httpgd offers features like thumbnail display for past plots, so it makes sense to prioritize httpgd over jgd in environments where httpgd is installed.

Another point that caught my attention is that the tests don't seem to have been ported from the jgd repository.

[renkun-ken]

Looks user-created R session does not respect r.plot.backend setting as is not reflected in the attach script.

[grantmcdermott]

Looks user-created R session does not respect r.plot.backend setting as is not reflected in the attach script.

Thanks. Should be fixed now.

Another point that caught my attention is that the tests don't seem to have been ported from the jgd repository.

I think these only touch the Deno server, right? Porting them to VS Code is out-of-scope IMO.

[eitsupi]

I think these only touch the Deno server, right? Porting them to VS Code is out-of-scope IMO.

I meant these files.
https://github.com/grantmcdermott/jgd/blob/b556d1cc0bc94a83628045e12ea5e71ec8342480/vscode-ext/src/plot-history.test.ts
https://github.com/grantmcdermott/jgd/blob/b556d1cc0bc94a83628045e12ea5e71ec8342480/vscode-ext/src/socket-server.test.ts

[grantmcdermott]

I think these only touch the Deno server, right? Porting them to VS Code is out-of-scope IMO.

I meant these files. https://github.com/grantmcdermott/jgd/blob/b556d1cc0bc94a83628045e12ea5e71ec8342480/vscode-ext/src/plot-history.test.ts https://github.com/grantmcdermott/jgd/blob/b556d1cc0bc94a83628045e12ea5e71ec8342480/vscode-ext/src/socket-server.test.ts

A good catch. I'll port those (second one might be a bit trickier given we changed some things from the original extension, but it should work...)

[renkun-ken]

if r.plot.backend = "auto":

From the perspective of current capabilities:

httpgd > jgd > svglite > png

From the perspective of hard dependencies, footprints, things to be loaded into the user session:

png > jgd > svglite > httpgd

From a balanced view of both perspectives as well as native extension support, jgd is a good choice I guess?
It largely depends how we weight each perspective.

From user-side, if loading a ton of hard dependencies is not a big deal, choosing the currently most capable installed backend might make more sense.

From the status quo, I guess a good proportion of existing users use httpgd as a suggested approach to view graphics, with their r.plot.useHttpgd being enabled, which will force using httpgd even if r.plot.backend = "auto" by default as we release. Even if we choose the behavior to make jgd > httpgd for "auto", if httpgd users wants to check out jgd, they have to disable r.plot.useHttpgd first and install jgd; if new users wants to check out jgd, just install it and it should work.

Considering all these, I think it is fair to make jgd > httpgd as we don't want to introduce a heavier dependency for user workflow by default.

The only thing that might be a bit drawback is that do we think jgd has reached a level of stability that could be released/set by default if installed?

[eitsupi]

In my opinion, if httpgd is removed from CRAN again (which is likely to happen in the not-too-distant future), leaving the useHttpgd setting in place will leave lasting problems.

[renkun-ken]

Sometimes when vscode is restarted, jgd::jgd() on startup in a unmanaged R session I start will usually not work:

jgd: cannot find socket path. Pass socket= to jgd() or start the rendering server.
Warning message:
In jgd::jgd() :
  jgd: could not connect to renderer. Plots will be recorded but not displayed until connection is established.

[grantmcdermott]

Sometimes when vscode is restarted, jgd::jgd() on startup in a unmanaged R session I start will usually not work:

jgd: cannot find socket path. Pass socket= to jgd() or start the rendering server.
Warning message:
In jgd::jgd() :
  jgd: could not connect to renderer. Plots will be recorded but not displayed until connection is established.

Thanks @renkun-ken. I think this should be fixed with 26f3bc5. (More details: grantmcdermott/jgd#63)

[grantmcdermott]

Coming back to this: The three CI tests were failing b/c the runner wasn't installing Suggests dependencies in the sess package (which I assume was a mistake). I've pushed a small change that should fix this in 22196fe

@renkun-ken @randy3k Can either of you trigger the workflows so that we can confirm?

P.S. The tradeoff is now that the CI runs will take slightly longer, since they will install all of the Suggests deps. It's not a huge number, but we can cut most of these if we switch away from testthat to tinytest (link). From my brief look at the sess codebase, the test surface is minimal and so this should be a low-risk change. I'm happy to confirm this locally and then, assuming everything ports, push this additional change as part of this PR if you agree?

[randy3k]

Hi Grant,

Thank you for the excellent work and sorry for the late reply. Obviously I am not very familar with the code for jgd. I have spawned multiple AI agents to review the code. Please review the following comments (and dismiss if you that they are not important / false alarm).

Anything below this line was written jointly with Gemini and Claude.

---

Draft Reply to PR #1706

---

Hi @grantmcdermott,

Thanks for this — really exciting to see JGD integrated directly. The architecture is clean, the plot history state machine is well thought out (especially the latestDeleted guard), and the test coverage is solid.

I went through the review above and agree with most of the points raised. Here's my consolidated take with some additional context and a couple of extra findings.

Should-fix

1. SVG attribute escaping

+1 to what was flagged above. svgGcStroke(), svgGcFill(), and the text/raster branches in plotToSvg() inject gc.col, gc.fill, f.family, col, and op.data into XML attributes without going through svgEsc(). The text content path correctly uses svgEsc(op.str), but the attributes are unescaped:

// Currently:
let s = ' stroke="' + gc.col + '"';
return ' fill="' + gc.fill + '"';

// Should be:
let s = ' stroke="' + svgEsc(String(gc.col)) + '"';
return ' fill="' + svgEsc(String(gc.fill)) + '"';

Same for f.family, col, and op.data (the raster href) in plotToSvg(). The values come from R's graphics device so the risk is low in practice, but it's good hygiene.

2. Large PNG export can crash the webview

In the PNG export handler:

const base64 = btoa(String.fromCharCode(...new Uint8Array(reader.result)));

The spread operator on a large Uint8Array will exceed the JS call stack at extreme export sizes (e.g., 20+ inches @ 300+ DPI). A chunked approach would fix this:

function uint8ToBase64(bytes) {
    let binary = '';
    const chunk = 8192;
    for (let i = 0; i < bytes.length; i += chunk) {
        binary += String.fromCharCode.apply(null, bytes.subarray(i, i + chunk));
    }
    return btoa(binary);
}

Or alternatively use FileReader.readAsDataURL() and strip the prefix.

3. Use a secure temp directory for the Unix socket

The random token (crypto.randomBytes(8)) already makes the path unpredictable, so this isn't a critical vulnerability. That said, using fs.mkdtempSync with 0o700 permissions would align with what we established in #1705 for the IPC pipes:

const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'jgd-'));
this.socketPath = path.join(tempDir, 'jgd.sock');

Remember to clean up the directory during stop().

4. Add a Content-Security-Policy to the webview

The webViewer already sets a CSP meta tag. Adding one here is good practice:

<meta http-equiv="Content-Security-Policy"
      content="default-src 'none'; style-src 'unsafe-inline'; script-src 'unsafe-inline'; img-src data: blob:;">

For what it's worth, httpgdViewer and standardViewer don't have one either — I'll open a separate issue to address those, so this isn't something unique to your PR.

5. Set the panel icon

One-liner for UI consistency with the other plot viewers — add after createWebviewPanel():

this.panel.iconPath = new UriIcon('graph');

Nice-to-have (can be follow-ups)

6. Changing r.plot.backend mid-session

Switching the backend setting without reloading VS Code won't start/stop the JGD server. This is the same limitation r.plot.useHttpgd has always had — we've never supported hot-swapping plot backends. I think the right move is to add "scope": "window" in package.json for r.plot.backend to signal that a reload is needed. No need to block on this.

7. Prune stale sessions from PlotHistory

The sessions Map grows on each R restart since sessions are never removed. Each entry's metadata is small and plots are already bounded by maxPlots, so this isn't urgent. A simple cap (e.g., evict empty sessions or keep only the 10 most recent) would be tidy.

8. Clean up pendingMetrics on panel disposal

onDidDispose clears metricsCache but not pendingMetrics. Resolving any in-flight promises there would avoid dangling 500ms timeouts:

this.panel.onDidDispose(() => {
    this.panel = null;
    this.metricsCache.clear();
    for (const [id, resolver] of this.pendingMetrics) {
        resolver({ type: 'metrics_response', id, width: 0, ascent: 0, descent: 0 });
    }
    this.pendingMetrics.clear();
});

---

Items 1–5 are things I'd like to see addressed in this PR. 6–8 can be follow-ups. None of these are merge-blockers — happy to approve once you've had a chance to look at them. Let me know if anything needs clarification!

[renkun-ken]

I'm using the grantmcdermott/jgd@ae3283b and the latest PR build. Also, I usually use self-managed R terminals (arf) running in tmux windows.

Here's my operation:

1. Start arf in a tmux window
2. Click attach R in the status bar
3. Send plot(rnorm(100)) to terminal will properly trigger the jgd plot viewer.
4. Now reload window and navigate to the existing arf terminal, click attach.
5. It seems there's no way to re-show the previously closed jgd plot viewer since "R Plot: Show viewers" command no longer works.

[grantmcdermott]

This is great, thanks both. I believe that I've address both sets of feedback. Everything look good on my side. But, again, please trigger the CI when you get a chance to confirm that I haven't introduced some inadvertent bugs.

@randy3k Those were all helpful ideas and I've actioned items 1-5. I agree with your agents' assessment that 6-8 are add-ons that should only really be address later, so I left those out.

@renkun-ken It took me a bit of back and forth, but I've reproduced your bug locally and have tested a local fix that resolves the issue in c3c6c7b. Specifically, sess now detects an open jgd device on attach, and then records its current plot, reopens against the live socket, and replays the plot. Note that running Developer: Reload Window from your VS Code command palette still clears the panel... but then re-running the attach command (R: Attach External R Session (copy command)) brings the previous plot back and adding new plots also works fine. (Note I bumped sess to 3.0.1 so the attach script reinstalls the corrected package. Could you confirm on your end when you get a chance?)

P.S. I ended up going ahead with the testthat -> tinytest change in sess. I had already implemented and sanity checked this on my side, so it was easiest just to roll these other new changes on top of that.

[grantmcdermott]

One minor test was failing b/c of a sess version mismatch. The easiest solution was just to revert the numbering back to 3.0.0, so that's what I did in c38b27c. Should be a clean result now once the workflows are triggered again. Sorry for the back and forth.

[Fred-Wu]

I'm using the grantmcdermott/jgd@ae3283b and the latest PR build. Also, I usually use self-managed R terminals (arf) running in tmux windows.

Here's my operation:

1. Start `arf` in a tmux window

2. Click attach R in the status bar

3. Send `plot(rnorm(100))` to terminal will properly trigger the jgd plot viewer.

4. Now reload window and navigate to the existing arf terminal, click attach.

5. It seems there's no way to re-show the previously closed jgd plot viewer since "R Plot: Show viewers" command no longer works.

Would like to mention that for Point 5. If a session is re-attached after detach due to, say, a third party extension gets restarted, "R Plot: Show viewers" actually never worked even in the current official release. I think R plot is currently not session based. It would be better to make it be aware of which R session it attaches to.

[renkun-ken]

Thanks for the quick fix. It works now. 👍

[grantmcdermott]

Great. Are we good to merge, then (perhaps after @randy3k's AI review)?

Two closing(?) notes from my side:

1. I'm not sure when you plan to submit the sess package to CRAN. But the DESCRIPTION file still needs to be configured with a proper @Author field and maintainer email. I didn't want to presume those entry values as part of this PR, but would like to add my name as a contributor once it's ready, since I've touched up several parts of the sess codebase.
2. @Fred-Wu Good call out on making the R plot viewer session aware. Given that this is a preexisting issue and requires a potentially non-trivial fix, I'd really prefer to tackle that as a separate issue once this PR is merged.

[renkun-ken]

From my usage in the recent months, I think this PR is good enough to merge. Let's see if @randy3k has more review?

[randy3k]

let's merge it then~

[Fred-Wu]

@grantmcdermott When R is launched, plot_backend is leaked to the global environment.

[grantmcdermott]

🎉

@Fred-Wu Good catch. I'll address that in a follow-up up. Let me know if anything else crops up.

P.S. I received an email with a reply from @chmzs, saying that no plot was appearing. My first thought is you haven't updated the sess package. But the reply seems to have been deleted, so I assume it's working now?

[chmzs]

🎉

@Fred-Wu Good catch. I'll address that in a follow-up up. Let me know if anything else crops up.

P.S. I received an email with a reply from @chmzs, saying that no plot was appearing. My first thought is you haven't updated the sess package. But the reply seems to have been deleted, so I assume it's working now?

Here's a natural English translation of your message:

I applied the update right away. At first, I thought it was an issue with .Rprofile, but after commenting it out, that wasn't the case. Eventually, I pinpointed the problem to a setting in the R extension. The option shown in the screenshot was originally enabled; once I turned it off, everything worked smoothly – good!
Therefore, I still suggest updating the usage instructions (README) to make things easier for everyone.

[grantmcdermott]

Thanks @chmzs. Good to hear you resolved it. I certainly agree that we should update the documentation and was planning to edit the plotting wiki page after this PR landed.

But FWIW I am unable to reproduce your error locally. Setting r.plot.useHttpgd: true on my system leads to the expected fallback described by the setting (and discussed above): i.e, it invokes httpgd as the backend instead of jgd.

Perhaps you need to update httpgd?

EDIT: I do think this example probably motivates a warning message (i.e., if the user tries to set httpgd as the backend but it isn't installed). Let me tackle that as a follow up item.