Skip to content

Refactor 3D scenes to a modular object system#6085

Open
HosseyNJF wants to merge 7 commits into
zauberzeug:mainfrom
HosseyNJF:3d-scene-modular-refactor
Open

Refactor 3D scenes to a modular object system#6085
HosseyNJF wants to merge 7 commits into
zauberzeug:mainfrom
HosseyNJF:3d-scene-modular-refactor

Conversation

@HosseyNJF
Copy link
Copy Markdown
Contributor

@HosseyNJF HosseyNJF commented May 31, 2026

Motivation

The NiceGUI 3D scene class has a lot of potential, but it's severly limited by the fact that users cannot create their own custom meshes inside JS, just like they do with Vue components. With this PR, users can create any complex meshes they need in JS directly, integrate any library authored for three.js into their objects, and even go as far as integrating a pre-existing three.js project into NiceGUI by just importing the project as a singular "Object3D" and exposing methods for the Python run_method.

Relevant: #4120

Implementation

The idea is to let children of the "Object3D" class define a "component" which gets registered as a library in the dependencies module, and loaded in the importmap. Then, the scene.js file imports those libraries, and hooks into them at various points (for creation, attaching, detaching, etc). This will allow a modular system where each component's code is encapsulated in a JS file.

As a side-effect, all pre-defined objects were also refactored to use this new system.

I still haven't figured out how to make the connection from JS -> Python more clear, for example letting components define their own events and emit them.

Progress

  • The PR title is a short phrase starting with a verb like "Add ...", "Fix ...", "Update ...", "Remove ...", etc.
  • The implementation is complete. (Otherwise, open a draft PR.)
  • This PR does not address a security issue. (Security fixes must be coordinated via the security advisory process before opening a PR.)
  • Pytests have been added/updated or are not necessary.
  • Documentation has been added/updated or is not necessary.
  • No breaking changes to the public API or migration steps are described above.

@HosseyNJF
Copy link
Copy Markdown
Contributor Author

I'm still doing work on this PR, but raised it here to get feedback early before I make the final touches.

@HosseyNJF HosseyNJF marked this pull request as ready for review June 1, 2026 09:18
@evnchn
Copy link
Copy Markdown
Collaborator

evnchn commented Jun 1, 2026

@HosseyNJF can you explain why 0de5653

Because:

  • You seem to not enabled workflow runs on your fork, so I must approve workflow, and I'm kinda hesitant to do so when there's workflow changes
  • If there really is a bug with our CI it's worth a separate PR instead of having that bug "bundled with a feature PR"

@HosseyNJF
Copy link
Copy Markdown
Contributor Author

@evnchn Yeah my bad, I'll create a separate PR for that. About the issue, there is a bug that if a screen test fails, the action won't upload the screenshot of selenium.

Btw workflows are enabled on my fork, maybe it's a security toggle that just doesn't let first-time contributors run the actions on a PR?

@HosseyNJF HosseyNJF force-pushed the 3d-scene-modular-refactor branch from bbcd474 to 32354a7 Compare June 1, 2026 13:55
@evnchn
Copy link
Copy Markdown
Collaborator

evnchn commented Jun 1, 2026

@HosseyNJF You sure enabled? 0 workflow runs at https://github.com/HosseyNJF/nicegui/actions tho

@HosseyNJF
Copy link
Copy Markdown
Contributor Author

Oops I was looking at the wrong toggle :) I just enabled it now, let me try a push

@HosseyNJF HosseyNJF force-pushed the 3d-scene-modular-refactor branch from 32354a7 to f2f63d5 Compare June 1, 2026 13:58
@evnchn
Copy link
Copy Markdown
Collaborator

evnchn commented Jun 1, 2026

Well still no. I pressed approve for you on this one.

image
Some GitHub cost-cutting shenanigans (worth a read if confused)So on my end, I can say is, sometime this year, GitHub definitely did change how the workflow runs are triggered, because it used to be "run on all branches" but some time they changed to "run only on opened PRs", hence why I am forced to track all my branches with PRs at https://github.com/evnchn/nicegui

Maybe due to that shenanigan, that this PR is not in the list of branches to monitor because of that mechanism.

I think the course of action is to:

  • bring 0de5653 which is needed anyways upstream in a separate PR (do double-check since I recalled it was working in the PR that introduced it?)
    • besides the value of the code, that PR will tell you whether you're "in the list" for workflow runs to come up on my end without needing my approval since you're using your run quota.
  • consider, in parallel, to open a PR of this branch inside your fork, best against a divergent branch (main is also OK if you promise to never accidentally-merge it). That could do the trick as well.

Comment thread nicegui/dependencies.py Outdated
@HosseyNJF
Copy link
Copy Markdown
Contributor Author

I don't know if I'm doing anything wrong, but I created the PR in my own fork and over there, the workflows are running. But not here for some reason.

@evnchn
Copy link
Copy Markdown
Collaborator

evnchn commented Jun 1, 2026

Well opening fork PR used to get the trick done. I just don't know how much GitHub wants to cost-cut its workflow runs by ruling more stuff out tho.

Then the only thing left to try is to bring 0de5653 out to its own PR. If that doesn't get actions to show up working maybe time for look for pre-existing issues and file it to them, no choice.

@HosseyNJF
Copy link
Copy Markdown
Contributor Author

HosseyNJF commented Jun 1, 2026

Questions for reviewers:

  • Does the API make sense? I made some decisions that I'm not fully sure about.
    • For example, the attached and detached methods receive the scene.js module instance, if they for any reason need it. I'm not sure if anyone would ever need it though, or if it's a good idea to expose NiceGUI internals to external components. Does this break encapsulation?
    • Should it be parent_mesh or the parent class instance itself which gets passed to other components' attached method? It's currently parent_mesh. Should we allow different components to communicate this way?
  • Do these APIs seem extensible? I don't want to define an interface that proves to be a pain to change/update in the future, because it breaks users' components. What if we want to add/remove an argument from a method in the future? There are two options to add an argument in the future:
    • Check the component's method's length property, and only pass the new arguments if the method supports it
    • Scrap the current arguments. Instead, make extensive use of object deconstruction. function attached({ scene, parent_mesh }). This way, adding extra arguments in the future won't break anything (but removal still does).

@HosseyNJF
Copy link
Copy Markdown
Contributor Author

Hello @evnchn , would you mind assigning the "review" label to this PR, or is there something else I need to do before it's considered ready?

@evnchn
Copy link
Copy Markdown
Collaborator

evnchn commented Jun 4, 2026

We are slight busy and may not able to handle big feat PRs in 3.13, sorry :(

Off the top of my head, I wonder really if there are no breaking changes with this PR. Because nicegui/elements/scene/scene_objects.py is gone, after all.

If there is breaking change, then we need to discuss if it is worth it, and if it is then we can ship it in 4.0

If there is none, we'll choose a milestone when we are comfortable with handling it and do it.

@HosseyNJF
Copy link
Copy Markdown
Contributor Author

I totally understand, I just didn't want it to get lost to time :) There are absolutely no breaking changes, since all objects were exported from ui.scene anyway, and this PR just replaces those exposed classes. Importing Object3Ds directly by name (which fails now because of import path change) is not documented and the official way is through ui.scene, which remain unaffected.

@evnchn
Copy link
Copy Markdown
Collaborator

evnchn commented Jun 4, 2026

Huh interesting.

So maybe this statement, when attested to be true, would help the review of the PR which, on paper, has 49 changed files, and +1,295 -609?

Most of this PR is moving things around. A refactor, basically.

I'll send my agents on this. @HosseyNJF lmk if you think it's true on your side.

@HosseyNJF
Copy link
Copy Markdown
Contributor Author

That is true, but I'd add this too (since it's an important aspect):

This refactor also enables users of NiceGUI to be able to define their own custom components in JS. Because of that, it's important to consider API design standards between components and the scene.js module.

@evnchn
Copy link
Copy Markdown
Collaborator

evnchn commented Jun 4, 2026

Reviewed via Claude Code (Claude Opus 4.8) on Evan's behalf — MREs run locally headless against your branch fd0d591 vs main.

Hey @HosseyNJF — short version: the refactor's solid and you're right it's mostly relocation (the 17 objects/*.py + objects/*.js are scene_objects.py and the old create() switch moved out near-verbatim), so "moving things around" is fair in spirit 👍. But two things surfaced when I sent agents over it: Ring is broken on the branch, and two changes are breaking by our bar — even though the ui.scene import surface you mentioned is intact.

Why we'd still call items 2 & 3 "breaking" (we may just define it differently)

Your bar for "absolutely no breaking changes" is the documented ui.scene import surface — and that's genuinely intact, agreed. Ours also covers the runtime behavior of documented usage (e.g. scene.box().material() after page load) and the extension API this PR itself documents (subclassing Object3D). By that bar, items 2 & 3 below change the behavior of code that works today.

Three minimal repros, each run headless against your branch and main:

1. 🔴 Ring renders nothing — must-fix

ring.py is the only object that kept its old type string:

super().__init__('ring', inner_radius, outer_radius, ...)   # every other object dropped the string

Since the new Object3D.__init__(self, *args) no longer consumes a leading type, 'ring' is forwarded as RingGeometry's innerRadius. Live readback of geometry.parameters:

main this branch
innerRadius 0.5 "ring" (string)
vertex positions finite all NaN → nothing renders

The fix is to drop that leftover 'ring' argument (verified locally — readback goes back to innerRadius=0.5, no NaN). We're a bit swamped right now so I'd rather not push to your branch, but ping me if you'd like the exact one-liner.

2. 🟠 Breaking — runtime .material() silently dropped (create() went async)

create() now returns a Promise (import(name).then(...)). At runtime this documented pattern:

ui.button('add', on_click=lambda: scene.box().material('#ff0000'))

sends create then material as separate messages. On the first (uncached) use of an object type, the import hasn't resolved yet, so material() hits if (!object) return; and is lost.

main this branch
box color after click #ff0000 (red) #ffffff (white — material lost)

The initial-scene path is safe (init_objects awaits each create); it's specifically runtime-created objects — which is every custom object, i.e. the headline feature. Options: keep built-ins statically imported (sync), or buffer per-object method calls until that object's import resolves.

3. 🟠 Breaking — subclassing Object3D raises at class-definition time

__init_subclass__ now requires component=:

class MyThing(Object3D):   # the pattern every built-in used pre-PR
    ...
main this branch
class definition OK TypeError: parameter 'component' must be supplied as a class named argument

Acceptable if we agree direct Object3D subclassing was never public — but worth stating explicitly, since this PR makes subclassing the official extension path.

The ask: Ring needs fixing. For items 2 & 3 — your call: fix them, or come ready to walk me through why they're intentional and we'll settle on the right approach together. Thanks for the contribution, genuinely nice work on the modular structure 🙏

@evnchn
Copy link
Copy Markdown
Collaborator

evnchn commented Jun 4, 2026

@HosseyNJF So in short: I spot at least 3 to-do items, so maybe this will go in 3.14, at least not 3.13 I guess.

3.13's packed not because of many tickets but because of low human bandwidth so yeah weight's off for this PR then.

@evnchn evnchn added this to the Next milestone Jun 4, 2026
@HosseyNJF
Copy link
Copy Markdown
Contributor Author

Thanks for the review! Yeah that works for me.

Off the top of my head, I'd say point 3 is incorrect because the component argument has a default value of None in __init_subclass__, and it uses the parent class's component argument if the value is None. It's also validated in the scene_documentation.py file (the CoordinateSystem class which renders correctly without errors in this PR).

For the rest, I'll take a look and get back to you with fixes.

@HosseyNJF
Copy link
Copy Markdown
Contributor Author

By the way, please let me know if I can be of any help in other issues/PRs for the 3.13 milestone.

@evnchn
Copy link
Copy Markdown
Collaborator

evnchn commented Jun 4, 2026

Party's public at https://github.com/zauberzeug/nicegui/milestone/169 so feel free to look at it yourself.

I can say the HMW statement is: "how might we help Falko with finding out ones best not in 3.13, or reviewing the PRs if they are really for 3.13"

I don't have a systemized way of doing this, though. If anything you should not fully listen to me and become another Evan because Falko is a very systemized person and I'm not (don't ask how we get along, that's a long story)

@falkoschindler do you wanna guide him a bit in the meantime? Actually @HosseyNJF general "what should I do as an open source contributor" gets you a long way.


Some braindump:

  • Some PRs keep getting deferred in milestone. Probably OK to defer one more.
  • Some are change requested, but normally we'd ask our agents to get doing, so the fact that something stays change requested it is often that we need to make a design decision and that's the halt reason
  • Some might just need a push or a fresh set of eyes to be frank <- I think here's the value?

Final note: not all open source repo keep the release milestone decision making process open. So "what should I do as an open source contributor" might not get you all the way. And I also think it's intentional because otherwise we feel kinda bad for just taking your code but looping you out of all decision making process?

@falkoschindler falkoschindler added feature Type/scope: New or intentionally changed behavior in progress Status: Someone is working on it labels Jun 5, 2026
@falkoschindler
Copy link
Copy Markdown
Contributor

falkoschindler commented Jun 5, 2026

Thanks a lot for this pull request, @HosseyNJF! I like the idea of splitting ui.scene into smaller, more manageable modules. I'm looking forward to reviewing it in detail!

And thanks for offering your help! This is very appreciated! Rather than answer one-on-one, we wrote up where contributions are most valuable right now: #6094. The 3D scene is clearly an area you know well, so pre-flighting and helping unblock scene-related PRs would be a natural fit — but anything on that list genuinely helps. 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature Type/scope: New or intentionally changed behavior in progress Status: Someone is working on it

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants