[Fiware-miwi] EntSys Sync & UI integration: separate EC data & view objects

[Toni Alatalo]

Another early w.i.p. report, now about the integration of the Sync GE with the rest of the client & 3DUI.

Summary: we propose separating the data & view parts of EC implementations. Main motivation is for client side network code to be independent of WebGL so that e.g. a 2d HTML client and headless Node.js bot / service development is easy. This is achieved by packaging the scene & sync code so that also without 3d rendering the scene gets populated normally and manipulating it works. This may sound obvious but is not how any of the existing realXtend clients have been implemented so far. The design is not verified by an implementation yet, Erno is working on it, but we discussed with Lasse last week and decided to give a shot.

Longer version:

Current Tundra clients all feature EC classes that bundle both 1) the data model definition as the set of attributes and 2) the implementation of the component (with e.g. 3d graphics, physics engine or audio playback) in the same class. For example in the C++ Tundra we have the data model of a Mesh in https://github.com/realXtend/naali/blob/tundra2/src/Core/OgreRenderingModule/EC_Mesh.cpp#L39 and the code to show the mesh with Ogre3D later in the same file in EC_Mesh::SetMesh, https://github.com/realXtend/naali/blob/tundra2/src/Core/OgreRenderingModule/EC_Mesh.cpp#L250 . The equivalent in Chiru-WebClient, using Three.js the same way, is in https://github.com/Chiru/Chiru-WebClient/blob/master/src/ecmodel/EC_Mesh.js (and AFAIK WebRocket is made the same way).

This introduces a hard dependency for the 3d rendering. Initially when Tundra (then known as Naali) was made as a 3d client only codebase, against Opensimulator, it was not a problem. There was already a networking-only client implementation of the same LLUDP protocol, LibOMV, which works well for making services, bots or custom 2d clients. When we switched to using the Naali codebase as the server too and renamed it to Tundra we lost that. Support for headless servers was hacked in by adding boolean !headless checks and also with the NullRenderer plugin to Ogre. But there’s no nice small network lib for custom clients.

Now we have a chance to get it right in the Web client code. Regarding use cases, we already have an existing custom 2d client for mobile phones from before, made by a 3rd party — NIMO project’s Daniele Zanni implemented a minimal 2d html GUI for mobile phones for his Street Art Gangs city game by using websockets to connect to Tundra to send entity actions for the game app running there .. so people can run around the city and make virtual graffititi, by pressing a big button in the phone display, no view of the 3d scene is necessary. This was easy against the naive 1st Tundra websocket server made by Playsign which used plain JSON for comms. It could easily be nice to have scene sync working there to for example show locations of other players on a map, better than just e.g. sending actions with HTTP in a disconnected mode. Another usage we envision is to run services or bots as e.g. Node.js apps, on external servers which can connect to Tundra servers. This might be the good solution for what the SceneAPI Epic is calling for.

With this in mind Lasse started the work on Sync GE on a specific repo, https://github.com/realXtend/WebTundraNetworking/ . I did not think this through then and thought it would mostly be the kNet and TundraProtocol parts, to manage the connection and parse messages etc. — not the ECs. But we soon realized that for nice usage the networking module should actually have the entire scene model. So for example a Node.js bot is written with the same API as 3D client functionality: the networking takes care that the scene is there with all the data and the app code can just use it. It would be all too low level for every client to need to handle the network messages themselves. This is identical to LibOMV: it provides a readily populated scene-like object which custom client / bot code can just read & manipulate. For example all Avatars in a sim are in this dict in the client side Simulator object: http://lib.openmetaverse.org/docs/trunk/html/F_OpenMetaverse_Simulator_ObjectsAvatars.htm

We plan to implement this by making the data & view parts of ECs separate classes, somehow. Lasse already has the data parts for a set of core components, e.g. Mesh in https://github.com/realXtend/WebTundraNetworking/blob/master/src/scene/EC_Mesh.js (the focus there is to have something to populate the data from the net, to very that the messaging works). Erno is working to get some kind of view objects to show the data with WebGL — but so that the client works also without those. 

Erno has also continued with the DOM integration investigations, testing the attribute mutation observers to inform the rest of the client of changes in the data (Lasse’s code doesn’t do that part). However, to support Node.js (and other similar) non-browser usage of the networking module it seems better to have the DOM integration as an optional feature (Node.js does not have nor plan to have a native DOM implementation). This way the internal data representation can use Lasse’s attribute objects which are typed and provide serialization via themselves, in https://github.com/realXtend/WebTundraNetworking/blob/master/src/scene/Attribute.js . We’ll see how the details with the DOM integration go as the implementation proceeds.

Again, this is now basically just an idea and the purpose of this post is to give an early warning and possibility to comment. Do note that none of us involved here are fans of fancy OO designs or layers of abstractions etc (this design may or may not be what MVC is :) and we definitely want to implement this separation of concerns in most straightforward and efficient way (e.g. not with events but for example by EC data objects having the active corresponding view implementation as a member it can call directly for updates, or vice versa .. let’s see). If we come up with a good design it might be interesting to refactor the native Tundra code to that as well (to not depend on Ogre for headless runs, and perhaps make it easier to implement alternative renderers .. is probably more complex in the land of static typing though).

One worry I had was that application programming would become more complex if app code would need to know about separate data & view objects and could not just manipulate the single ECs. But I think this worry was moot as the data / model object is always the interface, and the separated view is a hidden implementation mechanism behind that. Other worry is that implementing new components becomes more complex but that’s may not be the case either — should be simple to make two classes following a pattern well established by the core components, and they can probably be in a single JS file.

Finally about repositories: current plan is to do this all in a single repo by renaming the current WebTundraNetworking -> WebTundra and add a renderer/view directory there to accompany scene and networking. Then a headless client is just a configuration which does not need the view part. We can later split those to separate repos if needed, perhaps use submodules to aggregate them as the full 3d client etc., but we figured it’s best now to collaborate in a single one as the parts are tightly integrated and in flux.

Cheers,
~Toni