[Fiware-miwi] xflow + WebCL

[Jarkko Vatjus-Anttila]

Felix,

I am just thinking aloud here, and may be referencing to points which you
already marked down in you email, but I still want to raise two questions
which came up while browsing the code.

1) In order to be fully parallelised, Xflow needs to aware the submitted
work packages may be executed in some other context than the main thread
javascript is running. Typically the case would be that the computation is
executed in the GPU context. Hence, when executing a flow of nodes, the
main javascript must not block, but instead do other tasks while waiting
for the results to arrive (which in some cases might take a while). I call
this a concurrency problem. It seems to be that the current implementation
does not conceptually support this kind of execution, and hence would
require rework to the Xflow core.

2) The other problem I am calling a parallelism problem. If we have
primitive nodes implemented in Xflow, doing primitive operations, it is
likely that we will build chains of these operators to gain something more
valuable. If all of the chained primitives are computed at the GPU (or some
other remote context for that matter), it would require that none of the
intermediate results of the computation need to be transferred between the
GPU and main thread. Achieving this it would require that WebCL nodes can
pass contextual information to the next node (i.e. shared GPU textures) to
avoid unneeded data copying. This is also a feature which seems
non-existent to me.

Now raising these two problems, might fundamentally be exactly the ones
what you already described, Felix, but like I said: I wanted to raise this
up a little bit. What is your take in general to this?

- jarkko


On Tue, Sep 10, 2013 at 11:53 AM, Felix Klein <lachsen at cg.uni-saarland.de>wrote:

> Hi Jouni,
>
> sorry for answering so late. I was pretty busy the recent days and
> unfortunately won't have much time to help you with this topic at the
> moment.
>
> About the integration of parallel computing into Xflow:
>
> *What Xflow can do at the moment:*
> For the xml3d.js implementation, Xflow currently still runs only on the
> CPU using JavaScript. However, we're almost done with the integration into
> the vertex shader stage with GLSL. However, any other GLSL computation is
> not supported at the moment.
>
> *Isolated WebCL/GLSL operators:*
> The first point that Kristian/Philipp mentioned is relatively easy to
> implement, but has limited efficiency (unless you have large, complex
> operators).
> The idea here is, that you simply create an Xflow operator that has input
> and output CPU buffers (e.g. TypedArrays), converts the input data to the
> respective computation platform, performs the computation, and finally
> converts the data back.
>
> There isn't really much to explain here except to pointing you to a custom
> Xflow operator.
> You can see one here:
> http://xml3d.github.io/xml3d-examples/examples/xflowWave/xflow-wave.xhtml
> More precisely, here the custom operator file:
> http://xml3d.github.io/xml3d-examples/examples/xflowWave/myxflow.js
> And here some other operators used for simple image processing on the GPU:
> http://xml3d.github.io/xml3d-examples/script/xflip-operators.js
>
> Unfortunately, we're currently missing a proper documentation on how Xflow
> operators are implemented. I hope you can understand it looking at the code.
> If something is unclear, just contact me with specific questions.
> If you want to integrate WebCL or GLSL rendering as described previously,
> you simply have to do all the stuff necessary in the evaluate() function of
> an Xflow operator.
> This includes converting the buffers back and forth.
>
> *Communicating WebCL/GLSL buffers:*
> The idea here is that Xflow supports the transfer GLSL or WebCL specific
> buffers between hardware.
>
> After thinking about this issue in more detail, I realized that it's not
> quite as simple as I first thought.
>
> The following aspects need to be implemented to support this:
> 1. The Xflow system should implement the conversion of buffers in its
> infrastructure, not inside the operators
> 2. Operators must request which kind of buffer they need
> 3. The buffers data structure (i.e. Xflow.BufferEntry) needs to store
> values for each platform (WebCL, WebGL)
> 4. Computation in the Xflow graph need to marked for each platform
> separately.
> 5. In order to have efficient GLSL computation connected to the renderer
> we need to run operators and store buffers per *context (e.g. per XML3D
> scene)*
> *
> *
> The whole thing requires several changes in the Xflow interfaces. Thus, it
> is not quickly explained what kind of code needs to be changed at what
> point...
> I don't really think it's a huge feature, but it must be carefully
> implemented. If you want to take over this implementation, we should at
> least communicate changes in the interface together.
>
> Thus, doing this second step is a lot harder then the first. Especially
> since I'm currently quite busy.
>
> I suggest we start with the first idea and then discuss the second point
> in more detail, hopefully when I have more time.
>
> Bye
>
> Felix
>
>
>
>
> On Mon, Sep 9, 2013 at 2:53 PM, Jouni Mietola <
> jouni.mietola at cyberlightning.com> wrote:
>
>> Thank you for the comprehensive info. We have now forked xml3d repository
>> and going to work under webcl branch if you want to follow us here:
>> https://github.com/Cyberlightning/xml3d.js
>>
>> Felix if you are available and have time it would be great help to hear
>> more about the points (1) and (2).
>>
>> Respectfully,
>> Jouni Mietola
>> Software Engineer
>> Cyberlightning Ltd.
>>
>>
>>
>> On Thu, Sep 5, 2013 at 8:37 AM, Philipp Slusallek <
>> Philipp.Slusallek at dfki.de> wrote:
>>
>>> [I am CCing to the MiWi list, as this may be interesting also in a wider
>>> architectural context.]
>>>
>>> Hi Jouni,
>>>
>>> Welcome in the team! Integration of optimal HW support is a very
>>> important aspect of WP13, so your looking into this is very much
>>> appreciated.
>>>
>>> Here are the possible options that we see for implementing support for
>>> WebCL:
>>>
>>> 1. You should be able to implement individual kernels directly in Xflow,
>>> similar to the already existing support for glsl computations in Xflow
>>> nodes. Thanks to the modular design of Xflow this should be a a rather
>>> small change to Xflow and would already be a great first step. Felix (in
>>> CC) can point you to the right part of Xflow and get you a head start
>>> for implementing it.
>>>
>>> 2. Currently Xflow assumes that the results of Xflow nodes are available
>>> in JS after each node. This is obviously inefficient, if the next node
>>> is also implemented on the GPU as data would be up and downloaded after
>>> each node. There is no support for this in Xflow yet but it should not
>>> be very hard to implement this in the Xflow core. Again Felix would be
>>> the right person to talk to about that.
>>>
>>> 3. As yet another optimization one could even merge the code from two
>>> (or more) successive Xflow kernels into one and execute it as a single
>>> kernel. We already have implemented this (text-based) merging of code
>>> for the composition of vertex shaders. It would be an option to extend
>>> this scheme to also apply to WebCL code but it could be a bit more
>>> complicated here and programmers would have to follow a certain
>>> programming style and format. It might not be necessary to do this as we
>>> might have Option 4 available in time before you would get to this.
>>>
>>> 4. Kristian is currently developing a very nice framework for specifying
>>> shaders in JS ("shader.js"). It allows to write completely generic
>>> shaders in JS and uses a small compiler framework (also in JS) to
>>> cross-compile this code to different concrete shading languages. We are
>>> developing this to be able to have portable and platform neutral shaders
>>> that can be used with any rendering system. We are targeting
>>> feed-forward renderers (glsl), deferred renderer (glsl, OpenCL), and
>>> real-time ray-tracing renderers (Intel's Embree, for a start). The
>>> language features are compatible with those from the OpenSL (Sony) and
>>> MDL (Nvidia), they are compatible with (real-time) global illumination
>>> algorithms, and should work everywhere. So we should be able to target
>>> also high-end rendering with a single shader/material library. We have
>>> formed a small initial small consortium with the German industry to
>>> promote this idea.
>>>
>>> This last option is very general and uses a "real" compiler (in JS).
>>> With some additional effort it should be possible to extend this
>>> compiler to also support computational kernels and generate WebCL (or
>>> any other: CUDA, RiverTrail, C/C++ with intrinsics) code. Since the
>>> compiler "understands" the code, it should ideally be able to fully
>>> merge kernels without any formatting conventions that the programmer has
>>> to follow. Because we can do all sort of optimizations at the high level
>>> (in addition to the low-level optimizations that are still done by the
>>> glsl/WebCL/etc backend compiler) this will be the most performant and
>>> most general option.
>>>
>>> However, this is still work in progress and we plan to focus exclusively
>>> on shaders for now (upcoming paper deadline). Kristian is the contact
>>> person for this work.
>>>
>>> In terms of schedule, I suggest that you look at the options (1) and (2)
>>> first as low-hanging fruits that will give us most of the performance
>>> optimizations already. You can then still look at (3) or we might skip
>>> this and go straight to (4) depending on how far we are ready by that
>>> time.
>>>
>>> Feel free to contact Felix and Kristian as needed so they can point you
>>> in the right direction. We have lots of use cases for any speedup that
>>> we can achieve and so are looking forward to your work!
>>>
>>>
>>> Hope this helps,
>>>
>>>         Philipp
>>>
>>> Am 04.09.2013 11:46, schrieb Jouni Mietola:
>>> > Adding our CTO, Jarkko Vatjus-Anttila as cc recipient.
>>> >
>>> >
>>> > On Wed, Sep 4, 2013 at 11:43 AM, Jouni Mietola
>>> > <jouni.mietola at cyberlightning.com
>>> > <mailto:jouni.mietola at cyberlightning.com>> wrote:
>>> >
>>> >     Prof. Dr. Philipp Slusallek,
>>> >
>>> >
>>> >     I am a software engineer from Cyberlightning Ltd. We are currently
>>> >     working on EU's FI-WARE project and we are trying to turn the xflow
>>> >     modules into accelerated ones. Do you have recommendations how to
>>> >     approach this issue and are you planning to use webCL in near
>>> >     future? I have found out that you are using rivertrail for parallel
>>> >     computing. We are looking forward to use webCL.
>>> >
>>> >     Currently we are planning to use Nokia's webCL prototype for
>>> firefox
>>> >     which seems to be the starting point for the webCL's future
>>> >     development.
>>> >
>>> >     About the ongoing FI-WARE project:
>>> >
>>> http://forge.fi-ware.eu/plugins/mediawiki/wiki/fiware/index.php/FIWARE.Epic.AdvUI.AdvWebUI.DataflowProcessing
>>> >
>>> >     WebCL Working Draft:
>>> >
>>> https://cvs.khronos.org/svn/repos/registry/trunk/public/webcl/spec/latest/index.html
>>> >
>>> >     Nokia WebCL Prototype:
>>> >     http://webcl.nokiaresearch.com/
>>> >
>>> >     Respectfully,
>>> >     Jouni Mietola
>>> >     Software Engineer
>>> >     Cyberlightning Ltd.
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>>
>>>
>>> --
>>>
>>> -------------------------------------------------------------------------
>>> Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI) GmbH
>>> Trippstadter Strasse 122, D-67663 Kaiserslautern
>>>
>>> Geschäftsführung:
>>>   Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
>>>   Dr. Walter Olthoff
>>> Vorsitzender des Aufsichtsrats:
>>>   Prof. Dr. h.c. Hans A. Aukes
>>>
>>> Sitz der Gesellschaft: Kaiserslautern (HRB 2313)
>>> USt-Id.Nr.: DE 148646973, Steuernummer:  19/673/0060/3
>>>
>>> ---------------------------------------------------------------------------
>>>
>>
>>
>


-- 
Jarkko Vatjus-Anttila
VP, Technology
Cyberlightning Ltd.

mobile. +358 405245142
email. jarkko at cyberlightning.com

Enrich Your Presentations! New CyberSlide 2.0 released on February 27th.
Get your free evaluation version and buy it now! www.cybersli.de

www.cyberlightning.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/fiware-miwi/attachments/20130915/5677a2e2/attachment.html>