[Fiware-miwi] xflow + WebCL

[Philipp Slusallek]

[I am CCing to the MiWi list, as this may be interesting also in a wider
architectural context.]

Hi Jouni,

Welcome in the team! Integration of optimal HW support is a very
important aspect of WP13, so your looking into this is very much
appreciated.

Here are the possible options that we see for implementing support for
WebCL:

1. You should be able to implement individual kernels directly in Xflow,
similar to the already existing support for glsl computations in Xflow
nodes. Thanks to the modular design of Xflow this should be a a rather
small change to Xflow and would already be a great first step. Felix (in
CC) can point you to the right part of Xflow and get you a head start
for implementing it.

2. Currently Xflow assumes that the results of Xflow nodes are available
in JS after each node. This is obviously inefficient, if the next node
is also implemented on the GPU as data would be up and downloaded after
each node. There is no support for this in Xflow yet but it should not
be very hard to implement this in the Xflow core. Again Felix would be
the right person to talk to about that.

3. As yet another optimization one could even merge the code from two
(or more) successive Xflow kernels into one and execute it as a single
kernel. We already have implemented this (text-based) merging of code
for the composition of vertex shaders. It would be an option to extend
this scheme to also apply to WebCL code but it could be a bit more
complicated here and programmers would have to follow a certain
programming style and format. It might not be necessary to do this as we
might have Option 4 available in time before you would get to this.

4. Kristian is currently developing a very nice framework for specifying
shaders in JS ("shader.js"). It allows to write completely generic
shaders in JS and uses a small compiler framework (also in JS) to
cross-compile this code to different concrete shading languages. We are
developing this to be able to have portable and platform neutral shaders
that can be used with any rendering system. We are targeting
feed-forward renderers (glsl), deferred renderer (glsl, OpenCL), and
real-time ray-tracing renderers (Intel's Embree, for a start). The
language features are compatible with those from the OpenSL (Sony) and
MDL (Nvidia), they are compatible with (real-time) global illumination
algorithms, and should work everywhere. So we should be able to target
also high-end rendering with a single shader/material library. We have
formed a small initial small consortium with the German industry to
promote this idea.

This last option is very general and uses a "real" compiler (in JS).
With some additional effort it should be possible to extend this
compiler to also support computational kernels and generate WebCL (or
any other: CUDA, RiverTrail, C/C++ with intrinsics) code. Since the
compiler "understands" the code, it should ideally be able to fully
merge kernels without any formatting conventions that the programmer has
to follow. Because we can do all sort of optimizations at the high level
(in addition to the low-level optimizations that are still done by the
glsl/WebCL/etc backend compiler) this will be the most performant and
most general option.

However, this is still work in progress and we plan to focus exclusively
on shaders for now (upcoming paper deadline). Kristian is the contact
person for this work.

In terms of schedule, I suggest that you look at the options (1) and (2)
first as low-hanging fruits that will give us most of the performance
optimizations already. You can then still look at (3) or we might skip
this and go straight to (4) depending on how far we are ready by that time.

Feel free to contact Felix and Kristian as needed so they can point you
in the right direction. We have lots of use cases for any speedup that
we can achieve and so are looking forward to your work!


Hope this helps,

	Philipp

Am 04.09.2013 11:46, schrieb Jouni Mietola:
> Adding our CTO, Jarkko Vatjus-Anttila as cc recipient.
> 
> 
> On Wed, Sep 4, 2013 at 11:43 AM, Jouni Mietola
> <jouni.mietola at cyberlightning.com
> <mailto:jouni.mietola at cyberlightning.com>> wrote:
> 
>     Prof. Dr. Philipp Slusallek,
> 
> 
>     I am a software engineer from Cyberlightning Ltd. We are currently
>     working on EU's FI-WARE project and we are trying to turn the xflow
>     modules into accelerated ones. Do you have recommendations how to
>     approach this issue and are you planning to use webCL in near
>     future? I have found out that you are using rivertrail for parallel
>     computing. We are looking forward to use webCL.
> 
>     Currently we are planning to use Nokia's webCL prototype for firefox
>     which seems to be the starting point for the webCL's future
>     development.
> 
>     About the ongoing FI-WARE project:
>     http://forge.fi-ware.eu/plugins/mediawiki/wiki/fiware/index.php/FIWARE.Epic.AdvUI.AdvWebUI.DataflowProcessing
> 
>     WebCL Working Draft:
>     https://cvs.khronos.org/svn/repos/registry/trunk/public/webcl/spec/latest/index.html
> 
>     Nokia WebCL Prototype:
>     http://webcl.nokiaresearch.com/
> 
>     Respectfully,
>     Jouni Mietola
>     Software Engineer
>     Cyberlightning Ltd.
> 
> 
> 
> 
> 
> 
> 
> 


-- 

-------------------------------------------------------------------------
Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI) GmbH
Trippstadter Strasse 122, D-67663 Kaiserslautern

Geschäftsführung:
  Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
  Dr. Walter Olthoff
Vorsitzender des Aufsichtsrats:
  Prof. Dr. h.c. Hans A. Aukes

Sitz der Gesellschaft: Kaiserslautern (HRB 2313)
USt-Id.Nr.: DE 148646973, Steuernummer:  19/673/0060/3
---------------------------------------------------------------------------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: slusallek.vcf
Type: text/x-vcard
Size: 441 bytes
Desc: not available
URL: <https://lists.fiware.org/private/fiware-miwi/attachments/20130905/7468950f/attachment.vcf>