Hi everyone,
as we are talking about doing the live demo with Kurento we think its interesting and useful for us using it for image processing. We have deployed the versions 5.0.33 and 5.0.4 in fiware to test the tutorials that are available in https://github.com/Kurento/kurento-tutorial-js
We have only been able to send video via Kurento, we cannot do any image processing (that we think that is a very interesting feature for robotics area), so we put here the problems we have found so Luis can help us or put us in touch with someone who could help us with them, or maybe we can talk via skype or hangout to be more efficient:
* The bower files use adapter.js with kurento-alvar branch, but it doesn't exist, so we used master branch and in the html dependencies we have changed "bower_components/adapter.js/samples/web/js/adapter.js" with "bower_components/adapter.js/adapter.js" FIXED
* The kurento-crowddetector show us an error: 'Error: Unexpected error while processing method: Factory not found {code: -32603, stack: (...), message: "Unexpected error while processing method: Factory not found"}' I supposed that is related with something that is not installed, I have researched but I didn't find anything about this.
* The kurento-faceoverlay-generator doesn't add any overlay on the video and doesn't give us any error.
* The kurento-chroma doesn't show anything, but also we don't understand how it works, it tooks the upper left corner to do something but we don't know what.
* The kurento-background give us many errors:
* Bower doesn't know where kurento-module-backgroundextractor.js is I have search it on internet and finally I have been able to extract it from a jar file I found in a maven repository. FIXED
* After fixing that I have tried the demo but it gives me the same error as in the crowdetector (Factory not found)
* It also gives me another error: 'TypeError: Cannot read property 'processSdpAnswer' of null {stack: (...), message: "Cannot read property 'processSdpAnswer' of null"}' And I don't know how to continue.
I wanted to do some pull request fixing the html, bower and js issues I have found, but first of all I want to be sure that I can make the examples work. So any help would be great.
Regards,
Iñigo
On 18/02/15 12:41, Bollano Gianmario wrote:
Hi Luis,
I have found initially of documentation from the FIWARE pages and it was quite complex to understand where to get clear code and examples. Probably the catalog page it’s not the simplest page to start with, because it provides lot – for me too many - links for Kurento server and Client at Kurento Github<https://github.com/Kurento>, at NPM<http://npmsearch.com/?q=kurento-client>, at Maven Central<http://search.maven.org/#search%7Cga%7C1%7Ckurento>, Bower<http://bower.io/search/?q=kurento-client> and Kurento FIWARE Forge Repository<https://forge.fiware.org/frs/?group_id=7#title_data-kurento> … Really hard to understand which component may be needed and where to get them.
Information at Kurento.org is definitely more helpful; without digging deep in the technical details, from the Tutorial it is clear there are three different “usage scenarios” , also represented in the picture in the introducing_kurento<http://www.kurento.org/docs/current/introducing_kurento.html> tutorial (“Kurento Api, Client and Protocol” section).
In my understanding, these 3 scenario mapped to our environment becomes:
1. Robot equipped with Chrome + NodeJS + Bower + Kurento JS Client + Application Code. This allow to have Application server as localhost in the robot and to open the browser for transmitting video in the same environment. Having everything in the Robot allows to communicate directly with a Kurento media server. The Browser endpoint connects to Kurento server.
2. Robot with Chrome only, which connect to on external Linux App server with NodeJS + Bower + Kurento JS Client + Application Code (downloaded and launched in the Robot browser).
3. As 2, but working in Java EE App server and Java Kurento client instead of JS.
I believe the scenario 3 beyond the purposes of the Robotics live demo because it requires larger development effort and Java EE knowledge, however the Robotics developers may better evaluate this issue. The scenario 1. is preferable but we have to check if the Robot has the requirements to host all the Application Server software.
The scenario 2 move the Application server outside the Robot and is less resource consuming for the Robot, but it involves further issues concerning communication between Robot and the Application server. Robotics GE adopts ROS environment which already consider a Robot Entity composed of nodes communicating in their way, same of them placed externally to the Robot (in a service Space which runs also in an external server). Moreover, our demo also use an external application server to send and display Robots commands, which interact with Context broker on one side and calls Robotics management APIs, but the Robot ROS environment (between ROS nodes) is isolated (also for Robotics security reasons) and cannot communicate with that external server. This issue should be better investigated because it may be important also for the scenario 1.
The talk.io test may be come handy to test if WebRTC support from Chrome is working fine in the Robot, but does not require Kurento. The next steps will need to use a configuration including a Kurento Client (thus using Kurento protocol) and we’ll need an instance of Kurento Server. Do you think it would be possible in the next phase to use a FI-LAB instance of Kurento to make some test with a Kurento Client? The solution of installing a local Kurento Server wouldn’t probably simplify the communication issues with the Robotics platform, and probably would have less impact for Robotics and Kurento GEs demo. For the same reason we are using an external instance of the Context Broker: one of purpose of the live demo is to show interoperability of different FIWARE GEs in the Cloud.
We are still defining the Live demo, and at this stage it’s not yet completely defined if and how exploiting the Kurento advanced features (beyond the basic video streaming supported by WebRTC). One idea might be working on some pattern or face detection when we use an Infrared camera. This solution would obviously imply the usage of Kurento equipped with some processing function among the ones already developed and already made available with Kurento (no need to have ad-hoc functions, but use of one of the Kurento processing components available).
This is my general understanding and overview; all the other Robotics in the mailing list guys have deeper knowledge of the technical details and specifications of the Robotics Platforms GE components, they can arise more precise proposals or questions.
Thank you,
Gianmario
Da: Luis López Fernández [mailto:luis.lopez at urjc.es]
Inviato: martedì 17 febbraio 2015 19:05
A: Bollano Gianmario
Cc: Antonini Roberto; fiware-robotics at lists.fi-ware.org<mailto:fiware-robotics at lists.fi-ware.org>
Oggetto: Re: [Fiware-robotics] Robot characteristics
Hi
Answers inline
El 17/02/2015, a las 16:47, Bollano Gianmario <gianmario.bollano at telecomitalia.it<mailto:gianmario.bollano at telecomitalia.it>> escribió:
Hi Luis,
I think the questions arisen from Roberto and Davide can be summarized as:
- Assuming to use a chrome browser, do we need a specific “Kurento Client” code (e.g. a JS library for Kurento APIs) to call a Kurento function and transmit a video streaming to the media server?
The short answer is no. The long answer is that it depend on the architecture of your application. Probably, the simplest way for you to start working with kurento is following one of these tutorials:
1. Read first this and pay attention to the role of the application server (node or java based)
http://www.kurento.org/docs/current/introducing_kurento.html
After that, my opinion is that you should go to an architecture based on application server, so the browser does not speak directly with Kurento Media Server. You can find several tutorials of applications basing on such architecture. For example
In java
http://www.kurento.org/docs/current/tutorials/java/tutorial-1-helloworld.html
In JavaScript (for Node.js)
http://www.kurento.org/docs/current/tutorials/node/tutorial-1-helloworld.html
For making something more useful (e.g. making possible to robot to broadcast the video to a number of receivers), take a look to these tutorials:
In java
http://www.kurento.org/docs/current/tutorials/java/tutorial-3-one2many.html
In JavaScript (for Node.js)
http://www.kurento.org/docs/current/tutorials/node/tutorial-3-one2many.html
- Would it be possible to have a simple code snipped to try to transmit a video stream to Kurento and use it to check if it works immediately in our robot (equipped with chrome and linux)?
Install one of the tutorials listed above and open them from the robot using a chrome browser. That should do the job.
The point is not WebRTC, I think the different options for WebRTC clients are clear now, but instead identifying the simplest code (Kurento client library, snippet code, etc) required to make a quick test and transmit a video stream from a Robot to a real Kurento media server.
If you were able to execute the simple peer-to-peer example I provided in a previous mail (basing on talky.io for example), then you won't have problems for interoperating with Kurento.
This will help to see if the Robot environment is sufficiently compliant with WebRTC and Kurento requirements, and the robot have enough resources (CPU, memory) to allow a realtime transmission of a video stream from a webcam.
Again, you can check this with the basic peer-to-peer example based on talky.io without requiring Kurento at all for it.
Best.
Gianmario
Da: fiware-robotics-bounces at lists.fi-ware.org<mailto:fiware-robotics-bounces at lists.fi-ware.org> [mailto:fiware-robotics-bounces at lists.fi-ware.org<mailto:robotics-bounces at lists.fi-ware.org>] Per conto di Luis López Fernández
Inviato: martedì 17 febbraio 2015 16:20
A: Antonini Roberto
Cc: fiware-robotics at lists.fi-ware.org<mailto:fiware-robotics at lists.fi-ware.org>
Oggetto: Re: [Fiware-robotics] Robot characteristics
Hi Roberto,
Please, take into consideration the following information for having an understanding on how to use Kurento:
Kurento is:
- A server infrastructure (it receives streams from clients and "do things" with those streams such as transcoding, processing, recording, distributing, etc)
- A WebRTC capable server (it's capable of receiving WebRTC streams, among other types of streams)
Kurento is not:
- A WebRTC client (it cannot "open a webcam" and send the stream to a remote party)
Hence, in your architecture, you need two ingredients:
- A WebRTC client catching the stream from a Camera and sending the stream using WebRTC standard protocols
- A WebRTC server, for receiving the stream and "do things" with that stream.
The "WebRTC server" part is already solved by Kurento and you'll not need to do much for integrating with it.
The "WebRTC client" part is not solved by Kurento. You'll need to have your very own client and in my last e-mail I just provided different alternatives that you have. Kurento is not one of them because Kurento is not a client.
I hope this clarifies the issues.
Best.
El 17/02/2015, a las 16:12, Antonini Roberto <roberto1.antonini at telecomitalia.it<mailto:roberto1.antonini at telecomitalia.it>> escribió:
Hi Luis,
thanks for having promptly answered my e-mail .
Amongst the solutions you’ve proposed, it’s not clear (at least to me) which of these are using KURENTO GE.
I figured out that you provided us with some code snippet (with apposite KURENTO API usage) to build a client and connect your GE from our robot, I guess these are only a test cases to validate WebRTC technology on the scenario we shared, so I suppose I miss the link between these tests and KURENTO.
Anyway, just to better clarify what we meant to do:
On our side, we would like to use KURENTO javascript clients (through Google Chrome) to connect to your KURENTO media server (using WebRTC technology), is it feasible?
Please, let us know which are the steps (and consequently the complexity) to integrate the components above in our scenario, if we understand well.
BR
Roberto and Davide
Da: Luis López Fernández [mailto:luis.lopez at urjc.es<http://urjc.es>]
Inviato: giovedì 12 febbraio 2015 16:29
A: Antonini Roberto
Cc: Garino Pierangelo; Colombatto Davide; fabio.dibenedetto at consoft.it<mailto:fabio.dibenedetto at consoft.it>; Bollano Gianmario; Angel Hernandez; Iñigo Gonzalez (igonzalez at ikergune.com<mailto:igonzalez at ikergune.com>); Jose Jaime Ariza (jjaime at ikergune.com<mailto:jjaime at ikergune.com>); Fernando Herranz (fherranz at ikergune.com<mailto:fherranz at ikergune.com>); fiware-robotics at lists.fi-ware.org<mailto:fiware-robotics at lists.fi-ware.org>
Oggetto: Re: Robot characteristics
Hi Roberto,
basing on the specifications you are providing, my recommendation is to integrate WebRTC support into the robot in a native way. For this, you have several options.
1. Use the WebRTC Google stack.
* This is the most mature WebRTC stack providing lots of advanced features
* It's the stack used by Chrome
* Can be compiled and used independently on the browser (with some work)
* Only supports VP8 codec. Supporting H264 requires some complex integrations
* Details of it can be found here: http://www.webrtc.org/
2. Use the WebRTC Ericsson Stack
* Less mature but also less heavy
* Supports natively H.264, which is a more performant codec for your use case.
* Details can be found here: http://www.openwebrtc.io/
Here, my choice would be option 1, but you may wish to perform a deeper analysis.
For validating that you webcam works correctly with WebRTC, do the following:
- Provide internet connectivity to the robot
- Install chrome on it
- Open this URL
-- https://talky.io/
- Fill a string in the form (e.g. "aaa") and press the button.
- Accept to share the webcam
- The video should appear
- Open another browser into another computer with Internet connectivity and a webcam
- Open the url https://talky.io/aaa (where "aaa" is the string you filled previously in the form)
- Accept to share the webcam
- You should have a video conference between the two browsers
Best.
-----------------------------------------------------------
Luis López Fernández
Subdirector de Investigación y Relaciones con la Empresa
Escuela Técnica Superior de Ingenieros de Telecomunicación
Universidad Rey Juan Carlos
http://www.etsit.urjc.es
e-mail: luis.lopez at urjc.es<mailto:luis.lopez at urjc.es>
Tf1: +34 914 888 747
Tf2: + 34 914 888 713
El 11/02/2015, a las 13:54, Antonini Roberto <roberto1.antonini at telecomitalia.it<mailto:roberto1.antonini at telecomitalia.it>> escribió:
Hi Luis
As agreed, we start sending to you the characteristics of the robot we have here, that’s a turtlebot2 and the PC on board has Ubuntu 12.04 OS (more details available athttp://www.clearpathrobotics.com/turtlebot_2/tech-specs/))
The camera we intend to use is a Logitech c270 (more details available at http://www.logitech.com/it-it/product/hd-webcam-c270).
Let us know if this’s enough, we wait for instructions on how to setup a WebRTC video streaming, in order to solve technical issues and test the end-to-end communication.
Thank you.
Davide and Roberto
Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione, Grazie.
This e-mail and any attachments is confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorised. If you are not the intended recipient, please delete this message and any attachments and advise the sender by return e-mail, Thanks.
<logo Ambiente_foglia2.jpg>Rispetta l'ambiente. Non stampare questa mail se non è necessario.
<logo Ambiente_foglia2.jpg>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/fiware-robotics/attachments/20150219/8af2581f/attachment.html>
You can get more information about our cookies and privacy policies clicking on the following links: Privacy policy Cookies policy