From GLIKSON at il.ibm.com Mon Jun 1 08:28:11 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Mon, 1 Jun 2015 09:28:11 +0300 Subject: [Fiware-cloud-containers] Containers/Docker discussion Message-ID: Dear partners, We will defer the Architects discussion regarding Containers/Docker until the 15th. Until then, we will make progress within the dedicated task force that has been established. Regards, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 -------------- next part -------------- An HTML attachment was scrubbed... URL: From GLIKSON at il.ibm.com Mon Jun 1 08:31:13 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Mon, 1 Jun 2015 09:31:13 +0300 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: References: <5569EA0F.7010900@dfki.de> <504347CB-229B-4483-A453-A5565124AAFC@telefonica.com> Message-ID: We will hold the taskforce kickoff at a different time slot, to accommodate availability of key people. I will send a doodle later today. Meanwhile, we can try making a bit progress by email, following the summary I sent in a separate email. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Alex Glikson/Haifa/IBM at IBMIL To: "fiware-cloud-containers at lists.fiware.org" Date: 31/05/2015 10:59 PM Subject: Re: [Fiware-cloud-containers] task-force kickoff Sent by: fiware-cloud-containers-bounces at lists.fiware.org Before this task force has launched, discussion on containers has been scheduled for tomorrow, 10:30. Now, maybe it would make sense to make some progress within the task force before raising this with the (rest of the) architects - especially if some of the participants can't join tomorrow. Regards, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: FERNANDO LOPEZ AGUILAR To: JUAN JOSE HIERRO SUREDA , Alex Glikson/Haifa/IBM at IBMIL Cc: "fiware-cloud-containers at lists.fiware.org" Date: 31/05/2015 10:03 PM Subject: Re: [Fiware-cloud-containers] task-force kickoff Ok, then the question is we will have the kickoff of this task force tomorrow? And if it is true between which hours? We have tomorrow our internal sprint planning for the next sprint from 9:00 to 13:00. Fernando.- De: JUAN JOSE HIERRO SUREDA Fecha: Saturday 30 May 2015 20:31 Para: Alex Glikson CC: "fiware-cloud-containers at lists.fiware.org" < fiware-cloud-containers at lists.fiware.org> Asunto: Re: [Fiware-cloud-containers] task-force kickoff That's what I understood too Juanjo from iPhone El 30/5/2015, a las 19:52, Alex Glikson escribi?: I am pretty sure that the monthly review/demo meeting has been rescheduled, so it seems that the morning slot on the 1st is free.. ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Philipp Slusallek To: Alex Glikson/Haifa/IBM at IBMIL, fiware-cloud-containers at lists.fiware.org Date: 30/05/2015 07:49 PM Subject: Re: [Fiware-cloud-containers] task-force kickoff Hi, I thought the June 1 meetings will be moved. But I would join anyway. Best, Philipp Am 30.05.2015 um 18:06 schrieb Alex Glikson: > Dear all, > > Welcome to 'fiware-cloud-containers' mailing list! We will use this > mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. > support for Linux containers (such as Docker), including use-cases & > requirements, associated IaaS and PaaS-style capabilities, concrete > plans going forwards, potential collaboration with teams working on > similar/related technologies, etc. > > Currently we have the following people subscribed to this mailing list: > IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for > IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud > leader), Doron (working with Ezra on containers) > TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads > for the PaaS-related work in FIWARE Cloud) > DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter > leader/architect) > Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old > FI-WARE project) > > Let me know if anyone is missing. > > As a background, please, read the thread below, including the very nice > summary by Mario on the related investigation and implementation in > FI-Content2 project. I will send a separate email briefly summarizing > the situation in FIWARE. > > *All *-- notice that we are going to have a phone call to discuss this > topic during the upcoming FIWARE architects interlock on Monday, June > 1at, 10:30-12:30 CET. Let me know if you can't join. > *Mario *-- would you be able to join & explain/demonstrate the FIC2Lab > solution? > > Thanks, > Alex > > > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- > > fiware-chapter-architects-bounces at lists.fi-ware.org wrote on 26/05/2015 > 08:37:16 AM: > >> From: Alex Glikson/Haifa/IBM at IBMIL >> To: Philipp Slusallek >> Cc: fiware-chapter-architects at lists.fi-ware.org, Kenneth Nagin/ >> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >> Date: 26/05/2015 08:37 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> Sent by: fiware-chapter-architects-bounces at lists.fi-ware.org >> >> Hi Philipp, >> >> It is great to hear that FIC2 team is willing to contribute! >> In a nutshell, our current plan is to start with surfacing >> containers via Nova, and then gradually migrate to Magnum. In terms >> of orchestration, we currently work with Heat and Murano, and are >> assessing the approaches to support containers (OpenStack alignment >> being high priority, but also the ability to surface capabilities >> unique to containers). >> I suggest that we form a small taskforce to explore this topic. >> Please, send me a list of people to involve. >> >> Thanks, >> Alex >> >> > ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> >> >> >> From: Philipp Slusallek >> To: Juanjo Hierro , Alex >> Glikson/Haifa/IBM at IBMIL >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >> architects at lists.fi-ware.org, Kenneth Nagin/Haifa/IBM at IBMIL >> Date: 26/05/2015 07:21 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> >> >> >> Hi, >> >> Here is some first input from FIC2 on Docker compiled from Mario. As I >> mentioned before, we have a full system working that deploys entire sets >> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >> >> FIC2 would be happy to work with FIWARE to integrate them in whatever >> way we at FIWARE decide to implement Docker. Right now we deploy it on >> top of the VMs offered by FIWARE but a more native implementation would >> be fine as well and save much resources (which seems to be a major issue). >> >> A presentation by Canonical recently showed a >10x improvement in >> resource usage with their containers over plain OpenStack (which is >> probably where we are -- but then Intel claims even better numbers in >> their Clear Containers solution using traditional VMs). >> >> Since we have experience already, it seems very useful to make Docker >> available especially since deployment of SW is much easier too and the >> same SW can be deployed essentially anywhere (including a local machine). >> >> >> Best, >> >> Philipp >> >> > There are several ways in which FIWARE could support Docker. My >> assumption is that the plan is to add to OpenStack a Docker driver. >> It would enable the creation of instances which are not VMs but >> OpenStack containers, from images which are not ISOs but Docker >> images. It?s described here: https://wiki.openstack.org/wiki/Docker >> > >> > >> > >> > Advantages: >> > >> > + Launches faster than VM instances >> > >> > + Less resources used from quotas in FIWARE nodes >> > >> > + images of SEs smaller and easier to build (using Dockerfiles) >> > >> > >> > >> > Open questions / disadvantages: >> > >> > - Will it be possible to pull images from DockerHub (preferred >> approach), or do they need to be uploaded to the Glance image >> repository of each FIWARE node? >> > >> > - Is the only way to launch containers the Cloud Portal or >> OpenStack API? Not being able to target a Docker API would reduce >> the value of Docker and its ecosystem of tools, e.g.: docker-compose >> which enables launching applications composed of multiple containers. >> > >> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >> reverse proxy that forwards HTTP requests to container1.john- >> smith.cloud.fiware.org to the right container port? >> > >> > - How would links between containers running on different hosts >> work? Using Weave? >> > >> > - Mounting volumes in Docker is a very common way of passing >> configuration files, having persistence, etc. How would this work > inOpenStack? >> > >> > >> > >> > Our suggestion: >> > >> > * Ability to use the Docker API remotely (i.e. command-line tools >> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >> > >> > * Ability to pull containers directly from DockerHub with a common >> (transparent?) image mirror in each FIWARE node >> > >> > >> > >> > >> > >> > >> > >> > Regarding FIC2Lab runner, our wishlist was: >> > >> > 1. ability to use the same tools locally and remotely in any cloud >> (ideally use docker-machine against any cloud provider, in >> particular FIWARE nodes) >> > >> > 2. nice web user interface hiding the complexity of firewalling, >> security pairs, public IPs, etc. >> > >> > 3. compatibility with the tools of the docker ecosystem: docker >> CLI, compose, swarm ... >> > >> > >> > >> > We compared tools such as tutum (free multi-tenant online >> service), shipyard (open-source web UI) and panamax (open-source web >> UI and orchestrator). >> > >> > And we decided to go for Panamax, which is nice for #1 and #2 but >> not #3 so far. Here's a few pluses and minuses: >> > >> > + nice UI with DockerHub search >> > >> > + composite application, with application template search >> > >> > - It's not multi-tenant, which is good for portability but bad for >> software updates >> > >> > - its model for composite applications is different from docker- >> compose (for now) >> > >> > - the UI can't synchronize with actions performed in Docker CLI or >> API because it's using CoreOS underneath >> > >> > >> > >> > The FIC2Lab runner is documented here: http://fic2.github.io/runner >> > >> > >> >> >> >> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >> > Dear Alex, >> > >> > It's fine with me. >> > >> > I believe that we should collocate here the discussion with >> > FI-Content2 (Philipp) regarding the stuff they have developed for the >> > FIC2-Lab. >> > >> > @Philipp: could you share some material (document describing your work >> > on FIC2-Lab, slides, whatever) describing what you have done and send it >> > to the fiware-chapter-architects mailing list as preparation for the >> > discussion? >> > >> > @Alex: please also share the ideas you want to present on this matter. >> > >> > By exchanging material prior to the meeting, we would be able to have >> > a more fruitful meeting. >> > >> > Best regards, >> > >> > -- Juanjo >> > >> > On 18/05/15 11:07, Alex Glikson wrote: >> >> I am out of office on the 25th. Would the following Monday work? >> >> >> >> Thanks, >> >> Alex >> >> >> >> >> >> >> >> >> >> >> >> >> >> From: Juanjo Hierro >> >> To: Philipp Slusallek , Alex >> >> Glikson/Haifa/IBM at IBMIL, < fiware-chapter-architects at lists.fi-ware.org> >> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >> >> Date: 18/05/2015 11:53 AM >> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & > Docker >> >> > ------------------------------------------------------------------------ >> >> >> >> >> >> >> >> >> >> I would suggest that we book the next architecture session (Monday >> >> 25th) to cover this matter. >> >> >> >> Will you Alex be able to prepare a presentation? >> >> >> >> Cheers, >> >> >> >> -- Juanjo >> >> >> >> On 18/05/15 09:46, Philipp Slusallek wrote: >> >> > Hi, >> >> > >> >> > This sounds very useful to cover as well in this or one of the next >> >> > calls. It would also get the architectural activities started > again. It >> >> > might even make sense to invite (selected?) AB members from the UCs. >> >> > >> >> > For example, a lot of work that has been done within FIcontent to > deploy >> >> > our SEs and related GEs within Docker on top of FIWARE Lab. > Particularly >> >> > relevant could be the on-click deployment of entire arrangements > of GEs >> >> > and SEs. Better and more dedicated Docker support would certainly be >> >> > very welcome as well. >> >> > >> >> > Unfortunately, I have a lecture this morning and cannot join > until 12h. >> >> > But Stefan Lemme from my group will be on the call. He has been a > core >> >> > member of the relevant FIC2-Lab task force in FIcontent. >> >> > >> >> > >> >> > Best, >> >> > >> >> > Philipp >> >> > >> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >> >> >> Maybe we can also dedicate some time to have an initial > discussion on >> >> >> the roadmap to adopt Linux Containers and in particular Docker in >> >> FIWARE >> >> >> Lab. >> >> >> >> >> >> Regards, >> >> >> Alex >> >> >> > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers -------------- next part -------------- An HTML attachment was scrubbed... URL: From philipp.slusallek at dfki.de Mon Jun 1 08:32:48 2015 From: philipp.slusallek at dfki.de (Philipp Slusallek) Date: Mon, 01 Jun 2015 08:32:48 +0200 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: References: <5569EA0F.7010900@dfki.de> <504347CB-229B-4483-A453-A5565124AAFC@telefonica.com> Message-ID: <556BFC90.80901@dfki.de> OK, so no meeting today, right (disregard my other email, they crossed). Best, Philipp Am 01.06.2015 um 08:31 schrieb Alex Glikson: > We will hold the taskforce kickoff at a different time slot, to > accommodate availability of key people. I will send a doodle later > today. Meanwhile, we can try making a bit progress by email, following > the summary I sent in a separate email. > > Thanks, > Alex > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > > > > From: Alex Glikson/Haifa/IBM at IBMIL > To: "fiware-cloud-containers at lists.fiware.org" > > Date: 31/05/2015 10:59 PM > Subject: Re: [Fiware-cloud-containers] task-force kickoff > Sent by: fiware-cloud-containers-bounces at lists.fiware.org > ------------------------------------------------------------------------ > > > > Before this task force has launched, discussion on containers has been > scheduled for tomorrow, 10:30. > Now, maybe it would make sense to make some progress within the task > force before raising this with the (rest of the) architects - especially > if some of the participants can't join tomorrow. > > Regards, > Alex > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > > > > From: FERNANDO LOPEZ AGUILAR > To: JUAN JOSE HIERRO SUREDA , > Alex Glikson/Haifa/IBM at IBMIL > Cc: "fiware-cloud-containers at lists.fiware.org" > > Date: 31/05/2015 10:03 PM > Subject: Re: [Fiware-cloud-containers] task-force kickoff > ------------------------------------------------------------------------ > > > > Ok, then the question is we will have the kickoff of this task force > tomorrow? And if it is true between which hours? > > We have tomorrow our internal sprint planning for the next sprint from > 9:00 to 13:00. > > Fernando.- > * > De: *JUAN JOSE HIERRO SUREDA <_juanjose.hierro at telefonica.com_ > >* > Fecha: *Saturday 30 May 2015 20:31* > Para: *Alex Glikson <_GLIKSON at il.ibm.com_ >* > CC: *"_fiware-cloud-containers at lists.fiware.org_ > " > <_fiware-cloud-containers at lists.fiware.org_ > >* > Asunto: *Re: [Fiware-cloud-containers] task-force kickoff > > That's what I understood too > > Juanjo from iPhone > > El 30/5/2015, a las 19:52, Alex Glikson <_GLIKSON at il.ibm.com_ > > escribi?: > > > I am pretty sure that the monthly review/demo meeting has been > rescheduled, so it seems that the morning slot on the 1st is free.. > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: _glikson at il.ibm.com_ | Phone: > +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 > > > > > > From: Philipp Slusallek <_philipp.slusallek at dfki.de_ > > > To: Alex Glikson/Haifa/IBM at IBMIL, > _fiware-cloud-containers at lists.fiware.org_ > > Date: 30/05/2015 07:49 PM > Subject: Re: [Fiware-cloud-containers] task-force kickoff > ------------------------------------------------------------------------ > > > > Hi, > > I thought the June 1 meetings will be moved. But I would join anyway. > > Best, > > Philipp > > Am 30.05.2015 um 18:06 schrieb Alex Glikson: >> Dear all, >> >> Welcome to 'fiware-cloud-containers' mailing list! We will use this >> mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. >> support for Linux containers (such as Docker), including use-cases & >> requirements, associated IaaS and PaaS-style capabilities, concrete >> plans going forwards, potential collaboration with teams working on >> similar/related technologies, etc. >> >> Currently we have the following people subscribed to this mailing list: >> IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for >> IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud >> leader), Doron (working with Ezra on containers) >> TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads >> for the PaaS-related work in FIWARE Cloud) >> DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter >> leader/architect) >> Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old >> FI-WARE project) >> >> Let me know if anyone is missing. >> >> As a background, please, read the thread below, including the very nice >> summary by Mario on the related investigation and implementation in >> FI-Content2 project. I will send a separate email briefly summarizing >> the situation in FIWARE. >> >> *All *-- notice that we are going to have a phone call to discuss this >> topic during the upcoming FIWARE architects interlock on Monday, June >> 1at, 10:30-12:30 CET. Let me know if you can't join. >> *Mario *-- would you be able to join & explain/demonstrate the FIC2Lab >> solution? >> >> Thanks, >> Alex >> >> >> >> > ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: _glikson at il.ibm.com_ | Phone: > +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- >> >> _fiware-chapter-architects-bounces at lists.fi-ware.org_ > wrote on > 26/05/2015 >> 08:37:16 AM: >> >>> From: Alex Glikson/Haifa/IBM at IBMIL >>> To: Philipp Slusallek <_philipp.slusallek at dfki.de_ > > >>> Cc: _fiware-chapter-architects at lists.fi-ware.org_ > , Kenneth Nagin/ >>> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >>> Date: 26/05/2015 08:37 AM >>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >>> Sent by: _fiware-chapter-architects-bounces at lists.fi-ware.org_ > >>> >>> Hi Philipp, >>> >>> It is great to hear that FIC2 team is willing to contribute! >>> In a nutshell, our current plan is to start with surfacing >>> containers via Nova, and then gradually migrate to Magnum. In terms >>> of orchestration, we currently work with Heat and Murano, and are >>> assessing the approaches to support containers (OpenStack alignment >>> being high priority, but also the ability to surface capabilities >>> unique to containers). >>> I suggest that we form a small taskforce to explore this topic. >>> Please, send me a list of people to involve. >>> >>> Thanks, >>> Alex >>> >>> >> > ==================================================================================== >>> Alex Glikson >>> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >>> Email: _glikson at il.ibm.com_ | Phone: > +972-4-8281085 | Mobile: >>> +972-54-6466667 | Fax: +972-4-8296112 >>> >>> >>> >>> >>> >>> From: Philipp Slusallek <_philipp.slusallek at dfki.de_ > > >>> To: Juanjo Hierro <_juanjose.hierro at telefonica.com_ > >, Alex >>> Glikson/Haifa/IBM at IBMIL >>> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >>> _architects at lists.fi-ware.org_ , > Kenneth Nagin/Haifa/IBM at IBMIL >>> Date: 26/05/2015 07:21 AM >>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >>> >>> >>> >>> Hi, >>> >>> Here is some first input from FIC2 on Docker compiled from Mario. As I >>> mentioned before, we have a full system working that deploys entire sets >>> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >>> >>> FIC2 would be happy to work with FIWARE to integrate them in whatever >>> way we at FIWARE decide to implement Docker. Right now we deploy it on >>> top of the VMs offered by FIWARE but a more native implementation would >>> be fine as well and save much resources (which seems to be a major > issue). >>> >>> A presentation by Canonical recently showed a >10x improvement in >>> resource usage with their containers over plain OpenStack (which is >>> probably where we are -- but then Intel claims even better numbers in >>> their Clear Containers solution using traditional VMs). >>> >>> Since we have experience already, it seems very useful to make Docker >>> available especially since deployment of SW is much easier too and the >>> same SW can be deployed essentially anywhere (including a local machine). >>> >>> >>> Best, >>> >>> Philipp >>> >>> > There are several ways in which FIWARE could support Docker. My >>> assumption is that the plan is to add to OpenStack a Docker driver. >>> It would enable the creation of instances which are not VMs but >>> OpenStack containers, from images which are not ISOs but Docker >>> images. It?s described here: _https://wiki.openstack.org/wiki/Docker_ >>> > >>> > >>> > >>> > Advantages: >>> > >>> > + Launches faster than VM instances >>> > >>> > + Less resources used from quotas in FIWARE nodes >>> > >>> > + images of SEs smaller and easier to build (using Dockerfiles) >>> > >>> > >>> > >>> > Open questions / disadvantages: >>> > >>> > - Will it be possible to pull images from DockerHub (preferred >>> approach), or do they need to be uploaded to the Glance image >>> repository of each FIWARE node? >>> > >>> > - Is the only way to launch containers the Cloud Portal or >>> OpenStack API? Not being able to target a Docker API would reduce >>> the value of Docker and its ecosystem of tools, e.g.: docker-compose >>> which enables launching applications composed of multiple containers. >>> > >>> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >>> reverse proxy that forwards HTTP requests to container1.john- >>> _smith.cloud.fiware.org_ to the right > container port? >>> > >>> > - How would links between containers running on different hosts >>> work? Using Weave? >>> > >>> > - Mounting volumes in Docker is a very common way of passing >>> configuration files, having persistence, etc. How would this work >> inOpenStack? >>> > >>> > >>> > >>> > Our suggestion: >>> > >>> > * Ability to use the Docker API remotely (i.e. command-line tools >>> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >>> > >>> > * Ability to pull containers directly from DockerHub with a common >>> (transparent?) image mirror in each FIWARE node >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > Regarding FIC2Lab runner, our wishlist was: >>> > >>> > 1. ability to use the same tools locally and remotely in any cloud >>> (ideally use docker-machine against any cloud provider, in >>> particular FIWARE nodes) >>> > >>> > 2. nice web user interface hiding the complexity of firewalling, >>> security pairs, public IPs, etc. >>> > >>> > 3. compatibility with the tools of the docker ecosystem: docker >>> CLI, compose, swarm ... >>> > >>> > >>> > >>> > We compared tools such as tutum (free multi-tenant online >>> service), shipyard (open-source web UI) and panamax (open-source web >>> UI and orchestrator). >>> > >>> > And we decided to go for Panamax, which is nice for #1 and #2 but >>> not #3 so far. Here's a few pluses and minuses: >>> > >>> > + nice UI with DockerHub search >>> > >>> > + composite application, with application template search >>> > >>> > - It's not multi-tenant, which is good for portability but bad for >>> software updates >>> > >>> > - its model for composite applications is different from docker- >>> compose (for now) >>> > >>> > - the UI can't synchronize with actions performed in Docker CLI or >>> API because it's using CoreOS underneath >>> > >>> > >>> > >>> > The FIC2Lab runner is documented here: _http://fic2.github.io/runner_ >>> > >>> > >>> >>> >>> >>> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >>> > Dear Alex, >>> > >>> > It's fine with me. >>> > >>> > I believe that we should collocate here the discussion with >>> > FI-Content2 (Philipp) regarding the stuff they have developed for the >>> > FIC2-Lab. >>> > >>> > @Philipp: could you share some material (document describing your > work >>> > on FIC2-Lab, slides, whatever) describing what you have done and > send it >>> > to the fiware-chapter-architects mailing list as preparation for the >>> > discussion? >>> > >>> > @Alex: please also share the ideas you want to present on this > matter. >>> > >>> > By exchanging material prior to the meeting, we would be able to have >>> > a more fruitful meeting. >>> > >>> > Best regards, >>> > >>> > -- Juanjo >>> > >>> > On 18/05/15 11:07, Alex Glikson wrote: >>> >> I am out of office on the 25th. Would the following Monday work? >>> >> >>> >> Thanks, >>> >> Alex >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> From: Juanjo Hierro <_juanjose.hierro at telefonica.com_ > > >>> >> To: Philipp Slusallek <_philipp.slusallek at dfki.de_ > >, Alex >>> >> Glikson/Haifa/IBM at IBMIL, > <_fiware-chapter-architects at lists.fi-ware.org_ > > >>> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >>> >> Date: 18/05/2015 11:53 AM >>> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & >> Docker >>> >> >> ------------------------------------------------------------------------ >>> >> >>> >> >>> >> >>> >> >>> >> I would suggest that we book the next architecture session (Monday >>> >> 25th) to cover this matter. >>> >> >>> >> Will you Alex be able to prepare a presentation? >>> >> >>> >> Cheers, >>> >> >>> >> -- Juanjo >>> >> >>> >> On 18/05/15 09:46, Philipp Slusallek wrote: >>> >> > Hi, >>> >> > >>> >> > This sounds very useful to cover as well in this or one of the next >>> >> > calls. It would also get the architectural activities started >> again. It >>> >> > might even make sense to invite (selected?) AB members from the UCs. >>> >> > >>> >> > For example, a lot of work that has been done within FIcontent to >> deploy >>> >> > our SEs and related GEs within Docker on top of FIWARE Lab. >> Particularly >>> >> > relevant could be the on-click deployment of entire arrangements >> of GEs >>> >> > and SEs. Better and more dedicated Docker support would certainly be >>> >> > very welcome as well. >>> >> > >>> >> > Unfortunately, I have a lecture this morning and cannot join >> until 12h. >>> >> > But Stefan Lemme from my group will be on the call. He has been a >> core >>> >> > member of the relevant FIC2-Lab task force in FIcontent. >>> >> > >>> >> > >>> >> > Best, >>> >> > >>> >> > Philipp >>> >> > >>> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >>> >> >> Maybe we can also dedicate some time to have an initial >> discussion on >>> >> >> the roadmap to adopt Linux Containers and in particular Docker in >>> >> FIWARE >>> >> >> Lab. >>> >> >> >>> >> >> Regards, >>> >> >> Alex >>> >> >> >> >> >> _______________________________________________ >> Fiware-cloud-containers mailing list >> _Fiware-cloud-containers at lists.fiware.org_ > >> _https://lists.fiware.org/listinfo/fiware-cloud-containers_ >> > > -- > > ------------------------------------------------------------------------- > Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH > Trippstadter Strasse 122, D-67663 Kaiserslautern > > Gesch?ftsf?hrung: > Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) > Dr. Walter Olthoff > Vorsitzender des Aufsichtsrats: > Prof. Dr. h.c. Hans A. Aukes > > Sitz der Gesellschaft: Kaiserslautern (HRB 2313) > USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 > --------------------------------------------------------------------------- > [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] > _______________________________________________ > Fiware-cloud-containers mailing list_ > __Fiware-cloud-containers at lists.fiware.org_ > _ > __https://lists.fiware.org/listinfo/fiware-cloud-containers_ > > ------------------------------------------------------------------------ > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener informaci?n privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilizaci?n, > divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud > de la legislaci?n vigente. Si ha recibido este mensaje por error, le > rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda > a su destrucci?n. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution > or copying of this communication is strictly prohibited. If you have > received this transmission in error, do not read it. Please immediately > reply to the sender that you have received this communication in error > and then delete it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu > destinat?rio, pode conter informa??o privilegiada ou confidencial e ? > para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa > senhoria o destinat?rio indicado, fica notificado de que a leitura, > utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em > virtude da legisla??o vigente. Se recebeu esta mensagem por erro, > rogamos-lhe que nos o comunique imediatamente por esta mesma via e > proceda a sua destrui??o > > ------------------------------------------------------------------------ > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener informaci?n privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilizaci?n, > divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud > de la legislaci?n vigente. Si ha recibido este mensaje por error, le > rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda > a su destrucci?n. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution > or copying of this communication is strictly prohibited. If you have > received this transmission in error, do not read it. Please immediately > reply to the sender that you have received this communication in error > and then delete it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu > destinat?rio, pode conter informa??o privilegiada ou confidencial e ? > para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa > senhoria o destinat?rio indicado, fica notificado de que a leitura, > utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em > virtude da legisla??o vigente. Se recebeu esta mensagem por erro, > rogamos-lhe que nos o comunique imediatamente por esta mesma via e > proceda a sua destrui??o_______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: philipp_slusallek.vcf Type: text/x-vcard Size: 441 bytes Desc: not available URL: From GLIKSON at il.ibm.com Mon Jun 1 09:02:41 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Mon, 1 Jun 2015 10:02:41 +0300 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: <556BFC90.80901@dfki.de> References: <5569EA0F.7010900@dfki.de> <504347CB-229B-4483-A453-A5565124AAFC@telefonica.com> <556BFC90.80901@dfki.de> Message-ID: Right. Regards, Alex From: Philipp Slusallek To: Alex Glikson/Haifa/IBM at IBMIL, "fiware-cloud-containers at lists.fiware.org" Date: 01/06/2015 09:32 AM Subject: Re: [Fiware-cloud-containers] task-force kickoff OK, so no meeting today, right (disregard my other email, they crossed). Best, Philipp Am 01.06.2015 um 08:31 schrieb Alex Glikson: > We will hold the taskforce kickoff at a different time slot, to > accommodate availability of key people. I will send a doodle later > today. Meanwhile, we can try making a bit progress by email, following > the summary I sent in a separate email. > > Thanks, > Alex > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > > > > From: Alex Glikson/Haifa/IBM at IBMIL > To: "fiware-cloud-containers at lists.fiware.org" > > Date: 31/05/2015 10:59 PM > Subject: Re: [Fiware-cloud-containers] task-force kickoff > Sent by: fiware-cloud-containers-bounces at lists.fiware.org > ------------------------------------------------------------------------ > > > > Before this task force has launched, discussion on containers has been > scheduled for tomorrow, 10:30. > Now, maybe it would make sense to make some progress within the task > force before raising this with the (rest of the) architects - especially > if some of the participants can't join tomorrow. > > Regards, > Alex > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > > > > From: FERNANDO LOPEZ AGUILAR > To: JUAN JOSE HIERRO SUREDA , > Alex Glikson/Haifa/IBM at IBMIL > Cc: "fiware-cloud-containers at lists.fiware.org" > > Date: 31/05/2015 10:03 PM > Subject: Re: [Fiware-cloud-containers] task-force kickoff > ------------------------------------------------------------------------ > > > > Ok, then the question is we will have the kickoff of this task force > tomorrow? And if it is true between which hours? > > We have tomorrow our internal sprint planning for the next sprint from > 9:00 to 13:00. > > Fernando.- > * > De: *JUAN JOSE HIERRO SUREDA <_juanjose.hierro at telefonica.com_ > >* > Fecha: *Saturday 30 May 2015 20:31* > Para: *Alex Glikson <_GLIKSON at il.ibm.com_ >* > CC: *"_fiware-cloud-containers at lists.fiware.org_ > " > <_fiware-cloud-containers at lists.fiware.org_ > >* > Asunto: *Re: [Fiware-cloud-containers] task-force kickoff > > That's what I understood too > > Juanjo from iPhone > > El 30/5/2015, a las 19:52, Alex Glikson <_GLIKSON at il.ibm.com_ > > escribi?: > > > I am pretty sure that the monthly review/demo meeting has been > rescheduled, so it seems that the morning slot on the 1st is free.. > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: _glikson at il.ibm.com_ | Phone: > +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 > > > > > > From: Philipp Slusallek <_philipp.slusallek at dfki.de_ > > > To: Alex Glikson/Haifa/IBM at IBMIL, > _fiware-cloud-containers at lists.fiware.org_ > > Date: 30/05/2015 07:49 PM > Subject: Re: [Fiware-cloud-containers] task-force kickoff > ------------------------------------------------------------------------ > > > > Hi, > > I thought the June 1 meetings will be moved. But I would join anyway. > > Best, > > Philipp > > Am 30.05.2015 um 18:06 schrieb Alex Glikson: >> Dear all, >> >> Welcome to 'fiware-cloud-containers' mailing list! We will use this >> mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. >> support for Linux containers (such as Docker), including use-cases & >> requirements, associated IaaS and PaaS-style capabilities, concrete >> plans going forwards, potential collaboration with teams working on >> similar/related technologies, etc. >> >> Currently we have the following people subscribed to this mailing list: >> IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for >> IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud >> leader), Doron (working with Ezra on containers) >> TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads >> for the PaaS-related work in FIWARE Cloud) >> DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter >> leader/architect) >> Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old >> FI-WARE project) >> >> Let me know if anyone is missing. >> >> As a background, please, read the thread below, including the very nice >> summary by Mario on the related investigation and implementation in >> FI-Content2 project. I will send a separate email briefly summarizing >> the situation in FIWARE. >> >> *All *-- notice that we are going to have a phone call to discuss this >> topic during the upcoming FIWARE architects interlock on Monday, June >> 1at, 10:30-12:30 CET. Let me know if you can't join. >> *Mario *-- would you be able to join & explain/demonstrate the FIC2Lab >> solution? >> >> Thanks, >> Alex >> >> >> >> > ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: _glikson at il.ibm.com_ | Phone: > +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- >> >> _fiware-chapter-architects-bounces at lists.fi-ware.org_ > wrote on > 26/05/2015 >> 08:37:16 AM: >> >>> From: Alex Glikson/Haifa/IBM at IBMIL >>> To: Philipp Slusallek <_philipp.slusallek at dfki.de_ > > >>> Cc: _fiware-chapter-architects at lists.fi-ware.org_ > , Kenneth Nagin/ >>> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >>> Date: 26/05/2015 08:37 AM >>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >>> Sent by: _fiware-chapter-architects-bounces at lists.fi-ware.org_ > >>> >>> Hi Philipp, >>> >>> It is great to hear that FIC2 team is willing to contribute! >>> In a nutshell, our current plan is to start with surfacing >>> containers via Nova, and then gradually migrate to Magnum. In terms >>> of orchestration, we currently work with Heat and Murano, and are >>> assessing the approaches to support containers (OpenStack alignment >>> being high priority, but also the ability to surface capabilities >>> unique to containers). >>> I suggest that we form a small taskforce to explore this topic. >>> Please, send me a list of people to involve. >>> >>> Thanks, >>> Alex >>> >>> >> > ==================================================================================== >>> Alex Glikson >>> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >>> Email: _glikson at il.ibm.com_ | Phone: > +972-4-8281085 | Mobile: >>> +972-54-6466667 | Fax: +972-4-8296112 >>> >>> >>> >>> >>> >>> From: Philipp Slusallek <_philipp.slusallek at dfki.de_ > > >>> To: Juanjo Hierro <_juanjose.hierro at telefonica.com_ > >, Alex >>> Glikson/Haifa/IBM at IBMIL >>> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >>> _architects at lists.fi-ware.org_ , > Kenneth Nagin/Haifa/IBM at IBMIL >>> Date: 26/05/2015 07:21 AM >>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >>> >>> >>> >>> Hi, >>> >>> Here is some first input from FIC2 on Docker compiled from Mario. As I >>> mentioned before, we have a full system working that deploys entire sets >>> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >>> >>> FIC2 would be happy to work with FIWARE to integrate them in whatever >>> way we at FIWARE decide to implement Docker. Right now we deploy it on >>> top of the VMs offered by FIWARE but a more native implementation would >>> be fine as well and save much resources (which seems to be a major > issue). >>> >>> A presentation by Canonical recently showed a >10x improvement in >>> resource usage with their containers over plain OpenStack (which is >>> probably where we are -- but then Intel claims even better numbers in >>> their Clear Containers solution using traditional VMs). >>> >>> Since we have experience already, it seems very useful to make Docker >>> available especially since deployment of SW is much easier too and the >>> same SW can be deployed essentially anywhere (including a local machine). >>> >>> >>> Best, >>> >>> Philipp >>> >>> > There are several ways in which FIWARE could support Docker. My >>> assumption is that the plan is to add to OpenStack a Docker driver. >>> It would enable the creation of instances which are not VMs but >>> OpenStack containers, from images which are not ISOs but Docker >>> images. It?s described here: _https://wiki.openstack.org/wiki/Docker_ >>> > >>> > >>> > >>> > Advantages: >>> > >>> > + Launches faster than VM instances >>> > >>> > + Less resources used from quotas in FIWARE nodes >>> > >>> > + images of SEs smaller and easier to build (using Dockerfiles) >>> > >>> > >>> > >>> > Open questions / disadvantages: >>> > >>> > - Will it be possible to pull images from DockerHub (preferred >>> approach), or do they need to be uploaded to the Glance image >>> repository of each FIWARE node? >>> > >>> > - Is the only way to launch containers the Cloud Portal or >>> OpenStack API? Not being able to target a Docker API would reduce >>> the value of Docker and its ecosystem of tools, e.g.: docker-compose >>> which enables launching applications composed of multiple containers. >>> > >>> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >>> reverse proxy that forwards HTTP requests to container1.john- >>> _smith.cloud.fiware.org_ to the right > container port? >>> > >>> > - How would links between containers running on different hosts >>> work? Using Weave? >>> > >>> > - Mounting volumes in Docker is a very common way of passing >>> configuration files, having persistence, etc. How would this work >> inOpenStack? >>> > >>> > >>> > >>> > Our suggestion: >>> > >>> > * Ability to use the Docker API remotely (i.e. command-line tools >>> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >>> > >>> > * Ability to pull containers directly from DockerHub with a common >>> (transparent?) image mirror in each FIWARE node >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > Regarding FIC2Lab runner, our wishlist was: >>> > >>> > 1. ability to use the same tools locally and remotely in any cloud >>> (ideally use docker-machine against any cloud provider, in >>> particular FIWARE nodes) >>> > >>> > 2. nice web user interface hiding the complexity of firewalling, >>> security pairs, public IPs, etc. >>> > >>> > 3. compatibility with the tools of the docker ecosystem: docker >>> CLI, compose, swarm ... >>> > >>> > >>> > >>> > We compared tools such as tutum (free multi-tenant online >>> service), shipyard (open-source web UI) and panamax (open-source web >>> UI and orchestrator). >>> > >>> > And we decided to go for Panamax, which is nice for #1 and #2 but >>> not #3 so far. Here's a few pluses and minuses: >>> > >>> > + nice UI with DockerHub search >>> > >>> > + composite application, with application template search >>> > >>> > - It's not multi-tenant, which is good for portability but bad for >>> software updates >>> > >>> > - its model for composite applications is different from docker- >>> compose (for now) >>> > >>> > - the UI can't synchronize with actions performed in Docker CLI or >>> API because it's using CoreOS underneath >>> > >>> > >>> > >>> > The FIC2Lab runner is documented here: _http://fic2.github.io/runner_ >>> > >>> > >>> >>> >>> >>> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >>> > Dear Alex, >>> > >>> > It's fine with me. >>> > >>> > I believe that we should collocate here the discussion with >>> > FI-Content2 (Philipp) regarding the stuff they have developed for the >>> > FIC2-Lab. >>> > >>> > @Philipp: could you share some material (document describing your > work >>> > on FIC2-Lab, slides, whatever) describing what you have done and > send it >>> > to the fiware-chapter-architects mailing list as preparation for the >>> > discussion? >>> > >>> > @Alex: please also share the ideas you want to present on this > matter. >>> > >>> > By exchanging material prior to the meeting, we would be able to have >>> > a more fruitful meeting. >>> > >>> > Best regards, >>> > >>> > -- Juanjo >>> > >>> > On 18/05/15 11:07, Alex Glikson wrote: >>> >> I am out of office on the 25th. Would the following Monday work? >>> >> >>> >> Thanks, >>> >> Alex >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> From: Juanjo Hierro <_juanjose.hierro at telefonica.com_ > > >>> >> To: Philipp Slusallek <_philipp.slusallek at dfki.de_ > >, Alex >>> >> Glikson/Haifa/IBM at IBMIL, > <_fiware-chapter-architects at lists.fi-ware.org_ > > >>> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >>> >> Date: 18/05/2015 11:53 AM >>> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & >> Docker >>> >> >> ------------------------------------------------------------------------ >>> >> >>> >> >>> >> >>> >> >>> >> I would suggest that we book the next architecture session (Monday >>> >> 25th) to cover this matter. >>> >> >>> >> Will you Alex be able to prepare a presentation? >>> >> >>> >> Cheers, >>> >> >>> >> -- Juanjo >>> >> >>> >> On 18/05/15 09:46, Philipp Slusallek wrote: >>> >> > Hi, >>> >> > >>> >> > This sounds very useful to cover as well in this or one of the next >>> >> > calls. It would also get the architectural activities started >> again. It >>> >> > might even make sense to invite (selected?) AB members from the UCs. >>> >> > >>> >> > For example, a lot of work that has been done within FIcontent to >> deploy >>> >> > our SEs and related GEs within Docker on top of FIWARE Lab. >> Particularly >>> >> > relevant could be the on-click deployment of entire arrangements >> of GEs >>> >> > and SEs. Better and more dedicated Docker support would certainly be >>> >> > very welcome as well. >>> >> > >>> >> > Unfortunately, I have a lecture this morning and cannot join >> until 12h. >>> >> > But Stefan Lemme from my group will be on the call. He has been a >> core >>> >> > member of the relevant FIC2-Lab task force in FIcontent. >>> >> > >>> >> > >>> >> > Best, >>> >> > >>> >> > Philipp >>> >> > >>> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >>> >> >> Maybe we can also dedicate some time to have an initial >> discussion on >>> >> >> the roadmap to adopt Linux Containers and in particular Docker in >>> >> FIWARE >>> >> >> Lab. >>> >> >> >>> >> >> Regards, >>> >> >> Alex >>> >> >> >> >> >> _______________________________________________ >> Fiware-cloud-containers mailing list >> _Fiware-cloud-containers at lists.fiware.org_ > >> _https://lists.fiware.org/listinfo/fiware-cloud-containers_ >> > > -- > > ------------------------------------------------------------------------- > Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH > Trippstadter Strasse 122, D-67663 Kaiserslautern > > Gesch?ftsf?hrung: > Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) > Dr. Walter Olthoff > Vorsitzender des Aufsichtsrats: > Prof. Dr. h.c. Hans A. Aukes > > Sitz der Gesellschaft: Kaiserslautern (HRB 2313) > USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 > --------------------------------------------------------------------------- > [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] > _______________________________________________ > Fiware-cloud-containers mailing list_ > __Fiware-cloud-containers at lists.fiware.org_ > _ > __https://lists.fiware.org/listinfo/fiware-cloud-containers_ > > ------------------------------------------------------------------------ > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener informaci?n privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilizaci?n, > divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud > de la legislaci?n vigente. Si ha recibido este mensaje por error, le > rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda > a su destrucci?n. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution > or copying of this communication is strictly prohibited. If you have > received this transmission in error, do not read it. Please immediately > reply to the sender that you have received this communication in error > and then delete it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu > destinat?rio, pode conter informa??o privilegiada ou confidencial e ? > para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa > senhoria o destinat?rio indicado, fica notificado de que a leitura, > utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em > virtude da legisla??o vigente. Se recebeu esta mensagem por erro, > rogamos-lhe que nos o comunique imediatamente por esta mesma via e > proceda a sua destrui??o > > ------------------------------------------------------------------------ > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener informaci?n privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilizaci?n, > divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud > de la legislaci?n vigente. Si ha recibido este mensaje por error, le > rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda > a su destrucci?n. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution > or copying of this communication is strictly prohibited. If you have > received this transmission in error, do not read it. Please immediately > reply to the sender that you have received this communication in error > and then delete it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu > destinat?rio, pode conter informa??o privilegiada ou confidencial e ? > para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa > senhoria o destinat?rio indicado, fica notificado de que a leitura, > utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em > virtude da legisla??o vigente. Se recebeu esta mensagem por erro, > rogamos-lhe que nos o comunique imediatamente por esta mesma via e > proceda a sua destrui??o_______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] -------------- next part -------------- An HTML attachment was scrubbed... URL: From philipp.slusallek at dfki.de Mon Jun 1 09:11:55 2015 From: philipp.slusallek at dfki.de (Philipp Slusallek) Date: Mon, 01 Jun 2015 09:11:55 +0200 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: References: Message-ID: <556C05BB.5060900@dfki.de> Hi all, It is certainly the right thing to work on supporting containers natively in the Cloud chapter. It seems that a lot is currently being done in this regard within OpenStack and we should see how we can best contribute to that. However, I would like to point out another aspect that will probably have a much more direct and short-term impact on FIWARE and which we should be able to do in parallel: Focus on the user-perspective. I am talking about greatly simplifying the deployment of enablers for developers (and possibly end users) on FIWARE. Docker is an excellent tool for that as we have experienced for example in FIcontent. Its very easy for owners to provide their enabler and even easier for users to deploy and test them. As with the "Discover, Try, Tweak, Run" approach in FIcontent we can make it tremendously simpler for users to explore what we offer, try it out, share it with others via social networks, and deploy and use the necessary services for real. This is exactly what I believe FIWARE needs. The GEs are still somewhat difficult to access and the documentation for doing so leaves some things to be desired (at least judging from my reviewing job). Taking a much more user-focused approach is important now. So to make things concrete: We should explore (in parallel to your suggestions which is still needed, of course) to install something similar to FIC2-Lab (actually: now officially called "FIWARE Media&Content Lab"). We might be able to leverage the experience we got from there as a basis. But I am sure that together there are many improvements and extensions that should be done. In parallel, we would ask enabler owners to provide suitable dockerified versions that can easily be deployed via this system. This does not make sense for all and can be done incrementally as well. All that is then still needed is to create some nice examples that can be used for the Try and Tweak part, where developers can get first hand experience of enablers, by simply trying out, inspecting, and hacking on existing web-based applications that make use of one or more (bundles!) enablers from any Web browser and without ever having seen the cloud portal (they eventually need to but this is when they are hopefully hooked already). Again, this is not an alternative but a parallel activity that makes containers directly relevant for the developers right now (!) and which relies eventually on the ability to deploy enabler an easier, faster, and with a largely reduced resource footprint via the necessary technology in Cloud/OpenStack. Simply deploying docker within existing virtual machines should allow us to reduce resource usage by quite a bit. But we need to look into that. Best, Philipp Am 31.05.2015 um 22:36 schrieb Alex Glikson: > Below is a brief summary of the current FIWARE Cloud status with regards > to containers and Docker support. > > * FIWARE Cloud GEs currently deployed in FIWARE Lab provide VM-based > IaaS capabilities with OpenStack Nova (with KVM), Glance, Keystone, > Cinder and Neutron (with OVS). FIWARE Cloud also provides object > storage capabilities based on OpenStack Swift. Moreover, we support > PaaS-level capabilities, including the ability to provision and > auto-scale complex applications comprising sets of inter-dependent > VMs as well as the ability to install and configure the individual > software components within VMs using configuration management tools > such as Chef. The key user-visible notion behind this capability is > the notion of a Blueprint, which defines the application topology, > dependencies, auto-scaling rules, software configuration, etc > (conceptually similar to Amazon OpsWorks). > * In the last few months, the PaaS implementation is undergoing a > major transition from a 'proprietary' (although open source) code to > full adoption of OpenStack Heat and Murano (as well as contribution > of features to the Murano community, such as support for templates, > configuration management enhancements, etc). > * In parallel, we have started assessing (and prototyping) the options > to support/leverage Linux containers technologies, and Docker in > particular. We aim at two main use-cases: > 1. As a FIWARE Cloud provider (in FIWARE Lab), I want to continue > providing (almost?) the same PaaS/IaaS capabilities, but at the > same time being able to admit much more users/workloads on the > same hardware, by leveraging Containers technologies. > 2. As a FIWARE Cloud user, I want to be able to leverage the Docker > ecosystem, including public images available at Docker Hub as > well as existing tools (TBD: which tools exactly?), while > hosting my applications on the FIWARE Lab > * The 2nd use-case can be satisfied in several ways, such as: > 1. Treat Docker-based cluster as PaaS application (managed by > FIWARE Cloud PaaS), while the cloud middleware (IaaS and PaaS) > is not aware of individual containers, Docker images, etc. > 2. Make Docker containers 1st class citizens in PaaS -- e.g., as an > additional delivery mechanism in addition to Chef, deployed on > regular IaaS virtualized infra (providing isolation, accounting, > etc). In this case each Docker cluster is contained within a > single tenant (and even single application within a tenant?) > 3. Make Docker containers 1st class citizens in IaaS deployed on > bare-metal, making PaaS use it via a standardized API > * The current (tentative) plan is to apply a 2-phase approach to > introduce containers: phase-1: start by deploying nova-docker in > FIWARE lab in order to enable use-case n.1, and then phase-2: > gradually migrate to OpenStack Magnum, while incorporating the > Docker-native capabilities into Murano in order to support use-case > n.2. Of course, the devil is in the details, so we need to discuss > further, to understand the exact requirements and implications. > > > Regards, > Alex > > > > > From: Alex Glikson/Haifa/IBM at IBMIL > To: fiware-cloud-containers at lists.fiware.org > Date: 30/05/2015 07:06 PM > Subject: [Fiware-cloud-containers] task-force kickoff > Sent by: fiware-cloud-containers-bounces at lists.fiware.org > ------------------------------------------------------------------------ > > > > Dear all, > > Welcome to 'fiware-cloud-containers' mailing list! We will use this > mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. > support for Linux containers (such as Docker), including use-cases & > requirements, associated IaaS and PaaS-style capabilities, concrete > plans going forwards, potential collaboration with teams working on > similar/related technologies, etc. > > Currently we have the following people subscribed to this mailing list: > IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for > IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud > leader), Doron (working with Ezra on containers) > TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads > for the PaaS-related work in FIWARE Cloud) > DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter > leader/architect) > Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old > FI-WARE project) > > Let me know if anyone is missing. > > As a background, please, read the thread below, including the very nice > summary by Mario on the related investigation and implementation in > FI-Content2 project. I will send a separate email briefly summarizing > the situation in FIWARE. > * > All *-- notice that we are going to have a phone call to discuss this > topic during the upcoming FIWARE architects interlock on Monday, June > 1at, 10:30-12:30 CET. Let me know if you can't join.* > Mario *-- would you be able to join & explain/demonstrate the FIC2Lab > solution? > > Thanks, > Alex > > > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- > > fiware-chapter-architects-bounces at lists.fi-ware.org wrote on 26/05/2015 > 08:37:16 AM: > >> From: Alex Glikson/Haifa/IBM at IBMIL >> To: Philipp Slusallek >> Cc: fiware-chapter-architects at lists.fi-ware.org, Kenneth Nagin/ >> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >> Date: 26/05/2015 08:37 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> Sent by: fiware-chapter-architects-bounces at lists.fi-ware.org >> >> Hi Philipp, >> >> It is great to hear that FIC2 team is willing to contribute! >> In a nutshell, our current plan is to start with surfacing >> containers via Nova, and then gradually migrate to Magnum. In terms >> of orchestration, we currently work with Heat and Murano, and are >> assessing the approaches to support containers (OpenStack alignment >> being high priority, but also the ability to surface capabilities >> unique to containers). >> I suggest that we form a small taskforce to explore this topic. >> Please, send me a list of people to involve. >> >> Thanks, >> Alex >> >> > ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> >> >> >> From: Philipp Slusallek >> To: Juanjo Hierro , Alex >> Glikson/Haifa/IBM at IBMIL >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >> architects at lists.fi-ware.org, Kenneth Nagin/Haifa/IBM at IBMIL >> Date: 26/05/2015 07:21 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> >> >> >> Hi, >> >> Here is some first input from FIC2 on Docker compiled from Mario. As I >> mentioned before, we have a full system working that deploys entire sets >> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >> >> FIC2 would be happy to work with FIWARE to integrate them in whatever >> way we at FIWARE decide to implement Docker. Right now we deploy it on >> top of the VMs offered by FIWARE but a more native implementation would >> be fine as well and save much resources (which seems to be a major issue). >> >> A presentation by Canonical recently showed a >10x improvement in >> resource usage with their containers over plain OpenStack (which is >> probably where we are -- but then Intel claims even better numbers in >> their Clear Containers solution using traditional VMs). >> >> Since we have experience already, it seems very useful to make Docker >> available especially since deployment of SW is much easier too and the >> same SW can be deployed essentially anywhere (including a local machine). >> >> >> Best, >> >> Philipp >> >> > There are several ways in which FIWARE could support Docker. My >> assumption is that the plan is to add to OpenStack a Docker driver. >> It would enable the creation of instances which are not VMs but >> OpenStack containers, from images which are not ISOs but Docker >> images. It?s described here: _https://wiki.openstack.org/wiki/Docker_ >> > >> > >> > >> > Advantages: >> > >> > + Launches faster than VM instances >> > >> > + Less resources used from quotas in FIWARE nodes >> > >> > + images of SEs smaller and easier to build (using Dockerfiles) >> > >> > >> > >> > Open questions / disadvantages: >> > >> > - Will it be possible to pull images from DockerHub (preferred >> approach), or do they need to be uploaded to the Glance image >> repository of each FIWARE node? >> > >> > - Is the only way to launch containers the Cloud Portal or >> OpenStack API? Not being able to target a Docker API would reduce >> the value of Docker and its ecosystem of tools, e.g.: docker-compose >> which enables launching applications composed of multiple containers. >> > >> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >> reverse proxy that forwards HTTP requests to container1.john- >> smith.cloud.fiware.org to the right container port? >> > >> > - How would links between containers running on different hosts >> work? Using Weave? >> > >> > - Mounting volumes in Docker is a very common way of passing >> configuration files, having persistence, etc. How would this work > inOpenStack? >> > >> > >> > >> > Our suggestion: >> > >> > * Ability to use the Docker API remotely (i.e. command-line tools >> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >> > >> > * Ability to pull containers directly from DockerHub with a common >> (transparent?) image mirror in each FIWARE node >> > >> > >> > >> > >> > >> > >> > >> > Regarding FIC2Lab runner, our wishlist was: >> > >> > 1. ability to use the same tools locally and remotely in any cloud >> (ideally use docker-machine against any cloud provider, in >> particular FIWARE nodes) >> > >> > 2. nice web user interface hiding the complexity of firewalling, >> security pairs, public IPs, etc. >> > >> > 3. compatibility with the tools of the docker ecosystem: docker >> CLI, compose, swarm ... >> > >> > >> > >> > We compared tools such as tutum (free multi-tenant online >> service), shipyard (open-source web UI) and panamax (open-source web >> UI and orchestrator). >> > >> > And we decided to go for Panamax, which is nice for #1 and #2 but >> not #3 so far. Here's a few pluses and minuses: >> > >> > + nice UI with DockerHub search >> > >> > + composite application, with application template search >> > >> > - It's not multi-tenant, which is good for portability but bad for >> software updates >> > >> > - its model for composite applications is different from docker- >> compose (for now) >> > >> > - the UI can't synchronize with actions performed in Docker CLI or >> API because it's using CoreOS underneath >> > >> > >> > >> > The FIC2Lab runner is documented here: _http://fic2.github.io/runner_ >> > >> > >> >> >> >> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >> > Dear Alex, >> > >> > It's fine with me. >> > >> > I believe that we should collocate here the discussion with >> > FI-Content2 (Philipp) regarding the stuff they have developed for the >> > FIC2-Lab. >> > >> > @Philipp: could you share some material (document describing your work >> > on FIC2-Lab, slides, whatever) describing what you have done and send it >> > to the fiware-chapter-architects mailing list as preparation for the >> > discussion? >> > >> > @Alex: please also share the ideas you want to present on this matter. >> > >> > By exchanging material prior to the meeting, we would be able to have >> > a more fruitful meeting. >> > >> > Best regards, >> > >> > -- Juanjo >> > >> > On 18/05/15 11:07, Alex Glikson wrote: >> >> I am out of office on the 25th. Would the following Monday work? >> >> >> >> Thanks, >> >> Alex >> >> >> >> >> >> >> >> >> >> >> >> >> >> From: Juanjo Hierro >> >> To: Philipp Slusallek , Alex >> >> Glikson/Haifa/IBM at IBMIL, >> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >> >> Date: 18/05/2015 11:53 AM >> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & > Docker >> >> > ------------------------------------------------------------------------ >> >> >> >> >> >> >> >> >> >> I would suggest that we book the next architecture session (Monday >> >> 25th) to cover this matter. >> >> >> >> Will you Alex be able to prepare a presentation? >> >> >> >> Cheers, >> >> >> >> -- Juanjo >> >> >> >> On 18/05/15 09:46, Philipp Slusallek wrote: >> >> > Hi, >> >> > >> >> > This sounds very useful to cover as well in this or one of the next >> >> > calls. It would also get the architectural activities started > again. It >> >> > might even make sense to invite (selected?) AB members from the UCs. >> >> > >> >> > For example, a lot of work that has been done within FIcontent to > deploy >> >> > our SEs and related GEs within Docker on top of FIWARE Lab. > Particularly >> >> > relevant could be the on-click deployment of entire arrangements > of GEs >> >> > and SEs. Better and more dedicated Docker support would certainly be >> >> > very welcome as well. >> >> > >> >> > Unfortunately, I have a lecture this morning and cannot join > until 12h. >> >> > But Stefan Lemme from my group will be on the call. He has been a > core >> >> > member of the relevant FIC2-Lab task force in FIcontent. >> >> > >> >> > >> >> > Best, >> >> > >> >> > Philipp >> >> > >> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >> >> >> Maybe we can also dedicate some time to have an initial > discussion on >> >> >> the roadmap to adopt Linux Containers and in particular Docker in >> >> FIWARE >> >> >> Lab. >> >> >> >> >> >> Regards, >> >> >> Alex >> >> >>_______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: philipp_slusallek.vcf Type: text/x-vcard Size: 441 bytes Desc: not available URL: From GLIKSON at il.ibm.com Mon Jun 1 17:08:12 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Mon, 1 Jun 2015 18:08:12 +0300 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: References: <5569EA0F.7010900@dfki.de> <504347CB-229B-4483-A453-A5565124AAFC@telefonica.com> Message-ID: Regarding the phone call I mentioned below -- we already have the weekly call of the Cloud chapter on Wed 15:00-16:00 CET (with many overlapping participants), so I suggest that we shorten the regular call to 15:30-16:00, and dedicate 14:00-15:30 CET to this topic. Please, let me know if you can make it. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Alex Glikson/Haifa/IBM at IBMIL To: "fiware-cloud-containers at lists.fiware.org" Date: 01/06/2015 09:31 AM Subject: Re: [Fiware-cloud-containers] task-force kickoff Sent by: fiware-cloud-containers-bounces at lists.fiware.org We will hold the taskforce kickoff at a different time slot, to accommodate availability of key people. I will send a doodle later today. Meanwhile, we can try making a bit progress by email, following the summary I sent in a separate email. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Alex Glikson/Haifa/IBM at IBMIL To: "fiware-cloud-containers at lists.fiware.org" Date: 31/05/2015 10:59 PM Subject: Re: [Fiware-cloud-containers] task-force kickoff Sent by: fiware-cloud-containers-bounces at lists.fiware.org Before this task force has launched, discussion on containers has been scheduled for tomorrow, 10:30. Now, maybe it would make sense to make some progress within the task force before raising this with the (rest of the) architects - especially if some of the participants can't join tomorrow. Regards, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: FERNANDO LOPEZ AGUILAR To: JUAN JOSE HIERRO SUREDA , Alex Glikson/Haifa/IBM at IBMIL Cc: "fiware-cloud-containers at lists.fiware.org" Date: 31/05/2015 10:03 PM Subject: Re: [Fiware-cloud-containers] task-force kickoff Ok, then the question is we will have the kickoff of this task force tomorrow? And if it is true between which hours? We have tomorrow our internal sprint planning for the next sprint from 9:00 to 13:00. Fernando.- De: JUAN JOSE HIERRO SUREDA Fecha: Saturday 30 May 2015 20:31 Para: Alex Glikson CC: "fiware-cloud-containers at lists.fiware.org" < fiware-cloud-containers at lists.fiware.org> Asunto: Re: [Fiware-cloud-containers] task-force kickoff That's what I understood too Juanjo from iPhone El 30/5/2015, a las 19:52, Alex Glikson escribi?: I am pretty sure that the monthly review/demo meeting has been rescheduled, so it seems that the morning slot on the 1st is free.. ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Philipp Slusallek To: Alex Glikson/Haifa/IBM at IBMIL, fiware-cloud-containers at lists.fiware.org Date: 30/05/2015 07:49 PM Subject: Re: [Fiware-cloud-containers] task-force kickoff Hi, I thought the June 1 meetings will be moved. But I would join anyway. Best, Philipp Am 30.05.2015 um 18:06 schrieb Alex Glikson: > Dear all, > > Welcome to 'fiware-cloud-containers' mailing list! We will use this > mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. > support for Linux containers (such as Docker), including use-cases & > requirements, associated IaaS and PaaS-style capabilities, concrete > plans going forwards, potential collaboration with teams working on > similar/related technologies, etc. > > Currently we have the following people subscribed to this mailing list: > IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for > IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud > leader), Doron (working with Ezra on containers) > TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads > for the PaaS-related work in FIWARE Cloud) > DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter > leader/architect) > Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old > FI-WARE project) > > Let me know if anyone is missing. > > As a background, please, read the thread below, including the very nice > summary by Mario on the related investigation and implementation in > FI-Content2 project. I will send a separate email briefly summarizing > the situation in FIWARE. > > *All *-- notice that we are going to have a phone call to discuss this > topic during the upcoming FIWARE architects interlock on Monday, June > 1at, 10:30-12:30 CET. Let me know if you can't join. > *Mario *-- would you be able to join & explain/demonstrate the FIC2Lab > solution? > > Thanks, > Alex > > > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- > > fiware-chapter-architects-bounces at lists.fi-ware.org wrote on 26/05/2015 > 08:37:16 AM: > >> From: Alex Glikson/Haifa/IBM at IBMIL >> To: Philipp Slusallek >> Cc: fiware-chapter-architects at lists.fi-ware.org, Kenneth Nagin/ >> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >> Date: 26/05/2015 08:37 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> Sent by: fiware-chapter-architects-bounces at lists.fi-ware.org >> >> Hi Philipp, >> >> It is great to hear that FIC2 team is willing to contribute! >> In a nutshell, our current plan is to start with surfacing >> containers via Nova, and then gradually migrate to Magnum. In terms >> of orchestration, we currently work with Heat and Murano, and are >> assessing the approaches to support containers (OpenStack alignment >> being high priority, but also the ability to surface capabilities >> unique to containers). >> I suggest that we form a small taskforce to explore this topic. >> Please, send me a list of people to involve. >> >> Thanks, >> Alex >> >> > ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> >> >> >> From: Philipp Slusallek >> To: Juanjo Hierro , Alex >> Glikson/Haifa/IBM at IBMIL >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >> architects at lists.fi-ware.org, Kenneth Nagin/Haifa/IBM at IBMIL >> Date: 26/05/2015 07:21 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> >> >> >> Hi, >> >> Here is some first input from FIC2 on Docker compiled from Mario. As I >> mentioned before, we have a full system working that deploys entire sets >> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >> >> FIC2 would be happy to work with FIWARE to integrate them in whatever >> way we at FIWARE decide to implement Docker. Right now we deploy it on >> top of the VMs offered by FIWARE but a more native implementation would >> be fine as well and save much resources (which seems to be a major issue). >> >> A presentation by Canonical recently showed a >10x improvement in >> resource usage with their containers over plain OpenStack (which is >> probably where we are -- but then Intel claims even better numbers in >> their Clear Containers solution using traditional VMs). >> >> Since we have experience already, it seems very useful to make Docker >> available especially since deployment of SW is much easier too and the >> same SW can be deployed essentially anywhere (including a local machine). >> >> >> Best, >> >> Philipp >> >> > There are several ways in which FIWARE could support Docker. My >> assumption is that the plan is to add to OpenStack a Docker driver. >> It would enable the creation of instances which are not VMs but >> OpenStack containers, from images which are not ISOs but Docker >> images. It?s described here: https://wiki.openstack.org/wiki/Docker >> > >> > >> > >> > Advantages: >> > >> > + Launches faster than VM instances >> > >> > + Less resources used from quotas in FIWARE nodes >> > >> > + images of SEs smaller and easier to build (using Dockerfiles) >> > >> > >> > >> > Open questions / disadvantages: >> > >> > - Will it be possible to pull images from DockerHub (preferred >> approach), or do they need to be uploaded to the Glance image >> repository of each FIWARE node? >> > >> > - Is the only way to launch containers the Cloud Portal or >> OpenStack API? Not being able to target a Docker API would reduce >> the value of Docker and its ecosystem of tools, e.g.: docker-compose >> which enables launching applications composed of multiple containers. >> > >> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >> reverse proxy that forwards HTTP requests to container1.john- >> smith.cloud.fiware.org to the right container port? >> > >> > - How would links between containers running on different hosts >> work? Using Weave? >> > >> > - Mounting volumes in Docker is a very common way of passing >> configuration files, having persistence, etc. How would this work > inOpenStack? >> > >> > >> > >> > Our suggestion: >> > >> > * Ability to use the Docker API remotely (i.e. command-line tools >> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >> > >> > * Ability to pull containers directly from DockerHub with a common >> (transparent?) image mirror in each FIWARE node >> > >> > >> > >> > >> > >> > >> > >> > Regarding FIC2Lab runner, our wishlist was: >> > >> > 1. ability to use the same tools locally and remotely in any cloud >> (ideally use docker-machine against any cloud provider, in >> particular FIWARE nodes) >> > >> > 2. nice web user interface hiding the complexity of firewalling, >> security pairs, public IPs, etc. >> > >> > 3. compatibility with the tools of the docker ecosystem: docker >> CLI, compose, swarm ... >> > >> > >> > >> > We compared tools such as tutum (free multi-tenant online >> service), shipyard (open-source web UI) and panamax (open-source web >> UI and orchestrator). >> > >> > And we decided to go for Panamax, which is nice for #1 and #2 but >> not #3 so far. Here's a few pluses and minuses: >> > >> > + nice UI with DockerHub search >> > >> > + composite application, with application template search >> > >> > - It's not multi-tenant, which is good for portability but bad for >> software updates >> > >> > - its model for composite applications is different from docker- >> compose (for now) >> > >> > - the UI can't synchronize with actions performed in Docker CLI or >> API because it's using CoreOS underneath >> > >> > >> > >> > The FIC2Lab runner is documented here: http://fic2.github.io/runner >> > >> > >> >> >> >> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >> > Dear Alex, >> > >> > It's fine with me. >> > >> > I believe that we should collocate here the discussion with >> > FI-Content2 (Philipp) regarding the stuff they have developed for the >> > FIC2-Lab. >> > >> > @Philipp: could you share some material (document describing your work >> > on FIC2-Lab, slides, whatever) describing what you have done and send it >> > to the fiware-chapter-architects mailing list as preparation for the >> > discussion? >> > >> > @Alex: please also share the ideas you want to present on this matter. >> > >> > By exchanging material prior to the meeting, we would be able to have >> > a more fruitful meeting. >> > >> > Best regards, >> > >> > -- Juanjo >> > >> > On 18/05/15 11:07, Alex Glikson wrote: >> >> I am out of office on the 25th. Would the following Monday work? >> >> >> >> Thanks, >> >> Alex >> >> >> >> >> >> >> >> >> >> >> >> >> >> From: Juanjo Hierro >> >> To: Philipp Slusallek , Alex >> >> Glikson/Haifa/IBM at IBMIL, < fiware-chapter-architects at lists.fi-ware.org> >> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >> >> Date: 18/05/2015 11:53 AM >> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & > Docker >> >> > ------------------------------------------------------------------------ >> >> >> >> >> >> >> >> >> >> I would suggest that we book the next architecture session (Monday >> >> 25th) to cover this matter. >> >> >> >> Will you Alex be able to prepare a presentation? >> >> >> >> Cheers, >> >> >> >> -- Juanjo >> >> >> >> On 18/05/15 09:46, Philipp Slusallek wrote: >> >> > Hi, >> >> > >> >> > This sounds very useful to cover as well in this or one of the next >> >> > calls. It would also get the architectural activities started > again. It >> >> > might even make sense to invite (selected?) AB members from the UCs. >> >> > >> >> > For example, a lot of work that has been done within FIcontent to > deploy >> >> > our SEs and related GEs within Docker on top of FIWARE Lab. > Particularly >> >> > relevant could be the on-click deployment of entire arrangements > of GEs >> >> > and SEs. Better and more dedicated Docker support would certainly be >> >> > very welcome as well. >> >> > >> >> > Unfortunately, I have a lecture this morning and cannot join > until 12h. >> >> > But Stefan Lemme from my group will be on the call. He has been a > core >> >> > member of the relevant FIC2-Lab task force in FIcontent. >> >> > >> >> > >> >> > Best, >> >> > >> >> > Philipp >> >> > >> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >> >> >> Maybe we can also dedicate some time to have an initial > discussion on >> >> >> the roadmap to adopt Linux Containers and in particular Docker in >> >> FIWARE >> >> >> Lab. >> >> >> >> >> >> Regards, >> >> >> Alex >> >> >> > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers -------------- next part -------------- An HTML attachment was scrubbed... URL: From philipp.slusallek at dfki.de Mon Jun 1 17:13:48 2015 From: philipp.slusallek at dfki.de (Philipp Slusallek) Date: Mon, 01 Jun 2015 17:13:48 +0200 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: References: <5569EA0F.7010900@dfki.de> <504347CB-229B-4483-A453-A5565124AAFC@telefonica.com> Message-ID: <556C76AC.704@dfki.de> Hi, Yes, I can move another meeting. Best, Philipp Am 01.06.2015 um 17:08 schrieb Alex Glikson: > Regarding the phone call I mentioned below -- we already have the weekly > call of the Cloud chapter on Wed 15:00-16:00 CET (with many overlapping > participants), so I suggest that we shorten the regular call to > 15:30-16:00, and dedicate 14:00-15:30 CET to this topic. > Please, let me know if you can make it. > > Thanks, > Alex > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > > > > From: Alex Glikson/Haifa/IBM at IBMIL > To: "fiware-cloud-containers at lists.fiware.org" > > Date: 01/06/2015 09:31 AM > Subject: Re: [Fiware-cloud-containers] task-force kickoff > Sent by: fiware-cloud-containers-bounces at lists.fiware.org > ------------------------------------------------------------------------ > > > > We will hold the taskforce kickoff at a different time slot, to > accommodate availability of key people. I will send a doodle later > today. Meanwhile, we can try making a bit progress by email, following > the summary I sent in a separate email. > > Thanks, > Alex > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > > > > From: Alex Glikson/Haifa/IBM at IBMIL > To: "fiware-cloud-containers at lists.fiware.org" > > Date: 31/05/2015 10:59 PM > Subject: Re: [Fiware-cloud-containers] task-force kickoff > Sent by: fiware-cloud-containers-bounces at lists.fiware.org > ------------------------------------------------------------------------ > > > > Before this task force has launched, discussion on containers has been > scheduled for tomorrow, 10:30. > Now, maybe it would make sense to make some progress within the task > force before raising this with the (rest of the) architects - especially > if some of the participants can't join tomorrow. > > Regards, > Alex > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > > > > From: FERNANDO LOPEZ AGUILAR > To: JUAN JOSE HIERRO SUREDA , > Alex Glikson/Haifa/IBM at IBMIL > Cc: "fiware-cloud-containers at lists.fiware.org" > > Date: 31/05/2015 10:03 PM > Subject: Re: [Fiware-cloud-containers] task-force kickoff > ------------------------------------------------------------------------ > > > > Ok, then the question is we will have the kickoff of this task force > tomorrow? And if it is true between which hours? > > We have tomorrow our internal sprint planning for the next sprint from > 9:00 to 13:00. > > Fernando.-* > > De: *JUAN JOSE HIERRO SUREDA <_juanjose.hierro at telefonica.com_ > >* > Fecha: *Saturday 30 May 2015 20:31* > Para: *Alex Glikson <_GLIKSON at il.ibm.com_ >* > CC: *"_fiware-cloud-containers at lists.fiware.org_ > " > <_fiware-cloud-containers at lists.fiware.org_ > >* > Asunto: *Re: [Fiware-cloud-containers] task-force kickoff > > That's what I understood too > > Juanjo from iPhone > > El 30/5/2015, a las 19:52, Alex Glikson <_GLIKSON at il.ibm.com_ > > escribi?: > > > I am pretty sure that the monthly review/demo meeting has been > rescheduled, so it seems that the morning slot on the 1st is free.. > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: _glikson at il.ibm.com_ | Phone: > +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 > > > > > > From: Philipp Slusallek <_philipp.slusallek at dfki.de_ > > > To: Alex Glikson/Haifa/IBM at IBMIL, > _fiware-cloud-containers at lists.fiware.org_ > > Date: 30/05/2015 07:49 PM > Subject: Re: [Fiware-cloud-containers] task-force kickoff > ------------------------------------------------------------------------ > > > > Hi, > > I thought the June 1 meetings will be moved. But I would join anyway. > > Best, > > Philipp > > Am 30.05.2015 um 18:06 schrieb Alex Glikson: >> Dear all, >> >> Welcome to 'fiware-cloud-containers' mailing list! We will use this >> mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. >> support for Linux containers (such as Docker), including use-cases & >> requirements, associated IaaS and PaaS-style capabilities, concrete >> plans going forwards, potential collaboration with teams working on >> similar/related technologies, etc. >> >> Currently we have the following people subscribed to this mailing list: >> IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for >> IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud >> leader), Doron (working with Ezra on containers) >> TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads >> for the PaaS-related work in FIWARE Cloud) >> DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter >> leader/architect) >> Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old >> FI-WARE project) >> >> Let me know if anyone is missing. >> >> As a background, please, read the thread below, including the very nice >> summary by Mario on the related investigation and implementation in >> FI-Content2 project. I will send a separate email briefly summarizing >> the situation in FIWARE. >> >> *All *-- notice that we are going to have a phone call to discuss this >> topic during the upcoming FIWARE architects interlock on Monday, June >> 1at, 10:30-12:30 CET. Let me know if you can't join. >> *Mario *-- would you be able to join & explain/demonstrate the FIC2Lab >> solution? >> >> Thanks, >> Alex >> >> >> >> > ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: _glikson at il.ibm.com_ | Phone: > +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- >> >> _fiware-chapter-architects-bounces at lists.fi-ware.org_ > wrote on > 26/05/2015 >> 08:37:16 AM: >> >>> From: Alex Glikson/Haifa/IBM at IBMIL >>> To: Philipp Slusallek <_philipp.slusallek at dfki.de_ > > >>> Cc: _fiware-chapter-architects at lists.fi-ware.org_ > , Kenneth Nagin/ >>> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >>> Date: 26/05/2015 08:37 AM >>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >>> Sent by: _fiware-chapter-architects-bounces at lists.fi-ware.org_ > >>> >>> Hi Philipp, >>> >>> It is great to hear that FIC2 team is willing to contribute! >>> In a nutshell, our current plan is to start with surfacing >>> containers via Nova, and then gradually migrate to Magnum. In terms >>> of orchestration, we currently work with Heat and Murano, and are >>> assessing the approaches to support containers (OpenStack alignment >>> being high priority, but also the ability to surface capabilities >>> unique to containers). >>> I suggest that we form a small taskforce to explore this topic. >>> Please, send me a list of people to involve. >>> >>> Thanks, >>> Alex >>> >>> >> > ==================================================================================== >>> Alex Glikson >>> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >>> Email: _glikson at il.ibm.com_ | Phone: > +972-4-8281085 | Mobile: >>> +972-54-6466667 | Fax: +972-4-8296112 >>> >>> >>> >>> >>> >>> From: Philipp Slusallek <_philipp.slusallek at dfki.de_ > > >>> To: Juanjo Hierro <_juanjose.hierro at telefonica.com_ > >, Alex >>> Glikson/Haifa/IBM at IBMIL >>> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >>> _architects at lists.fi-ware.org_ , > Kenneth Nagin/Haifa/IBM at IBMIL >>> Date: 26/05/2015 07:21 AM >>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >>> >>> >>> >>> Hi, >>> >>> Here is some first input from FIC2 on Docker compiled from Mario. As I >>> mentioned before, we have a full system working that deploys entire sets >>> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >>> >>> FIC2 would be happy to work with FIWARE to integrate them in whatever >>> way we at FIWARE decide to implement Docker. Right now we deploy it on >>> top of the VMs offered by FIWARE but a more native implementation would >>> be fine as well and save much resources (which seems to be a major > issue). >>> >>> A presentation by Canonical recently showed a >10x improvement in >>> resource usage with their containers over plain OpenStack (which is >>> probably where we are -- but then Intel claims even better numbers in >>> their Clear Containers solution using traditional VMs). >>> >>> Since we have experience already, it seems very useful to make Docker >>> available especially since deployment of SW is much easier too and the >>> same SW can be deployed essentially anywhere (including a local machine). >>> >>> >>> Best, >>> >>> Philipp >>> >>> > There are several ways in which FIWARE could support Docker. My >>> assumption is that the plan is to add to OpenStack a Docker driver. >>> It would enable the creation of instances which are not VMs but >>> OpenStack containers, from images which are not ISOs but Docker >>> images. It?s described here: _https://wiki.openstack.org/wiki/Docker_ >>> > >>> > >>> > >>> > Advantages: >>> > >>> > + Launches faster than VM instances >>> > >>> > + Less resources used from quotas in FIWARE nodes >>> > >>> > + images of SEs smaller and easier to build (using Dockerfiles) >>> > >>> > >>> > >>> > Open questions / disadvantages: >>> > >>> > - Will it be possible to pull images from DockerHub (preferred >>> approach), or do they need to be uploaded to the Glance image >>> repository of each FIWARE node? >>> > >>> > - Is the only way to launch containers the Cloud Portal or >>> OpenStack API? Not being able to target a Docker API would reduce >>> the value of Docker and its ecosystem of tools, e.g.: docker-compose >>> which enables launching applications composed of multiple containers. >>> > >>> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >>> reverse proxy that forwards HTTP requests to container1.john- >>> _smith.cloud.fiware.org_ to the right > container port? >>> > >>> > - How would links between containers running on different hosts >>> work? Using Weave? >>> > >>> > - Mounting volumes in Docker is a very common way of passing >>> configuration files, having persistence, etc. How would this work >> inOpenStack? >>> > >>> > >>> > >>> > Our suggestion: >>> > >>> > * Ability to use the Docker API remotely (i.e. command-line tools >>> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >>> > >>> > * Ability to pull containers directly from DockerHub with a common >>> (transparent?) image mirror in each FIWARE node >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > Regarding FIC2Lab runner, our wishlist was: >>> > >>> > 1. ability to use the same tools locally and remotely in any cloud >>> (ideally use docker-machine against any cloud provider, in >>> particular FIWARE nodes) >>> > >>> > 2. nice web user interface hiding the complexity of firewalling, >>> security pairs, public IPs, etc. >>> > >>> > 3. compatibility with the tools of the docker ecosystem: docker >>> CLI, compose, swarm ... >>> > >>> > >>> > >>> > We compared tools such as tutum (free multi-tenant online >>> service), shipyard (open-source web UI) and panamax (open-source web >>> UI and orchestrator). >>> > >>> > And we decided to go for Panamax, which is nice for #1 and #2 but >>> not #3 so far. Here's a few pluses and minuses: >>> > >>> > + nice UI with DockerHub search >>> > >>> > + composite application, with application template search >>> > >>> > - It's not multi-tenant, which is good for portability but bad for >>> software updates >>> > >>> > - its model for composite applications is different from docker- >>> compose (for now) >>> > >>> > - the UI can't synchronize with actions performed in Docker CLI or >>> API because it's using CoreOS underneath >>> > >>> > >>> > >>> > The FIC2Lab runner is documented here: _http://fic2.github.io/runner_ >>> > >>> > >>> >>> >>> >>> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >>> > Dear Alex, >>> > >>> > It's fine with me. >>> > >>> > I believe that we should collocate here the discussion with >>> > FI-Content2 (Philipp) regarding the stuff they have developed for the >>> > FIC2-Lab. >>> > >>> > @Philipp: could you share some material (document describing your > work >>> > on FIC2-Lab, slides, whatever) describing what you have done and > send it >>> > to the fiware-chapter-architects mailing list as preparation for the >>> > discussion? >>> > >>> > @Alex: please also share the ideas you want to present on this > matter. >>> > >>> > By exchanging material prior to the meeting, we would be able to have >>> > a more fruitful meeting. >>> > >>> > Best regards, >>> > >>> > -- Juanjo >>> > >>> > On 18/05/15 11:07, Alex Glikson wrote: >>> >> I am out of office on the 25th. Would the following Monday work? >>> >> >>> >> Thanks, >>> >> Alex >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> From: Juanjo Hierro <_juanjose.hierro at telefonica.com_ > > >>> >> To: Philipp Slusallek <_philipp.slusallek at dfki.de_ > >, Alex >>> >> Glikson/Haifa/IBM at IBMIL, > <_fiware-chapter-architects at lists.fi-ware.org_ > > >>> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >>> >> Date: 18/05/2015 11:53 AM >>> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & >> Docker >>> >> >> ------------------------------------------------------------------------ >>> >> >>> >> >>> >> >>> >> >>> >> I would suggest that we book the next architecture session (Monday >>> >> 25th) to cover this matter. >>> >> >>> >> Will you Alex be able to prepare a presentation? >>> >> >>> >> Cheers, >>> >> >>> >> -- Juanjo >>> >> >>> >> On 18/05/15 09:46, Philipp Slusallek wrote: >>> >> > Hi, >>> >> > >>> >> > This sounds very useful to cover as well in this or one of the next >>> >> > calls. It would also get the architectural activities started >> again. It >>> >> > might even make sense to invite (selected?) AB members from the UCs. >>> >> > >>> >> > For example, a lot of work that has been done within FIcontent to >> deploy >>> >> > our SEs and related GEs within Docker on top of FIWARE Lab. >> Particularly >>> >> > relevant could be the on-click deployment of entire arrangements >> of GEs >>> >> > and SEs. Better and more dedicated Docker support would certainly be >>> >> > very welcome as well. >>> >> > >>> >> > Unfortunately, I have a lecture this morning and cannot join >> until 12h. >>> >> > But Stefan Lemme from my group will be on the call. He has been a >> core >>> >> > member of the relevant FIC2-Lab task force in FIcontent. >>> >> > >>> >> > >>> >> > Best, >>> >> > >>> >> > Philipp >>> >> > >>> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >>> >> >> Maybe we can also dedicate some time to have an initial >> discussion on >>> >> >> the roadmap to adopt Linux Containers and in particular Docker in >>> >> FIWARE >>> >> >> Lab. >>> >> >> >>> >> >> Regards, >>> >> >> Alex >>> >> >> >> >> >> _______________________________________________ >> Fiware-cloud-containers mailing list >> _Fiware-cloud-containers at lists.fiware.org_ > >> _https://lists.fiware.org/listinfo/fiware-cloud-containers_ >> > > -- > > ------------------------------------------------------------------------- > Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH > Trippstadter Strasse 122, D-67663 Kaiserslautern > > Gesch?ftsf?hrung: > Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) > Dr. Walter Olthoff > Vorsitzender des Aufsichtsrats: > Prof. Dr. h.c. Hans A. Aukes > > Sitz der Gesellschaft: Kaiserslautern (HRB 2313) > USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 > --------------------------------------------------------------------------- > [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] > _______________________________________________ > Fiware-cloud-containers mailing list_ > __Fiware-cloud-containers at lists.fiware.org_ > _ > __https://lists.fiware.org/listinfo/fiware-cloud-containers_ > > ------------------------------------------------------------------------ > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener informaci?n privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilizaci?n, > divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud > de la legislaci?n vigente. Si ha recibido este mensaje por error, le > rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda > a su destrucci?n. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution > or copying of this communication is strictly prohibited. If you have > received this transmission in error, do not read it. Please immediately > reply to the sender that you have received this communication in error > and then delete it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu > destinat?rio, pode conter informa??o privilegiada ou confidencial e ? > para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa > senhoria o destinat?rio indicado, fica notificado de que a leitura, > utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em > virtude da legisla??o vigente. Se recebeu esta mensagem por erro, > rogamos-lhe que nos o comunique imediatamente por esta mesma via e > proceda a sua destrui??o > > ------------------------------------------------------------------------ > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener informaci?n privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilizaci?n, > divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud > de la legislaci?n vigente. Si ha recibido este mensaje por error, le > rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda > a su destrucci?n. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution > or copying of this communication is strictly prohibited. If you have > received this transmission in error, do not read it. Please immediately > reply to the sender that you have received this communication in error > and then delete it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu > destinat?rio, pode conter informa??o privilegiada ou confidencial e ? > para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa > senhoria o destinat?rio indicado, fica notificado de que a leitura, > utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em > virtude da legisla??o vigente. Se recebeu esta mensagem por erro, > rogamos-lhe que nos o comunique imediatamente por esta mesma via e > proceda a sua destrui??o_______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org_ > __https://lists.fiware.org/listinfo/fiware-cloud-containers_ > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: philipp_slusallek.vcf Type: text/x-vcard Size: 441 bytes Desc: not available URL: From mario.lopezramos at thalesgroup.com Mon Jun 1 17:23:29 2015 From: mario.lopezramos at thalesgroup.com (LOPEZ RAMOS Mario) Date: Mon, 1 Jun 2015 17:23:29 +0200 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: References: <5569EA0F.7010900@dfki.de> <504347CB-229B-4483-A453-A5565124AAFC@telefonica.com> Message-ID: Hi Alex and all, Sorry, I have another meeting Wed 14:00-15:30. I can make it before or after that though. Best, Mario De : fiware-cloud-containers-bounces at lists.fiware.org [mailto:fiware-cloud-containers-bounces at lists.fiware.org] De la part de Alex Glikson Envoy? : lundi 1 juin 2015 17:08 ? : fiware-cloud-containers at lists.fiware.org Objet : Re: [Fiware-cloud-containers] task-force kickoff Regarding the phone call I mentioned below -- we already have the weekly call of the Cloud chapter on Wed 15:00-16:00 CET (with many overlapping participants), so I suggest that we shorten the regular call to 15:30-16:00, and dedicate 14:00-15:30 CET to this topic. Please, let me know if you can make it. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Alex Glikson/Haifa/IBM at IBMIL To: "fiware-cloud-containers at lists.fiware.org" > Date: 01/06/2015 09:31 AM Subject: Re: [Fiware-cloud-containers] task-force kickoff Sent by: fiware-cloud-containers-bounces at lists.fiware.org ________________________________ We will hold the taskforce kickoff at a different time slot, to accommodate availability of key people. I will send a doodle later today. Meanwhile, we can try making a bit progress by email, following the summary I sent in a separate email. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Alex Glikson/Haifa/IBM at IBMIL To: "fiware-cloud-containers at lists.fiware.org" > Date: 31/05/2015 10:59 PM Subject: Re: [Fiware-cloud-containers] task-force kickoff Sent by: fiware-cloud-containers-bounces at lists.fiware.org ________________________________ Before this task force has launched, discussion on containers has been scheduled for tomorrow, 10:30. Now, maybe it would make sense to make some progress within the task force before raising this with the (rest of the) architects - especially if some of the participants can't join tomorrow. Regards, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: FERNANDO LOPEZ AGUILAR > To: JUAN JOSE HIERRO SUREDA >, Alex Glikson/Haifa/IBM at IBMIL Cc: "fiware-cloud-containers at lists.fiware.org" > Date: 31/05/2015 10:03 PM Subject: Re: [Fiware-cloud-containers] task-force kickoff ________________________________ Ok, then the question is we will have the kickoff of this task force tomorrow? And if it is true between which hours? We have tomorrow our internal sprint planning for the next sprint from 9:00 to 13:00. Fernando.- De: JUAN JOSE HIERRO SUREDA > Fecha: Saturday 30 May 2015 20:31 Para: Alex Glikson > CC: "fiware-cloud-containers at lists.fiware.org" > Asunto: Re: [Fiware-cloud-containers] task-force kickoff That's what I understood too Juanjo from iPhone El 30/5/2015, a las 19:52, Alex Glikson > escribi?: I am pretty sure that the monthly review/demo meeting has been rescheduled, so it seems that the morning slot on the 1st is free.. ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Philipp Slusallek > To: Alex Glikson/Haifa/IBM at IBMIL, fiware-cloud-containers at lists.fiware.org Date: 30/05/2015 07:49 PM Subject: Re: [Fiware-cloud-containers] task-force kickoff ________________________________ Hi, I thought the June 1 meetings will be moved. But I would join anyway. Best, Philipp Am 30.05.2015 um 18:06 schrieb Alex Glikson: > Dear all, > > Welcome to 'fiware-cloud-containers' mailing list! We will use this > mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. > support for Linux containers (such as Docker), including use-cases & > requirements, associated IaaS and PaaS-style capabilities, concrete > plans going forwards, potential collaboration with teams working on > similar/related technologies, etc. > > Currently we have the following people subscribed to this mailing list: > IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for > IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud > leader), Doron (working with Ezra on containers) > TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads > for the PaaS-related work in FIWARE Cloud) > DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter > leader/architect) > Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old > FI-WARE project) > > Let me know if anyone is missing. > > As a background, please, read the thread below, including the very nice > summary by Mario on the related investigation and implementation in > FI-Content2 project. I will send a separate email briefly summarizing > the situation in FIWARE. > > *All *-- notice that we are going to have a phone call to discuss this > topic during the upcoming FIWARE architects interlock on Monday, June > 1at, 10:30-12:30 CET. Let me know if you can't join. > *Mario *-- would you be able to join & explain/demonstrate the FIC2Lab > solution? > > Thanks, > Alex > > > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- > > fiware-chapter-architects-bounces at lists.fi-ware.org wrote on 26/05/2015 > 08:37:16 AM: > >> From: Alex Glikson/Haifa/IBM at IBMIL >> To: Philipp Slusallek > >> Cc: fiware-chapter-architects at lists.fi-ware.org, Kenneth Nagin/ >> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >> Date: 26/05/2015 08:37 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> Sent by: fiware-chapter-architects-bounces at lists.fi-ware.org >> >> Hi Philipp, >> >> It is great to hear that FIC2 team is willing to contribute! >> In a nutshell, our current plan is to start with surfacing >> containers via Nova, and then gradually migrate to Magnum. In terms >> of orchestration, we currently work with Heat and Murano, and are >> assessing the approaches to support containers (OpenStack alignment >> being high priority, but also the ability to surface capabilities >> unique to containers). >> I suggest that we form a small taskforce to explore this topic. >> Please, send me a list of people to involve. >> >> Thanks, >> Alex >> >> > ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> >> >> >> From: Philipp Slusallek > >> To: Juanjo Hierro >, Alex >> Glikson/Haifa/IBM at IBMIL >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >> architects at lists.fi-ware.org, Kenneth Nagin/Haifa/IBM at IBMIL >> Date: 26/05/2015 07:21 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> >> >> >> Hi, >> >> Here is some first input from FIC2 on Docker compiled from Mario. As I >> mentioned before, we have a full system working that deploys entire sets >> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >> >> FIC2 would be happy to work with FIWARE to integrate them in whatever >> way we at FIWARE decide to implement Docker. Right now we deploy it on >> top of the VMs offered by FIWARE but a more native implementation would >> be fine as well and save much resources (which seems to be a major issue). >> >> A presentation by Canonical recently showed a >10x improvement in >> resource usage with their containers over plain OpenStack (which is >> probably where we are -- but then Intel claims even better numbers in >> their Clear Containers solution using traditional VMs). >> >> Since we have experience already, it seems very useful to make Docker >> available especially since deployment of SW is much easier too and the >> same SW can be deployed essentially anywhere (including a local machine). >> >> >> Best, >> >> Philipp >> >> > There are several ways in which FIWARE could support Docker. My >> assumption is that the plan is to add to OpenStack a Docker driver. >> It would enable the creation of instances which are not VMs but >> OpenStack containers, from images which are not ISOs but Docker >> images. It's described here: https://wiki.openstack.org/wiki/Docker >> > >> > >> > >> > Advantages: >> > >> > + Launches faster than VM instances >> > >> > + Less resources used from quotas in FIWARE nodes >> > >> > + images of SEs smaller and easier to build (using Dockerfiles) >> > >> > >> > >> > Open questions / disadvantages: >> > >> > - Will it be possible to pull images from DockerHub (preferred >> approach), or do they need to be uploaded to the Glance image >> repository of each FIWARE node? >> > >> > - Is the only way to launch containers the Cloud Portal or >> OpenStack API? Not being able to target a Docker API would reduce >> the value of Docker and its ecosystem of tools, e.g.: docker-compose >> which enables launching applications composed of multiple containers. >> > >> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >> reverse proxy that forwards HTTP requests to container1.john- >> smith.cloud.fiware.org to the right container port? >> > >> > - How would links between containers running on different hosts >> work? Using Weave? >> > >> > - Mounting volumes in Docker is a very common way of passing >> configuration files, having persistence, etc. How would this work > inOpenStack? >> > >> > >> > >> > Our suggestion: >> > >> > * Ability to use the Docker API remotely (i.e. command-line tools >> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >> > >> > * Ability to pull containers directly from DockerHub with a common >> (transparent?) image mirror in each FIWARE node >> > >> > >> > >> > >> > >> > >> > >> > Regarding FIC2Lab runner, our wishlist was: >> > >> > 1. ability to use the same tools locally and remotely in any cloud >> (ideally use docker-machine against any cloud provider, in >> particular FIWARE nodes) >> > >> > 2. nice web user interface hiding the complexity of firewalling, >> security pairs, public IPs, etc. >> > >> > 3. compatibility with the tools of the docker ecosystem: docker >> CLI, compose, swarm ... >> > >> > >> > >> > We compared tools such as tutum (free multi-tenant online >> service), shipyard (open-source web UI) and panamax (open-source web >> UI and orchestrator). >> > >> > And we decided to go for Panamax, which is nice for #1 and #2 but >> not #3 so far. Here's a few pluses and minuses: >> > >> > + nice UI with DockerHub search >> > >> > + composite application, with application template search >> > >> > - It's not multi-tenant, which is good for portability but bad for >> software updates >> > >> > - its model for composite applications is different from docker- >> compose (for now) >> > >> > - the UI can't synchronize with actions performed in Docker CLI or >> API because it's using CoreOS underneath >> > >> > >> > >> > The FIC2Lab runner is documented here: http://fic2.github.io/runner >> > >> > >> >> >> >> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >> > Dear Alex, >> > >> > It's fine with me. >> > >> > I believe that we should collocate here the discussion with >> > FI-Content2 (Philipp) regarding the stuff they have developed for the >> > FIC2-Lab. >> > >> > @Philipp: could you share some material (document describing your work >> > on FIC2-Lab, slides, whatever) describing what you have done and send it >> > to the fiware-chapter-architects mailing list as preparation for the >> > discussion? >> > >> > @Alex: please also share the ideas you want to present on this matter. >> > >> > By exchanging material prior to the meeting, we would be able to have >> > a more fruitful meeting. >> > >> > Best regards, >> > >> > -- Juanjo >> > >> > On 18/05/15 11:07, Alex Glikson wrote: >> >> I am out of office on the 25th. Would the following Monday work? >> >> >> >> Thanks, >> >> Alex >> >> >> >> >> >> >> >> >> >> >> >> >> >> From: Juanjo Hierro > >> >> To: Philipp Slusallek >, Alex >> >> Glikson/Haifa/IBM at IBMIL, > >> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >> >> Date: 18/05/2015 11:53 AM >> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & > Docker >> >> > ------------------------------------------------------------------------ >> >> >> >> >> >> >> >> >> >> I would suggest that we book the next architecture session (Monday >> >> 25th) to cover this matter. >> >> >> >> Will you Alex be able to prepare a presentation? >> >> >> >> Cheers, >> >> >> >> -- Juanjo >> >> >> >> On 18/05/15 09:46, Philipp Slusallek wrote: >> >> > Hi, >> >> > >> >> > This sounds very useful to cover as well in this or one of the next >> >> > calls. It would also get the architectural activities started > again. It >> >> > might even make sense to invite (selected?) AB members from the UCs. >> >> > >> >> > For example, a lot of work that has been done within FIcontent to > deploy >> >> > our SEs and related GEs within Docker on top of FIWARE Lab. > Particularly >> >> > relevant could be the on-click deployment of entire arrangements > of GEs >> >> > and SEs. Better and more dedicated Docker support would certainly be >> >> > very welcome as well. >> >> > >> >> > Unfortunately, I have a lecture this morning and cannot join > until 12h. >> >> > But Stefan Lemme from my group will be on the call. He has been a > core >> >> > member of the relevant FIC2-Lab task force in FIcontent. >> >> > >> >> > >> >> > Best, >> >> > >> >> > Philipp >> >> > >> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >> >> >> Maybe we can also dedicate some time to have an initial > discussion on >> >> >> the roadmap to adopt Linux Containers and in particular Docker in >> >> FIWARE >> >> >> Lab. >> >> >> >> >> >> Regards, >> >> >> Alex >> >> >> > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers ________________________________ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o ________________________________ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers -------------- next part -------------- An HTML attachment was scrubbed... URL: From fernando.lopezaguilar at telefonica.com Mon Jun 1 17:27:21 2015 From: fernando.lopezaguilar at telefonica.com (FERNANDO LOPEZ AGUILAR) Date: Mon, 1 Jun 2015 15:27:21 +0000 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: References: <5569EA0F.7010900@dfki.de> <504347CB-229B-4483-A453-A5565124AAFC@telefonica.com> Message-ID: For me is ok that time. De: Alex Glikson > Fecha: Monday 1 June 2015 17:08 Para: "fiware-cloud-containers at lists.fiware.org" > Asunto: Re: [Fiware-cloud-containers] task-force kickoff Regarding the phone call I mentioned below -- we already have the weekly call of the Cloud chapter on Wed 15:00-16:00 CET (with many overlapping participants), so I suggest that we shorten the regular call to 15:30-16:00, and dedicate 14:00-15:30 CET to this topic. Please, let me know if you can make it. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Alex Glikson/Haifa/IBM at IBMIL To: "fiware-cloud-containers at lists.fiware.org" > Date: 01/06/2015 09:31 AM Subject: Re: [Fiware-cloud-containers] task-force kickoff Sent by: fiware-cloud-containers-bounces at lists.fiware.org ________________________________ We will hold the taskforce kickoff at a different time slot, to accommodate availability of key people. I will send a doodle later today. Meanwhile, we can try making a bit progress by email, following the summary I sent in a separate email. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Alex Glikson/Haifa/IBM at IBMIL To: "fiware-cloud-containers at lists.fiware.org" > Date: 31/05/2015 10:59 PM Subject: Re: [Fiware-cloud-containers] task-force kickoff Sent by: fiware-cloud-containers-bounces at lists.fiware.org ________________________________ Before this task force has launched, discussion on containers has been scheduled for tomorrow, 10:30. Now, maybe it would make sense to make some progress within the task force before raising this with the (rest of the) architects - especially if some of the participants can't join tomorrow. Regards, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: FERNANDO LOPEZ AGUILAR > To: JUAN JOSE HIERRO SUREDA >, Alex Glikson/Haifa/IBM at IBMIL Cc: "fiware-cloud-containers at lists.fiware.org" > Date: 31/05/2015 10:03 PM Subject: Re: [Fiware-cloud-containers] task-force kickoff ________________________________ Ok, then the question is we will have the kickoff of this task force tomorrow? And if it is true between which hours? We have tomorrow our internal sprint planning for the next sprint from 9:00 to 13:00. Fernando.- De: JUAN JOSE HIERRO SUREDA > Fecha: Saturday 30 May 2015 20:31 Para: Alex Glikson > CC: "fiware-cloud-containers at lists.fiware.org" > Asunto: Re: [Fiware-cloud-containers] task-force kickoff That's what I understood too Juanjo from iPhone El 30/5/2015, a las 19:52, Alex Glikson > escribi?: I am pretty sure that the monthly review/demo meeting has been rescheduled, so it seems that the morning slot on the 1st is free.. ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Philipp Slusallek > To: Alex Glikson/Haifa/IBM at IBMIL, fiware-cloud-containers at lists.fiware.org Date: 30/05/2015 07:49 PM Subject: Re: [Fiware-cloud-containers] task-force kickoff ________________________________ Hi, I thought the June 1 meetings will be moved. But I would join anyway. Best, Philipp Am 30.05.2015 um 18:06 schrieb Alex Glikson: > Dear all, > > Welcome to 'fiware-cloud-containers' mailing list! We will use this > mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. > support for Linux containers (such as Docker), including use-cases & > requirements, associated IaaS and PaaS-style capabilities, concrete > plans going forwards, potential collaboration with teams working on > similar/related technologies, etc. > > Currently we have the following people subscribed to this mailing list: > IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for > IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud > leader), Doron (working with Ezra on containers) > TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads > for the PaaS-related work in FIWARE Cloud) > DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter > leader/architect) > Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old > FI-WARE project) > > Let me know if anyone is missing. > > As a background, please, read the thread below, including the very nice > summary by Mario on the related investigation and implementation in > FI-Content2 project. I will send a separate email briefly summarizing > the situation in FIWARE. > > *All *-- notice that we are going to have a phone call to discuss this > topic during the upcoming FIWARE architects interlock on Monday, June > 1at, 10:30-12:30 CET. Let me know if you can't join. > *Mario *-- would you be able to join & explain/demonstrate the FIC2Lab > solution? > > Thanks, > Alex > > > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- > > fiware-chapter-architects-bounces at lists.fi-ware.org wrote on 26/05/2015 > 08:37:16 AM: > >> From: Alex Glikson/Haifa/IBM at IBMIL >> To: Philipp Slusallek > >> Cc: fiware-chapter-architects at lists.fi-ware.org, Kenneth Nagin/ >> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >> Date: 26/05/2015 08:37 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> Sent by: fiware-chapter-architects-bounces at lists.fi-ware.org >> >> Hi Philipp, >> >> It is great to hear that FIC2 team is willing to contribute! >> In a nutshell, our current plan is to start with surfacing >> containers via Nova, and then gradually migrate to Magnum. In terms >> of orchestration, we currently work with Heat and Murano, and are >> assessing the approaches to support containers (OpenStack alignment >> being high priority, but also the ability to surface capabilities >> unique to containers). >> I suggest that we form a small taskforce to explore this topic. >> Please, send me a list of people to involve. >> >> Thanks, >> Alex >> >> > ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> >> >> >> From: Philipp Slusallek > >> To: Juanjo Hierro >, Alex >> Glikson/Haifa/IBM at IBMIL >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >> architects at lists.fi-ware.org, Kenneth Nagin/Haifa/IBM at IBMIL >> Date: 26/05/2015 07:21 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> >> >> >> Hi, >> >> Here is some first input from FIC2 on Docker compiled from Mario. As I >> mentioned before, we have a full system working that deploys entire sets >> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >> >> FIC2 would be happy to work with FIWARE to integrate them in whatever >> way we at FIWARE decide to implement Docker. Right now we deploy it on >> top of the VMs offered by FIWARE but a more native implementation would >> be fine as well and save much resources (which seems to be a major issue). >> >> A presentation by Canonical recently showed a >10x improvement in >> resource usage with their containers over plain OpenStack (which is >> probably where we are -- but then Intel claims even better numbers in >> their Clear Containers solution using traditional VMs). >> >> Since we have experience already, it seems very useful to make Docker >> available especially since deployment of SW is much easier too and the >> same SW can be deployed essentially anywhere (including a local machine). >> >> >> Best, >> >> Philipp >> >> > There are several ways in which FIWARE could support Docker. My >> assumption is that the plan is to add to OpenStack a Docker driver. >> It would enable the creation of instances which are not VMs but >> OpenStack containers, from images which are not ISOs but Docker >> images. It?s described here: https://wiki.openstack.org/wiki/Docker >> > >> > >> > >> > Advantages: >> > >> > + Launches faster than VM instances >> > >> > + Less resources used from quotas in FIWARE nodes >> > >> > + images of SEs smaller and easier to build (using Dockerfiles) >> > >> > >> > >> > Open questions / disadvantages: >> > >> > - Will it be possible to pull images from DockerHub (preferred >> approach), or do they need to be uploaded to the Glance image >> repository of each FIWARE node? >> > >> > - Is the only way to launch containers the Cloud Portal or >> OpenStack API? Not being able to target a Docker API would reduce >> the value of Docker and its ecosystem of tools, e.g.: docker-compose >> which enables launching applications composed of multiple containers. >> > >> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >> reverse proxy that forwards HTTP requests to container1.john- >> smith.cloud.fiware.org to the right container port? >> > >> > - How would links between containers running on different hosts >> work? Using Weave? >> > >> > - Mounting volumes in Docker is a very common way of passing >> configuration files, having persistence, etc. How would this work > inOpenStack? >> > >> > >> > >> > Our suggestion: >> > >> > * Ability to use the Docker API remotely (i.e. command-line tools >> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >> > >> > * Ability to pull containers directly from DockerHub with a common >> (transparent?) image mirror in each FIWARE node >> > >> > >> > >> > >> > >> > >> > >> > Regarding FIC2Lab runner, our wishlist was: >> > >> > 1. ability to use the same tools locally and remotely in any cloud >> (ideally use docker-machine against any cloud provider, in >> particular FIWARE nodes) >> > >> > 2. nice web user interface hiding the complexity of firewalling, >> security pairs, public IPs, etc. >> > >> > 3. compatibility with the tools of the docker ecosystem: docker >> CLI, compose, swarm ... >> > >> > >> > >> > We compared tools such as tutum (free multi-tenant online >> service), shipyard (open-source web UI) and panamax (open-source web >> UI and orchestrator). >> > >> > And we decided to go for Panamax, which is nice for #1 and #2 but >> not #3 so far. Here's a few pluses and minuses: >> > >> > + nice UI with DockerHub search >> > >> > + composite application, with application template search >> > >> > - It's not multi-tenant, which is good for portability but bad for >> software updates >> > >> > - its model for composite applications is different from docker- >> compose (for now) >> > >> > - the UI can't synchronize with actions performed in Docker CLI or >> API because it's using CoreOS underneath >> > >> > >> > >> > The FIC2Lab runner is documented here: http://fic2.github.io/runner >> > >> > >> >> >> >> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >> > Dear Alex, >> > >> > It's fine with me. >> > >> > I believe that we should collocate here the discussion with >> > FI-Content2 (Philipp) regarding the stuff they have developed for the >> > FIC2-Lab. >> > >> > @Philipp: could you share some material (document describing your work >> > on FIC2-Lab, slides, whatever) describing what you have done and send it >> > to the fiware-chapter-architects mailing list as preparation for the >> > discussion? >> > >> > @Alex: please also share the ideas you want to present on this matter. >> > >> > By exchanging material prior to the meeting, we would be able to have >> > a more fruitful meeting. >> > >> > Best regards, >> > >> > -- Juanjo >> > >> > On 18/05/15 11:07, Alex Glikson wrote: >> >> I am out of office on the 25th. Would the following Monday work? >> >> >> >> Thanks, >> >> Alex >> >> >> >> >> >> >> >> >> >> >> >> >> >> From: Juanjo Hierro > >> >> To: Philipp Slusallek >, Alex >> >> Glikson/Haifa/IBM at IBMIL, > >> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >> >> Date: 18/05/2015 11:53 AM >> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & > Docker >> >> > ------------------------------------------------------------------------ >> >> >> >> >> >> >> >> >> >> I would suggest that we book the next architecture session (Monday >> >> 25th) to cover this matter. >> >> >> >> Will you Alex be able to prepare a presentation? >> >> >> >> Cheers, >> >> >> >> -- Juanjo >> >> >> >> On 18/05/15 09:46, Philipp Slusallek wrote: >> >> > Hi, >> >> > >> >> > This sounds very useful to cover as well in this or one of the next >> >> > calls. It would also get the architectural activities started > again. It >> >> > might even make sense to invite (selected?) AB members from the UCs. >> >> > >> >> > For example, a lot of work that has been done within FIcontent to > deploy >> >> > our SEs and related GEs within Docker on top of FIWARE Lab. > Particularly >> >> > relevant could be the on-click deployment of entire arrangements > of GEs >> >> > and SEs. Better and more dedicated Docker support would certainly be >> >> > very welcome as well. >> >> > >> >> > Unfortunately, I have a lecture this morning and cannot join > until 12h. >> >> > But Stefan Lemme from my group will be on the call. He has been a > core >> >> > member of the relevant FIC2-Lab task force in FIcontent. >> >> > >> >> > >> >> > Best, >> >> > >> >> > Philipp >> >> > >> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >> >> >> Maybe we can also dedicate some time to have an initial > discussion on >> >> >> the roadmap to adopt Linux Containers and in particular Docker in >> >> FIWARE >> >> >> Lab. >> >> >> >> >> >> Regards, >> >> >> Alex >> >> >> > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers ________________________________ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o ________________________________ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o_______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers ________________________________ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o -------------- next part -------------- An HTML attachment was scrubbed... URL: From GLIKSON at il.ibm.com Mon Jun 1 17:40:12 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Mon, 1 Jun 2015 18:40:12 +0300 Subject: [Fiware-cloud-containers] Doodle Poll (RE: task-force kickoff) In-Reply-To: References: <5569EA0F.7010900@dfki.de> <504347CB-229B-4483-A453-A5565124AAFC@telefonica.com> Message-ID: OK, let's go for a Doodle poll then. http://doodle.com/nckkydftccztdyky Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: LOPEZ RAMOS Mario To: Alex Glikson/Haifa/IBM at IBMIL, "fiware-cloud-containers at lists.fiware.org" Date: 01/06/2015 06:24 PM Subject: RE: [Fiware-cloud-containers] task-force kickoff Hi Alex and all, Sorry, I have another meeting Wed 14:00-15:30. I can make it before or after that though. Best, Mario De : fiware-cloud-containers-bounces at lists.fiware.org [ mailto:fiware-cloud-containers-bounces at lists.fiware.org] De la part de Alex Glikson Envoy? : lundi 1 juin 2015 17:08 ? : fiware-cloud-containers at lists.fiware.org Objet : Re: [Fiware-cloud-containers] task-force kickoff Regarding the phone call I mentioned below -- we already have the weekly call of the Cloud chapter on Wed 15:00-16:00 CET (with many overlapping participants), so I suggest that we shorten the regular call to 15:30-16:00, and dedicate 14:00-15:30 CET to this topic. Please, let me know if you can make it. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Alex Glikson/Haifa/IBM at IBMIL To: "fiware-cloud-containers at lists.fiware.org" < fiware-cloud-containers at lists.fiware.org> Date: 01/06/2015 09:31 AM Subject: Re: [Fiware-cloud-containers] task-force kickoff Sent by: fiware-cloud-containers-bounces at lists.fiware.org We will hold the taskforce kickoff at a different time slot, to accommodate availability of key people. I will send a doodle later today. Meanwhile, we can try making a bit progress by email, following the summary I sent in a separate email. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Alex Glikson/Haifa/IBM at IBMIL To: "fiware-cloud-containers at lists.fiware.org" < fiware-cloud-containers at lists.fiware.org> Date: 31/05/2015 10:59 PM Subject: Re: [Fiware-cloud-containers] task-force kickoff Sent by: fiware-cloud-containers-bounces at lists.fiware.org Before this task force has launched, discussion on containers has been scheduled for tomorrow, 10:30. Now, maybe it would make sense to make some progress within the task force before raising this with the (rest of the) architects - especially if some of the participants can't join tomorrow. Regards, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: FERNANDO LOPEZ AGUILAR To: JUAN JOSE HIERRO SUREDA , Alex Glikson/Haifa/IBM at IBMIL Cc: "fiware-cloud-containers at lists.fiware.org" < fiware-cloud-containers at lists.fiware.org> Date: 31/05/2015 10:03 PM Subject: Re: [Fiware-cloud-containers] task-force kickoff Ok, then the question is we will have the kickoff of this task force tomorrow? And if it is true between which hours? We have tomorrow our internal sprint planning for the next sprint from 9:00 to 13:00. Fernando.- De: JUAN JOSE HIERRO SUREDA Fecha: Saturday 30 May 2015 20:31 Para: Alex Glikson CC: "fiware-cloud-containers at lists.fiware.org" < fiware-cloud-containers at lists.fiware.org> Asunto: Re: [Fiware-cloud-containers] task-force kickoff That's what I understood too Juanjo from iPhone El 30/5/2015, a las 19:52, Alex Glikson escribi?: I am pretty sure that the monthly review/demo meeting has been rescheduled, so it seems that the morning slot on the 1st is free.. ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Philipp Slusallek To: Alex Glikson/Haifa/IBM at IBMIL, fiware-cloud-containers at lists.fiware.org Date: 30/05/2015 07:49 PM Subject: Re: [Fiware-cloud-containers] task-force kickoff Hi, I thought the June 1 meetings will be moved. But I would join anyway. Best, Philipp Am 30.05.2015 um 18:06 schrieb Alex Glikson: > Dear all, > > Welcome to 'fiware-cloud-containers' mailing list! We will use this > mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. > support for Linux containers (such as Docker), including use-cases & > requirements, associated IaaS and PaaS-style capabilities, concrete > plans going forwards, potential collaboration with teams working on > similar/related technologies, etc. > > Currently we have the following people subscribed to this mailing list: > IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for > IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud > leader), Doron (working with Ezra on containers) > TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads > for the PaaS-related work in FIWARE Cloud) > DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter > leader/architect) > Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old > FI-WARE project) > > Let me know if anyone is missing. > > As a background, please, read the thread below, including the very nice > summary by Mario on the related investigation and implementation in > FI-Content2 project. I will send a separate email briefly summarizing > the situation in FIWARE. > > *All *-- notice that we are going to have a phone call to discuss this > topic during the upcoming FIWARE architects interlock on Monday, June > 1at, 10:30-12:30 CET. Let me know if you can't join. > *Mario *-- would you be able to join & explain/demonstrate the FIC2Lab > solution? > > Thanks, > Alex > > > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- > > fiware-chapter-architects-bounces at lists.fi-ware.org wrote on 26/05/2015 > 08:37:16 AM: > >> From: Alex Glikson/Haifa/IBM at IBMIL >> To: Philipp Slusallek >> Cc: fiware-chapter-architects at lists.fi-ware.org, Kenneth Nagin/ >> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >> Date: 26/05/2015 08:37 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> Sent by: fiware-chapter-architects-bounces at lists.fi-ware.org >> >> Hi Philipp, >> >> It is great to hear that FIC2 team is willing to contribute! >> In a nutshell, our current plan is to start with surfacing >> containers via Nova, and then gradually migrate to Magnum. In terms >> of orchestration, we currently work with Heat and Murano, and are >> assessing the approaches to support containers (OpenStack alignment >> being high priority, but also the ability to surface capabilities >> unique to containers). >> I suggest that we form a small taskforce to explore this topic. >> Please, send me a list of people to involve. >> >> Thanks, >> Alex >> >> > ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> >> >> >> From: Philipp Slusallek >> To: Juanjo Hierro , Alex >> Glikson/Haifa/IBM at IBMIL >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >> architects at lists.fi-ware.org, Kenneth Nagin/Haifa/IBM at IBMIL >> Date: 26/05/2015 07:21 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> >> >> >> Hi, >> >> Here is some first input from FIC2 on Docker compiled from Mario. As I >> mentioned before, we have a full system working that deploys entire sets >> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >> >> FIC2 would be happy to work with FIWARE to integrate them in whatever >> way we at FIWARE decide to implement Docker. Right now we deploy it on >> top of the VMs offered by FIWARE but a more native implementation would >> be fine as well and save much resources (which seems to be a major issue). >> >> A presentation by Canonical recently showed a >10x improvement in >> resource usage with their containers over plain OpenStack (which is >> probably where we are -- but then Intel claims even better numbers in >> their Clear Containers solution using traditional VMs). >> >> Since we have experience already, it seems very useful to make Docker >> available especially since deployment of SW is much easier too and the >> same SW can be deployed essentially anywhere (including a local machine). >> >> >> Best, >> >> Philipp >> >> > There are several ways in which FIWARE could support Docker. My >> assumption is that the plan is to add to OpenStack a Docker driver. >> It would enable the creation of instances which are not VMs but >> OpenStack containers, from images which are not ISOs but Docker >> images. It?s described here: https://wiki.openstack.org/wiki/Docker >> > >> > >> > >> > Advantages: >> > >> > + Launches faster than VM instances >> > >> > + Less resources used from quotas in FIWARE nodes >> > >> > + images of SEs smaller and easier to build (using Dockerfiles) >> > >> > >> > >> > Open questions / disadvantages: >> > >> > - Will it be possible to pull images from DockerHub (preferred >> approach), or do they need to be uploaded to the Glance image >> repository of each FIWARE node? >> > >> > - Is the only way to launch containers the Cloud Portal or >> OpenStack API? Not being able to target a Docker API would reduce >> the value of Docker and its ecosystem of tools, e.g.: docker-compose >> which enables launching applications composed of multiple containers. >> > >> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >> reverse proxy that forwards HTTP requests to container1.john- >> smith.cloud.fiware.org to the right container port? >> > >> > - How would links between containers running on different hosts >> work? Using Weave? >> > >> > - Mounting volumes in Docker is a very common way of passing >> configuration files, having persistence, etc. How would this work > inOpenStack? >> > >> > >> > >> > Our suggestion: >> > >> > * Ability to use the Docker API remotely (i.e. command-line tools >> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >> > >> > * Ability to pull containers directly from DockerHub with a common >> (transparent?) image mirror in each FIWARE node >> > >> > >> > >> > >> > >> > >> > >> > Regarding FIC2Lab runner, our wishlist was: >> > >> > 1. ability to use the same tools locally and remotely in any cloud >> (ideally use docker-machine against any cloud provider, in >> particular FIWARE nodes) >> > >> > 2. nice web user interface hiding the complexity of firewalling, >> security pairs, public IPs, etc. >> > >> > 3. compatibility with the tools of the docker ecosystem: docker >> CLI, compose, swarm ... >> > >> > >> > >> > We compared tools such as tutum (free multi-tenant online >> service), shipyard (open-source web UI) and panamax (open-source web >> UI and orchestrator). >> > >> > And we decided to go for Panamax, which is nice for #1 and #2 but >> not #3 so far. Here's a few pluses and minuses: >> > >> > + nice UI with DockerHub search >> > >> > + composite application, with application template search >> > >> > - It's not multi-tenant, which is good for portability but bad for >> software updates >> > >> > - its model for composite applications is different from docker- >> compose (for now) >> > >> > - the UI can't synchronize with actions performed in Docker CLI or >> API because it's using CoreOS underneath >> > >> > >> > >> > The FIC2Lab runner is documented here: http://fic2.github.io/runner >> > >> > >> >> >> >> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >> > Dear Alex, >> > >> > It's fine with me. >> > >> > I believe that we should collocate here the discussion with >> > FI-Content2 (Philipp) regarding the stuff they have developed for the >> > FIC2-Lab. >> > >> > @Philipp: could you share some material (document describing your work >> > on FIC2-Lab, slides, whatever) describing what you have done and send it >> > to the fiware-chapter-architects mailing list as preparation for the >> > discussion? >> > >> > @Alex: please also share the ideas you want to present on this matter. >> > >> > By exchanging material prior to the meeting, we would be able to have >> > a more fruitful meeting. >> > >> > Best regards, >> > >> > -- Juanjo >> > >> > On 18/05/15 11:07, Alex Glikson wrote: >> >> I am out of office on the 25th. Would the following Monday work? >> >> >> >> Thanks, >> >> Alex >> >> >> >> >> >> >> >> >> >> >> >> >> >> From: Juanjo Hierro >> >> To: Philipp Slusallek , Alex >> >> Glikson/Haifa/IBM at IBMIL, < fiware-chapter-architects at lists.fi-ware.org> >> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >> >> Date: 18/05/2015 11:53 AM >> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & > Docker >> >> > ------------------------------------------------------------------------ >> >> >> >> >> >> >> >> >> >> I would suggest that we book the next architecture session (Monday >> >> 25th) to cover this matter. >> >> >> >> Will you Alex be able to prepare a presentation? >> >> >> >> Cheers, >> >> >> >> -- Juanjo >> >> >> >> On 18/05/15 09:46, Philipp Slusallek wrote: >> >> > Hi, >> >> > >> >> > This sounds very useful to cover as well in this or one of the next >> >> > calls. It would also get the architectural activities started > again. It >> >> > might even make sense to invite (selected?) AB members from the UCs. >> >> > >> >> > For example, a lot of work that has been done within FIcontent to > deploy >> >> > our SEs and related GEs within Docker on top of FIWARE Lab. > Particularly >> >> > relevant could be the on-click deployment of entire arrangements > of GEs >> >> > and SEs. Better and more dedicated Docker support would certainly be >> >> > very welcome as well. >> >> > >> >> > Unfortunately, I have a lecture this morning and cannot join > until 12h. >> >> > But Stefan Lemme from my group will be on the call. He has been a > core >> >> > member of the relevant FIC2-Lab task force in FIcontent. >> >> > >> >> > >> >> > Best, >> >> > >> >> > Philipp >> >> > >> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >> >> >> Maybe we can also dedicate some time to have an initial > discussion on >> >> >> the roadmap to adopt Linux Containers and in particular Docker in >> >> FIWARE >> >> >> Lab. >> >> >> >> >> >> Regards, >> >> >> Alex >> >> >> > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers -------------- next part -------------- An HTML attachment was scrubbed... URL: From GLIKSON at il.ibm.com Mon Jun 1 22:36:14 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Mon, 1 Jun 2015 23:36:14 +0300 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: <556C05BB.5060900@dfki.de> References: <556C05BB.5060900@dfki.de> Message-ID: I agree that user/developer perspective is important. However, there are several things that we need to take into account: For users/developers who just want to *use* GEs (rather than develop new ones), FIWARE already provides tools to easily set them up - pre-built VM images, as well as SDC/Chef recipes and PaaS Blueprints (for bundles). Significant effort has been invested by GE owners to populate the corresponding artefacts and enable the automated/seamless deployment in FIWARE Lab. Moreover, as I mentioned, we are currently transforming those tools to better align with OpenStack, and we are also constantly improving them to make more capable/stable. Having said that, there is clearly space for further improvement (usability, documentation, features, integration, etc). In this context, it is not entirely clear whether/how containers (including Docker) can help, besides achieving higher density. We do appreciate the value proposition of Docker as a new emerging standard for application packaging and life cycle management -- especially for 'cloud-born' applications. However, the tooling around Docker is rather diverse and immature, and we should be careful before we 'bet' on one (or more) of the options to become part of FIWARE (which aims at becoming a sustainable standard). For example, I haven't seen yet a good DevOps tool that would manage rolling upgrades (and other 'traditional' configuration management scenarios) for Docker-based applications. It is likely that we would need to support both 'old' and 'new' ways to design and manage applications for the upcoming few years. Moreover, new paradigms may arise, addressing additional 'niches' on the spectrum between traditional servers (running full operating system plus an arbitrary software stack) and application containers (running a single process, possible with it's children processes). In my opinion, one of our challenges is to come up with a unified solution (and a roadmap) which would provide a reasonable value proposition in the short term, but would also like to be forward-compatible with new developments which will surely emerge around containers in the near future. For example, if we think that Chef is the way to go from configuration management perspective, maybe we should use and extend the Chef-Docker integration [1] . Alternatively (or in addition?), if we want to stick with Heat for orchestration, maybe we can invest in adding support for Magnum containers as resources in Heat, as well as LXD driver in Nova (for system containers). IMO, this is the problem space we need to further explore (and we started doing so). Regards, Alex [1] https://www.chef.io/solutions/containers/ ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Philipp Slusallek To: Alex Glikson/Haifa/IBM at IBMIL, fiware-cloud-containers at lists.fiware.org Date: 01/06/2015 10:12 AM Subject: Re: [Fiware-cloud-containers] task-force kickoff Hi all, It is certainly the right thing to work on supporting containers natively in the Cloud chapter. It seems that a lot is currently being done in this regard within OpenStack and we should see how we can best contribute to that. However, I would like to point out another aspect that will probably have a much more direct and short-term impact on FIWARE and which we should be able to do in parallel: Focus on the user-perspective. I am talking about greatly simplifying the deployment of enablers for developers (and possibly end users) on FIWARE. Docker is an excellent tool for that as we have experienced for example in FIcontent. Its very easy for owners to provide their enabler and even easier for users to deploy and test them. As with the "Discover, Try, Tweak, Run" approach in FIcontent we can make it tremendously simpler for users to explore what we offer, try it out, share it with others via social networks, and deploy and use the necessary services for real. This is exactly what I believe FIWARE needs. The GEs are still somewhat difficult to access and the documentation for doing so leaves some things to be desired (at least judging from my reviewing job). Taking a much more user-focused approach is important now. So to make things concrete: We should explore (in parallel to your suggestions which is still needed, of course) to install something similar to FIC2-Lab (actually: now officially called "FIWARE Media&Content Lab"). We might be able to leverage the experience we got from there as a basis. But I am sure that together there are many improvements and extensions that should be done. In parallel, we would ask enabler owners to provide suitable dockerified versions that can easily be deployed via this system. This does not make sense for all and can be done incrementally as well. All that is then still needed is to create some nice examples that can be used for the Try and Tweak part, where developers can get first hand experience of enablers, by simply trying out, inspecting, and hacking on existing web-based applications that make use of one or more (bundles!) enablers from any Web browser and without ever having seen the cloud portal (they eventually need to but this is when they are hopefully hooked already). Again, this is not an alternative but a parallel activity that makes containers directly relevant for the developers right now (!) and which relies eventually on the ability to deploy enabler an easier, faster, and with a largely reduced resource footprint via the necessary technology in Cloud/OpenStack. Simply deploying docker within existing virtual machines should allow us to reduce resource usage by quite a bit. But we need to look into that. Best, Philipp Am 31.05.2015 um 22:36 schrieb Alex Glikson: > Below is a brief summary of the current FIWARE Cloud status with regards > to containers and Docker support. > > * FIWARE Cloud GEs currently deployed in FIWARE Lab provide VM-based > IaaS capabilities with OpenStack Nova (with KVM), Glance, Keystone, > Cinder and Neutron (with OVS). FIWARE Cloud also provides object > storage capabilities based on OpenStack Swift. Moreover, we support > PaaS-level capabilities, including the ability to provision and > auto-scale complex applications comprising sets of inter-dependent > VMs as well as the ability to install and configure the individual > software components within VMs using configuration management tools > such as Chef. The key user-visible notion behind this capability is > the notion of a Blueprint, which defines the application topology, > dependencies, auto-scaling rules, software configuration, etc > (conceptually similar to Amazon OpsWorks). > * In the last few months, the PaaS implementation is undergoing a > major transition from a 'proprietary' (although open source) code to > full adoption of OpenStack Heat and Murano (as well as contribution > of features to the Murano community, such as support for templates, > configuration management enhancements, etc). > * In parallel, we have started assessing (and prototyping) the options > to support/leverage Linux containers technologies, and Docker in > particular. We aim at two main use-cases: > 1. As a FIWARE Cloud provider (in FIWARE Lab), I want to continue > providing (almost?) the same PaaS/IaaS capabilities, but at the > same time being able to admit much more users/workloads on the > same hardware, by leveraging Containers technologies. > 2. As a FIWARE Cloud user, I want to be able to leverage the Docker > ecosystem, including public images available at Docker Hub as > well as existing tools (TBD: which tools exactly?), while > hosting my applications on the FIWARE Lab > * The 2nd use-case can be satisfied in several ways, such as: > 1. Treat Docker-based cluster as PaaS application (managed by > FIWARE Cloud PaaS), while the cloud middleware (IaaS and PaaS) > is not aware of individual containers, Docker images, etc. > 2. Make Docker containers 1st class citizens in PaaS -- e.g., as an > additional delivery mechanism in addition to Chef, deployed on > regular IaaS virtualized infra (providing isolation, accounting, > etc). In this case each Docker cluster is contained within a > single tenant (and even single application within a tenant?) > 3. Make Docker containers 1st class citizens in IaaS deployed on > bare-metal, making PaaS use it via a standardized API > * The current (tentative) plan is to apply a 2-phase approach to > introduce containers: phase-1: start by deploying nova-docker in > FIWARE lab in order to enable use-case n.1, and then phase-2: > gradually migrate to OpenStack Magnum, while incorporating the > Docker-native capabilities into Murano in order to support use-case > n.2. Of course, the devil is in the details, so we need to discuss > further, to understand the exact requirements and implications. > > > Regards, > Alex > > > > > From: Alex Glikson/Haifa/IBM at IBMIL > To: fiware-cloud-containers at lists.fiware.org > Date: 30/05/2015 07:06 PM > Subject: [Fiware-cloud-containers] task-force kickoff > Sent by: fiware-cloud-containers-bounces at lists.fiware.org > ------------------------------------------------------------------------ > > > > Dear all, > > Welcome to 'fiware-cloud-containers' mailing list! We will use this > mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. > support for Linux containers (such as Docker), including use-cases & > requirements, associated IaaS and PaaS-style capabilities, concrete > plans going forwards, potential collaboration with teams working on > similar/related technologies, etc. > > Currently we have the following people subscribed to this mailing list: > IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for > IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud > leader), Doron (working with Ezra on containers) > TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads > for the PaaS-related work in FIWARE Cloud) > DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter > leader/architect) > Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old > FI-WARE project) > > Let me know if anyone is missing. > > As a background, please, read the thread below, including the very nice > summary by Mario on the related investigation and implementation in > FI-Content2 project. I will send a separate email briefly summarizing > the situation in FIWARE. > * > All *-- notice that we are going to have a phone call to discuss this > topic during the upcoming FIWARE architects interlock on Monday, June > 1at, 10:30-12:30 CET. Let me know if you can't join.* > Mario *-- would you be able to join & explain/demonstrate the FIC2Lab > solution? > > Thanks, > Alex > > > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- > > fiware-chapter-architects-bounces at lists.fi-ware.org wrote on 26/05/2015 > 08:37:16 AM: > >> From: Alex Glikson/Haifa/IBM at IBMIL >> To: Philipp Slusallek >> Cc: fiware-chapter-architects at lists.fi-ware.org, Kenneth Nagin/ >> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >> Date: 26/05/2015 08:37 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> Sent by: fiware-chapter-architects-bounces at lists.fi-ware.org >> >> Hi Philipp, >> >> It is great to hear that FIC2 team is willing to contribute! >> In a nutshell, our current plan is to start with surfacing >> containers via Nova, and then gradually migrate to Magnum. In terms >> of orchestration, we currently work with Heat and Murano, and are >> assessing the approaches to support containers (OpenStack alignment >> being high priority, but also the ability to surface capabilities >> unique to containers). >> I suggest that we form a small taskforce to explore this topic. >> Please, send me a list of people to involve. >> >> Thanks, >> Alex >> >> > ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> >> >> >> From: Philipp Slusallek >> To: Juanjo Hierro , Alex >> Glikson/Haifa/IBM at IBMIL >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >> architects at lists.fi-ware.org, Kenneth Nagin/Haifa/IBM at IBMIL >> Date: 26/05/2015 07:21 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> >> >> >> Hi, >> >> Here is some first input from FIC2 on Docker compiled from Mario. As I >> mentioned before, we have a full system working that deploys entire sets >> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >> >> FIC2 would be happy to work with FIWARE to integrate them in whatever >> way we at FIWARE decide to implement Docker. Right now we deploy it on >> top of the VMs offered by FIWARE but a more native implementation would >> be fine as well and save much resources (which seems to be a major issue). >> >> A presentation by Canonical recently showed a >10x improvement in >> resource usage with their containers over plain OpenStack (which is >> probably where we are -- but then Intel claims even better numbers in >> their Clear Containers solution using traditional VMs). >> >> Since we have experience already, it seems very useful to make Docker >> available especially since deployment of SW is much easier too and the >> same SW can be deployed essentially anywhere (including a local machine). >> >> >> Best, >> >> Philipp >> >> > There are several ways in which FIWARE could support Docker. My >> assumption is that the plan is to add to OpenStack a Docker driver. >> It would enable the creation of instances which are not VMs but >> OpenStack containers, from images which are not ISOs but Docker >> images. It?s described here: _https://wiki.openstack.org/wiki/Docker_ >> > >> > >> > >> > Advantages: >> > >> > + Launches faster than VM instances >> > >> > + Less resources used from quotas in FIWARE nodes >> > >> > + images of SEs smaller and easier to build (using Dockerfiles) >> > >> > >> > >> > Open questions / disadvantages: >> > >> > - Will it be possible to pull images from DockerHub (preferred >> approach), or do they need to be uploaded to the Glance image >> repository of each FIWARE node? >> > >> > - Is the only way to launch containers the Cloud Portal or >> OpenStack API? Not being able to target a Docker API would reduce >> the value of Docker and its ecosystem of tools, e.g.: docker-compose >> which enables launching applications composed of multiple containers. >> > >> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >> reverse proxy that forwards HTTP requests to container1.john- >> smith.cloud.fiware.org to the right container port? >> > >> > - How would links between containers running on different hosts >> work? Using Weave? >> > >> > - Mounting volumes in Docker is a very common way of passing >> configuration files, having persistence, etc. How would this work > inOpenStack? >> > >> > >> > >> > Our suggestion: >> > >> > * Ability to use the Docker API remotely (i.e. command-line tools >> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >> > >> > * Ability to pull containers directly from DockerHub with a common >> (transparent?) image mirror in each FIWARE node >> > >> > >> > >> > >> > >> > >> > >> > Regarding FIC2Lab runner, our wishlist was: >> > >> > 1. ability to use the same tools locally and remotely in any cloud >> (ideally use docker-machine against any cloud provider, in >> particular FIWARE nodes) >> > >> > 2. nice web user interface hiding the complexity of firewalling, >> security pairs, public IPs, etc. >> > >> > 3. compatibility with the tools of the docker ecosystem: docker >> CLI, compose, swarm ... >> > >> > >> > >> > We compared tools such as tutum (free multi-tenant online >> service), shipyard (open-source web UI) and panamax (open-source web >> UI and orchestrator). >> > >> > And we decided to go for Panamax, which is nice for #1 and #2 but >> not #3 so far. Here's a few pluses and minuses: >> > >> > + nice UI with DockerHub search >> > >> > + composite application, with application template search >> > >> > - It's not multi-tenant, which is good for portability but bad for >> software updates >> > >> > - its model for composite applications is different from docker- >> compose (for now) >> > >> > - the UI can't synchronize with actions performed in Docker CLI or >> API because it's using CoreOS underneath >> > >> > >> > >> > The FIC2Lab runner is documented here: _http://fic2.github.io/runner_ >> > >> > >> >> >> >> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >> > Dear Alex, >> > >> > It's fine with me. >> > >> > I believe that we should collocate here the discussion with >> > FI-Content2 (Philipp) regarding the stuff they have developed for the >> > FIC2-Lab. >> > >> > @Philipp: could you share some material (document describing your work >> > on FIC2-Lab, slides, whatever) describing what you have done and send it >> > to the fiware-chapter-architects mailing list as preparation for the >> > discussion? >> > >> > @Alex: please also share the ideas you want to present on this matter. >> > >> > By exchanging material prior to the meeting, we would be able to have >> > a more fruitful meeting. >> > >> > Best regards, >> > >> > -- Juanjo >> > >> > On 18/05/15 11:07, Alex Glikson wrote: >> >> I am out of office on the 25th. Would the following Monday work? >> >> >> >> Thanks, >> >> Alex >> >> >> >> >> >> >> >> >> >> >> >> >> >> From: Juanjo Hierro >> >> To: Philipp Slusallek , Alex >> >> Glikson/Haifa/IBM at IBMIL, >> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >> >> Date: 18/05/2015 11:53 AM >> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & > Docker >> >> > ------------------------------------------------------------------------ >> >> >> >> >> >> >> >> >> >> I would suggest that we book the next architecture session (Monday >> >> 25th) to cover this matter. >> >> >> >> Will you Alex be able to prepare a presentation? >> >> >> >> Cheers, >> >> >> >> -- Juanjo >> >> >> >> On 18/05/15 09:46, Philipp Slusallek wrote: >> >> > Hi, >> >> > >> >> > This sounds very useful to cover as well in this or one of the next >> >> > calls. It would also get the architectural activities started > again. It >> >> > might even make sense to invite (selected?) AB members from the UCs. >> >> > >> >> > For example, a lot of work that has been done within FIcontent to > deploy >> >> > our SEs and related GEs within Docker on top of FIWARE Lab. > Particularly >> >> > relevant could be the on-click deployment of entire arrangements > of GEs >> >> > and SEs. Better and more dedicated Docker support would certainly be >> >> > very welcome as well. >> >> > >> >> > Unfortunately, I have a lecture this morning and cannot join > until 12h. >> >> > But Stefan Lemme from my group will be on the call. He has been a > core >> >> > member of the relevant FIC2-Lab task force in FIcontent. >> >> > >> >> > >> >> > Best, >> >> > >> >> > Philipp >> >> > >> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >> >> >> Maybe we can also dedicate some time to have an initial > discussion on >> >> >> the roadmap to adopt Linux Containers and in particular Docker in >> >> FIWARE >> >> >> Lab. >> >> >> >> >> >> Regards, >> >> >> Alex >> >> >>_______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] -------------- next part -------------- An HTML attachment was scrubbed... URL: From philipp.slusallek at dfki.de Tue Jun 2 06:29:01 2015 From: philipp.slusallek at dfki.de (Philipp Slusallek) Date: Tue, 02 Jun 2015 06:29:01 +0200 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: References: <556C05BB.5060900@dfki.de> Message-ID: <556D310D.2080405@dfki.de> Hi Alex, all, Am 01.06.2015 um 22:36 schrieb Alex Glikson: > 1. For users/developers who just want to *use* GEs (rather than develop > new ones), FIWARE already provides tools to easily set them up - > pre-built VM images, as well as SDC/Chef recipes and PaaS Blueprints > (for bundles). Significant effort has been invested by GE owners to > populate the corresponding artefacts and enable the > automated/seamless deployment in FIWARE Lab. Moreover, as I > mentioned, we are currently transforming those tools to better align > with OpenStack, and we are also constantly improving them to make > more capable/stable. Having said that, there is clearly space for > further improvement (usability, documentation, features, > integration, etc). In this context, it is not entirely clear > whether/how containers (including Docker) can help, besides > achieving higher density. Docker is just dramatically simpler to deploy on any infrastructure including your home PC (e.g. for debugging, testing, or whatever). VM images are a pain and are very inefficient if I have many smaller services that I need to deploy. Bundles help here, but are too inflexible as the combination I want probably does not exist. Building a setup myself is hard particularly for novices. With docker we can deploy whatever combination of services with a few commands and very efficiently, too. Actually we do not even want to deal with Docker. Its just a tools that simplifies our lives. Our users just want services and deploy them with a few clicks. That is what we now offer in FIC2-Lab. Of course, this is not for setting up a large data center or special deployment but then this is not what people typically do in FIWARE Lab either. > 2. We do appreciate the value proposition of Docker as a new emerging > standard for application packaging and life cycle management -- > especially for 'cloud-born' applications. However, the tooling > around Docker is rather diverse and immature, and we should be > careful before we 'bet' on one (or more) of the options to become > part of FIWARE (which aims at becoming a sustainable standard). For > example, I haven't seen yet a good DevOps tool that would manage > rolling upgrades (and other 'traditional' configuration management > scenarios) for Docker-based applications. Fully agree, its certainly not as evolved as other infrastructures. But does this mean we should not start using them, particularly if we can offer significant easy of use to our users? We can "learn on the job" at Internet time :-). > 3. It is likely that we would need to support both 'old' and 'new' ways > to design and manage applications for the upcoming few years. > Moreover, new paradigms may arise, addressing additional 'niches' on > the spectrum between traditional servers (running full operating > system plus an arbitrary software stack) and application containers > (running a single process, possible with it's children processes). > In my opinion, one of our challenges is to come up with a unified > solution (and a roadmap) which would provide a reasonable value > proposition in the short term, but would also like to be > forward-compatible with new developments which will surely emerge > around containers in the near future. For example, if we think that > Chef is the way to go from configuration management perspective, > maybe we should use and extend the Chef-Docker integration [1] . > Alternatively (or in addition?), if we want to stick with Heat for > orchestration, maybe we can invest in adding support for Magnum > containers as resources in Heat, as well as LXD driver in Nova (for > system containers). IMO, this is the problem space we need to > further explore (and we started doing so). As I said, I agree here. But this is the backend and something for the at least medium term future. Currently any FIWARE user has to first set up his machines in the Cloud portal even if all he wants to do is try out and play with one service (or a couple of them). This is crazy. Why can't we just give him the services he wants to play with -- with the click of a button ("Try"). Another button and he has his own copy and can modify it to his/her hearts contain ("Tweak"). Another click and he has his setup be deployed in some cloud or even his home machine ("Run"). And if he came up with some nice ideas allow whim to share them easily through Github and let others build on top of that. Think of this as a github for services. This is something that has not been done yet and no infrastructure yet allows people to do this easily. We have the means to very easily offer that (FIC2-Lab already does exactly this). It seems that this could be a huge hit for developers, similar to github. We need to provide better solutions for our users NOW. We cannot wait until all of this infrastructure has settled down. And there are solutions that already work NOW. Can we back all this up with better backend technology -- of course we can and we should! But these are (largely) independent activities from my point of view. Best, Philipp > > > Regards, > Alex > > [1] https://www.chef.io/solutions/containers/ > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > > > > From: Philipp Slusallek > To: Alex Glikson/Haifa/IBM at IBMIL, > fiware-cloud-containers at lists.fiware.org > Date: 01/06/2015 10:12 AM > Subject: Re: [Fiware-cloud-containers] task-force kickoff > ------------------------------------------------------------------------ > > > > Hi all, > > It is certainly the right thing to work on supporting containers > natively in the Cloud chapter. It seems that a lot is currently being > done in this regard within OpenStack and we should see how we can best > contribute to that. > > However, I would like to point out another aspect that will probably > have a much more direct and short-term impact on FIWARE and which we > should be able to do in parallel: Focus on the user-perspective. I am > talking about greatly simplifying the deployment of enablers for > developers (and possibly end users) on FIWARE. > > Docker is an excellent tool for that as we have experienced for example > in FIcontent. Its very easy for owners to provide their enabler and even > easier for users to deploy and test them. As with the "Discover, Try, > Tweak, Run" approach in FIcontent we can make it tremendously simpler > for users to explore what we offer, try it out, share it with others via > social networks, and deploy and use the necessary services for real. > > This is exactly what I believe FIWARE needs. The GEs are still somewhat > difficult to access and the documentation for doing so leaves some > things to be desired (at least judging from my reviewing job). Taking a > much more user-focused approach is important now. > > So to make things concrete: We should explore (in parallel to your > suggestions which is still needed, of course) to install something > similar to FIC2-Lab (actually: now officially called "FIWARE > Media&Content Lab"). We might be able to leverage the experience we got > from there as a basis. But I am sure that together there are many > improvements and extensions that should be done. > > In parallel, we would ask enabler owners to provide suitable dockerified > versions that can easily be deployed via this system. This does not make > sense for all and can be done incrementally as well. All that is then > still needed is to create some nice examples that can be used for the > Try and Tweak part, where developers can get first hand experience of > enablers, by simply trying out, inspecting, and hacking on existing > web-based applications that make use of one or more (bundles!) enablers > from any Web browser and without ever having seen the cloud portal (they > eventually need to but this is when they are hopefully hooked already). > > Again, this is not an alternative but a parallel activity that makes > containers directly relevant for the developers right now (!) and which > relies eventually on the ability to deploy enabler an easier, faster, > and with a largely reduced resource footprint via the necessary > technology in Cloud/OpenStack. Simply deploying docker within existing > virtual machines should allow us to reduce resource usage by quite a > bit. But we need to look into that. > > > Best, > > Philipp > > Am 31.05.2015 um 22:36 schrieb Alex Glikson: >> Below is a brief summary of the current FIWARE Cloud status with regards >> to containers and Docker support. >> >> * FIWARE Cloud GEs currently deployed in FIWARE Lab provide VM-based >> IaaS capabilities with OpenStack Nova (with KVM), Glance, Keystone, >> Cinder and Neutron (with OVS). FIWARE Cloud also provides object >> storage capabilities based on OpenStack Swift. Moreover, we support >> PaaS-level capabilities, including the ability to provision and >> auto-scale complex applications comprising sets of inter-dependent >> VMs as well as the ability to install and configure the individual >> software components within VMs using configuration management tools >> such as Chef. The key user-visible notion behind this capability is >> the notion of a Blueprint, which defines the application topology, >> dependencies, auto-scaling rules, software configuration, etc >> (conceptually similar to Amazon OpsWorks). >> * In the last few months, the PaaS implementation is undergoing a >> major transition from a 'proprietary' (although open source) code to >> full adoption of OpenStack Heat and Murano (as well as contribution >> of features to the Murano community, such as support for templates, >> configuration management enhancements, etc). >> * In parallel, we have started assessing (and prototyping) the options >> to support/leverage Linux containers technologies, and Docker in >> particular. We aim at two main use-cases: >> 1. As a FIWARE Cloud provider (in FIWARE Lab), I want to continue >> providing (almost?) the same PaaS/IaaS capabilities, but at the >> same time being able to admit much more users/workloads on the >> same hardware, by leveraging Containers technologies. >> 2. As a FIWARE Cloud user, I want to be able to leverage the Docker >> ecosystem, including public images available at Docker Hub as >> well as existing tools (TBD: which tools exactly?), while >> hosting my applications on the FIWARE Lab >> * The 2nd use-case can be satisfied in several ways, such as: >> 1. Treat Docker-based cluster as PaaS application (managed by >> FIWARE Cloud PaaS), while the cloud middleware (IaaS and PaaS) >> is not aware of individual containers, Docker images, etc. >> 2. Make Docker containers 1st class citizens in PaaS -- e.g., as an >> additional delivery mechanism in addition to Chef, deployed on >> regular IaaS virtualized infra (providing isolation, accounting, >> etc). In this case each Docker cluster is contained within a >> single tenant (and even single application within a tenant?) >> 3. Make Docker containers 1st class citizens in IaaS deployed on >> bare-metal, making PaaS use it via a standardized API >> * The current (tentative) plan is to apply a 2-phase approach to >> introduce containers: phase-1: start by deploying nova-docker in >> FIWARE lab in order to enable use-case n.1, and then phase-2: >> gradually migrate to OpenStack Magnum, while incorporating the >> Docker-native capabilities into Murano in order to support use-case >> n.2. Of course, the devil is in the details, so we need to discuss >> further, to understand the exact requirements and implications. >> >> >> Regards, >> Alex >> >> >> >> >> From: Alex Glikson/Haifa/IBM at IBMIL >> To: fiware-cloud-containers at lists.fiware.org >> Date: 30/05/2015 07:06 PM >> Subject: [Fiware-cloud-containers] task-force kickoff >> Sent by: fiware-cloud-containers-bounces at lists.fiware.org >> ------------------------------------------------------------------------ >> >> >> >> Dear all, >> >> Welcome to 'fiware-cloud-containers' mailing list! We will use this >> mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. >> support for Linux containers (such as Docker), including use-cases & >> requirements, associated IaaS and PaaS-style capabilities, concrete >> plans going forwards, potential collaboration with teams working on >> similar/related technologies, etc. >> >> Currently we have the following people subscribed to this mailing list: >> IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for >> IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud >> leader), Doron (working with Ezra on containers) >> TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads >> for the PaaS-related work in FIWARE Cloud) >> DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter >> leader/architect) >> Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old >> FI-WARE project) >> >> Let me know if anyone is missing. >> >> As a background, please, read the thread below, including the very nice >> summary by Mario on the related investigation and implementation in >> FI-Content2 project. I will send a separate email briefly summarizing >> the situation in FIWARE. >> * >> All *-- notice that we are going to have a phone call to discuss this >> topic during the upcoming FIWARE architects interlock on Monday, June >> 1at, 10:30-12:30 CET. Let me know if you can't join.* >> Mario *-- would you be able to join & explain/demonstrate the FIC2Lab >> solution? >> >> Thanks, >> Alex >> >> >> >> > ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- >> >> fiware-chapter-architects-bounces at lists.fi-ware.org wrote on 26/05/2015 >> 08:37:16 AM: >> >>> From: Alex Glikson/Haifa/IBM at IBMIL >>> To: Philipp Slusallek >>> Cc: fiware-chapter-architects at lists.fi-ware.org, Kenneth Nagin/ >>> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >>> Date: 26/05/2015 08:37 AM >>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >>> Sent by: fiware-chapter-architects-bounces at lists.fi-ware.org >>> >>> Hi Philipp, >>> >>> It is great to hear that FIC2 team is willing to contribute! >>> In a nutshell, our current plan is to start with surfacing >>> containers via Nova, and then gradually migrate to Magnum. In terms >>> of orchestration, we currently work with Heat and Murano, and are >>> assessing the approaches to support containers (OpenStack alignment >>> being high priority, but also the ability to surface capabilities >>> unique to containers). >>> I suggest that we form a small taskforce to explore this topic. >>> Please, send me a list of people to involve. >>> >>> Thanks, >>> Alex >>> >>> >> > ==================================================================================== >>> Alex Glikson >>> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >>> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: >>> +972-54-6466667 | Fax: +972-4-8296112 >>> >>> >>> >>> >>> >>> From: Philipp Slusallek >>> To: Juanjo Hierro , Alex >>> Glikson/Haifa/IBM at IBMIL >>> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >>> architects at lists.fi-ware.org, Kenneth Nagin/Haifa/IBM at IBMIL >>> Date: 26/05/2015 07:21 AM >>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >>> >>> >>> >>> Hi, >>> >>> Here is some first input from FIC2 on Docker compiled from Mario. As I >>> mentioned before, we have a full system working that deploys entire sets >>> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >>> >>> FIC2 would be happy to work with FIWARE to integrate them in whatever >>> way we at FIWARE decide to implement Docker. Right now we deploy it on >>> top of the VMs offered by FIWARE but a more native implementation would >>> be fine as well and save much resources (which seems to be a major > issue). >>> >>> A presentation by Canonical recently showed a >10x improvement in >>> resource usage with their containers over plain OpenStack (which is >>> probably where we are -- but then Intel claims even better numbers in >>> their Clear Containers solution using traditional VMs). >>> >>> Since we have experience already, it seems very useful to make Docker >>> available especially since deployment of SW is much easier too and the >>> same SW can be deployed essentially anywhere (including a local machine). >>> >>> >>> Best, >>> >>> Philipp >>> >>> > There are several ways in which FIWARE could support Docker. My >>> assumption is that the plan is to add to OpenStack a Docker driver. >>> It would enable the creation of instances which are not VMs but >>> OpenStack containers, from images which are not ISOs but Docker >>> images. It?s described here: _https://wiki.openstack.org/wiki/Docker_ >>> > >>> > >>> > >>> > Advantages: >>> > >>> > + Launches faster than VM instances >>> > >>> > + Less resources used from quotas in FIWARE nodes >>> > >>> > + images of SEs smaller and easier to build (using Dockerfiles) >>> > >>> > >>> > >>> > Open questions / disadvantages: >>> > >>> > - Will it be possible to pull images from DockerHub (preferred >>> approach), or do they need to be uploaded to the Glance image >>> repository of each FIWARE node? >>> > >>> > - Is the only way to launch containers the Cloud Portal or >>> OpenStack API? Not being able to target a Docker API would reduce >>> the value of Docker and its ecosystem of tools, e.g.: docker-compose >>> which enables launching applications composed of multiple containers. >>> > >>> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >>> reverse proxy that forwards HTTP requests to container1.john- >>> smith.cloud.fiware.org to the right container port? >>> > >>> > - How would links between containers running on different hosts >>> work? Using Weave? >>> > >>> > - Mounting volumes in Docker is a very common way of passing >>> configuration files, having persistence, etc. How would this work >> inOpenStack? >>> > >>> > >>> > >>> > Our suggestion: >>> > >>> > * Ability to use the Docker API remotely (i.e. command-line tools >>> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >>> > >>> > * Ability to pull containers directly from DockerHub with a common >>> (transparent?) image mirror in each FIWARE node >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > Regarding FIC2Lab runner, our wishlist was: >>> > >>> > 1. ability to use the same tools locally and remotely in any cloud >>> (ideally use docker-machine against any cloud provider, in >>> particular FIWARE nodes) >>> > >>> > 2. nice web user interface hiding the complexity of firewalling, >>> security pairs, public IPs, etc. >>> > >>> > 3. compatibility with the tools of the docker ecosystem: docker >>> CLI, compose, swarm ... >>> > >>> > >>> > >>> > We compared tools such as tutum (free multi-tenant online >>> service), shipyard (open-source web UI) and panamax (open-source web >>> UI and orchestrator). >>> > >>> > And we decided to go for Panamax, which is nice for #1 and #2 but >>> not #3 so far. Here's a few pluses and minuses: >>> > >>> > + nice UI with DockerHub search >>> > >>> > + composite application, with application template search >>> > >>> > - It's not multi-tenant, which is good for portability but bad for >>> software updates >>> > >>> > - its model for composite applications is different from docker- >>> compose (for now) >>> > >>> > - the UI can't synchronize with actions performed in Docker CLI or >>> API because it's using CoreOS underneath >>> > >>> > >>> > >>> > The FIC2Lab runner is documented here: _http://fic2.github.io/runner_ >>> > >>> > >>> >>> >>> >>> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >>> > Dear Alex, >>> > >>> > It's fine with me. >>> > >>> > I believe that we should collocate here the discussion with >>> > FI-Content2 (Philipp) regarding the stuff they have developed for the >>> > FIC2-Lab. >>> > >>> > @Philipp: could you share some material (document describing your > work >>> > on FIC2-Lab, slides, whatever) describing what you have done and > send it >>> > to the fiware-chapter-architects mailing list as preparation for the >>> > discussion? >>> > >>> > @Alex: please also share the ideas you want to present on this > matter. >>> > >>> > By exchanging material prior to the meeting, we would be able to have >>> > a more fruitful meeting. >>> > >>> > Best regards, >>> > >>> > -- Juanjo >>> > >>> > On 18/05/15 11:07, Alex Glikson wrote: >>> >> I am out of office on the 25th. Would the following Monday work? >>> >> >>> >> Thanks, >>> >> Alex >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> From: Juanjo Hierro >>> >> To: Philipp Slusallek , Alex >>> >> Glikson/Haifa/IBM at IBMIL, >>> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >>> >> Date: 18/05/2015 11:53 AM >>> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & >> Docker >>> >> >> ------------------------------------------------------------------------ >>> >> >>> >> >>> >> >>> >> >>> >> I would suggest that we book the next architecture session (Monday >>> >> 25th) to cover this matter. >>> >> >>> >> Will you Alex be able to prepare a presentation? >>> >> >>> >> Cheers, >>> >> >>> >> -- Juanjo >>> >> >>> >> On 18/05/15 09:46, Philipp Slusallek wrote: >>> >> > Hi, >>> >> > >>> >> > This sounds very useful to cover as well in this or one of the next >>> >> > calls. It would also get the architectural activities started >> again. It >>> >> > might even make sense to invite (selected?) AB members from the UCs. >>> >> > >>> >> > For example, a lot of work that has been done within FIcontent to >> deploy >>> >> > our SEs and related GEs within Docker on top of FIWARE Lab. >> Particularly >>> >> > relevant could be the on-click deployment of entire arrangements >> of GEs >>> >> > and SEs. Better and more dedicated Docker support would certainly be >>> >> > very welcome as well. >>> >> > >>> >> > Unfortunately, I have a lecture this morning and cannot join >> until 12h. >>> >> > But Stefan Lemme from my group will be on the call. He has been a >> core >>> >> > member of the relevant FIC2-Lab task force in FIcontent. >>> >> > >>> >> > >>> >> > Best, >>> >> > >>> >> > Philipp >>> >> > >>> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >>> >> >> Maybe we can also dedicate some time to have an initial >> discussion on >>> >> >> the roadmap to adopt Linux Containers and in particular Docker in >>> >> FIWARE >>> >> >> Lab. >>> >> >> >>> >> >> Regards, >>> >> >> Alex >>> >> >>_______________________________________________ >> Fiware-cloud-containers mailing list >> Fiware-cloud-containers at lists.fiware.org >> https://lists.fiware.org/listinfo/fiware-cloud-containers >> >> >> >> _______________________________________________ >> Fiware-cloud-containers mailing list >> Fiware-cloud-containers at lists.fiware.org >> https://lists.fiware.org/listinfo/fiware-cloud-containers >> > > -- > > ------------------------------------------------------------------------- > Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH > Trippstadter Strasse 122, D-67663 Kaiserslautern > > Gesch?ftsf?hrung: > Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) > Dr. Walter Olthoff > Vorsitzender des Aufsichtsrats: > Prof. Dr. h.c. Hans A. Aukes > > Sitz der Gesellschaft: Kaiserslautern (HRB 2313) > USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 > --------------------------------------------------------------------------- > [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: philipp_slusallek.vcf Type: text/x-vcard Size: 441 bytes Desc: not available URL: From henar.munozfrutos at telefonica.com Tue Jun 2 09:32:51 2015 From: henar.munozfrutos at telefonica.com (=?Windows-1252?Q?HENAR_MU=D1OZ_FRUTOS?=) Date: Tue, 2 Jun 2015 07:32:51 +0000 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: References: <556C05BB.5060900@dfki.de> Message-ID: Hi Did you hear about https://www.mirantis.com/blog/integrating-openstack-and-kubernetes-with-murano/? Regards, Henar De: "GLIKSON at il.ibm.com" > Fecha: lunes, 1 de junio de 2015 22:36 Para: Philipp Slusallek > CC: "fiware-cloud-containers at lists.fiware.org" > Asunto: Re: [Fiware-cloud-containers] task-force kickoff I agree that user/developer perspective is important. However, there are several things that we need to take into account: 1. For users/developers who just want to *use* GEs (rather than develop new ones), FIWARE already provides tools to easily set them up - pre-built VM images, as well as SDC/Chef recipes and PaaS Blueprints (for bundles). Significant effort has been invested by GE owners to populate the corresponding artefacts and enable the automated/seamless deployment in FIWARE Lab. Moreover, as I mentioned, we are currently transforming those tools to better align with OpenStack, and we are also constantly improving them to make more capable/stable. Having said that, there is clearly space for further improvement (usability, documentation, features, integration, etc). In this context, it is not entirely clear whether/how containers (including Docker) can help, besides achieving higher density. 2. We do appreciate the value proposition of Docker as a new emerging standard for application packaging and life cycle management -- especially for 'cloud-born' applications. However, the tooling around Docker is rather diverse and immature, and we should be careful before we 'bet' on one (or more) of the options to become part of FIWARE (which aims at becoming a sustainable standard). For example, I haven't seen yet a good DevOps tool that would manage rolling upgrades (and other 'traditional' configuration management scenarios) for Docker-based applications. 3. It is likely that we would need to support both 'old' and 'new' ways to design and manage applications for the upcoming few years. Moreover, new paradigms may arise, addressing additional 'niches' on the spectrum between traditional servers (running full operating system plus an arbitrary software stack) and application containers (running a single process, possible with it's children processes). In my opinion, one of our challenges is to come up with a unified solution (and a roadmap) which would provide a reasonable value proposition in the short term, but would also like to be forward-compatible with new developments which will surely emerge around containers in the near future. For example, if we think that Chef is the way to go from configuration management perspective, maybe we should use and extend the Chef-Docker integration [1] . Alternatively (or in addition?), if we want to stick with Heat for orchestration, maybe we can invest in adding support for Magnum containers as resources in Heat, as well as LXD driver in Nova (for system containers). IMO, this is the problem space we need to further explore (and we started doing so). Regards, Alex [1] https://www.chef.io/solutions/containers/ ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Philipp Slusallek > To: Alex Glikson/Haifa/IBM at IBMIL, fiware-cloud-containers at lists.fiware.org Date: 01/06/2015 10:12 AM Subject: Re: [Fiware-cloud-containers] task-force kickoff ________________________________ Hi all, It is certainly the right thing to work on supporting containers natively in the Cloud chapter. It seems that a lot is currently being done in this regard within OpenStack and we should see how we can best contribute to that. However, I would like to point out another aspect that will probably have a much more direct and short-term impact on FIWARE and which we should be able to do in parallel: Focus on the user-perspective. I am talking about greatly simplifying the deployment of enablers for developers (and possibly end users) on FIWARE. Docker is an excellent tool for that as we have experienced for example in FIcontent. Its very easy for owners to provide their enabler and even easier for users to deploy and test them. As with the "Discover, Try, Tweak, Run" approach in FIcontent we can make it tremendously simpler for users to explore what we offer, try it out, share it with others via social networks, and deploy and use the necessary services for real. This is exactly what I believe FIWARE needs. The GEs are still somewhat difficult to access and the documentation for doing so leaves some things to be desired (at least judging from my reviewing job). Taking a much more user-focused approach is important now. So to make things concrete: We should explore (in parallel to your suggestions which is still needed, of course) to install something similar to FIC2-Lab (actually: now officially called "FIWARE Media&Content Lab"). We might be able to leverage the experience we got from there as a basis. But I am sure that together there are many improvements and extensions that should be done. In parallel, we would ask enabler owners to provide suitable dockerified versions that can easily be deployed via this system. This does not make sense for all and can be done incrementally as well. All that is then still needed is to create some nice examples that can be used for the Try and Tweak part, where developers can get first hand experience of enablers, by simply trying out, inspecting, and hacking on existing web-based applications that make use of one or more (bundles!) enablers from any Web browser and without ever having seen the cloud portal (they eventually need to but this is when they are hopefully hooked already). Again, this is not an alternative but a parallel activity that makes containers directly relevant for the developers right now (!) and which relies eventually on the ability to deploy enabler an easier, faster, and with a largely reduced resource footprint via the necessary technology in Cloud/OpenStack. Simply deploying docker within existing virtual machines should allow us to reduce resource usage by quite a bit. But we need to look into that. Best, Philipp Am 31.05.2015 um 22:36 schrieb Alex Glikson: > Below is a brief summary of the current FIWARE Cloud status with regards > to containers and Docker support. > > * FIWARE Cloud GEs currently deployed in FIWARE Lab provide VM-based > IaaS capabilities with OpenStack Nova (with KVM), Glance, Keystone, > Cinder and Neutron (with OVS). FIWARE Cloud also provides object > storage capabilities based on OpenStack Swift. Moreover, we support > PaaS-level capabilities, including the ability to provision and > auto-scale complex applications comprising sets of inter-dependent > VMs as well as the ability to install and configure the individual > software components within VMs using configuration management tools > such as Chef. The key user-visible notion behind this capability is > the notion of a Blueprint, which defines the application topology, > dependencies, auto-scaling rules, software configuration, etc > (conceptually similar to Amazon OpsWorks). > * In the last few months, the PaaS implementation is undergoing a > major transition from a 'proprietary' (although open source) code to > full adoption of OpenStack Heat and Murano (as well as contribution > of features to the Murano community, such as support for templates, > configuration management enhancements, etc). > * In parallel, we have started assessing (and prototyping) the options > to support/leverage Linux containers technologies, and Docker in > particular. We aim at two main use-cases: > 1. As a FIWARE Cloud provider (in FIWARE Lab), I want to continue > providing (almost?) the same PaaS/IaaS capabilities, but at the > same time being able to admit much more users/workloads on the > same hardware, by leveraging Containers technologies. > 2. As a FIWARE Cloud user, I want to be able to leverage the Docker > ecosystem, including public images available at Docker Hub as > well as existing tools (TBD: which tools exactly?), while > hosting my applications on the FIWARE Lab > * The 2nd use-case can be satisfied in several ways, such as: > 1. Treat Docker-based cluster as PaaS application (managed by > FIWARE Cloud PaaS), while the cloud middleware (IaaS and PaaS) > is not aware of individual containers, Docker images, etc. > 2. Make Docker containers 1st class citizens in PaaS -- e.g., as an > additional delivery mechanism in addition to Chef, deployed on > regular IaaS virtualized infra (providing isolation, accounting, > etc). In this case each Docker cluster is contained within a > single tenant (and even single application within a tenant?) > 3. Make Docker containers 1st class citizens in IaaS deployed on > bare-metal, making PaaS use it via a standardized API > * The current (tentative) plan is to apply a 2-phase approach to > introduce containers: phase-1: start by deploying nova-docker in > FIWARE lab in order to enable use-case n.1, and then phase-2: > gradually migrate to OpenStack Magnum, while incorporating the > Docker-native capabilities into Murano in order to support use-case > n.2. Of course, the devil is in the details, so we need to discuss > further, to understand the exact requirements and implications. > > > Regards, > Alex > > > > > From: Alex Glikson/Haifa/IBM at IBMIL > To: fiware-cloud-containers at lists.fiware.org > Date: 30/05/2015 07:06 PM > Subject: [Fiware-cloud-containers] task-force kickoff > Sent by: fiware-cloud-containers-bounces at lists.fiware.org > ------------------------------------------------------------------------ > > > > Dear all, > > Welcome to 'fiware-cloud-containers' mailing list! We will use this > mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. > support for Linux containers (such as Docker), including use-cases & > requirements, associated IaaS and PaaS-style capabilities, concrete > plans going forwards, potential collaboration with teams working on > similar/related technologies, etc. > > Currently we have the following people subscribed to this mailing list: > IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for > IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud > leader), Doron (working with Ezra on containers) > TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads > for the PaaS-related work in FIWARE Cloud) > DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter > leader/architect) > Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old > FI-WARE project) > > Let me know if anyone is missing. > > As a background, please, read the thread below, including the very nice > summary by Mario on the related investigation and implementation in > FI-Content2 project. I will send a separate email briefly summarizing > the situation in FIWARE. > * > All *-- notice that we are going to have a phone call to discuss this > topic during the upcoming FIWARE architects interlock on Monday, June > 1at, 10:30-12:30 CET. Let me know if you can't join.* > Mario *-- would you be able to join & explain/demonstrate the FIC2Lab > solution? > > Thanks, > Alex > > > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- > > fiware-chapter-architects-bounces at lists.fi-ware.org wrote on 26/05/2015 > 08:37:16 AM: > >> From: Alex Glikson/Haifa/IBM at IBMIL >> To: Philipp Slusallek > >> Cc: fiware-chapter-architects at lists.fi-ware.org, Kenneth Nagin/ >> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >> Date: 26/05/2015 08:37 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> Sent by: fiware-chapter-architects-bounces at lists.fi-ware.org >> >> Hi Philipp, >> >> It is great to hear that FIC2 team is willing to contribute! >> In a nutshell, our current plan is to start with surfacing >> containers via Nova, and then gradually migrate to Magnum. In terms >> of orchestration, we currently work with Heat and Murano, and are >> assessing the approaches to support containers (OpenStack alignment >> being high priority, but also the ability to surface capabilities >> unique to containers). >> I suggest that we form a small taskforce to explore this topic. >> Please, send me a list of people to involve. >> >> Thanks, >> Alex >> >> > ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> >> >> >> From: Philipp Slusallek > >> To: Juanjo Hierro >, Alex >> Glikson/Haifa/IBM at IBMIL >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >> architects at lists.fi-ware.org, Kenneth Nagin/Haifa/IBM at IBMIL >> Date: 26/05/2015 07:21 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> >> >> >> Hi, >> >> Here is some first input from FIC2 on Docker compiled from Mario. As I >> mentioned before, we have a full system working that deploys entire sets >> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >> >> FIC2 would be happy to work with FIWARE to integrate them in whatever >> way we at FIWARE decide to implement Docker. Right now we deploy it on >> top of the VMs offered by FIWARE but a more native implementation would >> be fine as well and save much resources (which seems to be a major issue). >> >> A presentation by Canonical recently showed a >10x improvement in >> resource usage with their containers over plain OpenStack (which is >> probably where we are -- but then Intel claims even better numbers in >> their Clear Containers solution using traditional VMs). >> >> Since we have experience already, it seems very useful to make Docker >> available especially since deployment of SW is much easier too and the >> same SW can be deployed essentially anywhere (including a local machine). >> >> >> Best, >> >> Philipp >> >> > There are several ways in which FIWARE could support Docker. My >> assumption is that the plan is to add to OpenStack a Docker driver. >> It would enable the creation of instances which are not VMs but >> OpenStack containers, from images which are not ISOs but Docker >> images. It?s described here: _https://wiki.openstack.org/wiki/Docker_ >> > >> > >> > >> > Advantages: >> > >> > + Launches faster than VM instances >> > >> > + Less resources used from quotas in FIWARE nodes >> > >> > + images of SEs smaller and easier to build (using Dockerfiles) >> > >> > >> > >> > Open questions / disadvantages: >> > >> > - Will it be possible to pull images from DockerHub (preferred >> approach), or do they need to be uploaded to the Glance image >> repository of each FIWARE node? >> > >> > - Is the only way to launch containers the Cloud Portal or >> OpenStack API? Not being able to target a Docker API would reduce >> the value of Docker and its ecosystem of tools, e.g.: docker-compose >> which enables launching applications composed of multiple containers. >> > >> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >> reverse proxy that forwards HTTP requests to container1.john- >> smith.cloud.fiware.org to the right container port? >> > >> > - How would links between containers running on different hosts >> work? Using Weave? >> > >> > - Mounting volumes in Docker is a very common way of passing >> configuration files, having persistence, etc. How would this work > inOpenStack? >> > >> > >> > >> > Our suggestion: >> > >> > * Ability to use the Docker API remotely (i.e. command-line tools >> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >> > >> > * Ability to pull containers directly from DockerHub with a common >> (transparent?) image mirror in each FIWARE node >> > >> > >> > >> > >> > >> > >> > >> > Regarding FIC2Lab runner, our wishlist was: >> > >> > 1. ability to use the same tools locally and remotely in any cloud >> (ideally use docker-machine against any cloud provider, in >> particular FIWARE nodes) >> > >> > 2. nice web user interface hiding the complexity of firewalling, >> security pairs, public IPs, etc. >> > >> > 3. compatibility with the tools of the docker ecosystem: docker >> CLI, compose, swarm ... >> > >> > >> > >> > We compared tools such as tutum (free multi-tenant online >> service), shipyard (open-source web UI) and panamax (open-source web >> UI and orchestrator). >> > >> > And we decided to go for Panamax, which is nice for #1 and #2 but >> not #3 so far. Here's a few pluses and minuses: >> > >> > + nice UI with DockerHub search >> > >> > + composite application, with application template search >> > >> > - It's not multi-tenant, which is good for portability but bad for >> software updates >> > >> > - its model for composite applications is different from docker- >> compose (for now) >> > >> > - the UI can't synchronize with actions performed in Docker CLI or >> API because it's using CoreOS underneath >> > >> > >> > >> > The FIC2Lab runner is documented here: _http://fic2.github.io/runner_ >> > >> > >> >> >> >> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >> > Dear Alex, >> > >> > It's fine with me. >> > >> > I believe that we should collocate here the discussion with >> > FI-Content2 (Philipp) regarding the stuff they have developed for the >> > FIC2-Lab. >> > >> > @Philipp: could you share some material (document describing your work >> > on FIC2-Lab, slides, whatever) describing what you have done and send it >> > to the fiware-chapter-architects mailing list as preparation for the >> > discussion? >> > >> > @Alex: please also share the ideas you want to present on this matter. >> > >> > By exchanging material prior to the meeting, we would be able to have >> > a more fruitful meeting. >> > >> > Best regards, >> > >> > -- Juanjo >> > >> > On 18/05/15 11:07, Alex Glikson wrote: >> >> I am out of office on the 25th. Would the following Monday work? >> >> >> >> Thanks, >> >> Alex >> >> >> >> >> >> >> >> >> >> >> >> >> >> From: Juanjo Hierro > >> >> To: Philipp Slusallek >, Alex >> >> Glikson/Haifa/IBM at IBMIL, > >> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >> >> Date: 18/05/2015 11:53 AM >> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & > Docker >> >> > ------------------------------------------------------------------------ >> >> >> >> >> >> >> >> >> >> I would suggest that we book the next architecture session (Monday >> >> 25th) to cover this matter. >> >> >> >> Will you Alex be able to prepare a presentation? >> >> >> >> Cheers, >> >> >> >> -- Juanjo >> >> >> >> On 18/05/15 09:46, Philipp Slusallek wrote: >> >> > Hi, >> >> > >> >> > This sounds very useful to cover as well in this or one of the next >> >> > calls. It would also get the architectural activities started > again. It >> >> > might even make sense to invite (selected?) AB members from the UCs. >> >> > >> >> > For example, a lot of work that has been done within FIcontent to > deploy >> >> > our SEs and related GEs within Docker on top of FIWARE Lab. > Particularly >> >> > relevant could be the on-click deployment of entire arrangements > of GEs >> >> > and SEs. Better and more dedicated Docker support would certainly be >> >> > very welcome as well. >> >> > >> >> > Unfortunately, I have a lecture this morning and cannot join > until 12h. >> >> > But Stefan Lemme from my group will be on the call. He has been a > core >> >> > member of the relevant FIC2-Lab task force in FIcontent. >> >> > >> >> > >> >> > Best, >> >> > >> >> > Philipp >> >> > >> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >> >> >> Maybe we can also dedicate some time to have an initial > discussion on >> >> >> the roadmap to adopt Linux Containers and in particular Docker in >> >> FIWARE >> >> >> Lab. >> >> >> >> >> >> Regards, >> >> >> Alex >> >> >>_______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] ________________________________ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o -------------- next part -------------- An HTML attachment was scrubbed... URL: From EZRA at il.ibm.com Tue Jun 2 09:39:26 2015 From: EZRA at il.ibm.com (Ezra Silvera) Date: Tue, 2 Jun 2015 10:39:26 +0300 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: <556D310D.2080405@dfki.de> References: <556C05BB.5060900@dfki.de> <556D310D.2080405@dfki.de> Message-ID: Thanks Ezra Silvera fiware-cloud-containers-bounces at lists.fiware.org wrote on 02/06/2015 07:29:01 AM: > From: Philipp Slusallek > To: Alex Glikson/Haifa/IBM at IBMIL > Cc: fiware-cloud-containers at lists.fiware.org > Date: 02/06/2015 07:29 AM > Subject: Re: [Fiware-cloud-containers] task-force kickoff > Sent by: fiware-cloud-containers-bounces at lists.fiware.org > > Hi Alex, all, > > Am 01.06.2015 um 22:36 schrieb Alex Glikson: > > 1. For users/developers who just want to *use* GEs (rather than develop > > new ones), FIWARE already provides tools to easily set them up - > > pre-built VM images, as well as SDC/Chef recipes and PaaS Blueprints > > (for bundles). Significant effort has been invested by GE owners to > > populate the corresponding artefacts and enable the > > automated/seamless deployment in FIWARE Lab. Moreover, as I > > mentioned, we are currently transforming those tools to better align > > with OpenStack, and we are also constantly improving them to make > > more capable/stable. Having said that, there is clearly space for > > further improvement (usability, documentation, features, > > integration, etc). In this context, it is not entirely clear > > whether/how containers (including Docker) can help, besides > > achieving higher density. > > Docker is just dramatically simpler to deploy on any infrastructure > including your home PC (e.g. for debugging, testing, or whatever). VM > images are a pain and are very inefficient if I have many smaller > services that I need to deploy. Bundles help here, but are too > inflexible as the combination I want probably does not exist. Building a > setup myself is hard particularly for novices. With docker we can deploy > whatever combination of services with a few commands and very > efficiently, too. > > Actually we do not even want to deal with Docker. Its just a tools that > simplifies our lives. Our users just want services and deploy them with > a few clicks. That is what we now offer in FIC2-Lab. > > Of course, this is not for setting up a large data center or special > deployment but then this is not what people typically do in FIWARE Lab > either. > > > 2. We do appreciate the value proposition of Docker as a new emerging > > standard for application packaging and life cycle management -- > > especially for 'cloud-born' applications. However, the tooling > > around Docker is rather diverse and immature, and we should be > > careful before we 'bet' on one (or more) of the options to become > > part of FIWARE (which aims at becoming a sustainable standard). For > > example, I haven't seen yet a good DevOps tool that would manage > > rolling upgrades (and other 'traditional' configuration management > > scenarios) for Docker-based applications. > > Fully agree, its certainly not as evolved as other infrastructures. But > does this mean we should not start using them, particularly if we can > offer significant easy of use to our users? We can "learn on the job" at > Internet time :-). > > > 3. It is likely that we would need to support both 'old' and 'new' ways > > to design and manage applications for the upcoming few years. > > Moreover, new paradigms may arise, addressing additional 'niches' on > > the spectrum between traditional servers (running full operating > > system plus an arbitrary software stack) and application containers > > (running a single process, possible with it's children processes). > > In my opinion, one of our challenges is to come up with a unified > > solution (and a roadmap) which would provide a reasonable value > > proposition in the short term, but would also like to be > > forward-compatible with new developments which will surely emerge > > around containers in the near future. For example, if we think that > > Chef is the way to go from configuration management perspective, > > maybe we should use and extend the Chef-Docker integration [1] . > > Alternatively (or in addition?), if we want to stick with Heat for > > orchestration, maybe we can invest in adding support for Magnum > > containers as resources in Heat, as well as LXD driver in Nova (for > > system containers). IMO, this is the problem space we need to > > further explore (and we started doing so). > > As I said, I agree here. But this is the backend and something for the > at least medium term future. > > Currently any FIWARE user has to first set up his machines in the Cloud > portal even if all he wants to do is try out and play with one service > (or a couple of them). This is crazy. > > Why can't we just give him the services he wants to play with -- with > the click of a button ("Try"). Another button and he has his own copy > and can modify it to his/her hearts contain ("Tweak"). Another click and > he has his setup be deployed in some cloud or even his home machine > ("Run"). > I think this is somewhat orthogonal to the Docker discussion. Although it's true that with Docker you can achieve such "one-click" behavior, you can build such capability with Vms as well. I'm not claiming that this is what we should do but we probably should separate the discussion of completely new capabilities and overall change to the service life cycle from the discussion of how to integrate Docker (and which tools to use) Also let's not forget that there are also limitations to containers (security, isolation, gaps in networking, etc..) so adding to the immaturity of containers today we are probably going to still see a lot of demand for VM based solutions. In general I don't think the discussion now is whether Docker/containers are better then Vms but rather what is the best plan to **smoothly** introduce containers support into FIWARE. > And if he came up with some nice ideas allow whim to share them easily > through Github and let others build on top of that. > > Think of this as a github for services. This is something that has not > been done yet and no infrastructure yet allows people to do this easily. > We have the means to very easily offer that (FIC2-Lab already does > exactly this). It seems that this could be a huge hit for developers, > similar to github. > > We need to provide better solutions for our users NOW. We cannot wait > until all of this infrastructure has settled down. And there are > solutions that already work NOW. > > Can we back all this up with better backend technology -- of course we > can and we should! But these are (largely) independent activities from > my point of view. > > > Best, > > Philipp > > > > > > Regards, > > Alex > > > > [1] https://www.chef.io/solutions/containers/ > > > > > ==================================================================================== > > Alex Glikson > > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > > +972-54-6466667 | Fax: +972-4-8296112 > > > > > > > > > > > > From: Philipp Slusallek > > To: Alex Glikson/Haifa/IBM at IBMIL, > > fiware-cloud-containers at lists.fiware.org > > Date: 01/06/2015 10:12 AM > > Subject: Re: [Fiware-cloud-containers] task-force kickoff > > ------------------------------------------------------------------------ > > > > > > > > Hi all, > > > > It is certainly the right thing to work on supporting containers > > natively in the Cloud chapter. It seems that a lot is currently being > > done in this regard within OpenStack and we should see how we can best > > contribute to that. > > > > However, I would like to point out another aspect that will probably > > have a much more direct and short-term impact on FIWARE and which we > > should be able to do in parallel: Focus on the user-perspective. I am > > talking about greatly simplifying the deployment of enablers for > > developers (and possibly end users) on FIWARE. > > > > Docker is an excellent tool for that as we have experienced for example > > in FIcontent. Its very easy for owners to provide their enabler and even > > easier for users to deploy and test them. As with the "Discover, Try, > > Tweak, Run" approach in FIcontent we can make it tremendously simpler > > for users to explore what we offer, try it out, share it with others via > > social networks, and deploy and use the necessary services for real. > > > > This is exactly what I believe FIWARE needs. The GEs are still somewhat > > difficult to access and the documentation for doing so leaves some > > things to be desired (at least judging from my reviewing job). Taking a > > much more user-focused approach is important now. > > > > So to make things concrete: We should explore (in parallel to your > > suggestions which is still needed, of course) to install something > > similar to FIC2-Lab (actually: now officially called "FIWARE > > Media&Content Lab"). We might be able to leverage the experience we got > > from there as a basis. But I am sure that together there are many > > improvements and extensions that should be done. > > > > In parallel, we would ask enabler owners to provide suitable dockerified > > versions that can easily be deployed via this system. This does not make > > sense for all and can be done incrementally as well. All that is then > > still needed is to create some nice examples that can be used for the > > Try and Tweak part, where developers can get first hand experience of > > enablers, by simply trying out, inspecting, and hacking on existing > > web-based applications that make use of one or more (bundles!) enablers > > from any Web browser and without ever having seen the cloud portal (they > > eventually need to but this is when they are hopefully hooked already). > > > > Again, this is not an alternative but a parallel activity that makes > > containers directly relevant for the developers right now (!) and which > > relies eventually on the ability to deploy enabler an easier, faster, > > and with a largely reduced resource footprint via the necessary > > technology in Cloud/OpenStack. Simply deploying docker within existing > > virtual machines should allow us to reduce resource usage by quite a > > bit. But we need to look into that. > > > > > > Best, > > > > Philipp > > > > Am 31.05.2015 um 22:36 schrieb Alex Glikson: > >> Below is a brief summary of the current FIWARE Cloud status with regards > >> to containers and Docker support. > >> > >> * FIWARE Cloud GEs currently deployed in FIWARE Lab provide VM-based > >> IaaS capabilities with OpenStack Nova (with KVM), Glance, Keystone, > >> Cinder and Neutron (with OVS). FIWARE Cloud also provides object > >> storage capabilities based on OpenStack Swift. Moreover, we support > >> PaaS-level capabilities, including the ability to provision and > >> auto-scale complex applications comprising sets of inter-dependent > >> VMs as well as the ability to install and configure the individual > >> software components within VMs using configuration management tools > >> such as Chef. The key user-visible notion behind this capability is > >> the notion of a Blueprint, which defines the application topology, > >> dependencies, auto-scaling rules, software configuration, etc > >> (conceptually similar to Amazon OpsWorks). > >> * In the last few months, the PaaS implementation is undergoing a > >> major transition from a 'proprietary' (although open source) code to > >> full adoption of OpenStack Heat and Murano (as well as contribution > >> of features to the Murano community, such as support for templates, > >> configuration management enhancements, etc). > >> * In parallel, we have started assessing (and prototyping) the options > >> to support/leverage Linux containers technologies, and Docker in > >> particular. We aim at two main use-cases: > >> 1. As a FIWARE Cloud provider (in FIWARE Lab), I want to continue > >> providing (almost?) the same PaaS/IaaS capabilities, but at the > >> same time being able to admit much more users/workloads on the > >> same hardware, by leveraging Containers technologies. > >> 2. As a FIWARE Cloud user, I want to be able to leverage the Docker > >> ecosystem, including public images available at Docker Hub as > >> well as existing tools (TBD: which tools exactly?), while > >> hosting my applications on the FIWARE Lab > >> * The 2nd use-case can be satisfied in several ways, such as: > >> 1. Treat Docker-based cluster as PaaS application (managed by > >> FIWARE Cloud PaaS), while the cloud middleware (IaaS and PaaS) > >> is not aware of individual containers, Docker images, etc. > >> 2. Make Docker containers 1st class citizens in PaaS -- e.g., as an > >> additional delivery mechanism in addition to Chef, deployed on > >> regular IaaS virtualized infra (providing isolation, accounting, > >> etc). In this case each Docker cluster is contained within a > >> single tenant (and even single application within a tenant?) > >> 3. Make Docker containers 1st class citizens in IaaS deployed on > >> bare-metal, making PaaS use it via a standardized API > >> * The current (tentative) plan is to apply a 2-phase approach to > >> introduce containers: phase-1: start by deploying nova-docker in > >> FIWARE lab in order to enable use-case n.1, and then phase-2: > >> gradually migrate to OpenStack Magnum, while incorporating the > >> Docker-native capabilities into Murano in order to support use-case > >> n.2. Of course, the devil is in the details, so we need to discuss > >> further, to understand the exact requirements and implications. > >> > >> > >> Regards, > >> Alex > >> > >> > >> > >> > >> From: Alex Glikson/Haifa/IBM at IBMIL > >> To: fiware-cloud-containers at lists.fiware.org > >> Date: 30/05/2015 07:06 PM > >> Subject: [Fiware-cloud-containers] task-force kickoff > >> Sent by: fiware-cloud-containers-bounces at lists.fiware.org > >> ------------------------------------------------------------------------ > >> > >> > >> > >> Dear all, > >> > >> Welcome to 'fiware-cloud-containers' mailing list! We will use this > >> mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. > >> support for Linux containers (such as Docker), including use-cases & > >> requirements, associated IaaS and PaaS-style capabilities, concrete > >> plans going forwards, potential collaboration with teams working on > >> similar/related technologies, etc. > >> > >> Currently we have the following people subscribed to this mailing list: > >> IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for > >> IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud > >> leader), Doron (working with Ezra on containers) > >> TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads > >> for the PaaS-related work in FIWARE Cloud) > >> DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter > >> leader/architect) > >> Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old > >> FI-WARE project) > >> > >> Let me know if anyone is missing. > >> > >> As a background, please, read the thread below, including the very nice > >> summary by Mario on the related investigation and implementation in > >> FI-Content2 project. I will send a separate email briefly summarizing > >> the situation in FIWARE. > >> * > >> All *-- notice that we are going to have a phone call to discuss this > >> topic during the upcoming FIWARE architects interlock on Monday, June > >> 1at, 10:30-12:30 CET. Let me know if you can't join.* > >> Mario *-- would you be able to join & explain/demonstrate the FIC2Lab > >> solution? > >> > >> Thanks, > >> Alex > >> > >> > >> > >> > > > ==================================================================================== > >> Alex Glikson > >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > >> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > >> +972-54-6466667 | Fax: +972-4-8296112 > >> > >> > >> ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- > >> > >> fiware-chapter-architects-bounces at lists.fi-ware.org wrote on 26/05/2015 > >> 08:37:16 AM: > >> > >>> From: Alex Glikson/Haifa/IBM at IBMIL > >>> To: Philipp Slusallek > >>> Cc: fiware-chapter-architects at lists.fi-ware.org, Kenneth Nagin/ > >>> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL > >>> Date: 26/05/2015 08:37 AM > >>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker > >>> Sent by: fiware-chapter-architects-bounces at lists.fi-ware.org > >>> > >>> Hi Philipp, > >>> > >>> It is great to hear that FIC2 team is willing to contribute! > >>> In a nutshell, our current plan is to start with surfacing > >>> containers via Nova, and then gradually migrate to Magnum. In terms > >>> of orchestration, we currently work with Heat and Murano, and are > >>> assessing the approaches to support containers (OpenStack alignment > >>> being high priority, but also the ability to surface capabilities > >>> unique to containers). > >>> I suggest that we form a small taskforce to explore this topic. > >>> Please, send me a list of people to involve. > >>> > >>> Thanks, > >>> Alex > >>> > >>> > >> > > > ==================================================================================== > >>> Alex Glikson > >>> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > >>> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > >>> +972-54-6466667 | Fax: +972-4-8296112 > >>> > >>> > >>> > >>> > >>> > >>> From: Philipp Slusallek > >>> To: Juanjo Hierro , Alex > >>> Glikson/Haifa/IBM at IBMIL > >>> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- > >>> architects at lists.fi-ware.org, Kenneth Nagin/Haifa/IBM at IBMIL > >>> Date: 26/05/2015 07:21 AM > >>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker > >>> > >>> > >>> > >>> Hi, > >>> > >>> Here is some first input from FIC2 on Docker compiled from Mario. As I > >>> mentioned before, we have a full system working that deploys entire sets > >>> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. > >>> > >>> FIC2 would be happy to work with FIWARE to integrate them in whatever > >>> way we at FIWARE decide to implement Docker. Right now we deploy it on > >>> top of the VMs offered by FIWARE but a more native implementation would > >>> be fine as well and save much resources (which seems to be a major > > issue). > >>> > >>> A presentation by Canonical recently showed a >10x improvement in > >>> resource usage with their containers over plain OpenStack (which is > >>> probably where we are -- but then Intel claims even better numbers in > >>> their Clear Containers solution using traditional VMs). > >>> > >>> Since we have experience already, it seems very useful to make Docker > >>> available especially since deployment of SW is much easier too and the > >>> same SW can be deployed essentially anywhere (including a local machine). > >>> > >>> > >>> Best, > >>> > >>> Philipp > >>> > >>> > There are several ways in which FIWARE could support Docker. My > >>> assumption is that the plan is to add to OpenStack a Docker driver. > >>> It would enable the creation of instances which are not VMs but > >>> OpenStack containers, from images which are not ISOs but Docker > >>> images. It?s described here: _https://wiki.openstack.org/wiki/Docker_ > >>> > > >>> > > >>> > > >>> > Advantages: > >>> > > >>> > + Launches faster than VM instances > >>> > > >>> > + Less resources used from quotas in FIWARE nodes > >>> > > >>> > + images of SEs smaller and easier to build (using Dockerfiles) > >>> > > >>> > > >>> > > >>> > Open questions / disadvantages: > >>> > > >>> > - Will it be possible to pull images from DockerHub (preferred > >>> approach), or do they need to be uploaded to the Glance image > >>> repository of each FIWARE node? > >>> > > >>> > - Is the only way to launch containers the Cloud Portal or > >>> OpenStack API? Not being able to target a Docker API would reduce > >>> the value of Docker and its ecosystem of tools, e.g.: docker-compose > >>> which enables launching applications composed of multiple containers. > >>> > > >>> > - How to solve the limits in terms of IPs? Maybe with a DNS-based > >>> reverse proxy that forwards HTTP requests to container1.john- > >>> smith.cloud.fiware.org to the right container port? > >>> > > >>> > - How would links between containers running on different hosts > >>> work? Using Weave? > >>> > > >>> > - Mounting volumes in Docker is a very common way of passing > >>> configuration files, having persistence, etc. How would this work > >> inOpenStack? > >>> > > >>> > > >>> > > >>> > Our suggestion: > >>> > > >>> > * Ability to use the Docker API remotely (i.e. command-line tools > >>> such as docker and docker-compose, Kitematic for a Mac OS X GUI) > >>> > > >>> > * Ability to pull containers directly from DockerHub with a common > >>> (transparent?) image mirror in each FIWARE node > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > Regarding FIC2Lab runner, our wishlist was: > >>> > > >>> > 1. ability to use the same tools locally and remotely in any cloud > >>> (ideally use docker-machine against any cloud provider, in > >>> particular FIWARE nodes) > >>> > > >>> > 2. nice web user interface hiding the complexity of firewalling, > >>> security pairs, public IPs, etc. > >>> > > >>> > 3. compatibility with the tools of the docker ecosystem: docker > >>> CLI, compose, swarm ... > >>> > > >>> > > >>> > > >>> > We compared tools such as tutum (free multi-tenant online > >>> service), shipyard (open-source web UI) and panamax (open-source web > >>> UI and orchestrator). > >>> > > >>> > And we decided to go for Panamax, which is nice for #1 and #2 but > >>> not #3 so far. Here's a few pluses and minuses: > >>> > > >>> > + nice UI with DockerHub search > >>> > > >>> > + composite application, with application template search > >>> > > >>> > - It's not multi-tenant, which is good for portability but bad for > >>> software updates > >>> > > >>> > - its model for composite applications is different from docker- > >>> compose (for now) > >>> > > >>> > - the UI can't synchronize with actions performed in Docker CLI or > >>> API because it's using CoreOS underneath > >>> > > >>> > > >>> > > >>> > The FIC2Lab runner is documented here: _http://fic2.github.io/runner_ > >>> > > >>> > > >>> > >>> > >>> > >>> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: > >>> > Dear Alex, > >>> > > >>> > It's fine with me. > >>> > > >>> > I believe that we should collocate here the discussion with > >>> > FI-Content2 (Philipp) regarding the stuff they have developed for the > >>> > FIC2-Lab. > >>> > > >>> > @Philipp: could you share some material (document describing your > > work > >>> > on FIC2-Lab, slides, whatever) describing what you have done and > > send it > >>> > to the fiware-chapter-architects mailing list as preparation for the > >>> > discussion? > >>> > > >>> > @Alex: please also share the ideas you want to present on this > > matter. > >>> > > >>> > By exchanging material prior to the meeting, we would be able to have > >>> > a more fruitful meeting. > >>> > > >>> > Best regards, > >>> > > >>> > -- Juanjo > >>> > > >>> > On 18/05/15 11:07, Alex Glikson wrote: > >>> >> I am out of office on the 25th. Would the following Monday work? > >>> >> > >>> >> Thanks, > >>> >> Alex > >>> >> > >>> >> > >>> >> > >>> >> > >>> >> > >>> >> > >>> >> From: Juanjo Hierro > >>> >> To: Philipp Slusallek , Alex > >>> >> Glikson/Haifa/IBM at IBMIL, > >>> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL > >>> >> Date: 18/05/2015 11:53 AM > >>> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & > >> Docker > >>> >> > >> ------------------------------------------------------------------------ > >>> >> > >>> >> > >>> >> > >>> >> > >>> >> I would suggest that we book the next architecture session (Monday > >>> >> 25th) to cover this matter. > >>> >> > >>> >> Will you Alex be able to prepare a presentation? > >>> >> > >>> >> Cheers, > >>> >> > >>> >> -- Juanjo > >>> >> > >>> >> On 18/05/15 09:46, Philipp Slusallek wrote: > >>> >> > Hi, > >>> >> > > >>> >> > This sounds very useful to cover as well in this or one of the next > >>> >> > calls. It would also get the architectural activities started > >> again. It > >>> >> > might even make sense to invite (selected?) AB members from the UCs. > >>> >> > > >>> >> > For example, a lot of work that has been done within FIcontent to > >> deploy > >>> >> > our SEs and related GEs within Docker on top of FIWARE Lab. > >> Particularly > >>> >> > relevant could be the on-click deployment of entire arrangements > >> of GEs > >>> >> > and SEs. Better and more dedicated Docker support would certainly be > >>> >> > very welcome as well. > >>> >> > > >>> >> > Unfortunately, I have a lecture this morning and cannot join > >> until 12h. > >>> >> > But Stefan Lemme from my group will be on the call. He has been a > >> core > >>> >> > member of the relevant FIC2-Lab task force in FIcontent. > >>> >> > > >>> >> > > >>> >> > Best, > >>> >> > > >>> >> > Philipp > >>> >> > > >>> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: > >>> >> >> Maybe we can also dedicate some time to have an initial > >> discussion on > >>> >> >> the roadmap to adopt Linux Containers and in particular Docker in > >>> >> FIWARE > >>> >> >> Lab. > >>> >> >> > >>> >> >> Regards, > >>> >> >> Alex > >>> >> >>_______________________________________________ > >> Fiware-cloud-containers mailing list > >> Fiware-cloud-containers at lists.fiware.org > >> https://lists.fiware.org/listinfo/fiware-cloud-containers > >> > >> > >> > >> _______________________________________________ > >> Fiware-cloud-containers mailing list > >> Fiware-cloud-containers at lists.fiware.org > >> https://lists.fiware.org/listinfo/fiware-cloud-containers > >> > > > > -- > > > > ------------------------------------------------------------------------- > > Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH > > Trippstadter Strasse 122, D-67663 Kaiserslautern > > > > Gesch?ftsf?hrung: > > Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) > > Dr. Walter Olthoff > > Vorsitzender des Aufsichtsrats: > > Prof. Dr. h.c. Hans A. Aukes > > > > Sitz der Gesellschaft: Kaiserslautern (HRB 2313) > > USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 > > --------------------------------------------------------------------------- > > [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] > > -- > > ------------------------------------------------------------------------- > Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH > Trippstadter Strasse 122, D-67663 Kaiserslautern > > Gesch?ftsf?hrung: > Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) > Dr. Walter Olthoff > Vorsitzender des Aufsichtsrats: > Prof. Dr. h.c. Hans A. Aukes > > Sitz der Gesellschaft: Kaiserslautern (HRB 2313) > USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 > --------------------------------------------------------------------------- > [attachment "philipp_slusallek.vcf" deleted by Ezra Silvera/Haifa/ > IBM] _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers -------------- next part -------------- An HTML attachment was scrubbed... URL: From GLIKSON at il.ibm.com Tue Jun 2 10:04:01 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Tue, 2 Jun 2015 11:04:01 +0300 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: References: <556C05BB.5060900@dfki.de> Message-ID: Yes. It addresses use-case 2.1 in my list down the thread, which is also pretty much the mode FC2Lab is addressing --- deployment of a Docker-based cluster as an 'application' managed by Heat/Murano. Regards, Alex From: HENAR MU?OZ FRUTOS To: Alex Glikson/Haifa/IBM at IBMIL, Philipp Slusallek Cc: "fiware-cloud-containers at lists.fiware.org" Date: 02/06/2015 10:33 AM Subject: Re: [Fiware-cloud-containers] task-force kickoff Hi Did you hear about https://www.mirantis.com/blog/integrating-openstack-and-kubernetes-with-murano/ ? Regards, Henar De: "GLIKSON at il.ibm.com" Fecha: lunes, 1 de junio de 2015 22:36 Para: Philipp Slusallek CC: "fiware-cloud-containers at lists.fiware.org" < fiware-cloud-containers at lists.fiware.org> Asunto: Re: [Fiware-cloud-containers] task-force kickoff I agree that user/developer perspective is important. However, there are several things that we need to take into account: 1. For users/developers who just want to *use* GEs (rather than develop new ones), FIWARE already provides tools to easily set them up - pre-built VM images, as well as SDC/Chef recipes and PaaS Blueprints (for bundles). Significant effort has been invested by GE owners to populate the corresponding artefacts and enable the automated/seamless deployment in FIWARE Lab. Moreover, as I mentioned, we are currently transforming those tools to better align with OpenStack, and we are also constantly improving them to make more capable/stable. Having said that, there is clearly space for further improvement (usability, documentation, features, integration, etc). In this context, it is not entirely clear whether/how containers (including Docker) can help, besides achieving higher density. 2. We do appreciate the value proposition of Docker as a new emerging standard for application packaging and life cycle management -- especially for 'cloud-born' applications. However, the tooling around Docker is rather diverse and immature, and we should be careful before we 'bet' on one (or more) of the options to become part of FIWARE (which aims at becoming a sustainable standard). For example, I haven't seen yet a good DevOps tool that would manage rolling upgrades (and other 'traditional' configuration management scenarios) for Docker-based applications. 3. It is likely that we would need to support both 'old' and 'new' ways to design and manage applications for the upcoming few years. Moreover, new paradigms may arise, addressing additional 'niches' on the spectrum between traditional servers (running full operating system plus an arbitrary software stack) and application containers (running a single process, possible with it's children processes). In my opinion, one of our challenges is to come up with a unified solution (and a roadmap) which would provide a reasonable value proposition in the short term, but would also like to be forward-compatible with new developments which will surely emerge around containers in the near future. For example, if we think that Chef is the way to go from configuration management perspective, maybe we should use and extend the Chef-Docker integration [1] . Alternatively (or in addition?), if we want to stick with Heat for orchestration, maybe we can invest in adding support for Magnum containers as resources in Heat, as well as LXD driver in Nova (for system containers). IMO, this is the problem space we need to further explore (and we started doing so). Regards, Alex [1] https://www.chef.io/solutions/containers/ ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Philipp Slusallek To: Alex Glikson/Haifa/IBM at IBMIL, fiware-cloud-containers at lists.fiware.org Date: 01/06/2015 10:12 AM Subject: Re: [Fiware-cloud-containers] task-force kickoff Hi all, It is certainly the right thing to work on supporting containers natively in the Cloud chapter. It seems that a lot is currently being done in this regard within OpenStack and we should see how we can best contribute to that. However, I would like to point out another aspect that will probably have a much more direct and short-term impact on FIWARE and which we should be able to do in parallel: Focus on the user-perspective. I am talking about greatly simplifying the deployment of enablers for developers (and possibly end users) on FIWARE. Docker is an excellent tool for that as we have experienced for example in FIcontent. Its very easy for owners to provide their enabler and even easier for users to deploy and test them. As with the "Discover, Try, Tweak, Run" approach in FIcontent we can make it tremendously simpler for users to explore what we offer, try it out, share it with others via social networks, and deploy and use the necessary services for real. This is exactly what I believe FIWARE needs. The GEs are still somewhat difficult to access and the documentation for doing so leaves some things to be desired (at least judging from my reviewing job). Taking a much more user-focused approach is important now. So to make things concrete: We should explore (in parallel to your suggestions which is still needed, of course) to install something similar to FIC2-Lab (actually: now officially called "FIWARE Media&Content Lab"). We might be able to leverage the experience we got from there as a basis. But I am sure that together there are many improvements and extensions that should be done. In parallel, we would ask enabler owners to provide suitable dockerified versions that can easily be deployed via this system. This does not make sense for all and can be done incrementally as well. All that is then still needed is to create some nice examples that can be used for the Try and Tweak part, where developers can get first hand experience of enablers, by simply trying out, inspecting, and hacking on existing web-based applications that make use of one or more (bundles!) enablers from any Web browser and without ever having seen the cloud portal (they eventually need to but this is when they are hopefully hooked already). Again, this is not an alternative but a parallel activity that makes containers directly relevant for the developers right now (!) and which relies eventually on the ability to deploy enabler an easier, faster, and with a largely reduced resource footprint via the necessary technology in Cloud/OpenStack. Simply deploying docker within existing virtual machines should allow us to reduce resource usage by quite a bit. But we need to look into that. Best, Philipp Am 31.05.2015 um 22:36 schrieb Alex Glikson: > Below is a brief summary of the current FIWARE Cloud status with regards > to containers and Docker support. > > * FIWARE Cloud GEs currently deployed in FIWARE Lab provide VM-based > IaaS capabilities with OpenStack Nova (with KVM), Glance, Keystone, > Cinder and Neutron (with OVS). FIWARE Cloud also provides object > storage capabilities based on OpenStack Swift. Moreover, we support > PaaS-level capabilities, including the ability to provision and > auto-scale complex applications comprising sets of inter-dependent > VMs as well as the ability to install and configure the individual > software components within VMs using configuration management tools > such as Chef. The key user-visible notion behind this capability is > the notion of a Blueprint, which defines the application topology, > dependencies, auto-scaling rules, software configuration, etc > (conceptually similar to Amazon OpsWorks). > * In the last few months, the PaaS implementation is undergoing a > major transition from a 'proprietary' (although open source) code to > full adoption of OpenStack Heat and Murano (as well as contribution > of features to the Murano community, such as support for templates, > configuration management enhancements, etc). > * In parallel, we have started assessing (and prototyping) the options > to support/leverage Linux containers technologies, and Docker in > particular. We aim at two main use-cases: > 1. As a FIWARE Cloud provider (in FIWARE Lab), I want to continue > providing (almost?) the same PaaS/IaaS capabilities, but at the > same time being able to admit much more users/workloads on the > same hardware, by leveraging Containers technologies. > 2. As a FIWARE Cloud user, I want to be able to leverage the Docker > ecosystem, including public images available at Docker Hub as > well as existing tools (TBD: which tools exactly?), while > hosting my applications on the FIWARE Lab > * The 2nd use-case can be satisfied in several ways, such as: > 1. Treat Docker-based cluster as PaaS application (managed by > FIWARE Cloud PaaS), while the cloud middleware (IaaS and PaaS) > is not aware of individual containers, Docker images, etc. > 2. Make Docker containers 1st class citizens in PaaS -- e.g., as an > additional delivery mechanism in addition to Chef, deployed on > regular IaaS virtualized infra (providing isolation, accounting, > etc). In this case each Docker cluster is contained within a > single tenant (and even single application within a tenant?) > 3. Make Docker containers 1st class citizens in IaaS deployed on > bare-metal, making PaaS use it via a standardized API > * The current (tentative) plan is to apply a 2-phase approach to > introduce containers: phase-1: start by deploying nova-docker in > FIWARE lab in order to enable use-case n.1, and then phase-2: > gradually migrate to OpenStack Magnum, while incorporating the > Docker-native capabilities into Murano in order to support use-case > n.2. Of course, the devil is in the details, so we need to discuss > further, to understand the exact requirements and implications. > > > Regards, > Alex > > > > > From: Alex Glikson/Haifa/IBM at IBMIL > To: fiware-cloud-containers at lists.fiware.org > Date: 30/05/2015 07:06 PM > Subject: [Fiware-cloud-containers] task-force kickoff > Sent by: fiware-cloud-containers-bounces at lists.fiware.org > ------------------------------------------------------------------------ > > > > Dear all, > > Welcome to 'fiware-cloud-containers' mailing list! We will use this > mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. > support for Linux containers (such as Docker), including use-cases & > requirements, associated IaaS and PaaS-style capabilities, concrete > plans going forwards, potential collaboration with teams working on > similar/related technologies, etc. > > Currently we have the following people subscribed to this mailing list: > IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for > IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud > leader), Doron (working with Ezra on containers) > TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads > for the PaaS-related work in FIWARE Cloud) > DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter > leader/architect) > Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old > FI-WARE project) > > Let me know if anyone is missing. > > As a background, please, read the thread below, including the very nice > summary by Mario on the related investigation and implementation in > FI-Content2 project. I will send a separate email briefly summarizing > the situation in FIWARE. > * > All *-- notice that we are going to have a phone call to discuss this > topic during the upcoming FIWARE architects interlock on Monday, June > 1at, 10:30-12:30 CET. Let me know if you can't join.* > Mario *-- would you be able to join & explain/demonstrate the FIC2Lab > solution? > > Thanks, > Alex > > > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- > > fiware-chapter-architects-bounces at lists.fi-ware.org wrote on 26/05/2015 > 08:37:16 AM: > >> From: Alex Glikson/Haifa/IBM at IBMIL >> To: Philipp Slusallek >> Cc: fiware-chapter-architects at lists.fi-ware.org, Kenneth Nagin/ >> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >> Date: 26/05/2015 08:37 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> Sent by: fiware-chapter-architects-bounces at lists.fi-ware.org >> >> Hi Philipp, >> >> It is great to hear that FIC2 team is willing to contribute! >> In a nutshell, our current plan is to start with surfacing >> containers via Nova, and then gradually migrate to Magnum. In terms >> of orchestration, we currently work with Heat and Murano, and are >> assessing the approaches to support containers (OpenStack alignment >> being high priority, but also the ability to surface capabilities >> unique to containers). >> I suggest that we form a small taskforce to explore this topic. >> Please, send me a list of people to involve. >> >> Thanks, >> Alex >> >> > ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> >> >> >> From: Philipp Slusallek >> To: Juanjo Hierro , Alex >> Glikson/Haifa/IBM at IBMIL >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >> architects at lists.fi-ware.org, Kenneth Nagin/Haifa/IBM at IBMIL >> Date: 26/05/2015 07:21 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> >> >> >> Hi, >> >> Here is some first input from FIC2 on Docker compiled from Mario. As I >> mentioned before, we have a full system working that deploys entire sets >> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >> >> FIC2 would be happy to work with FIWARE to integrate them in whatever >> way we at FIWARE decide to implement Docker. Right now we deploy it on >> top of the VMs offered by FIWARE but a more native implementation would >> be fine as well and save much resources (which seems to be a major issue). >> >> A presentation by Canonical recently showed a >10x improvement in >> resource usage with their containers over plain OpenStack (which is >> probably where we are -- but then Intel claims even better numbers in >> their Clear Containers solution using traditional VMs). >> >> Since we have experience already, it seems very useful to make Docker >> available especially since deployment of SW is much easier too and the >> same SW can be deployed essentially anywhere (including a local machine). >> >> >> Best, >> >> Philipp >> >> > There are several ways in which FIWARE could support Docker. My >> assumption is that the plan is to add to OpenStack a Docker driver. >> It would enable the creation of instances which are not VMs but >> OpenStack containers, from images which are not ISOs but Docker >> images. It?s described here: _https://wiki.openstack.org/wiki/Docker_ >> > >> > >> > >> > Advantages: >> > >> > + Launches faster than VM instances >> > >> > + Less resources used from quotas in FIWARE nodes >> > >> > + images of SEs smaller and easier to build (using Dockerfiles) >> > >> > >> > >> > Open questions / disadvantages: >> > >> > - Will it be possible to pull images from DockerHub (preferred >> approach), or do they need to be uploaded to the Glance image >> repository of each FIWARE node? >> > >> > - Is the only way to launch containers the Cloud Portal or >> OpenStack API? Not being able to target a Docker API would reduce >> the value of Docker and its ecosystem of tools, e.g.: docker-compose >> which enables launching applications composed of multiple containers. >> > >> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >> reverse proxy that forwards HTTP requests to container1.john- >> smith.cloud.fiware.org to the right container port? >> > >> > - How would links between containers running on different hosts >> work? Using Weave? >> > >> > - Mounting volumes in Docker is a very common way of passing >> configuration files, having persistence, etc. How would this work > inOpenStack? >> > >> > >> > >> > Our suggestion: >> > >> > * Ability to use the Docker API remotely (i.e. command-line tools >> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >> > >> > * Ability to pull containers directly from DockerHub with a common >> (transparent?) image mirror in each FIWARE node >> > >> > >> > >> > >> > >> > >> > >> > Regarding FIC2Lab runner, our wishlist was: >> > >> > 1. ability to use the same tools locally and remotely in any cloud >> (ideally use docker-machine against any cloud provider, in >> particular FIWARE nodes) >> > >> > 2. nice web user interface hiding the complexity of firewalling, >> security pairs, public IPs, etc. >> > >> > 3. compatibility with the tools of the docker ecosystem: docker >> CLI, compose, swarm ... >> > >> > >> > >> > We compared tools such as tutum (free multi-tenant online >> service), shipyard (open-source web UI) and panamax (open-source web >> UI and orchestrator). >> > >> > And we decided to go for Panamax, which is nice for #1 and #2 but >> not #3 so far. Here's a few pluses and minuses: >> > >> > + nice UI with DockerHub search >> > >> > + composite application, with application template search >> > >> > - It's not multi-tenant, which is good for portability but bad for >> software updates >> > >> > - its model for composite applications is different from docker- >> compose (for now) >> > >> > - the UI can't synchronize with actions performed in Docker CLI or >> API because it's using CoreOS underneath >> > >> > >> > >> > The FIC2Lab runner is documented here: _http://fic2.github.io/runner_ >> > >> > >> >> >> >> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >> > Dear Alex, >> > >> > It's fine with me. >> > >> > I believe that we should collocate here the discussion with >> > FI-Content2 (Philipp) regarding the stuff they have developed for the >> > FIC2-Lab. >> > >> > @Philipp: could you share some material (document describing your work >> > on FIC2-Lab, slides, whatever) describing what you have done and send it >> > to the fiware-chapter-architects mailing list as preparation for the >> > discussion? >> > >> > @Alex: please also share the ideas you want to present on this matter. >> > >> > By exchanging material prior to the meeting, we would be able to have >> > a more fruitful meeting. >> > >> > Best regards, >> > >> > -- Juanjo >> > >> > On 18/05/15 11:07, Alex Glikson wrote: >> >> I am out of office on the 25th. Would the following Monday work? >> >> >> >> Thanks, >> >> Alex >> >> >> >> >> >> >> >> >> >> >> >> >> >> From: Juanjo Hierro >> >> To: Philipp Slusallek , Alex >> >> Glikson/Haifa/IBM at IBMIL, < fiware-chapter-architects at lists.fi-ware.org> >> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >> >> Date: 18/05/2015 11:53 AM >> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & > Docker >> >> > ------------------------------------------------------------------------ >> >> >> >> >> >> >> >> >> >> I would suggest that we book the next architecture session (Monday >> >> 25th) to cover this matter. >> >> >> >> Will you Alex be able to prepare a presentation? >> >> >> >> Cheers, >> >> >> >> -- Juanjo >> >> >> >> On 18/05/15 09:46, Philipp Slusallek wrote: >> >> > Hi, >> >> > >> >> > This sounds very useful to cover as well in this or one of the next >> >> > calls. It would also get the architectural activities started > again. It >> >> > might even make sense to invite (selected?) AB members from the UCs. >> >> > >> >> > For example, a lot of work that has been done within FIcontent to > deploy >> >> > our SEs and related GEs within Docker on top of FIWARE Lab. > Particularly >> >> > relevant could be the on-click deployment of entire arrangements > of GEs >> >> > and SEs. Better and more dedicated Docker support would certainly be >> >> > very welcome as well. >> >> > >> >> > Unfortunately, I have a lecture this morning and cannot join > until 12h. >> >> > But Stefan Lemme from my group will be on the call. He has been a > core >> >> > member of the relevant FIC2-Lab task force in FIcontent. >> >> > >> >> > >> >> > Best, >> >> > >> >> > Philipp >> >> > >> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >> >> >> Maybe we can also dedicate some time to have an initial > discussion on >> >> >> the roadmap to adopt Linux Containers and in particular Docker in >> >> FIWARE >> >> >> Lab. >> >> >> >> >> >> Regards, >> >> >> Alex >> >> >>_______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o -------------- next part -------------- An HTML attachment was scrubbed... URL: From EZRA at il.ibm.com Tue Jun 2 10:06:24 2015 From: EZRA at il.ibm.com (Ezra Silvera) Date: Tue, 2 Jun 2015 11:06:24 +0300 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: References: <556C05BB.5060900@dfki.de> Message-ID: Interesting. "Kubernetes does the same things you expect it to, such as providing Pods that implement the Docker service, monitoring availability and load of the Pods, and scaling Pods up and down based on the Kubernetes configuration. It also coordinates connectivity between the Pods and the underlying infrastructure. Meanwhile, Murano manages and orchestrates that underlying infrastructure, which consists of OpenStack resources. It configures the virtual network for Kubernetes and the Pods, and uses OpenStack Orchestration (Heat) to provision the resources Kubernetes needs, such as virtual machines and interface connections, network and subnet configs, security groups, router configurations, and storage." Seems like many of these tasks are covered by Magnum. So how about Murano + Magnum ? Does it make sense? BTW, we probably should keep an eye on Solum (I'm sure you are :-) ) Although it's currently seems immature, it does look promissing as a future PaaS framework Thanks Ezra Silvera From: HENAR MU?OZ FRUTOS To: Alex Glikson/Haifa/IBM at IBMIL, Philipp Slusallek Cc: "fiware-cloud-containers at lists.fiware.org" Date: 02/06/2015 10:33 AM Subject: Re: [Fiware-cloud-containers] task-force kickoff Sent by: fiware-cloud-containers-bounces at lists.fiware.org Hi Did you hear about https://www.mirantis.com/blog/integrating-openstack-and-kubernetes-with-murano/ ? Regards, Henar De: "GLIKSON at il.ibm.com" Fecha: lunes, 1 de junio de 2015 22:36 Para: Philipp Slusallek CC: "fiware-cloud-containers at lists.fiware.org" < fiware-cloud-containers at lists.fiware.org> Asunto: Re: [Fiware-cloud-containers] task-force kickoff I agree that user/developer perspective is important. However, there are several things that we need to take into account: 1. For users/developers who just want to *use* GEs (rather than develop new ones), FIWARE already provides tools to easily set them up - pre-built VM images, as well as SDC/Chef recipes and PaaS Blueprints (for bundles). Significant effort has been invested by GE owners to populate the corresponding artefacts and enable the automated/seamless deployment in FIWARE Lab. Moreover, as I mentioned, we are currently transforming those tools to better align with OpenStack, and we are also constantly improving them to make more capable/stable. Having said that, there is clearly space for further improvement (usability, documentation, features, integration, etc). In this context, it is not entirely clear whether/how containers (including Docker) can help, besides achieving higher density. 2. We do appreciate the value proposition of Docker as a new emerging standard for application packaging and life cycle management -- especially for 'cloud-born' applications. However, the tooling around Docker is rather diverse and immature, and we should be careful before we 'bet' on one (or more) of the options to become part of FIWARE (which aims at becoming a sustainable standard). For example, I haven't seen yet a good DevOps tool that would manage rolling upgrades (and other 'traditional' configuration management scenarios) for Docker-based applications. 3. It is likely that we would need to support both 'old' and 'new' ways to design and manage applications for the upcoming few years. Moreover, new paradigms may arise, addressing additional 'niches' on the spectrum between traditional servers (running full operating system plus an arbitrary software stack) and application containers (running a single process, possible with it's children processes). In my opinion, one of our challenges is to come up with a unified solution (and a roadmap) which would provide a reasonable value proposition in the short term, but would also like to be forward-compatible with new developments which will surely emerge around containers in the near future. For example, if we think that Chef is the way to go from configuration management perspective, maybe we should use and extend the Chef-Docker integration [1] . Alternatively (or in addition?), if we want to stick with Heat for orchestration, maybe we can invest in adding support for Magnum containers as resources in Heat, as well as LXD driver in Nova (for system containers). IMO, this is the problem space we need to further explore (and we started doing so). Regards, Alex [1] https://www.chef.io/solutions/containers/ ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Philipp Slusallek To: Alex Glikson/Haifa/IBM at IBMIL, fiware-cloud-containers at lists.fiware.org Date: 01/06/2015 10:12 AM Subject: Re: [Fiware-cloud-containers] task-force kickoff Hi all, It is certainly the right thing to work on supporting containers natively in the Cloud chapter. It seems that a lot is currently being done in this regard within OpenStack and we should see how we can best contribute to that. However, I would like to point out another aspect that will probably have a much more direct and short-term impact on FIWARE and which we should be able to do in parallel: Focus on the user-perspective. I am talking about greatly simplifying the deployment of enablers for developers (and possibly end users) on FIWARE. Docker is an excellent tool for that as we have experienced for example in FIcontent. Its very easy for owners to provide their enabler and even easier for users to deploy and test them. As with the "Discover, Try, Tweak, Run" approach in FIcontent we can make it tremendously simpler for users to explore what we offer, try it out, share it with others via social networks, and deploy and use the necessary services for real. This is exactly what I believe FIWARE needs. The GEs are still somewhat difficult to access and the documentation for doing so leaves some things to be desired (at least judging from my reviewing job). Taking a much more user-focused approach is important now. So to make things concrete: We should explore (in parallel to your suggestions which is still needed, of course) to install something similar to FIC2-Lab (actually: now officially called "FIWARE Media&Content Lab"). We might be able to leverage the experience we got from there as a basis. But I am sure that together there are many improvements and extensions that should be done. In parallel, we would ask enabler owners to provide suitable dockerified versions that can easily be deployed via this system. This does not make sense for all and can be done incrementally as well. All that is then still needed is to create some nice examples that can be used for the Try and Tweak part, where developers can get first hand experience of enablers, by simply trying out, inspecting, and hacking on existing web-based applications that make use of one or more (bundles!) enablers from any Web browser and without ever having seen the cloud portal (they eventually need to but this is when they are hopefully hooked already). Again, this is not an alternative but a parallel activity that makes containers directly relevant for the developers right now (!) and which relies eventually on the ability to deploy enabler an easier, faster, and with a largely reduced resource footprint via the necessary technology in Cloud/OpenStack. Simply deploying docker within existing virtual machines should allow us to reduce resource usage by quite a bit. But we need to look into that. Best, Philipp Am 31.05.2015 um 22:36 schrieb Alex Glikson: > Below is a brief summary of the current FIWARE Cloud status with regards > to containers and Docker support. > > * FIWARE Cloud GEs currently deployed in FIWARE Lab provide VM-based > IaaS capabilities with OpenStack Nova (with KVM), Glance, Keystone, > Cinder and Neutron (with OVS). FIWARE Cloud also provides object > storage capabilities based on OpenStack Swift. Moreover, we support > PaaS-level capabilities, including the ability to provision and > auto-scale complex applications comprising sets of inter-dependent > VMs as well as the ability to install and configure the individual > software components within VMs using configuration management tools > such as Chef. The key user-visible notion behind this capability is > the notion of a Blueprint, which defines the application topology, > dependencies, auto-scaling rules, software configuration, etc > (conceptually similar to Amazon OpsWorks). > * In the last few months, the PaaS implementation is undergoing a > major transition from a 'proprietary' (although open source) code to > full adoption of OpenStack Heat and Murano (as well as contribution > of features to the Murano community, such as support for templates, > configuration management enhancements, etc). > * In parallel, we have started assessing (and prototyping) the options > to support/leverage Linux containers technologies, and Docker in > particular. We aim at two main use-cases: > 1. As a FIWARE Cloud provider (in FIWARE Lab), I want to continue > providing (almost?) the same PaaS/IaaS capabilities, but at the > same time being able to admit much more users/workloads on the > same hardware, by leveraging Containers technologies. > 2. As a FIWARE Cloud user, I want to be able to leverage the Docker > ecosystem, including public images available at Docker Hub as > well as existing tools (TBD: which tools exactly?), while > hosting my applications on the FIWARE Lab > * The 2nd use-case can be satisfied in several ways, such as: > 1. Treat Docker-based cluster as PaaS application (managed by > FIWARE Cloud PaaS), while the cloud middleware (IaaS and PaaS) > is not aware of individual containers, Docker images, etc. > 2. Make Docker containers 1st class citizens in PaaS -- e.g., as an > additional delivery mechanism in addition to Chef, deployed on > regular IaaS virtualized infra (providing isolation, accounting, > etc). In this case each Docker cluster is contained within a > single tenant (and even single application within a tenant?) > 3. Make Docker containers 1st class citizens in IaaS deployed on > bare-metal, making PaaS use it via a standardized API > * The current (tentative) plan is to apply a 2-phase approach to > introduce containers: phase-1: start by deploying nova-docker in > FIWARE lab in order to enable use-case n.1, and then phase-2: > gradually migrate to OpenStack Magnum, while incorporating the > Docker-native capabilities into Murano in order to support use-case > n.2. Of course, the devil is in the details, so we need to discuss > further, to understand the exact requirements and implications. > > > Regards, > Alex > > > > > From: Alex Glikson/Haifa/IBM at IBMIL > To: fiware-cloud-containers at lists.fiware.org > Date: 30/05/2015 07:06 PM > Subject: [Fiware-cloud-containers] task-force kickoff > Sent by: fiware-cloud-containers-bounces at lists.fiware.org > ------------------------------------------------------------------------ > > > > Dear all, > > Welcome to 'fiware-cloud-containers' mailing list! We will use this > mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. > support for Linux containers (such as Docker), including use-cases & > requirements, associated IaaS and PaaS-style capabilities, concrete > plans going forwards, potential collaboration with teams working on > similar/related technologies, etc. > > Currently we have the following people subscribed to this mailing list: > IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for > IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud > leader), Doron (working with Ezra on containers) > TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads > for the PaaS-related work in FIWARE Cloud) > DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter > leader/architect) > Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old > FI-WARE project) > > Let me know if anyone is missing. > > As a background, please, read the thread below, including the very nice > summary by Mario on the related investigation and implementation in > FI-Content2 project. I will send a separate email briefly summarizing > the situation in FIWARE. > * > All *-- notice that we are going to have a phone call to discuss this > topic during the upcoming FIWARE architects interlock on Monday, June > 1at, 10:30-12:30 CET. Let me know if you can't join.* > Mario *-- would you be able to join & explain/demonstrate the FIC2Lab > solution? > > Thanks, > Alex > > > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- > > fiware-chapter-architects-bounces at lists.fi-ware.org wrote on 26/05/2015 > 08:37:16 AM: > >> From: Alex Glikson/Haifa/IBM at IBMIL >> To: Philipp Slusallek >> Cc: fiware-chapter-architects at lists.fi-ware.org, Kenneth Nagin/ >> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >> Date: 26/05/2015 08:37 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> Sent by: fiware-chapter-architects-bounces at lists.fi-ware.org >> >> Hi Philipp, >> >> It is great to hear that FIC2 team is willing to contribute! >> In a nutshell, our current plan is to start with surfacing >> containers via Nova, and then gradually migrate to Magnum. In terms >> of orchestration, we currently work with Heat and Murano, and are >> assessing the approaches to support containers (OpenStack alignment >> being high priority, but also the ability to surface capabilities >> unique to containers). >> I suggest that we form a small taskforce to explore this topic. >> Please, send me a list of people to involve. >> >> Thanks, >> Alex >> >> > ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> >> >> >> From: Philipp Slusallek >> To: Juanjo Hierro , Alex >> Glikson/Haifa/IBM at IBMIL >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >> architects at lists.fi-ware.org, Kenneth Nagin/Haifa/IBM at IBMIL >> Date: 26/05/2015 07:21 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> >> >> >> Hi, >> >> Here is some first input from FIC2 on Docker compiled from Mario. As I >> mentioned before, we have a full system working that deploys entire sets >> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >> >> FIC2 would be happy to work with FIWARE to integrate them in whatever >> way we at FIWARE decide to implement Docker. Right now we deploy it on >> top of the VMs offered by FIWARE but a more native implementation would >> be fine as well and save much resources (which seems to be a major issue). >> >> A presentation by Canonical recently showed a >10x improvement in >> resource usage with their containers over plain OpenStack (which is >> probably where we are -- but then Intel claims even better numbers in >> their Clear Containers solution using traditional VMs). >> >> Since we have experience already, it seems very useful to make Docker >> available especially since deployment of SW is much easier too and the >> same SW can be deployed essentially anywhere (including a local machine). >> >> >> Best, >> >> Philipp >> >> > There are several ways in which FIWARE could support Docker. My >> assumption is that the plan is to add to OpenStack a Docker driver. >> It would enable the creation of instances which are not VMs but >> OpenStack containers, from images which are not ISOs but Docker >> images. It?s described here: _https://wiki.openstack.org/wiki/Docker_ >> > >> > >> > >> > Advantages: >> > >> > + Launches faster than VM instances >> > >> > + Less resources used from quotas in FIWARE nodes >> > >> > + images of SEs smaller and easier to build (using Dockerfiles) >> > >> > >> > >> > Open questions / disadvantages: >> > >> > - Will it be possible to pull images from DockerHub (preferred >> approach), or do they need to be uploaded to the Glance image >> repository of each FIWARE node? >> > >> > - Is the only way to launch containers the Cloud Portal or >> OpenStack API? Not being able to target a Docker API would reduce >> the value of Docker and its ecosystem of tools, e.g.: docker-compose >> which enables launching applications composed of multiple containers. >> > >> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >> reverse proxy that forwards HTTP requests to container1.john- >> smith.cloud.fiware.org to the right container port? >> > >> > - How would links between containers running on different hosts >> work? Using Weave? >> > >> > - Mounting volumes in Docker is a very common way of passing >> configuration files, having persistence, etc. How would this work > inOpenStack? >> > >> > >> > >> > Our suggestion: >> > >> > * Ability to use the Docker API remotely (i.e. command-line tools >> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >> > >> > * Ability to pull containers directly from DockerHub with a common >> (transparent?) image mirror in each FIWARE node >> > >> > >> > >> > >> > >> > >> > >> > Regarding FIC2Lab runner, our wishlist was: >> > >> > 1. ability to use the same tools locally and remotely in any cloud >> (ideally use docker-machine against any cloud provider, in >> particular FIWARE nodes) >> > >> > 2. nice web user interface hiding the complexity of firewalling, >> security pairs, public IPs, etc. >> > >> > 3. compatibility with the tools of the docker ecosystem: docker >> CLI, compose, swarm ... >> > >> > >> > >> > We compared tools such as tutum (free multi-tenant online >> service), shipyard (open-source web UI) and panamax (open-source web >> UI and orchestrator). >> > >> > And we decided to go for Panamax, which is nice for #1 and #2 but >> not #3 so far. Here's a few pluses and minuses: >> > >> > + nice UI with DockerHub search >> > >> > + composite application, with application template search >> > >> > - It's not multi-tenant, which is good for portability but bad for >> software updates >> > >> > - its model for composite applications is different from docker- >> compose (for now) >> > >> > - the UI can't synchronize with actions performed in Docker CLI or >> API because it's using CoreOS underneath >> > >> > >> > >> > The FIC2Lab runner is documented here: _http://fic2.github.io/runner_ >> > >> > >> >> >> >> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >> > Dear Alex, >> > >> > It's fine with me. >> > >> > I believe that we should collocate here the discussion with >> > FI-Content2 (Philipp) regarding the stuff they have developed for the >> > FIC2-Lab. >> > >> > @Philipp: could you share some material (document describing your work >> > on FIC2-Lab, slides, whatever) describing what you have done and send it >> > to the fiware-chapter-architects mailing list as preparation for the >> > discussion? >> > >> > @Alex: please also share the ideas you want to present on this matter. >> > >> > By exchanging material prior to the meeting, we would be able to have >> > a more fruitful meeting. >> > >> > Best regards, >> > >> > -- Juanjo >> > >> > On 18/05/15 11:07, Alex Glikson wrote: >> >> I am out of office on the 25th. Would the following Monday work? >> >> >> >> Thanks, >> >> Alex >> >> >> >> >> >> >> >> >> >> >> >> >> >> From: Juanjo Hierro >> >> To: Philipp Slusallek , Alex >> >> Glikson/Haifa/IBM at IBMIL, < fiware-chapter-architects at lists.fi-ware.org> >> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >> >> Date: 18/05/2015 11:53 AM >> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & > Docker >> >> > ------------------------------------------------------------------------ >> >> >> >> >> >> >> >> >> >> I would suggest that we book the next architecture session (Monday >> >> 25th) to cover this matter. >> >> >> >> Will you Alex be able to prepare a presentation? >> >> >> >> Cheers, >> >> >> >> -- Juanjo >> >> >> >> On 18/05/15 09:46, Philipp Slusallek wrote: >> >> > Hi, >> >> > >> >> > This sounds very useful to cover as well in this or one of the next >> >> > calls. It would also get the architectural activities started > again. It >> >> > might even make sense to invite (selected?) AB members from the UCs. >> >> > >> >> > For example, a lot of work that has been done within FIcontent to > deploy >> >> > our SEs and related GEs within Docker on top of FIWARE Lab. > Particularly >> >> > relevant could be the on-click deployment of entire arrangements > of GEs >> >> > and SEs. Better and more dedicated Docker support would certainly be >> >> > very welcome as well. >> >> > >> >> > Unfortunately, I have a lecture this morning and cannot join > until 12h. >> >> > But Stefan Lemme from my group will be on the call. He has been a > core >> >> > member of the relevant FIC2-Lab task force in FIcontent. >> >> > >> >> > >> >> > Best, >> >> > >> >> > Philipp >> >> > >> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >> >> >> Maybe we can also dedicate some time to have an initial > discussion on >> >> >> the roadmap to adopt Linux Containers and in particular Docker in >> >> FIWARE >> >> >> Lab. >> >> >> >> >> >> Regards, >> >> >> Alex >> >> >>_______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers -------------- next part -------------- An HTML attachment was scrubbed... URL: From philipp.slusallek at dfki.de Tue Jun 2 10:11:04 2015 From: philipp.slusallek at dfki.de (Philipp Slusallek) Date: Tue, 02 Jun 2015 10:11:04 +0200 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: References: <556C05BB.5060900@dfki.de> Message-ID: <556D6518.1000701@dfki.de> Hi, Yes, this is waht we use. But FIC2-Lab actually does much more than that. The functionality to deploy docker images is just an underlying functionality to make services available (internally and on behalf of the user in his environment). However, the key feature is the user-facing part that allows to try, tweak, and run services and Web-apps using them, as well as share the results. best, Philipp Am 02.06.2015 um 10:04 schrieb Alex Glikson: > Yes. It addresses use-case 2.1 in my list down the thread, which is also > pretty much the mode FC2Lab is addressing --- deployment of a > Docker-based cluster as an 'application' managed by Heat/Murano. > > Regards, > Alex > > > > > From: HENAR MU?OZ FRUTOS > To: Alex Glikson/Haifa/IBM at IBMIL, Philipp Slusallek > > Cc: "fiware-cloud-containers at lists.fiware.org" > > Date: 02/06/2015 10:33 AM > Subject: Re: [Fiware-cloud-containers] task-force kickoff > ------------------------------------------------------------------------ > > > > Hi > > Did you hear about > _https://www.mirantis.com/blog/integrating-openstack-and-kubernetes-with-murano/_? > > Regards, > Henar > > *De: *"_GLIKSON at il.ibm.com_ " > <_GLIKSON at il.ibm.com_ >* > Fecha: *lunes, 1 de junio de 2015 22:36* > Para: *Philipp Slusallek <_Philipp.Slusallek at dfki.de_ > >* > CC: *"_fiware-cloud-containers at lists.fiware.org_ > " > <_fiware-cloud-containers at lists.fiware.org_ > >* > Asunto: *Re: [Fiware-cloud-containers] task-force kickoff > > I agree that user/developer perspective is important. > However, there are several things that we need to take into account: > 1. For users/developers who just want to *use* GEs (rather than > develop new ones), FIWARE already provides tools to easily set them up - > pre-built VM images, as well as SDC/Chef recipes and PaaS Blueprints > (for bundles). Significant effort has been invested by GE owners to > populate the corresponding artefacts and enable the automated/seamless > deployment in FIWARE Lab. Moreover, as I mentioned, we are currently > transforming those tools to better align with OpenStack, and we are also > constantly improving them to make more capable/stable. Having said that, > there is clearly space for further improvement (usability, > documentation, features, integration, etc). In this context, it is not > entirely clear whether/how containers (including Docker) can help, > besides achieving higher density. > 2. We do appreciate the value proposition of Docker as a new > emerging standard for application packaging and life cycle management -- > especially for 'cloud-born' applications. However, the tooling around > Docker is rather diverse and immature, and we should be careful before > we 'bet' on one (or more) of the options to become part of FIWARE (which > aims at becoming a sustainable standard). For example, I haven't seen > yet a good DevOps tool that would manage rolling upgrades (and other > 'traditional' configuration management scenarios) for Docker-based > applications. > 3. It is likely that we would need to support both 'old' and > 'new' ways to design and manage applications for the upcoming few years. > Moreover, new paradigms may arise, addressing additional 'niches' on the > spectrum between traditional servers (running full operating system plus > an arbitrary software stack) and application containers (running a > single process, possible with it's children processes). In my opinion, > one of our challenges is to come up with a unified solution (and a > roadmap) which would provide a reasonable value proposition in the short > term, but would also like to be forward-compatible with new developments > which will surely emerge around containers in the near future. For > example, if we think that Chef is the way to go from configuration > management perspective, maybe we should use and extend the Chef-Docker > integration [1] . Alternatively (or in addition?), if we want to stick > with Heat for orchestration, maybe we can invest in adding support for > Magnum containers as resources in Heat, as well as LXD driver in Nova > (for system containers). IMO, this is the problem space we need to > further explore (and we started doing so). > > Regards, > Alex > > [1] _https://www.chef.io/solutions/containers/_ > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: _glikson at il.ibm.com_ | Phone: > +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 > > > > > > From: Philipp Slusallek <_philipp.slusallek at dfki.de_ > > > To: Alex Glikson/Haifa/IBM at IBMIL, > _fiware-cloud-containers at lists.fiware.org_ > > Date: 01/06/2015 10:12 AM > Subject: Re: [Fiware-cloud-containers] task-force kickoff > ------------------------------------------------------------------------ > > > > Hi all, > > It is certainly the right thing to work on supporting containers > natively in the Cloud chapter. It seems that a lot is currently being > done in this regard within OpenStack and we should see how we can best > contribute to that. > > However, I would like to point out another aspect that will probably > have a much more direct and short-term impact on FIWARE and which we > should be able to do in parallel: Focus on the user-perspective. I am > talking about greatly simplifying the deployment of enablers for > developers (and possibly end users) on FIWARE. > > Docker is an excellent tool for that as we have experienced for example > in FIcontent. Its very easy for owners to provide their enabler and even > easier for users to deploy and test them. As with the "Discover, Try, > Tweak, Run" approach in FIcontent we can make it tremendously simpler > for users to explore what we offer, try it out, share it with others via > social networks, and deploy and use the necessary services for real. > > This is exactly what I believe FIWARE needs. The GEs are still somewhat > difficult to access and the documentation for doing so leaves some > things to be desired (at least judging from my reviewing job). Taking a > much more user-focused approach is important now. > > So to make things concrete: We should explore (in parallel to your > suggestions which is still needed, of course) to install something > similar to FIC2-Lab (actually: now officially called "FIWARE > Media&Content Lab"). We might be able to leverage the experience we got > from there as a basis. But I am sure that together there are many > improvements and extensions that should be done. > > In parallel, we would ask enabler owners to provide suitable dockerified > versions that can easily be deployed via this system. This does not make > sense for all and can be done incrementally as well. All that is then > still needed is to create some nice examples that can be used for the > Try and Tweak part, where developers can get first hand experience of > enablers, by simply trying out, inspecting, and hacking on existing > web-based applications that make use of one or more (bundles!) enablers > from any Web browser and without ever having seen the cloud portal (they > eventually need to but this is when they are hopefully hooked already). > > Again, this is not an alternative but a parallel activity that makes > containers directly relevant for the developers right now (!) and which > relies eventually on the ability to deploy enabler an easier, faster, > and with a largely reduced resource footprint via the necessary > technology in Cloud/OpenStack. Simply deploying docker within existing > virtual machines should allow us to reduce resource usage by quite a > bit. But we need to look into that. > > > Best, > > Philipp > > Am 31.05.2015 um 22:36 schrieb Alex Glikson: >> Below is a brief summary of the current FIWARE Cloud status with regards >> to containers and Docker support. >> >> * FIWARE Cloud GEs currently deployed in FIWARE Lab provide VM-based >> IaaS capabilities with OpenStack Nova (with KVM), Glance, Keystone, >> Cinder and Neutron (with OVS). FIWARE Cloud also provides object >> storage capabilities based on OpenStack Swift. Moreover, we support >> PaaS-level capabilities, including the ability to provision and >> auto-scale complex applications comprising sets of inter-dependent >> VMs as well as the ability to install and configure the individual >> software components within VMs using configuration management tools >> such as Chef. The key user-visible notion behind this capability is >> the notion of a Blueprint, which defines the application topology, >> dependencies, auto-scaling rules, software configuration, etc >> (conceptually similar to Amazon OpsWorks). >> * In the last few months, the PaaS implementation is undergoing a >> major transition from a 'proprietary' (although open source) code to >> full adoption of OpenStack Heat and Murano (as well as contribution >> of features to the Murano community, such as support for templates, >> configuration management enhancements, etc). >> * In parallel, we have started assessing (and prototyping) the options >> to support/leverage Linux containers technologies, and Docker in >> particular. We aim at two main use-cases: >> 1. As a FIWARE Cloud provider (in FIWARE Lab), I want to continue >> providing (almost?) the same PaaS/IaaS capabilities, but at the >> same time being able to admit much more users/workloads on the >> same hardware, by leveraging Containers technologies. >> 2. As a FIWARE Cloud user, I want to be able to leverage the Docker >> ecosystem, including public images available at Docker Hub as >> well as existing tools (TBD: which tools exactly?), while >> hosting my applications on the FIWARE Lab >> * The 2nd use-case can be satisfied in several ways, such as: >> 1. Treat Docker-based cluster as PaaS application (managed by >> FIWARE Cloud PaaS), while the cloud middleware (IaaS and PaaS) >> is not aware of individual containers, Docker images, etc. >> 2. Make Docker containers 1st class citizens in PaaS -- e.g., as an >> additional delivery mechanism in addition to Chef, deployed on >> regular IaaS virtualized infra (providing isolation, accounting, >> etc). In this case each Docker cluster is contained within a >> single tenant (and even single application within a tenant?) >> 3. Make Docker containers 1st class citizens in IaaS deployed on >> bare-metal, making PaaS use it via a standardized API >> * The current (tentative) plan is to apply a 2-phase approach to >> introduce containers: phase-1: start by deploying nova-docker in >> FIWARE lab in order to enable use-case n.1, and then phase-2: >> gradually migrate to OpenStack Magnum, while incorporating the >> Docker-native capabilities into Murano in order to support use-case >> n.2. Of course, the devil is in the details, so we need to discuss >> further, to understand the exact requirements and implications. >> >> >> Regards, >> Alex >> >> >> >> >> From: Alex Glikson/Haifa/IBM at IBMIL >> To: _fiware-cloud-containers at lists.fiware.org_ > >> Date: 30/05/2015 07:06 PM >> Subject: [Fiware-cloud-containers] task-force kickoff >> Sent by: _fiware-cloud-containers-bounces at lists.fiware.org_ > >> ------------------------------------------------------------------------ >> >> >> >> Dear all, >> >> Welcome to 'fiware-cloud-containers' mailing list! We will use this >> mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. >> support for Linux containers (such as Docker), including use-cases & >> requirements, associated IaaS and PaaS-style capabilities, concrete >> plans going forwards, potential collaboration with teams working on >> similar/related technologies, etc. >> >> Currently we have the following people subscribed to this mailing list: >> IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for >> IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud >> leader), Doron (working with Ezra on containers) >> TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads >> for the PaaS-related work in FIWARE Cloud) >> DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter >> leader/architect) >> Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old >> FI-WARE project) >> >> Let me know if anyone is missing. >> >> As a background, please, read the thread below, including the very nice >> summary by Mario on the related investigation and implementation in >> FI-Content2 project. I will send a separate email briefly summarizing >> the situation in FIWARE. >> * >> All *-- notice that we are going to have a phone call to discuss this >> topic during the upcoming FIWARE architects interlock on Monday, June >> 1at, 10:30-12:30 CET. Let me know if you can't join.* >> Mario *-- would you be able to join & explain/demonstrate the FIC2Lab >> solution? >> >> Thanks, >> Alex >> >> >> >> > ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: _glikson at il.ibm.com_ | Phone: > +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- >> >> _fiware-chapter-architects-bounces at lists.fi-ware.org_ > wrote on > 26/05/2015 >> 08:37:16 AM: >> >>> From: Alex Glikson/Haifa/IBM at IBMIL >>> To: Philipp Slusallek <_philipp.slusallek at dfki.de_ > > >>> Cc: _fiware-chapter-architects at lists.fi-ware.org_ > , Kenneth Nagin/ >>> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >>> Date: 26/05/2015 08:37 AM >>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >>> Sent by: _fiware-chapter-architects-bounces at lists.fi-ware.org_ > >>> >>> Hi Philipp, >>> >>> It is great to hear that FIC2 team is willing to contribute! >>> In a nutshell, our current plan is to start with surfacing >>> containers via Nova, and then gradually migrate to Magnum. In terms >>> of orchestration, we currently work with Heat and Murano, and are >>> assessing the approaches to support containers (OpenStack alignment >>> being high priority, but also the ability to surface capabilities >>> unique to containers). >>> I suggest that we form a small taskforce to explore this topic. >>> Please, send me a list of people to involve. >>> >>> Thanks, >>> Alex >>> >>> >> > ==================================================================================== >>> Alex Glikson >>> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >>> Email: _glikson at il.ibm.com_ | Phone: > +972-4-8281085 | Mobile: >>> +972-54-6466667 | Fax: +972-4-8296112 >>> >>> >>> >>> >>> >>> From: Philipp Slusallek <_philipp.slusallek at dfki.de_ > > >>> To: Juanjo Hierro <_juanjose.hierro at telefonica.com_ > >, Alex >>> Glikson/Haifa/IBM at IBMIL >>> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >>> _architects at lists.fi-ware.org_ , > Kenneth Nagin/Haifa/IBM at IBMIL >>> Date: 26/05/2015 07:21 AM >>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >>> >>> >>> >>> Hi, >>> >>> Here is some first input from FIC2 on Docker compiled from Mario. As I >>> mentioned before, we have a full system working that deploys entire sets >>> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >>> >>> FIC2 would be happy to work with FIWARE to integrate them in whatever >>> way we at FIWARE decide to implement Docker. Right now we deploy it on >>> top of the VMs offered by FIWARE but a more native implementation would >>> be fine as well and save much resources (which seems to be a major > issue). >>> >>> A presentation by Canonical recently showed a >10x improvement in >>> resource usage with their containers over plain OpenStack (which is >>> probably where we are -- but then Intel claims even better numbers in >>> their Clear Containers solution using traditional VMs). >>> >>> Since we have experience already, it seems very useful to make Docker >>> available especially since deployment of SW is much easier too and the >>> same SW can be deployed essentially anywhere (including a local machine). >>> >>> >>> Best, >>> >>> Philipp >>> >>> > There are several ways in which FIWARE could support Docker. My >>> assumption is that the plan is to add to OpenStack a Docker driver. >>> It would enable the creation of instances which are not VMs but >>> OpenStack containers, from images which are not ISOs but Docker >>> images. It?s described here: __https://wiki.openstack.org/wiki/Docker__ >>> > >>> > >>> > >>> > Advantages: >>> > >>> > + Launches faster than VM instances >>> > >>> > + Less resources used from quotas in FIWARE nodes >>> > >>> > + images of SEs smaller and easier to build (using Dockerfiles) >>> > >>> > >>> > >>> > Open questions / disadvantages: >>> > >>> > - Will it be possible to pull images from DockerHub (preferred >>> approach), or do they need to be uploaded to the Glance image >>> repository of each FIWARE node? >>> > >>> > - Is the only way to launch containers the Cloud Portal or >>> OpenStack API? Not being able to target a Docker API would reduce >>> the value of Docker and its ecosystem of tools, e.g.: docker-compose >>> which enables launching applications composed of multiple containers. >>> > >>> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >>> reverse proxy that forwards HTTP requests to container1.john- >>> smith.cloud.fiware.org to the right container port? >>> > >>> > - How would links between containers running on different hosts >>> work? Using Weave? >>> > >>> > - Mounting volumes in Docker is a very common way of passing >>> configuration files, having persistence, etc. How would this work >> inOpenStack? >>> > >>> > >>> > >>> > Our suggestion: >>> > >>> > * Ability to use the Docker API remotely (i.e. command-line tools >>> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >>> > >>> > * Ability to pull containers directly from DockerHub with a common >>> (transparent?) image mirror in each FIWARE node >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > Regarding FIC2Lab runner, our wishlist was: >>> > >>> > 1. ability to use the same tools locally and remotely in any cloud >>> (ideally use docker-machine against any cloud provider, in >>> particular FIWARE nodes) >>> > >>> > 2. nice web user interface hiding the complexity of firewalling, >>> security pairs, public IPs, etc. >>> > >>> > 3. compatibility with the tools of the docker ecosystem: docker >>> CLI, compose, swarm ... >>> > >>> > >>> > >>> > We compared tools such as tutum (free multi-tenant online >>> service), shipyard (open-source web UI) and panamax (open-source web >>> UI and orchestrator). >>> > >>> > And we decided to go for Panamax, which is nice for #1 and #2 but >>> not #3 so far. Here's a few pluses and minuses: >>> > >>> > + nice UI with DockerHub search >>> > >>> > + composite application, with application template search >>> > >>> > - It's not multi-tenant, which is good for portability but bad for >>> software updates >>> > >>> > - its model for composite applications is different from docker- >>> compose (for now) >>> > >>> > - the UI can't synchronize with actions performed in Docker CLI or >>> API because it's using CoreOS underneath >>> > >>> > >>> > >>> > The FIC2Lab runner is documented here: __http://fic2.github.io/runner__ >>> > >>> > >>> >>> >>> >>> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >>> > Dear Alex, >>> > >>> > It's fine with me. >>> > >>> > I believe that we should collocate here the discussion with >>> > FI-Content2 (Philipp) regarding the stuff they have developed for the >>> > FIC2-Lab. >>> > >>> > @Philipp: could you share some material (document describing your > work >>> > on FIC2-Lab, slides, whatever) describing what you have done and > send it >>> > to the fiware-chapter-architects mailing list as preparation for the >>> > discussion? >>> > >>> > @Alex: please also share the ideas you want to present on this > matter. >>> > >>> > By exchanging material prior to the meeting, we would be able to have >>> > a more fruitful meeting. >>> > >>> > Best regards, >>> > >>> > -- Juanjo >>> > >>> > On 18/05/15 11:07, Alex Glikson wrote: >>> >> I am out of office on the 25th. Would the following Monday work? >>> >> >>> >> Thanks, >>> >> Alex >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> From: Juanjo Hierro <_juanjose.hierro at telefonica.com_ > > >>> >> To: Philipp Slusallek <_philipp.slusallek at dfki.de_ > >, Alex >>> >> Glikson/Haifa/IBM at IBMIL, > <_fiware-chapter-architects at lists.fi-ware.org_ > > >>> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >>> >> Date: 18/05/2015 11:53 AM >>> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & >> Docker >>> >> >> ------------------------------------------------------------------------ >>> >> >>> >> >>> >> >>> >> >>> >> I would suggest that we book the next architecture session (Monday >>> >> 25th) to cover this matter. >>> >> >>> >> Will you Alex be able to prepare a presentation? >>> >> >>> >> Cheers, >>> >> >>> >> -- Juanjo >>> >> >>> >> On 18/05/15 09:46, Philipp Slusallek wrote: >>> >> > Hi, >>> >> > >>> >> > This sounds very useful to cover as well in this or one of the next >>> >> > calls. It would also get the architectural activities started >> again. It >>> >> > might even make sense to invite (selected?) AB members from the UCs. >>> >> > >>> >> > For example, a lot of work that has been done within FIcontent to >> deploy >>> >> > our SEs and related GEs within Docker on top of FIWARE Lab. >> Particularly >>> >> > relevant could be the on-click deployment of entire arrangements >> of GEs >>> >> > and SEs. Better and more dedicated Docker support would certainly be >>> >> > very welcome as well. >>> >> > >>> >> > Unfortunately, I have a lecture this morning and cannot join >> until 12h. >>> >> > But Stefan Lemme from my group will be on the call. He has been a >> core >>> >> > member of the relevant FIC2-Lab task force in FIcontent. >>> >> > >>> >> > >>> >> > Best, >>> >> > >>> >> > Philipp >>> >> > >>> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >>> >> >> Maybe we can also dedicate some time to have an initial >> discussion on >>> >> >> the roadmap to adopt Linux Containers and in particular Docker in >>> >> FIWARE >>> >> >> Lab. >>> >> >> >>> >> >> Regards, >>> >> >> Alex >>> >> >>_______________________________________________ >> Fiware-cloud-containers mailing list >> _Fiware-cloud-containers at lists.fiware.org_ > >> _https://lists.fiware.org/listinfo/fiware-cloud-containers_ >> >> >> >> _______________________________________________ >> Fiware-cloud-containers mailing list >> _Fiware-cloud-containers at lists.fiware.org_ > >> _https://lists.fiware.org/listinfo/fiware-cloud-containers_ >> > > -- > > ------------------------------------------------------------------------- > Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH > Trippstadter Strasse 122, D-67663 Kaiserslautern > > Gesch?ftsf?hrung: > Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) > Dr. Walter Olthoff > Vorsitzender des Aufsichtsrats: > Prof. Dr. h.c. Hans A. Aukes > > Sitz der Gesellschaft: Kaiserslautern (HRB 2313) > USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 > --------------------------------------------------------------------------- > [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] > > ------------------------------------------------------------------------ > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener informaci?n privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilizaci?n, > divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud > de la legislaci?n vigente. Si ha recibido este mensaje por error, le > rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda > a su destrucci?n. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution > or copying of this communication is strictly prohibited. If you have > received this transmission in error, do not read it. Please immediately > reply to the sender that you have received this communication in error > and then delete it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu > destinat?rio, pode conter informa??o privilegiada ou confidencial e ? > para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa > senhoria o destinat?rio indicado, fica notificado de que a leitura, > utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em > virtude da legisla??o vigente. Se recebeu esta mensagem por erro, > rogamos-lhe que nos o comunique imediatamente por esta mesma via e > proceda a sua destrui??o > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: philipp_slusallek.vcf Type: text/x-vcard Size: 441 bytes Desc: not available URL: From mario.lopezramos at thalesgroup.com Tue Jun 2 11:03:28 2015 From: mario.lopezramos at thalesgroup.com (LOPEZ RAMOS Mario) Date: Tue, 2 Jun 2015 11:03:28 +0200 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: <556D6518.1000701@dfki.de> References: <556C05BB.5060900@dfki.de> <556D6518.1000701@dfki.de> Message-ID: Hi all, Just to be clear with respect to Philipp's statement about FIC2-Lab: It has a *similar* approach to the link sent by Henar and Alex's item 2.1 in that it automates the provisioning of a Docker host + network + keypair public IP on behalf of an OpenStack tenant. This is something that could have been easier with Heat if it was available. However, currently it deploys a single Docker host and therefore no scheduler (we have envisioned multiple Docker hosts with Swarm, but resources are scarce for Community accounts and 1 VM is often enough). And containers are obviously not integrated as OpenStack resources. Best regards, Mario -----Message d'origine----- De : fiware-cloud-containers-bounces at lists.fiware.org [mailto:fiware-cloud-containers-bounces at lists.fiware.org] De la part de Philipp Slusallek Envoy? : mardi 2 juin 2015 10:11 ? : Alex Glikson; fiware-cloud-containers at lists.fiware.org Objet : Re: [Fiware-cloud-containers] task-force kickoff Hi, Yes, this is waht we use. But FIC2-Lab actually does much more than that. The functionality to deploy docker images is just an underlying functionality to make services available (internally and on behalf of the user in his environment). However, the key feature is the user-facing part that allows to try, tweak, and run services and Web-apps using them, as well as share the results. best, Philipp Am 02.06.2015 um 10:04 schrieb Alex Glikson: > Yes. It addresses use-case 2.1 in my list down the thread, which is > also pretty much the mode FC2Lab is addressing --- deployment of a > Docker-based cluster as an 'application' managed by Heat/Murano. > > Regards, > Alex > > > > > From: HENAR MU?OZ FRUTOS > To: Alex Glikson/Haifa/IBM at IBMIL, Philipp Slusallek > > Cc: "fiware-cloud-containers at lists.fiware.org" > > Date: 02/06/2015 10:33 AM > Subject: Re: [Fiware-cloud-containers] task-force kickoff > ---------------------------------------------------------------------- > -- > > > > Hi > > Did you hear about > _https://www.mirantis.com/blog/integrating-openstack-and-kubernetes-with-murano/_? > > Regards, > Henar > > *De: *"_GLIKSON at il.ibm.com_ " > <_GLIKSON at il.ibm.com_ >* > Fecha: *lunes, 1 de junio de 2015 22:36* > Para: *Philipp Slusallek <_Philipp.Slusallek at dfki.de_ > >* > CC: *"_fiware-cloud-containers at lists.fiware.org_ > " > <_fiware-cloud-containers at lists.fiware.org_ > >* > Asunto: *Re: [Fiware-cloud-containers] task-force kickoff > > I agree that user/developer perspective is important. > However, there are several things that we need to take into account: > 1. For users/developers who just want to *use* GEs (rather than > develop new ones), FIWARE already provides tools to easily set them up > - pre-built VM images, as well as SDC/Chef recipes and PaaS Blueprints > (for bundles). Significant effort has been invested by GE owners to > populate the corresponding artefacts and enable the automated/seamless > deployment in FIWARE Lab. Moreover, as I mentioned, we are currently > transforming those tools to better align with OpenStack, and we are > also constantly improving them to make more capable/stable. Having > said that, there is clearly space for further improvement (usability, > documentation, features, integration, etc). In this context, it is not > entirely clear whether/how containers (including Docker) can help, > besides achieving higher density. > 2. We do appreciate the value proposition of Docker as a new > emerging standard for application packaging and life cycle management > -- especially for 'cloud-born' applications. However, the tooling > around Docker is rather diverse and immature, and we should be careful > before we 'bet' on one (or more) of the options to become part of > FIWARE (which aims at becoming a sustainable standard). For example, I > haven't seen yet a good DevOps tool that would manage rolling upgrades > (and other 'traditional' configuration management scenarios) for > Docker-based applications. > 3. It is likely that we would need to support both 'old' and > 'new' ways to design and manage applications for the upcoming few years. > Moreover, new paradigms may arise, addressing additional 'niches' on > the spectrum between traditional servers (running full operating > system plus an arbitrary software stack) and application containers > (running a single process, possible with it's children processes). In > my opinion, one of our challenges is to come up with a unified > solution (and a > roadmap) which would provide a reasonable value proposition in the > short term, but would also like to be forward-compatible with new > developments which will surely emerge around containers in the near > future. For example, if we think that Chef is the way to go from > configuration management perspective, maybe we should use and extend > the Chef-Docker integration [1] . Alternatively (or in addition?), if > we want to stick with Heat for orchestration, maybe we can invest in > adding support for Magnum containers as resources in Heat, as well as > LXD driver in Nova (for system containers). IMO, this is the problem > space we need to further explore (and we started doing so). > > Regards, > Alex > > [1] _https://www.chef.io/solutions/containers/_ > > ====================================================================== > ============== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: _glikson at il.ibm.com_ | Phone: > +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 > > > > > > From: Philipp Slusallek <_philipp.slusallek at dfki.de_ > > > To: Alex Glikson/Haifa/IBM at IBMIL, > _fiware-cloud-containers at lists.fiware.org_ > > Date: 01/06/2015 10:12 AM > Subject: Re: [Fiware-cloud-containers] task-force kickoff > ---------------------------------------------------------------------- > -- > > > > Hi all, > > It is certainly the right thing to work on supporting containers > natively in the Cloud chapter. It seems that a lot is currently being > done in this regard within OpenStack and we should see how we can best > contribute to that. > > However, I would like to point out another aspect that will probably > have a much more direct and short-term impact on FIWARE and which we > should be able to do in parallel: Focus on the user-perspective. I am > talking about greatly simplifying the deployment of enablers for > developers (and possibly end users) on FIWARE. > > Docker is an excellent tool for that as we have experienced for > example in FIcontent. Its very easy for owners to provide their > enabler and even easier for users to deploy and test them. As with the > "Discover, Try, Tweak, Run" approach in FIcontent we can make it > tremendously simpler for users to explore what we offer, try it out, > share it with others via social networks, and deploy and use the necessary services for real. > > This is exactly what I believe FIWARE needs. The GEs are still > somewhat difficult to access and the documentation for doing so leaves > some things to be desired (at least judging from my reviewing job). > Taking a much more user-focused approach is important now. > > So to make things concrete: We should explore (in parallel to your > suggestions which is still needed, of course) to install something > similar to FIC2-Lab (actually: now officially called "FIWARE > Media&Content Lab"). We might be able to leverage the experience we > got from there as a basis. But I am sure that together there are many > improvements and extensions that should be done. > > In parallel, we would ask enabler owners to provide suitable > dockerified versions that can easily be deployed via this system. This > does not make sense for all and can be done incrementally as well. All > that is then still needed is to create some nice examples that can be > used for the Try and Tweak part, where developers can get first hand > experience of enablers, by simply trying out, inspecting, and hacking > on existing web-based applications that make use of one or more > (bundles!) enablers from any Web browser and without ever having seen > the cloud portal (they eventually need to but this is when they are hopefully hooked already). > > Again, this is not an alternative but a parallel activity that makes > containers directly relevant for the developers right now (!) and > which relies eventually on the ability to deploy enabler an easier, > faster, and with a largely reduced resource footprint via the > necessary technology in Cloud/OpenStack. Simply deploying docker > within existing virtual machines should allow us to reduce resource > usage by quite a bit. But we need to look into that. > > > Best, > > Philipp > > Am 31.05.2015 um 22:36 schrieb Alex Glikson: >> Below is a brief summary of the current FIWARE Cloud status with >> regards to containers and Docker support. >> >> * FIWARE Cloud GEs currently deployed in FIWARE Lab provide VM-based >> IaaS capabilities with OpenStack Nova (with KVM), Glance, Keystone, >> Cinder and Neutron (with OVS). FIWARE Cloud also provides object >> storage capabilities based on OpenStack Swift. Moreover, we support >> PaaS-level capabilities, including the ability to provision and >> auto-scale complex applications comprising sets of inter-dependent >> VMs as well as the ability to install and configure the individual >> software components within VMs using configuration management tools >> such as Chef. The key user-visible notion behind this capability is >> the notion of a Blueprint, which defines the application topology, >> dependencies, auto-scaling rules, software configuration, etc >> (conceptually similar to Amazon OpsWorks). >> * In the last few months, the PaaS implementation is undergoing a >> major transition from a 'proprietary' (although open source) code to >> full adoption of OpenStack Heat and Murano (as well as contribution >> of features to the Murano community, such as support for templates, >> configuration management enhancements, etc). >> * In parallel, we have started assessing (and prototyping) the options >> to support/leverage Linux containers technologies, and Docker in >> particular. We aim at two main use-cases: >> 1. As a FIWARE Cloud provider (in FIWARE Lab), I want to continue >> providing (almost?) the same PaaS/IaaS capabilities, but at the >> same time being able to admit much more users/workloads on the >> same hardware, by leveraging Containers technologies. >> 2. As a FIWARE Cloud user, I want to be able to leverage the Docker >> ecosystem, including public images available at Docker Hub as >> well as existing tools (TBD: which tools exactly?), while >> hosting my applications on the FIWARE Lab >> * The 2nd use-case can be satisfied in several ways, such as: >> 1. Treat Docker-based cluster as PaaS application (managed by >> FIWARE Cloud PaaS), while the cloud middleware (IaaS and PaaS) >> is not aware of individual containers, Docker images, etc. >> 2. Make Docker containers 1st class citizens in PaaS -- e.g., as an >> additional delivery mechanism in addition to Chef, deployed on >> regular IaaS virtualized infra (providing isolation, accounting, >> etc). In this case each Docker cluster is contained within a >> single tenant (and even single application within a tenant?) >> 3. Make Docker containers 1st class citizens in IaaS deployed on >> bare-metal, making PaaS use it via a standardized API >> * The current (tentative) plan is to apply a 2-phase approach to >> introduce containers: phase-1: start by deploying nova-docker in >> FIWARE lab in order to enable use-case n.1, and then phase-2: >> gradually migrate to OpenStack Magnum, while incorporating the >> Docker-native capabilities into Murano in order to support use-case >> n.2. Of course, the devil is in the details, so we need to discuss >> further, to understand the exact requirements and implications. >> >> >> Regards, >> Alex >> >> >> >> >> From: Alex Glikson/Haifa/IBM at IBMIL >> To: _fiware-cloud-containers at lists.fiware.org_ > >> Date: 30/05/2015 07:06 PM >> Subject: [Fiware-cloud-containers] task-force kickoff >> Sent by: _fiware-cloud-containers-bounces at lists.fiware.org_ > >> --------------------------------------------------------------------- >> --- >> >> >> >> Dear all, >> >> Welcome to 'fiware-cloud-containers' mailing list! We will use this >> mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. >> support for Linux containers (such as Docker), including use-cases & >> requirements, associated IaaS and PaaS-style capabilities, concrete >> plans going forwards, potential collaboration with teams working on >> similar/related technologies, etc. >> >> Currently we have the following people subscribed to this mailing list: >> IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for >> IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud >> leader), Doron (working with Ezra on containers) >> TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical >> leads for the PaaS-related work in FIWARE Cloud) >> DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter >> leader/architect) >> Thales: Mario (FI-Content2, also used to be in the Cloud WP in the >> old FI-WARE project) >> >> Let me know if anyone is missing. >> >> As a background, please, read the thread below, including the very >> nice summary by Mario on the related investigation and implementation >> in >> FI-Content2 project. I will send a separate email briefly summarizing >> the situation in FIWARE. >> * >> All *-- notice that we are going to have a phone call to discuss this >> topic during the upcoming FIWARE architects interlock on Monday, June >> 1at, 10:30-12:30 CET. Let me know if you can't join.* Mario *-- would >> you be able to join & explain/demonstrate the FIC2Lab solution? >> >> Thanks, >> Alex >> >> >> >> > ====================================================================== > ============== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: _glikson at il.ibm.com_ | Phone: > +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM >> ----- >> >> _fiware-chapter-architects-bounces at lists.fi-ware.org_ > wrote on > 26/05/2015 >> 08:37:16 AM: >> >>> From: Alex Glikson/Haifa/IBM at IBMIL >>> To: Philipp Slusallek <_philipp.slusallek at dfki.de_ > > >>> Cc: _fiware-chapter-architects at lists.fi-ware.org_ > , Kenneth Nagin/ >>> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >>> Date: 26/05/2015 08:37 AM >>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >>> Sent by: _fiware-chapter-architects-bounces at lists.fi-ware.org_ > >>> >>> Hi Philipp, >>> >>> It is great to hear that FIC2 team is willing to contribute! >>> In a nutshell, our current plan is to start with surfacing >>> containers via Nova, and then gradually migrate to Magnum. In terms >>> of orchestration, we currently work with Heat and Murano, and are >>> assessing the approaches to support containers (OpenStack alignment >>> being high priority, but also the ability to surface capabilities >>> unique to containers). >>> I suggest that we form a small taskforce to explore this topic. >>> Please, send me a list of people to involve. >>> >>> Thanks, >>> Alex >>> >>> >> > ====================================================================== > ============== >>> Alex Glikson >>> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >>> Email: _glikson at il.ibm.com_ | Phone: > +972-4-8281085 | Mobile: >>> +972-54-6466667 | Fax: +972-4-8296112 >>> >>> >>> >>> >>> >>> From: Philipp Slusallek <_philipp.slusallek at dfki.de_ > > >>> To: Juanjo Hierro <_juanjose.hierro at telefonica.com_ > >, Alex >>> Glikson/Haifa/IBM at IBMIL >>> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >>> _architects at lists.fi-ware.org_ >>> , > Kenneth Nagin/Haifa/IBM at IBMIL >>> Date: 26/05/2015 07:21 AM >>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >>> >>> >>> >>> Hi, >>> >>> Here is some first input from FIC2 on Docker compiled from Mario. As >>> I mentioned before, we have a full system working that deploys >>> entire sets of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >>> >>> FIC2 would be happy to work with FIWARE to integrate them in >>> whatever way we at FIWARE decide to implement Docker. Right now we >>> deploy it on top of the VMs offered by FIWARE but a more native >>> implementation would be fine as well and save much resources (which >>> seems to be a major > issue). >>> >>> A presentation by Canonical recently showed a >10x improvement in >>> resource usage with their containers over plain OpenStack (which is >>> probably where we are -- but then Intel claims even better numbers >>> in their Clear Containers solution using traditional VMs). >>> >>> Since we have experience already, it seems very useful to make >>> Docker available especially since deployment of SW is much easier >>> too and the same SW can be deployed essentially anywhere (including a local machine). >>> >>> >>> Best, >>> >>> Philipp >>> >>> > There are several ways in which FIWARE could support Docker. My >>> assumption is that the plan is to add to OpenStack a Docker driver. >>> It would enable the creation of instances which are not VMs but >>> OpenStack containers, from images which are not ISOs but Docker >>> images. It's described here: >>> __https://wiki.openstack.org/wiki/Docker__ >>> > >>> > >>> > >>> > Advantages: >>> > >>> > + Launches faster than VM instances >>> > >>> > + Less resources used from quotas in FIWARE nodes >>> > >>> > + images of SEs smaller and easier to build (using Dockerfiles) >>> > >>> > >>> > >>> > Open questions / disadvantages: >>> > >>> > - Will it be possible to pull images from DockerHub (preferred >>> approach), or do they need to be uploaded to the Glance image >>> repository of each FIWARE node? >>> > >>> > - Is the only way to launch containers the Cloud Portal or >>> OpenStack API? Not being able to target a Docker API would reduce >>> the value of Docker and its ecosystem of tools, e.g.: docker-compose >>> which enables launching applications composed of multiple containers. >>> > >>> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >>> reverse proxy that forwards HTTP requests to container1.john- >>> smith.cloud.fiware.org to the right container port? >>> > >>> > - How would links between containers running on different hosts >>> work? Using Weave? >>> > >>> > - Mounting volumes in Docker is a very common way of passing >>> configuration files, having persistence, etc. How would this work >> inOpenStack? >>> > >>> > >>> > >>> > Our suggestion: >>> > >>> > * Ability to use the Docker API remotely (i.e. command-line tools >>> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >>> > >>> > * Ability to pull containers directly from DockerHub with a common >>> (transparent?) image mirror in each FIWARE node >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > Regarding FIC2Lab runner, our wishlist was: >>> > >>> > 1. ability to use the same tools locally and remotely in any cloud >>> (ideally use docker-machine against any cloud provider, in >>> particular FIWARE nodes) >>> > >>> > 2. nice web user interface hiding the complexity of firewalling, >>> security pairs, public IPs, etc. >>> > >>> > 3. compatibility with the tools of the docker ecosystem: docker >>> CLI, compose, swarm ... >>> > >>> > >>> > >>> > We compared tools such as tutum (free multi-tenant online >>> service), shipyard (open-source web UI) and panamax (open-source web >>> UI and orchestrator). >>> > >>> > And we decided to go for Panamax, which is nice for #1 and #2 but >>> not #3 so far. Here's a few pluses and minuses: >>> > >>> > + nice UI with DockerHub search >>> > >>> > + composite application, with application template search >>> > >>> > - It's not multi-tenant, which is good for portability but bad for >>> software updates >>> > >>> > - its model for composite applications is different from docker- >>> compose (for now) >>> > >>> > - the UI can't synchronize with actions performed in Docker CLI or >>> API because it's using CoreOS underneath >>> > >>> > >>> > >>> > The FIC2Lab runner is documented here: >>> > __http://fic2.github.io/runner__ >>> > >>> > >>> >>> >>> >>> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >>> > Dear Alex, >>> > >>> > It's fine with me. >>> > >>> > I believe that we should collocate here the discussion with >>> > FI-Content2 (Philipp) regarding the stuff they have developed for >>> > the FIC2-Lab. >>> > >>> > @Philipp: could you share some material (document describing >>> > your > work >>> > on FIC2-Lab, slides, whatever) describing what you have done and > send it >>> > to the fiware-chapter-architects mailing list as preparation for >>> > the discussion? >>> > >>> > @Alex: please also share the ideas you want to present on this > matter. >>> > >>> > By exchanging material prior to the meeting, we would be able to >>> > have a more fruitful meeting. >>> > >>> > Best regards, >>> > >>> > -- Juanjo >>> > >>> > On 18/05/15 11:07, Alex Glikson wrote: >>> >> I am out of office on the 25th. Would the following Monday work? >>> >> >>> >> Thanks, >>> >> Alex >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> From: Juanjo Hierro <_juanjose.hierro at telefonica.com_ > > >>> >> To: Philipp Slusallek <_philipp.slusallek at dfki.de_ > >, Alex >>> >> Glikson/Haifa/IBM at IBMIL, > <_fiware-chapter-architects at lists.fi-ware.org_ > > >>> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >>> >> Date: 18/05/2015 11:53 AM >>> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & >> Docker >>> >> >> --------------------------------------------------------------------- >> --- >>> >> >>> >> >>> >> >>> >> >>> >> I would suggest that we book the next architecture session >>> >> (Monday >>> >> 25th) to cover this matter. >>> >> >>> >> Will you Alex be able to prepare a presentation? >>> >> >>> >> Cheers, >>> >> >>> >> -- Juanjo >>> >> >>> >> On 18/05/15 09:46, Philipp Slusallek wrote: >>> >> > Hi, >>> >> > >>> >> > This sounds very useful to cover as well in this or one of the >>> >> > next calls. It would also get the architectural activities >>> >> > started >> again. It >>> >> > might even make sense to invite (selected?) AB members from the UCs. >>> >> > >>> >> > For example, a lot of work that has been done within FIcontent >>> >> > to >> deploy >>> >> > our SEs and related GEs within Docker on top of FIWARE Lab. >> Particularly >>> >> > relevant could be the on-click deployment of entire >>> >> > arrangements >> of GEs >>> >> > and SEs. Better and more dedicated Docker support would >>> >> > certainly be very welcome as well. >>> >> > >>> >> > Unfortunately, I have a lecture this morning and cannot join >> until 12h. >>> >> > But Stefan Lemme from my group will be on the call. He has been >>> >> > a >> core >>> >> > member of the relevant FIC2-Lab task force in FIcontent. >>> >> > >>> >> > >>> >> > Best, >>> >> > >>> >> > Philipp >>> >> > >>> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >>> >> >> Maybe we can also dedicate some time to have an initial >> discussion on >>> >> >> the roadmap to adopt Linux Containers and in particular Docker >>> >> >> in >>> >> FIWARE >>> >> >> Lab. >>> >> >> >>> >> >> Regards, >>> >> >> Alex >>> >> >>_______________________________________________ >> Fiware-cloud-containers mailing list >> _Fiware-cloud-containers at lists.fiware.org_ > >> _https://lists.fiware.org/listinfo/fiware-cloud-containers_ >> >> >> >> _______________________________________________ >> Fiware-cloud-containers mailing list >> _Fiware-cloud-containers at lists.fiware.org_ > >> _https://lists.fiware.org/listinfo/fiware-cloud-containers_ >> > > -- > > ---------------------------------------------------------------------- > --- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH > Trippstadter Strasse 122, D-67663 Kaiserslautern > > Gesch?ftsf?hrung: > Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter > Olthoff Vorsitzender des Aufsichtsrats: > Prof. Dr. h.c. Hans A. Aukes > > Sitz der Gesellschaft: Kaiserslautern (HRB 2313) > USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 > ---------------------------------------------------------------------- > ----- [attachment "philipp_slusallek.vcf" deleted by Alex > Glikson/Haifa/IBM] > > ---------------------------------------------------------------------- > -- > > Este mensaje y sus adjuntos se dirigen exclusivamente a su > destinatario, puede contener informaci?n privilegiada o confidencial y > es para uso exclusivo de la persona o entidad de destino. Si no es > usted. el destinatario indicado, queda notificado de que la lectura, > utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar > prohibida en virtud de la legislaci?n vigente. Si ha recibido este > mensaje por error, le rogamos que nos lo comunique inmediatamente por > esta misma v?a y proceda a su destrucci?n. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual > or entity named above. If the reader of this message is not the > intended recipient, you are hereby notified that any dissemination, > distribution or copying of this communication is strictly prohibited. > If you have received this transmission in error, do not read it. > Please immediately reply to the sender that you have received this > communication in error and then delete it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu > destinat?rio, pode conter informa??o privilegiada ou confidencial e ? > para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa > senhoria o destinat?rio indicado, fica notificado de que a leitura, > utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida > em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, > rogamos-lhe que nos o comunique imediatamente por esta mesma via e > proceda a sua destrui??o > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- From philipp.slusallek at dfki.de Tue Jun 2 11:07:01 2015 From: philipp.slusallek at dfki.de (Philipp Slusallek) Date: Tue, 02 Jun 2015 11:07:01 +0200 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: References: <556C05BB.5060900@dfki.de> <556D6518.1000701@dfki.de> Message-ID: <556D7235.9040306@dfki.de> Hi, Yes, I was only talking about the principle functionality, not the actual implementation. Thanks for clarifying! Best, Philipp Am 02.06.2015 um 11:03 schrieb LOPEZ RAMOS Mario: > Hi all, > > Just to be clear with respect to Philipp's statement about FIC2-Lab: > > It has a *similar* approach to the link sent by Henar and Alex's item 2.1 in that it automates the provisioning of a Docker host + network + keypair public IP on behalf of an OpenStack tenant. This is something that could have been easier with Heat if it was available. > > However, currently it deploys a single Docker host and therefore no scheduler (we have envisioned multiple Docker hosts with Swarm, but resources are scarce for Community accounts and 1 VM is often enough). And containers are obviously not integrated as OpenStack resources. > > > Best regards, > > Mario > > > -----Message d'origine----- > De : fiware-cloud-containers-bounces at lists.fiware.org [mailto:fiware-cloud-containers-bounces at lists.fiware.org] De la part de Philipp Slusallek > Envoy? : mardi 2 juin 2015 10:11 > ? : Alex Glikson; fiware-cloud-containers at lists.fiware.org > Objet : Re: [Fiware-cloud-containers] task-force kickoff > > Hi, > > Yes, this is waht we use. But FIC2-Lab actually does much more than that. The functionality to deploy docker images is just an underlying functionality to make services available (internally and on behalf of the user in his environment). > > However, the key feature is the user-facing part that allows to try, tweak, and run services and Web-apps using them, as well as share the results. > > > best, > > Philipp > > Am 02.06.2015 um 10:04 schrieb Alex Glikson: >> Yes. It addresses use-case 2.1 in my list down the thread, which is >> also pretty much the mode FC2Lab is addressing --- deployment of a >> Docker-based cluster as an 'application' managed by Heat/Murano. >> >> Regards, >> Alex >> >> >> >> >> From: HENAR MU?OZ FRUTOS >> To: Alex Glikson/Haifa/IBM at IBMIL, Philipp Slusallek >> >> Cc: "fiware-cloud-containers at lists.fiware.org" >> >> Date: 02/06/2015 10:33 AM >> Subject: Re: [Fiware-cloud-containers] task-force kickoff >> ---------------------------------------------------------------------- >> -- >> >> >> >> Hi >> >> Did you hear about >> _https://www.mirantis.com/blog/integrating-openstack-and-kubernetes-with-murano/_? >> >> Regards, >> Henar >> >> *De: *"_GLIKSON at il.ibm.com_ " >> <_GLIKSON at il.ibm.com_ >* >> Fecha: *lunes, 1 de junio de 2015 22:36* >> Para: *Philipp Slusallek <_Philipp.Slusallek at dfki.de_ >> >* >> CC: *"_fiware-cloud-containers at lists.fiware.org_ >> " >> <_fiware-cloud-containers at lists.fiware.org_ >> >* >> Asunto: *Re: [Fiware-cloud-containers] task-force kickoff >> >> I agree that user/developer perspective is important. >> However, there are several things that we need to take into account: >> 1. For users/developers who just want to *use* GEs (rather than >> develop new ones), FIWARE already provides tools to easily set them up >> - pre-built VM images, as well as SDC/Chef recipes and PaaS Blueprints >> (for bundles). Significant effort has been invested by GE owners to >> populate the corresponding artefacts and enable the automated/seamless >> deployment in FIWARE Lab. Moreover, as I mentioned, we are currently >> transforming those tools to better align with OpenStack, and we are >> also constantly improving them to make more capable/stable. Having >> said that, there is clearly space for further improvement (usability, >> documentation, features, integration, etc). In this context, it is not >> entirely clear whether/how containers (including Docker) can help, >> besides achieving higher density. >> 2. We do appreciate the value proposition of Docker as a new >> emerging standard for application packaging and life cycle management >> -- especially for 'cloud-born' applications. However, the tooling >> around Docker is rather diverse and immature, and we should be careful >> before we 'bet' on one (or more) of the options to become part of >> FIWARE (which aims at becoming a sustainable standard). For example, I >> haven't seen yet a good DevOps tool that would manage rolling upgrades >> (and other 'traditional' configuration management scenarios) for >> Docker-based applications. >> 3. It is likely that we would need to support both 'old' and >> 'new' ways to design and manage applications for the upcoming few years. >> Moreover, new paradigms may arise, addressing additional 'niches' on >> the spectrum between traditional servers (running full operating >> system plus an arbitrary software stack) and application containers >> (running a single process, possible with it's children processes). In >> my opinion, one of our challenges is to come up with a unified >> solution (and a >> roadmap) which would provide a reasonable value proposition in the >> short term, but would also like to be forward-compatible with new >> developments which will surely emerge around containers in the near >> future. For example, if we think that Chef is the way to go from >> configuration management perspective, maybe we should use and extend >> the Chef-Docker integration [1] . Alternatively (or in addition?), if >> we want to stick with Heat for orchestration, maybe we can invest in >> adding support for Magnum containers as resources in Heat, as well as >> LXD driver in Nova (for system containers). IMO, this is the problem >> space we need to further explore (and we started doing so). >> >> Regards, >> Alex >> >> [1] _https://www.chef.io/solutions/containers/_ >> >> ====================================================================== >> ============== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: _glikson at il.ibm.com_ | Phone: >> +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 >> >> >> >> >> >> From: Philipp Slusallek <_philipp.slusallek at dfki.de_ >> > >> To: Alex Glikson/Haifa/IBM at IBMIL, >> _fiware-cloud-containers at lists.fiware.org_ >> >> Date: 01/06/2015 10:12 AM >> Subject: Re: [Fiware-cloud-containers] task-force kickoff >> ---------------------------------------------------------------------- >> -- >> >> >> >> Hi all, >> >> It is certainly the right thing to work on supporting containers >> natively in the Cloud chapter. It seems that a lot is currently being >> done in this regard within OpenStack and we should see how we can best >> contribute to that. >> >> However, I would like to point out another aspect that will probably >> have a much more direct and short-term impact on FIWARE and which we >> should be able to do in parallel: Focus on the user-perspective. I am >> talking about greatly simplifying the deployment of enablers for >> developers (and possibly end users) on FIWARE. >> >> Docker is an excellent tool for that as we have experienced for >> example in FIcontent. Its very easy for owners to provide their >> enabler and even easier for users to deploy and test them. As with the >> "Discover, Try, Tweak, Run" approach in FIcontent we can make it >> tremendously simpler for users to explore what we offer, try it out, >> share it with others via social networks, and deploy and use the necessary services for real. >> >> This is exactly what I believe FIWARE needs. The GEs are still >> somewhat difficult to access and the documentation for doing so leaves >> some things to be desired (at least judging from my reviewing job). >> Taking a much more user-focused approach is important now. >> >> So to make things concrete: We should explore (in parallel to your >> suggestions which is still needed, of course) to install something >> similar to FIC2-Lab (actually: now officially called "FIWARE >> Media&Content Lab"). We might be able to leverage the experience we >> got from there as a basis. But I am sure that together there are many >> improvements and extensions that should be done. >> >> In parallel, we would ask enabler owners to provide suitable >> dockerified versions that can easily be deployed via this system. This >> does not make sense for all and can be done incrementally as well. All >> that is then still needed is to create some nice examples that can be >> used for the Try and Tweak part, where developers can get first hand >> experience of enablers, by simply trying out, inspecting, and hacking >> on existing web-based applications that make use of one or more >> (bundles!) enablers from any Web browser and without ever having seen >> the cloud portal (they eventually need to but this is when they are hopefully hooked already). >> >> Again, this is not an alternative but a parallel activity that makes >> containers directly relevant for the developers right now (!) and >> which relies eventually on the ability to deploy enabler an easier, >> faster, and with a largely reduced resource footprint via the >> necessary technology in Cloud/OpenStack. Simply deploying docker >> within existing virtual machines should allow us to reduce resource >> usage by quite a bit. But we need to look into that. >> >> >> Best, >> >> Philipp >> >> Am 31.05.2015 um 22:36 schrieb Alex Glikson: >>> Below is a brief summary of the current FIWARE Cloud status with >>> regards to containers and Docker support. >>> >>> * FIWARE Cloud GEs currently deployed in FIWARE Lab provide VM-based >>> IaaS capabilities with OpenStack Nova (with KVM), Glance, Keystone, >>> Cinder and Neutron (with OVS). FIWARE Cloud also provides object >>> storage capabilities based on OpenStack Swift. Moreover, we support >>> PaaS-level capabilities, including the ability to provision and >>> auto-scale complex applications comprising sets of inter-dependent >>> VMs as well as the ability to install and configure the individual >>> software components within VMs using configuration management tools >>> such as Chef. The key user-visible notion behind this capability is >>> the notion of a Blueprint, which defines the application topology, >>> dependencies, auto-scaling rules, software configuration, etc >>> (conceptually similar to Amazon OpsWorks). >>> * In the last few months, the PaaS implementation is undergoing a >>> major transition from a 'proprietary' (although open source) code to >>> full adoption of OpenStack Heat and Murano (as well as contribution >>> of features to the Murano community, such as support for templates, >>> configuration management enhancements, etc). >>> * In parallel, we have started assessing (and prototyping) the options >>> to support/leverage Linux containers technologies, and Docker in >>> particular. We aim at two main use-cases: >>> 1. As a FIWARE Cloud provider (in FIWARE Lab), I want to continue >>> providing (almost?) the same PaaS/IaaS capabilities, but at the >>> same time being able to admit much more users/workloads on the >>> same hardware, by leveraging Containers technologies. >>> 2. As a FIWARE Cloud user, I want to be able to leverage the Docker >>> ecosystem, including public images available at Docker Hub as >>> well as existing tools (TBD: which tools exactly?), while >>> hosting my applications on the FIWARE Lab >>> * The 2nd use-case can be satisfied in several ways, such as: >>> 1. Treat Docker-based cluster as PaaS application (managed by >>> FIWARE Cloud PaaS), while the cloud middleware (IaaS and PaaS) >>> is not aware of individual containers, Docker images, etc. >>> 2. Make Docker containers 1st class citizens in PaaS -- e.g., as an >>> additional delivery mechanism in addition to Chef, deployed on >>> regular IaaS virtualized infra (providing isolation, accounting, >>> etc). In this case each Docker cluster is contained within a >>> single tenant (and even single application within a tenant?) >>> 3. Make Docker containers 1st class citizens in IaaS deployed on >>> bare-metal, making PaaS use it via a standardized API >>> * The current (tentative) plan is to apply a 2-phase approach to >>> introduce containers: phase-1: start by deploying nova-docker in >>> FIWARE lab in order to enable use-case n.1, and then phase-2: >>> gradually migrate to OpenStack Magnum, while incorporating the >>> Docker-native capabilities into Murano in order to support use-case >>> n.2. Of course, the devil is in the details, so we need to discuss >>> further, to understand the exact requirements and implications. >>> >>> >>> Regards, >>> Alex >>> >>> >>> >>> >>> From: Alex Glikson/Haifa/IBM at IBMIL >>> To: _fiware-cloud-containers at lists.fiware.org_ >> >>> Date: 30/05/2015 07:06 PM >>> Subject: [Fiware-cloud-containers] task-force kickoff >>> Sent by: _fiware-cloud-containers-bounces at lists.fiware.org_ >> >>> --------------------------------------------------------------------- >>> --- >>> >>> >>> >>> Dear all, >>> >>> Welcome to 'fiware-cloud-containers' mailing list! We will use this >>> mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. >>> support for Linux containers (such as Docker), including use-cases & >>> requirements, associated IaaS and PaaS-style capabilities, concrete >>> plans going forwards, potential collaboration with teams working on >>> similar/related technologies, etc. >>> >>> Currently we have the following people subscribed to this mailing list: >>> IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for >>> IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud >>> leader), Doron (working with Ezra on containers) >>> TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical >>> leads for the PaaS-related work in FIWARE Cloud) >>> DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter >>> leader/architect) >>> Thales: Mario (FI-Content2, also used to be in the Cloud WP in the >>> old FI-WARE project) >>> >>> Let me know if anyone is missing. >>> >>> As a background, please, read the thread below, including the very >>> nice summary by Mario on the related investigation and implementation >>> in >>> FI-Content2 project. I will send a separate email briefly summarizing >>> the situation in FIWARE. >>> * >>> All *-- notice that we are going to have a phone call to discuss this >>> topic during the upcoming FIWARE architects interlock on Monday, June >>> 1at, 10:30-12:30 CET. Let me know if you can't join.* Mario *-- would >>> you be able to join & explain/demonstrate the FIC2Lab solution? >>> >>> Thanks, >>> Alex >>> >>> >>> >>> >> ====================================================================== >> ============== >>> Alex Glikson >>> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >>> Email: _glikson at il.ibm.com_ | Phone: >> +972-4-8281085 | Mobile: >>> +972-54-6466667 | Fax: +972-4-8296112 >>> >>> >>> ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM >>> ----- >>> >>> _fiware-chapter-architects-bounces at lists.fi-ware.org_ >> wrote on >> 26/05/2015 >>> 08:37:16 AM: >>> >>>> From: Alex Glikson/Haifa/IBM at IBMIL >>>> To: Philipp Slusallek <_philipp.slusallek at dfki.de_ >> > >>>> Cc: _fiware-chapter-architects at lists.fi-ware.org_ >> , Kenneth Nagin/ >>>> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >>>> Date: 26/05/2015 08:37 AM >>>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >>>> Sent by: _fiware-chapter-architects-bounces at lists.fi-ware.org_ >> >>>> >>>> Hi Philipp, >>>> >>>> It is great to hear that FIC2 team is willing to contribute! >>>> In a nutshell, our current plan is to start with surfacing >>>> containers via Nova, and then gradually migrate to Magnum. In terms >>>> of orchestration, we currently work with Heat and Murano, and are >>>> assessing the approaches to support containers (OpenStack alignment >>>> being high priority, but also the ability to surface capabilities >>>> unique to containers). >>>> I suggest that we form a small taskforce to explore this topic. >>>> Please, send me a list of people to involve. >>>> >>>> Thanks, >>>> Alex >>>> >>>> >>> >> ====================================================================== >> ============== >>>> Alex Glikson >>>> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >>>> Email: _glikson at il.ibm.com_ | Phone: >> +972-4-8281085 | Mobile: >>>> +972-54-6466667 | Fax: +972-4-8296112 >>>> >>>> >>>> >>>> >>>> >>>> From: Philipp Slusallek <_philipp.slusallek at dfki.de_ >> > >>>> To: Juanjo Hierro <_juanjose.hierro at telefonica.com_ >> >, Alex >>>> Glikson/Haifa/IBM at IBMIL >>>> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >>>> _architects at lists.fi-ware.org_ >>>> , >> Kenneth Nagin/Haifa/IBM at IBMIL >>>> Date: 26/05/2015 07:21 AM >>>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >>>> >>>> >>>> >>>> Hi, >>>> >>>> Here is some first input from FIC2 on Docker compiled from Mario. As >>>> I mentioned before, we have a full system working that deploys >>>> entire sets of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >>>> >>>> FIC2 would be happy to work with FIWARE to integrate them in >>>> whatever way we at FIWARE decide to implement Docker. Right now we >>>> deploy it on top of the VMs offered by FIWARE but a more native >>>> implementation would be fine as well and save much resources (which >>>> seems to be a major >> issue). >>>> >>>> A presentation by Canonical recently showed a >10x improvement in >>>> resource usage with their containers over plain OpenStack (which is >>>> probably where we are -- but then Intel claims even better numbers >>>> in their Clear Containers solution using traditional VMs). >>>> >>>> Since we have experience already, it seems very useful to make >>>> Docker available especially since deployment of SW is much easier >>>> too and the same SW can be deployed essentially anywhere (including a local machine). >>>> >>>> >>>> Best, >>>> >>>> Philipp >>>> >>>>> There are several ways in which FIWARE could support Docker. My >>>> assumption is that the plan is to add to OpenStack a Docker driver. >>>> It would enable the creation of instances which are not VMs but >>>> OpenStack containers, from images which are not ISOs but Docker >>>> images. It's described here: >>>> __https://wiki.openstack.org/wiki/Docker__ >>>>> >>>>> >>>>> >>>>> Advantages: >>>>> >>>>> + Launches faster than VM instances >>>>> >>>>> + Less resources used from quotas in FIWARE nodes >>>>> >>>>> + images of SEs smaller and easier to build (using Dockerfiles) >>>>> >>>>> >>>>> >>>>> Open questions / disadvantages: >>>>> >>>>> - Will it be possible to pull images from DockerHub (preferred >>>> approach), or do they need to be uploaded to the Glance image >>>> repository of each FIWARE node? >>>>> >>>>> - Is the only way to launch containers the Cloud Portal or >>>> OpenStack API? Not being able to target a Docker API would reduce >>>> the value of Docker and its ecosystem of tools, e.g.: docker-compose >>>> which enables launching applications composed of multiple containers. >>>>> >>>>> - How to solve the limits in terms of IPs? Maybe with a DNS-based >>>> reverse proxy that forwards HTTP requests to container1.john- >>>> smith.cloud.fiware.org to the right container port? >>>>> >>>>> - How would links between containers running on different hosts >>>> work? Using Weave? >>>>> >>>>> - Mounting volumes in Docker is a very common way of passing >>>> configuration files, having persistence, etc. How would this work >>> inOpenStack? >>>>> >>>>> >>>>> >>>>> Our suggestion: >>>>> >>>>> * Ability to use the Docker API remotely (i.e. command-line tools >>>> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >>>>> >>>>> * Ability to pull containers directly from DockerHub with a common >>>> (transparent?) image mirror in each FIWARE node >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Regarding FIC2Lab runner, our wishlist was: >>>>> >>>>> 1. ability to use the same tools locally and remotely in any cloud >>>> (ideally use docker-machine against any cloud provider, in >>>> particular FIWARE nodes) >>>>> >>>>> 2. nice web user interface hiding the complexity of firewalling, >>>> security pairs, public IPs, etc. >>>>> >>>>> 3. compatibility with the tools of the docker ecosystem: docker >>>> CLI, compose, swarm ... >>>>> >>>>> >>>>> >>>>> We compared tools such as tutum (free multi-tenant online >>>> service), shipyard (open-source web UI) and panamax (open-source web >>>> UI and orchestrator). >>>>> >>>>> And we decided to go for Panamax, which is nice for #1 and #2 but >>>> not #3 so far. Here's a few pluses and minuses: >>>>> >>>>> + nice UI with DockerHub search >>>>> >>>>> + composite application, with application template search >>>>> >>>>> - It's not multi-tenant, which is good for portability but bad for >>>> software updates >>>>> >>>>> - its model for composite applications is different from docker- >>>> compose (for now) >>>>> >>>>> - the UI can't synchronize with actions performed in Docker CLI or >>>> API because it's using CoreOS underneath >>>>> >>>>> >>>>> >>>>> The FIC2Lab runner is documented here: >>>>> __http://fic2.github.io/runner__ >>>>> >>>>> >>>> >>>> >>>> >>>> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >>>>> Dear Alex, >>>>> >>>>> It's fine with me. >>>>> >>>>> I believe that we should collocate here the discussion with >>>>> FI-Content2 (Philipp) regarding the stuff they have developed for >>>>> the FIC2-Lab. >>>>> >>>>> @Philipp: could you share some material (document describing >>>>> your >> work >>>>> on FIC2-Lab, slides, whatever) describing what you have done and >> send it >>>>> to the fiware-chapter-architects mailing list as preparation for >>>>> the discussion? >>>>> >>>>> @Alex: please also share the ideas you want to present on this >> matter. >>>>> >>>>> By exchanging material prior to the meeting, we would be able to >>>>> have a more fruitful meeting. >>>>> >>>>> Best regards, >>>>> >>>>> -- Juanjo >>>>> >>>>> On 18/05/15 11:07, Alex Glikson wrote: >>>>>> I am out of office on the 25th. Would the following Monday work? >>>>>> >>>>>> Thanks, >>>>>> Alex >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> From: Juanjo Hierro <_juanjose.hierro at telefonica.com_ >> > >>>>>> To: Philipp Slusallek <_philipp.slusallek at dfki.de_ >> >, Alex >>>>>> Glikson/Haifa/IBM at IBMIL, >> <_fiware-chapter-architects at lists.fi-ware.org_ >> > >>>>>> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >>>>>> Date: 18/05/2015 11:53 AM >>>>>> Subject: Re: [Fiware-chapter-architects] Linux Containers & >>> Docker >>>>>> >>> --------------------------------------------------------------------- >>> --- >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> I would suggest that we book the next architecture session >>>>>> (Monday >>>>>> 25th) to cover this matter. >>>>>> >>>>>> Will you Alex be able to prepare a presentation? >>>>>> >>>>>> Cheers, >>>>>> >>>>>> -- Juanjo >>>>>> >>>>>> On 18/05/15 09:46, Philipp Slusallek wrote: >>>>>>> Hi, >>>>>>> >>>>>>> This sounds very useful to cover as well in this or one of the >>>>>>> next calls. It would also get the architectural activities >>>>>>> started >>> again. It >>>>>>> might even make sense to invite (selected?) AB members from the UCs. >>>>>>> >>>>>>> For example, a lot of work that has been done within FIcontent >>>>>>> to >>> deploy >>>>>>> our SEs and related GEs within Docker on top of FIWARE Lab. >>> Particularly >>>>>>> relevant could be the on-click deployment of entire >>>>>>> arrangements >>> of GEs >>>>>>> and SEs. Better and more dedicated Docker support would >>>>>>> certainly be very welcome as well. >>>>>>> >>>>>>> Unfortunately, I have a lecture this morning and cannot join >>> until 12h. >>>>>>> But Stefan Lemme from my group will be on the call. He has been >>>>>>> a >>> core >>>>>>> member of the relevant FIC2-Lab task force in FIcontent. >>>>>>> >>>>>>> >>>>>>> Best, >>>>>>> >>>>>>> Philipp >>>>>>> >>>>>>> Am 18.05.2015 um 09:28 schrieb Alex Glikson: >>>>>>>> Maybe we can also dedicate some time to have an initial >>> discussion on >>>>>>>> the roadmap to adopt Linux Containers and in particular Docker >>>>>>>> in >>>>>> FIWARE >>>>>>>> Lab. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Alex >>>>>>>> _______________________________________________ >>> Fiware-cloud-containers mailing list >>> _Fiware-cloud-containers at lists.fiware.org_ >> >>> _https://lists.fiware.org/listinfo/fiware-cloud-containers_ >>> >>> >>> >>> _______________________________________________ >>> Fiware-cloud-containers mailing list >>> _Fiware-cloud-containers at lists.fiware.org_ >> >>> _https://lists.fiware.org/listinfo/fiware-cloud-containers_ >>> >> >> -- >> >> ---------------------------------------------------------------------- >> --- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH >> Trippstadter Strasse 122, D-67663 Kaiserslautern >> >> Gesch?ftsf?hrung: >> Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter >> Olthoff Vorsitzender des Aufsichtsrats: >> Prof. Dr. h.c. Hans A. Aukes >> >> Sitz der Gesellschaft: Kaiserslautern (HRB 2313) >> USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 >> ---------------------------------------------------------------------- >> ----- [attachment "philipp_slusallek.vcf" deleted by Alex >> Glikson/Haifa/IBM] >> >> ---------------------------------------------------------------------- >> -- >> >> Este mensaje y sus adjuntos se dirigen exclusivamente a su >> destinatario, puede contener informaci?n privilegiada o confidencial y >> es para uso exclusivo de la persona o entidad de destino. Si no es >> usted. el destinatario indicado, queda notificado de que la lectura, >> utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar >> prohibida en virtud de la legislaci?n vigente. Si ha recibido este >> mensaje por error, le rogamos que nos lo comunique inmediatamente por >> esta misma v?a y proceda a su destrucci?n. >> >> The information contained in this transmission is privileged and >> confidential information intended only for the use of the individual >> or entity named above. If the reader of this message is not the >> intended recipient, you are hereby notified that any dissemination, >> distribution or copying of this communication is strictly prohibited. >> If you have received this transmission in error, do not read it. >> Please immediately reply to the sender that you have received this >> communication in error and then delete it. >> >> Esta mensagem e seus anexos se dirigem exclusivamente ao seu >> destinat?rio, pode conter informa??o privilegiada ou confidencial e ? >> para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa >> senhoria o destinat?rio indicado, fica notificado de que a leitura, >> utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida >> em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, >> rogamos-lhe que nos o comunique imediatamente por esta mesma via e >> proceda a sua destrui??o >> >> >> _______________________________________________ >> Fiware-cloud-containers mailing list >> Fiware-cloud-containers at lists.fiware.org >> https://lists.fiware.org/listinfo/fiware-cloud-containers >> > > -- > > ------------------------------------------------------------------------- > Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern > > Gesch?ftsf?hrung: > Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) > Dr. Walter Olthoff > Vorsitzender des Aufsichtsrats: > Prof. Dr. h.c. Hans A. Aukes > > Sitz der Gesellschaft: Kaiserslautern (HRB 2313) > USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 > --------------------------------------------------------------------------- > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: philipp_slusallek.vcf Type: text/x-vcard Size: 441 bytes Desc: not available URL: From mario.lopezramos at thalesgroup.com Tue Jun 2 11:51:48 2015 From: mario.lopezramos at thalesgroup.com (LOPEZ RAMOS Mario) Date: Tue, 2 Jun 2015 11:51:48 +0200 Subject: [Fiware-cloud-containers] task-force kickoff In-Reply-To: References: <556C05BB.5060900@dfki.de> Message-ID: Hi all, See my comments inline. Best, Mario De : fiware-cloud-containers-bounces at lists.fiware.org [mailto:fiware-cloud-containers-bounces at lists.fiware.org] De la part de Alex Glikson Envoy? : lundi 1 juin 2015 22:36 ? : Philipp Slusallek Cc : fiware-cloud-containers at lists.fiware.org Objet : Re: [Fiware-cloud-containers] task-force kickoff I agree that user/developer perspective is important. However, there are several things that we need to take into account: 1. For users/developers who just want to *use* GEs (rather than develop new ones), FIWARE already provides tools to easily set them up - pre-built VM images, as well as SDC/Chef recipes and PaaS Blueprints (for bundles). Significant effort has been invested by GE owners to populate the corresponding artefacts and enable the automated/seamless deployment in FIWARE Lab. Moreover, as I mentioned, we are currently transforming those tools to better align with OpenStack, and we are also constantly improving them to make more capable/stable. Having said that, there is clearly space for further improvement (usability, documentation, features, integration, etc). In this context, it is not entirely clear whether/how containers (including Docker) can help, besides achieving higher density. [Mario] OK, but: a) It's still not that easy from the perspective of non-expert users. Understanding the concepts linked to SDC and Blueprints requires effort. Is there a way for GE/SE providers to create a 'Launch' button that provisions everything (security groups, IPs, etc.) to have an instance running? With Heat we could do something similar to this: https://blogs.aws.amazon.com/application-management/post/Tx2YSVJV4VMPBHI/Construct-Your-Own-Launch-Stack-URL b) The argument "we spent a lot of effort doing it" may not seem like a valid one from the customer perspective. 2. We do appreciate the value proposition of Docker as a new emerging standard for application packaging and life cycle management -- especially for 'cloud-born' applications. However, the tooling around Docker is rather diverse and immature, and we should be careful before we 'bet' on one (or more) of the options to become part of FIWARE (which aims at becoming a sustainable standard). For example, I haven't seen yet a good DevOps tool that would manage rolling upgrades (and other 'traditional' configuration management scenarios) for Docker-based applications. [Mario] Agreed regarding the maturity. But let's start quick with something simple such as exposing a Docker API endpoint. Users can then use the Docker commandline just like in their local machine, and easily plug their own tools (e.g. docker-compose). Later we can 'bet' on one scheduler/orchestrator or another. FYI: I documented how to use docker-machine to launch a Docker instance on FIWARE, but it should be simpler!: http://fic2.github.io/runner/#run-enablers-using-command-line-tools-in-a-remote-machine-in-fiware-cloud With respect to rolling upgrades, for old stuff you can still use Chef. For 'cloud-born' applications you might as well replace an old container with a new one. Should you take care of your instances like pets or cattle? https://blog.engineyard.com/2014/pets-vs-cattle 3. It is likely that we would need to support both 'old' and 'new' ways to design and manage applications for the upcoming few years. Moreover, new paradigms may arise, addressing additional 'niches' on the spectrum between traditional servers (running full operating system plus an arbitrary software stack) and application containers (running a single process, possible with it's children processes). In my opinion, one of our challenges is to come up with a unified solution (and a roadmap) which would provide a reasonable value proposition in the short term, but would also like to be forward-compatible with new developments which will surely emerge around containers in the near future. For example, if we think that Chef is the way to go from configuration management perspective, maybe we should use and extend the Chef-Docker integration [1] . Alternatively (or in addition?), if we want to stick with Heat for orchestration, maybe we can invest in adding support for Magnum containers as resources in Heat, as well as LXD driver in Nova (for system containers). IMO, this is the problem space we need to further explore (and we started doing so). [Mario] My opinion: - Chef-Docker integration: NO. Chef is for pets and Docker for cattle. Choose either the 'old' or the 'new' way, but not both. - Medium-term roadmap: Heat for orchestration + Magnum for container orchestration + Designate for DNSaaS (important since IPs are scarce!) - Short-term roadmap: Docker host deployment automation (using Heat or docker-machine or something similar to FIC2-Lab) Regards, Alex [1] https://www.chef.io/solutions/containers/ ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Philipp Slusallek > To: Alex Glikson/Haifa/IBM at IBMIL, fiware-cloud-containers at lists.fiware.org Date: 01/06/2015 10:12 AM Subject: Re: [Fiware-cloud-containers] task-force kickoff ________________________________ Hi all, It is certainly the right thing to work on supporting containers natively in the Cloud chapter. It seems that a lot is currently being done in this regard within OpenStack and we should see how we can best contribute to that. However, I would like to point out another aspect that will probably have a much more direct and short-term impact on FIWARE and which we should be able to do in parallel: Focus on the user-perspective. I am talking about greatly simplifying the deployment of enablers for developers (and possibly end users) on FIWARE. Docker is an excellent tool for that as we have experienced for example in FIcontent. Its very easy for owners to provide their enabler and even easier for users to deploy and test them. As with the "Discover, Try, Tweak, Run" approach in FIcontent we can make it tremendously simpler for users to explore what we offer, try it out, share it with others via social networks, and deploy and use the necessary services for real. This is exactly what I believe FIWARE needs. The GEs are still somewhat difficult to access and the documentation for doing so leaves some things to be desired (at least judging from my reviewing job). Taking a much more user-focused approach is important now. So to make things concrete: We should explore (in parallel to your suggestions which is still needed, of course) to install something similar to FIC2-Lab (actually: now officially called "FIWARE Media&Content Lab"). We might be able to leverage the experience we got from there as a basis. But I am sure that together there are many improvements and extensions that should be done. In parallel, we would ask enabler owners to provide suitable dockerified versions that can easily be deployed via this system. This does not make sense for all and can be done incrementally as well. All that is then still needed is to create some nice examples that can be used for the Try and Tweak part, where developers can get first hand experience of enablers, by simply trying out, inspecting, and hacking on existing web-based applications that make use of one or more (bundles!) enablers from any Web browser and without ever having seen the cloud portal (they eventually need to but this is when they are hopefully hooked already). Again, this is not an alternative but a parallel activity that makes containers directly relevant for the developers right now (!) and which relies eventually on the ability to deploy enabler an easier, faster, and with a largely reduced resource footprint via the necessary technology in Cloud/OpenStack. Simply deploying docker within existing virtual machines should allow us to reduce resource usage by quite a bit. But we need to look into that. Best, Philipp Am 31.05.2015 um 22:36 schrieb Alex Glikson: > Below is a brief summary of the current FIWARE Cloud status with regards > to containers and Docker support. > > * FIWARE Cloud GEs currently deployed in FIWARE Lab provide VM-based > IaaS capabilities with OpenStack Nova (with KVM), Glance, Keystone, > Cinder and Neutron (with OVS). FIWARE Cloud also provides object > storage capabilities based on OpenStack Swift. Moreover, we support > PaaS-level capabilities, including the ability to provision and > auto-scale complex applications comprising sets of inter-dependent > VMs as well as the ability to install and configure the individual > software components within VMs using configuration management tools > such as Chef. The key user-visible notion behind this capability is > the notion of a Blueprint, which defines the application topology, > dependencies, auto-scaling rules, software configuration, etc > (conceptually similar to Amazon OpsWorks). > * In the last few months, the PaaS implementation is undergoing a > major transition from a 'proprietary' (although open source) code to > full adoption of OpenStack Heat and Murano (as well as contribution > of features to the Murano community, such as support for templates, > configuration management enhancements, etc). > * In parallel, we have started assessing (and prototyping) the options > to support/leverage Linux containers technologies, and Docker in > particular. We aim at two main use-cases: > 1. As a FIWARE Cloud provider (in FIWARE Lab), I want to continue > providing (almost?) the same PaaS/IaaS capabilities, but at the > same time being able to admit much more users/workloads on the > same hardware, by leveraging Containers technologies. > 2. As a FIWARE Cloud user, I want to be able to leverage the Docker > ecosystem, including public images available at Docker Hub as > well as existing tools (TBD: which tools exactly?), while > hosting my applications on the FIWARE Lab > * The 2nd use-case can be satisfied in several ways, such as: > 1. Treat Docker-based cluster as PaaS application (managed by > FIWARE Cloud PaaS), while the cloud middleware (IaaS and PaaS) > is not aware of individual containers, Docker images, etc. > 2. Make Docker containers 1st class citizens in PaaS -- e.g., as an > additional delivery mechanism in addition to Chef, deployed on > regular IaaS virtualized infra (providing isolation, accounting, > etc). In this case each Docker cluster is contained within a > single tenant (and even single application within a tenant?) > 3. Make Docker containers 1st class citizens in IaaS deployed on > bare-metal, making PaaS use it via a standardized API > * The current (tentative) plan is to apply a 2-phase approach to > introduce containers: phase-1: start by deploying nova-docker in > FIWARE lab in order to enable use-case n.1, and then phase-2: > gradually migrate to OpenStack Magnum, while incorporating the > Docker-native capabilities into Murano in order to support use-case > n.2. Of course, the devil is in the details, so we need to discuss > further, to understand the exact requirements and implications. > > > Regards, > Alex > > > > > From: Alex Glikson/Haifa/IBM at IBMIL > To: fiware-cloud-containers at lists.fiware.org > Date: 30/05/2015 07:06 PM > Subject: [Fiware-cloud-containers] task-force kickoff > Sent by: fiware-cloud-containers-bounces at lists.fiware.org > ------------------------------------------------------------------------ > > > > Dear all, > > Welcome to 'fiware-cloud-containers' mailing list! We will use this > mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. > support for Linux containers (such as Docker), including use-cases & > requirements, associated IaaS and PaaS-style capabilities, concrete > plans going forwards, potential collaboration with teams working on > similar/related technologies, etc. > > Currently we have the following people subscribed to this mailing list: > IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for > IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud > leader), Doron (working with Ezra on containers) > TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads > for the PaaS-related work in FIWARE Cloud) > DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter > leader/architect) > Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old > FI-WARE project) > > Let me know if anyone is missing. > > As a background, please, read the thread below, including the very nice > summary by Mario on the related investigation and implementation in > FI-Content2 project. I will send a separate email briefly summarizing > the situation in FIWARE. > * > All *-- notice that we are going to have a phone call to discuss this > topic during the upcoming FIWARE architects interlock on Monday, June > 1at, 10:30-12:30 CET. Let me know if you can't join.* > Mario *-- would you be able to join & explain/demonstrate the FIC2Lab > solution? > > Thanks, > Alex > > > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- > > fiware-chapter-architects-bounces at lists.fi-ware.org wrote on 26/05/2015 > 08:37:16 AM: > >> From: Alex Glikson/Haifa/IBM at IBMIL >> To: Philipp Slusallek > >> Cc: fiware-chapter-architects at lists.fi-ware.org, Kenneth Nagin/ >> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL >> Date: 26/05/2015 08:37 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> Sent by: fiware-chapter-architects-bounces at lists.fi-ware.org >> >> Hi Philipp, >> >> It is great to hear that FIC2 team is willing to contribute! >> In a nutshell, our current plan is to start with surfacing >> containers via Nova, and then gradually migrate to Magnum. In terms >> of orchestration, we currently work with Heat and Murano, and are >> assessing the approaches to support containers (OpenStack alignment >> being high priority, but also the ability to surface capabilities >> unique to containers). >> I suggest that we form a small taskforce to explore this topic. >> Please, send me a list of people to involve. >> >> Thanks, >> Alex >> >> > ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> >> >> >> From: Philipp Slusallek > >> To: Juanjo Hierro >, Alex >> Glikson/Haifa/IBM at IBMIL >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter- >> architects at lists.fi-ware.org, Kenneth Nagin/Haifa/IBM at IBMIL >> Date: 26/05/2015 07:21 AM >> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker >> >> >> >> Hi, >> >> Here is some first input from FIC2 on Docker compiled from Mario. As I >> mentioned before, we have a full system working that deploys entire sets >> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs. >> >> FIC2 would be happy to work with FIWARE to integrate them in whatever >> way we at FIWARE decide to implement Docker. Right now we deploy it on >> top of the VMs offered by FIWARE but a more native implementation would >> be fine as well and save much resources (which seems to be a major issue). >> >> A presentation by Canonical recently showed a >10x improvement in >> resource usage with their containers over plain OpenStack (which is >> probably where we are -- but then Intel claims even better numbers in >> their Clear Containers solution using traditional VMs). >> >> Since we have experience already, it seems very useful to make Docker >> available especially since deployment of SW is much easier too and the >> same SW can be deployed essentially anywhere (including a local machine). >> >> >> Best, >> >> Philipp >> >> > There are several ways in which FIWARE could support Docker. My >> assumption is that the plan is to add to OpenStack a Docker driver. >> It would enable the creation of instances which are not VMs but >> OpenStack containers, from images which are not ISOs but Docker >> images. It's described here: _https://wiki.openstack.org/wiki/Docker_ >> > >> > >> > >> > Advantages: >> > >> > + Launches faster than VM instances >> > >> > + Less resources used from quotas in FIWARE nodes >> > >> > + images of SEs smaller and easier to build (using Dockerfiles) >> > >> > >> > >> > Open questions / disadvantages: >> > >> > - Will it be possible to pull images from DockerHub (preferred >> approach), or do they need to be uploaded to the Glance image >> repository of each FIWARE node? >> > >> > - Is the only way to launch containers the Cloud Portal or >> OpenStack API? Not being able to target a Docker API would reduce >> the value of Docker and its ecosystem of tools, e.g.: docker-compose >> which enables launching applications composed of multiple containers. >> > >> > - How to solve the limits in terms of IPs? Maybe with a DNS-based >> reverse proxy that forwards HTTP requests to container1.john- >> smith.cloud.fiware.org to the right container port? >> > >> > - How would links between containers running on different hosts >> work? Using Weave? >> > >> > - Mounting volumes in Docker is a very common way of passing >> configuration files, having persistence, etc. How would this work > inOpenStack? >> > >> > >> > >> > Our suggestion: >> > >> > * Ability to use the Docker API remotely (i.e. command-line tools >> such as docker and docker-compose, Kitematic for a Mac OS X GUI) >> > >> > * Ability to pull containers directly from DockerHub with a common >> (transparent?) image mirror in each FIWARE node >> > >> > >> > >> > >> > >> > >> > >> > Regarding FIC2Lab runner, our wishlist was: >> > >> > 1. ability to use the same tools locally and remotely in any cloud >> (ideally use docker-machine against any cloud provider, in >> particular FIWARE nodes) >> > >> > 2. nice web user interface hiding the complexity of firewalling, >> security pairs, public IPs, etc. >> > >> > 3. compatibility with the tools of the docker ecosystem: docker >> CLI, compose, swarm ... >> > >> > >> > >> > We compared tools such as tutum (free multi-tenant online >> service), shipyard (open-source web UI) and panamax (open-source web >> UI and orchestrator). >> > >> > And we decided to go for Panamax, which is nice for #1 and #2 but >> not #3 so far. Here's a few pluses and minuses: >> > >> > + nice UI with DockerHub search >> > >> > + composite application, with application template search >> > >> > - It's not multi-tenant, which is good for portability but bad for >> software updates >> > >> > - its model for composite applications is different from docker- >> compose (for now) >> > >> > - the UI can't synchronize with actions performed in Docker CLI or >> API because it's using CoreOS underneath >> > >> > >> > >> > The FIC2Lab runner is documented here: _http://fic2.github.io/runner_ >> > >> > >> >> >> >> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro: >> > Dear Alex, >> > >> > It's fine with me. >> > >> > I believe that we should collocate here the discussion with >> > FI-Content2 (Philipp) regarding the stuff they have developed for the >> > FIC2-Lab. >> > >> > @Philipp: could you share some material (document describing your work >> > on FIC2-Lab, slides, whatever) describing what you have done and send it >> > to the fiware-chapter-architects mailing list as preparation for the >> > discussion? >> > >> > @Alex: please also share the ideas you want to present on this matter. >> > >> > By exchanging material prior to the meeting, we would be able to have >> > a more fruitful meeting. >> > >> > Best regards, >> > >> > -- Juanjo >> > >> > On 18/05/15 11:07, Alex Glikson wrote: >> >> I am out of office on the 25th. Would the following Monday work? >> >> >> >> Thanks, >> >> Alex >> >> >> >> >> >> >> >> >> >> >> >> >> >> From: Juanjo Hierro > >> >> To: Philipp Slusallek >, Alex >> >> Glikson/Haifa/IBM at IBMIL, > >> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth Nagin/Haifa/IBM at IBMIL >> >> Date: 18/05/2015 11:53 AM >> >> Subject: Re: [Fiware-chapter-architects] Linux Containers & > Docker >> >> > ------------------------------------------------------------------------ >> >> >> >> >> >> >> >> >> >> I would suggest that we book the next architecture session (Monday >> >> 25th) to cover this matter. >> >> >> >> Will you Alex be able to prepare a presentation? >> >> >> >> Cheers, >> >> >> >> -- Juanjo >> >> >> >> On 18/05/15 09:46, Philipp Slusallek wrote: >> >> > Hi, >> >> > >> >> > This sounds very useful to cover as well in this or one of the next >> >> > calls. It would also get the architectural activities started > again. It >> >> > might even make sense to invite (selected?) AB members from the UCs. >> >> > >> >> > For example, a lot of work that has been done within FIcontent to > deploy >> >> > our SEs and related GEs within Docker on top of FIWARE Lab. > Particularly >> >> > relevant could be the on-click deployment of entire arrangements > of GEs >> >> > and SEs. Better and more dedicated Docker support would certainly be >> >> > very welcome as well. >> >> > >> >> > Unfortunately, I have a lecture this morning and cannot join > until 12h. >> >> > But Stefan Lemme from my group will be on the call. He has been a > core >> >> > member of the relevant FIC2-Lab task force in FIcontent. >> >> > >> >> > >> >> > Best, >> >> > >> >> > Philipp >> >> > >> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson: >> >> >> Maybe we can also dedicate some time to have an initial > discussion on >> >> >> the roadmap to adopt Linux Containers and in particular Docker in >> >> FIWARE >> >> >> Lab. >> >> >> >> >> >> Regards, >> >> >> Alex >> >> >>_______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] -------------- next part -------------- An HTML attachment was scrubbed... URL: From GLIKSON at il.ibm.com Tue Jun 2 23:28:27 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Wed, 3 Jun 2015 00:28:27 +0300 Subject: [Fiware-cloud-containers] Invitation: FIWARE Cloud containers taskforce (4 Jun 11:00 AM ZE2 in 695/HL-Haifa-Site@Research) Message-ID: Call-in numbers: https://www.teleconference.att.com/servlet/glbAccess?process=1&accessCode=5546993&accessNumber=1809417783#C2 Participant Code: 5546993 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 1849 bytes Desc: not available URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: c002814.ics Type: application/octet-stream Size: 1849 bytes Desc: not available URL: From philipp.slusallek at dfki.de Wed Jun 3 06:28:52 2015 From: philipp.slusallek at dfki.de (Philipp Slusallek) Date: Wed, 03 Jun 2015 06:28:52 +0200 Subject: [Fiware-cloud-containers] Invitation: FIWARE Cloud containers taskforce (4 Jun 11:00 AM ZE2 in 695/HL-Haifa-Site@Research) In-Reply-To: References: Message-ID: <556E8284.8080106@dfki.de> Hi all, Thursday is a holiday in some of the Germany states (including mine). I am not sure yet but I might be able to call in, still. Best, Philipp Am 02.06.2015 um 23:28 schrieb Alex Glikson: > Event Invitation > > Title: > > > > FIWARE Cloud containers taskforce > > Location: > > > > When: > > > > Thu 4 Jun 2015 10:00 ? 11:30 > > > > > > Organizer: > > > > Alex Glikson/Haifa/IBM > > Description: > > > > Call-in numbers: > https://www.teleconference.att.com/servlet/glbAccess?process=1&accessCode=5546993&accessNumber=1809417783#C2 > Participant Code: 5546993 > > Comment: > > > > Attendees: > > > > > > Alex Glikson/Haifa/IBM > > fiware-cloud-containers at lists.fiware.org > > 695/HL-Haifa-Site <695/HL-Haifa-Site at d06ml319.portsmouth.uk.ibm.com> > > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: philipp_slusallek.vcf Type: text/x-vcard Size: 441 bytes Desc: not available URL: From GLIKSON at il.ibm.com Wed Jun 3 06:59:39 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Wed, 3 Jun 2015 07:59:39 +0300 Subject: [Fiware-cloud-containers] Invitation: FIWARE Cloud containers taskforce (4 Jun 11:00 AM ZE2 in 695/HL-Haifa-Site@Research) In-Reply-To: <556E8284.8080106@dfki.de> References: <556E8284.8080106@dfki.de> Message-ID: Unfortunately, there was no slot when everyone was available in Doodle -- hope this can work. In any case, I think we made good progress over email sharing initial perspectives on considerations and requirements (and it seems that we are generally in agreement) , so now the discussion can probably be more technical/practical. Regards, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Philipp Slusallek To: Alex Glikson/Haifa/IBM at IBMIL, fiware-cloud-containers at lists.fiware.org Date: 03/06/2015 07:29 AM Subject: Re: [Fiware-cloud-containers] Invitation: FIWARE Cloud containers taskforce (4 Jun 11:00 AM ZE2 in 695/HL-Haifa-Site at Research) Hi all, Thursday is a holiday in some of the Germany states (including mine). I am not sure yet but I might be able to call in, still. Best, Philipp Am 02.06.2015 um 23:28 schrieb Alex Glikson: > Event Invitation > > Title: > > > > FIWARE Cloud containers taskforce > > Location: > > > > When: > > > > Thu 4 Jun 2015 10:00 ? 11:30 > > > > > > Organizer: > > > > Alex Glikson/Haifa/IBM > > Description: > > > > Call-in numbers: > https://www.teleconference.att.com/servlet/glbAccess?process=1&accessCode=5546993&accessNumber=1809417783#C2 > Participant Code: 5546993 > > Comment: > > > > Attendees: > > > > > > Alex Glikson/Haifa/IBM > > fiware-cloud-containers at lists.fiware.org > > 695/HL-Haifa-Site <695/HL-Haifa-Site at d06ml319.portsmouth.uk.ibm.com> > > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- [attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM] -------------- next part -------------- An HTML attachment was scrubbed... URL: From NAGIN at il.ibm.com Thu Jun 4 10:33:07 2015 From: NAGIN at il.ibm.com (Kenneth Nagin) Date: Thu, 4 Jun 2015 11:33:07 +0300 Subject: [Fiware-cloud-containers] fiware-cloud-containters meeting notes Message-ID: https://docs.google.com/document/d/1yjcGc6GxKA4sotbQr9J2oSpdtI9mCvs7JFKgiE6Ytuc/edit# Best Regards, Kenneth Nagin Ph: +972-4-8296227 Cell: 054-6976227 Fx: +972-4- 8296114 http://researcher.ibm.com/view.php?person=il-NAGIN -------------- next part -------------- An HTML attachment was scrubbed... URL: From GLIKSON at il.ibm.com Wed Jun 10 09:02:12 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Wed, 10 Jun 2015 10:02:12 +0300 Subject: [Fiware-cloud-containers] fiware-cloud-containters meeting notes In-Reply-To: References: Message-ID: Reminder to send me names of people to be added to this mailing list for the user experience discussion. Thanks, Alex From: Kenneth Nagin/Haifa/IBM at IBMIL To: fiware-cloud-containers at lists.fiware.org Date: 04/06/2015 11:33 AM Subject: [Fiware-cloud-containers] fiware-cloud-containters meeting notes Sent by: fiware-cloud-containers-bounces at lists.fiware.org https://docs.google.com/document/d/1yjcGc6GxKA4sotbQr9J2oSpdtI9mCvs7JFKgiE6Ytuc/edit# Best Regards, Kenneth Nagin Ph: +972-4-8296227 Cell: 054-6976227 Fx: +972-4- 8296114 http://researcher.ibm.com/view.php?person=il-NAGIN _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers -------------- next part -------------- An HTML attachment was scrubbed... URL: From GLIKSON at il.ibm.com Tue Jun 30 20:34:52 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Tue, 30 Jun 2015 21:34:52 +0300 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker Message-ID: Dear partners, Following the initial discussion at the containers task force, we identified two follow-on (related) discussion tracks: 1) expected FIWARE Lab user/developer experience with Docker 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker (and enable requirements derived from #1) This email refers to topic (1). I will try to summarize the initial set of usage scenarios that we may want to support. We can then iterate on this over email, and have a phone call if needed, when we feel that we are close to a convergence point. Please, notice that I've subscribed several additional people relevant for the discussion (who could provide input related to UI, developers perspectives and operations perspectives). Assumptions: Notice that a prerequisite for all the usage scenarios is that all the FIWARE GEs (and SEs) are packages as Docker images and are kept in a central repository, preferably the docker hub (under a 'to-be-created' "fiware" namespace). Also, there is an assumption that the users/developers would want to work with Docker tools as much as possible (surfacing some or all of the capabilities via the FIWARE Cloud Portal too). Juanjo will elaborate on the approach we are thinking of to promote this with GE/SE owners and the broader developer community. Group 1: Basic Docker environment setup Usage scenario #1.1: A user wants a local Docker runtime on his laptop. He follows the standard instructions on setting up a docker host (e.g., running within a VirtalBox VM, using docker-machine / boot2docker) Usage scenario #1.2: A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab (where he would then run various Docker containers). He uses the standard docker-machine tool, specifying the URI of the OpenStack Keystone in FIWARE Lab (and additional parameters, as needed). The tool creates a VM using standard OpenStack APIs (natively supported by FIWARE Lab) and configures Docker within the VM. The VM would need to have a public IP (naturally). Group 2: Basic life cycle of individual containers running GEs/SEs Usage scenario #2.1: A developer wants to publish (a version of) a GE/SE. After he is done creating the new Docker image, he pushes the new version of the GE/SE to Docker hub under corresponding FIWARE namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the image points to the new version. Usage scenario #2.2: A user/developer wants to deploy locally an instance of a certain GE/SE. He uses the standard docker CLI to locally provision a container, referring to the corresponding image at Docker hub -- e.g.: "$ docker -H boot2docker-vm:2376 run fiware/GE/cb-orion" Usage scenario #2.3: A user/developer wants to deploy an instance of a certain GE/SE within his Docker VM on FIWARE Lab He uses the standard docker CLI to provision a container, referring to the location of his Docker VM as well as the corresponding image at Docker hub -- e.g.: "$ docker -H mydocker-vm37.lab.fiware.org:2376 run fiware/GE/cb-orion". ISSUE: the user would need to open the corresponding firewall ports in his VM (same as those of the GE/SE, or following the mapping performed during container provisioning) in order to make the GE/SE accessible. Ideally, this should be done in a scalable but secure manner. An easy solution is to open up-front (during VM provisioning) a range of ports (via setting up corresponding security group). A more advanced solution is to update the security group dynamically. Usage scenario #2.4: A user/developer wants to update a container comprising certain GE/SE with the latest version recently published in the dedicated namespace of the Docker Hub. He pulls the latest version from the Docker Hub, kills the old container, and starts a new one (attaching to the same resources). Note that this would work well when the application is properly designed for Docker (e.g., the container itself is stateless), and when the previous version of the container has been provisioned manually (as in #2, #4 above) by the developer (and he knows which resources to connect to). Group 3: Support for 'bundles' of GEs/SEs that together perform a certain complex function Usage scenario #3.1: A developer wants to publish (a version of) a 'bundle' of GEs/SEs that together perform a certain complex function. He creates a 'template' (e.g., following the format of docker-compose), referring to the individual GE/SE images as well as their interdependencies (e.g., links) and other composition properties. This might be done using a text editor, a Web UI provided by the FIWARE Cloud (conceptually similar to today's UI for creation of blueprints), or other tools from Docker ecosystem. He then uploads the template to the centralized templats repository. Note: it is likely that docker-compose will be able to use Docker Registry/Hub as a repository for templates (including versioning, push/pull, etc). Usage scenario #3.2: A user wants to provision locally a set of GEs/SEs, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the local Docker URL. Usage scenario #3.3: A user wants to provision a set of GEs/SEs in FIWARE Lab, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the target Docker URL in FIWARE Lab. Alternatively, he uses the FIWARE Portal UI to do the same (e.g., with Murano backend invoking docker-compose). Usage scenario #3.4: A user wants to update his Docker environment (local or on FIWARE Lab) with the latest version of a certain 'bundle' Note: may require enancements to docker-compose Group 4: Advanced scenarios Usage scenario #4.1: A user/developer wants to provision a cluster of VMs on FIWARE Lab that would host his Docker cluster (managed with Swarm or Kubernetes) He uses corresponding Murano/Heat template to provision the VMs and to configure the Docker/Swarm/Kubernetes cluster. Usage scenario #4.2: A user/developer wants to access a global instance of a Docker service in FIWARE Lab (shared, scalable, managed), so that he doesn't need to manage the corresponding VM(s) by himself. He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab Docker API endpoint with the standard Docker tools or FIWARE-specific tools (as outlined above ). Usage scenario #4.3: A user/developer wants to manage access control for Docker images among FIWARE Lab users. He starts using the Docker Registry/Hub deployed within the FIWARE Lab. Notice that this is a very initial list -- I am sure that there are many inaccuracies and gaps. Feel free to comment. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 -------------- next part -------------- An HTML attachment was scrubbed... URL: From juanjose.hierro at telefonica.com Tue Jun 30 22:05:50 2015 From: juanjose.hierro at telefonica.com (Juanjo Hierro) Date: Tue, 30 Jun 2015 22:05:50 +0200 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: References: Message-ID: <5592F69E.4060803@telefonica.com> An HTML attachment was scrubbed... URL: