I agree that user/developer perspective is important.
However, there are several things that we need to take into account:
For users/developers who just want to *use* GEs (rather than develop new
ones), FIWARE already provides tools to easily set them up - pre-built VM
images, as well as SDC/Chef recipes and PaaS Blueprints (for bundles).
Significant effort has been invested by GE owners to populate the
corresponding artefacts and enable the automated/seamless deployment in
FIWARE Lab. Moreover, as I mentioned, we are currently transforming those
tools to better align with OpenStack, and we are also constantly improving
them to make more capable/stable. Having said that, there is clearly space
for further improvement (usability, documentation, features, integration,
etc). In this context, it is not entirely clear whether/how containers
(including Docker) can help, besides achieving higher density.
We do appreciate the value proposition of Docker as a new emerging
standard for application packaging and life cycle management -- especially
for 'cloud-born' applications. However, the tooling around Docker is
rather diverse and immature, and we should be careful before we 'bet' on
one (or more) of the options to become part of FIWARE (which aims at
becoming a sustainable standard). For example, I haven't seen yet a good
DevOps tool that would manage rolling upgrades (and other 'traditional'
configuration management scenarios) for Docker-based applications.
It is likely that we would need to support both 'old' and 'new' ways to
design and manage applications for the upcoming few years. Moreover, new
paradigms may arise, addressing additional 'niches' on the spectrum
between traditional servers (running full operating system plus an
arbitrary software stack) and application containers (running a single
process, possible with it's children processes). In my opinion, one of our
challenges is to come up with a unified solution (and a roadmap) which
would provide a reasonable value proposition in the short term, but would
also like to be forward-compatible with new developments which will surely
emerge around containers in the near future. For example, if we think that
Chef is the way to go from configuration management perspective, maybe we
should use and extend the Chef-Docker integration [1] . Alternatively (or
in addition?), if we want to stick with Heat for orchestration, maybe we
can invest in adding support for Magnum containers as resources in Heat,
as well as LXD driver in Nova (for system containers). IMO, this is the
problem space we need to further explore (and we started doing so).
Regards,
Alex
[1] https://www.chef.io/solutions/containers/
====================================================================================
Alex Glikson
Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab
Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile:
+972-54-6466667 | Fax: +972-4-8296112
From: Philipp Slusallek <philipp.slusallek at dfki.de>
To: Alex Glikson/Haifa/IBM at IBMIL,
fiware-cloud-containers at lists.fiware.org
Date: 01/06/2015 10:12 AM
Subject: Re: [Fiware-cloud-containers] task-force kickoff
Hi all,
It is certainly the right thing to work on supporting containers
natively in the Cloud chapter. It seems that a lot is currently being
done in this regard within OpenStack and we should see how we can best
contribute to that.
However, I would like to point out another aspect that will probably
have a much more direct and short-term impact on FIWARE and which we
should be able to do in parallel: Focus on the user-perspective. I am
talking about greatly simplifying the deployment of enablers for
developers (and possibly end users) on FIWARE.
Docker is an excellent tool for that as we have experienced for example
in FIcontent. Its very easy for owners to provide their enabler and even
easier for users to deploy and test them. As with the "Discover, Try,
Tweak, Run" approach in FIcontent we can make it tremendously simpler
for users to explore what we offer, try it out, share it with others via
social networks, and deploy and use the necessary services for real.
This is exactly what I believe FIWARE needs. The GEs are still somewhat
difficult to access and the documentation for doing so leaves some
things to be desired (at least judging from my reviewing job). Taking a
much more user-focused approach is important now.
So to make things concrete: We should explore (in parallel to your
suggestions which is still needed, of course) to install something
similar to FIC2-Lab (actually: now officially called "FIWARE
Media&Content Lab"). We might be able to leverage the experience we got
from there as a basis. But I am sure that together there are many
improvements and extensions that should be done.
In parallel, we would ask enabler owners to provide suitable dockerified
versions that can easily be deployed via this system. This does not make
sense for all and can be done incrementally as well. All that is then
still needed is to create some nice examples that can be used for the
Try and Tweak part, where developers can get first hand experience of
enablers, by simply trying out, inspecting, and hacking on existing
web-based applications that make use of one or more (bundles!) enablers
from any Web browser and without ever having seen the cloud portal (they
eventually need to but this is when they are hopefully hooked already).
Again, this is not an alternative but a parallel activity that makes
containers directly relevant for the developers right now (!) and which
relies eventually on the ability to deploy enabler an easier, faster,
and with a largely reduced resource footprint via the necessary
technology in Cloud/OpenStack. Simply deploying docker within existing
virtual machines should allow us to reduce resource usage by quite a
bit. But we need to look into that.
Best,
Philipp
Am 31.05.2015 um 22:36 schrieb Alex Glikson:
> Below is a brief summary of the current FIWARE Cloud status with regards
> to containers and Docker support.
>
> * FIWARE Cloud GEs currently deployed in FIWARE Lab provide VM-based
> IaaS capabilities with OpenStack Nova (with KVM), Glance, Keystone,
> Cinder and Neutron (with OVS). FIWARE Cloud also provides object
> storage capabilities based on OpenStack Swift. Moreover, we support
> PaaS-level capabilities, including the ability to provision and
> auto-scale complex applications comprising sets of inter-dependent
> VMs as well as the ability to install and configure the individual
> software components within VMs using configuration management tools
> such as Chef. The key user-visible notion behind this capability is
> the notion of a Blueprint, which defines the application topology,
> dependencies, auto-scaling rules, software configuration, etc
> (conceptually similar to Amazon OpsWorks).
> * In the last few months, the PaaS implementation is undergoing a
> major transition from a 'proprietary' (although open source) code to
> full adoption of OpenStack Heat and Murano (as well as contribution
> of features to the Murano community, such as support for templates,
> configuration management enhancements, etc).
> * In parallel, we have started assessing (and prototyping) the options
> to support/leverage Linux containers technologies, and Docker in
> particular. We aim at two main use-cases:
> 1. As a FIWARE Cloud provider (in FIWARE Lab), I want to continue
> providing (almost?) the same PaaS/IaaS capabilities, but at the
> same time being able to admit much more users/workloads on the
> same hardware, by leveraging Containers technologies.
> 2. As a FIWARE Cloud user, I want to be able to leverage the Docker
> ecosystem, including public images available at Docker Hub as
> well as existing tools (TBD: which tools exactly?), while
> hosting my applications on the FIWARE Lab
> * The 2nd use-case can be satisfied in several ways, such as:
> 1. Treat Docker-based cluster as PaaS application (managed by
> FIWARE Cloud PaaS), while the cloud middleware (IaaS and PaaS)
> is not aware of individual containers, Docker images, etc.
> 2. Make Docker containers 1st class citizens in PaaS -- e.g., as an
> additional delivery mechanism in addition to Chef, deployed on
> regular IaaS virtualized infra (providing isolation, accounting,
> etc). In this case each Docker cluster is contained within a
> single tenant (and even single application within a tenant?)
> 3. Make Docker containers 1st class citizens in IaaS deployed on
> bare-metal, making PaaS use it via a standardized API
> * The current (tentative) plan is to apply a 2-phase approach to
> introduce containers: phase-1: start by deploying nova-docker in
> FIWARE lab in order to enable use-case n.1, and then phase-2:
> gradually migrate to OpenStack Magnum, while incorporating the
> Docker-native capabilities into Murano in order to support use-case
> n.2. Of course, the devil is in the details, so we need to discuss
> further, to understand the exact requirements and implications.
>
>
> Regards,
> Alex
>
>
>
>
> From: Alex Glikson/Haifa/IBM at IBMIL
> To: fiware-cloud-containers at lists.fiware.org
> Date: 30/05/2015 07:06 PM
> Subject: [Fiware-cloud-containers] task-force kickoff
> Sent by: fiware-cloud-containers-bounces at lists.fiware.org
> ------------------------------------------------------------------------
>
>
>
> Dear all,
>
> Welcome to 'fiware-cloud-containers' mailing list! We will use this
> mailing list to discuss the details of FIWARE Cloud roadmap w.r.t.
> support for Linux containers (such as Docker), including use-cases &
> requirements, associated IaaS and PaaS-style capabilities, concrete
> plans going forwards, potential collaboration with teams working on
> similar/related technologies, etc.
>
> Currently we have the following people subscribed to this mailing list:
> IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for
> IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud
> leader), Doron (working with Ezra on containers)
> TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads
> for the PaaS-related work in FIWARE Cloud)
> DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter
> leader/architect)
> Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old
> FI-WARE project)
>
> Let me know if anyone is missing.
>
> As a background, please, read the thread below, including the very nice
> summary by Mario on the related investigation and implementation in
> FI-Content2 project. I will send a separate email briefly summarizing
> the situation in FIWARE.
> *
> All *-- notice that we are going to have a phone call to discuss this
> topic during the upcoming FIWARE architects interlock on Monday, June
> 1at, 10:30-12:30 CET. Let me know if you can't join.*
> Mario *-- would you be able to join & explain/demonstrate the FIC2Lab
> solution?
>
> Thanks,
> Alex
>
>
>
>
====================================================================================
> Alex Glikson
> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab
> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile:
> +972-54-6466667 | Fax: +972-4-8296112
>
>
> ----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM -----
>
> fiware-chapter-architects-bounces at lists.fi-ware.org wrote on 26/05/2015
> 08:37:16 AM:
>
>> From: Alex Glikson/Haifa/IBM at IBMIL
>> To: Philipp Slusallek <philipp.slusallek at dfki.de>
>> Cc: fiware-chapter-architects at lists.fi-ware.org, Kenneth Nagin/
>> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL
>> Date: 26/05/2015 08:37 AM
>> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker
>> Sent by: fiware-chapter-architects-bounces at lists.fi-ware.org
>>
>> Hi Philipp,
>>
>> It is great to hear that FIC2 team is willing to contribute!
>> In a nutshell, our current plan is to start with surfacing
>> containers via Nova, and then gradually migrate to Magnum. In terms
>> of orchestration, we currently work with Heat and Murano, and are
>> assessing the approaches to support containers (OpenStack alignment
>> being high priority, but also the ability to surface capabilities
>> unique to containers).
>> I suggest that we form a small taskforce to explore this topic.
>> Please, send me a list of people to involve.
>>
>> Thanks,
>> Alex
>>
>>
>
====================================================================================
>> Alex Glikson
>> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab
>> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile:
>> +972-54-6466667 | Fax: +972-4-8296112
>>
>>
>>
>>
>>
>> From: Philipp Slusallek <philipp.slusallek at dfki.de>
>> To: Juanjo Hierro <juanjose.hierro at telefonica.com>, Alex
>> Glikson/Haifa/IBM at IBMIL
>> Cc: Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter-
>> architects at lists.fi-ware.org, Kenneth Nagin/Haifa/IBM at IBMIL
>> Date: 26/05/2015 07:21 AM
>> Subject: Re: [Fiware-chapter-architects] Linux Containers &
Docker
>>
>>
>>
>> Hi,
>>
>> Here is some first input from FIC2 on Docker compiled from Mario. As I
>> mentioned before, we have a full system working that deploys entire
sets
>> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs.
>>
>> FIC2 would be happy to work with FIWARE to integrate them in whatever
>> way we at FIWARE decide to implement Docker. Right now we deploy it on
>> top of the VMs offered by FIWARE but a more native implementation would
>> be fine as well and save much resources (which seems to be a major
issue).
>>
>> A presentation by Canonical recently showed a >10x improvement in
>> resource usage with their containers over plain OpenStack (which is
>> probably where we are -- but then Intel claims even better numbers in
>> their Clear Containers solution using traditional VMs).
>>
>> Since we have experience already, it seems very useful to make Docker
>> available especially since deployment of SW is much easier too and the
>> same SW can be deployed essentially anywhere (including a local
machine).
>>
>>
>> Best,
>>
>> Philipp
>>
>> > There are several ways in which FIWARE could support Docker. My
>> assumption is that the plan is to add to OpenStack a Docker driver.
>> It would enable the creation of instances which are not VMs but
>> OpenStack containers, from images which are not ISOs but Docker
>> images. It?s described here: _https://wiki.openstack.org/wiki/Docker_
>> >
>> >
>> >
>> > Advantages:
>> >
>> > + Launches faster than VM instances
>> >
>> > + Less resources used from quotas in FIWARE nodes
>> >
>> > + images of SEs smaller and easier to build (using Dockerfiles)
>> >
>> >
>> >
>> > Open questions / disadvantages:
>> >
>> > - Will it be possible to pull images from DockerHub (preferred
>> approach), or do they need to be uploaded to the Glance image
>> repository of each FIWARE node?
>> >
>> > - Is the only way to launch containers the Cloud Portal or
>> OpenStack API? Not being able to target a Docker API would reduce
>> the value of Docker and its ecosystem of tools, e.g.: docker-compose
>> which enables launching applications composed of multiple containers.
>> >
>> > - How to solve the limits in terms of IPs? Maybe with a DNS-based
>> reverse proxy that forwards HTTP requests to container1.john-
>> smith.cloud.fiware.org to the right container port?
>> >
>> > - How would links between containers running on different hosts
>> work? Using Weave?
>> >
>> > - Mounting volumes in Docker is a very common way of passing
>> configuration files, having persistence, etc. How would this work
> inOpenStack?
>> >
>> >
>> >
>> > Our suggestion:
>> >
>> > * Ability to use the Docker API remotely (i.e. command-line tools
>> such as docker and docker-compose, Kitematic for a Mac OS X GUI)
>> >
>> > * Ability to pull containers directly from DockerHub with a common
>> (transparent?) image mirror in each FIWARE node
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > Regarding FIC2Lab runner, our wishlist was:
>> >
>> > 1. ability to use the same tools locally and remotely in any cloud
>> (ideally use docker-machine against any cloud provider, in
>> particular FIWARE nodes)
>> >
>> > 2. nice web user interface hiding the complexity of firewalling,
>> security pairs, public IPs, etc.
>> >
>> > 3. compatibility with the tools of the docker ecosystem: docker
>> CLI, compose, swarm ...
>> >
>> >
>> >
>> > We compared tools such as tutum (free multi-tenant online
>> service), shipyard (open-source web UI) and panamax (open-source web
>> UI and orchestrator).
>> >
>> > And we decided to go for Panamax, which is nice for #1 and #2 but
>> not #3 so far. Here's a few pluses and minuses:
>> >
>> > + nice UI with DockerHub search
>> >
>> > + composite application, with application template search
>> >
>> > - It's not multi-tenant, which is good for portability but bad for
>> software updates
>> >
>> > - its model for composite applications is different from docker-
>> compose (for now)
>> >
>> > - the UI can't synchronize with actions performed in Docker CLI or
>> API because it's using CoreOS underneath
>> >
>> >
>> >
>> > The FIC2Lab runner is documented here: _http://fic2.github.io/runner_
>> >
>> >
>>
>>
>>
>> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro:
>> > Dear Alex,
>> >
>> > It's fine with me.
>> >
>> > I believe that we should collocate here the discussion with
>> > FI-Content2 (Philipp) regarding the stuff they have developed for the
>> > FIC2-Lab.
>> >
>> > @Philipp: could you share some material (document describing your
work
>> > on FIC2-Lab, slides, whatever) describing what you have done and send
it
>> > to the fiware-chapter-architects mailing list as preparation for the
>> > discussion?
>> >
>> > @Alex: please also share the ideas you want to present on this
matter.
>> >
>> > By exchanging material prior to the meeting, we would be able to
have
>> > a more fruitful meeting.
>> >
>> > Best regards,
>> >
>> > -- Juanjo
>> >
>> > On 18/05/15 11:07, Alex Glikson wrote:
>> >> I am out of office on the 25th. Would the following Monday work?
>> >>
>> >> Thanks,
>> >> Alex
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> From: Juanjo Hierro <juanjose.hierro at telefonica.com>
>> >> To: Philipp Slusallek <philipp.slusallek at dfki.de>, Alex
>> >> Glikson/Haifa/IBM at IBMIL,
<fiware-chapter-architects at lists.fi-ware.org>
>> >> Cc: Ezra Silvera/Haifa/IBM at IBMIL, Kenneth
Nagin/Haifa/IBM at IBMIL
>> >> Date: 18/05/2015 11:53 AM
>> >> Subject: Re: [Fiware-chapter-architects] Linux Containers &
> Docker
>> >>
> ------------------------------------------------------------------------
>> >>
>> >>
>> >>
>> >>
>> >> I would suggest that we book the next architecture session (Monday
>> >> 25th) to cover this matter.
>> >>
>> >> Will you Alex be able to prepare a presentation?
>> >>
>> >> Cheers,
>> >>
>> >> -- Juanjo
>> >>
>> >> On 18/05/15 09:46, Philipp Slusallek wrote:
>> >> > Hi,
>> >> >
>> >> > This sounds very useful to cover as well in this or one of the
next
>> >> > calls. It would also get the architectural activities started
> again. It
>> >> > might even make sense to invite (selected?) AB members from the
UCs.
>> >> >
>> >> > For example, a lot of work that has been done within FIcontent to
> deploy
>> >> > our SEs and related GEs within Docker on top of FIWARE Lab.
> Particularly
>> >> > relevant could be the on-click deployment of entire arrangements
> of GEs
>> >> > and SEs. Better and more dedicated Docker support would certainly
be
>> >> > very welcome as well.
>> >> >
>> >> > Unfortunately, I have a lecture this morning and cannot join
> until 12h.
>> >> > But Stefan Lemme from my group will be on the call. He has been a
> core
>> >> > member of the relevant FIC2-Lab task force in FIcontent.
>> >> >
>> >> >
>> >> > Best,
>> >> >
>> >> > Philipp
>> >> >
>> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson:
>> >> >> Maybe we can also dedicate some time to have an initial
> discussion on
>> >> >> the roadmap to adopt Linux Containers and in particular Docker in
>> >> FIWARE
>> >> >> Lab.
>> >> >>
>> >> >> Regards,
>> >> >> Alex
>> >> >>_______________________________________________
> Fiware-cloud-containers mailing list
> Fiware-cloud-containers at lists.fiware.org
> https://lists.fiware.org/listinfo/fiware-cloud-containers
>
>
>
> _______________________________________________
> Fiware-cloud-containers mailing list
> Fiware-cloud-containers at lists.fiware.org
> https://lists.fiware.org/listinfo/fiware-cloud-containers
>
--
-------------------------------------------------------------------------
Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI) GmbH
Trippstadter Strasse 122, D-67663 Kaiserslautern
Geschäftsführung:
Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
Dr. Walter Olthoff
Vorsitzender des Aufsichtsrats:
Prof. Dr. h.c. Hans A. Aukes
Sitz der Gesellschaft: Kaiserslautern (HRB 2313)
USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3
---------------------------------------------------------------------------
[attachment "philipp_slusallek.vcf" deleted by Alex Glikson/Haifa/IBM]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/fiware-cloud-containers/attachments/20150601/419b3fd2/attachment.html>
You can get more information about our cookies and privacy policies clicking on the following links: Privacy policy Cookies policy