[Fiware-cloud-containers] task-force kickoff

[Alex Glikson]

Below is a brief summary of the current FIWARE Cloud status with regards 
to containers and Docker support.

FIWARE Cloud GEs currently deployed in FIWARE Lab provide VM-based IaaS 
capabilities with OpenStack Nova (with KVM), Glance, Keystone, Cinder and 
Neutron (with OVS). FIWARE Cloud also provides object storage capabilities 
based on OpenStack Swift. Moreover, we support PaaS-level capabilities, 
including the ability to provision and auto-scale complex applications 
comprising sets of inter-dependent VMs as well as the ability to install 
and configure the individual software components within VMs using 
configuration management tools such as Chef. The key user-visible notion 
behind this capability is the notion of a Blueprint, which defines the 
application topology, dependencies, auto-scaling rules, software 
configuration, etc (conceptually similar to Amazon OpsWorks). 
In the last few months, the PaaS implementation is undergoing a major 
transition from a 'proprietary' (although open source) code to full 
adoption of OpenStack Heat and Murano (as well as contribution of features 
to the Murano community, such as support for templates, configuration 
management enhancements, etc).
In parallel, we have started assessing (and prototyping) the options to 
support/leverage Linux containers technologies, and Docker in particular. 
We aim at two main use-cases:
As a FIWARE Cloud provider (in FIWARE Lab), I want to continue providing 
(almost?) the same PaaS/IaaS capabilities, but at the same time being able 
to admit much more users/workloads on the same hardware, by leveraging 
Containers technologies.
As a FIWARE Cloud user, I want to be able to leverage the Docker 
ecosystem, including public images available at Docker Hub as well as 
existing tools (TBD: which tools exactly?), while hosting my applications 
on the FIWARE Lab
The 2nd use-case can be satisfied in several ways, such as:
Treat Docker-based cluster as PaaS application (managed by FIWARE Cloud 
PaaS), while the cloud middleware (IaaS and PaaS) is not aware of 
individual containers, Docker images, etc.
Make Docker containers 1st class citizens in PaaS -- e.g., as an 
additional delivery mechanism in addition to Chef, deployed on regular 
IaaS virtualized infra (providing isolation, accounting, etc). In this 
case each Docker cluster is contained within a single tenant (and even 
single application within a tenant?)
Make Docker containers 1st class citizens in IaaS deployed on bare-metal, 
making PaaS use it via a standardized API
The current (tentative) plan is to apply a 2-phase approach to introduce 
containers: phase-1: start by deploying nova-docker in FIWARE lab in order 
to enable use-case n.1, and then phase-2: gradually migrate to OpenStack 
Magnum, while incorporating the Docker-native capabilities into Murano in 
order to support use-case n.2. Of course, the devil is in the details, so 
we need to discuss further, to understand the exact requirements and 
implications.

Regards,
Alex




From:   Alex Glikson/Haifa/IBM at IBMIL
To:     fiware-cloud-containers at lists.fiware.org
Date:   30/05/2015 07:06 PM
Subject:        [Fiware-cloud-containers] task-force kickoff
Sent by:        fiware-cloud-containers-bounces at lists.fiware.org



Dear all, 

Welcome to 'fiware-cloud-containers' mailing list! We will use this 
mailing list to discuss the details of FIWARE Cloud roadmap w.r.t. support 
for Linux containers (such as Docker), including use-cases & requirements, 
associated IaaS and PaaS-style capabilities, concrete plans going 
forwards, potential collaboration with teams working on similar/related 
technologies, etc. 

Currently we have the following people subscribed to this mailing list: 
IBM: Alex (FIWARE Cloud architect), Ezra (technical lead for 
IaaS/compute-related work in FIWARE Cloud), Kenneth (FIWARE Cloud leader), 
Doron (working with Ezra on containers) 
TID: Juanjo (FIWARE chief architect), Fernando & Henar (technical leads 
for the PaaS-related work in FIWARE Cloud) 
DFKI: Philipp (FI-Content2 architect, FIWARE Advanced WebUI chapter 
leader/architect) 
Thales: Mario (FI-Content2, also used to be in the Cloud WP in the old 
FI-WARE project) 

Let me know if anyone is missing. 

As a background, please, read the thread below, including the very nice 
summary by Mario on the related investigation and implementation in 
FI-Content2 project. I will send a separate email briefly summarizing the 
situation in FIWARE. 

All -- notice that we are going to have a phone call to discuss this topic 
during the upcoming FIWARE architects interlock on Monday, June 1at, 
10:30-12:30 CET. Let me know if you can't join. 
Mario -- would you be able to join & explain/demonstrate the FIC2Lab 
solution? 

Thanks, 
Alex 



====================================================================================
Alex Glikson
Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab
Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: 
+972-54-6466667 | Fax: +972-4-8296112


----- Forwarded by Alex Glikson/Haifa/IBM on 30/05/2015 06:36 PM ----- 

fiware-chapter-architects-bounces at lists.fi-ware.org wrote on 26/05/2015 
08:37:16 AM:

> From: Alex Glikson/Haifa/IBM at IBMIL 
> To: Philipp Slusallek <philipp.slusallek at dfki.de> 
> Cc: fiware-chapter-architects at lists.fi-ware.org, Kenneth Nagin/
> Haifa/IBM at IBMIL, Ezra Silvera/Haifa/IBM at IBMIL 
> Date: 26/05/2015 08:37 AM 
> Subject: Re: [Fiware-chapter-architects] Linux Containers & Docker 
> Sent by: fiware-chapter-architects-bounces at lists.fi-ware.org 
> 
> Hi Philipp, 
> 
> It is great to hear that FIC2 team is willing to contribute! 
> In a nutshell, our current plan is to start with surfacing 
> containers via Nova, and then gradually migrate to Magnum. In terms 
> of orchestration, we currently work with Heat and Murano, and are 
> assessing the approaches to support containers (OpenStack alignment 
> being high priority, but also the ability to surface capabilities 
> unique to containers). 
> I suggest that we form a small taskforce to explore this topic. 
> Please, send me a list of people to involve. 
> 
> Thanks, 
> Alex 
> 
> 
====================================================================================
> Alex Glikson
> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab
> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: 
> +972-54-6466667 | Fax: +972-4-8296112
> 
> 
> 
> 
> 
> From:        Philipp Slusallek <philipp.slusallek at dfki.de> 
> To:        Juanjo Hierro <juanjose.hierro at telefonica.com>, Alex 
> Glikson/Haifa/IBM at IBMIL 
> Cc:        Ezra Silvera/Haifa/IBM at IBMIL, fiware-chapter-
> architects at lists.fi-ware.org, Kenneth Nagin/Haifa/IBM at IBMIL 
> Date:        26/05/2015 07:21 AM 
> Subject:        Re: [Fiware-chapter-architects] Linux Containers & 
Docker 
> 
> 
> 
> Hi,
> 
> Here is some first input from FIC2 on Docker compiled from Mario. As I
> mentioned before, we have a full system working that deploys entire sets
> of GEs/SEs using this mechanism on FIWARE (and AWS) VMs.
> 
> FIC2 would be happy to work with FIWARE to integrate them in whatever
> way we at FIWARE decide to implement Docker. Right now we deploy it on
> top of the VMs offered by FIWARE but a more native implementation would
> be fine as well and save much resources (which seems to be a major 
issue).
> 
> A presentation by Canonical recently showed a >10x improvement in
> resource usage with their containers over plain OpenStack (which is
> probably where we are -- but then Intel claims even better numbers in
> their Clear Containers solution using traditional VMs).
> 
> Since we have experience already, it seems very useful to make Docker
> available especially since deployment of SW is much easier too and the
> same SW can be deployed essentially anywhere (including a local 
machine).
> 
> 
> Best,
> 
>                 Philipp
> 
> > There are several ways in which FIWARE could support Docker. My 
> assumption is that the plan is to add to OpenStack a Docker driver. 
> It would enable the creation of instances which are not VMs but 
> OpenStack containers, from images which are not ISOs but Docker 
> images. It?s described here: https://wiki.openstack.org/wiki/Docker
> > 
> > 
> > 
> > Advantages:
> > 
> > + Launches faster than VM instances
> > 
> > + Less resources used from quotas in FIWARE nodes
> > 
> > + images of SEs smaller and easier to build (using Dockerfiles)
> > 
> > 
> > 
> > Open questions / disadvantages:
> > 
> > - Will it be possible to pull images from DockerHub (preferred 
> approach), or do they need to be uploaded to the Glance image 
> repository of each FIWARE node?
> > 
> > - Is the only way to launch containers the Cloud Portal or 
> OpenStack API? Not being able to target a Docker API would reduce 
> the value of Docker and its ecosystem of tools, e.g.: docker-compose
> which enables launching applications composed of multiple containers.
> > 
> > - How to solve the limits in terms of IPs? Maybe with a DNS-based 
> reverse proxy that forwards HTTP requests to container1.john-
> smith.cloud.fiware.org to the right container port?
> > 
> > - How would links between containers running on different hosts 
> work? Using Weave?
> > 
> > - Mounting volumes in Docker is a very common way of passing 
> configuration files, having persistence, etc. How would this work 
inOpenStack?
> > 
> > 
> > 
> > Our suggestion:
> > 
> > * Ability to use the Docker API remotely (i.e. command-line tools 
> such as docker and docker-compose, Kitematic for a Mac OS X GUI)
> > 
> > * Ability to pull containers directly from DockerHub with a common
> (transparent?) image mirror in each FIWARE node
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > Regarding FIC2Lab runner, our wishlist was:
> > 
> > 1. ability to use the same tools locally and remotely in any cloud
> (ideally use docker-machine against any cloud provider, in 
> particular FIWARE nodes)
> > 
> > 2. nice web user interface hiding the complexity of firewalling, 
> security pairs, public IPs, etc.
> > 
> > 3. compatibility with the tools of the docker ecosystem: docker 
> CLI, compose, swarm ...
> > 
> > 
> > 
> > We compared tools such as tutum (free multi-tenant online 
> service), shipyard (open-source web UI) and panamax (open-source web
> UI and orchestrator).
> > 
> > And we decided to go for Panamax, which is nice for #1 and #2 but 
> not #3 so far. Here's a few pluses and minuses:
> > 
> > + nice UI with DockerHub search
> > 
> > + composite application, with application template search
> > 
> > - It's not multi-tenant, which is good for portability but bad for
> software updates
> > 
> > - its model for composite applications is different from docker-
> compose (for now)
> > 
> > - the UI can't synchronize with actions performed in Docker CLI or
> API because it's using CoreOS underneath
> > 
> > 
> > 
> > The FIC2Lab runner is documented here: http://fic2.github.io/runner
> > 
> > 
> 
> 
> 
> Am 20.05.2015 um 17:48 schrieb Juanjo Hierro:
> > Dear Alex,
> > 
> >   It's fine with me.
> > 
> >   I believe that we should collocate here the discussion with
> > FI-Content2 (Philipp) regarding the stuff they have developed for the
> > FIC2-Lab.
> > 
> >   @Philipp: could you share some material (document describing your 
work
> > on FIC2-Lab, slides, whatever) describing what you have done and send 
it
> > to the fiware-chapter-architects mailing list as preparation for the
> > discussion?
> > 
> >   @Alex: please also share the ideas you want to present on this 
matter.
> > 
> >   By exchanging material prior to the meeting, we would be able to 
have
> > a more fruitful meeting.
> > 
> >   Best regards,
> > 
> > -- Juanjo
> > 
> > On 18/05/15 11:07, Alex Glikson wrote:
> >> I am out of office on the 25th. Would the following Monday work?
> >>
> >> Thanks,
> >> Alex
> >>
> >>
> >>
> >>
> >>
> >>
> >> From:        Juanjo Hierro <juanjose.hierro at telefonica.com>
> >> To:        Philipp Slusallek <philipp.slusallek at dfki.de>, Alex
> >> Glikson/Haifa/IBM at IBMIL, 
<fiware-chapter-architects at lists.fi-ware.org>
> >> Cc:        Ezra Silvera/Haifa/IBM at IBMIL, Kenneth 
Nagin/Haifa/IBM at IBMIL
> >> Date:        18/05/2015 11:53 AM
> >> Subject:        Re: [Fiware-chapter-architects] Linux Containers & 
Docker
> >> 
------------------------------------------------------------------------
> >>
> >>
> >>
> >>
> >>   I would suggest that we book the next architecture session (Monday
> >> 25th) to cover this matter.
> >>
> >>   Will you Alex be able to prepare a presentation?
> >>
> >>   Cheers,
> >>
> >> -- Juanjo
> >>
> >> On 18/05/15 09:46, Philipp Slusallek wrote:
> >> > Hi,
> >> >
> >> > This sounds very useful to cover as well in this or one of the next
> >> > calls. It would also get the architectural activities started 
again. It
> >> > might even make sense to invite (selected?) AB members from the 
UCs.
> >> >
> >> > For example, a lot of work that has been done within FIcontent to 
deploy
> >> > our SEs and related GEs within Docker on top of FIWARE Lab. 
Particularly
> >> > relevant could be the on-click deployment of entire arrangements of 
GEs
> >> > and SEs. Better and more dedicated Docker support would certainly 
be
> >> > very welcome as well.
> >> >
> >> > Unfortunately, I have a lecture this morning and cannot join until 
12h.
> >> > But Stefan Lemme from my group will be on the call. He has been a 
core
> >> > member of the relevant FIC2-Lab task force in FIcontent.
> >> >
> >> >
> >> > Best,
> >> >
> >> >       Philipp
> >> >
> >> > Am 18.05.2015 um 09:28 schrieb Alex Glikson:
> >> >> Maybe we can also dedicate some time to have an initial discussion 
on
> >> >> the roadmap to adopt Linux Containers and in particular Docker in
> >> FIWARE
> >> >> Lab.
> >> >>
> >> >> Regards,
> >> >> Alex
> >> >>_______________________________________________
Fiware-cloud-containers mailing list
Fiware-cloud-containers at lists.fiware.org
https://lists.fiware.org/listinfo/fiware-cloud-containers

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/fiware-cloud-containers/attachments/20150531/d7c3ace0/attachment.html>