[Fiware-cloud-containers] FIWARE developer experience with Docker

[Alex Glikson]

Dear partners,

Following the initial discussion at the containers task force, we 
identified two follow-on (related) discussion tracks:
1) expected FIWARE Lab user/developer experience with Docker
2) Enhancements to FIWARE Cloud Hosting architecture to support Docker 
(and enable requirements derived from #1)

This email refers to topic (1).

I will try to summarize the initial set of usage scenarios that we may 
want to support. We can then iterate on this over email, and have a phone 
call if needed, when we feel that we are close to a convergence point.

Please, notice that I've subscribed several additional people relevant for 
the discussion (who could provide input related to UI, developers 
perspectives and operations perspectives).

Assumptions:
Notice that a prerequisite for all the usage scenarios is that all the 
FIWARE GEs (and SEs) are packages as Docker images and are kept in a 
central repository, preferably the docker hub (under a 'to-be-created' 
"fiware" namespace). Also, there is an assumption that the 
users/developers would want to work with Docker tools as much as possible 
(surfacing some or all of the capabilities via the FIWARE Cloud Portal 
too). Juanjo will elaborate on the approach we are thinking of to promote 
this with GE/SE owners and the broader developer community.

Group 1: Basic Docker environment setup
Usage scenario #1.1:
A user wants a local Docker runtime on his laptop.
He follows the standard instructions on setting up a docker host (e.g., 
running within a VirtalBox VM, using docker-machine / boot2docker)

Usage scenario #1.2:
A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab 
(where he would then run various Docker containers).
He uses the standard docker-machine tool, specifying the URI of the 
OpenStack Keystone in FIWARE Lab (and additional parameters, as needed). 
The tool creates a VM using standard OpenStack APIs (natively supported by 
FIWARE Lab) and configures Docker within the VM. The VM would need to have 
a public IP (naturally).

Group 2: Basic life cycle of individual containers running GEs/SEs
Usage scenario #2.1:
A developer wants to publish (a version of) a GE/SE.
After he is done creating the new Docker image, he pushes the new version 
of the GE/SE to Docker hub under corresponding FIWARE namespace (e.g., 
fiware/GE/cb-orion). Now the 'latest' version of the image points to the 
new version.

Usage scenario #2.2:
A user/developer wants to deploy locally an instance of a certain GE/SE.
He uses the standard docker CLI to locally provision a container, 
referring to the corresponding image at Docker hub -- e.g.: "$ docker -H 
boot2docker-vm:2376 run fiware/GE/cb-orion"

Usage scenario #2.3:
A user/developer wants to deploy an instance of a certain GE/SE within his 
Docker VM on FIWARE Lab
He uses the standard docker CLI to provision a container, referring to the 
location of his Docker VM as well as the corresponding image at Docker hub 
-- e.g.: "$ docker -H mydocker-vm37.lab.fiware.org:2376 run 
fiware/GE/cb-orion". ISSUE: the user would need to open the corresponding 
firewall ports in his VM (same as those of the GE/SE, or following the 
mapping performed during container provisioning) in order to make the 
GE/SE accessible. Ideally, this should be done in a scalable but secure 
manner. An easy solution is to open up-front (during VM provisioning) a 
range of ports (via setting up corresponding security group). A more 
advanced solution is to update the security group dynamically.

Usage scenario #2.4:
A user/developer wants to update a container comprising certain GE/SE with 
the latest version recently published in the dedicated namespace of the 
Docker Hub.
He pulls the latest version from the Docker Hub, kills the old container, 
and starts a new one (attaching to the same resources). Note that this 
would work well when the application is properly designed for Docker 
(e.g., the container itself is stateless), and when the previous version 
of the container has been provisioned manually (as in #2, #4 above) by the 
developer (and he knows which resources to connect to).

Group 3: Support for 'bundles' of GEs/SEs that together perform a certain 
complex function
Usage scenario #3.1:
A developer wants to publish (a version of) a 'bundle' of GEs/SEs that 
together perform a certain complex function.
He creates a 'template' (e.g., following the format of docker-compose), 
referring to the individual GE/SE images as well as their 
interdependencies (e.g., links) and other composition properties. This 
might be done using a text editor, a Web UI provided by the FIWARE Cloud 
(conceptually similar to today's UI for creation of blueprints), or other 
tools from Docker ecosystem. He then uploads the template to the 
centralized templats repository. Note: it is likely that docker-compose 
will be able to use Docker Registry/Hub as a repository for templates 
(including versioning, push/pull, etc).

Usage scenario #3.2:
A user wants to provision locally a set of GEs/SEs, using a pre-defined 
template (comprising a 'bundle')
He uses the standard docker-compose tool referring to the 'template' 
artifcat as well as the local Docker URL.

Usage scenario #3.3:
A user wants to provision a set of GEs/SEs in FIWARE Lab, using a 
pre-defined template (comprising a 'bundle')
He uses the standard docker-compose tool referring to the 'template' 
artifcat as well as the target Docker URL in FIWARE Lab. Alternatively, he 
uses the FIWARE Portal UI to do the same (e.g., with Murano backend 
invoking docker-compose).

Usage scenario #3.4:
A user wants to update his Docker environment (local or on FIWARE Lab) 
with the latest version of a certain 'bundle'
Note: may require enancements to docker-compose

Group 4: Advanced scenarios
Usage scenario #4.1:
A user/developer wants to provision a cluster of VMs on FIWARE Lab that 
would host his Docker cluster (managed with Swarm or Kubernetes)
He uses corresponding Murano/Heat template to provision the VMs and to 
configure the Docker/Swarm/Kubernetes cluster.

Usage scenario #4.2:
A user/developer wants to access a global instance of a Docker service in 
FIWARE Lab (shared, scalable, managed), so that he doesn't need to manage 
the corresponding VM(s) by himself.
He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab 
Docker API endpoint with the standard Docker tools or FIWARE-specific 
tools (as outlined above ).

Usage scenario #4.3:
A user/developer wants to manage access control for Docker images among 
FIWARE Lab users.
He starts using the Docker Registry/Hub deployed within the FIWARE Lab.


Notice that this is a very initial list -- I am sure that there are many 
inaccuracies and gaps. Feel free to comment.

Thanks,
Alex

====================================================================================
Alex Glikson
Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab
Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: 
+972-54-6466667 | Fax: +972-4-8296112

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/fiware-cloud-containers/attachments/20150630/9c5be52b/attachment.html>