From NAGIN at il.ibm.com Tue Jul 7 15:36:18 2015 From: NAGIN at il.ibm.com (Kenneth Nagin) Date: Tue, 7 Jul 2015 16:36:18 +0300 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: <5592F69E.4060803@telefonica.com> References: <5592F69E.4060803@telefonica.com> Message-ID: The Cloud chapter has already begun focusing on providing Docker support. IBM is preparing a study of how FIWARE developers can leverage the current FIWARE lab to host Docker while using Docker ecosystem tools locally to develop and deploy their applications. We will verify 1) Docker-Engine and Docker-Machine for setting up their remote hosting environment, 2) Docker-Compose for constructing and running multi-container applications hosted on FIWARE, 3) Docker-Swarm for host clustering and container scheduling. (Issue CLD-574). The output of the study will be to describe the capabilities, limitations and gaps of hosting Docker on FIWARE. We presented preliminary results at the sprint closing chapter review (Monday July 6). IBM documented the required set up by a FIWARE developer for hosting Docker on FIWARE. This documentation is available at http://www.slideshare.net/knagin/simple-docker-hosting-on-fiware-lab. (Issue CLD-576). This is basically the outputs of <1> and can be contributed to item <2> in Juano's attached note. We are currently studying the limitation of this environment and a roadmap to address these limitations . We will share it with the task force once it is completed. TID is preparing a demonstration of Murano Docker Support (Issues CLD-560, CLD-568 ). This would allow a FIWARE developer to deploy Docker containers using the Murano API. Henar will demonstrate deploying docker with Murano at the sprint closing chapter review (Monday July 6). UPM is estimating the work effort to expose the Murano to the user on the cloud portal. This involves adapting the current blue print views to the Murano API (Issue CLD-584). Best Regards, Kenneth Nagin Ph: +972-4-8296227 Cell: 054-6976227 Fx: +972-4- 8296114 http://researcher.ibm.com/view.php?person=il-NAGIN From: Juanjo Hierro To: Alex Glikson/Haifa/IBM at IBMIL, Date: 30/06/2015 11:06 PM Subject: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Sent by: fiware-cloud-containers-bounces at lists.fiware.org Dear all, Thanks Alex for launching these discussions tracks. Regarding point (1) as anticipated by Alex, here it is the concrete plan we aim at implementing to promote usage of docker tools by GE/SE owners and the broader developer community. 1. Make info about Docker images and instruction to setup docker containers linked to FIWARE GEris available in the FIWARE Catalogue: We will specify a general template of a new section which will be included in the "Creating instances" tab of entries linked to FIWARE GEris. This section will be elaborate on "Deploying a dedicated GE instance using Docker technology" Each FIWARE GEri owner will be asked to setup a Docker image for their GEs and register it in Docker Hub. This task can start in parallel to the previous one. Each FIWARE GEri owner will be asked to update the "Creating instance" tab of the entry linked to the FIWARE GEri so that it provides instructions about deploying a dedicated instance of the GEri using docker. For this purpose, the owner will follow the template defined in step 1. 2. We will incorporate the description about how to create FIWARE GEri instances and try them using docker within the FIWARE Tour Guide for developers (which will effectively become the landing page of http://developers.fiware.org) as a mean to shorten the learning curve with FIWARE GEris Somewhere at the beginning of the guided tour, we will explain the developer how he can setup the basic docker environment either locally or on the FIWARE Lab on which he will be able to rely to try the different FIWARE GEris (Group 1 scenarios described by Alex) A demo application will be developed that will help the developer to try each FIWARE GEri instance deployed using docker with concrete data. Developers will be able to deploy this application also using docker. It is the intention that this application will be helpful to show how several FIWARE GEris can be used in an integrated way. Each of the chapters of the FIWARE Tour guide for developers will integrate a "Try it yourself" section which will help the developer to create an instance of the FIWARE GEris referred in the chapter and try them. In addition, developers will be able to deploy the demo application, in order to learn from a more elaborated example how the FIWARE GEris can be used, and providing instructions about how to play with the FIWARE GEris, also "tweak" the application. Deployment of integrated FIWARE GEris and the demo application will be made feasible using docker composition tools Jos?-Manuel Cantera will drive the implementation of these concrete actions. Regarding point 2, Bitergia has already developed a first bundle of FIWARE GEris deployable using Docker compose (as a whole or individually). They will work in extending this bundle integrating additional FIWARE GEris in collaboration with the corresponding FIWARE GEri owners. Bitergia and the ULPG will work together in the development of a first version of the demo application. Eventually, additional partners might be incorporated as additional skills and/or resources are needed. We will address first those scenarios dealing with local deployment, then we will incorporate those related to deployment on other infrastructures such as the FIWARE Lab as soon as the necessary preliminary work and/or detailed guidelines are completed (this would be done under coordination by Alex and I assume we will leverage on experience developed in FI-Content2). Feedback is welcome. Best regards, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 30/06/15 20:34, Alex Glikson wrote: Dear partners, Following the initial discussion at the containers task force, we identified two follow-on (related) discussion tracks: 1) expected FIWARE Lab user/developer experience with Docker 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker (and enable requirements derived from #1) This email refers to topic (1). I will try to summarize the initial set of usage scenarios that we may want to support. We can then iterate on this over email, and have a phone call if needed, when we feel that we are close to a convergence point. Please, notice that I've subscribed several additional people relevant for the discussion (who could provide input related to UI, developers perspectives and operations perspectives). Assumptions: Notice that a prerequisite for all the usage scenarios is that all the FIWARE GEs (and SEs) are packages as Docker images and are kept in a central repository, preferably the docker hub (under a 'to-be-created' "fiware" namespace). Also, there is an assumption that the users/developers would want to work with Docker tools as much as possible (surfacing some or all of the capabilities via the FIWARE Cloud Portal too). Juanjo will elaborate on the approach we are thinking of to promote this with GE/SE owners and the broader developer community. Group 1: Basic Docker environment setup Usage scenario #1.1: A user wants a local Docker runtime on his laptop. He follows the standard instructions on setting up a docker host (e.g., running within a VirtalBox VM, using docker-machine / boot2docker) Usage scenario #1.2: A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab (where he would then run various Docker containers). He uses the standard docker-machine tool, specifying the URI of the OpenStack Keystone in FIWARE Lab (and additional parameters, as needed). The tool creates a VM using standard OpenStack APIs (natively supported by FIWARE Lab) and configures Docker within the VM. The VM would need to have a public IP (naturally). Group 2: Basic life cycle of individual containers running GEs/SEs Usage scenario #2.1: A developer wants to publish (a version of) a GE/SE. After he is done creating the new Docker image, he pushes the new version of the GE/SE to Docker hub under corresponding FIWARE namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the image points to the new version. Usage scenario #2.2: A user/developer wants to deploy locally an instance of a certain GE/SE. He uses the standard docker CLI to locally provision a container, referring to the corresponding image at Docker hub -- e.g.: "$ docker -H boot2docker-vm:2376 run fiware/GE/cb-orion" Usage scenario #2.3: A user/developer wants to deploy an instance of a certain GE/SE within his Docker VM on FIWARE Lab He uses the standard docker CLI to provision a container, referring to the location of his Docker VM as well as the corresponding image at Docker hub -- e.g.: "$ docker -H mydocker-vm37.lab.fiware.org:2376 run fiware/GE/cb-orion". ISSUE: the user would need to open the corresponding firewall ports in his VM (same as those of the GE/SE, or following the mapping performed during container provisioning) in order to make the GE/SE accessible. Ideally, this should be done in a scalable but secure manner. An easy solution is to open up-front (during VM provisioning) a range of ports (via setting up corresponding security group). A more advanced solution is to update the security group dynamically. Usage scenario #2.4: A user/developer wants to update a container comprising certain GE/SE with the latest version recently published in the dedicated namespace of the Docker Hub. He pulls the latest version from the Docker Hub, kills the old container, and starts a new one (attaching to the same resources). Note that this would work well when the application is properly designed for Docker (e.g., the container itself is stateless), and when the previous version of the container has been provisioned manually (as in #2, #4 above) by the developer (and he knows which resources to connect to). Group 3: Support for 'bundles' of GEs/SEs that together perform a certain complex function Usage scenario #3.1: A developer wants to publish (a version of) a 'bundle' of GEs/SEs that together perform a certain complex function. He creates a 'template' (e.g., following the format of docker-compose), referring to the individual GE/SE images as well as their interdependencies (e.g., links) and other composition properties. This might be done using a text editor, a Web UI provided by the FIWARE Cloud (conceptually similar to today's UI for creation of blueprints), or other tools from Docker ecosystem. He then uploads the template to the centralized templats repository. Note: it is likely that docker-compose will be able to use Docker Registry/Hub as a repository for templates (including versioning, push/pull, etc). Usage scenario #3.2: A user wants to provision locally a set of GEs/SEs, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the local Docker URL. Usage scenario #3.3: A user wants to provision a set of GEs/SEs in FIWARE Lab, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the target Docker URL in FIWARE Lab. Alternatively, he uses the FIWARE Portal UI to do the same (e.g., with Murano backend invoking docker-compose). Usage scenario #3.4: A user wants to update his Docker environment (local or on FIWARE Lab) with the latest version of a certain 'bundle' Note: may require enancements to docker-compose Group 4: Advanced scenarios Usage scenario #4.1: A user/developer wants to provision a cluster of VMs on FIWARE Lab that would host his Docker cluster (managed with Swarm or Kubernetes) He uses corresponding Murano/Heat template to provision the VMs and to configure the Docker/Swarm/Kubernetes cluster. Usage scenario #4.2: A user/developer wants to access a global instance of a Docker service in FIWARE Lab (shared, scalable, managed), so that he doesn't need to manage the corresponding VM(s) by himself. He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab Docker API endpoint with the standard Docker tools or FIWARE-specific tools (as outlined above ). Usage scenario #4.3: A user/developer wants to manage access control for Docker images among FIWARE Lab users. He starts using the Docker Registry/Hub deployed within the FIWARE Lab. Notice that this is a very initial list -- I am sure that there are many inaccuracies and gaps. Feel free to comment. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers -------------- next part -------------- An HTML attachment was scrubbed... URL: From federico.facca at create-net.org Tue Jul 7 15:57:17 2015 From: federico.facca at create-net.org (Federico Michele Facca) Date: Tue, 7 Jul 2015 15:57:17 +0200 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: References: <5592F69E.4060803@telefonica.com> Message-ID: hi guys, i think the introduction of docker is a very interesting evolution of fiware offer. my two cents (following also some remarks from the commission) is that whatever is done needs to take into consideration the user experience and making life easier for the developer. - most of the complexity (unless the developer is really interested into getting into it) should be hidden to him (maybe this simply means more pre-cooked food / recipies for him in the blueprint/murano engine); - maybe we can learn from paas that are successful: can we have a docker environment for 2-3 development platforms/frameworks (django, ruby-on-rails) and simplify allow developers to deploy their code on it from a github repository? more things are possible (natting like in cloudfounfry and automatic dns registration) that may improve the experience, i am just brainstorming :) some of the things can be pushed as well to WP2.1. hope it helps! fede PS: do we really need 1 docker vm for 1 user? (or actually do we need VMs?) c.f. Usage scenario #2.3, is this related to current limits of neutron w.r.t. docker-scheduler in OpenStack? On Tue, Jul 7, 2015 at 3:36 PM, Kenneth Nagin wrote: > The Cloud chapter has already begun focusing on providing Docker support. > > 1. IBM is preparing a study of how FIWARE developers can leverage the > current FIWARE lab to host Docker while using Docker ecosystem tools > locally to develop and deploy their applications. We will verify 1) > Docker-Engine and Docker-Machine for setting up their remote hosting > environment, 2) Docker-Compose for constructing and running > multi-container applications hosted on FIWARE, 3) Docker-Swarm for host > clustering and container scheduling. (Issue CLD-574). The output of the > study will be to describe the capabilities, limitations and gaps of hosting > Docker on FIWARE. We presented preliminary results at the sprint closing > chapter review (Monday July 6). > 2. IBM documented the required set up by a FIWARE developer for > hosting Docker on FIWARE. This documentation is available at > http://www.slideshare.net/knagin/simple-docker-hosting-on-fiware-lab. > (Issue CLD-576). This is basically the outputs of <1> and can be > contributed to item <2> in Juano's attached note. We are currently > studying the limitation of this environment and a roadmap to address these > limitations . We will share it with the task force once it is completed. > 3. TID is preparing a demonstration of Murano Docker Support (Issues > CLD-560, CLD-568 ). This would allow a FIWARE developer to deploy Docker > containers using the Murano API. Henar will demonstrate deploying docker > with Murano at the sprint closing chapter review (Monday July 6). > 4. UPM is estimating the work effort to expose the Murano to the user > on the cloud portal. This involves adapting the current blue print views to > the Murano API (Issue CLD-584). > > > Best Regards, > > Kenneth Nagin > Ph: +972-4-8296227 > Cell: 054-6976227 > Fx: +972-4- 8296114 > http://researcher.ibm.com/view.php?person=il-NAGIN > > > > > > > From: Juanjo Hierro > To: Alex Glikson/Haifa/IBM at IBMIL, < > fiware-cloud-containers at lists.fiware.org> > Date: 30/06/2015 11:06 PM > Subject: Re: [Fiware-cloud-containers] FIWARE developer experience > with Docker > Sent by: fiware-cloud-containers-bounces at lists.fiware.org > ------------------------------ > > > > Dear all, > > Thanks Alex for launching these discussions tracks. > > Regarding point (1) as anticipated by Alex, here it is the concrete plan > we aim at implementing to promote usage of docker tools by GE/SE owners and > the broader developer community. > 1. Make info about Docker images and instruction to setup docker > containers linked to FIWARE GEris available in the FIWARE Catalogue: > > - We will specify a general template of a new section which will be > included in the "Creating instances" tab of entries linked to FIWARE GEris. > This section will be elaborate on "Deploying a dedicated GE instance > using Docker technology" > - Each FIWARE GEri owner will be asked to setup a Docker image for > their GEs and register it in Docker Hub. This task can start in parallel > to the previous one. > - Each FIWARE GEri owner will be asked to update the "Creating > instance" tab of the entry linked to the FIWARE GEri so that it provides > instructions about deploying a dedicated instance of the GEri using docker. > For this purpose, the owner will follow the template defined in step 1. > > 2. We will incorporate the description about how to create FIWARE > GEri instances and try them using docker within the FIWARE Tour Guide for > developers (which will effectively become the landing page of > *http://developers.fiware.org* ) as a mean > to shorten the learning curve with FIWARE GEris > > - Somewhere at the beginning of the guided tour, we will explain the > developer how he can setup the basic docker environment either locally or > on the FIWARE Lab on which he will be able to rely to try the different > FIWARE GEris (Group 1 scenarios described by Alex) > - A demo application will be developed that will help the developer to > try each FIWARE GEri instance deployed using docker with concrete data. > Developers will be able to deploy this application also using docker. It > is the intention that this application will be helpful to show how several > FIWARE GEris can be used in an integrated way. > - Each of the chapters of the FIWARE Tour guide for developers will > integrate a "Try it yourself" section which will help the developer to > create an instance of the FIWARE GEris referred in the chapter and try > them. In addition, developers will be able to deploy the demo > application, in order to learn from a more elaborated example how the > FIWARE GEris can be used, and providing instructions about how to play with > the FIWARE GEris, also "tweak" the application. > - Deployment of integrated FIWARE GEris and the demo application will > be made feasible using docker composition tools > > > Jos?-Manuel Cantera will drive the implementation of these concrete > actions. > > Regarding point 2, Bitergia has already developed a first bundle of > FIWARE GEris deployable using Docker compose (as a whole or individually). > They will work in extending this bundle integrating additional FIWARE > GEris in collaboration with the corresponding FIWARE GEri owners. > Bitergia and the ULPG will work together in the development of a first > version of the demo application. Eventually, additional partners might be > incorporated as additional skills and/or resources are needed. We will > address first those scenarios dealing with local deployment, then we will > incorporate those related to deployment on other infrastructures such as > the FIWARE Lab as soon as the necessary preliminary work and/or detailed > guidelines are completed (this would be done under coordination by Alex and > I assume we will leverage on experience developed in FI-Content2). > > Feedback is welcome. > > Best regards, > > -- Juanjo > ______________________________________________________ > > Coordinator and Chief Architect, FIWARE platform > CTO Industrial IoT, Telef?nica > > email: *juanjose.hierro at telefonica.com* > twitter: @JuanjoHierro > > You can follow FIWARE at: > website: *http://www.fiware.org* > twitter: @FIWARE > facebook: *http://www.facebook.com/pages/FI-WARE/251366491587242* > > linkedIn: *http://www.linkedin.com/groups/FIWARE-4239932* > > > On 30/06/15 20:34, Alex Glikson wrote: > Dear partners, > > Following the initial discussion at the containers task force, we > identified two follow-on (related) discussion tracks: > 1) expected FIWARE Lab user/developer experience with Docker > 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker > (and enable requirements derived from #1) > > This email refers to topic (1). > > I will try to summarize the initial set of usage scenarios that we may > want to support. We can then iterate on this over email, and have a phone > call if needed, when we feel that we are close to a convergence point. > > Please, notice that I've subscribed several additional people relevant for > the discussion (who could provide input related to UI, developers > perspectives and operations perspectives). > > * Assumptions*: > Notice that a prerequisite for all the usage scenarios is that all the > FIWARE GEs (and SEs) are packages as Docker images and are kept in a > central repository, preferably the docker hub (under a 'to-be-created' > "fiware" namespace). Also, there is an assumption that the users/developers > would want to work with Docker tools as much as possible (surfacing some or > all of the capabilities via the FIWARE Cloud Portal too). Juanjo will > elaborate on the approach we are thinking of to promote this with GE/SE > owners and the broader developer community. > > * Group 1: Basic Docker environment setup* > Usage scenario #1.1: > A user wants a local Docker runtime on his laptop. > He follows the standard instructions on setting up a docker host (e.g., > running within a VirtalBox VM, using docker-machine / boot2docker) > > Usage scenario #1.2: > A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab > (where he would then run various Docker containers). > He uses the standard docker-machine tool, specifying the URI of the > OpenStack Keystone in FIWARE Lab (and additional parameters, as needed). > The tool creates a VM using standard OpenStack APIs (natively supported by > FIWARE Lab) and configures Docker within the VM. The VM would need to have > a public IP (naturally). > > * Group 2: Basic life cycle of individual containers running GEs/SEs* > Usage scenario #2.1: > A developer wants to publish (a version of) a GE/SE. > After he is done creating the new Docker image, he pushes the new version > of the GE/SE to Docker hub under corresponding FIWARE namespace (e.g., > fiware/GE/cb-orion). Now the 'latest' version of the image points to the > new version. > > Usage scenario #2.2: > A user/developer wants to deploy locally an instance of a certain GE/SE. > He uses the standard docker CLI to locally provision a container, > referring to the corresponding image at Docker hub -- e.g.: *"$ docker -H > boot2docker-vm:2376 run fiware/GE/cb-orion"* > > Usage scenario #2.3: > A user/developer wants to deploy an instance of a certain GE/SE within his > Docker VM on FIWARE Lab > He uses the standard docker CLI to provision a container, referring to the > location of his Docker VM as well as the corresponding image at Docker hub > -- e.g.: *"$ docker -H mydocker-vm37.lab.fiware.org:2376 > run fiware/GE/cb-orion". *ISSUE: > the user would need to open the corresponding firewall ports in his VM > (same as those of the GE/SE, or following the mapping performed during > container provisioning) in order to make the GE/SE accessible. Ideally, > this should be done in a scalable but secure manner. An easy solution is to > open up-front (during VM provisioning) a range of ports (via setting up > corresponding security group). A more advanced solution is to update the > security group dynamically. > > Usage scenario #2.4: > A user/developer wants to update a container comprising certain GE/SE with > the latest version recently published in the dedicated namespace of the > Docker Hub. > He pulls the latest version from the Docker Hub, kills the old container, > and starts a new one (attaching to the same resources). Note that this > would work well when the application is properly designed for Docker (e.g., > the container itself is stateless), and when the previous version of the > container has been provisioned manually (as in #2, #4 above) by the > developer (and he knows which resources to connect to). > > * Group 3: Support for 'bundles' of GEs/SEs that together perform a > certain complex function* > Usage scenario #3.1: > A developer wants to publish (a version of) a 'bundle' of GEs/SEs that > together perform a certain complex function. > He creates a 'template' (e.g., following the format of docker-compose), > referring to the individual GE/SE images as well as their interdependencies > (e.g., links) and other composition properties. This might be done using a > text editor, a Web UI provided by the FIWARE Cloud (conceptually similar to > today's UI for creation of blueprints), or other tools from Docker > ecosystem. He then uploads the template to the centralized templats > repository. Note: it is likely that docker-compose will be able to use > Docker Registry/Hub as a repository for templates (including versioning, > push/pull, etc). > > Usage scenario #3.2: > A user wants to provision locally a set of GEs/SEs, using a pre-defined > template (comprising a 'bundle') > He uses the standard docker-compose tool referring to the 'template' > artifcat as well as the local Docker URL. > > Usage scenario #3.3: > A user wants to provision a set of GEs/SEs in FIWARE Lab, using a > pre-defined template (comprising a 'bundle') > He uses the standard docker-compose tool referring to the 'template' > artifcat as well as the target Docker URL in FIWARE Lab. Alternatively, he > uses the FIWARE Portal UI to do the same (e.g., with Murano backend > invoking docker-compose). > > Usage scenario #3.4: > A user wants to update his Docker environment (local or on FIWARE Lab) > with the latest version of a certain 'bundle' > Note: may require enancements to docker-compose > > * Group 4: Advanced scenarios* > Usage scenario #4.1: > A user/developer wants to provision a cluster of VMs on FIWARE Lab that > would host his Docker cluster (managed with Swarm or Kubernetes) > He uses corresponding Murano/Heat template to provision the VMs and to > configure the Docker/Swarm/Kubernetes cluster. > > Usage scenario #4.2: > A user/developer wants to access a global instance of a Docker service in > FIWARE Lab (shared, scalable, managed), so that he doesn't need to manage > the corresponding VM(s) by himself. > He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab > Docker API endpoint with the standard Docker tools or FIWARE-specific tools > (as outlined above ). > > Usage scenario #4.3: > A user/developer wants to manage access control for Docker images among > FIWARE Lab users. > He starts using the Docker Registry/Hub deployed within the FIWARE Lab. > > > Notice that this is a very initial list -- I am sure that there are many > inaccuracies and gaps. Feel free to comment. > > Thanks, > Alex > > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: *glikson at il.ibm.com* | Phone: +972-4-8281085 > | Mobile: +972-54-6466667 | Fax: +972-4-8296112 > > > > _______________________________________________ > Fiware-cloud-containers mailing list > *Fiware-cloud-containers at lists.fiware.org* > > *https://lists.fiware.org/listinfo/fiware-cloud-containers* > > > > > ------------------------------ > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener informaci?n privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilizaci?n, > divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de > la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos > que nos lo comunique inmediatamente por esta misma v?a y proceda a su > destrucci?n. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution or > copying of this communication is strictly prohibited. If you have received > this transmission in error, do not read it. Please immediately reply to the > sender that you have received this communication in error and then delete > it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, > pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo > da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio > indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou > c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. > Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique > imediatamente por esta mesma via e proceda a sua destrui??o > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > > -- -- Future Internet is closer than you think! http://www.fiware.org Official Mirantis partner for OpenStack Training https://www.create-net.org/community/openstack-training -- Dr. Federico M. Facca CREATE-NET Via alla Cascata 56/D 38123 Povo Trento (Italy) P +39 0461 312471 M +39 334 6049758 E federico.facca at create-net.org T @chicco785 W www.create-net.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From philipp.slusallek at dfki.de Wed Jul 8 07:46:22 2015 From: philipp.slusallek at dfki.de (Philipp Slusallek) Date: Wed, 08 Jul 2015 07:46:22 +0200 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: <5592F69E.4060803@telefonica.com> References: <5592F69E.4060803@telefonica.com> Message-ID: <559CB92E.3070002@dfki.de> Hi all, This all sounds good! But what happened to your planned next meetings. FIC2 is still highly interested in working with FIWARE to deploy the infrastructure developed there (running on FIWARE Lab) also for FIWARE. I thought the idea was to explore these options in a joint next call. Best, Philipp Am 30.06.2015 um 22:05 schrieb Juanjo Hierro: > Dear all, > > Thanks Alex for launching these discussions tracks. > > Regarding point (1) as anticipated by Alex, here it is the concrete > plan we aim at implementing to promote usage of docker tools by GE/SE > owners and the broader developer community. > > 1. Make info about Docker images and instruction to setup docker > containers linked to FIWARE GEris available in the FIWARE Catalogue: > * We will specify a general template of a new section which will > be included in the "Creating instances" tab of entries linked to > FIWARE GEris. This section will be elaborate on "Deploying a > dedicated GE instance using Docker technology" > * Each FIWARE GEri owner will be asked to setup a Docker image for > their GEs and register it in Docker Hub. This task can start > in parallel to the previous one. > * Each FIWARE GEri owner will be asked to update the "Creating > instance" tab of the entry linked to the FIWARE GEri so that it > provides instructions about deploying a dedicated instance of > the GEri using docker. For this purpose, the owner will follow > the template defined in step 1. > 2. We will incorporate the description about how to create FIWARE GEri > instances and try them using docker within the FIWARE Tour Guide for > developers (which will effectively become the landing page of > http://developers.fiware.org) as a mean to shorten the learning > curve with FIWARE GEris > * Somewhere at the beginning of the guided tour, we will explain > the developer how he can setup the basic docker environment > either locally or on the FIWARE Lab on which he will be able to > rely to try the different FIWARE GEris (Group 1 scenarios > described by Alex) > * A demo application will be developed that will help the > developer to try each FIWARE GEri instance deployed using docker > with concrete data. Developers will be able to deploy this > application also using docker. It is the intention that this > application will be helpful to show how several FIWARE GEris can > be used in an integrated way. > * Each of the chapters of the FIWARE Tour guide for developers > will integrate a "Try it yourself" section which will help the > developer to create an instance of the FIWARE GEris referred in > the chapter and try them. In addition, developers will be able > to deploy the demo application, in order to learn from a more > elaborated example how the FIWARE GEris can be used, and > providing instructions about how to play with the FIWARE GEris, > also "tweak" the application. > * Deployment of integrated FIWARE GEris and the demo application > will be made feasible using docker composition tools > > > Jos?-Manuel Cantera will drive the implementation of these concrete > actions. > > Regarding point 2, Bitergia has already developed a first bundle of > FIWARE GEris deployable using Docker compose (as a whole or > individually). They will work in extending this bundle integrating > additional FIWARE GEris in collaboration with the corresponding FIWARE > GEri owners. Bitergia and the ULPG will work together in the > development of a first version of the demo application. Eventually, > additional partners might be incorporated as additional skills and/or > resources are needed. We will address first those scenarios dealing > with local deployment, then we will incorporate those related to > deployment on other infrastructures such as the FIWARE Lab as soon as > the necessary preliminary work and/or detailed guidelines are completed > (this would be done under coordination by Alex and I assume we will > leverage on experience developed in FI-Content2). > > Feedback is welcome. > > Best regards, > > -- Juanjo > > ______________________________________________________ > > Coordinator and Chief Architect, FIWARE platform > CTO Industrial IoT, Telef?nica > > email: juanjose.hierro at telefonica.com > twitter: @JuanjoHierro > > You can follow FIWARE at: > website: http://www.fiware.org > twitter: @FIWARE > facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 > linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 > > On 30/06/15 20:34, Alex Glikson wrote: >> Dear partners, >> >> Following the initial discussion at the containers task force, we >> identified two follow-on (related) discussion tracks: >> 1) expected FIWARE Lab user/developer experience with Docker >> 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker >> (and enable requirements derived from #1) >> >> This email refers to topic (1). >> >> I will try to summarize the initial set of usage scenarios that we may >> want to support. We can then iterate on this over email, and have a >> phone call if needed, when we feel that we are close to a convergence >> point. >> >> Please, notice that I've subscribed several additional people relevant >> for the discussion (who could provide input related to UI, developers >> perspectives and operations perspectives). >> >> *Assumptions*: >> Notice that a prerequisite for all the usage scenarios is that all the >> FIWARE GEs (and SEs) are packages as Docker images and are kept in a >> central repository, preferably the docker hub (under a 'to-be-created' >> "fiware" namespace). Also, there is an assumption that the >> users/developers would want to work with Docker tools as much as >> possible (surfacing some or all of the capabilities via the FIWARE >> Cloud Portal too). Juanjo will elaborate on the approach we are >> thinking of to promote this with GE/SE owners and the broader >> developer community. >> >> *Group 1: Basic Docker environment setup* >> Usage scenario #1.1: >> A user wants a local Docker runtime on his laptop. >> He follows the standard instructions on setting up a docker host >> (e.g., running within a VirtalBox VM, using docker-machine / boot2docker) >> >> Usage scenario #1.2: >> A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab >> (where he would then run various Docker containers). >> He uses the standard docker-machine tool, specifying the URI of the >> OpenStack Keystone in FIWARE Lab (and additional parameters, as >> needed). The tool creates a VM using standard OpenStack APIs (natively >> supported by FIWARE Lab) and configures Docker within the VM. The VM >> would need to have a public IP (naturally). >> >> *Group 2: Basic life cycle of individual containers running GEs/SEs* >> Usage scenario #2.1: >> A developer wants to publish (a version of) a GE/SE. >> After he is done creating the new Docker image, he pushes the new >> version of the GE/SE to Docker hub under corresponding FIWARE >> namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the >> image points to the new version. >> >> Usage scenario #2.2: >> A user/developer wants to deploy locally an instance of a certain GE/SE. >> He uses the standard docker CLI to locally provision a container, >> referring to the corresponding image at Docker hub -- e.g.: /"$ docker >> -H boot2docker-vm:2376 run fiware/GE/cb-orion"/ >> >> Usage scenario #2.3: >> A user/developer wants to deploy an instance of a certain GE/SE within >> his Docker VM on FIWARE Lab >> He uses the standard docker CLI to provision a container, referring to >> the location of his Docker VM as well as the corresponding image at >> Docker hub -- e.g.: /"$ docker -H mydocker-vm37.lab.fiware.org:2376 >> run fiware/GE/cb-orion". /ISSUE: the user would need to open the >> corresponding firewall ports in his VM (same as those of the GE/SE, or >> following the mapping performed during container provisioning) in >> order to make the GE/SE accessible. Ideally, this should be done in a >> scalable but secure manner. An easy solution is to open up-front >> (during VM provisioning) a range of ports (via setting up >> corresponding security group). A more advanced solution is to update >> the security group dynamically. >> >> Usage scenario #2.4: >> A user/developer wants to update a container comprising certain GE/SE >> with the latest version recently published in the dedicated namespace >> of the Docker Hub. >> He pulls the latest version from the Docker Hub, kills the old >> container, and starts a new one (attaching to the same resources). >> Note that this would work well when the application is properly >> designed for Docker (e.g., the container itself is stateless), and >> when the previous version of the container has been provisioned >> manually (as in #2, #4 above) by the developer (and he knows which >> resources to connect to). >> >> *Group 3: Support for 'bundles' of GEs/SEs that together perform a >> certain complex function* >> Usage scenario #3.1: >> A developer wants to publish (a version of) a 'bundle' of GEs/SEs that >> together perform a certain complex function. >> He creates a 'template' (e.g., following the format of >> docker-compose), referring to the individual GE/SE images as well as >> their interdependencies (e.g., links) and other composition >> properties. This might be done using a text editor, a Web UI provided >> by the FIWARE Cloud (conceptually similar to today's UI for creation >> of blueprints), or other tools from Docker ecosystem. He then uploads >> the template to the centralized templats repository. Note: it is >> likely that docker-compose will be able to use Docker Registry/Hub as >> a repository for templates (including versioning, push/pull, etc). >> >> Usage scenario #3.2: >> A user wants to provision locally a set of GEs/SEs, using a >> pre-defined template (comprising a 'bundle') >> He uses the standard docker-compose tool referring to the 'template' >> artifcat as well as the local Docker URL. >> >> Usage scenario #3.3: >> A user wants to provision a set of GEs/SEs in FIWARE Lab, using a >> pre-defined template (comprising a 'bundle') >> He uses the standard docker-compose tool referring to the 'template' >> artifcat as well as the target Docker URL in FIWARE Lab. >> Alternatively, he uses the FIWARE Portal UI to do the same (e.g., with >> Murano backend invoking docker-compose). >> >> Usage scenario #3.4: >> A user wants to update his Docker environment (local or on FIWARE Lab) >> with the latest version of a certain 'bundle' >> Note: may require enancements to docker-compose >> >> *Group 4: Advanced scenarios* >> Usage scenario #4.1: >> A user/developer wants to provision a cluster of VMs on FIWARE Lab >> that would host his Docker cluster (managed with Swarm or Kubernetes) >> He uses corresponding Murano/Heat template to provision the VMs and to >> configure the Docker/Swarm/Kubernetes cluster. >> >> Usage scenario #4.2: >> A user/developer wants to access a global instance of a Docker service >> in FIWARE Lab (shared, scalable, managed), so that he doesn't need to >> manage the corresponding VM(s) by himself. >> He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab >> Docker API endpoint with the standard Docker tools or FIWARE-specific >> tools (as outlined above ). >> >> Usage scenario #4.3: >> A user/developer wants to manage access control for Docker images >> among FIWARE Lab users. >> He starts using the Docker Registry/Hub deployed within the FIWARE Lab. >> >> >> Notice that this is a very initial list -- I am sure that there are >> many inaccuracies and gaps. Feel free to comment. >> >> Thanks, >> Alex >> >> ==================================================================================== >> Alex Glikson >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: >> +972-54-6466667 | Fax: +972-4-8296112 >> >> >> >> _______________________________________________ >> Fiware-cloud-containers mailing list >> Fiware-cloud-containers at lists.fiware.org >> https://lists.fiware.org/listinfo/fiware-cloud-containers > > > ------------------------------------------------------------------------ > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener informaci?n privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilizaci?n, > divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud > de la legislaci?n vigente. Si ha recibido este mensaje por error, le > rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda > a su destrucci?n. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution > or copying of this communication is strictly prohibited. If you have > received this transmission in error, do not read it. Please immediately > reply to the sender that you have received this communication in error > and then delete it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu > destinat?rio, pode conter informa??o privilegiada ou confidencial e ? > para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa > senhoria o destinat?rio indicado, fica notificado de que a leitura, > utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em > virtude da legisla??o vigente. Se recebeu esta mensagem por erro, > rogamos-lhe que nos o comunique imediatamente por esta mesma via e > proceda a sua destrui??o > > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: philipp_slusallek.vcf Type: text/x-vcard Size: 441 bytes Desc: not available URL: From federico.facca at create-net.org Wed Jul 8 08:12:40 2015 From: federico.facca at create-net.org (Federico Michele Facca) Date: Wed, 8 Jul 2015 08:12:40 +0200 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: <559CB92E.3070002@dfki.de> References: <5592F69E.4060803@telefonica.com> <559CB92E.3070002@dfki.de> Message-ID: hi philipp, correct me if I am wrong fic2 developed a paas based on cloudfoundry. so as such you are using containers in vms, eventhough a different type of containers from docker. federico On Wed, Jul 8, 2015 at 7:46 AM, Philipp Slusallek wrote: > Hi all, > > This all sounds good! > > But what happened to your planned next meetings. FIC2 is still highly > interested in working with FIWARE to deploy the infrastructure developed > there (running on FIWARE Lab) also for FIWARE. > > I thought the idea was to explore these options in a joint next call. > > Best, > > Philipp > > Am 30.06.2015 um 22:05 schrieb Juanjo Hierro: > > Dear all, > > > > Thanks Alex for launching these discussions tracks. > > > > Regarding point (1) as anticipated by Alex, here it is the concrete > > plan we aim at implementing to promote usage of docker tools by GE/SE > > owners and the broader developer community. > > > > 1. Make info about Docker images and instruction to setup docker > > containers linked to FIWARE GEris available in the FIWARE Catalogue: > > * We will specify a general template of a new section which will > > be included in the "Creating instances" tab of entries linked to > > FIWARE GEris. This section will be elaborate on "Deploying a > > dedicated GE instance using Docker technology" > > * Each FIWARE GEri owner will be asked to setup a Docker image for > > their GEs and register it in Docker Hub. This task can start > > in parallel to the previous one. > > * Each FIWARE GEri owner will be asked to update the "Creating > > instance" tab of the entry linked to the FIWARE GEri so that it > > provides instructions about deploying a dedicated instance of > > the GEri using docker. For this purpose, the owner will follow > > the template defined in step 1. > > 2. We will incorporate the description about how to create FIWARE GEri > > instances and try them using docker within the FIWARE Tour Guide for > > developers (which will effectively become the landing page of > > http://developers.fiware.org) as a mean to shorten the learning > > curve with FIWARE GEris > > * Somewhere at the beginning of the guided tour, we will explain > > the developer how he can setup the basic docker environment > > either locally or on the FIWARE Lab on which he will be able to > > rely to try the different FIWARE GEris (Group 1 scenarios > > described by Alex) > > * A demo application will be developed that will help the > > developer to try each FIWARE GEri instance deployed using docker > > with concrete data. Developers will be able to deploy this > > application also using docker. It is the intention that this > > application will be helpful to show how several FIWARE GEris can > > be used in an integrated way. > > * Each of the chapters of the FIWARE Tour guide for developers > > will integrate a "Try it yourself" section which will help the > > developer to create an instance of the FIWARE GEris referred in > > the chapter and try them. In addition, developers will be able > > to deploy the demo application, in order to learn from a more > > elaborated example how the FIWARE GEris can be used, and > > providing instructions about how to play with the FIWARE GEris, > > also "tweak" the application. > > * Deployment of integrated FIWARE GEris and the demo application > > will be made feasible using docker composition tools > > > > > > Jos?-Manuel Cantera will drive the implementation of these concrete > > actions. > > > > Regarding point 2, Bitergia has already developed a first bundle of > > FIWARE GEris deployable using Docker compose (as a whole or > > individually). They will work in extending this bundle integrating > > additional FIWARE GEris in collaboration with the corresponding FIWARE > > GEri owners. Bitergia and the ULPG will work together in the > > development of a first version of the demo application. Eventually, > > additional partners might be incorporated as additional skills and/or > > resources are needed. We will address first those scenarios dealing > > with local deployment, then we will incorporate those related to > > deployment on other infrastructures such as the FIWARE Lab as soon as > > the necessary preliminary work and/or detailed guidelines are completed > > (this would be done under coordination by Alex and I assume we will > > leverage on experience developed in FI-Content2). > > > > Feedback is welcome. > > > > Best regards, > > > > -- Juanjo > > > > ______________________________________________________ > > > > Coordinator and Chief Architect, FIWARE platform > > CTO Industrial IoT, Telef?nica > > > > email: juanjose.hierro at telefonica.com > > twitter: @JuanjoHierro > > > > You can follow FIWARE at: > > website: http://www.fiware.org > > twitter: @FIWARE > > facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 > > linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 > > > > On 30/06/15 20:34, Alex Glikson wrote: > >> Dear partners, > >> > >> Following the initial discussion at the containers task force, we > >> identified two follow-on (related) discussion tracks: > >> 1) expected FIWARE Lab user/developer experience with Docker > >> 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker > >> (and enable requirements derived from #1) > >> > >> This email refers to topic (1). > >> > >> I will try to summarize the initial set of usage scenarios that we may > >> want to support. We can then iterate on this over email, and have a > >> phone call if needed, when we feel that we are close to a convergence > >> point. > >> > >> Please, notice that I've subscribed several additional people relevant > >> for the discussion (who could provide input related to UI, developers > >> perspectives and operations perspectives). > >> > >> *Assumptions*: > >> Notice that a prerequisite for all the usage scenarios is that all the > >> FIWARE GEs (and SEs) are packages as Docker images and are kept in a > >> central repository, preferably the docker hub (under a 'to-be-created' > >> "fiware" namespace). Also, there is an assumption that the > >> users/developers would want to work with Docker tools as much as > >> possible (surfacing some or all of the capabilities via the FIWARE > >> Cloud Portal too). Juanjo will elaborate on the approach we are > >> thinking of to promote this with GE/SE owners and the broader > >> developer community. > >> > >> *Group 1: Basic Docker environment setup* > >> Usage scenario #1.1: > >> A user wants a local Docker runtime on his laptop. > >> He follows the standard instructions on setting up a docker host > >> (e.g., running within a VirtalBox VM, using docker-machine / > boot2docker) > >> > >> Usage scenario #1.2: > >> A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab > >> (where he would then run various Docker containers). > >> He uses the standard docker-machine tool, specifying the URI of the > >> OpenStack Keystone in FIWARE Lab (and additional parameters, as > >> needed). The tool creates a VM using standard OpenStack APIs (natively > >> supported by FIWARE Lab) and configures Docker within the VM. The VM > >> would need to have a public IP (naturally). > >> > >> *Group 2: Basic life cycle of individual containers running GEs/SEs* > >> Usage scenario #2.1: > >> A developer wants to publish (a version of) a GE/SE. > >> After he is done creating the new Docker image, he pushes the new > >> version of the GE/SE to Docker hub under corresponding FIWARE > >> namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the > >> image points to the new version. > >> > >> Usage scenario #2.2: > >> A user/developer wants to deploy locally an instance of a certain GE/SE. > >> He uses the standard docker CLI to locally provision a container, > >> referring to the corresponding image at Docker hub -- e.g.: /"$ docker > >> -H boot2docker-vm:2376 run fiware/GE/cb-orion"/ > >> > >> Usage scenario #2.3: > >> A user/developer wants to deploy an instance of a certain GE/SE within > >> his Docker VM on FIWARE Lab > >> He uses the standard docker CLI to provision a container, referring to > >> the location of his Docker VM as well as the corresponding image at > >> Docker hub -- e.g.: /"$ docker -H mydocker-vm37.lab.fiware.org:2376 > >> run fiware/GE/cb-orion". /ISSUE: the user would need to open the > >> corresponding firewall ports in his VM (same as those of the GE/SE, or > >> following the mapping performed during container provisioning) in > >> order to make the GE/SE accessible. Ideally, this should be done in a > >> scalable but secure manner. An easy solution is to open up-front > >> (during VM provisioning) a range of ports (via setting up > >> corresponding security group). A more advanced solution is to update > >> the security group dynamically. > >> > >> Usage scenario #2.4: > >> A user/developer wants to update a container comprising certain GE/SE > >> with the latest version recently published in the dedicated namespace > >> of the Docker Hub. > >> He pulls the latest version from the Docker Hub, kills the old > >> container, and starts a new one (attaching to the same resources). > >> Note that this would work well when the application is properly > >> designed for Docker (e.g., the container itself is stateless), and > >> when the previous version of the container has been provisioned > >> manually (as in #2, #4 above) by the developer (and he knows which > >> resources to connect to). > >> > >> *Group 3: Support for 'bundles' of GEs/SEs that together perform a > >> certain complex function* > >> Usage scenario #3.1: > >> A developer wants to publish (a version of) a 'bundle' of GEs/SEs that > >> together perform a certain complex function. > >> He creates a 'template' (e.g., following the format of > >> docker-compose), referring to the individual GE/SE images as well as > >> their interdependencies (e.g., links) and other composition > >> properties. This might be done using a text editor, a Web UI provided > >> by the FIWARE Cloud (conceptually similar to today's UI for creation > >> of blueprints), or other tools from Docker ecosystem. He then uploads > >> the template to the centralized templats repository. Note: it is > >> likely that docker-compose will be able to use Docker Registry/Hub as > >> a repository for templates (including versioning, push/pull, etc). > >> > >> Usage scenario #3.2: > >> A user wants to provision locally a set of GEs/SEs, using a > >> pre-defined template (comprising a 'bundle') > >> He uses the standard docker-compose tool referring to the 'template' > >> artifcat as well as the local Docker URL. > >> > >> Usage scenario #3.3: > >> A user wants to provision a set of GEs/SEs in FIWARE Lab, using a > >> pre-defined template (comprising a 'bundle') > >> He uses the standard docker-compose tool referring to the 'template' > >> artifcat as well as the target Docker URL in FIWARE Lab. > >> Alternatively, he uses the FIWARE Portal UI to do the same (e.g., with > >> Murano backend invoking docker-compose). > >> > >> Usage scenario #3.4: > >> A user wants to update his Docker environment (local or on FIWARE Lab) > >> with the latest version of a certain 'bundle' > >> Note: may require enancements to docker-compose > >> > >> *Group 4: Advanced scenarios* > >> Usage scenario #4.1: > >> A user/developer wants to provision a cluster of VMs on FIWARE Lab > >> that would host his Docker cluster (managed with Swarm or Kubernetes) > >> He uses corresponding Murano/Heat template to provision the VMs and to > >> configure the Docker/Swarm/Kubernetes cluster. > >> > >> Usage scenario #4.2: > >> A user/developer wants to access a global instance of a Docker service > >> in FIWARE Lab (shared, scalable, managed), so that he doesn't need to > >> manage the corresponding VM(s) by himself. > >> He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab > >> Docker API endpoint with the standard Docker tools or FIWARE-specific > >> tools (as outlined above ). > >> > >> Usage scenario #4.3: > >> A user/developer wants to manage access control for Docker images > >> among FIWARE Lab users. > >> He starts using the Docker Registry/Hub deployed within the FIWARE Lab. > >> > >> > >> Notice that this is a very initial list -- I am sure that there are > >> many inaccuracies and gaps. Feel free to comment. > >> > >> Thanks, > >> Alex > >> > >> > ==================================================================================== > >> Alex Glikson > >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > >> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > >> +972-54-6466667 | Fax: +972-4-8296112 > >> > >> > >> > >> _______________________________________________ > >> Fiware-cloud-containers mailing list > >> Fiware-cloud-containers at lists.fiware.org > >> https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > > > ------------------------------------------------------------------------ > > > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > > puede contener informaci?n privilegiada o confidencial y es para uso > > exclusivo de la persona o entidad de destino. Si no es usted. el > > destinatario indicado, queda notificado de que la lectura, utilizaci?n, > > divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud > > de la legislaci?n vigente. Si ha recibido este mensaje por error, le > > rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda > > a su destrucci?n. > > > > The information contained in this transmission is privileged and > > confidential information intended only for the use of the individual or > > entity named above. If the reader of this message is not the intended > > recipient, you are hereby notified that any dissemination, distribution > > or copying of this communication is strictly prohibited. If you have > > received this transmission in error, do not read it. Please immediately > > reply to the sender that you have received this communication in error > > and then delete it. > > > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu > > destinat?rio, pode conter informa??o privilegiada ou confidencial e ? > > para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa > > senhoria o destinat?rio indicado, fica notificado de que a leitura, > > utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em > > virtude da legisla??o vigente. Se recebeu esta mensagem por erro, > > rogamos-lhe que nos o comunique imediatamente por esta mesma via e > > proceda a sua destrui??o > > > > > > _______________________________________________ > > Fiware-cloud-containers mailing list > > Fiware-cloud-containers at lists.fiware.org > > https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > -- > > ------------------------------------------------------------------------- > Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH > Trippstadter Strasse 122, D-67663 Kaiserslautern > > Gesch?ftsf?hrung: > Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) > Dr. Walter Olthoff > Vorsitzender des Aufsichtsrats: > Prof. Dr. h.c. Hans A. Aukes > > Sitz der Gesellschaft: Kaiserslautern (HRB 2313) > USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 > --------------------------------------------------------------------------- > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > https://lists.fiware.org/listinfo/fiware-cloud-containers > > -- -- Future Internet is closer than you think! http://www.fiware.org Official Mirantis partner for OpenStack Training https://www.create-net.org/community/openstack-training -- Dr. Federico M. Facca CREATE-NET Via alla Cascata 56/D 38123 Povo Trento (Italy) P +39 0461 312471 M +39 334 6049758 E federico.facca at create-net.org T @chicco785 W www.create-net.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From philipp.slusallek at dfki.de Wed Jul 8 08:53:53 2015 From: philipp.slusallek at dfki.de (Philipp Slusallek) Date: Wed, 08 Jul 2015 08:53:53 +0200 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: References: <5592F69E.4060803@telefonica.com> <559CB92E.3070002@dfki.de> Message-ID: <559CC901.1020606@dfki.de> Hi, For the new "FIWARE Media & content lab" (aka FIC2 Lab) we exclusively use docker deployed within FIWARE Lab VMs. We use a dedicated set of VMs for the Tweak functionality and can do 1-click deployments of GEs and SEs (or groups thereof) to a dockerized VW of a FIWARE Lab user. We also offer instructions for users to deploy the functionality to private docker installations. All relevant FIcontent SEs (and needed FIWARE GEs) have been dockerized for this new offering of FIcontent. A few more will follow You can try FIWARE media & contet lab here: http://lab.mediafi.org/. Best, Philipp Am 08.07.2015 um 08:12 schrieb Federico Michele Facca: > hi philipp, > correct me if I am wrong fic2 developed a paas based on cloudfoundry. so > as such you are using containers in vms, > eventhough a different type of containers from docker. > > federico > > On Wed, Jul 8, 2015 at 7:46 AM, Philipp Slusallek > > wrote: > > Hi all, > > This all sounds good! > > But what happened to your planned next meetings. FIC2 is still highly > interested in working with FIWARE to deploy the infrastructure developed > there (running on FIWARE Lab) also for FIWARE. > > I thought the idea was to explore these options in a joint next call. > > Best, > > Philipp > > Am 30.06.2015 um 22:05 schrieb Juanjo Hierro: > > Dear all, > > > > Thanks Alex for launching these discussions tracks. > > > > Regarding point (1) as anticipated by Alex, here it is the concrete > > plan we aim at implementing to promote usage of docker tools by GE/SE > > owners and the broader developer community. > > > > 1. Make info about Docker images and instruction to setup docker > > containers linked to FIWARE GEris available in the FIWARE Catalogue: > > * We will specify a general template of a new section which will > > be included in the "Creating instances" tab of entries linked to > > FIWARE GEris. This section will be elaborate on "Deploying a > > dedicated GE instance using Docker technology" > > * Each FIWARE GEri owner will be asked to setup a Docker > image for > > their GEs and register it in Docker Hub. This task can start > > in parallel to the previous one. > > * Each FIWARE GEri owner will be asked to update the "Creating > > instance" tab of the entry linked to the FIWARE GEri so that it > > provides instructions about deploying a dedicated instance of > > the GEri using docker. For this purpose, the owner will follow > > the template defined in step 1. > > 2. We will incorporate the description about how to create FIWARE > GEri > > instances and try them using docker within the FIWARE Tour Guide for > > developers (which will effectively become the landing page of > > http://developers.fiware.org) as a mean to shorten the learning > > curve with FIWARE GEris > > * Somewhere at the beginning of the guided tour, we will explain > > the developer how he can setup the basic docker environment > > either locally or on the FIWARE Lab on which he will be able to > > rely to try the different FIWARE GEris (Group 1 scenarios > > described by Alex) > > * A demo application will be developed that will help the > > developer to try each FIWARE GEri instance deployed using docker > > with concrete data. Developers will be able to deploy this > > application also using docker. It is the intention that this > > application will be helpful to show how several FIWARE GEris can > > be used in an integrated way. > > * Each of the chapters of the FIWARE Tour guide for developers > > will integrate a "Try it yourself" section which will help the > > developer to create an instance of the FIWARE GEris referred in > > the chapter and try them. In addition, developers will be able > > to deploy the demo application, in order to learn from a more > > elaborated example how the FIWARE GEris can be used, and > > providing instructions about how to play with the FIWARE GEris, > > also "tweak" the application. > > * Deployment of integrated FIWARE GEris and the demo application > > will be made feasible using docker composition tools > > > > > > Jos?-Manuel Cantera will drive the implementation of these concrete > > actions. > > > > Regarding point 2, Bitergia has already developed a first bundle of > > FIWARE GEris deployable using Docker compose (as a whole or > > individually). They will work in extending this bundle integrating > > additional FIWARE GEris in collaboration with the corresponding FIWARE > > GEri owners. Bitergia and the ULPG will work together in the > > development of a first version of the demo application. Eventually, > > additional partners might be incorporated as additional skills and/or > > resources are needed. We will address first those scenarios dealing > > with local deployment, then we will incorporate those related to > > deployment on other infrastructures such as the FIWARE Lab as soon as > > the necessary preliminary work and/or detailed guidelines are > completed > > (this would be done under coordination by Alex and I assume we will > > leverage on experience developed in FI-Content2). > > > > Feedback is welcome. > > > > Best regards, > > > > -- Juanjo > > > > ______________________________________________________ > > > > Coordinator and Chief Architect, FIWARE platform > > CTO Industrial IoT, Telef?nica > > > > email: juanjose.hierro at telefonica.com > > > twitter: @JuanjoHierro > > > > You can follow FIWARE at: > > website: http://www.fiware.org > > twitter: @FIWARE > > facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 > > linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 > > > > On 30/06/15 20:34, Alex Glikson wrote: > >> Dear partners, > >> > >> Following the initial discussion at the containers task force, we > >> identified two follow-on (related) discussion tracks: > >> 1) expected FIWARE Lab user/developer experience with Docker > >> 2) Enhancements to FIWARE Cloud Hosting architecture to support > Docker > >> (and enable requirements derived from #1) > >> > >> This email refers to topic (1). > >> > >> I will try to summarize the initial set of usage scenarios that > we may > >> want to support. We can then iterate on this over email, and have a > >> phone call if needed, when we feel that we are close to a convergence > >> point. > >> > >> Please, notice that I've subscribed several additional people > relevant > >> for the discussion (who could provide input related to UI, developers > >> perspectives and operations perspectives). > >> > >> *Assumptions*: > >> Notice that a prerequisite for all the usage scenarios is that all the > >> FIWARE GEs (and SEs) are packages as Docker images and are kept in a > >> central repository, preferably the docker hub (under a 'to-be-created' > >> "fiware" namespace). Also, there is an assumption that the > >> users/developers would want to work with Docker tools as much as > >> possible (surfacing some or all of the capabilities via the FIWARE > >> Cloud Portal too). Juanjo will elaborate on the approach we are > >> thinking of to promote this with GE/SE owners and the broader > >> developer community. > >> > >> *Group 1: Basic Docker environment setup* > >> Usage scenario #1.1: > >> A user wants a local Docker runtime on his laptop. > >> He follows the standard instructions on setting up a docker host > >> (e.g., running within a VirtalBox VM, using docker-machine / boot2docker) > >> > >> Usage scenario #1.2: > >> A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab > >> (where he would then run various Docker containers). > >> He uses the standard docker-machine tool, specifying the URI of the > >> OpenStack Keystone in FIWARE Lab (and additional parameters, as > >> needed). The tool creates a VM using standard OpenStack APIs (natively > >> supported by FIWARE Lab) and configures Docker within the VM. The VM > >> would need to have a public IP (naturally). > >> > >> *Group 2: Basic life cycle of individual containers running GEs/SEs* > >> Usage scenario #2.1: > >> A developer wants to publish (a version of) a GE/SE. > >> After he is done creating the new Docker image, he pushes the new > >> version of the GE/SE to Docker hub under corresponding FIWARE > >> namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the > >> image points to the new version. > >> > >> Usage scenario #2.2: > >> A user/developer wants to deploy locally an instance of a certain GE/SE. > >> He uses the standard docker CLI to locally provision a container, > >> referring to the corresponding image at Docker hub -- e.g.: /"$ > docker > >> -H boot2docker-vm:2376 run fiware/GE/cb-orion"/ > >> > >> Usage scenario #2.3: > >> A user/developer wants to deploy an instance of a certain GE/SE within > >> his Docker VM on FIWARE Lab > >> He uses the standard docker CLI to provision a container, referring to > >> the location of his Docker VM as well as the corresponding image at > >> Docker hub -- e.g.: /"$ docker -H > mydocker-vm37.lab.fiware.org:2376 > > >> run fiware/GE/cb-orion". /ISSUE: the user would need to open the > >> corresponding firewall ports in his VM (same as those of the GE/SE, or > >> following the mapping performed during container provisioning) in > >> order to make the GE/SE accessible. Ideally, this should be done in a > >> scalable but secure manner. An easy solution is to open up-front > >> (during VM provisioning) a range of ports (via setting up > >> corresponding security group). A more advanced solution is to update > >> the security group dynamically. > >> > >> Usage scenario #2.4: > >> A user/developer wants to update a container comprising certain GE/SE > >> with the latest version recently published in the dedicated namespace > >> of the Docker Hub. > >> He pulls the latest version from the Docker Hub, kills the old > >> container, and starts a new one (attaching to the same resources). > >> Note that this would work well when the application is properly > >> designed for Docker (e.g., the container itself is stateless), and > >> when the previous version of the container has been provisioned > >> manually (as in #2, #4 above) by the developer (and he knows which > >> resources to connect to). > >> > >> *Group 3: Support for 'bundles' of GEs/SEs that together perform a > >> certain complex function* > >> Usage scenario #3.1: > >> A developer wants to publish (a version of) a 'bundle' of GEs/SEs > that > >> together perform a certain complex function. > >> He creates a 'template' (e.g., following the format of > >> docker-compose), referring to the individual GE/SE images as well as > >> their interdependencies (e.g., links) and other composition > >> properties. This might be done using a text editor, a Web UI provided > >> by the FIWARE Cloud (conceptually similar to today's UI for creation > >> of blueprints), or other tools from Docker ecosystem. He then uploads > >> the template to the centralized templats repository. Note: it is > >> likely that docker-compose will be able to use Docker Registry/Hub as > >> a repository for templates (including versioning, push/pull, etc). > >> > >> Usage scenario #3.2: > >> A user wants to provision locally a set of GEs/SEs, using a > >> pre-defined template (comprising a 'bundle') > >> He uses the standard docker-compose tool referring to the 'template' > >> artifcat as well as the local Docker URL. > >> > >> Usage scenario #3.3: > >> A user wants to provision a set of GEs/SEs in FIWARE Lab, using a > >> pre-defined template (comprising a 'bundle') > >> He uses the standard docker-compose tool referring to the 'template' > >> artifcat as well as the target Docker URL in FIWARE Lab. > >> Alternatively, he uses the FIWARE Portal UI to do the same (e.g., > with > >> Murano backend invoking docker-compose). > >> > >> Usage scenario #3.4: > >> A user wants to update his Docker environment (local or on FIWARE > Lab) > >> with the latest version of a certain 'bundle' > >> Note: may require enancements to docker-compose > >> > >> *Group 4: Advanced scenarios* > >> Usage scenario #4.1: > >> A user/developer wants to provision a cluster of VMs on FIWARE Lab > >> that would host his Docker cluster (managed with Swarm or Kubernetes) > >> He uses corresponding Murano/Heat template to provision the VMs > and to > >> configure the Docker/Swarm/Kubernetes cluster. > >> > >> Usage scenario #4.2: > >> A user/developer wants to access a global instance of a Docker > service > >> in FIWARE Lab (shared, scalable, managed), so that he doesn't need to > >> manage the corresponding VM(s) by himself. > >> He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab > >> Docker API endpoint with the standard Docker tools or FIWARE-specific > >> tools (as outlined above ). > >> > >> Usage scenario #4.3: > >> A user/developer wants to manage access control for Docker images > >> among FIWARE Lab users. > >> He starts using the Docker Registry/Hub deployed within the > FIWARE Lab. > >> > >> > >> Notice that this is a very initial list -- I am sure that there are > >> many inaccuracies and gaps. Feel free to comment. > >> > >> Thanks, > >> Alex > >> > >> > ==================================================================================== > >> Alex Glikson > >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > >> Email: glikson at il.ibm.com | Phone: > +972-4-8281085 | Mobile: > >> +972-54-6466667 | Fax: +972-4-8296112 > > >> > >> > >> > >> _______________________________________________ > >> Fiware-cloud-containers mailing list > >> Fiware-cloud-containers at lists.fiware.org > > >> https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > > > > ------------------------------------------------------------------------ > > > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > > puede contener informaci?n privilegiada o confidencial y es para uso > > exclusivo de la persona o entidad de destino. Si no es usted. el > > destinatario indicado, queda notificado de que la lectura, utilizaci?n, > > divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud > > de la legislaci?n vigente. Si ha recibido este mensaje por error, le > > rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda > > a su destrucci?n. > > > > The information contained in this transmission is privileged and > > confidential information intended only for the use of the individual or > > entity named above. If the reader of this message is not the intended > > recipient, you are hereby notified that any dissemination, distribution > > or copying of this communication is strictly prohibited. If you have > > received this transmission in error, do not read it. Please immediately > > reply to the sender that you have received this communication in error > > and then delete it. > > > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu > > destinat?rio, pode conter informa??o privilegiada ou confidencial e ? > > para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa > > senhoria o destinat?rio indicado, fica notificado de que a leitura, > > utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em > > virtude da legisla??o vigente. Se recebeu esta mensagem por erro, > > rogamos-lhe que nos o comunique imediatamente por esta mesma via e > > proceda a sua destrui??o > > > > > > _______________________________________________ > > Fiware-cloud-containers mailing list > > Fiware-cloud-containers at lists.fiware.org > > > https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > -- > > ------------------------------------------------------------------------- > Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH > Trippstadter Strasse 122, D-67663 Kaiserslautern > > Gesch?ftsf?hrung: > Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) > Dr. Walter Olthoff > Vorsitzender des Aufsichtsrats: > Prof. Dr. h.c. Hans A. Aukes > > Sitz der Gesellschaft: Kaiserslautern (HRB 2313) > USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 > --------------------------------------------------------------------------- > > _______________________________________________ > Fiware-cloud-containers mailing list > Fiware-cloud-containers at lists.fiware.org > > https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > > -- > -- > Future Internet is closer than you think! > http://www.fiware.org > > Official Mirantis partner for OpenStack Training > https://www.create-net.org/community/openstack-training > > -- > Dr. Federico M. Facca > > CREATE-NET > Via alla Cascata 56/D > 38123 Povo Trento (Italy) > > P +39 0461 312471 > M +39 334 6049758 > E federico.facca at create-net.org > T @chicco785 > W www.create-net.org -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: philipp_slusallek.vcf Type: text/x-vcard Size: 441 bytes Desc: not available URL: From federico.facca at create-net.org Wed Jul 8 09:03:26 2015 From: federico.facca at create-net.org (Federico Michele Facca) Date: Wed, 8 Jul 2015 09:03:26 +0200 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: <559CC901.1020606@dfki.de> References: <5592F69E.4060803@telefonica.com> <559CB92E.3070002@dfki.de> <559CC901.1020606@dfki.de> Message-ID: nice :) On Wed, Jul 8, 2015 at 8:53 AM, Philipp Slusallek wrote: > Hi, > > For the new "FIWARE Media & content lab" (aka FIC2 Lab) we exclusively > use docker deployed within FIWARE Lab VMs. We use a dedicated set of VMs > for the Tweak functionality and can do 1-click deployments of GEs and > SEs (or groups thereof) to a dockerized VW of a FIWARE Lab user. We also > offer instructions for users to deploy the functionality to private > docker installations. > > All relevant FIcontent SEs (and needed FIWARE GEs) have been dockerized > for this new offering of FIcontent. A few more will follow > > You can try FIWARE media & contet lab here: http://lab.mediafi.org/. > > > Best, > > Philipp > > Am 08.07.2015 um 08:12 schrieb Federico Michele Facca: > > hi philipp, > > correct me if I am wrong fic2 developed a paas based on cloudfoundry. so > > as such you are using containers in vms, > > eventhough a different type of containers from docker. > > > > federico > > > > On Wed, Jul 8, 2015 at 7:46 AM, Philipp Slusallek > > > wrote: > > > > Hi all, > > > > This all sounds good! > > > > But what happened to your planned next meetings. FIC2 is still highly > > interested in working with FIWARE to deploy the infrastructure > developed > > there (running on FIWARE Lab) also for FIWARE. > > > > I thought the idea was to explore these options in a joint next call. > > > > Best, > > > > Philipp > > > > Am 30.06.2015 um 22:05 schrieb Juanjo Hierro: > > > Dear all, > > > > > > Thanks Alex for launching these discussions tracks. > > > > > > Regarding point (1) as anticipated by Alex, here it is the > concrete > > > plan we aim at implementing to promote usage of docker tools by > GE/SE > > > owners and the broader developer community. > > > > > > 1. Make info about Docker images and instruction to setup docker > > > containers linked to FIWARE GEris available in the FIWARE > Catalogue: > > > * We will specify a general template of a new section which > will > > > be included in the "Creating instances" tab of entries > linked to > > > FIWARE GEris. This section will be elaborate on > "Deploying a > > > dedicated GE instance using Docker technology" > > > * Each FIWARE GEri owner will be asked to setup a Docker > > image for > > > their GEs and register it in Docker Hub. This task can > start > > > in parallel to the previous one. > > > * Each FIWARE GEri owner will be asked to update the > "Creating > > > instance" tab of the entry linked to the FIWARE GEri so > that it > > > provides instructions about deploying a dedicated instance > of > > > the GEri using docker. For this purpose, the owner will > follow > > > the template defined in step 1. > > > 2. We will incorporate the description about how to create FIWARE > > GEri > > > instances and try them using docker within the FIWARE Tour > Guide for > > > developers (which will effectively become the landing page of > > > http://developers.fiware.org) as a mean to shorten the > learning > > > curve with FIWARE GEris > > > * Somewhere at the beginning of the guided tour, we will > explain > > > the developer how he can setup the basic docker environment > > > either locally or on the FIWARE Lab on which he will be > able to > > > rely to try the different FIWARE GEris (Group 1 scenarios > > > described by Alex) > > > * A demo application will be developed that will help the > > > developer to try each FIWARE GEri instance deployed using > docker > > > with concrete data. Developers will be able to deploy this > > > application also using docker. It is the intention that > this > > > application will be helpful to show how several FIWARE > GEris can > > > be used in an integrated way. > > > * Each of the chapters of the FIWARE Tour guide for > developers > > > will integrate a "Try it yourself" section which will help > the > > > developer to create an instance of the FIWARE GEris > referred in > > > the chapter and try them. In addition, developers will > be able > > > to deploy the demo application, in order to learn from a > more > > > elaborated example how the FIWARE GEris can be used, and > > > providing instructions about how to play with the FIWARE > GEris, > > > also "tweak" the application. > > > * Deployment of integrated FIWARE GEris and the demo > application > > > will be made feasible using docker composition tools > > > > > > > > > Jos?-Manuel Cantera will drive the implementation of these > concrete > > > actions. > > > > > > Regarding point 2, Bitergia has already developed a first bundle > of > > > FIWARE GEris deployable using Docker compose (as a whole or > > > individually). They will work in extending this bundle > integrating > > > additional FIWARE GEris in collaboration with the corresponding > FIWARE > > > GEri owners. Bitergia and the ULPG will work together in the > > > development of a first version of the demo application. > Eventually, > > > additional partners might be incorporated as additional skills > and/or > > > resources are needed. We will address first those scenarios > dealing > > > with local deployment, then we will incorporate those related to > > > deployment on other infrastructures such as the FIWARE Lab as soon > as > > > the necessary preliminary work and/or detailed guidelines are > > completed > > > (this would be done under coordination by Alex and I assume we will > > > leverage on experience developed in FI-Content2). > > > > > > Feedback is welcome. > > > > > > Best regards, > > > > > > -- Juanjo > > > > > > ______________________________________________________ > > > > > > Coordinator and Chief Architect, FIWARE platform > > > CTO Industrial IoT, Telef?nica > > > > > > email: juanjose.hierro at telefonica.com > > > > > twitter: @JuanjoHierro > > > > > > You can follow FIWARE at: > > > website: http://www.fiware.org > > > twitter: @FIWARE > > > facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 > > > linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 > > > > > > On 30/06/15 20:34, Alex Glikson wrote: > > >> Dear partners, > > >> > > >> Following the initial discussion at the containers task force, we > > >> identified two follow-on (related) discussion tracks: > > >> 1) expected FIWARE Lab user/developer experience with Docker > > >> 2) Enhancements to FIWARE Cloud Hosting architecture to support > > Docker > > >> (and enable requirements derived from #1) > > >> > > >> This email refers to topic (1). > > >> > > >> I will try to summarize the initial set of usage scenarios that > > we may > > >> want to support. We can then iterate on this over email, and have > a > > >> phone call if needed, when we feel that we are close to a > convergence > > >> point. > > >> > > >> Please, notice that I've subscribed several additional people > > relevant > > >> for the discussion (who could provide input related to UI, > developers > > >> perspectives and operations perspectives). > > >> > > >> *Assumptions*: > > >> Notice that a prerequisite for all the usage scenarios is that > all the > > >> FIWARE GEs (and SEs) are packages as Docker images and are kept > in a > > >> central repository, preferably the docker hub (under a > 'to-be-created' > > >> "fiware" namespace). Also, there is an assumption that the > > >> users/developers would want to work with Docker tools as much as > > >> possible (surfacing some or all of the capabilities via the FIWARE > > >> Cloud Portal too). Juanjo will elaborate on the approach we are > > >> thinking of to promote this with GE/SE owners and the broader > > >> developer community. > > >> > > >> *Group 1: Basic Docker environment setup* > > >> Usage scenario #1.1: > > >> A user wants a local Docker runtime on his laptop. > > >> He follows the standard instructions on setting up a docker host > > >> (e.g., running within a VirtalBox VM, using docker-machine / > boot2docker) > > >> > > >> Usage scenario #1.2: > > >> A user/developer wants to deploy a dedicated Docker VM on FIWARE > Lab > > >> (where he would then run various Docker containers). > > >> He uses the standard docker-machine tool, specifying the URI of > the > > >> OpenStack Keystone in FIWARE Lab (and additional parameters, as > > >> needed). The tool creates a VM using standard OpenStack APIs > (natively > > >> supported by FIWARE Lab) and configures Docker within the VM. The > VM > > >> would need to have a public IP (naturally). > > >> > > >> *Group 2: Basic life cycle of individual containers running > GEs/SEs* > > >> Usage scenario #2.1: > > >> A developer wants to publish (a version of) a GE/SE. > > >> After he is done creating the new Docker image, he pushes the new > > >> version of the GE/SE to Docker hub under corresponding FIWARE > > >> namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of > the > > >> image points to the new version. > > >> > > >> Usage scenario #2.2: > > >> A user/developer wants to deploy locally an instance of a certain > GE/SE. > > >> He uses the standard docker CLI to locally provision a container, > > >> referring to the corresponding image at Docker hub -- e.g.: /"$ > > docker > > >> -H boot2docker-vm:2376 run fiware/GE/cb-orion"/ > > >> > > >> Usage scenario #2.3: > > >> A user/developer wants to deploy an instance of a certain GE/SE > within > > >> his Docker VM on FIWARE Lab > > >> He uses the standard docker CLI to provision a container, > referring to > > >> the location of his Docker VM as well as the corresponding image > at > > >> Docker hub -- e.g.: /"$ docker -H > > mydocker-vm37.lab.fiware.org:2376 > > > > >> run fiware/GE/cb-orion". /ISSUE: the user would need to open the > > >> corresponding firewall ports in his VM (same as those of the > GE/SE, or > > >> following the mapping performed during container provisioning) in > > >> order to make the GE/SE accessible. Ideally, this should be done > in a > > >> scalable but secure manner. An easy solution is to open up-front > > >> (during VM provisioning) a range of ports (via setting up > > >> corresponding security group). A more advanced solution is to > update > > >> the security group dynamically. > > >> > > >> Usage scenario #2.4: > > >> A user/developer wants to update a container comprising certain > GE/SE > > >> with the latest version recently published in the dedicated > namespace > > >> of the Docker Hub. > > >> He pulls the latest version from the Docker Hub, kills the old > > >> container, and starts a new one (attaching to the same resources). > > >> Note that this would work well when the application is properly > > >> designed for Docker (e.g., the container itself is stateless), and > > >> when the previous version of the container has been provisioned > > >> manually (as in #2, #4 above) by the developer (and he knows which > > >> resources to connect to). > > >> > > >> *Group 3: Support for 'bundles' of GEs/SEs that together perform a > > >> certain complex function* > > >> Usage scenario #3.1: > > >> A developer wants to publish (a version of) a 'bundle' of GEs/SEs > > that > > >> together perform a certain complex function. > > >> He creates a 'template' (e.g., following the format of > > >> docker-compose), referring to the individual GE/SE images as well > as > > >> their interdependencies (e.g., links) and other composition > > >> properties. This might be done using a text editor, a Web UI > provided > > >> by the FIWARE Cloud (conceptually similar to today's UI for > creation > > >> of blueprints), or other tools from Docker ecosystem. He then > uploads > > >> the template to the centralized templats repository. Note: it is > > >> likely that docker-compose will be able to use Docker > Registry/Hub as > > >> a repository for templates (including versioning, push/pull, etc). > > >> > > >> Usage scenario #3.2: > > >> A user wants to provision locally a set of GEs/SEs, using a > > >> pre-defined template (comprising a 'bundle') > > >> He uses the standard docker-compose tool referring to the > 'template' > > >> artifcat as well as the local Docker URL. > > >> > > >> Usage scenario #3.3: > > >> A user wants to provision a set of GEs/SEs in FIWARE Lab, using a > > >> pre-defined template (comprising a 'bundle') > > >> He uses the standard docker-compose tool referring to the > 'template' > > >> artifcat as well as the target Docker URL in FIWARE Lab. > > >> Alternatively, he uses the FIWARE Portal UI to do the same (e.g., > > with > > >> Murano backend invoking docker-compose). > > >> > > >> Usage scenario #3.4: > > >> A user wants to update his Docker environment (local or on FIWARE > > Lab) > > >> with the latest version of a certain 'bundle' > > >> Note: may require enancements to docker-compose > > >> > > >> *Group 4: Advanced scenarios* > > >> Usage scenario #4.1: > > >> A user/developer wants to provision a cluster of VMs on FIWARE Lab > > >> that would host his Docker cluster (managed with Swarm or > Kubernetes) > > >> He uses corresponding Murano/Heat template to provision the VMs > > and to > > >> configure the Docker/Swarm/Kubernetes cluster. > > >> > > >> Usage scenario #4.2: > > >> A user/developer wants to access a global instance of a Docker > > service > > >> in FIWARE Lab (shared, scalable, managed), so that he doesn't > need to > > >> manage the corresponding VM(s) by himself. > > >> He authenticates with FIWARE Lab, and starts accessing the FIWARE > Lab > > >> Docker API endpoint with the standard Docker tools or > FIWARE-specific > > >> tools (as outlined above ). > > >> > > >> Usage scenario #4.3: > > >> A user/developer wants to manage access control for Docker images > > >> among FIWARE Lab users. > > >> He starts using the Docker Registry/Hub deployed within the > > FIWARE Lab. > > >> > > >> > > >> Notice that this is a very initial list -- I am sure that there > are > > >> many inaccuracies and gaps. Feel free to comment. > > >> > > >> Thanks, > > >> Alex > > >> > > >> > > > ==================================================================================== > > >> Alex Glikson > > >> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > > >> Email: glikson at il.ibm.com | Phone: > > +972-4-8281085 | Mobile: > > >> +972-54-6466667 | Fax: +972-4-8296112 > > > > >> > > >> > > >> > > >> _______________________________________________ > > >> Fiware-cloud-containers mailing list > > >> Fiware-cloud-containers at lists.fiware.org > > > > >> https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > > > > > > > > > ------------------------------------------------------------------------ > > > > > > Este mensaje y sus adjuntos se dirigen exclusivamente a su > destinatario, > > > puede contener informaci?n privilegiada o confidencial y es para > uso > > > exclusivo de la persona o entidad de destino. Si no es usted. el > > > destinatario indicado, queda notificado de que la lectura, > utilizaci?n, > > > divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en > virtud > > > de la legislaci?n vigente. Si ha recibido este mensaje por error, > le > > > rogamos que nos lo comunique inmediatamente por esta misma v?a y > proceda > > > a su destrucci?n. > > > > > > The information contained in this transmission is privileged and > > > confidential information intended only for the use of the > individual or > > > entity named above. If the reader of this message is not the > intended > > > recipient, you are hereby notified that any dissemination, > distribution > > > or copying of this communication is strictly prohibited. If you > have > > > received this transmission in error, do not read it. Please > immediately > > > reply to the sender that you have received this communication in > error > > > and then delete it. > > > > > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu > > > destinat?rio, pode conter informa??o privilegiada ou confidencial > e ? > > > para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa > > > senhoria o destinat?rio indicado, fica notificado de que a leitura, > > > utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar > proibida em > > > virtude da legisla??o vigente. Se recebeu esta mensagem por erro, > > > rogamos-lhe que nos o comunique imediatamente por esta mesma via e > > > proceda a sua destrui??o > > > > > > > > > _______________________________________________ > > > Fiware-cloud-containers mailing list > > > Fiware-cloud-containers at lists.fiware.org > > > > > https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > > > > -- > > > > > ------------------------------------------------------------------------- > > Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH > > Trippstadter Strasse 122, D-67663 Kaiserslautern > > > > Gesch?ftsf?hrung: > > Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) > > Dr. Walter Olthoff > > Vorsitzender des Aufsichtsrats: > > Prof. Dr. h.c. Hans A. Aukes > > > > Sitz der Gesellschaft: Kaiserslautern (HRB 2313) > > USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 > > > --------------------------------------------------------------------------- > > > > _______________________________________________ > > Fiware-cloud-containers mailing list > > Fiware-cloud-containers at lists.fiware.org > > > > https://lists.fiware.org/listinfo/fiware-cloud-containers > > > > > > > > > > -- > > -- > > Future Internet is closer than you think! > > http://www.fiware.org > > > > Official Mirantis partner for OpenStack Training > > https://www.create-net.org/community/openstack-training > > > > -- > > Dr. Federico M. Facca > > > > CREATE-NET > > Via alla Cascata 56/D > > 38123 Povo Trento (Italy) > > > > P +39 0461 312471 > > M +39 334 6049758 > > E federico.facca at create-net.org > > T @chicco785 > > W www.create-net.org > > -- > > ------------------------------------------------------------------------- > Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH > Trippstadter Strasse 122, D-67663 Kaiserslautern > > Gesch?ftsf?hrung: > Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) > Dr. Walter Olthoff > Vorsitzender des Aufsichtsrats: > Prof. Dr. h.c. Hans A. Aukes > > Sitz der Gesellschaft: Kaiserslautern (HRB 2313) > USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 > --------------------------------------------------------------------------- > -- -- Future Internet is closer than you think! http://www.fiware.org Official Mirantis partner for OpenStack Training https://www.create-net.org/community/openstack-training -- Dr. Federico M. Facca CREATE-NET Via alla Cascata 56/D 38123 Povo Trento (Italy) P +39 0461 312471 M +39 334 6049758 E federico.facca at create-net.org T @chicco785 W www.create-net.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From juanjose.hierro at telefonica.com Wed Jul 8 09:45:14 2015 From: juanjose.hierro at telefonica.com (Juanjo Hierro) Date: Wed, 8 Jul 2015 09:45:14 +0200 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: <559CB92E.3070002@dfki.de> References: <5592F69E.4060803@telefonica.com> <559CB92E.3070002@dfki.de> Message-ID: <559CD50A.9060403@telefonica.com> An HTML attachment was scrubbed... URL: From NAGIN at il.ibm.com Wed Jul 8 10:16:29 2015 From: NAGIN at il.ibm.com (Kenneth Nagin) Date: Wed, 8 Jul 2015 11:16:29 +0300 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: References: <5592F69E.4060803@telefonica.com> Message-ID: Your "two cents" is very important to us. In fact, we welcome feedback from all, since we are now identifying the current limitations of hosting docker on the FIWARE Lab. Our target audience is developers familiar with docker that are creating services based on the FIWARE GEs . The use case is that they remotely manage their docker hosts and containers with local docker clients. Our design objectives are: Maximise the management capabilities; Enable automation; Make the user experience as frictionless as possible; Hide the underlying infrastructure, i.e. Openstack, so that developers won't need to learn yet another paradigm; Promote more efficient resource utilisation from the provider's point of view, i.e. FIWARE Lab; Allow other Docker DevOps tool sets to easily interact with the FIWARE Docker hosting infrastructure; etc. (your input is welcome) responses to your ps: >do we really need 1 docker vm for 1 user? That is the current state, but we need to do better. At least we need to expand the usage to the level of an organization with many members. >(or actually do we need VMs?) Bare metal and more efficient multi-tenancy will be in our road map, but we need to walk before we run. >Usage scenario #2.3, is this related to current limits of neutron w.r.t. docker-scheduler in OpenStack? We need to investigate further, but I don't think this is an OpenStack limitation, rather it looks like a limitation in docker-machine's OpenStack driver. Best Regards, Kenneth Nagin Ph: +972-4-8296227 Cell: 054-6976227 Fx: +972-4- 8296114 http://researcher.ibm.com/view.php?person=il-NAGIN From: Federico Michele Facca To: Kenneth Nagin/Haifa/IBM at IBMIL Cc: fiware-cloud-containers at lists.fiware.org Date: 07/07/2015 04:58 PM Subject: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker hi guys, i think the introduction of docker is a very interesting evolution of fiware offer. my two cents (following also some remarks from the commission) is that whatever is done needs to take into consideration the user experience and making life easier for the developer. - most of the complexity (unless the developer is really interested into getting into it) should be hidden to him (maybe this simply means more pre-cooked food / recipies for him in the blueprint/murano engine); - maybe we can learn from paas that are successful: can we have a docker environment for 2-3 development platforms/frameworks (django, ruby-on-rails) and simplify allow developers to deploy their code on it from a github repository? more things are possible (natting like in cloudfounfry and automatic dns registration) that may improve the experience, i am just brainstorming :) some of the things can be pushed as well to WP2.1. hope it helps! fede PS: do we really need 1 docker vm for 1 user? (or actually do we need VMs?) c.f. Usage scenario #2.3, is this related to current limits of neutron w.r.t. docker-scheduler in OpenStack? On Tue, Jul 7, 2015 at 3:36 PM, Kenneth Nagin wrote: The Cloud chapter has already begun focusing on providing Docker support. 1. IBM is preparing a study of how FIWARE developers can leverage the current FIWARE lab to host Docker while using Docker ecosystem tools locally to develop and deploy their applications. We will verify 1) Docker-Engine and Docker-Machine for setting up their remote hosting environment, 2) Docker-Compose for constructing and running multi-container applications hosted on FIWARE, 3) Docker-Swarm for host clustering and container scheduling. (Issue CLD-574). The output of the study will be to describe the capabilities, limitations and gaps of hosting Docker on FIWARE. We presented preliminary results at the sprint closing chapter review (Monday July 6). 2. IBM documented the required set up by a FIWARE developer for hosting Docker on FIWARE. This documentation is available at http://www.slideshare.net/knagin/simple-docker-hosting-on-fiware-lab. (Issue CLD-576). This is basically the outputs of <1> and can be contributed to item <2> in Juano's attached note. We are currently studying the limitation of this environment and a roadmap to address these limitations . We will share it with the task force once it is completed. 3. TID is preparing a demonstration of Murano Docker Support (Issues CLD-560, CLD-568 ). This would allow a FIWARE developer to deploy Docker containers using the Murano API. Henar will demonstrate deploying docker with Murano at the sprint closing chapter review (Monday July 6). 4. UPM is estimating the work effort to expose the Murano to the user on the cloud portal. This involves adapting the current blue print views to the Murano API (Issue CLD-584). Best Regards, Kenneth Nagin Ph: +972-4-8296227 Cell: 054-6976227 Fx: +972-4- 8296114 http://researcher.ibm.com/view.php?person=il-NAGIN From: Juanjo Hierro To: Alex Glikson/Haifa/IBM at IBMIL, < fiware-cloud-containers at lists.fiware.org> Date: 30/06/2015 11:06 PM Subject: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Sent by: fiware-cloud-containers-bounces at lists.fiware.org Dear all, Thanks Alex for launching these discussions tracks. Regarding point (1) as anticipated by Alex, here it is the concrete plan we aim at implementing to promote usage of docker tools by GE/SE owners and the broader developer community. 1. Make info about Docker images and instruction to setup docker containers linked to FIWARE GEris available in the FIWARE Catalogue: We will specify a general template of a new section which will be included in the "Creating instances" tab of entries linked to FIWARE GEris. This section will be elaborate on "Deploying a dedicated GE instance using Docker technology" Each FIWARE GEri owner will be asked to setup a Docker image for their GEs and register it in Docker Hub. This task can start in parallel to the previous one. Each FIWARE GEri owner will be asked to update the "Creating instance" tab of the entry linked to the FIWARE GEri so that it provides instructions about deploying a dedicated instance of the GEri using docker. For this purpose, the owner will follow the template defined in step 1. 2. We will incorporate the description about how to create FIWARE GEri instances and try them using docker within the FIWARE Tour Guide for developers (which will effectively become the landing page of http://developers.fiware.org) as a mean to shorten the learning curve with FIWARE GEris Somewhere at the beginning of the guided tour, we will explain the developer how he can setup the basic docker environment either locally or on the FIWARE Lab on which he will be able to rely to try the different FIWARE GEris (Group 1 scenarios described by Alex) A demo application will be developed that will help the developer to try each FIWARE GEri instance deployed using docker with concrete data. Developers will be able to deploy this application also using docker. It is the intention that this application will be helpful to show how several FIWARE GEris can be used in an integrated way. Each of the chapters of the FIWARE Tour guide for developers will integrate a "Try it yourself" section which will help the developer to create an instance of the FIWARE GEris referred in the chapter and try them. In addition, developers will be able to deploy the demo application, in order to learn from a more elaborated example how the FIWARE GEris can be used, and providing instructions about how to play with the FIWARE GEris, also "tweak" the application. Deployment of integrated FIWARE GEris and the demo application will be made feasible using docker composition tools Jos?-Manuel Cantera will drive the implementation of these concrete actions. Regarding point 2, Bitergia has already developed a first bundle of FIWARE GEris deployable using Docker compose (as a whole or individually). They will work in extending this bundle integrating additional FIWARE GEris in collaboration with the corresponding FIWARE GEri owners. Bitergia and the ULPG will work together in the development of a first version of the demo application. Eventually, additional partners might be incorporated as additional skills and/or resources are needed. We will address first those scenarios dealing with local deployment, then we will incorporate those related to deployment on other infrastructures such as the FIWARE Lab as soon as the necessary preliminary work and/or detailed guidelines are completed (this would be done under coordination by Alex and I assume we will leverage on experience developed in FI-Content2). Feedback is welcome. Best regards, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 30/06/15 20:34, Alex Glikson wrote: Dear partners, Following the initial discussion at the containers task force, we identified two follow-on (related) discussion tracks: 1) expected FIWARE Lab user/developer experience with Docker 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker (and enable requirements derived from #1) This email refers to topic (1). I will try to summarize the initial set of usage scenarios that we may want to support. We can then iterate on this over email, and have a phone call if needed, when we feel that we are close to a convergence point. Please, notice that I've subscribed several additional people relevant for the discussion (who could provide input related to UI, developers perspectives and operations perspectives). Assumptions: Notice that a prerequisite for all the usage scenarios is that all the FIWARE GEs (and SEs) are packages as Docker images and are kept in a central repository, preferably the docker hub (under a 'to-be-created' "fiware" namespace). Also, there is an assumption that the users/developers would want to work with Docker tools as much as possible (surfacing some or all of the capabilities via the FIWARE Cloud Portal too). Juanjo will elaborate on the approach we are thinking of to promote this with GE/SE owners and the broader developer community. Group 1: Basic Docker environment setup Usage scenario #1.1: A user wants a local Docker runtime on his laptop. He follows the standard instructions on setting up a docker host (e.g., running within a VirtalBox VM, using docker-machine / boot2docker) Usage scenario #1.2: A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab (where he would then run various Docker containers). He uses the standard docker-machine tool, specifying the URI of the OpenStack Keystone in FIWARE Lab (and additional parameters, as needed). The tool creates a VM using standard OpenStack APIs (natively supported by FIWARE Lab) and configures Docker within the VM. The VM would need to have a public IP (naturally). Group 2: Basic life cycle of individual containers running GEs/SEs Usage scenario #2.1: A developer wants to publish (a version of) a GE/SE. After he is done creating the new Docker image, he pushes the new version of the GE/SE to Docker hub under corresponding FIWARE namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the image points to the new version. Usage scenario #2.2: A user/developer wants to deploy locally an instance of a certain GE/SE. He uses the standard docker CLI to locally provision a container, referring to the corresponding image at Docker hub -- e.g.: "$ docker -H boot2docker-vm:2376 run fiware/GE/cb-orion" Usage scenario #2.3: A user/developer wants to deploy an instance of a certain GE/SE within his Docker VM on FIWARE Lab He uses the standard docker CLI to provision a container, referring to the location of his Docker VM as well as the corresponding image at Docker hub -- e.g.: "$ docker -H mydocker-vm37.lab.fiware.org:2376 run fiware/GE/cb-orion". ISSUE: the user would need to open the corresponding firewall ports in his VM (same as those of the GE/SE, or following the mapping performed during container provisioning) in order to make the GE/SE accessible. Ideally, this should be done in a scalable but secure manner. An easy solution is to open up-front (during VM provisioning) a range of ports (via setting up corresponding security group). A more advanced solution is to update the security group dynamically. Usage scenario #2.4: A user/developer wants to update a container comprising certain GE/SE with the latest version recently published in the dedicated namespace of the Docker Hub. He pulls the latest version from the Docker Hub, kills the old container, and starts a new one (attaching to the same resources). Note that this would work well when the application is properly designed for Docker (e.g., the container itself is stateless), and when the previous version of the container has been provisioned manually (as in #2, #4 above) by the developer (and he knows which resources to connect to). Group 3: Support for 'bundles' of GEs/SEs that together perform a certain complex function Usage scenario #3.1: A developer wants to publish (a version of) a 'bundle' of GEs/SEs that together perform a certain complex function. He creates a 'template' (e.g., following the format of docker-compose), referring to the individual GE/SE images as well as their interdependencies (e.g., links) and other composition properties. This might be done using a text editor, a Web UI provided by the FIWARE Cloud (conceptually similar to today's UI for creation of blueprints), or other tools from Docker ecosystem. He then uploads the template to the centralized templats repository. Note: it is likely that docker-compose will be able to use Docker Registry/Hub as a repository for templates (including versioning, push/pull, etc). Usage scenario #3.2: A user wants to provision locally a set of GEs/SEs, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the local Docker URL. Usage scenario #3.3: A user wants to provision a set of GEs/SEs in FIWARE Lab, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the target Docker URL in FIWARE Lab. Alternatively, he uses the FIWARE Portal UI to do the same (e.g., with Murano backend invoking docker-compose). Usage scenario #3.4: A user wants to update his Docker environment (local or on FIWARE Lab) with the latest version of a certain 'bundle' Note: may require enancements to docker-compose Group 4: Advanced scenarios Usage scenario #4.1: A user/developer wants to provision a cluster of VMs on FIWARE Lab that would host his Docker cluster (managed with Swarm or Kubernetes) He uses corresponding Murano/Heat template to provision the VMs and to configure the Docker/Swarm/Kubernetes cluster. Usage scenario #4.2: A user/developer wants to access a global instance of a Docker service in FIWARE Lab (shared, scalable, managed), so that he doesn't need to manage the corresponding VM(s) by himself. He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab Docker API endpoint with the standard Docker tools or FIWARE-specific tools (as outlined above ). Usage scenario #4.3: A user/developer wants to manage access control for Docker images among FIWARE Lab users. He starts using the Docker Registry/Hub deployed within the FIWARE Lab. Notice that this is a very initial list -- I am sure that there are many inaccuracies and gaps. Feel free to comment. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers -- -- Future Internet is closer than you think! http://www.fiware.org Official Mirantis partner for OpenStack Training https://www.create-net.org/community/openstack-training -- Dr. Federico M. Facca CREATE-NET Via alla Cascata 56/D 38123 Povo Trento (Italy) P +39 0461 312471 M +39 334 6049758 E federico.facca at create-net.org T @chicco785 W www.create-net.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From GLIKSON at il.ibm.com Wed Jul 8 15:41:58 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Wed, 8 Jul 2015 16:41:58 +0300 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: <559CD50A.9060403@telefonica.com> References: <5592F69E.4060803@telefonica.com> <559CB92E.3070002@dfki.de> <559CD50A.9060403@telefonica.com> Message-ID: Dear Juanjo, Philipp, all, Tomorrow would be a too short notice for me. Also, the idea was to iterate on the usage scenarios over email, and schedule a meeting to discuss the outcomes of the discussion, open questions and next steps. Regarding reuse of FIC2Lab assets, from user experience perspective -- there are two main approaches to consider. CLI-based and WebUI-based. >From CLI-based usage perspective -- we plan to adopt the approach FIC2Lab is proposing, using docker-machine etc, and make it the 'official' way to get started with Docker on FIWARE Lab (on all the nodes). In parallel, from the cloud hosting enablement perspective, we will start addressing the gaps which can not be easily addressed by this simplistic approach (very briefly outlined on the last slide of Kenneth's deck). >From WebUI perspective -- we would need to align with the existing FIWARE Portal UI. Also, we would need to choose underlying technologies and APIs for which we are confident enough regarding their long-term sustainability. I am not sure whether the Panamax-based solution developed in FIC2Lab meets these criteria. We can (and should) certainly discuss the options in more detail (here is the first point to discuss, I guess -- but we need to do the 'homework' first, being more precise of what usage scenarios from the list below we are trying to address, etc). In the very short term, we need to decide on priorities. For example, if our top-priority goal is to make it easy to deploy GEs/SEs and GE/SE bundles on FIWARE Lab using native docker CLI-based tools -- we can document how to do it (leveraging docker-machine and docker-compose, based on [1] and [2]), and then consider various simplification/automation tasks (we have started making a list of such potential enhancements). Regards, Alex [1] http://www.slideshare.net/knagin/simple-docker-hosting-on-fiware-lab [2] http://fic2.github.io/runner/ ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Juanjo Hierro To: Philipp Slusallek , Alex Glikson/Haifa/IBM at IBMIL, Date: 08/07/2015 10:45 AM Subject: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker I would suggest we setup a confcall to coordinate actions. Next architects confcall should be devoted to the discussion on POIs. We may schedule the next architects confcall on July 20th on this matter or, try to plan a confcall earlier. Next week it is going to be difficult for Jos? Manuel and myself. Jos? Manuel is travelling to Shangai this Friday and will come back Thursday next week. Then, Friday 17th wouldn't work for Israel We could make it on Thursday from 12:00 to 14:00 or from 14:30 on, starting not later than 16:00 ... otherwise go for the 20th. I have setup a doodle poll with the different options: http://doodle.com/66zyea2kiuh5vy98 Cheers, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 08/07/15 07:46, Philipp Slusallek wrote: Hi all, This all sounds good! But what happened to your planned next meetings. FIC2 is still highly interested in working with FIWARE to deploy the infrastructure developed there (running on FIWARE Lab) also for FIWARE. I thought the idea was to explore these options in a joint next call. Best, Philipp Am 30.06.2015 um 22:05 schrieb Juanjo Hierro: Dear all, Thanks Alex for launching these discussions tracks. Regarding point (1) as anticipated by Alex, here it is the concrete plan we aim at implementing to promote usage of docker tools by GE/SE owners and the broader developer community. 1. Make info about Docker images and instruction to setup docker containers linked to FIWARE GEris available in the FIWARE Catalogue: * We will specify a general template of a new section which will be included in the "Creating instances" tab of entries linked to FIWARE GEris. This section will be elaborate on "Deploying a dedicated GE instance using Docker technology" * Each FIWARE GEri owner will be asked to setup a Docker image for their GEs and register it in Docker Hub. This task can start in parallel to the previous one. * Each FIWARE GEri owner will be asked to update the "Creating instance" tab of the entry linked to the FIWARE GEri so that it provides instructions about deploying a dedicated instance of the GEri using docker. For this purpose, the owner will follow the template defined in step 1. 2. We will incorporate the description about how to create FIWARE GEri instances and try them using docker within the FIWARE Tour Guide for developers (which will effectively become the landing page of http://developers.fiware.org) as a mean to shorten the learning curve with FIWARE GEris * Somewhere at the beginning of the guided tour, we will explain the developer how he can setup the basic docker environment either locally or on the FIWARE Lab on which he will be able to rely to try the different FIWARE GEris (Group 1 scenarios described by Alex) * A demo application will be developed that will help the developer to try each FIWARE GEri instance deployed using docker with concrete data. Developers will be able to deploy this application also using docker. It is the intention that this application will be helpful to show how several FIWARE GEris can be used in an integrated way. * Each of the chapters of the FIWARE Tour guide for developers will integrate a "Try it yourself" section which will help the developer to create an instance of the FIWARE GEris referred in the chapter and try them. In addition, developers will be able to deploy the demo application, in order to learn from a more elaborated example how the FIWARE GEris can be used, and providing instructions about how to play with the FIWARE GEris, also "tweak" the application. * Deployment of integrated FIWARE GEris and the demo application will be made feasible using docker composition tools Jos?-Manuel Cantera will drive the implementation of these concrete actions. Regarding point 2, Bitergia has already developed a first bundle of FIWARE GEris deployable using Docker compose (as a whole or individually). They will work in extending this bundle integrating additional FIWARE GEris in collaboration with the corresponding FIWARE GEri owners. Bitergia and the ULPG will work together in the development of a first version of the demo application. Eventually, additional partners might be incorporated as additional skills and/or resources are needed. We will address first those scenarios dealing with local deployment, then we will incorporate those related to deployment on other infrastructures such as the FIWARE Lab as soon as the necessary preliminary work and/or detailed guidelines are completed (this would be done under coordination by Alex and I assume we will leverage on experience developed in FI-Content2). Feedback is welcome. Best regards, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 30/06/15 20:34, Alex Glikson wrote: Dear partners, Following the initial discussion at the containers task force, we identified two follow-on (related) discussion tracks: 1) expected FIWARE Lab user/developer experience with Docker 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker (and enable requirements derived from #1) This email refers to topic (1). I will try to summarize the initial set of usage scenarios that we may want to support. We can then iterate on this over email, and have a phone call if needed, when we feel that we are close to a convergence point. Please, notice that I've subscribed several additional people relevant for the discussion (who could provide input related to UI, developers perspectives and operations perspectives). *Assumptions*: Notice that a prerequisite for all the usage scenarios is that all the FIWARE GEs (and SEs) are packages as Docker images and are kept in a central repository, preferably the docker hub (under a 'to-be-created' "fiware" namespace). Also, there is an assumption that the users/developers would want to work with Docker tools as much as possible (surfacing some or all of the capabilities via the FIWARE Cloud Portal too). Juanjo will elaborate on the approach we are thinking of to promote this with GE/SE owners and the broader developer community. *Group 1: Basic Docker environment setup* Usage scenario #1.1: A user wants a local Docker runtime on his laptop. He follows the standard instructions on setting up a docker host (e.g., running within a VirtalBox VM, using docker-machine / boot2docker) Usage scenario #1.2: A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab (where he would then run various Docker containers). He uses the standard docker-machine tool, specifying the URI of the OpenStack Keystone in FIWARE Lab (and additional parameters, as needed). The tool creates a VM using standard OpenStack APIs (natively supported by FIWARE Lab) and configures Docker within the VM. The VM would need to have a public IP (naturally). *Group 2: Basic life cycle of individual containers running GEs/SEs* Usage scenario #2.1: A developer wants to publish (a version of) a GE/SE. After he is done creating the new Docker image, he pushes the new version of the GE/SE to Docker hub under corresponding FIWARE namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the image points to the new version. Usage scenario #2.2: A user/developer wants to deploy locally an instance of a certain GE/SE. He uses the standard docker CLI to locally provision a container, referring to the corresponding image at Docker hub -- e.g.: /"$ docker -H boot2docker-vm:2376 run fiware/GE/cb-orion"/ Usage scenario #2.3: A user/developer wants to deploy an instance of a certain GE/SE within his Docker VM on FIWARE Lab He uses the standard docker CLI to provision a container, referring to the location of his Docker VM as well as the corresponding image at Docker hub -- e.g.: /"$ docker -H mydocker-vm37.lab.fiware.org:2376 run fiware/GE/cb-orion". /ISSUE: the user would need to open the corresponding firewall ports in his VM (same as those of the GE/SE, or following the mapping performed during container provisioning) in order to make the GE/SE accessible. Ideally, this should be done in a scalable but secure manner. An easy solution is to open up-front (during VM provisioning) a range of ports (via setting up corresponding security group). A more advanced solution is to update the security group dynamically. Usage scenario #2.4: A user/developer wants to update a container comprising certain GE/SE with the latest version recently published in the dedicated namespace of the Docker Hub. He pulls the latest version from the Docker Hub, kills the old container, and starts a new one (attaching to the same resources). Note that this would work well when the application is properly designed for Docker (e.g., the container itself is stateless), and when the previous version of the container has been provisioned manually (as in #2, #4 above) by the developer (and he knows which resources to connect to). *Group 3: Support for 'bundles' of GEs/SEs that together perform a certain complex function* Usage scenario #3.1: A developer wants to publish (a version of) a 'bundle' of GEs/SEs that together perform a certain complex function. He creates a 'template' (e.g., following the format of docker-compose), referring to the individual GE/SE images as well as their interdependencies (e.g., links) and other composition properties. This might be done using a text editor, a Web UI provided by the FIWARE Cloud (conceptually similar to today's UI for creation of blueprints), or other tools from Docker ecosystem. He then uploads the template to the centralized templats repository. Note: it is likely that docker-compose will be able to use Docker Registry/Hub as a repository for templates (including versioning, push/pull, etc). Usage scenario #3.2: A user wants to provision locally a set of GEs/SEs, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the local Docker URL. Usage scenario #3.3: A user wants to provision a set of GEs/SEs in FIWARE Lab, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the target Docker URL in FIWARE Lab. Alternatively, he uses the FIWARE Portal UI to do the same (e.g., with Murano backend invoking docker-compose). Usage scenario #3.4: A user wants to update his Docker environment (local or on FIWARE Lab) with the latest version of a certain 'bundle' Note: may require enancements to docker-compose *Group 4: Advanced scenarios* Usage scenario #4.1: A user/developer wants to provision a cluster of VMs on FIWARE Lab that would host his Docker cluster (managed with Swarm or Kubernetes) He uses corresponding Murano/Heat template to provision the VMs and to configure the Docker/Swarm/Kubernetes cluster. Usage scenario #4.2: A user/developer wants to access a global instance of a Docker service in FIWARE Lab (shared, scalable, managed), so that he doesn't need to manage the corresponding VM(s) by himself. He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab Docker API endpoint with the standard Docker tools or FIWARE-specific tools (as outlined above ). Usage scenario #4.3: A user/developer wants to manage access control for Docker images among FIWARE Lab users. He starts using the Docker Registry/Hub deployed within the FIWARE Lab. Notice that this is a very initial list -- I am sure that there are many inaccuracies and gaps. Feel free to comment. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers ------------------------------------------------------------------------ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o -------------- next part -------------- An HTML attachment was scrubbed... URL: From juanjose.hierro at telefonica.com Thu Jul 9 08:36:13 2015 From: juanjose.hierro at telefonica.com (Juanjo Hierro) Date: Thu, 9 Jul 2015 08:36:13 +0200 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: <559CD50A.9060403@telefonica.com> References: <5592F69E.4060803@telefonica.com> <559CB92E.3070002@dfki.de> <559CD50A.9060403@telefonica.com> Message-ID: <559E165D.5090908@telefonica.com> An HTML attachment was scrubbed... URL: From philipp.slusallek at dfki.de Thu Jul 9 09:35:49 2015 From: philipp.slusallek at dfki.de (Philipp Slusallek) Date: Thu, 09 Jul 2015 09:35:49 +0200 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: <559E165D.5090908@telefonica.com> References: <5592F69E.4060803@telefonica.com> <559CB92E.3070002@dfki.de> <559CD50A.9060403@telefonica.com> <559E165D.5090908@telefonica.com> Message-ID: <559E2455.60706@dfki.de> Hi, I will join but have a PhD proposal talk until about 10h and so will be a bit late. Best, Philipp Am 09.07.2015 um 08:36 schrieb Juanjo Hierro: > Hi all, > > Based on results from the doodle poll, we will setup the confcall on > Monday 20th, from 09:30 to 11:30. > > Cheers, > > -- Juanjo > > ______________________________________________________ > > Coordinator and Chief Architect, FIWARE platform > CTO Industrial IoT, Telef?nica > > email: juanjose.hierro at telefonica.com > twitter: @JuanjoHierro > > You can follow FIWARE at: > website: http://www.fiware.org > twitter: @FIWARE > facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 > linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 > > On 08/07/15 09:45, Juanjo Hierro wrote: >> >> I would suggest we setup a confcall to coordinate actions. >> >> Next architects confcall should be devoted to the discussion on POIs. >> >> We may schedule the next architects confcall on July 20th on this >> matter or, try to plan a confcall earlier. >> >> Next week it is going to be difficult for Jos? Manuel and myself. >> Jos? Manuel is travelling to Shangai this Friday and will come back >> Thursday next week. Then, Friday 17th wouldn't work for Israel >> >> We could make it on Thursday from 12:00 to 14:00 or from 14:30 on, >> starting not later than 16:00 ... otherwise go for the 20th. >> >> I have setup a doodle poll with the different options: >> >> http://doodle.com/66zyea2kiuh5vy98 >> >> >> Cheers, >> >> -- Juanjo >> ______________________________________________________ >> >> Coordinator and Chief Architect, FIWARE platform >> CTO Industrial IoT, Telef?nica >> >> email: juanjose.hierro at telefonica.com >> twitter: @JuanjoHierro >> >> You can follow FIWARE at: >> website: http://www.fiware.org >> twitter: @FIWARE >> facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 >> linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 >> On 08/07/15 07:46, Philipp Slusallek wrote: >>> Hi all, >>> >>> This all sounds good! >>> >>> But what happened to your planned next meetings. FIC2 is still highly >>> interested in working with FIWARE to deploy the infrastructure developed >>> there (running on FIWARE Lab) also for FIWARE. >>> >>> I thought the idea was to explore these options in a joint next call. >>> >>> Best, >>> >>> Philipp >>> >>> Am 30.06.2015 um 22:05 schrieb Juanjo Hierro: >>>> Dear all, >>>> >>>> Thanks Alex for launching these discussions tracks. >>>> >>>> Regarding point (1) as anticipated by Alex, here it is the concrete >>>> plan we aim at implementing to promote usage of docker tools by GE/SE >>>> owners and the broader developer community. >>>> >>>> 1. Make info about Docker images and instruction to setup docker >>>> containers linked to FIWARE GEris available in the FIWARE Catalogue: >>>> * We will specify a general template of a new section which will >>>> be included in the "Creating instances" tab of entries linked to >>>> FIWARE GEris. This section will be elaborate on "Deploying a >>>> dedicated GE instance using Docker technology" >>>> * Each FIWARE GEri owner will be asked to setup a Docker image for >>>> their GEs and register it in Docker Hub. This task can start >>>> in parallel to the previous one. >>>> * Each FIWARE GEri owner will be asked to update the "Creating >>>> instance" tab of the entry linked to the FIWARE GEri so that it >>>> provides instructions about deploying a dedicated instance of >>>> the GEri using docker. For this purpose, the owner will follow >>>> the template defined in step 1. >>>> 2. We will incorporate the description about how to create FIWARE GEri >>>> instances and try them using docker within the FIWARE Tour Guide for >>>> developers (which will effectively become the landing page of >>>> http://developers.fiware.org) as a mean to shorten the learning >>>> curve with FIWARE GEris >>>> * Somewhere at the beginning of the guided tour, we will explain >>>> the developer how he can setup the basic docker environment >>>> either locally or on the FIWARE Lab on which he will be able to >>>> rely to try the different FIWARE GEris (Group 1 scenarios >>>> described by Alex) >>>> * A demo application will be developed that will help the >>>> developer to try each FIWARE GEri instance deployed using docker >>>> with concrete data. Developers will be able to deploy this >>>> application also using docker. It is the intention that this >>>> application will be helpful to show how several FIWARE GEris can >>>> be used in an integrated way. >>>> * Each of the chapters of the FIWARE Tour guide for developers >>>> will integrate a "Try it yourself" section which will help the >>>> developer to create an instance of the FIWARE GEris referred in >>>> the chapter and try them. In addition, developers will be able >>>> to deploy the demo application, in order to learn from a more >>>> elaborated example how the FIWARE GEris can be used, and >>>> providing instructions about how to play with the FIWARE GEris, >>>> also "tweak" the application. >>>> * Deployment of integrated FIWARE GEris and the demo application >>>> will be made feasible using docker composition tools >>>> >>>> >>>> Jos?-Manuel Cantera will drive the implementation of these concrete >>>> actions. >>>> >>>> Regarding point 2, Bitergia has already developed a first bundle of >>>> FIWARE GEris deployable using Docker compose (as a whole or >>>> individually). They will work in extending this bundle integrating >>>> additional FIWARE GEris in collaboration with the corresponding FIWARE >>>> GEri owners. Bitergia and the ULPG will work together in the >>>> development of a first version of the demo application. Eventually, >>>> additional partners might be incorporated as additional skills and/or >>>> resources are needed. We will address first those scenarios dealing >>>> with local deployment, then we will incorporate those related to >>>> deployment on other infrastructures such as the FIWARE Lab as soon as >>>> the necessary preliminary work and/or detailed guidelines are completed >>>> (this would be done under coordination by Alex and I assume we will >>>> leverage on experience developed in FI-Content2). >>>> >>>> Feedback is welcome. >>>> >>>> Best regards, >>>> >>>> -- Juanjo >>>> >>>> ______________________________________________________ >>>> >>>> Coordinator and Chief Architect, FIWARE platform >>>> CTO Industrial IoT, Telef?nica >>>> >>>> email: juanjose.hierro at telefonica.com >>>> twitter: @JuanjoHierro >>>> >>>> You can follow FIWARE at: >>>> website: http://www.fiware.org >>>> twitter: @FIWARE >>>> facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 >>>> linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 >>>> >>>> On 30/06/15 20:34, Alex Glikson wrote: >>>>> Dear partners, >>>>> >>>>> Following the initial discussion at the containers task force, we >>>>> identified two follow-on (related) discussion tracks: >>>>> 1) expected FIWARE Lab user/developer experience with Docker >>>>> 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker >>>>> (and enable requirements derived from #1) >>>>> >>>>> This email refers to topic (1). >>>>> >>>>> I will try to summarize the initial set of usage scenarios that we may >>>>> want to support. We can then iterate on this over email, and have a >>>>> phone call if needed, when we feel that we are close to a convergence >>>>> point. >>>>> >>>>> Please, notice that I've subscribed several additional people relevant >>>>> for the discussion (who could provide input related to UI, developers >>>>> perspectives and operations perspectives). >>>>> >>>>> *Assumptions*: >>>>> Notice that a prerequisite for all the usage scenarios is that all the >>>>> FIWARE GEs (and SEs) are packages as Docker images and are kept in a >>>>> central repository, preferably the docker hub (under a 'to-be-created' >>>>> "fiware" namespace). Also, there is an assumption that the >>>>> users/developers would want to work with Docker tools as much as >>>>> possible (surfacing some or all of the capabilities via the FIWARE >>>>> Cloud Portal too). Juanjo will elaborate on the approach we are >>>>> thinking of to promote this with GE/SE owners and the broader >>>>> developer community. >>>>> >>>>> *Group 1: Basic Docker environment setup* >>>>> Usage scenario #1.1: >>>>> A user wants a local Docker runtime on his laptop. >>>>> He follows the standard instructions on setting up a docker host >>>>> (e.g., running within a VirtalBox VM, using docker-machine / boot2docker) >>>>> >>>>> Usage scenario #1.2: >>>>> A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab >>>>> (where he would then run various Docker containers). >>>>> He uses the standard docker-machine tool, specifying the URI of the >>>>> OpenStack Keystone in FIWARE Lab (and additional parameters, as >>>>> needed). The tool creates a VM using standard OpenStack APIs (natively >>>>> supported by FIWARE Lab) and configures Docker within the VM. The VM >>>>> would need to have a public IP (naturally). >>>>> >>>>> *Group 2: Basic life cycle of individual containers running GEs/SEs* >>>>> Usage scenario #2.1: >>>>> A developer wants to publish (a version of) a GE/SE. >>>>> After he is done creating the new Docker image, he pushes the new >>>>> version of the GE/SE to Docker hub under corresponding FIWARE >>>>> namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the >>>>> image points to the new version. >>>>> >>>>> Usage scenario #2.2: >>>>> A user/developer wants to deploy locally an instance of a certain GE/SE. >>>>> He uses the standard docker CLI to locally provision a container, >>>>> referring to the corresponding image at Docker hub -- e.g.: /"$ docker >>>>> -H boot2docker-vm:2376 run fiware/GE/cb-orion"/ >>>>> >>>>> Usage scenario #2.3: >>>>> A user/developer wants to deploy an instance of a certain GE/SE within >>>>> his Docker VM on FIWARE Lab >>>>> He uses the standard docker CLI to provision a container, referring to >>>>> the location of his Docker VM as well as the corresponding image at >>>>> Docker hub -- e.g.: /"$ docker -H mydocker-vm37.lab.fiware.org:2376 >>>>> run fiware/GE/cb-orion". /ISSUE: the user would need to open the >>>>> corresponding firewall ports in his VM (same as those of the GE/SE, or >>>>> following the mapping performed during container provisioning) in >>>>> order to make the GE/SE accessible. Ideally, this should be done in a >>>>> scalable but secure manner. An easy solution is to open up-front >>>>> (during VM provisioning) a range of ports (via setting up >>>>> corresponding security group). A more advanced solution is to update >>>>> the security group dynamically. >>>>> >>>>> Usage scenario #2.4: >>>>> A user/developer wants to update a container comprising certain GE/SE >>>>> with the latest version recently published in the dedicated namespace >>>>> of the Docker Hub. >>>>> He pulls the latest version from the Docker Hub, kills the old >>>>> container, and starts a new one (attaching to the same resources). >>>>> Note that this would work well when the application is properly >>>>> designed for Docker (e.g., the container itself is stateless), and >>>>> when the previous version of the container has been provisioned >>>>> manually (as in #2, #4 above) by the developer (and he knows which >>>>> resources to connect to). >>>>> >>>>> *Group 3: Support for 'bundles' of GEs/SEs that together perform a >>>>> certain complex function* >>>>> Usage scenario #3.1: >>>>> A developer wants to publish (a version of) a 'bundle' of GEs/SEs that >>>>> together perform a certain complex function. >>>>> He creates a 'template' (e.g., following the format of >>>>> docker-compose), referring to the individual GE/SE images as well as >>>>> their interdependencies (e.g., links) and other composition >>>>> properties. This might be done using a text editor, a Web UI provided >>>>> by the FIWARE Cloud (conceptually similar to today's UI for creation >>>>> of blueprints), or other tools from Docker ecosystem. He then uploads >>>>> the template to the centralized templats repository. Note: it is >>>>> likely that docker-compose will be able to use Docker Registry/Hub as >>>>> a repository for templates (including versioning, push/pull, etc). >>>>> >>>>> Usage scenario #3.2: >>>>> A user wants to provision locally a set of GEs/SEs, using a >>>>> pre-defined template (comprising a 'bundle') >>>>> He uses the standard docker-compose tool referring to the 'template' >>>>> artifcat as well as the local Docker URL. >>>>> >>>>> Usage scenario #3.3: >>>>> A user wants to provision a set of GEs/SEs in FIWARE Lab, using a >>>>> pre-defined template (comprising a 'bundle') >>>>> He uses the standard docker-compose tool referring to the 'template' >>>>> artifcat as well as the target Docker URL in FIWARE Lab. >>>>> Alternatively, he uses the FIWARE Portal UI to do the same (e.g., with >>>>> Murano backend invoking docker-compose). >>>>> >>>>> Usage scenario #3.4: >>>>> A user wants to update his Docker environment (local or on FIWARE Lab) >>>>> with the latest version of a certain 'bundle' >>>>> Note: may require enancements to docker-compose >>>>> >>>>> *Group 4: Advanced scenarios* >>>>> Usage scenario #4.1: >>>>> A user/developer wants to provision a cluster of VMs on FIWARE Lab >>>>> that would host his Docker cluster (managed with Swarm or Kubernetes) >>>>> He uses corresponding Murano/Heat template to provision the VMs and to >>>>> configure the Docker/Swarm/Kubernetes cluster. >>>>> >>>>> Usage scenario #4.2: >>>>> A user/developer wants to access a global instance of a Docker service >>>>> in FIWARE Lab (shared, scalable, managed), so that he doesn't need to >>>>> manage the corresponding VM(s) by himself. >>>>> He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab >>>>> Docker API endpoint with the standard Docker tools or FIWARE-specific >>>>> tools (as outlined above ). >>>>> >>>>> Usage scenario #4.3: >>>>> A user/developer wants to manage access control for Docker images >>>>> among FIWARE Lab users. >>>>> He starts using the Docker Registry/Hub deployed within the FIWARE Lab. >>>>> >>>>> >>>>> Notice that this is a very initial list -- I am sure that there are >>>>> many inaccuracies and gaps. Feel free to comment. >>>>> >>>>> Thanks, >>>>> Alex >>>>> >>>>> ==================================================================================== >>>>> Alex Glikson >>>>> Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab >>>>> Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: >>>>> +972-54-6466667 | Fax: +972-4-8296112 >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Fiware-cloud-containers mailing list >>>>> Fiware-cloud-containers at lists.fiware.org >>>>> https://lists.fiware.org/listinfo/fiware-cloud-containers >>>> ------------------------------------------------------------------------ >>>> >>>> Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, >>>> puede contener informaci?n privilegiada o confidencial y es para uso >>>> exclusivo de la persona o entidad de destino. Si no es usted. el >>>> destinatario indicado, queda notificado de que la lectura, utilizaci?n, >>>> divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud >>>> de la legislaci?n vigente. Si ha recibido este mensaje por error, le >>>> rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda >>>> a su destrucci?n. >>>> >>>> The information contained in this transmission is privileged and >>>> confidential information intended only for the use of the individual or >>>> entity named above. If the reader of this message is not the intended >>>> recipient, you are hereby notified that any dissemination, distribution >>>> or copying of this communication is strictly prohibited. If you have >>>> received this transmission in error, do not read it. Please immediately >>>> reply to the sender that you have received this communication in error >>>> and then delete it. >>>> >>>> Esta mensagem e seus anexos se dirigem exclusivamente ao seu >>>> destinat?rio, pode conter informa??o privilegiada ou confidencial e ? >>>> para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa >>>> senhoria o destinat?rio indicado, fica notificado de que a leitura, >>>> utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em >>>> virtude da legisla??o vigente. Se recebeu esta mensagem por erro, >>>> rogamos-lhe que nos o comunique imediatamente por esta mesma via e >>>> proceda a sua destrui??o >>>> >>>> >>>> _______________________________________________ >>>> Fiware-cloud-containers mailing list >>>> Fiware-cloud-containers at lists.fiware.org >>>> https://lists.fiware.org/listinfo/fiware-cloud-containers >>>> >> >> >> ------------------------------------------------------------------------ >> >> Este mensaje y sus adjuntos se dirigen exclusivamente a su >> destinatario, puede contener informaci?n privilegiada o confidencial y >> es para uso exclusivo de la persona o entidad de destino. Si no es >> usted. el destinatario indicado, queda notificado de que la lectura, >> utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar >> prohibida en virtud de la legislaci?n vigente. Si ha recibido este >> mensaje por error, le rogamos que nos lo comunique inmediatamente por >> esta misma v?a y proceda a su destrucci?n. >> >> The information contained in this transmission is privileged and >> confidential information intended only for the use of the individual >> or entity named above. If the reader of this message is not the >> intended recipient, you are hereby notified that any dissemination, >> distribution or copying of this communication is strictly prohibited. >> If you have received this transmission in error, do not read it. >> Please immediately reply to the sender that you have received this >> communication in error and then delete it. >> >> Esta mensagem e seus anexos se dirigem exclusivamente ao seu >> destinat?rio, pode conter informa??o privilegiada ou confidencial e ? >> para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa >> senhoria o destinat?rio indicado, fica notificado de que a leitura, >> utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida >> em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, >> rogamos-lhe que nos o comunique imediatamente por esta mesma via e >> proceda a sua destrui??o >> >> >> _______________________________________________ >> Fiware-cloud-containers mailing list >> Fiware-cloud-containers at lists.fiware.org >> https://lists.fiware.org/listinfo/fiware-cloud-containers > > > ------------------------------------------------------------------------ > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener informaci?n privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilizaci?n, > divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud > de la legislaci?n vigente. Si ha recibido este mensaje por error, le > rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda > a su destrucci?n. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution > or copying of this communication is strictly prohibited. If you have > received this transmission in error, do not read it. Please immediately > reply to the sender that you have received this communication in error > and then delete it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu > destinat?rio, pode conter informa??o privilegiada ou confidencial e ? > para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa > senhoria o destinat?rio indicado, fica notificado de que a leitura, > utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em > virtude da legisla??o vigente. Se recebeu esta mensagem por erro, > rogamos-lhe que nos o comunique imediatamente por esta mesma via e > proceda a sua destrui??o -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: philipp_slusallek.vcf Type: text/x-vcard Size: 441 bytes Desc: not available URL: From mario.lopezramos at thalesgroup.com Thu Jul 9 09:52:24 2015 From: mario.lopezramos at thalesgroup.com (LOPEZ RAMOS Mario) Date: Thu, 9 Jul 2015 09:52:24 +0200 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: References: <5592F69E.4060803@telefonica.com> <559CB92E.3070002@dfki.de> <559CD50A.9060403@telefonica.com> Message-ID: Dear Alex and all, I agree with your approach: - documentation and examples of a CLI-based solution first (with potential simplification/automation tasks), - and later a WebUI-based solution that needs to be further discussed. Could you share with us the list of the potential enhancements you are considering for the CLI-based solution? A Google Doc would be a good way for us to add our suggestions. I will join the telco on the 20th. Best regards, Mario De : fiware-cloud-containers-bounces at lists.fiware.org [mailto:fiware-cloud-containers-bounces at lists.fiware.org] De la part de Alex Glikson Envoy? : mercredi 8 juillet 2015 15:42 ? : Juanjo Hierro Cc : fiware-cloud-containers at lists.fiware.org Objet : Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Dear Juanjo, Philipp, all, Tomorrow would be a too short notice for me. Also, the idea was to iterate on the usage scenarios over email, and schedule a meeting to discuss the outcomes of the discussion, open questions and next steps. Regarding reuse of FIC2Lab assets, from user experience perspective -- there are two main approaches to consider. CLI-based and WebUI-based. >From CLI-based usage perspective -- we plan to adopt the approach FIC2Lab is proposing, using docker-machine etc, and make it the 'official' way to get started with Docker on FIWARE Lab (on all the nodes). In parallel, from the cloud hosting enablement perspective, we will start addressing the gaps which can not be easily addressed by this simplistic approach (very briefly outlined on the last slide of Kenneth's deck). >From WebUI perspective -- we would need to align with the existing FIWARE Portal UI. Also, we would need to choose underlying technologies and APIs for which we are confident enough regarding their long-term sustainability. I am not sure whether the Panamax-based solution developed in FIC2Lab meets these criteria. We can (and should) certainly discuss the options in more detail (here is the first point to discuss, I guess -- but we need to do the 'homework' first, being more precise of what usage scenarios from the list below we are trying to address, etc). In the very short term, we need to decide on priorities. For example, if our top-priority goal is to make it easy to deploy GEs/SEs and GE/SE bundles on FIWARE Lab using native docker CLI-based tools -- we can document how to do it (leveraging docker-machine and docker-compose, based on [1] and [2]), and then consider various simplification/automation tasks (we have started making a list of such potential enhancements). Regards, Alex [1] http://www.slideshare.net/knagin/simple-docker-hosting-on-fiware-lab [2] http://fic2.github.io/runner/ ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Juanjo Hierro > To: Philipp Slusallek >, Alex Glikson/Haifa/IBM at IBMIL, > Date: 08/07/2015 10:45 AM Subject: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker ________________________________ I would suggest we setup a confcall to coordinate actions. Next architects confcall should be devoted to the discussion on POIs. We may schedule the next architects confcall on July 20th on this matter or, try to plan a confcall earlier. Next week it is going to be difficult for Jos? Manuel and myself. Jos? Manuel is travelling to Shangai this Friday and will come back Thursday next week. Then, Friday 17th wouldn't work for Israel We could make it on Thursday from 12:00 to 14:00 or from 14:30 on, starting not later than 16:00 ... otherwise go for the 20th. I have setup a doodle poll with the different options: http://doodle.com/66zyea2kiuh5vy98 Cheers, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 08/07/15 07:46, Philipp Slusallek wrote: Hi all, This all sounds good! But what happened to your planned next meetings. FIC2 is still highly interested in working with FIWARE to deploy the infrastructure developed there (running on FIWARE Lab) also for FIWARE. I thought the idea was to explore these options in a joint next call. Best, Philipp Am 30.06.2015 um 22:05 schrieb Juanjo Hierro: Dear all, Thanks Alex for launching these discussions tracks. Regarding point (1) as anticipated by Alex, here it is the concrete plan we aim at implementing to promote usage of docker tools by GE/SE owners and the broader developer community. 1. Make info about Docker images and instruction to setup docker containers linked to FIWARE GEris available in the FIWARE Catalogue: * We will specify a general template of a new section which will be included in the "Creating instances" tab of entries linked to FIWARE GEris. This section will be elaborate on "Deploying a dedicated GE instance using Docker technology" * Each FIWARE GEri owner will be asked to setup a Docker image for their GEs and register it in Docker Hub. This task can start in parallel to the previous one. * Each FIWARE GEri owner will be asked to update the "Creating instance" tab of the entry linked to the FIWARE GEri so that it provides instructions about deploying a dedicated instance of the GEri using docker. For this purpose, the owner will follow the template defined in step 1. 2. We will incorporate the description about how to create FIWARE GEri instances and try them using docker within the FIWARE Tour Guide for developers (which will effectively become the landing page of http://developers.fiware.org) as a mean to shorten the learning curve with FIWARE GEris * Somewhere at the beginning of the guided tour, we will explain the developer how he can setup the basic docker environment either locally or on the FIWARE Lab on which he will be able to rely to try the different FIWARE GEris (Group 1 scenarios described by Alex) * A demo application will be developed that will help the developer to try each FIWARE GEri instance deployed using docker with concrete data. Developers will be able to deploy this application also using docker. It is the intention that this application will be helpful to show how several FIWARE GEris can be used in an integrated way. * Each of the chapters of the FIWARE Tour guide for developers will integrate a "Try it yourself" section which will help the developer to create an instance of the FIWARE GEris referred in the chapter and try them. In addition, developers will be able to deploy the demo application, in order to learn from a more elaborated example how the FIWARE GEris can be used, and providing instructions about how to play with the FIWARE GEris, also "tweak" the application. * Deployment of integrated FIWARE GEris and the demo application will be made feasible using docker composition tools Jos?-Manuel Cantera will drive the implementation of these concrete actions. Regarding point 2, Bitergia has already developed a first bundle of FIWARE GEris deployable using Docker compose (as a whole or individually). They will work in extending this bundle integrating additional FIWARE GEris in collaboration with the corresponding FIWARE GEri owners. Bitergia and the ULPG will work together in the development of a first version of the demo application. Eventually, additional partners might be incorporated as additional skills and/or resources are needed. We will address first those scenarios dealing with local deployment, then we will incorporate those related to deployment on other infrastructures such as the FIWARE Lab as soon as the necessary preliminary work and/or detailed guidelines are completed (this would be done under coordination by Alex and I assume we will leverage on experience developed in FI-Content2). Feedback is welcome. Best regards, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 30/06/15 20:34, Alex Glikson wrote: Dear partners, Following the initial discussion at the containers task force, we identified two follow-on (related) discussion tracks: 1) expected FIWARE Lab user/developer experience with Docker 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker (and enable requirements derived from #1) This email refers to topic (1). I will try to summarize the initial set of usage scenarios that we may want to support. We can then iterate on this over email, and have a phone call if needed, when we feel that we are close to a convergence point. Please, notice that I've subscribed several additional people relevant for the discussion (who could provide input related to UI, developers perspectives and operations perspectives). *Assumptions*: Notice that a prerequisite for all the usage scenarios is that all the FIWARE GEs (and SEs) are packages as Docker images and are kept in a central repository, preferably the docker hub (under a 'to-be-created' "fiware" namespace). Also, there is an assumption that the users/developers would want to work with Docker tools as much as possible (surfacing some or all of the capabilities via the FIWARE Cloud Portal too). Juanjo will elaborate on the approach we are thinking of to promote this with GE/SE owners and the broader developer community. *Group 1: Basic Docker environment setup* Usage scenario #1.1: A user wants a local Docker runtime on his laptop. He follows the standard instructions on setting up a docker host (e.g., running within a VirtalBox VM, using docker-machine / boot2docker) Usage scenario #1.2: A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab (where he would then run various Docker containers). He uses the standard docker-machine tool, specifying the URI of the OpenStack Keystone in FIWARE Lab (and additional parameters, as needed). The tool creates a VM using standard OpenStack APIs (natively supported by FIWARE Lab) and configures Docker within the VM. The VM would need to have a public IP (naturally). *Group 2: Basic life cycle of individual containers running GEs/SEs* Usage scenario #2.1: A developer wants to publish (a version of) a GE/SE. After he is done creating the new Docker image, he pushes the new version of the GE/SE to Docker hub under corresponding FIWARE namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the image points to the new version. Usage scenario #2.2: A user/developer wants to deploy locally an instance of a certain GE/SE. He uses the standard docker CLI to locally provision a container, referring to the corresponding image at Docker hub -- e.g.: /"$ docker -H boot2docker-vm:2376 run fiware/GE/cb-orion"/ Usage scenario #2.3: A user/developer wants to deploy an instance of a certain GE/SE within his Docker VM on FIWARE Lab He uses the standard docker CLI to provision a container, referring to the location of his Docker VM as well as the corresponding image at Docker hub -- e.g.: /"$ docker -H mydocker-vm37.lab.fiware.org:2376 run fiware/GE/cb-orion". /ISSUE: the user would need to open the corresponding firewall ports in his VM (same as those of the GE/SE, or following the mapping performed during container provisioning) in order to make the GE/SE accessible. Ideally, this should be done in a scalable but secure manner. An easy solution is to open up-front (during VM provisioning) a range of ports (via setting up corresponding security group). A more advanced solution is to update the security group dynamically. Usage scenario #2.4: A user/developer wants to update a container comprising certain GE/SE with the latest version recently published in the dedicated namespace of the Docker Hub. He pulls the latest version from the Docker Hub, kills the old container, and starts a new one (attaching to the same resources). Note that this would work well when the application is properly designed for Docker (e.g., the container itself is stateless), and when the previous version of the container has been provisioned manually (as in #2, #4 above) by the developer (and he knows which resources to connect to). *Group 3: Support for 'bundles' of GEs/SEs that together perform a certain complex function* Usage scenario #3.1: A developer wants to publish (a version of) a 'bundle' of GEs/SEs that together perform a certain complex function. He creates a 'template' (e.g., following the format of docker-compose), referring to the individual GE/SE images as well as their interdependencies (e.g., links) and other composition properties. This might be done using a text editor, a Web UI provided by the FIWARE Cloud (conceptually similar to today's UI for creation of blueprints), or other tools from Docker ecosystem. He then uploads the template to the centralized templats repository. Note: it is likely that docker-compose will be able to use Docker Registry/Hub as a repository for templates (including versioning, push/pull, etc). Usage scenario #3.2: A user wants to provision locally a set of GEs/SEs, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the local Docker URL. Usage scenario #3.3: A user wants to provision a set of GEs/SEs in FIWARE Lab, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the target Docker URL in FIWARE Lab. Alternatively, he uses the FIWARE Portal UI to do the same (e.g., with Murano backend invoking docker-compose). Usage scenario #3.4: A user wants to update his Docker environment (local or on FIWARE Lab) with the latest version of a certain 'bundle' Note: may require enancements to docker-compose *Group 4: Advanced scenarios* Usage scenario #4.1: A user/developer wants to provision a cluster of VMs on FIWARE Lab that would host his Docker cluster (managed with Swarm or Kubernetes) He uses corresponding Murano/Heat template to provision the VMs and to configure the Docker/Swarm/Kubernetes cluster. Usage scenario #4.2: A user/developer wants to access a global instance of a Docker service in FIWARE Lab (shared, scalable, managed), so that he doesn't need to manage the corresponding VM(s) by himself. He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab Docker API endpoint with the standard Docker tools or FIWARE-specific tools (as outlined above ). Usage scenario #4.3: A user/developer wants to manage access control for Docker images among FIWARE Lab users. He starts using the Docker Registry/Hub deployed within the FIWARE Lab. Notice that this is a very initial list -- I am sure that there are many inaccuracies and gaps. Feel free to comment. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers ------------------------------------------------------------------------ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers ________________________________ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o -------------- next part -------------- An HTML attachment was scrubbed... URL: From GLIKSON at il.ibm.com Sun Jul 19 10:05:20 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Sun, 19 Jul 2015 11:05:20 +0300 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: <559E165D.5090908@telefonica.com> References: <5592F69E.4060803@telefonica.com> <559CB92E.3070002@dfki.de> <559CD50A.9060403@telefonica.com> <559E165D.5090908@telefonica.com> Message-ID: Can we do it 11am-1pm? Or maybe 11-12:30? Doodle indicated that this slot is available too. Juanjo -- were you going to update the architects meeting invitation (inviting additional people, etc), or should I send a separate one, with my bridge details? Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab; FIWARE Cloud Leader & Architect Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Juanjo Hierro To: Philipp Slusallek , Alex Glikson/Haifa/IBM at IBMIL, Date: 09/07/2015 09:36 AM Subject: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Hi all, Based on results from the doodle poll, we will setup the confcall on Monday 20th, from 09:30 to 11:30. Cheers, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 08/07/15 09:45, Juanjo Hierro wrote: I would suggest we setup a confcall to coordinate actions. Next architects confcall should be devoted to the discussion on POIs. We may schedule the next architects confcall on July 20th on this matter or, try to plan a confcall earlier. Next week it is going to be difficult for Jos? Manuel and myself. Jos? Manuel is travelling to Shangai this Friday and will come back Thursday next week. Then, Friday 17th wouldn't work for Israel We could make it on Thursday from 12:00 to 14:00 or from 14:30 on, starting not later than 16:00 ... otherwise go for the 20th. I have setup a doodle poll with the different options: http://doodle.com/66zyea2kiuh5vy98 Cheers, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 08/07/15 07:46, Philipp Slusallek wrote: Hi all, This all sounds good! But what happened to your planned next meetings. FIC2 is still highly interested in working with FIWARE to deploy the infrastructure developed there (running on FIWARE Lab) also for FIWARE. I thought the idea was to explore these options in a joint next call. Best, Philipp Am 30.06.2015 um 22:05 schrieb Juanjo Hierro: Dear all, Thanks Alex for launching these discussions tracks. Regarding point (1) as anticipated by Alex, here it is the concrete plan we aim at implementing to promote usage of docker tools by GE/SE owners and the broader developer community. 1. Make info about Docker images and instruction to setup docker containers linked to FIWARE GEris available in the FIWARE Catalogue: * We will specify a general template of a new section which will be included in the "Creating instances" tab of entries linked to FIWARE GEris. This section will be elaborate on "Deploying a dedicated GE instance using Docker technology" * Each FIWARE GEri owner will be asked to setup a Docker image for their GEs and register it in Docker Hub. This task can start in parallel to the previous one. * Each FIWARE GEri owner will be asked to update the "Creating instance" tab of the entry linked to the FIWARE GEri so that it provides instructions about deploying a dedicated instance of the GEri using docker. For this purpose, the owner will follow the template defined in step 1. 2. We will incorporate the description about how to create FIWARE GEri instances and try them using docker within the FIWARE Tour Guide for developers (which will effectively become the landing page of http://developers.fiware.org) as a mean to shorten the learning curve with FIWARE GEris * Somewhere at the beginning of the guided tour, we will explain the developer how he can setup the basic docker environment either locally or on the FIWARE Lab on which he will be able to rely to try the different FIWARE GEris (Group 1 scenarios described by Alex) * A demo application will be developed that will help the developer to try each FIWARE GEri instance deployed using docker with concrete data. Developers will be able to deploy this application also using docker. It is the intention that this application will be helpful to show how several FIWARE GEris can be used in an integrated way. * Each of the chapters of the FIWARE Tour guide for developers will integrate a "Try it yourself" section which will help the developer to create an instance of the FIWARE GEris referred in the chapter and try them. In addition, developers will be able to deploy the demo application, in order to learn from a more elaborated example how the FIWARE GEris can be used, and providing instructions about how to play with the FIWARE GEris, also "tweak" the application. * Deployment of integrated FIWARE GEris and the demo application will be made feasible using docker composition tools Jos?-Manuel Cantera will drive the implementation of these concrete actions. Regarding point 2, Bitergia has already developed a first bundle of FIWARE GEris deployable using Docker compose (as a whole or individually). They will work in extending this bundle integrating additional FIWARE GEris in collaboration with the corresponding FIWARE GEri owners. Bitergia and the ULPG will work together in the development of a first version of the demo application. Eventually, additional partners might be incorporated as additional skills and/or resources are needed. We will address first those scenarios dealing with local deployment, then we will incorporate those related to deployment on other infrastructures such as the FIWARE Lab as soon as the necessary preliminary work and/or detailed guidelines are completed (this would be done under coordination by Alex and I assume we will leverage on experience developed in FI-Content2). Feedback is welcome. Best regards, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 30/06/15 20:34, Alex Glikson wrote: Dear partners, Following the initial discussion at the containers task force, we identified two follow-on (related) discussion tracks: 1) expected FIWARE Lab user/developer experience with Docker 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker (and enable requirements derived from #1) This email refers to topic (1). I will try to summarize the initial set of usage scenarios that we may want to support. We can then iterate on this over email, and have a phone call if needed, when we feel that we are close to a convergence point. Please, notice that I've subscribed several additional people relevant for the discussion (who could provide input related to UI, developers perspectives and operations perspectives). *Assumptions*: Notice that a prerequisite for all the usage scenarios is that all the FIWARE GEs (and SEs) are packages as Docker images and are kept in a central repository, preferably the docker hub (under a 'to-be-created' "fiware" namespace). Also, there is an assumption that the users/developers would want to work with Docker tools as much as possible (surfacing some or all of the capabilities via the FIWARE Cloud Portal too). Juanjo will elaborate on the approach we are thinking of to promote this with GE/SE owners and the broader developer community. *Group 1: Basic Docker environment setup* Usage scenario #1.1: A user wants a local Docker runtime on his laptop. He follows the standard instructions on setting up a docker host (e.g., running within a VirtalBox VM, using docker-machine / boot2docker) Usage scenario #1.2: A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab (where he would then run various Docker containers). He uses the standard docker-machine tool, specifying the URI of the OpenStack Keystone in FIWARE Lab (and additional parameters, as needed). The tool creates a VM using standard OpenStack APIs (natively supported by FIWARE Lab) and configures Docker within the VM. The VM would need to have a public IP (naturally). *Group 2: Basic life cycle of individual containers running GEs/SEs* Usage scenario #2.1: A developer wants to publish (a version of) a GE/SE. After he is done creating the new Docker image, he pushes the new version of the GE/SE to Docker hub under corresponding FIWARE namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the image points to the new version. Usage scenario #2.2: A user/developer wants to deploy locally an instance of a certain GE/SE. He uses the standard docker CLI to locally provision a container, referring to the corresponding image at Docker hub -- e.g.: /"$ docker -H boot2docker-vm:2376 run fiware/GE/cb-orion"/ Usage scenario #2.3: A user/developer wants to deploy an instance of a certain GE/SE within his Docker VM on FIWARE Lab He uses the standard docker CLI to provision a container, referring to the location of his Docker VM as well as the corresponding image at Docker hub -- e.g.: /"$ docker -H mydocker-vm37.lab.fiware.org:2376 run fiware/GE/cb-orion". /ISSUE: the user would need to open the corresponding firewall ports in his VM (same as those of the GE/SE, or following the mapping performed during container provisioning) in order to make the GE/SE accessible. Ideally, this should be done in a scalable but secure manner. An easy solution is to open up-front (during VM provisioning) a range of ports (via setting up corresponding security group). A more advanced solution is to update the security group dynamically. Usage scenario #2.4: A user/developer wants to update a container comprising certain GE/SE with the latest version recently published in the dedicated namespace of the Docker Hub. He pulls the latest version from the Docker Hub, kills the old container, and starts a new one (attaching to the same resources). Note that this would work well when the application is properly designed for Docker (e.g., the container itself is stateless), and when the previous version of the container has been provisioned manually (as in #2, #4 above) by the developer (and he knows which resources to connect to). *Group 3: Support for 'bundles' of GEs/SEs that together perform a certain complex function* Usage scenario #3.1: A developer wants to publish (a version of) a 'bundle' of GEs/SEs that together perform a certain complex function. He creates a 'template' (e.g., following the format of docker-compose), referring to the individual GE/SE images as well as their interdependencies (e.g., links) and other composition properties. This might be done using a text editor, a Web UI provided by the FIWARE Cloud (conceptually similar to today's UI for creation of blueprints), or other tools from Docker ecosystem. He then uploads the template to the centralized templats repository. Note: it is likely that docker-compose will be able to use Docker Registry/Hub as a repository for templates (including versioning, push/pull, etc). Usage scenario #3.2: A user wants to provision locally a set of GEs/SEs, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the local Docker URL. Usage scenario #3.3: A user wants to provision a set of GEs/SEs in FIWARE Lab, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the target Docker URL in FIWARE Lab. Alternatively, he uses the FIWARE Portal UI to do the same (e.g., with Murano backend invoking docker-compose). Usage scenario #3.4: A user wants to update his Docker environment (local or on FIWARE Lab) with the latest version of a certain 'bundle' Note: may require enancements to docker-compose *Group 4: Advanced scenarios* Usage scenario #4.1: A user/developer wants to provision a cluster of VMs on FIWARE Lab that would host his Docker cluster (managed with Swarm or Kubernetes) He uses corresponding Murano/Heat template to provision the VMs and to configure the Docker/Swarm/Kubernetes cluster. Usage scenario #4.2: A user/developer wants to access a global instance of a Docker service in FIWARE Lab (shared, scalable, managed), so that he doesn't need to manage the corresponding VM(s) by himself. He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab Docker API endpoint with the standard Docker tools or FIWARE-specific tools (as outlined above ). Usage scenario #4.3: A user/developer wants to manage access control for Docker images among FIWARE Lab users. He starts using the Docker Registry/Hub deployed within the FIWARE Lab. Notice that this is a very initial list -- I am sure that there are many inaccuracies and gaps. Feel free to comment. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers ------------------------------------------------------------------------ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o -------------- next part -------------- An HTML attachment was scrubbed... URL: From philipp.slusallek at dfki.de Mon Jul 20 07:18:47 2015 From: philipp.slusallek at dfki.de (Philipp Slusallek) Date: Mon, 20 Jul 2015 07:18:47 +0200 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: References: <5592F69E.4060803@telefonica.com> <559CB92E.3070002@dfki.de> <559CD50A.9060403@telefonica.com> <559E165D.5090908@telefonica.com> Message-ID: <55AC84B7.4080705@dfki.de> Hi, I have a lecture from 10-12 so cannot really attend either way. But see my other emails. Best, Philipp Am 19.07.2015 um 10:05 schrieb Alex Glikson: > Can we do it 11am-1pm? Or maybe 11-12:30? Doodle indicated that this > slot is available too. > Juanjo -- were you going to update the architects meeting invitation > (inviting additional people, etc), or should I send a separate one, with > my bridge details? > > Thanks, > Alex > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab; FIWARE > Cloud Leader & Architect > Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > > > > From: Juanjo Hierro > To: Philipp Slusallek , Alex > Glikson/Haifa/IBM at IBMIL, > Date: 09/07/2015 09:36 AM > Subject: Re: [Fiware-cloud-containers] FIWARE developer > experience with Docker > ------------------------------------------------------------------------ > > > > Hi all, > > Based on results from the doodle poll, we will setup the confcall on > Monday 20th, from 09:30 to 11:30. > > Cheers, > > -- Juanjo > ______________________________________________________ > > Coordinator and Chief Architect, FIWARE platform > CTO Industrial IoT, Telef?nica > > email: _juanjose.hierro at telefonica.com_ > > twitter: @JuanjoHierro > > You can follow FIWARE at: > website: _http://www.fiware.org_ > twitter: @FIWARE > facebook: _http://www.facebook.com/pages/FI-WARE/251366491587242_ > linkedIn: _http://www.linkedin.com/groups/FIWARE-4239932_ > > On 08/07/15 09:45, Juanjo Hierro wrote: > > I would suggest we setup a confcall to coordinate actions. > > Next architects confcall should be devoted to the discussion on POIs. > > We may schedule the next architects confcall on July 20th on this > matter or, try to plan a confcall earlier. > > Next week it is going to be difficult for Jos? Manuel and myself. > Jos? Manuel is travelling to Shangai this Friday and will come back > Thursday next week. Then, Friday 17th wouldn't work for Israel > > We could make it on Thursday from 12:00 to 14:00 or from 14:30 on, > starting not later than 16:00 ... otherwise go for the 20th. > > I have setup a doodle poll with the different options: > _http://doodle.com/66zyea2kiuh5vy98_ > > Cheers, > > -- Juanjo > ______________________________________________________ > > Coordinator and Chief Architect, FIWARE platform > CTO Industrial IoT, Telef?nica > > email: _juanjose.hierro at telefonica.com_ > > twitter: @JuanjoHierro > > You can follow FIWARE at: > website: _http://www.fiware.org_ > twitter: @FIWARE > facebook: _http://www.facebook.com/pages/FI-WARE/251366491587242_ > linkedIn: _http://www.linkedin.com/groups/FIWARE-4239932_ > > On 08/07/15 07:46, Philipp Slusallek wrote: > Hi all, > > This all sounds good! > > But what happened to your planned next meetings. FIC2 is still highly > interested in working with FIWARE to deploy the infrastructure developed > there (running on FIWARE Lab) also for FIWARE. > > I thought the idea was to explore these options in a joint next call. > > Best, > > Philipp > > Am 30.06.2015 um 22:05 schrieb Juanjo Hierro: > > Dear all, > > Thanks Alex for launching these discussions tracks. > > Regarding point (1) as anticipated by Alex, here it is the concrete > plan we aim at implementing to promote usage of docker tools by GE/SE > owners and the broader developer community. > > 1. Make info about Docker images and instruction to setup docker > containers linked to FIWARE GEris available in the FIWARE Catalogue: > * We will specify a general template of a new section which will > be included in the "Creating instances" tab of entries linked to > FIWARE GEris. This section will be elaborate on "Deploying a > dedicated GE instance using Docker technology" > * Each FIWARE GEri owner will be asked to setup a Docker image for > their GEs and register it in Docker Hub. This task can start > in parallel to the previous one. > * Each FIWARE GEri owner will be asked to update the "Creating > instance" tab of the entry linked to the FIWARE GEri so that it > provides instructions about deploying a dedicated instance of > the GEri using docker. For this purpose, the owner will follow > the template defined in step 1. > 2. We will incorporate the description about how to create FIWARE GEri > instances and try them using docker within the FIWARE Tour Guide for > developers (which will effectively become the landing page of > _http://developers.fiware.org_ ) as a > mean to shorten the learning > curve with FIWARE GEris > * Somewhere at the beginning of the guided tour, we will explain > the developer how he can setup the basic docker environment > either locally or on the FIWARE Lab on which he will be able to > rely to try the different FIWARE GEris (Group 1 scenarios > described by Alex) > * A demo application will be developed that will help the > developer to try each FIWARE GEri instance deployed using docker > with concrete data. Developers will be able to deploy this > application also using docker. It is the intention that this > application will be helpful to show how several FIWARE GEris can > be used in an integrated way. > * Each of the chapters of the FIWARE Tour guide for developers > will integrate a "Try it yourself" section which will help the > developer to create an instance of the FIWARE GEris referred in > the chapter and try them. In addition, developers will be able > to deploy the demo application, in order to learn from a more > elaborated example how the FIWARE GEris can be used, and > providing instructions about how to play with the FIWARE GEris, > also "tweak" the application. > * Deployment of integrated FIWARE GEris and the demo application > will be made feasible using docker composition tools > > > Jos?-Manuel Cantera will drive the implementation of these concrete > actions. > > Regarding point 2, Bitergia has already developed a first bundle of > FIWARE GEris deployable using Docker compose (as a whole or > individually). They will work in extending this bundle integrating > additional FIWARE GEris in collaboration with the corresponding FIWARE > GEri owners. Bitergia and the ULPG will work together in the > development of a first version of the demo application. Eventually, > additional partners might be incorporated as additional skills and/or > resources are needed. We will address first those scenarios dealing > with local deployment, then we will incorporate those related to > deployment on other infrastructures such as the FIWARE Lab as soon as > the necessary preliminary work and/or detailed guidelines are completed > (this would be done under coordination by Alex and I assume we will > leverage on experience developed in FI-Content2). > > Feedback is welcome. > > Best regards, > > -- Juanjo > > ______________________________________________________ > > Coordinator and Chief Architect, FIWARE platform > CTO Industrial IoT, Telef?nica > > email: _juanjose.hierro at telefonica.com_ > > twitter: @JuanjoHierro > > You can follow FIWARE at: > website: _http://www.fiware.org_ > twitter: @FIWARE > facebook: _http://www.facebook.com/pages/FI-WARE/251366491587242_ > linkedIn: _http://www.linkedin.com/groups/FIWARE-4239932_ > > On 30/06/15 20:34, Alex Glikson wrote: > > Dear partners, > > Following the initial discussion at the containers task force, we > identified two follow-on (related) discussion tracks: > 1) expected FIWARE Lab user/developer experience with Docker > 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker > (and enable requirements derived from #1) > > This email refers to topic (1). > > I will try to summarize the initial set of usage scenarios that we may > want to support. We can then iterate on this over email, and have a > phone call if needed, when we feel that we are close to a convergence > point. > > Please, notice that I've subscribed several additional people relevant > for the discussion (who could provide input related to UI, developers > perspectives and operations perspectives). > > *Assumptions*: > Notice that a prerequisite for all the usage scenarios is that all the > FIWARE GEs (and SEs) are packages as Docker images and are kept in a > central repository, preferably the docker hub (under a 'to-be-created' > "fiware" namespace). Also, there is an assumption that the > users/developers would want to work with Docker tools as much as > possible (surfacing some or all of the capabilities via the FIWARE > Cloud Portal too). Juanjo will elaborate on the approach we are > thinking of to promote this with GE/SE owners and the broader > developer community. > > *Group 1: Basic Docker environment setup* > Usage scenario #1.1: > A user wants a local Docker runtime on his laptop. > He follows the standard instructions on setting up a docker host > (e.g., running within a VirtalBox VM, using docker-machine / boot2docker) > > Usage scenario #1.2: > A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab > (where he would then run various Docker containers). > He uses the standard docker-machine tool, specifying the URI of the > OpenStack Keystone in FIWARE Lab (and additional parameters, as > needed). The tool creates a VM using standard OpenStack APIs (natively > supported by FIWARE Lab) and configures Docker within the VM. The VM > would need to have a public IP (naturally). > > *Group 2: Basic life cycle of individual containers running GEs/SEs* > Usage scenario #2.1: > A developer wants to publish (a version of) a GE/SE. > After he is done creating the new Docker image, he pushes the new > version of the GE/SE to Docker hub under corresponding FIWARE > namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the > image points to the new version. > > Usage scenario #2.2: > A user/developer wants to deploy locally an instance of a certain GE/SE. > He uses the standard docker CLI to locally provision a container, > referring to the corresponding image at Docker hub -- e.g.: /"$ docker > -H boot2docker-vm:2376 run fiware/GE/cb-orion"/ > > Usage scenario #2.3: > A user/developer wants to deploy an instance of a certain GE/SE within > his Docker VM on FIWARE Lab > He uses the standard docker CLI to provision a container, referring to > the location of his Docker VM as well as the corresponding image at > Docker hub -- e.g.: /"$ docker -H mydocker-vm37.lab.fiware.org:2376 > run fiware/GE/cb-orion". /ISSUE: the user would need to open the > corresponding firewall ports in his VM (same as those of the GE/SE, or > following the mapping performed during container provisioning) in > order to make the GE/SE accessible. Ideally, this should be done in a > scalable but secure manner. An easy solution is to open up-front > (during VM provisioning) a range of ports (via setting up > corresponding security group). A more advanced solution is to update > the security group dynamically. > > Usage scenario #2.4: > A user/developer wants to update a container comprising certain GE/SE > with the latest version recently published in the dedicated namespace > of the Docker Hub. > He pulls the latest version from the Docker Hub, kills the old > container, and starts a new one (attaching to the same resources). > Note that this would work well when the application is properly > designed for Docker (e.g., the container itself is stateless), and > when the previous version of the container has been provisioned > manually (as in #2, #4 above) by the developer (and he knows which > resources to connect to). > > *Group 3: Support for 'bundles' of GEs/SEs that together perform a > certain complex function* > Usage scenario #3.1: > A developer wants to publish (a version of) a 'bundle' of GEs/SEs that > together perform a certain complex function. > He creates a 'template' (e.g., following the format of > docker-compose), referring to the individual GE/SE images as well as > their interdependencies (e.g., links) and other composition > properties. This might be done using a text editor, a Web UI provided > by the FIWARE Cloud (conceptually similar to today's UI for creation > of blueprints), or other tools from Docker ecosystem. He then uploads > the template to the centralized templats repository. Note: it is > likely that docker-compose will be able to use Docker Registry/Hub as > a repository for templates (including versioning, push/pull, etc). > > Usage scenario #3.2: > A user wants to provision locally a set of GEs/SEs, using a > pre-defined template (comprising a 'bundle') > He uses the standard docker-compose tool referring to the 'template' > artifcat as well as the local Docker URL. > > Usage scenario #3.3: > A user wants to provision a set of GEs/SEs in FIWARE Lab, using a > pre-defined template (comprising a 'bundle') > He uses the standard docker-compose tool referring to the 'template' > artifcat as well as the target Docker URL in FIWARE Lab. > Alternatively, he uses the FIWARE Portal UI to do the same (e.g., with > Murano backend invoking docker-compose). > > Usage scenario #3.4: > A user wants to update his Docker environment (local or on FIWARE Lab) > with the latest version of a certain 'bundle' > Note: may require enancements to docker-compose > > *Group 4: Advanced scenarios* > Usage scenario #4.1: > A user/developer wants to provision a cluster of VMs on FIWARE Lab > that would host his Docker cluster (managed with Swarm or Kubernetes) > He uses corresponding Murano/Heat template to provision the VMs and to > configure the Docker/Swarm/Kubernetes cluster. > > Usage scenario #4.2: > A user/developer wants to access a global instance of a Docker service > in FIWARE Lab (shared, scalable, managed), so that he doesn't need to > manage the corresponding VM(s) by himself. > He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab > Docker API endpoint with the standard Docker tools or FIWARE-specific > tools (as outlined above ). > > Usage scenario #4.3: > A user/developer wants to manage access control for Docker images > among FIWARE Lab users. > He starts using the Docker Registry/Hub deployed within the FIWARE Lab. > > > Notice that this is a very initial list -- I am sure that there are > many inaccuracies and gaps. Feel free to comment. > > Thanks, > Alex > > ==================================================================================== > Alex Glikson > Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab > Email: _glikson at il.ibm.com_ | Phone: > +972-4-8281085 | Mobile: > +972-54-6466667 | Fax: +972-4-8296112 > > > > _______________________________________________ > Fiware-cloud-containers mailing list > _Fiware-cloud-containers at lists.fiware.org_ > > _https://lists.fiware.org/listinfo/fiware-cloud-containers_ > > > ------------------------------------------------------------------------ > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener informaci?n privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilizaci?n, > divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud > de la legislaci?n vigente. Si ha recibido este mensaje por error, le > rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda > a su destrucci?n. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution > or copying of this communication is strictly prohibited. If you have > received this transmission in error, do not read it. Please immediately > reply to the sender that you have received this communication in error > and then delete it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu > destinat?rio, pode conter informa??o privilegiada ou confidencial e ? > para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa > senhoria o destinat?rio indicado, fica notificado de que a leitura, > utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em > virtude da legisla??o vigente. Se recebeu esta mensagem por erro, > rogamos-lhe que nos o comunique imediatamente por esta mesma via e > proceda a sua destrui??o > > > _______________________________________________ > Fiware-cloud-containers mailing list > _Fiware-cloud-containers at lists.fiware.org_ > > _https://lists.fiware.org/listinfo/fiware-cloud-containers_ > > > > > ------------------------------------------------------------------------ > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener informaci?n privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilizaci?n, > divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud > de la legislaci?n vigente. Si ha recibido este mensaje por error, le > rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda > a su destrucci?n. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution > or copying of this communication is strictly prohibited. If you have > received this transmission in error, do not read it. Please immediately > reply to the sender that you have received this communication in error > and then delete it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu > destinat?rio, pode conter informa??o privilegiada ou confidencial e ? > para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa > senhoria o destinat?rio indicado, fica notificado de que a leitura, > utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em > virtude da legisla??o vigente. Se recebeu esta mensagem por erro, > rogamos-lhe que nos o comunique imediatamente por esta mesma via e > proceda a sua destrui??o > > > _______________________________________________ > Fiware-cloud-containers mailing list > _Fiware-cloud-containers at lists.fiware.org_ > > _https://lists.fiware.org/listinfo/fiware-cloud-containers_ > > > > ------------------------------------------------------------------------ > > Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, > puede contener informaci?n privilegiada o confidencial y es para uso > exclusivo de la persona o entidad de destino. Si no es usted. el > destinatario indicado, queda notificado de que la lectura, utilizaci?n, > divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud > de la legislaci?n vigente. Si ha recibido este mensaje por error, le > rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda > a su destrucci?n. > > The information contained in this transmission is privileged and > confidential information intended only for the use of the individual or > entity named above. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution > or copying of this communication is strictly prohibited. If you have > received this transmission in error, do not read it. Please immediately > reply to the sender that you have received this communication in error > and then delete it. > > Esta mensagem e seus anexos se dirigem exclusivamente ao seu > destinat?rio, pode conter informa??o privilegiada ou confidencial e ? > para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa > senhoria o destinat?rio indicado, fica notificado de que a leitura, > utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em > virtude da legisla??o vigente. Se recebeu esta mensagem por erro, > rogamos-lhe que nos o comunique imediatamente por esta mesma via e > proceda a sua destrui??o -- ------------------------------------------------------------------------- Deutsches Forschungszentrum f?r K?nstliche Intelligenz (DFKI) GmbH Trippstadter Strasse 122, D-67663 Kaiserslautern Gesch?ftsf?hrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) Dr. Walter Olthoff Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes Sitz der Gesellschaft: Kaiserslautern (HRB 2313) USt-Id.Nr.: DE 148646973, Steuernummer: 19/673/0060/3 --------------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: philipp_slusallek.vcf Type: text/x-vcard Size: 441 bytes Desc: not available URL: From juanjose.hierro at telefonica.com Mon Jul 20 07:22:12 2015 From: juanjose.hierro at telefonica.com (Juanjo Hierro) Date: Mon, 20 Jul 2015 07:22:12 +0200 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: References: <5592F69E.4060803@telefonica.com> <559CB92E.3070002@dfki.de> <559CD50A.9060403@telefonica.com> <559E165D.5090908@telefonica.com> Message-ID: <55AC8584.9030803@telefonica.com> An HTML attachment was scrubbed... URL: From GLIKSON at il.ibm.com Mon Jul 20 08:02:40 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Mon, 20 Jul 2015 09:02:40 +0300 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: <55AC8584.9030803@telefonica.com> References: <5592F69E.4060803@telefonica.com> <559CB92E.3070002@dfki.de> <559CD50A.9060403@telefonica.com> <559E165D.5090908@telefonica.com> <55AC8584.9030803@telefonica.com> Message-ID: Thanks, Juanjo. I assume we can use the regular TID bridge... For those not on the invitation, the dial-in details are: + PIN: 050662 + Local dial-in phone numbers at: http://pdf.powwownow.com/pdf/USA_en_pwn-dial-in-numbers.pdf Talk to (most of) you at 11 CET then. I am not entirely sure whether this will be productive for all the other architects who didn't participate the task force, though.. I will try giving a quick recap -- hopefully this would help. Regards, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab; FIWARE Cloud Leader & Architect Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Juanjo Hierro To: Alex Glikson/Haifa/IBM at IBMIL Cc: , Philipp Slusallek Date: 20/07/2015 08:22 AM Subject: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Hi, Sorry that I didn't see your message and couldn't answer before ... For me, it's fine to change the time so that we start at 11:00am ... I have updated the invitation to the FIWARE Architects follow-up confcall where the discussion was going to be placed. Note that I'm currently in Mexico and there are 7 hours of difference. I'm not pretty sure whether I will be able to make it so please don't wait for me ... Best regards, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 19/07/15 10:05, Alex Glikson wrote: Can we do it 11am-1pm? Or maybe 11-12:30? Doodle indicated that this slot is available too. Juanjo -- were you going to update the architects meeting invitation (inviting additional people, etc), or should I send a separate one, with my bridge details? Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab; FIWARE Cloud Leader & Architect Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Juanjo Hierro To: Philipp Slusallek , Alex Glikson/Haifa/IBM at IBMIL, Date: 09/07/2015 09:36 AM Subject: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Hi all, Based on results from the doodle poll, we will setup the confcall on Monday 20th, from 09:30 to 11:30. Cheers, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 08/07/15 09:45, Juanjo Hierro wrote: I would suggest we setup a confcall to coordinate actions. Next architects confcall should be devoted to the discussion on POIs. We may schedule the next architects confcall on July 20th on this matter or, try to plan a confcall earlier. Next week it is going to be difficult for Jos? Manuel and myself. Jos? Manuel is travelling to Shangai this Friday and will come back Thursday next week. Then, Friday 17th wouldn't work for Israel We could make it on Thursday from 12:00 to 14:00 or from 14:30 on, starting not later than 16:00 ... otherwise go for the 20th. I have setup a doodle poll with the different options: http://doodle.com/66zyea2kiuh5vy98 Cheers, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 08/07/15 07:46, Philipp Slusallek wrote: Hi all, This all sounds good! But what happened to your planned next meetings. FIC2 is still highly interested in working with FIWARE to deploy the infrastructure developed there (running on FIWARE Lab) also for FIWARE. I thought the idea was to explore these options in a joint next call. Best, Philipp Am 30.06.2015 um 22:05 schrieb Juanjo Hierro: Dear all, Thanks Alex for launching these discussions tracks. Regarding point (1) as anticipated by Alex, here it is the concrete plan we aim at implementing to promote usage of docker tools by GE/SE owners and the broader developer community. 1. Make info about Docker images and instruction to setup docker containers linked to FIWARE GEris available in the FIWARE Catalogue: * We will specify a general template of a new section which will be included in the "Creating instances" tab of entries linked to FIWARE GEris. This section will be elaborate on "Deploying a dedicated GE instance using Docker technology" * Each FIWARE GEri owner will be asked to setup a Docker image for their GEs and register it in Docker Hub. This task can start in parallel to the previous one. * Each FIWARE GEri owner will be asked to update the "Creating instance" tab of the entry linked to the FIWARE GEri so that it provides instructions about deploying a dedicated instance of the GEri using docker. For this purpose, the owner will follow the template defined in step 1. 2. We will incorporate the description about how to create FIWARE GEri instances and try them using docker within the FIWARE Tour Guide for developers (which will effectively become the landing page of http://developers.fiware.org) as a mean to shorten the learning curve with FIWARE GEris * Somewhere at the beginning of the guided tour, we will explain the developer how he can setup the basic docker environment either locally or on the FIWARE Lab on which he will be able to rely to try the different FIWARE GEris (Group 1 scenarios described by Alex) * A demo application will be developed that will help the developer to try each FIWARE GEri instance deployed using docker with concrete data. Developers will be able to deploy this application also using docker. It is the intention that this application will be helpful to show how several FIWARE GEris can be used in an integrated way. * Each of the chapters of the FIWARE Tour guide for developers will integrate a "Try it yourself" section which will help the developer to create an instance of the FIWARE GEris referred in the chapter and try them. In addition, developers will be able to deploy the demo application, in order to learn from a more elaborated example how the FIWARE GEris can be used, and providing instructions about how to play with the FIWARE GEris, also "tweak" the application. * Deployment of integrated FIWARE GEris and the demo application will be made feasible using docker composition tools Jos?-Manuel Cantera will drive the implementation of these concrete actions. Regarding point 2, Bitergia has already developed a first bundle of FIWARE GEris deployable using Docker compose (as a whole or individually). They will work in extending this bundle integrating additional FIWARE GEris in collaboration with the corresponding FIWARE GEri owners. Bitergia and the ULPG will work together in the development of a first version of the demo application. Eventually, additional partners might be incorporated as additional skills and/or resources are needed. We will address first those scenarios dealing with local deployment, then we will incorporate those related to deployment on other infrastructures such as the FIWARE Lab as soon as the necessary preliminary work and/or detailed guidelines are completed (this would be done under coordination by Alex and I assume we will leverage on experience developed in FI-Content2). Feedback is welcome. Best regards, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 30/06/15 20:34, Alex Glikson wrote: Dear partners, Following the initial discussion at the containers task force, we identified two follow-on (related) discussion tracks: 1) expected FIWARE Lab user/developer experience with Docker 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker (and enable requirements derived from #1) This email refers to topic (1). I will try to summarize the initial set of usage scenarios that we may want to support. We can then iterate on this over email, and have a phone call if needed, when we feel that we are close to a convergence point. Please, notice that I've subscribed several additional people relevant for the discussion (who could provide input related to UI, developers perspectives and operations perspectives). *Assumptions*: Notice that a prerequisite for all the usage scenarios is that all the FIWARE GEs (and SEs) are packages as Docker images and are kept in a central repository, preferably the docker hub (under a 'to-be-created' "fiware" namespace). Also, there is an assumption that the users/developers would want to work with Docker tools as much as possible (surfacing some or all of the capabilities via the FIWARE Cloud Portal too). Juanjo will elaborate on the approach we are thinking of to promote this with GE/SE owners and the broader developer community. *Group 1: Basic Docker environment setup* Usage scenario #1.1: A user wants a local Docker runtime on his laptop. He follows the standard instructions on setting up a docker host (e.g., running within a VirtalBox VM, using docker-machine / boot2docker) Usage scenario #1.2: A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab (where he would then run various Docker containers). He uses the standard docker-machine tool, specifying the URI of the OpenStack Keystone in FIWARE Lab (and additional parameters, as needed). The tool creates a VM using standard OpenStack APIs (natively supported by FIWARE Lab) and configures Docker within the VM. The VM would need to have a public IP (naturally). *Group 2: Basic life cycle of individual containers running GEs/SEs* Usage scenario #2.1: A developer wants to publish (a version of) a GE/SE. After he is done creating the new Docker image, he pushes the new version of the GE/SE to Docker hub under corresponding FIWARE namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the image points to the new version. Usage scenario #2.2: A user/developer wants to deploy locally an instance of a certain GE/SE. He uses the standard docker CLI to locally provision a container, referring to the corresponding image at Docker hub -- e.g.: /"$ docker -H boot2docker-vm:2376 run fiware/GE/cb-orion"/ Usage scenario #2.3: A user/developer wants to deploy an instance of a certain GE/SE within his Docker VM on FIWARE Lab He uses the standard docker CLI to provision a container, referring to the location of his Docker VM as well as the corresponding image at Docker hub -- e.g.: /"$ docker -H mydocker-vm37.lab.fiware.org:2376 run fiware/GE/cb-orion". /ISSUE: the user would need to open the corresponding firewall ports in his VM (same as those of the GE/SE, or following the mapping performed during container provisioning) in order to make the GE/SE accessible. Ideally, this should be done in a scalable but secure manner. An easy solution is to open up-front (during VM provisioning) a range of ports (via setting up corresponding security group). A more advanced solution is to update the security group dynamically. Usage scenario #2.4: A user/developer wants to update a container comprising certain GE/SE with the latest version recently published in the dedicated namespace of the Docker Hub. He pulls the latest version from the Docker Hub, kills the old container, and starts a new one (attaching to the same resources). Note that this would work well when the application is properly designed for Docker (e.g., the container itself is stateless), and when the previous version of the container has been provisioned manually (as in #2, #4 above) by the developer (and he knows which resources to connect to). *Group 3: Support for 'bundles' of GEs/SEs that together perform a certain complex function* Usage scenario #3.1: A developer wants to publish (a version of) a 'bundle' of GEs/SEs that together perform a certain complex function. He creates a 'template' (e.g., following the format of docker-compose), referring to the individual GE/SE images as well as their interdependencies (e.g., links) and other composition properties. This might be done using a text editor, a Web UI provided by the FIWARE Cloud (conceptually similar to today's UI for creation of blueprints), or other tools from Docker ecosystem. He then uploads the template to the centralized templats repository. Note: it is likely that docker-compose will be able to use Docker Registry/Hub as a repository for templates (including versioning, push/pull, etc). Usage scenario #3.2: A user wants to provision locally a set of GEs/SEs, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the local Docker URL. Usage scenario #3.3: A user wants to provision a set of GEs/SEs in FIWARE Lab, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the target Docker URL in FIWARE Lab. Alternatively, he uses the FIWARE Portal UI to do the same (e.g., with Murano backend invoking docker-compose). Usage scenario #3.4: A user wants to update his Docker environment (local or on FIWARE Lab) with the latest version of a certain 'bundle' Note: may require enancements to docker-compose *Group 4: Advanced scenarios* Usage scenario #4.1: A user/developer wants to provision a cluster of VMs on FIWARE Lab that would host his Docker cluster (managed with Swarm or Kubernetes) He uses corresponding Murano/Heat template to provision the VMs and to configure the Docker/Swarm/Kubernetes cluster. Usage scenario #4.2: A user/developer wants to access a global instance of a Docker service in FIWARE Lab (shared, scalable, managed), so that he doesn't need to manage the corresponding VM(s) by himself. He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab Docker API endpoint with the standard Docker tools or FIWARE-specific tools (as outlined above ). Usage scenario #4.3: A user/developer wants to manage access control for Docker images among FIWARE Lab users. He starts using the Docker Registry/Hub deployed within the FIWARE Lab. Notice that this is a very initial list -- I am sure that there are many inaccuracies and gaps. Feel free to comment. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers ------------------------------------------------------------------------ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o -------------- next part -------------- An HTML attachment was scrubbed... URL: From GLIKSON at il.ibm.com Mon Jul 20 08:10:16 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Mon, 20 Jul 2015 09:10:16 +0300 Subject: [Fiware-cloud-containers] today's architects call -- Docker Message-ID: Dear chapter architects, all, Today's meeting (starting at 11am CET) will be dedicated to a recap of the activities of the task force on docker containers, and a discussion regarding roadmap and next steps. Regards, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab; FIWARE Cloud Leader & Architect Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 -------------- next part -------------- An HTML attachment was scrubbed... URL: From GLIKSON at il.ibm.com Mon Jul 20 10:58:42 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Mon, 20 Jul 2015 11:58:42 +0300 Subject: [Fiware-cloud-containers] [Fiware-chapter-architects] today's architects call -- Docker In-Reply-To: References: Message-ID: Slides for today's discussion: https://docs.google.com/presentation/d/1GE5jjyFspn540pW2lzQjaC3BNVvB43TjO3kUY_ix4PI/edit#slide=id.p4 Minutes will go here: https://docs.google.com/document/d/1JT6wp_KSN6qUbn5d61MA52DKZPVWe8S0K-Osnr4PG8Y/edit# Talk to you in few minutes. Regards, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab; FIWARE Cloud Leader & Architect Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: Alex Glikson/Haifa/IBM at IBMIL To: fiware-chapter-architects at lists.fi-ware.org Cc: fiware-cloud-containers at lists.fiware.org, fiware-chapter-leaders at lists.fi-ware.org, fiware-wg-leaders at lists.fi-ware.org Date: 20/07/2015 09:10 AM Subject: [Fiware-chapter-architects] today's architects call -- Docker Sent by: fiware-chapter-architects-bounces at lists.fi-ware.org Dear chapter architects, all, Today's meeting (starting at 11am CET) will be dedicated to a recap of the activities of the task force on docker containers, and a discussion regarding roadmap and next steps. Regards, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab; FIWARE Cloud Leader & Architect Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 _______________________________________________ Fiware-chapter-architects mailing list Fiware-chapter-architects at lists.fi-ware.org https://lists.fi-ware.org/listinfo/fiware-chapter-architects -------------- next part -------------- An HTML attachment was scrubbed... URL: From josemanuel.canterafonseca at telefonica.com Tue Jul 28 17:16:24 2015 From: josemanuel.canterafonseca at telefonica.com (JOSE MANUEL CANTERA FONSECA) Date: Tue, 28 Jul 2015 15:16:24 +0000 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: <5592F69E.4060803@telefonica.com> References: <5592F69E.4060803@telefonica.com> Message-ID: Dear all, In order to move forward regarding the Docker Activities related to developer experience I have drafted a first version of the Gei Dockerization (aka containerization) guidelines. It follows the same approach as the Developer Guidelines with must, should and may actions. It includes already working examples that can be used as a reference. https://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/Docker Please let us know your thoughts Many thanks All the best De: > on behalf of Juanjo Hierro > Fecha: martes, 30 de junio de 2015, 22:05 Para: Alex Glikson >, "fiware-cloud-containers at lists.fiware.org" > Asunto: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Dear all, Thanks Alex for launching these discussions tracks. Regarding point (1) as anticipated by Alex, here it is the concrete plan we aim at implementing to promote usage of docker tools by GE/SE owners and the broader developer community. 1. Make info about Docker images and instruction to setup docker containers linked to FIWARE GEris available in the FIWARE Catalogue: * We will specify a general template of a new section which will be included in the "Creating instances" tab of entries linked to FIWARE GEris. This section will be elaborate on "Deploying a dedicated GE instance using Docker technology" * Each FIWARE GEri owner will be asked to setup a Docker image for their GEs and register it in Docker Hub. This task can start in parallel to the previous one. * Each FIWARE GEri owner will be asked to update the "Creating instance" tab of the entry linked to the FIWARE GEri so that it provides instructions about deploying a dedicated instance of the GEri using docker. For this purpose, the owner will follow the template defined in step 1. 2. We will incorporate the description about how to create FIWARE GEri instances and try them using docker within the FIWARE Tour Guide for developers (which will effectively become the landing page of http://developers.fiware.org) as a mean to shorten the learning curve with FIWARE GEris * Somewhere at the beginning of the guided tour, we will explain the developer how he can setup the basic docker environment either locally or on the FIWARE Lab on which he will be able to rely to try the different FIWARE GEris (Group 1 scenarios described by Alex) * A demo application will be developed that will help the developer to try each FIWARE GEri instance deployed using docker with concrete data. Developers will be able to deploy this application also using docker. It is the intention that this application will be helpful to show how several FIWARE GEris can be used in an integrated way. * Each of the chapters of the FIWARE Tour guide for developers will integrate a "Try it yourself" section which will help the developer to create an instance of the FIWARE GEris referred in the chapter and try them. In addition, developers will be able to deploy the demo application, in order to learn from a more elaborated example how the FIWARE GEris can be used, and providing instructions about how to play with the FIWARE GEris, also "tweak" the application. * Deployment of integrated FIWARE GEris and the demo application will be made feasible using docker composition tools Jos?-Manuel Cantera will drive the implementation of these concrete actions. Regarding point 2, Bitergia has already developed a first bundle of FIWARE GEris deployable using Docker compose (as a whole or individually). They will work in extending this bundle integrating additional FIWARE GEris in collaboration with the corresponding FIWARE GEri owners. Bitergia and the ULPG will work together in the development of a first version of the demo application. Eventually, additional partners might be incorporated as additional skills and/or resources are needed. We will address first those scenarios dealing with local deployment, then we will incorporate those related to deployment on other infrastructures such as the FIWARE Lab as soon as the necessary preliminary work and/or detailed guidelines are completed (this would be done under coordination by Alex and I assume we will leverage on experience developed in FI-Content2). Feedback is welcome. Best regards, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 30/06/15 20:34, Alex Glikson wrote: Dear partners, Following the initial discussion at the containers task force, we identified two follow-on (related) discussion tracks: 1) expected FIWARE Lab user/developer experience with Docker 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker (and enable requirements derived from #1) This email refers to topic (1). I will try to summarize the initial set of usage scenarios that we may want to support. We can then iterate on this over email, and have a phone call if needed, when we feel that we are close to a convergence point. Please, notice that I've subscribed several additional people relevant for the discussion (who could provide input related to UI, developers perspectives and operations perspectives). Assumptions: Notice that a prerequisite for all the usage scenarios is that all the FIWARE GEs (and SEs) are packages as Docker images and are kept in a central repository, preferably the docker hub (under a 'to-be-created' "fiware" namespace). Also, there is an assumption that the users/developers would want to work with Docker tools as much as possible (surfacing some or all of the capabilities via the FIWARE Cloud Portal too). Juanjo will elaborate on the approach we are thinking of to promote this with GE/SE owners and the broader developer community. Group 1: Basic Docker environment setup Usage scenario #1.1: A user wants a local Docker runtime on his laptop. He follows the standard instructions on setting up a docker host (e.g., running within a VirtalBox VM, using docker-machine / boot2docker) Usage scenario #1.2: A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab (where he would then run various Docker containers). He uses the standard docker-machine tool, specifying the URI of the OpenStack Keystone in FIWARE Lab (and additional parameters, as needed). The tool creates a VM using standard OpenStack APIs (natively supported by FIWARE Lab) and configures Docker within the VM. The VM would need to have a public IP (naturally). Group 2: Basic life cycle of individual containers running GEs/SEs Usage scenario #2.1: A developer wants to publish (a version of) a GE/SE. After he is done creating the new Docker image, he pushes the new version of the GE/SE to Docker hub under corresponding FIWARE namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the image points to the new version. Usage scenario #2.2: A user/developer wants to deploy locally an instance of a certain GE/SE. He uses the standard docker CLI to locally provision a container, referring to the corresponding image at Docker hub -- e.g.: "$ docker -H boot2docker-vm:2376 run fiware/GE/cb-orion" Usage scenario #2.3: A user/developer wants to deploy an instance of a certain GE/SE within his Docker VM on FIWARE Lab He uses the standard docker CLI to provision a container, referring to the location of his Docker VM as well as the corresponding image at Docker hub -- e.g.: "$ docker -H mydocker-vm37.lab.fiware.org:2376 run fiware/GE/cb-orion". ISSUE: the user would need to open the corresponding firewall ports in his VM (same as those of the GE/SE, or following the mapping performed during container provisioning) in order to make the GE/SE accessible. Ideally, this should be done in a scalable but secure manner. An easy solution is to open up-front (during VM provisioning) a range of ports (via setting up corresponding security group). A more advanced solution is to update the security group dynamically. Usage scenario #2.4: A user/developer wants to update a container comprising certain GE/SE with the latest version recently published in the dedicated namespace of the Docker Hub. He pulls the latest version from the Docker Hub, kills the old container, and starts a new one (attaching to the same resources). Note that this would work well when the application is properly designed for Docker (e.g., the container itself is stateless), and when the previous version of the container has been provisioned manually (as in #2, #4 above) by the developer (and he knows which resources to connect to). Group 3: Support for 'bundles' of GEs/SEs that together perform a certain complex function Usage scenario #3.1: A developer wants to publish (a version of) a 'bundle' of GEs/SEs that together perform a certain complex function. He creates a 'template' (e.g., following the format of docker-compose), referring to the individual GE/SE images as well as their interdependencies (e.g., links) and other composition properties. This might be done using a text editor, a Web UI provided by the FIWARE Cloud (conceptually similar to today's UI for creation of blueprints), or other tools from Docker ecosystem. He then uploads the template to the centralized templats repository. Note: it is likely that docker-compose will be able to use Docker Registry/Hub as a repository for templates (including versioning, push/pull, etc). Usage scenario #3.2: A user wants to provision locally a set of GEs/SEs, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the local Docker URL. Usage scenario #3.3: A user wants to provision a set of GEs/SEs in FIWARE Lab, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the target Docker URL in FIWARE Lab. Alternatively, he uses the FIWARE Portal UI to do the same (e.g., with Murano backend invoking docker-compose). Usage scenario #3.4: A user wants to update his Docker environment (local or on FIWARE Lab) with the latest version of a certain 'bundle' Note: may require enancements to docker-compose Group 4: Advanced scenarios Usage scenario #4.1: A user/developer wants to provision a cluster of VMs on FIWARE Lab that would host his Docker cluster (managed with Swarm or Kubernetes) He uses corresponding Murano/Heat template to provision the VMs and to configure the Docker/Swarm/Kubernetes cluster. Usage scenario #4.2: A user/developer wants to access a global instance of a Docker service in FIWARE Lab (shared, scalable, managed), so that he doesn't need to manage the corresponding VM(s) by himself. He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab Docker API endpoint with the standard Docker tools or FIWARE-specific tools (as outlined above ). Usage scenario #4.3: A user/developer wants to manage access control for Docker images among FIWARE Lab users. He starts using the Docker Registry/Hub deployed within the FIWARE Lab. Notice that this is a very initial list -- I am sure that there are many inaccuracies and gaps. Feel free to comment. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.orghttps://lists.fiware.org/listinfo/fiware-cloud-containers ________________________________ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o ________________________________ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o -------------- next part -------------- An HTML attachment was scrubbed... URL: From GLIKSON at il.ibm.com Wed Jul 29 07:39:30 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Wed, 29 Jul 2015 08:39:30 +0300 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: References: <5592F69E.4060803@telefonica.com> Message-ID: Dear Jose Manuel, This is certainly a good start. Before going into specific guidelines, maybe you can outline the envisioned/recommended build/packaging/release workflow that we would expect GEi developers to follow? Let's say I am a GE owner, my code is in github (following the Developer Guidelines.. Can we use the 'fiware' repository in github?), and I want monthly releases (at the end of each sprint, after having an internal testing cycle). What should I do? It looks like I would need scripts that build DEB packages out of my github code (is there a FIWARE packages repository that I can use?), some mechanism to invoke those scripts and populate a new version of the packages periodically (e.g., I can do it manually after each sprint), a Dockerfile that would describe how to use those packages to build a GEi image, to have tags per release on github, and to have a mechanism that would perform periodic builds from the Dockerfile that would populate the Dockerhub repository (under FIWARE org/namespace?) with new images and respective tags (would I be able to configure automatic builds associated with the FIWARE repository in Docker Hub to automatically build images out of my github repository, or fiware github repository?). At least for me such a workflow would be very helpful, to make sure we don't miss important steps in the guidelines (or prerequisites to make them easy to follow). Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab; FIWARE Cloud Leader & Architect Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: JOSE MANUEL CANTERA FONSECA To: JUAN JOSE HIERRO SUREDA , Alex Glikson/Haifa/IBM at IBMIL Cc: "fiware-cloud-containers at lists.fiware.org" , "fiware-chapter-architects at lists.fiware.org" Date: 28/07/2015 06:16 PM Subject: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Dear all, In order to move forward regarding the Docker Activities related to developer experience I have drafted a first version of the Gei Dockerization (aka containerization) guidelines. It follows the same approach as the Developer Guidelines with must, should and may actions. It includes already working examples that can be used as a reference. https://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/Docker Please let us know your thoughts Many thanks All the best De: on behalf of Juanjo Hierro Fecha: martes, 30 de junio de 2015, 22:05 Para: Alex Glikson , " fiware-cloud-containers at lists.fiware.org" < fiware-cloud-containers at lists.fiware.org> Asunto: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Dear all, Thanks Alex for launching these discussions tracks. Regarding point (1) as anticipated by Alex, here it is the concrete plan we aim at implementing to promote usage of docker tools by GE/SE owners and the broader developer community. 1. Make info about Docker images and instruction to setup docker containers linked to FIWARE GEris available in the FIWARE Catalogue: We will specify a general template of a new section which will be included in the "Creating instances" tab of entries linked to FIWARE GEris. This section will be elaborate on "Deploying a dedicated GE instance using Docker technology" Each FIWARE GEri owner will be asked to setup a Docker image for their GEs and register it in Docker Hub. This task can start in parallel to the previous one. Each FIWARE GEri owner will be asked to update the "Creating instance" tab of the entry linked to the FIWARE GEri so that it provides instructions about deploying a dedicated instance of the GEri using docker. For this purpose, the owner will follow the template defined in step 1. 2. We will incorporate the description about how to create FIWARE GEri instances and try them using docker within the FIWARE Tour Guide for developers (which will effectively become the landing page of http://developers.fiware.org) as a mean to shorten the learning curve with FIWARE GEris Somewhere at the beginning of the guided tour, we will explain the developer how he can setup the basic docker environment either locally or on the FIWARE Lab on which he will be able to rely to try the different FIWARE GEris (Group 1 scenarios described by Alex) A demo application will be developed that will help the developer to try each FIWARE GEri instance deployed using docker with concrete data. Developers will be able to deploy this application also using docker. It is the intention that this application will be helpful to show how several FIWARE GEris can be used in an integrated way. Each of the chapters of the FIWARE Tour guide for developers will integrate a "Try it yourself" section which will help the developer to create an instance of the FIWARE GEris referred in the chapter and try them. In addition, developers will be able to deploy the demo application, in order to learn from a more elaborated example how the FIWARE GEris can be used, and providing instructions about how to play with the FIWARE GEris, also "tweak" the application. Deployment of integrated FIWARE GEris and the demo application will be made feasible using docker composition tools Jos?-Manuel Cantera will drive the implementation of these concrete actions. Regarding point 2, Bitergia has already developed a first bundle of FIWARE GEris deployable using Docker compose (as a whole or individually). They will work in extending this bundle integrating additional FIWARE GEris in collaboration with the corresponding FIWARE GEri owners. Bitergia and the ULPG will work together in the development of a first version of the demo application. Eventually, additional partners might be incorporated as additional skills and/or resources are needed. We will address first those scenarios dealing with local deployment, then we will incorporate those related to deployment on other infrastructures such as the FIWARE Lab as soon as the necessary preliminary work and/or detailed guidelines are completed (this would be done under coordination by Alex and I assume we will leverage on experience developed in FI-Content2). Feedback is welcome. Best regards, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 30/06/15 20:34, Alex Glikson wrote: Dear partners, Following the initial discussion at the containers task force, we identified two follow-on (related) discussion tracks: 1) expected FIWARE Lab user/developer experience with Docker 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker (and enable requirements derived from #1) This email refers to topic (1). I will try to summarize the initial set of usage scenarios that we may want to support. We can then iterate on this over email, and have a phone call if needed, when we feel that we are close to a convergence point. Please, notice that I've subscribed several additional people relevant for the discussion (who could provide input related to UI, developers perspectives and operations perspectives). Assumptions: Notice that a prerequisite for all the usage scenarios is that all the FIWARE GEs (and SEs) are packages as Docker images and are kept in a central repository, preferably the docker hub (under a 'to-be-created' "fiware" namespace). Also, there is an assumption that the users/developers would want to work with Docker tools as much as possible (surfacing some or all of the capabilities via the FIWARE Cloud Portal too). Juanjo will elaborate on the approach we are thinking of to promote this with GE/SE owners and the broader developer community. Group 1: Basic Docker environment setup Usage scenario #1.1: A user wants a local Docker runtime on his laptop. He follows the standard instructions on setting up a docker host (e.g., running within a VirtalBox VM, using docker-machine / boot2docker) Usage scenario #1.2: A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab (where he would then run various Docker containers). He uses the standard docker-machine tool, specifying the URI of the OpenStack Keystone in FIWARE Lab (and additional parameters, as needed). The tool creates a VM using standard OpenStack APIs (natively supported by FIWARE Lab) and configures Docker within the VM. The VM would need to have a public IP (naturally). Group 2: Basic life cycle of individual containers running GEs/SEs Usage scenario #2.1: A developer wants to publish (a version of) a GE/SE. After he is done creating the new Docker image, he pushes the new version of the GE/SE to Docker hub under corresponding FIWARE namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the image points to the new version. Usage scenario #2.2: A user/developer wants to deploy locally an instance of a certain GE/SE. He uses the standard docker CLI to locally provision a container, referring to the corresponding image at Docker hub -- e.g.: "$ docker -H boot2docker-vm:2376 run fiware/GE/cb-orion" Usage scenario #2.3: A user/developer wants to deploy an instance of a certain GE/SE within his Docker VM on FIWARE Lab He uses the standard docker CLI to provision a container, referring to the location of his Docker VM as well as the corresponding image at Docker hub -- e.g.: "$ docker -H mydocker-vm37.lab.fiware.org:2376 run fiware/GE/cb-orion". ISSUE: the user would need to open the corresponding firewall ports in his VM (same as those of the GE/SE, or following the mapping performed during container provisioning) in order to make the GE/SE accessible. Ideally, this should be done in a scalable but secure manner. An easy solution is to open up-front (during VM provisioning) a range of ports (via setting up corresponding security group). A more advanced solution is to update the security group dynamically. Usage scenario #2.4: A user/developer wants to update a container comprising certain GE/SE with the latest version recently published in the dedicated namespace of the Docker Hub. He pulls the latest version from the Docker Hub, kills the old container, and starts a new one (attaching to the same resources). Note that this would work well when the application is properly designed for Docker (e.g., the container itself is stateless), and when the previous version of the container has been provisioned manually (as in #2, #4 above) by the developer (and he knows which resources to connect to). Group 3: Support for 'bundles' of GEs/SEs that together perform a certain complex function Usage scenario #3.1: A developer wants to publish (a version of) a 'bundle' of GEs/SEs that together perform a certain complex function. He creates a 'template' (e.g., following the format of docker-compose), referring to the individual GE/SE images as well as their interdependencies (e.g., links) and other composition properties. This might be done using a text editor, a Web UI provided by the FIWARE Cloud (conceptually similar to today's UI for creation of blueprints), or other tools from Docker ecosystem. He then uploads the template to the centralized templats repository. Note: it is likely that docker-compose will be able to use Docker Registry/Hub as a repository for templates (including versioning, push/pull, etc). Usage scenario #3.2: A user wants to provision locally a set of GEs/SEs, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the local Docker URL. Usage scenario #3.3: A user wants to provision a set of GEs/SEs in FIWARE Lab, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the target Docker URL in FIWARE Lab. Alternatively, he uses the FIWARE Portal UI to do the same (e.g., with Murano backend invoking docker-compose). Usage scenario #3.4: A user wants to update his Docker environment (local or on FIWARE Lab) with the latest version of a certain 'bundle' Note: may require enancements to docker-compose Group 4: Advanced scenarios Usage scenario #4.1: A user/developer wants to provision a cluster of VMs on FIWARE Lab that would host his Docker cluster (managed with Swarm or Kubernetes) He uses corresponding Murano/Heat template to provision the VMs and to configure the Docker/Swarm/Kubernetes cluster. Usage scenario #4.2: A user/developer wants to access a global instance of a Docker service in FIWARE Lab (shared, scalable, managed), so that he doesn't need to manage the corresponding VM(s) by himself. He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab Docker API endpoint with the standard Docker tools or FIWARE-specific tools (as outlined above ). Usage scenario #4.3: A user/developer wants to manage access control for Docker images among FIWARE Lab users. He starts using the Docker Registry/Hub deployed within the FIWARE Lab. Notice that this is a very initial list -- I am sure that there are many inaccuracies and gaps. Feel free to comment. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o -------------- next part -------------- An HTML attachment was scrubbed... URL: From josemanuel.canterafonseca at telefonica.com Wed Jul 29 13:36:02 2015 From: josemanuel.canterafonseca at telefonica.com (JOSE MANUEL CANTERA FONSECA) Date: Wed, 29 Jul 2015 11:36:02 +0000 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: References: <5592F69E.4060803@telefonica.com> Message-ID: Please see responses below De: Alex Glikson > Fecha: mi?rcoles, 29 de julio de 2015, 7:39 Para: Jose Manuel Cantera Fonseca > CC: "fiware-chapter-architects at lists.fiware.org" >, "fiware-cloud-containers at lists.fiware.org" >, JUAN JOSE HIERRO SUREDA > Asunto: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Dear Jose Manuel, This is certainly a good start. Before going into specific guidelines, maybe you can outline the envisioned/recommended build/packaging/release workflow that we would expect GEi developers to follow? Let's say I am a GE owner, my code is in github (following the Developer Guidelines.. Can we use the 'fiware' repository in github?), >> You mean the ?fiware? github account? Apart from a couple of projects nobody is using it as github repos are associated to each organization or owner. For instance IDAs or Orion repos are hosted under the TID organization in Github. and I want monthly releases (at the end of each sprint, after having an internal testing cycle). What should I do? It looks like I would need scripts that build DEB packages out of my github code (is there a FIWARE packages repository that I can use?), some mechanism to invoke those scripts and populate a new version of the packages periodically (e.g., I can do it manually after each sprint) >> the kind of packages you generate is up to each Gei . The Orion example generates rpm packages for the yum repository but that?s only an example. Depending on how do you manage packaging the Dockerfile will have a different structure. Or the Docker image itself can launch the build process so we leave that up to the Gei owner. See for instance https://registry.hub.docker.com/u/bitergia/fiware-orion/dockerfile/ that?s an alternative way of generating a Docker container for Orion ? , a Dockerfile that would describe how to use those packages to build a GEi image, to have tags per release on github, and to have a mechanism that would perform periodic builds from the Dockerfile that would populate the Dockerhub repository (under FIWARE org/namespace?) with new images and respective tags (would I be able to configure automatic builds associated with the FIWARE repository in Docker Hub to automatically build images out of my github repository, or fiware github repository?). That?s done automatically for you once you link a GH repository with a Dockerhub repository. You associate once branch in GH with a tag in Dockerhub and that?s all. The rest is managed automatically by Dockerhub. At least for me such a workflow would be very helpful, to make sure we don't miss important steps in the guidelines (or prerequisites to make them easy to follow). I agree that some clarifications might be needed on the Guidelines just to make them more ?usable? particularly for people without previous experience with Docker. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab; FIWARE Cloud Leader & Architect Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: JOSE MANUEL CANTERA FONSECA > To: JUAN JOSE HIERRO SUREDA >, Alex Glikson/Haifa/IBM at IBMIL Cc: "fiware-cloud-containers at lists.fiware.org" >, "fiware-chapter-architects at lists.fiware.org" > Date: 28/07/2015 06:16 PM Subject: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker ________________________________ Dear all, In order to move forward regarding the Docker Activities related to developer experience I have drafted a first version of the Gei Dockerization (aka containerization) guidelines. It follows the same approach as the Developer Guidelines with must, should and may actions. It includes already working examples that can be used as a reference. https://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/Docker Please let us know your thoughts Many thanks All the best De: > on behalf of Juanjo Hierro > Fecha: martes, 30 de junio de 2015, 22:05 Para: Alex Glikson >, "fiware-cloud-containers at lists.fiware.org" > Asunto: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Dear all, Thanks Alex for launching these discussions tracks. Regarding point (1) as anticipated by Alex, here it is the concrete plan we aim at implementing to promote usage of docker tools by GE/SE owners and the broader developer community. 1. Make info about Docker images and instruction to setup docker containers linked to FIWARE GEris available in the FIWARE Catalogue: * We will specify a general template of a new section which will be included in the "Creating instances" tab of entries linked to FIWARE GEris. This section will be elaborate on "Deploying a dedicated GE instance using Docker technology" * Each FIWARE GEri owner will be asked to setup a Docker image for their GEs and register it in Docker Hub. This task can start in parallel to the previous one. * Each FIWARE GEri owner will be asked to update the "Creating instance" tab of the entry linked to the FIWARE GEri so that it provides instructions about deploying a dedicated instance of the GEri using docker. For this purpose, the owner will follow the template defined in step 1. 2. We will incorporate the description about how to create FIWARE GEri instances and try them using docker within the FIWARE Tour Guide for developers (which will effectively become the landing page of http://developers.fiware.org) as a mean to shorten the learning curve with FIWARE GEris * Somewhere at the beginning of the guided tour, we will explain the developer how he can setup the basic docker environment either locally or on the FIWARE Lab on which he will be able to rely to try the different FIWARE GEris (Group 1 scenarios described by Alex) * A demo application will be developed that will help the developer to try each FIWARE GEri instance deployed using docker with concrete data. Developers will be able to deploy this application also using docker. It is the intention that this application will be helpful to show how several FIWARE GEris can be used in an integrated way. * Each of the chapters of the FIWARE Tour guide for developers will integrate a "Try it yourself" section which will help the developer to create an instance of the FIWARE GEris referred in the chapter and try them. In addition, developers will be able to deploy the demo application, in order to learn from a more elaborated example how the FIWARE GEris can be used, and providing instructions about how to play with the FIWARE GEris, also "tweak" the application. * Deployment of integrated FIWARE GEris and the demo application will be made feasible using docker composition tools Jos?-Manuel Cantera will drive the implementation of these concrete actions. Regarding point 2, Bitergia has already developed a first bundle of FIWARE GEris deployable using Docker compose (as a whole or individually). They will work in extending this bundle integrating additional FIWARE GEris in collaboration with the corresponding FIWARE GEri owners. Bitergia and the ULPG will work together in the development of a first version of the demo application. Eventually, additional partners might be incorporated as additional skills and/or resources are needed. We will address first those scenarios dealing with local deployment, then we will incorporate those related to deployment on other infrastructures such as the FIWARE Lab as soon as the necessary preliminary work and/or detailed guidelines are completed (this would be done under coordination by Alex and I assume we will leverage on experience developed in FI-Content2). Feedback is welcome. Best regards, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 30/06/15 20:34, Alex Glikson wrote: Dear partners, Following the initial discussion at the containers task force, we identified two follow-on (related) discussion tracks: 1) expected FIWARE Lab user/developer experience with Docker 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker (and enable requirements derived from #1) This email refers to topic (1). I will try to summarize the initial set of usage scenarios that we may want to support. We can then iterate on this over email, and have a phone call if needed, when we feel that we are close to a convergence point. Please, notice that I've subscribed several additional people relevant for the discussion (who could provide input related to UI, developers perspectives and operations perspectives). Assumptions: Notice that a prerequisite for all the usage scenarios is that all the FIWARE GEs (and SEs) are packages as Docker images and are kept in a central repository, preferably the docker hub (under a 'to-be-created' "fiware" namespace). Also, there is an assumption that the users/developers would want to work with Docker tools as much as possible (surfacing some or all of the capabilities via the FIWARE Cloud Portal too). Juanjo will elaborate on the approach we are thinking of to promote this with GE/SE owners and the broader developer community. Group 1: Basic Docker environment setup Usage scenario #1.1: A user wants a local Docker runtime on his laptop. He follows the standard instructions on setting up a docker host (e.g., running within a VirtalBox VM, using docker-machine / boot2docker) Usage scenario #1.2: A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab (where he would then run various Docker containers). He uses the standard docker-machine tool, specifying the URI of the OpenStack Keystone in FIWARE Lab (and additional parameters, as needed). The tool creates a VM using standard OpenStack APIs (natively supported by FIWARE Lab) and configures Docker within the VM. The VM would need to have a public IP (naturally). Group 2: Basic life cycle of individual containers running GEs/SEs Usage scenario #2.1: A developer wants to publish (a version of) a GE/SE. After he is done creating the new Docker image, he pushes the new version of the GE/SE to Docker hub under corresponding FIWARE namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the image points to the new version. Usage scenario #2.2: A user/developer wants to deploy locally an instance of a certain GE/SE. He uses the standard docker CLI to locally provision a container, referring to the corresponding image at Docker hub -- e.g.: "$ docker -H boot2docker-vm:2376 run fiware/GE/cb-orion" Usage scenario #2.3: A user/developer wants to deploy an instance of a certain GE/SE within his Docker VM on FIWARE Lab He uses the standard docker CLI to provision a container, referring to the location of his Docker VM as well as the corresponding image at Docker hub -- e.g.: "$ docker -H mydocker-vm37.lab.fiware.org:2376 run fiware/GE/cb-orion". ISSUE: the user would need to open the corresponding firewall ports in his VM (same as those of the GE/SE, or following the mapping performed during container provisioning) in order to make the GE/SE accessible. Ideally, this should be done in a scalable but secure manner. An easy solution is to open up-front (during VM provisioning) a range of ports (via setting up corresponding security group). A more advanced solution is to update the security group dynamically. Usage scenario #2.4: A user/developer wants to update a container comprising certain GE/SE with the latest version recently published in the dedicated namespace of the Docker Hub. He pulls the latest version from the Docker Hub, kills the old container, and starts a new one (attaching to the same resources). Note that this would work well when the application is properly designed for Docker (e.g., the container itself is stateless), and when the previous version of the container has been provisioned manually (as in #2, #4 above) by the developer (and he knows which resources to connect to). Group 3: Support for 'bundles' of GEs/SEs that together perform a certain complex function Usage scenario #3.1: A developer wants to publish (a version of) a 'bundle' of GEs/SEs that together perform a certain complex function. He creates a 'template' (e.g., following the format of docker-compose), referring to the individual GE/SE images as well as their interdependencies (e.g., links) and other composition properties. This might be done using a text editor, a Web UI provided by the FIWARE Cloud (conceptually similar to today's UI for creation of blueprints), or other tools from Docker ecosystem. He then uploads the template to the centralized templats repository. Note: it is likely that docker-compose will be able to use Docker Registry/Hub as a repository for templates (including versioning, push/pull, etc). Usage scenario #3.2: A user wants to provision locally a set of GEs/SEs, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the local Docker URL. Usage scenario #3.3: A user wants to provision a set of GEs/SEs in FIWARE Lab, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the target Docker URL in FIWARE Lab. Alternatively, he uses the FIWARE Portal UI to do the same (e.g., with Murano backend invoking docker-compose). Usage scenario #3.4: A user wants to update his Docker environment (local or on FIWARE Lab) with the latest version of a certain 'bundle' Note: may require enancements to docker-compose Group 4: Advanced scenarios Usage scenario #4.1: A user/developer wants to provision a cluster of VMs on FIWARE Lab that would host his Docker cluster (managed with Swarm or Kubernetes) He uses corresponding Murano/Heat template to provision the VMs and to configure the Docker/Swarm/Kubernetes cluster. Usage scenario #4.2: A user/developer wants to access a global instance of a Docker service in FIWARE Lab (shared, scalable, managed), so that he doesn't need to manage the corresponding VM(s) by himself. He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab Docker API endpoint with the standard Docker tools or FIWARE-specific tools (as outlined above ). Usage scenario #4.3: A user/developer wants to manage access control for Docker images among FIWARE Lab users. He starts using the Docker Registry/Hub deployed within the FIWARE Lab. Notice that this is a very initial list -- I am sure that there are many inaccuracies and gaps. Feel free to comment. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.orghttps://lists.fiware.org/listinfo/fiware-cloud-containers ________________________________ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o ________________________________ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o ________________________________ Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o -------------- next part -------------- An HTML attachment was scrubbed... URL: From GLIKSON at il.ibm.com Wed Jul 29 14:35:08 2015 From: GLIKSON at il.ibm.com (Alex Glikson) Date: Wed, 29 Jul 2015 15:35:08 +0300 Subject: [Fiware-cloud-containers] FIWARE developer experience with Docker In-Reply-To: References: <5592F69E.4060803@telefonica.com> Message-ID: IMO, it might be very useful not to leave each GE owner so much freedom on one hand, and so many things they need to handle on their own on the other hand.. Or at least offer a 'default' which would be maximally automated and easy to use (like it is proposed to do with apiary, for example). If each GE owner would need to maintain their own github repository, docker hub repository, packages repository, etc -- it might complicate things quite a lot. This would be especially true once the community and the foundation are in place -- it could make a lot of sense to have a single githiub/dockerhub repository and a single packaging/build infrastructure (ideally also CI). Regards, Alex From: JOSE MANUEL CANTERA FONSECA To: Alex Glikson/Haifa/IBM at IBMIL Cc: "fiware-chapter-architects at lists.fiware.org" , "fiware-cloud-containers at lists.fiware.org" , JUAN JOSE HIERRO SUREDA Date: 29/07/2015 02:36 PM Subject: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Please see responses below De: Alex Glikson Fecha: mi?rcoles, 29 de julio de 2015, 7:39 Para: Jose Manuel Cantera Fonseca < josemanuel.canterafonseca at telefonica.com> CC: "fiware-chapter-architects at lists.fiware.org" < fiware-chapter-architects at lists.fiware.org>, " fiware-cloud-containers at lists.fiware.org" < fiware-cloud-containers at lists.fiware.org>, JUAN JOSE HIERRO SUREDA < juanjose.hierro at telefonica.com> Asunto: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Dear Jose Manuel, This is certainly a good start. Before going into specific guidelines, maybe you can outline the envisioned/recommended build/packaging/release workflow that we would expect GEi developers to follow? Let's say I am a GE owner, my code is in github (following the Developer Guidelines.. Can we use the 'fiware' repository in github?), >> You mean the ?fiware? github account? Apart from a couple of projects nobody is using it as github repos are associated to each organization or owner. For instance IDAs or Orion repos are hosted under the TID organization in Github. and I want monthly releases (at the end of each sprint, after having an internal testing cycle). What should I do? It looks like I would need scripts that build DEB packages out of my github code (is there a FIWARE packages repository that I can use?), some mechanism to invoke those scripts and populate a new version of the packages periodically (e.g., I can do it manually after each sprint) >> the kind of packages you generate is up to each Gei . The Orion example generates rpm packages for the yum repository but that?s only an example. Depending on how do you manage packaging the Dockerfile will have a different structure. Or the Docker image itself can launch the build process so we leave that up to the Gei owner. See for instance https://registry.hub.docker.com/u/bitergia/fiware-orion/dockerfile/ that?s an alternative way of generating a Docker container for Orion ? , a Dockerfile that would describe how to use those packages to build a GEi image, to have tags per release on github, and to have a mechanism that would perform periodic builds from the Dockerfile that would populate the Dockerhub repository (under FIWARE org/namespace?) with new images and respective tags (would I be able to configure automatic builds associated with the FIWARE repository in Docker Hub to automatically build images out of my github repository, or fiware github repository?). That?s done automatically for you once you link a GH repository with a Dockerhub repository. You associate once branch in GH with a tag in Dockerhub and that?s all. The rest is managed automatically by Dockerhub. At least for me such a workflow would be very helpful, to make sure we don't miss important steps in the guidelines (or prerequisites to make them easy to follow). I agree that some clarifications might be needed on the Guidelines just to make them more ?usable? particularly for people without previous experience with Docker. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab; FIWARE Cloud Leader & Architect Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: JOSE MANUEL CANTERA FONSECA < josemanuel.canterafonseca at telefonica.com> To: JUAN JOSE HIERRO SUREDA , Alex Glikson/Haifa/IBM at IBMIL Cc: "fiware-cloud-containers at lists.fiware.org" < fiware-cloud-containers at lists.fiware.org>, " fiware-chapter-architects at lists.fiware.org" < fiware-chapter-architects at lists.fiware.org> Date: 28/07/2015 06:16 PM Subject: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Dear all, In order to move forward regarding the Docker Activities related to developer experience I have drafted a first version of the Gei Dockerization (aka containerization) guidelines. It follows the same approach as the Developer Guidelines with must, should and may actions. It includes already working examples that can be used as a reference. https://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/Docker Please let us know your thoughts Many thanks All the best De: on behalf of Juanjo Hierro Fecha: martes, 30 de junio de 2015, 22:05 Para: Alex Glikson , " fiware-cloud-containers at lists.fiware.org" < fiware-cloud-containers at lists.fiware.org> Asunto: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Dear all, Thanks Alex for launching these discussions tracks. Regarding point (1) as anticipated by Alex, here it is the concrete plan we aim at implementing to promote usage of docker tools by GE/SE owners and the broader developer community. 1. Make info about Docker images and instruction to setup docker containers linked to FIWARE GEris available in the FIWARE Catalogue: We will specify a general template of a new section which will be included in the "Creating instances" tab of entries linked to FIWARE GEris. This section will be elaborate on "Deploying a dedicated GE instance using Docker technology" Each FIWARE GEri owner will be asked to setup a Docker image for their GEs and register it in Docker Hub. This task can start in parallel to the previous one. Each FIWARE GEri owner will be asked to update the "Creating instance" tab of the entry linked to the FIWARE GEri so that it provides instructions about deploying a dedicated instance of the GEri using docker. For this purpose, the owner will follow the template defined in step 1. 2. We will incorporate the description about how to create FIWARE GEri instances and try them using docker within the FIWARE Tour Guide for developers (which will effectively become the landing page of http://developers.fiware.org) as a mean to shorten the learning curve with FIWARE GEris Somewhere at the beginning of the guided tour, we will explain the developer how he can setup the basic docker environment either locally or on the FIWARE Lab on which he will be able to rely to try the different FIWARE GEris (Group 1 scenarios described by Alex) A demo application will be developed that will help the developer to try each FIWARE GEri instance deployed using docker with concrete data. Developers will be able to deploy this application also using docker. It is the intention that this application will be helpful to show how several FIWARE GEris can be used in an integrated way. Each of the chapters of the FIWARE Tour guide for developers will integrate a "Try it yourself" section which will help the developer to create an instance of the FIWARE GEris referred in the chapter and try them. In addition, developers will be able to deploy the demo application, in order to learn from a more elaborated example how the FIWARE GEris can be used, and providing instructions about how to play with the FIWARE GEris, also "tweak" the application. Deployment of integrated FIWARE GEris and the demo application will be made feasible using docker composition tools Jos?-Manuel Cantera will drive the implementation of these concrete actions. Regarding point 2, Bitergia has already developed a first bundle of FIWARE GEris deployable using Docker compose (as a whole or individually). They will work in extending this bundle integrating additional FIWARE GEris in collaboration with the corresponding FIWARE GEri owners. Bitergia and the ULPG will work together in the development of a first version of the demo application. Eventually, additional partners might be incorporated as additional skills and/or resources are needed. We will address first those scenarios dealing with local deployment, then we will incorporate those related to deployment on other infrastructures such as the FIWARE Lab as soon as the necessary preliminary work and/or detailed guidelines are completed (this would be done under coordination by Alex and I assume we will leverage on experience developed in FI-Content2). Feedback is welcome. Best regards, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telef?nica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 30/06/15 20:34, Alex Glikson wrote: Dear partners, Following the initial discussion at the containers task force, we identified two follow-on (related) discussion tracks: 1) expected FIWARE Lab user/developer experience with Docker 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker (and enable requirements derived from #1) This email refers to topic (1). I will try to summarize the initial set of usage scenarios that we may want to support. We can then iterate on this over email, and have a phone call if needed, when we feel that we are close to a convergence point. Please, notice that I've subscribed several additional people relevant for the discussion (who could provide input related to UI, developers perspectives and operations perspectives). Assumptions: Notice that a prerequisite for all the usage scenarios is that all the FIWARE GEs (and SEs) are packages as Docker images and are kept in a central repository, preferably the docker hub (under a 'to-be-created' "fiware" namespace). Also, there is an assumption that the users/developers would want to work with Docker tools as much as possible (surfacing some or all of the capabilities via the FIWARE Cloud Portal too). Juanjo will elaborate on the approach we are thinking of to promote this with GE/SE owners and the broader developer community. Group 1: Basic Docker environment setup Usage scenario #1.1: A user wants a local Docker runtime on his laptop. He follows the standard instructions on setting up a docker host (e.g., running within a VirtalBox VM, using docker-machine / boot2docker) Usage scenario #1.2: A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab (where he would then run various Docker containers). He uses the standard docker-machine tool, specifying the URI of the OpenStack Keystone in FIWARE Lab (and additional parameters, as needed). The tool creates a VM using standard OpenStack APIs (natively supported by FIWARE Lab) and configures Docker within the VM. The VM would need to have a public IP (naturally). Group 2: Basic life cycle of individual containers running GEs/SEs Usage scenario #2.1: A developer wants to publish (a version of) a GE/SE. After he is done creating the new Docker image, he pushes the new version of the GE/SE to Docker hub under corresponding FIWARE namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the image points to the new version. Usage scenario #2.2: A user/developer wants to deploy locally an instance of a certain GE/SE. He uses the standard docker CLI to locally provision a container, referring to the corresponding image at Docker hub -- e.g.: "$ docker -H boot2docker-vm:2376 run fiware/GE/cb-orion" Usage scenario #2.3: A user/developer wants to deploy an instance of a certain GE/SE within his Docker VM on FIWARE Lab He uses the standard docker CLI to provision a container, referring to the location of his Docker VM as well as the corresponding image at Docker hub -- e.g.: "$ docker -H mydocker-vm37.lab.fiware.org:2376 run fiware/GE/cb-orion". ISSUE: the user would need to open the corresponding firewall ports in his VM (same as those of the GE/SE, or following the mapping performed during container provisioning) in order to make the GE/SE accessible. Ideally, this should be done in a scalable but secure manner. An easy solution is to open up-front (during VM provisioning) a range of ports (via setting up corresponding security group). A more advanced solution is to update the security group dynamically. Usage scenario #2.4: A user/developer wants to update a container comprising certain GE/SE with the latest version recently published in the dedicated namespace of the Docker Hub. He pulls the latest version from the Docker Hub, kills the old container, and starts a new one (attaching to the same resources). Note that this would work well when the application is properly designed for Docker (e.g., the container itself is stateless), and when the previous version of the container has been provisioned manually (as in #2, #4 above) by the developer (and he knows which resources to connect to). Group 3: Support for 'bundles' of GEs/SEs that together perform a certain complex function Usage scenario #3.1: A developer wants to publish (a version of) a 'bundle' of GEs/SEs that together perform a certain complex function. He creates a 'template' (e.g., following the format of docker-compose), referring to the individual GE/SE images as well as their interdependencies (e.g., links) and other composition properties. This might be done using a text editor, a Web UI provided by the FIWARE Cloud (conceptually similar to today's UI for creation of blueprints), or other tools from Docker ecosystem. He then uploads the template to the centralized templats repository. Note: it is likely that docker-compose will be able to use Docker Registry/Hub as a repository for templates (including versioning, push/pull, etc). Usage scenario #3.2: A user wants to provision locally a set of GEs/SEs, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the local Docker URL. Usage scenario #3.3: A user wants to provision a set of GEs/SEs in FIWARE Lab, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the target Docker URL in FIWARE Lab. Alternatively, he uses the FIWARE Portal UI to do the same (e.g., with Murano backend invoking docker-compose). Usage scenario #3.4: A user wants to update his Docker environment (local or on FIWARE Lab) with the latest version of a certain 'bundle' Note: may require enancements to docker-compose Group 4: Advanced scenarios Usage scenario #4.1: A user/developer wants to provision a cluster of VMs on FIWARE Lab that would host his Docker cluster (managed with Swarm or Kubernetes) He uses corresponding Murano/Heat template to provision the VMs and to configure the Docker/Swarm/Kubernetes cluster. Usage scenario #4.2: A user/developer wants to access a global instance of a Docker service in FIWARE Lab (shared, scalable, managed), so that he doesn't need to manage the corresponding VM(s) by himself. He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab Docker API endpoint with the standard Docker tools or FIWARE-specific tools (as outlined above ). Usage scenario #4.3: A user/developer wants to manage access control for Docker images among FIWARE Lab users. He starts using the Docker Registry/Hub deployed within the FIWARE Lab. Notice that this is a very initial list -- I am sure that there are many inaccuracies and gaps. Feel free to comment. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener informaci?n privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilizaci?n, divulgaci?n y/o copia sin autorizaci?n puede estar prohibida en virtud de la legislaci?n vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma v?a y proceda a su destrucci?n. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinat?rio, pode conter informa??o privilegiada ou confidencial e ? para uso exclusivo da pessoa ou entidade de destino. Se n?o ? vossa senhoria o destinat?rio indicado, fica notificado de que a leitura, utiliza??o, divulga??o e/ou c?pia sem autoriza??o pode estar proibida em virtude da legisla??o vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destrui??o -------------- next part -------------- An HTML attachment was scrubbed... URL: