IMO, it might be very useful not to leave each GE owner so much freedom on one hand, and so many things they need to handle on their own on the other hand.. Or at least offer a 'default' which would be maximally automated and easy to use (like it is proposed to do with apiary, for example). If each GE owner would need to maintain their own github repository, docker hub repository, packages repository, etc -- it might complicate things quite a lot. This would be especially true once the community and the foundation are in place -- it could make a lot of sense to have a single githiub/dockerhub repository and a single packaging/build infrastructure (ideally also CI). Regards, Alex From: JOSE MANUEL CANTERA FONSECA <josemanuel.canterafonseca at telefonica.com> To: Alex Glikson/Haifa/IBM at IBMIL Cc: "fiware-chapter-architects at lists.fiware.org" <fiware-chapter-architects at lists.fiware.org>, "fiware-cloud-containers at lists.fiware.org" <fiware-cloud-containers at lists.fiware.org>, JUAN JOSE HIERRO SUREDA <juanjose.hierro at telefonica.com> Date: 29/07/2015 02:36 PM Subject: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Please see responses below De: Alex Glikson <GLIKSON at il.ibm.com> Fecha: miércoles, 29 de julio de 2015, 7:39 Para: Jose Manuel Cantera Fonseca < josemanuel.canterafonseca at telefonica.com> CC: "fiware-chapter-architects at lists.fiware.org" < fiware-chapter-architects at lists.fiware.org>, " fiware-cloud-containers at lists.fiware.org" < fiware-cloud-containers at lists.fiware.org>, JUAN JOSE HIERRO SUREDA < juanjose.hierro at telefonica.com> Asunto: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Dear Jose Manuel, This is certainly a good start. Before going into specific guidelines, maybe you can outline the envisioned/recommended build/packaging/release workflow that we would expect GEi developers to follow? Let's say I am a GE owner, my code is in github (following the Developer Guidelines.. Can we use the 'fiware' repository in github?), >> You mean the ?fiware? github account? Apart from a couple of projects nobody is using it as github repos are associated to each organization or owner. For instance IDAs or Orion repos are hosted under the TID organization in Github. and I want monthly releases (at the end of each sprint, after having an internal testing cycle). What should I do? It looks like I would need scripts that build DEB packages out of my github code (is there a FIWARE packages repository that I can use?), some mechanism to invoke those scripts and populate a new version of the packages periodically (e.g., I can do it manually after each sprint) >> the kind of packages you generate is up to each Gei . The Orion example generates rpm packages for the yum repository but that?s only an example. Depending on how do you manage packaging the Dockerfile will have a different structure. Or the Docker image itself can launch the build process so we leave that up to the Gei owner. See for instance https://registry.hub.docker.com/u/bitergia/fiware-orion/dockerfile/ that?s an alternative way of generating a Docker container for Orion ? , a Dockerfile that would describe how to use those packages to build a GEi image, to have tags per release on github, and to have a mechanism that would perform periodic builds from the Dockerfile that would populate the Dockerhub repository (under FIWARE org/namespace?) with new images and respective tags (would I be able to configure automatic builds associated with the FIWARE repository in Docker Hub to automatically build images out of my github repository, or fiware github repository?). That?s done automatically for you once you link a GH repository with a Dockerhub repository. You associate once branch in GH with a tag in Dockerhub and that?s all. The rest is managed automatically by Dockerhub. At least for me such a workflow would be very helpful, to make sure we don't miss important steps in the guidelines (or prerequisites to make them easy to follow). I agree that some clarifications might be needed on the Guidelines just to make them more ?usable? particularly for people without previous experience with Docker. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab; FIWARE Cloud Leader & Architect Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 From: JOSE MANUEL CANTERA FONSECA < josemanuel.canterafonseca at telefonica.com> To: JUAN JOSE HIERRO SUREDA <juanjose.hierro at telefonica.com>, Alex Glikson/Haifa/IBM at IBMIL Cc: "fiware-cloud-containers at lists.fiware.org" < fiware-cloud-containers at lists.fiware.org>, " fiware-chapter-architects at lists.fiware.org" < fiware-chapter-architects at lists.fiware.org> Date: 28/07/2015 06:16 PM Subject: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Dear all, In order to move forward regarding the Docker Activities related to developer experience I have drafted a first version of the Gei Dockerization (aka containerization) guidelines. It follows the same approach as the Developer Guidelines with must, should and may actions. It includes already working examples that can be used as a reference. https://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/Docker Please let us know your thoughts Many thanks All the best De: <fiware-cloud-containers-bounces at lists.fiware.org> on behalf of Juanjo Hierro <juanjose.hierro at telefonica.com> Fecha: martes, 30 de junio de 2015, 22:05 Para: Alex Glikson <GLIKSON at il.ibm.com>, " fiware-cloud-containers at lists.fiware.org" < fiware-cloud-containers at lists.fiware.org> Asunto: Re: [Fiware-cloud-containers] FIWARE developer experience with Docker Dear all, Thanks Alex for launching these discussions tracks. Regarding point (1) as anticipated by Alex, here it is the concrete plan we aim at implementing to promote usage of docker tools by GE/SE owners and the broader developer community. 1. Make info about Docker images and instruction to setup docker containers linked to FIWARE GEris available in the FIWARE Catalogue: We will specify a general template of a new section which will be included in the "Creating instances" tab of entries linked to FIWARE GEris. This section will be elaborate on "Deploying a dedicated GE instance using Docker technology" Each FIWARE GEri owner will be asked to setup a Docker image for their GEs and register it in Docker Hub. This task can start in parallel to the previous one. Each FIWARE GEri owner will be asked to update the "Creating instance" tab of the entry linked to the FIWARE GEri so that it provides instructions about deploying a dedicated instance of the GEri using docker. For this purpose, the owner will follow the template defined in step 1. 2. We will incorporate the description about how to create FIWARE GEri instances and try them using docker within the FIWARE Tour Guide for developers (which will effectively become the landing page of http://developers.fiware.org) as a mean to shorten the learning curve with FIWARE GEris Somewhere at the beginning of the guided tour, we will explain the developer how he can setup the basic docker environment either locally or on the FIWARE Lab on which he will be able to rely to try the different FIWARE GEris (Group 1 scenarios described by Alex) A demo application will be developed that will help the developer to try each FIWARE GEri instance deployed using docker with concrete data. Developers will be able to deploy this application also using docker. It is the intention that this application will be helpful to show how several FIWARE GEris can be used in an integrated way. Each of the chapters of the FIWARE Tour guide for developers will integrate a "Try it yourself" section which will help the developer to create an instance of the FIWARE GEris referred in the chapter and try them. In addition, developers will be able to deploy the demo application, in order to learn from a more elaborated example how the FIWARE GEris can be used, and providing instructions about how to play with the FIWARE GEris, also "tweak" the application. Deployment of integrated FIWARE GEris and the demo application will be made feasible using docker composition tools José-Manuel Cantera will drive the implementation of these concrete actions. Regarding point 2, Bitergia has already developed a first bundle of FIWARE GEris deployable using Docker compose (as a whole or individually). They will work in extending this bundle integrating additional FIWARE GEris in collaboration with the corresponding FIWARE GEri owners. Bitergia and the ULPG will work together in the development of a first version of the demo application. Eventually, additional partners might be incorporated as additional skills and/or resources are needed. We will address first those scenarios dealing with local deployment, then we will incorporate those related to deployment on other infrastructures such as the FIWARE Lab as soon as the necessary preliminary work and/or detailed guidelines are completed (this would be done under coordination by Alex and I assume we will leverage on experience developed in FI-Content2). Feedback is welcome. Best regards, -- Juanjo ______________________________________________________ Coordinator and Chief Architect, FIWARE platform CTO Industrial IoT, Telefónica email: juanjose.hierro at telefonica.com twitter: @JuanjoHierro You can follow FIWARE at: website: http://www.fiware.org twitter: @FIWARE facebook: http://www.facebook.com/pages/FI-WARE/251366491587242 linkedIn: http://www.linkedin.com/groups/FIWARE-4239932 On 30/06/15 20:34, Alex Glikson wrote: Dear partners, Following the initial discussion at the containers task force, we identified two follow-on (related) discussion tracks: 1) expected FIWARE Lab user/developer experience with Docker 2) Enhancements to FIWARE Cloud Hosting architecture to support Docker (and enable requirements derived from #1) This email refers to topic (1). I will try to summarize the initial set of usage scenarios that we may want to support. We can then iterate on this over email, and have a phone call if needed, when we feel that we are close to a convergence point. Please, notice that I've subscribed several additional people relevant for the discussion (who could provide input related to UI, developers perspectives and operations perspectives). Assumptions: Notice that a prerequisite for all the usage scenarios is that all the FIWARE GEs (and SEs) are packages as Docker images and are kept in a central repository, preferably the docker hub (under a 'to-be-created' "fiware" namespace). Also, there is an assumption that the users/developers would want to work with Docker tools as much as possible (surfacing some or all of the capabilities via the FIWARE Cloud Portal too). Juanjo will elaborate on the approach we are thinking of to promote this with GE/SE owners and the broader developer community. Group 1: Basic Docker environment setup Usage scenario #1.1: A user wants a local Docker runtime on his laptop. He follows the standard instructions on setting up a docker host (e.g., running within a VirtalBox VM, using docker-machine / boot2docker) Usage scenario #1.2: A user/developer wants to deploy a dedicated Docker VM on FIWARE Lab (where he would then run various Docker containers). He uses the standard docker-machine tool, specifying the URI of the OpenStack Keystone in FIWARE Lab (and additional parameters, as needed). The tool creates a VM using standard OpenStack APIs (natively supported by FIWARE Lab) and configures Docker within the VM. The VM would need to have a public IP (naturally). Group 2: Basic life cycle of individual containers running GEs/SEs Usage scenario #2.1: A developer wants to publish (a version of) a GE/SE. After he is done creating the new Docker image, he pushes the new version of the GE/SE to Docker hub under corresponding FIWARE namespace (e.g., fiware/GE/cb-orion). Now the 'latest' version of the image points to the new version. Usage scenario #2.2: A user/developer wants to deploy locally an instance of a certain GE/SE. He uses the standard docker CLI to locally provision a container, referring to the corresponding image at Docker hub -- e.g.: "$ docker -H boot2docker-vm:2376 run fiware/GE/cb-orion" Usage scenario #2.3: A user/developer wants to deploy an instance of a certain GE/SE within his Docker VM on FIWARE Lab He uses the standard docker CLI to provision a container, referring to the location of his Docker VM as well as the corresponding image at Docker hub -- e.g.: "$ docker -H mydocker-vm37.lab.fiware.org:2376 run fiware/GE/cb-orion". ISSUE: the user would need to open the corresponding firewall ports in his VM (same as those of the GE/SE, or following the mapping performed during container provisioning) in order to make the GE/SE accessible. Ideally, this should be done in a scalable but secure manner. An easy solution is to open up-front (during VM provisioning) a range of ports (via setting up corresponding security group). A more advanced solution is to update the security group dynamically. Usage scenario #2.4: A user/developer wants to update a container comprising certain GE/SE with the latest version recently published in the dedicated namespace of the Docker Hub. He pulls the latest version from the Docker Hub, kills the old container, and starts a new one (attaching to the same resources). Note that this would work well when the application is properly designed for Docker (e.g., the container itself is stateless), and when the previous version of the container has been provisioned manually (as in #2, #4 above) by the developer (and he knows which resources to connect to). Group 3: Support for 'bundles' of GEs/SEs that together perform a certain complex function Usage scenario #3.1: A developer wants to publish (a version of) a 'bundle' of GEs/SEs that together perform a certain complex function. He creates a 'template' (e.g., following the format of docker-compose), referring to the individual GE/SE images as well as their interdependencies (e.g., links) and other composition properties. This might be done using a text editor, a Web UI provided by the FIWARE Cloud (conceptually similar to today's UI for creation of blueprints), or other tools from Docker ecosystem. He then uploads the template to the centralized templats repository. Note: it is likely that docker-compose will be able to use Docker Registry/Hub as a repository for templates (including versioning, push/pull, etc). Usage scenario #3.2: A user wants to provision locally a set of GEs/SEs, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the local Docker URL. Usage scenario #3.3: A user wants to provision a set of GEs/SEs in FIWARE Lab, using a pre-defined template (comprising a 'bundle') He uses the standard docker-compose tool referring to the 'template' artifcat as well as the target Docker URL in FIWARE Lab. Alternatively, he uses the FIWARE Portal UI to do the same (e.g., with Murano backend invoking docker-compose). Usage scenario #3.4: A user wants to update his Docker environment (local or on FIWARE Lab) with the latest version of a certain 'bundle' Note: may require enancements to docker-compose Group 4: Advanced scenarios Usage scenario #4.1: A user/developer wants to provision a cluster of VMs on FIWARE Lab that would host his Docker cluster (managed with Swarm or Kubernetes) He uses corresponding Murano/Heat template to provision the VMs and to configure the Docker/Swarm/Kubernetes cluster. Usage scenario #4.2: A user/developer wants to access a global instance of a Docker service in FIWARE Lab (shared, scalable, managed), so that he doesn't need to manage the corresponding VM(s) by himself. He authenticates with FIWARE Lab, and starts accessing the FIWARE Lab Docker API endpoint with the standard Docker tools or FIWARE-specific tools (as outlined above ). Usage scenario #4.3: A user/developer wants to manage access control for Docker images among FIWARE Lab users. He starts using the Docker Registry/Hub deployed within the FIWARE Lab. Notice that this is a very initial list -- I am sure that there are many inaccuracies and gaps. Feel free to comment. Thanks, Alex ==================================================================================== Alex Glikson Manager, Cloud Infrastructure Solutions, IBM Haifa Research Lab Email: glikson at il.ibm.com | Phone: +972-4-8281085 | Mobile: +972-54-6466667 | Fax: +972-4-8296112 _______________________________________________ Fiware-cloud-containers mailing list Fiware-cloud-containers at lists.fiware.org https://lists.fiware.org/listinfo/fiware-cloud-containers Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción. The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it. Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://lists.fiware.org/private/fiware-chapter-architects/attachments/20150729/6b42ff0c/attachment.html>
You can get more information about our cookies and privacy policies clicking on the following links: Privacy policy Cookies policy