[Fiware] VERY IMPORTANT: Inmediate actions and plan for Friendly testing of FIWARE and the FIWARE Lab

[Franck Le Gall]

Dear Juanjo, all,

This point is really important and the labelling ideas we are pushing forward could help as they will oblige to start with some internal quality monitoring.

As a starting point, I can  transfer some of the work we do in FISTAR. It would start with  the use of the ‘Reusability Readiness Level (RRL)” framework. This framework is a companion of the NASA TRL framework but adapted to the software. Once in place, it would become a good communication tool.

It looks at 9 dimensions being:

·         Documentation: information that describes the software asset and how to use it.

·         Extensibility: the ability of the asset to be grown beyond its current context.

·         Intellectual Property: the legal rights for obtaining, using, modifying and distributing the asset.

·         Modularity: the degree of segregation and containment of an asset or components of an asset.

·         Packaging: the methodology and technology for assembling and encapsulating the components of a software asset.

·         Portability: the independence of an asset from platform-specific technologies.

·         Standards Compliance: the adherence of an asset to accepted technology definitions.

·         Support: the amount and type of assistance available to users of the asset.

·         Verification and Testing: the degree to which the functionality and applicability of the asset has been demonstrated.

Each having to be evaluated against a scale.
As an example, in the case of the documentation dimension, the scale levels are:

·         Level 1 (Little or no internal or external documentation available): source code is available, with little or no useful internal or external documentation.

·         Level 2 (Partially to fully commented source code available): source code is available and fully commented, but no other documentation is provided. It may be challenging for a good programmer to determine how to reuse the software.

·         Level 3 (Basic external documentation for sophisticated users available): for example, a README file, a “man” page, or command line usage examples. This type of documentation would be sufficient for a sophisticated user to figure out how to use the software, but probably not a general user.

·         Level 4 (Reference manual available): reference manual provides complete documentation on use of the software, but may not be easily approached or accessed by general users. Some documentation relevant to customization is available.

·         Level 5 (User manual available): user manual allows a “normal” or general user to understand how to use and possibly customize aspects of the software.

·         Level 6 (Tutorials available): step-by-step walkthroughs of how the software is customized and used in various scenarios, demos, etc. This makes it very easy to understand/teach the software and use it in a new project.

·         Level 7 (Interface guide available): documentation describes how to customize and interface the software with other software, programmatic interfaces, APIs, etc., so that it can more easily be embedded in a larger system.

·         Level 8 (Extension guide and/or design/developers guide available): an extension guide provides information on how to customize and add to the software, add plug-ins and the like, use internal programming “languages”, etc. A design/developers guide provides a description of internals, design documentation, internal documentation, etc. that is sufficient for someone “skilled in the art” to contribute to the development of the software or take over maintenance of the software.

·          Level 9 (Documentation on design, customization, testing, use, and reuse is available): all stages of the software engineering lifecycle are fully documented. This includes design and review artifacts, testing artifacts, customization, and regression tests. The documentation provided is easy to read/access and is appropriate for different categories of users.

Each published version of FIWARE GEs should probably target 8 or 9
I could further document this as part of T3.2 and suggest to run such RRL evaluation for all published versions of GERis, and monitor progresses over time.

This RRL framework misses important dimensions such as the quality of experience (QoE) of the GE/GERi’ users but is a starting point. I have some ideas for QoE evaluation as well but they will have to be run through the accelerators, using the coaches connections. The Friendly-testers could be the beta tester of the QoE evaluation.

Feedback welcome,

Kind regards
Franck




De : fiware-bounces at lists.fi-ware.org [mailto:fiware-bounces at lists.fi-ware.org] De la part de Juanjo Hierro
Envoyé : vendredi 3 octobre 2014 07:42
À : fiware at lists.fi-ware.eu
Cc : fiware-chapter-leaders at lists.fi-ware.org; fiware-chapter-architects at lists.fi-ware.org
Objet : [Fiware] VERY IMPORTANT: Inmediate actions and plan for Friendly testing of FIWARE and the FIWARE Lab

Dear all,

  The FIWARE Accelerator projects and the EC have expressed their strong concerns about quality of FIWARE GEs, associated documentation as well as stability of the FIWARE Lab.   It is not just for free, they are receiving continuous complains from some users that are approaching FIWARE for their first time (and, as always, it is the voice of those who rant that is heard).   Of course, the report produced by HLRS (attached for your convenience if your chapter leader didn't share it with you already), despite we know that is not correct in many aspects doesn't help too.

  All this requires immediate and executive action.   This is the plan:

  *   All FIWARE GE owners, under supervision of the corresponding chapter leaders should complete, if not already, all taks that are needed to solve issues raised in the HLRS report:
     *   Fix broken links in entries of the FIWARE Catalogue
     *   Make sure that downloadable software is in the Downloads tab
     *   Make sure that associated documentation (relevant specs as well as manuals/guides and tutorials) are available in the Documentation tab and map the software
     *   For all the FIWARE Cloud GEris, GEris linked to the Business Framework, the Security Monitoring GEri and the OFNIC GEi, add the following note in the Overview:
        *   IMPORTANT NOTE: This GE reference implementation product is only of interest to potential FIWARE instance providers and therefore has been used to build the basic FIWARE platform core infrastructure of the FIWARE Lab.   If you are an application developer, you don't need to create a complete FIWARE instance locally in order to start building applications based on FIWARE.   You may rely on instances of FIWARE GEris linked to the Data/Media Context Management chapter, the IoT Services Enablement chapter, the Advanced Web-based User Interface chapter or some GEris of the Applications/Services Ecosystem and Delivery Framework chapter (WireCloud) as well as the Security chapter (Access Control).   Those instances are either global instances or instances you can create on the FIWARE Lab but can also be instance you can create by means of downloading, installing and configuring the corresponding software in your own premises.
     *   Make sure that available videos and webinars are up to date.

  *   An urgent Task Force will be created recruiting "friendly testers" that will carry out a general testing of both the FIWARE Lab as well as those FIWARE GEris that it makes sense to download and deploy locally.
     *   This Task Force will last until end of the year.    Organizations providing friendly testers will be funded under the FI-WARE project (which is the project that cover the first phase of FIWARE and FIWARE Lab development and where there is still some funding pending of reallocation)
     *   We have activated the University of Las Palmas whose entire team plus two additional engineers will be devoted to this task until things get reasonably estabilized.   We will devote 10.000 € of funding from the FI-WARE project for this purpose.
     *   Telefónica will transfer part of its funding to Bitergia under FI-Core to recruit three resources that will be fully devoted to this task until the end of the year.  Partners who also volunteer to transfer part of their funding in FI-WARE or FI-Core to other organizations that may also devoted to this task are welcome.
     *   Friendly testing will be coordinated by Telefónica
     *   If someone knows about a reliable organization we can involve in this task force and can perform a good job as friendly testers please let us know so that we can evaluate the proposal and assign the necessary funding under the FI-WARE umbrella.   Please note that incorporating them should be rather agile, i.e., they can be easily incorporated as partners through an amendment to the contract.   Also note that this would be funding of a RTD activity, so not 100% funded.   Partners that are owners of FIWARE GEris are excluded.  Subcontracting is an option you can propose, but then you have to explain who will address covering the part that is not covered by the funding.
     *   FIWARE GEri owners should pay attention to solve issues reported through the channels that have recently been established (check previous email on the matter) and assign those maximum priority, which may impact planning of the Sprint under way.   Such impact will be measured at the end of each Sprint so that user stories or work items which couldn't be covered as initially planned are planned in the next Sprint.   We will come with more details on the process very soon.

  *   A task for continuous "Friendly Testing" will be defined in FI-Core to be performed by a partner to be selected by agreement of FIWARE chapter leaders.   500-600 K€ of funding will be deducted from the funding assigned to activities linked to development of Live demos, development of FIWARE GEris as well as development of FIWARE Ops tools.   This was a measure that we initially intend to put under voting in the consortium but the EC has asked to take inmediate executive action and it hast remind us that this measure could be considered as part of the measures required to be implemented as part of the negotiations that were still not closed despite the contract was signed.    Nevertheless, it is clear that this activity is needed and will only make the life of all of us easier.   This task will be either under overall Technical Coordination WP in FI-Core or under coordination of FIWARE Lab activities.

  A message describing the new approach looking for more agility regarding software updates (including updates of accompanying documentation as well as info provided in the Catalogue) will follow but that one will be sent to the whole FIWARE (FI-Core) partners.   Please stay tuned.

  Your collaboration is crucial.  Please don't scrimp on resources to be assigned to solve identified issues.

  Additional constructive proposals are welcome.

  Best regards,



-- Juanjo

-------------

Telefonica - Product Development and Innovation (PDI)

website: www.tid.es<http://www.tid.es>

email: juanjose.hierro at telefonica.com<mailto:juanjose.hierro at telefonica.com>

twitter: twitter.com/JuanjoHierro



FI-WARE (European Future Internet Core Platform) Coordinator

and Chief Architect



You can follow FI-WARE at:

  website:  http://www.fi-ware.org

  facebook: http://www.facebook.com/pages/FI-WARE/251366491587242

  twitter:  http://twitter.com/FIware

  linkedIn: http://www.linkedin.com/groups/FIWARE-4239932

________________________________

Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción.

The information contained in this transmission is privileged and confidential information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it.

Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/fiware/attachments/20141004/44f3c200/attachment.html>