Dear All,
I guess this could be interest for you.
Best regards
Matteo
-------- Messaggio originale --------
Oggetto: [Fiware-wpa] Fwd: FI-WARE: extract executive summary draft
review report
Data: Mon, 30 Jul 2012 10:58:31 +0200
Mittente: Juanjo Hierro <jhierro at tid.es>
A: fiware-wpl at lists.fi-ware.eu <fiware-wpl at lists.fi-ware.eu>,
fiware-wpa at lists.fi-ware.eu <fiware-wpa at lists.fi-ware.eu>
Dear colleagues,
We have just received the following extract of the executive summary
draft review report.
We haven't had time to review it. But certainly the review report
is not as positive as we thought after the first year review meeting.
Overall, it is worth to highlight that the project's assestment is:
* Unsatisfactory progress (The project has failed to achieve key
objectives and/or is not at all on schedule)"
This, among other things, confirms us that the recent measurements
put in place were required.
Now, it's critical to demonstrate that we are going to deliver what
was due and that the testbed will not get delayed.
We'll come with additional comments later.
Best regards,
-- Juanjo
-------- Original Message --------
Subject: FI-WARE: extract executive summary draft review report
Date: Mon, 30 Jul 2012 08:42:05 +0000
From: <Arian.ZWEGERS at ec.europa.eu>
To: <jimenez at tid.es>, <jhierro at tid.es>
CC: <jdps at tid.es>, <mcp at tid.es>, <CNECT-ICT-285248 at ec.europa.eu>
Dear all,
Below is an extract from the draft executive summary of the draft M12
review report.
The extract may deviate from the executive summary in the final version
of the review report.
It is sent since it contains some information, e.g. about deliverable
acceptance, that you may want to know asap.
Best regards,
Arian.
======================
The objectives of the project during this period remain unchanged, but
are brought into sharp focus due to the delay of two months plus now
apparent, and the critical need to release the complete and
comprehensive Generic Enabler (GE) Open Specifications and accompanying
reference code as soon as possible in order to not risk destabilizing
the entire FI PPP.
Many primary technical deliverables expected at M12 are not yet
available, which is extremely poor progress given that they are some of
the main achievements planned for Month 12 and create the first useable
foundation for Use Cases projects to build upon. Compounding this
failure is the poor quality and incompleteness of various technical
deliverables; a quite astounding result given the resources available to
the project.
According to the Description of Work (DoW), the main achievements for
this period should be the first version of the GE open specifications,
software prototypes and related guidance material and test plan. Only
the specifications were delivered so far. However, they are inadequate
for communication and unified understanding with the Use Case projects,
are not of the required quality to serve the needs of those projects,
and do not provide sufficient basis for practical implementation by
software developers - all of which are fundamental to the rationale for
and objectives of FI-WARE. It is unclear in many of the GE
specifications which are the reusable and commonly shared functions, and
which usage areas across various sectors would utilize these GEs. There
are also no details on the protocols that support interoperability with
other GEs or third party products or how this interoperability would be
achieved. Moreover, a consolidated and consistent presentation of the
specifications is still missing.
There is a lot of confusion about what the GEs really are, or meant to
be, within the FI-WARE consortium, and most probably the UC projects,
not to mention the world at large. They are a mixed bag, including
functional descriptions, specifications of (not necessarily
well-defined) technical functions, specification of access and other
operational protocols, specification of access to other existing
"modules"/"components"/"devices" and so on, specifications to enable
interfaces between existing protocols, specifications to enable
deployment of other collection of protocols as "engines", etc.
As mentioned in the DoW, "GE Open Specifications will contain all the
information required in order to build compliant products which can work
as alternative implementations of GEs developed in FI-WARE and therefore
may replace a GE implementation developed in FI-WARE within a particular
FI-WARE Instance." However, given the current state of the GE
deliverables, there are severe doubts about how a FI-WARE instance
("platform") could be built from the GEs without a massive amount of
"imagination" and tweaking to make them work together. There are also
severe doubts about how a "Future Internet Application" could be
portable across different FI-WARE Instances that implement the (same set
of?) GEs that the Future Internet Application relies on.
The GEs as currently defined are not, by themselves, implementable in
the sense of delivering a practical software solution ("platform"). This
is compounded by the difficulty of understanding what is really meant by
a "coherent" set of GEs. Some GEs seem to have closer
inter-relationships than others.
Notwithstanding the (useful) clarifications and information orally
provided at the review meeting, the inadequate and indeed scarce
information within these specifications and the discrepancies between
the Technical Roadmap and the available GE contents lead the reviewers
to cast doubts on the project achievement so far, and its prospect.
*FI-WARE failed to meet its milestones for Month 12. *
Given the above, *technical progress of the project for the first year
is unacceptable*. Moreover, any further delay is likely to significantly
risk integrity of the entire FI-PPP and the smooth transition to Phase 2.
The consortium made a generally positive and concerted effort so far as
the review meeting is concerned. It presented itself as a credible team.
The verbal presentations made at the meeting were typically more
informative and explanatory than the written deliverables would lead
readers to expect (some material within the presentation, such as the
matrix of use cases using GEs, is highly appreciated by the reviewers).
With evident technical capability, but lack of timely and quality
delivery, *managerial action is an absolute and urgent must for an
accelerated FI-WARE recovery, and to ensure that the FI-PPP programme
will deliver meaningful innovations*.
Collaboration with Use Case projects appears to be improving markedly
thanks to key face-to-face training weeks, and enhancements to the way
the Fusion Forge system is managed. However, there are still significant
issues concerning the processing and documentation of the
requirements/backlogs. *A main conclusion from the review is that the
Use Cases requirements are still not considered as high priority by
FI-WARE*. Feedback from Use Case projects, including implications for
the technology perspective and choices made by FI-WARE, should be (more
visibly) taken into account and demonstrated as such. Traceability is
crucial for the credibility of the FI-WARE technical deliverables and
uptake beyond the consortium; and accountability is required to justify
financing from the public purse.
The recently conducted architecture training weeks should be used as a
stepping stone towards building an FI-PPP "culture" between all
Programme participants. More such sessions should be planned,
particularly around the test-bed integration deployment and use of the
DevComE toolset. For its own part, DevComE is presently a positive
highlight of the project. Integration planning for the test-bed is sound
and detailed.
The testing and the integration activities are among the project
highlights. Both are carefully planned and well specified. As the
development of the GEs is delayed, the integration activities still lack
concreteness in terms of specific scenarios in which several of the GEs
will be involved, which is a pity.
Work undertaken regarding communication and dissemination is showing
significant advancement, albeit with still substantial opportunity for
improvement.
Given the overall delay in the work plan, the postponement of many key
deliverables, and the insufficient quality (and quantity) of the
technical specifications, the reviewers consider that the resources were
not utilized effectively within the first project year.
The consortium has put reasonable effort into analysing and positioning
FI-WARE in the current market context. The exploitation within the scope
of "Smart Cities" is promising and welcome, but the Use Cases projects
should be targeted as prime users. The business strategy is however
absent, in so far that no credible and preliminary quantified business
case has been presented yet. The consortium's position that the business
case will arrive in the second year of the project is a matter of
serious concerns; for example, the reviewers have the strong impression
that nobody in the consortium has any notion of the overall amount of
investment required to take FI-WARE results to the market, and the work
presented is clearly (still) lacking substantive input from the business
departments of the industrial partners. Despite the extensive comments
in the Month 6 Review Report and notwithstanding the voluminous
deliverable D11.2.1, there is still no unified or compelling marketing
message, including no compelling unique selling proposition, of the
FI-WARE results. The individual exploitation plans of the partners are
timid. There is inadequate consideration of third party development and
SME exploitation at the business level. The globalisation dimension of
the exploitation plan is unconvincing. The consortium is reminded that
the success of FI-WARE depends on the delivery of a genuine global
solution, exploitable by partners inside and outside the consortium and
beyond Europe. In summary, *there is a glaring and alarming discrepancy
between the high ambitions of FI-WARE given in the DoW and the very
limited perspective of exploitation offered so far.*
The reviewers are disappointed that many of the recommendations made --
dating back to the Month 6 review or even earlier - have not been
sufficiently considered. Additionally, the reviewers are frustrated by
the pattern of (extremely) late submission of the majority of the
deliverables, and hastily communicated rescheduling of other
deliverables with debatable arguments. Such behaviour is not acceptable
in the business world; it is equally unacceptable in the context of
European collaborative activity, especially in view of the public
funding involved.
b. Recommendations concerning the period under review
The following deliverables require re-submission:
. D2.3.1 by Month 15 (from Month 9 review)
. D2.4.1 by Month 15 (from Month 9 review)
. D2.1.2 by Month 18
. D3.1.1 by Month 18
. D4.1.1 by Month 18
. D5.1.1 by Month 18
. D6.1.1 by Month 18
. D7.1.1 by Month 18
. D8.1.1 by Month 18
. D11.1.1 by Month 18
. D11.2.1 by Month 18
. D11.3.1 by Month 18
The following recommendations are reiterated from the Month 6 review
report (with the timeframe for R[1][2]1 adjusted in light of the Month 9
review) and were expected to be addressed in a satisfactory manner by
the Month 12 review (original recommendations re-produced in italics):
/R [1-3]1. Given FI-WARE's intent to remain domain
neutral, a comprehensive technology map must be created that clearly and
unambiguously illustrates the relationships between all Generic Enablers
to be produced by FI-WARE. This was expected to be documented in
deliverable D2.3.1 originally due Month 9, for which re-submission is
now due Month 15. /
Still to be addressed in the forthcoming resubmission of D2.3.1 due
Month 15.
/As regards the "transparent encompassing architecture" for FI-WARE as a
whole also asked for already in the first recommendation of the review
report at M3, we advise the consortium to keep in mind and log this
recommendation, and re-visit it in 2013, after the projects selected
under Phase 2 have joined the FI PPP./
/R [1-3]3. Ensure meaningful interaction with standards
bodies including Internet-related standards groups such as the IETF, the
W3C, the IEEE, the ITU-T, the OMA, and the 3GPP/2. This is expected to
be reported in deliverable D12.3.2 due Month 12 and/or deliverable
D11.4.1 due Month 9./
Still to be addressed in the forthcoming D11.4.2 due Month 15.
/R [1-3]6. Ensure there is a focus on attracting a
development community for FI-WARE, and not only within the FI-PPP, but
where possible within the partner organisations and the open community
of potential users. This is expected to be considered in relation to
deliverable D2.5.1 and reported in deliverable D12.2.2./
Still to be addressed in the forthcoming D2.5.1.
/R [1-3]7. As there is substantial reliance on external
technology sources, e.g., other FP7 projects and open source projects,
contingencies should be prepared which address what actions to take
should those projects fail to deliver, or are delayed with planned
delivery upon which FI-WARE depends. This is expected to be documented
in deliverable D1.1.2 due Month 12./
The sourcing of technology is still not sufficiently clear. Contingency
planning is still not adequately provided for or documented in D1.1.2.
Please provide an adequate plan in the online version of the Project
Management Handbook asap.
/R [1-3]8. There is a reasonable likelihood that FI-WARE
chapters will be unable to achieve all that they would like to. A risk
mitigation strategy should be put in place for this, with thought given
in advance on how the project plans to prioritise resources if
insufficient resources are available to cope with all planned work. This
is expected to be documented in deliverable D1.1.2. The consortium is
reminded that effort and funding allocations in the DoW are indicative
and not sacrosanct./
A robust risk management strategy is still missing from D1.1.2. Please
provide such a strategy in the online version of the Project Management
Handbook asap.
/R [1-3]9. Make sustained effort to enrol the support of
stakeholders from the business and marketing departments of all major
commercial partners in the project. External to the consortium,
dissemination should go considerably beyond the "traditional groups"
that are usually targeted for dissemination in FP7 projects. This
concerns both the RTD communities and the business communities at large,
within as well as outside Europe. Consumer organisations should also be
considered and inputs be sought, to make sure that the FI-PPP results
will be successfully adopted by the mass market. This is expected to be
reported in deliverable D12.2.2./
Progress has been made in relation to dissemination. The active
involvement and support of the business and marketing personnel from the
industrial project partners is still largely absent.
/R [1-3]10. Ensure that planning of the Open Calls starts
early and the Open Calls are inclusive enough to attract any prospective
submitters including specialist SMEs (without making a priori
assumptions about who might be interested in the calls). The Open Calls
should be used as a strategic opportunity for disseminating FI-WARE to
FI stakeholders and engaging their interest in FI-WARE outputs. The Open
Call processes and experience, including lessons learnt, should be
documented and assimilated by the consortium for subsequent calls, and
included in the relevant editions of deliverable D1.2.x for reporting
and auditing purposes. Please take into full account our remarks on the
management and administration of the Open Calls in Section 1a of the
previous (M6) Review Report, and clarify the rationale for the selection
of the topic(s) for the forthcoming first Open Call and to what extent
the use case projects have a voice in the topic selection in the first
and subsequent calls./
The reviewers are still awaiting the reporting of the first Open Call.
Progress to be re-assessed in relation to the planning/reporting of the
next Open Call in the forthcoming D1.3.2.
/R [1-3]11. The reviewers cannot over-emphasise the
importance of exploitation planning, including detailed documentation of
IPR management. In our view, this is also intimately linked to
enablement of third party exploitation. Please refer to our remarks in
Sections 1a and 1b of the previous (M6) Review Report. It is obvious
that high expectations are placed by the reviewers on deliverables
D11.2.1 and D2.5.1 due Month 12. Please do not treat this as a paper
exercise for satisfying the reviewers. Instead, they should be treated
as initial blueprints for realistic business models backed by genuine
commitment from especially the industrial partners./
Unsatisfactory addressed. Exploitation planning lacks credibility and
market traction. To be addressed in the resubmission of D11.2.1 due M18.
/R [1-3]12. All future deliverables should continue to be
made available to the reviewers in pdf format. They should also be
submitted on time. The consortium should investigate whether posting
such files on its website or wiki might provide value for potential
users, in addition to the wiki pages (at least those files relating to
key deliverables). Please report the findings of the investigation at
the next review./
Unsatisfactory addressed. Large numbers of deliverables not submitted on
time, yet again. Others have been rescheduled in a hasty manner. The
reviewers strongly recommend corrective actions.
Additionally, the reviewers request that project deliverables be made
available as a consolidated file for future reviews.
The consortium has no inclination to make deliverables available in a
file format, other than to the EC and reviewers for pure compliance
purposes, after evident resistance. The need to investigate the
usefulness of such files to would-be users was dismissed.
/R [1-3]13. Improve the usefulness and attractiveness of the
project website: make the information (even) more easy to find, bearing
in mind that users might not be familiar with the FI-PPP; enhance the
website as a marketing tool to would-be third parties and 'customers' of
FI-WARE results (make the website answer the question to companies not
involved in EU funded activities: why should I be interested in what
FI-WARE is doing, and what is in it for me?). Consider the use of
creditable Search Engine Optimization techniques./
Unsatisfactorily addressed -- no evidence of website improvement as a
marketing tool. Significant improvement required for Month 17.
The consortium is also encouraged to bring forward the implementation of
the FI-WARE GE portal (not in DoW, but highly welcome) currently planned
for Year 3.
/R [1-3]14. Put in place contingencies for loss of key
people from the project. This is expected to be documented in
deliverable D1.1.2./
Recommendation addressed to some degree, for example by involving WP
leaders in the management of backlog. Please provide a clear update in
the online version of the Project Management Handbook asap.
Recommendations still outstanding from the initial review of D2.2.1:
/. R2: the relationship between GEs and the Specific
Enablers needs to be clarified and documented, bearing in mind that the
former are potential candidates for standardisation in due course, and
the latter are critical from the view point of making business out of
FI-WARE results. Documentation is now expected at Month 9 with
deliverable D2.3.1./
Still to be addressed in the resubmitted version of the D2.3.1 due Month
15.
/. R9: The requested delivery schedule for the GEs, or at a
minimum indication of prioritisation, has not been presented. This is
now expected for deliverable D2.4.1 at Month 9. The information should
be made visibly available in the public domain./
Still not addressed, with mismatches between the Backlog, the Technical
Roadmap and the Open Specifications delivered; To be addressed in the
resubmitted version of the D2.4.1 due Month 15.
c. Recommendations concerning future work
R[3]15. Ensure that the quality of the GE specifications is high and
consistent. Use GE specification for WP8 as a template for all other WPs.
R[3]16. Clean up the backlog, and keep it up to date at all times.
Specific resources must be dedicated to this.
R[3]17. Focus on delivery of critical-path, high-priority (for the Use
Cases) GEs.
R[3]18. GE code releases must be synchronized with GE priorities
indicated by Use Case projects.
R[3]19. Ensure that architectural documentation clearly and
unambiguously indicates the trace of source requirements and justification.
R[3]20. Transparency and visibility in what was delivered and what is
going to be delivered by the consortium in all future FI-WARE Releases.
R[3]21. Meeting the "check points" set by the reviewers.
For the next period leading up to the M18 review, the reviewers will be
monitoring and assessing project progress against the following key
"check points", in addition to DoW compliance and assessment of the
project deliverables due:
1. Technical Paper including common usage scenarios for the GEs for wide
dissemination (incl. to Use Case projects and third parties), month 15
2. Public availability matrix of use cases using the GEs; continuous
update of the matrix, month 15 & thereafter
3. FI-WARE software release, month 15
4. Public availability of the SAP GEs in WP3, month 15 (done)
5. Testbed in operation, feedback from UC projects on using the testbed,
month 17
6. Enhancement/re-design of the current website for impact creation,
month 17
7. Hold additional architectural weeks, develop training plan for use of
the DevComE framework, consider inviting 3rd party developers, month 18
8. Develop and publish a plan for fostering developer communities, month 18
9. Live demonstration of the FI-WARE test-bed with deployed GE software,
Next review meeting
10. Presentation by senior business personnel from the main commercial
partners of the consortium (at a minimum: TID, SAP, TI, Orange/FT) on
corporate plans to bring the FI-WARE key results to the market, Next
review meeting
R[3]22. For the next period overall project resource allocation should
be reviewed to identify the specific weaknesses leading to the failings
identified in this review project; resources should be re-planned and
re-allocated to rectify the failings where necessary.
d. Assessment
Unsatisfactory progress (The project has failed to achieve key
objectives and/or is not at all on schedule)
==============================
Some other considerations:
After one year in the project, one may want to evaluate what has been
achieved, esp taking into account that 12 MEuro has been spent. A
specific feature of the first reporting period, and a direct result of
the delays, is that efforts have been spent on tasks that did not result
in submitted deliverables. The economy, effectiveness and efficiency of
these efforts can therefore not be evaluated.
More importantly, after one year in the project, one may want to
evaluate what can be realistically achieved at the end of the project:
-The status and maturity of single GEs differ a lot. More consistency
and maturity in the specification of GEs needs to be achieved. The
litmus test is that these specifications need to be sufficiently mature
and complete so that independent software developers can use them to
develop implementations that are interchangeable. Prospect: good.
-It is not clear to what extent the GE Open Specifications satisfy the
requirements from use cases, FI-WARE organisations, or otherwise. There
is a risk that at the end of the project the GE Open Specifications do
not satisfy the requirements of the use cases.
-The baseline assets of each individual GE are not clear. Partly because
of this, the roadmap is not clear. At the end of the project, there will
be reference implementations, no doubt, but it is not clear to what
extent the GE Open Specifications follow the existing baseline assets or
v.v. There is a risk that the reference implementations do not satisfy
the requirements of the use cases.
-The IPR protection of the reference implementations is unclear. At the
end of the project, the use cases need clarity on this. It is essential
that this information becomes available as soon as possible.
-Likewise, the exploitation plans of the owners of the reference
implementations are unclear. At the end of the project, the use cases
need clarity on this. It is essential that this information becomes
available as soon as possible.
-The division of budgets in FI-WARE is (largely) based on the existence
of baseline assets. The more baseline assets a partner brought into the
project, the larger its budget in FI-WARE. The advantage is that the
work does not start from scratch. It is likely that these baseline
assets are typically proprietary solutions. It can be expected that
partners will give priority to their baseline assets, further developing
these assets, and give less priority to fulfilling use case
requirements. Although access to these assets by use case projects after
the end of the FI-WARE project is addressed by clause 41, the exact
conditions for access, the availability of the assets, and their
maturity are unknown. This leaves use cases, possible FI-WARE Instance
providers, and third party application developers in an uncertain
situation, and clarity is necessary.
-There is a high risk that current developments on GE reference
implementations do not fulfil the requirements of the use cases. This
could be accepted to some extent if these reference implementations
would be based on clear requirements from the business departments of
the technology providers, and if there were clear exploitation plans for
these reference implementations. In such situation, usefulness to use
cases would be offset by wide availability and accessibility. However,
at the moment there is a high risk that at the end of the FI-WARE
project the reference implementations will not fulfil the requirements
from the use cases AND will not be accessible.
The month 18 review and the Call 2 evaluation will provide an
opportunity to analyse the situation with respect to a) the FI-WARE GEs
and their reference implementations, b) the priorities of the phase 2
use cases, and c) the commitments of the FI-WARE beneficiaries.
Regarding the FI-WARE GEs and their reference implementations,
information is needed a.o. concerning:
-The final list of GEs for which Open Specifications will be written
-The extent to which these GEs fulfil requirements (from all sources),
providing justification and accountability
-For each GE: the reference implementation(s) of the GE
-For each GE reference implementation: the baseline asset(s) upon which
the reference implementation will be based, the current status, the
owner, the relation to other GE (reference implementations)
Regarding the phase 2 use cases:
-The list of GEs the phase 2 use cases plan to use, and their priority
-The list of GE reference implementations the phase 2 use cases plan to
use, and their priority
Regarding the commitment of the FI-WARE beneficiaries:
-For each GE reference implementation: the conditions under which it
will be available for FI-PPP programme participants and third parties
beyond the lifetime of FI-WARE
With respect to the last point, stated intentions in confidential
exploitation plans are not sufficient. True commitment and therefore
true assurance of the use cases and third parties of the sustainability
of their efforts building on FI-WARE can be shown via e.g. a public
statement by the GE reference implementation IPR holders. Such a public
statement could follow lines as sketched below:
-"Our developments in FI-WARE/FI-PPP will be made available as
open-source, and/or
-Our developments in FI-WARE will be proprietary, conforming to the GE
Open Specifications, and we guarantee that they will be available on the
market as of [date] under FRAND conditions and will be available for at
least [y] years. The exact conditions will be specified within [x]
months. In case our companies determine the product will no longer be
commercially offered, the source code will be made available as open
source and donated to [xyz], and/or
-Company X is developing an implementation of a certain GE. Within
FI-WARE, an open source reference implementation of the same GE is being
developed. Company X builds a proprietary implementation, outside the
FI-WARE project, using own funds, and/or
-Our companies encourage the development of multiple implementations of
a certain GE, will develop a reference implementation for each of the
specified GEs within the context of the FI-WARE project, using public
money, and will develop additional proprietary implementations [for GEs
g,h,k] using own funds
-We commit to safeguard the evolution and nature of the GEs for at least
[x] years after the FI-PPP programme.
-Etc"
------------------------------------------------------------------------
Este mensaje se dirige exclusivamente a su destinatario. Puede consultar
nuestra política de envío y recepción de correo electrónico en el enlace
situado más abajo.
This message is intended exclusively for its addressee. We only send and
receive email on the basis of the terms set out at.
http://www.tid.es/ES/PAGINAS/disclaimer.aspx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fiware.org/private/old-fiware-tools/attachments/20120731/a043dba5/attachment.html>
-------------- next part --------------
_______________________________________________
Fiware-wpa mailing list
Fiware-wpa at lists.fi-ware.eu
http://lists.fi-ware.eu/listinfo/fiware-wpa
-------------- next part --------------
A non-text attachment was scrubbed...
Name: matteo_melideo.vcf
Type: text/x-vcard
Size: 354 bytes
Desc: not available
URL: <https://lists.fiware.org/private/old-fiware-tools/attachments/20120731/a043dba5/attachment.vcf>
You can get more information about our cookies and privacy policies clicking on the following links: Privacy policy Cookies policy