Current press releases and speeches

PDF adobe graphic. View the Annexes in pdf format (162kb)

Annex D to Review of the Safety and Functionality of HMNZS Canterbury

Contents | Previous

(Edited) Review of HMNZS Canterbury Functional Performance Specification

Lt Cdr C.J. Gordon, RNZN and Mr Neil Muncey (Beca Applied Technologies) were tasked to provide assistance to the Review Team to conduct a comparison of the Functional Performance Specification (FPS), Ref A, to the delivered ship and also comparison of the FPS to the Contract, Ref B.

To Provide a Comparison Between the FPS and HMNZS Canterbury.

  1. The following methodology explains how the comparison was achieved.

FPS vs. Ship

  1. The process conducted involved linking the method of verifying how a FPS capability had been delivered and what was physically installed on the ship. The mandatory or non mandatory aspect of any given FPS requirement was ignored through this phase. It was accepted that if the capability/requirement was recorded in the FPS it was required and if there was a delta it was noted. This process enabled the reviewing officers to concentrate on areas where there were perceived deltas between the FPS and delivered vessel. The reviewing officers spent the period 30 June - 4 July onboard CANTERBURY to sight, discuss and verify whether the capabilities sought in the FPS have been delivered.
  2. The mechanism to assess the Ship vs. FPS is broken down into the five categories listed and defined below:
    1. Achieved. The stated requirement, preference or desired feature of the respective FPS clause has been achieved and delivered. This includes mandatory (shall), non binding (should) and permissive requirements (desirable).
    2. Partially achieved. The majority of the sub-components contributing to the full delivery of the stated requirement, preference or desired feature of the respective FPS clause has been partially achieved and delivered. This includes mandatory (shall), non binding (should) and permissive requirements (desirable).
    3. Not Achieved. The stated requirement, preference or desired feature of the respective FPS clause has not been achieved and delivered.
    4. Not assessed. The stated requirement, preference or desired feature of the respective FPS clause was not assessed due to:
      1. Insufficient time.
      2. Equipment/systems not available.
      3. In sufficient detail from which a sound assessment could be derived.
    5. Not applicable. The information contained within the clause had no associated deliverable or quantifiable value.
  3. The key results will show:
    1. Achievement, or otherwise of the FPS with respect to all clauses irrespective of whether they are mandatory, non binding or desirable.
    2. Achievement or otherwise of the FPS with respect to each type of clause.

FPS vs. Contract Comparison

  1. This activity involved matching the individual FPS items to the Contract Specification to determine whether the stated level of requirement of the contract met, exceeded or failed to meet the FPS. Data matching was achieved by comparing the Ship Functional Baseline Specification (Tenix derived document at Ref C) to the FPS. The Ship Functional Baseline Specification provides a direct linkage between the contract and the FPS (in that order). What was required was a linkage between the FPS and the contract. Where appropriate the Contract Specification references a FPS clause although the contract specification also contains numerous clauses without direct linkage to the FPS.
  2. The comparison between the FPS and the Contract Specification was conducted by a comparison of the stated level of compliance required. In simple terms a comparison between the ‘shall’, ‘should’ and desirable’ clauses of the FPS and appropriate corresponding reference in the Contract Specification. The Ship Functional Baseline Specification document was used to provide the linkage between the two documents.
  3. The mechanism to assess FPS vs. Contract specification is broken down into the three categories listed and defined below:
    1. Required Contract compliance item exceeds FPS required compliance.
    2. Required Contract compliance achieves the FPS required compliance.
    3. Required Contract compliance does not achieve the FPS required compliance.

Limitations

  1. Due to the short duration available to conduct these comparisons it is important to note the inherent limitations of the data, conduct and reporting process. It is considered that to undertake a rigorous review, including full verification and cross referencing would take in the vicinity of 600 hours (or one hour per FPS clause). This report should not be interpreted as being definitive. While potentially lacking scientific rigour this approach is consistent with what the reviewing officers were asked to provide in terms of a ‘pragmatic’ interpretation of whether the ships delivered capability achieved what the FPS sought.
  2. In addition to the observations at paragraph 17; the inherent limitations of this report are as follows:
    1. Technical interpretation. It has not been possible for the reviewing officers to develop a detailed understanding of the myriad of regulations relating to; SOLAS, IMO, New Zealand legislation, classification society requirements, AS/NZ standards, naval regulations, military standards etc. This has necessitated a reliance on existing knowledge combined with applying a logical process discipline.
    2. MARIN Report. A lack of operational sea time by CANTERBURY required the MARIN report to form the basis of assessing sea keeping capabilities.
    3. Verification. The ability to verify delivered capability against that sought in the FPS was dependent on a strong involvement with CANTERBURY's Ships Staff to advise the reviewing officers where the delta lies between that sought in the FPS and that delivered ship.
    4. Acceptance methodology. Due to time constraints the reviewing officers have not considered the impact of the selected acceptance testing method.
    5. Contract Change Proposals. Due to time constraints the impact of Contract Change Proposals has not been considered in the assessment. This includes changes to performance criteria required of the vessel.
    6. Unresolved ship issues. These issues potentially cloud the interpretation with respect to CANTERBURY's current and delivered status. For example the status of the sea-boats and the results of the Lloyds Register Reassessment of Classification and Statutory Certification.
    7. Imposed operating restrictions. The restrictions imposed by Navy on the CANTERBURY’s operations have prevented the ship from validating capabilities which may have been reflected in this report.
    8. Sequencing. Logical sequencing would have the reviewing officers conduct the FPS vs ship comparison first. With limited time it was considered undertaking the FPS v Contract comparison first provided an opportunity for the reviewing officers to gauge the problem and attempt to plan how best to conduct the remaining activities. This time would also allow co-ordination of the ship visit phase to occur.

Results

FPS vs Ship

  1. The FPS contained 596 clauses containing a capability requirement expressed as either mandatory (shall), non binding preference (should) or permissive requirement (desirable). The results are summarised below:
    1. With respect to the total number of FPS clauses and irrespective of the expected compliance level (i.e. shall, should, desirable):
      1. 71.1% of all requirements stated in the FPS, mandatory, preferred or desired, have been achieved/delivered.
      2. 9.4% of all requirements stated in the FPS, mandatory, preferred or desired, have been partially achieved/delivered.
      3. 9.7% of all requirements stated in the FPS, mandatory, preferred or desired, have not been achieved/delivered.
      4. 9.1% of all requirements stated in the FPS, either mandatory, preferred or desired, have not been assessed for achievement/delivery.
      5. 0.7% of all requirements stated in the FPS, either mandatory, preferred or desired do not have an associated deliverable.
    2. When broken down by type of clause (i.e. shall, should, desirable):
      1. 72.6% of all mandatory clauses (shall) have been achieved.
      2. 68.9% of all non-binding clauses (should) have been achieved.
      3. 80.1% of all desired clauses (desirable) have been achieved.
  2. Appendix A provides an overall table of results.

Key Not Achieved Items

  1. Key mandatory items that do not achieve the mandatory FPS requirement are as follows:
    1. The ship has not been effectively demonstrated to be capable of sustained patrol in seas and winds associated with the top of sea state 7. The MARIN photographs show clear propeller and rudder emergence combined with excessive roll and the flooding of the RHIB alcoves. Refer to FPS 5.7.1.3 (mandatory).
    2. While the anchors can achieve the requirement that the to holding the MRV by the bow in up to 50 metres of water, with a sand and shale sea bottom, with 35 knots of wind and 4 knots of current or tidal stream and with 90% of the fitted cable deployed, there are warranty related issues with the anchoring system, specifically the winches. Furthermore the it is not uncommon for the ship to swing through large arcs (up to 140°) while at anchor. Refer to FPS clause 7.3.1.1 (mandatory).
    3. The MRV is not fitted with sufficient provisions store rooms for the total number of persons in the vessel to the limit of the vessels endurance. This applies to freezer and dry store capability in particular. To achieve the required endurance of frozen goods necessitates the use of a reefer container thereby reducing refrigerated cargo capacity. Refer to FPS clause 7.18.2.1 (mandatory).
    4. The ship does not comply with the requirements SOLAS chapter III, particularly the International Life Saving Code due to the ongoing issues surrounding certification and fitness for purpose of the RHIBs. Refer to FPS clause 7.20.1 (mandatory).
    5. Inability to automatically track and control aircraft by day and night due at ranges up 40 miles and altitudes of 10 000 feet due to inadequate radar performance. Refer to FPS clause 7.22.1.3 (mandatory).
    6. Inability of the ships sewage treatment plant to achieve a discharge that complies with MARPOL. Refer to FPS clause 9.6.1.1: (mandatory).
    7. The communications and radar direction finding equipment failed contractor sea trials and is not for purpose. This is reflected in the VRTM at Ref D. Refer to FPS clause 16.3.1.1 (mandatory).
    8. The forward looking obstacle avoidance sonar is unfit for purpose. Refer to FPS clause 16.6.1.1 (mandatory). It is a warranty item.
    9. The ship’s staff advice is the primary Armament cannot be accurately trained, elevated and fired utilising the current configuration of EOSS and searchlights. Refer to FPS clause 17.2.1.5 (mandatory). It is a warranty item.
    10. The ship has not been effectively designed to enable the safe launch and recovery of sea boats in seas and winds associated with the top of sea state 4. This is reflected in the risks associated with alcove flooding and the certification issues surrounding the RHIBs and lifting gear. Refer to FPS clause 18.3.1.4 (mandatory).
    11. The ship has not been effectively designed to transfer or recover by sea personnel, cargo and transportable equipment from the MRV to shore. The inability to operate the LCM’s due to bow ramp design and hinge cracking precludes this capability. Refer FPS clause 21.1.1 (mandatory)
  2. The items below provide a selection of ‘non-binding’ items contained within the FPS that have not been achieved. Furthermore, with the exception of paragraph 16.a. these requirements were expressed as a mandatory requirement in the Contract Specification. The following list provides typical example of ‘non-binding’ items contained within the FPS that have not been achieved:
    1. The ship has not been effectively designed to provide directional stability +/-2° of a desired heading, including with following seas to the top of sea state 6. The MARIN data indicates ship performance fails to achieve this requirement. Refer to FPS clause 5.6.1.2 (non binding).
    2. The ship has not been effectively designed to be capable of surviving and continuing with its mission after exposure to seas and winds associated with the top of sea state 9 Refer to FPS clause 5.7.1.2 (non binding).
    3. The MRV is not designed, constructed and fitted out such that the vessel is able to sustain a peak operations requirement of 35 days with the Core Complement plus trainees plus 185 soldiers embarked. Specifically this refers to the lack of installed freezer capability volume of the Dry Store. Additional freezer capability is achieved through the use of ISO reefer containers thereby reducing refrigerated cargo capacity. Refer to FPS clause 6.2.1.2 (non binding).
    4. The MRV’s sea-boats have not been designed, constructed and fitted out such that each sea-boat is self-righting. This was revealed during the RHIB accident. Refer to FPS clause 18.1.1.2 (non binding).
    5. Sea-boats cannot be launched while fully loaded due to the interference from the stropping arrangement. Refer to FPS clause 18.3.1.8 (non binding).
  3. The non binding FPS clauses listed above were mandatory requirements upon transfer to the Contract Specification.

FPS vs Contract

  1. The results of the data matching between the FPS and the Contract Specification achieved by utilising Ref D reveal the FPS is directly liked to 918 contract clauses or sub clauses are:
    1. 44% of Contract Specification compliance wording exceed the FPS requirement (e.g. contract = shall: FPS = should; and equivalent derivations).
    2. 53% of Contract Specification compliance wording meet the FPS requirement (e.g. contract = shall: FPS = shall; and equivalent derivations).
    3. 2.5% of Contract Specification compliance wording fail to meet the FPS requirement (e.g. contract = should: FPS = shall: and equivalent derivations).
  2. The raw data can be found at Table 4 of Appendix A.
  3. 26 FPS clauses do not appear to have direct linkage to a Contract Specification clause. How or why this occurred has not been determined. In summary:
    1. 11 of the clauses, not transferred to the Contract Specification, have been achieved.
    2. 3 of the clauses, not transferred to the Contract Specification, have been partially achieved.
    3. 9 of the clauses not transferred to the Contract Specification have not been delivered.
  4. Overall the Contract Specification compliance requirement (should, shall, desirable) either meets or exceeds the FPS in all but 2.5% of situations where the FPS is referenced in the Contract Specification. Note: This comparison does not reflect Contract Specification clauses where the contracted performance criteria have changed. Accordingly it is possible for a Contract Specification requirement to meet the FPS requirement when in fact the performance requirements stated in the contract differ from that originally appearing in the FPS.

Conclusions

  1. In broad terms the ship has delivered the capabilities required in the FPS. There are a number of capabilities which have not been achieved and when compared to the FPS may not have been required to be achieved, but the delivery of which, could enable or enhance the delivery of subordinate capabilities. For example sea keeping requirements have effects to boat, aviation and patrol operations.
  2. It is arguable whether CANTERBURY can achieve the sea keeping performance requirements sought in the FPS and Contract Specification. This requirement is expressed as a preference (should) in the FPS and mandatory (shall) in the contract.
  3. From observation and discussion many of the ships features show signs of poor design, although are technically compliant with the FPS. Opportunities to identify areas of poor design have been hampered by the ships restricted operating programme.
  4. The restrictions imposed on the ship, either since delivery or the RHIB accident, has limited the ability to prove capability or highlight further areas of concern. It is only through the practical employment of the vessel that many of the unintended impacts, good and bad, will become apparent.
  5. Overall, HMNZS CANTERBURY broadly achieves the majority of mandatory FPS requirements. In a number of areas the ship partially achieves the sought after capabilities, yet fails to follow through on the full delivery of the sought capability. The results show a trend, and given the ‘pragmatic’ approach to determining FPS compliance, should be interpreted holistically.

Appendix:

  1. Tables of Results

References:

  1. Function and Performance Specification: Multi-Role Vessel for the Royal New Zealand Navy dated 26 May 2003
  2. Contract No. 322-04: RNZN Projector Contract between Her Majesty the Queen in Right of New Zealand and Tenix Pty Ltd (the Contract)
  3. DID E05-00 MRV Ship Functional Baseline Specification
  4. Multi Role Vessel Verification Requirement Traceability Matrix dated 12 Nov 07

Tables of Results

Table 1: Achievement of FPS Requirements

Category


(a)
Type of clause


(b)
No of clauses per category

(c)
Percentage breakdown by category
(d)
Category Total as a percentage of all Clauses
(e)
Achievement of the FPS requirement        
  Shall 233 55.0%  
  Should 162 38.2%
  
  Desirable 29 6.8%
  
  Sub Total 424 100.0% 71.1%
Partial achievement of the FPS requirement        
  Shall 24 42.9%  
  Should 29 51.8%  
  Desirable 3 5.4%  
  Sub Total 56 100.0% 9.4%
Requirement not achieved        
  Shall 34 58.6%  
  Should 21 36.2%  
  Desirable 3 5.2%  
  Sub Total 58 100.0% 9.7%
Not assessed        
  Shall 30 55.6%  
  Should 23 42.6%  
  Desirable 1 1.9%  
  Sub Total 54 100.0% 9.1%
Not Applicable   4   0.7%
Total Clauses   596   100.0%

Table 2: Compliance with FPS Requirement by type of clause

Achievement by Category & Contractural Compliance
(a)

Requirement stipulation

 
(b)
Shall
(c)
Should
(d)
Desirable
Achieved 72.6% 68.9% 80.6%
Partially Achieved 7.5% 12.3% 8.3%
Not Achieved 10.6% 8.9% 8.3%
Not Assessed 9.3% 9.8% 2.8%
  100.0% 100.0% 100.0%

Table 3: Total Number of Clauses by Type

Breakdown of all clauses
(a)
No. of Clauses
Shall
(b)
Percentage

(c)
Total Shall 321 53.9%
Total Should 235 39.4%
Total Desirable 36 6.0%
Not Applicable 4 0.7%
  596 100.0%

Table 4: Contract Specification vs FPS

Contract clause in relation to FPS
(a)
No. of Clauses

(b)
Percentage

(c)
Definition

(d)
FPS Contract
Exceeds FPS 393 44% Should
Desirable		 Shall
Should
Equal to FPS 476 53% Shall
Should
Desirable		 Shall
Should
Desirable
Fails to meet FPS 23 2.8% Shall
Shall
Should		 Should
Desirable
Desirable
Total 892 100.0%  

Top | Previous

Page contents Accessibility (list of Access Keys) Sitemap Homepage About us Defence Policy Acquisition Activities Reports & Publications Links Contact Us Search box New Zealand Government websites homepage