Date: 2018-06-13

Time: 13:00-15:00 CEST

Type of meeting: Gotomeeting

Participants and absents

Chair

Berg, Anna-Lis <anna-lis.berg@difi.no>

Attendees

Aksamit, Mikael <mikael@aksamit.se>

Berg, Hans <hans.berg@tickstar.com>

Collanus, Risto <risto.collanus@visma.com>

Helger, Philip <philip.helger@brz.gv.at>

Langøy, Bård <bard.langoy@pagero.com>

Agenda

Topics not discussed

1. -

Information items

1. -

Decisions

1. It was decided to have not more than one virtual meeting per week
2. As the size of the team is reasonable small it was decided to allow for ad-hoc meetings via Skype
3. Terminology decision: "Testbed" is the thing as a whole, "Testbed Controller" is the software to be created
4. Development is done in the name of OpenPEPPOL
5. Hosting is done by a 3rd party company appointed by OO. The hosting solution must be able to run Docker images.
6. Controlling/administration of the created software is done by OO
7. OpenPEPPOL Operations should have a possibility to view the logfiles of the hosted solution
8. It must be able to easily and automatically access the logfiles of the Backbone solutions

Action items

1. -

Subject of discussion

*Distinguish the software as “Testbed controller”

Requirements for implementing the centralized “Testbed Controller”

In scope for Version 1.0

1. Testbed Controller covers listed test cases and operates with a minimum of manual intervention
2. Ability to verify compliance on a technical level in terms of the PEPPOL AS2 profile
3. Ability to verify compliance on a technical level in terms of the PEPPOL AS4 profile
4. Ability to verify compliance on a technical level in terms of the E-SENS AS4 profile
5. Implemented as a web-based application with a self-service user interface. Self-identification based on PEPPOL test certificate based on PKI v3 is foreseen. Not open to everyone.
6. Provide efficient and (best effort) scalable self-service Testbed on-boarding, for connectivity testing (not for Backbone AP integration)
   
7. The main purpose of the Testbed from the viewpoint of the tester: to support developers and operations with manual processes
8. The main purpose of the Testbed from the viewpoint of OpenPEPPOL: to check the maturity and verify the compliance of an onboarding entity
9. The Testbed Controller is controlled (managed, maintained, operated and supported) by OpenPEPPOL operations.
10. The Testbed Controller is being hosted by a 3rd party in a generic setup (provider) and implemented in a way to ensure minimal maintenance activities. SLA is mandatory between 3rd party host and OpenPEPPOL
11. The Testbed should enable the usage of different PEPPOL compliant AP implementations (like Oxalis, ph-as2 ...) as Backbone APs. The same implementation can be available in different versions.
    
12. The Testbed should produce a report with the result of executed test cases and which Backbone AP implementations were tested. It should be possible to assert the authenticity of the report.
13. Testbed Controller implementation strategy should cater for scalability in scope and functionality

Out of scope for version 1.0, in scope for a later version

1. BIS document validation and content conformance
2. SMP onboarding test
3. Local or regional requirements mandated by OpenPEPPOL Authorities.
4. Having a history of reports sent out (in scope for future Implementations)
5. Identify scenarios where a production certificate might make sense

Out of scope forever (until management decides different)

1. Hosting of 3rd party Backbone AP implementations on behalf of OpenPEPPOL are not considered as part of the Testbed operation

Testbed AccessPoint (TB AP)

• TB AP is a virtual AP - equals the sum of all available, active and selected Backbone AP instances
• AS2 specific settings
  • SMP registration is required
  • The SMP must be connected to the SMK
  • https with TLS 1.2 must be used (and enforced)
  • The MIC algorithm for AS2 must be SHA-1 (and foresee SHA-256 for the future)

Test cases covered by Testbed v1.0

The first version (1.0) of the Testbed will cover the following PEPPOL AS2 and AS4 profile use cases:

1. System under test (SP AP) is receiving
   1. Positive tests
      1. TB AP sends a valid document to service provider (SP) AP using a valid certificate (should pass) (of reasonable size)
         
         1. Document type ID/process ID/no SMP lookup (URL must be provided by test user in the Test Controller, forwarded to Backbone APs)
            1. Tast for the Testbed Controller: it must be possible to identify which Backbone AP sent out which file - correlate Backbone AP and sent out file;
         2. Verification; SP AP send file back to Testbed AP (see 2. a. i.) (maybe a problem because AP software changes are required!)
            
         3. Handle different Content-Transfer-Encodings (binary, base64, 7bit etc.) - for the payload (AS2 specific)
         4. Support SHA-1 and SHA-256 as MIC algorithms (AS2 specific)
         5. Try to verify if all the AS2 MDN HTTP headers are according to the specs (if proxying is possible) (AS2 specific)
      2. TB AP sends a valid large (50 MB+) document to SP AP (should pass)
         1. Verification; SP AP sends file back in 2.a.ii
         2. Same verification steps as described above should be applied
      3. The SP AP implementation (not the instance under test!) needs to be on the list of CEF eDelivery compliant implementations (https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/eDelivery+AS4+conformant+solutions) (AS4 specific)
   2. Negative tests
      1. TB AP sends an invalid document (valid SBDH without a payload) to SP AP (should fail with negative MDN)
      2. TB AP sends an invalid document (No SBDH, no payload at all) to SP AP (should fail with negative MDN)
      3. TB AP sends an invalid document (No SBDH, valid payload only) to SP AP (should fail with negative MDN)
      4. TB AP sends an invalid document (Invalid SBDH, XML syntax error) to SP AP (should fail with negative MDN)
      5. TB AP sends an invalid document (Invalid SBDH, missing mandatory field) to SP AP (should fail with negative MDN)
      6. TB AP sends an invalid document (SBDH endpoint ID differs from the Payload endpoint ID/or document type mismatch) to SP AP (out of scope for V1)
      7. TB AP sends a valid document to SP AP using an invalid certificate (expired PEPPOL AP certificate) (should fail)
      8. TB AP sends a valid document to SP AP using an invalid certificate (revoked PEPPOL AP certificate) (should fail) - may cause problems  in practise; Error level warning only?
         
      9. TB AP sends a valid document to SP AP using an invalid certificate (with a non-PEPPOL certificate) (should fail)
      10. TB AP sends a valid document to SP AP using an invalid certificate (with a PEPPOL SMP certificate) (should fail)
      11. TB AP sends a valid document to SP AP using an old TLS version (e.g. TLS 1.1) (should be rejected by SSL handshake)
2. System under test is sending
   1. Positive tests
      1. SP AP sends a valid document to TB AP using a valid certificate
         1. Sending more than one different document type might be an issue for testing with real-life-test systems (out of scope for V1)
         2. Handle different Content-Transfer-Encodings (binary, base64, 7bit etc.) - for the MDN (AS2 specific)
         3. Verify that all the AS2 HTTP headers have correct values (according to the AS2 specs and the PEPPOL profile) (AS2 specific)
      2. SP AP sends a valid large (50 MB+) document using a valid certificate
         1. Consider reasonable read timeout values
   2. Negative tests
      1. SP AP is sending a document type that is not supported by the TB AP (e.g. OrderResponse) (should fail with negative MDN/Receipt) (out of scope for V1)
         1. Verification: verify that the negative MDN was received
      2. Out of scope version 1: list of negative self-tests, that cannot be verified by the Testbed

In a long term scenario the centralized Testbed should be used for:

1. Onboarding test (in TEST environment using SMK)
2. Compliance test (in PRODUCTION environment using SML)
3. Compliance monitoring (automated monitoring of deployed APs and SMPs)
4. BIS document validation (automated Schematron validation of BIS profiles – Self test)

Requirements for the Testbed software

• What about GITB
• REST API needs to be defined

Next meeting: June 27th, 13:00 to 15:00 CEST

Attachments