Testbed Taskforce 2018-07-11 minutes

Owned by Mikael Aksamit
Last updated: Jul 12, 2018 by Klaus Vilstrup Pedersen (Unlicensed)
6 min read

Date: 2018-07-11

Time: 13:00-15:00 CEST

Type of meeting: Gotomeeting

Participants and absents


Chair

Klaus Vilstrup Pedersen <kpe@difi.no>

Lefteris Leontaridis


Attendees

Aksamit, Mikael <mikael@aksamit.se>

Helger, Philip <philip.helger@brz.gv.at>

Erlend Klakegg Bergheim

Even Østvold

Edmund Gray

Jens Aabol

Fabio Vigna

Luca Chiecchio


Agenda


Topics not discussed

  1. Core requirements mentions that TB AP should be able to be used "as they are", only minimum intervention should be needed in order to support the proposed REST API. This needs to be emphasized in the scope of the non-functional requirements as encourage a high adoption by possible TB APs.

Information item

Decisions

  1. -

Action items

  1. -

Subject of discussion


Rest API

Supported actions

AS2

  • TB AP sends a transaction to a system under test (SUT)
    • Prerequisites:
      • SUT URL and AP certificate must be user provided
      • SUT must be automatically registered in a (new) TB SMP with special ID, using the Proxy AP URL and the SUT AP certificate
        • DNS timing must be considered
        • Proxy AP must find the correct URL from the Orchestration layer based on SBDH Instance ID; it must than proxy the binary stream "as is" to SUT AP URL
    • Input params:
      • SBDH - XML binary serialized (xs:base64binary)
        • Document Type ID
        • Process ID
        • Sending Participant Identifier
        • Receiver Participant Identifier (SUT participant ID)
        • Transmission Instance ID (Orchestration layer provided)
        • Payload
    • Return values:
      • None
  • TB AP send return value for transmission back to Orchestration Layer (asynchronous response to the above call)
    • Input Params:
      • Original MDN as byte[]
      • REM or some other generic XML format
    • Return values:
      • None
  • TB AP receives a transaction from a SUT AP.
    • Prerequisite:
      • Every TB AP must be registered in an SMP with its own ID, using the Proxy AP URL and the TB AP certificate
      • Each TB AP gets its own certificate
      • Each test transmission targets exactly one TB AP (and not all in parallel)
      • Each SUT AP must perform an SMP client lookup with the TB AP participant ID
      • Received AS2 message (byte[]) including the AS2 headers - should be recorded by Proxy AP (to Orchestration layer)
    • Input params:
      • SBDH values for matching with the recorded values (see prerequisites)
        • Document Type ID (must be predefined by OpenPEPPOL)
        • Process ID (must be predefined by OpenPEPPOL)
        • Sending Participant Identifier (SUT participant ID)
        • Receiver Participant Identifier (TB AP participant ID)
        • Transmission Instance ID
    • Things to be communicated to Orchestration layer
      • Transmission Instance ID as proof that message was received by TB AP
    • Return values to sending SUT AP:
      • MDN
        • MDN is proxied by the Proxy AP which records the contents to the Orchestration Layer
    • Verification of MDN received by TB AP:
      • Manually only atm

AS4

  • Basically identical to AS2
  • Problem: payload encryption
    • SUT sends to TB AP
      • Encrypted with Proxy AP Cert
      • Send to Proxy AP
      • Proxy AP decrypts Message, sends to Orchestration Layer, re-encrypts and sends to TB AP
      • TB AP creates Receipt
      • Encrypted with Proxy AP Cert
      • Send to Proxy AP
      • Proxy AP decrypts Receipt, sends to Orchestration Layer, re-encrypts and sends to SUT AP
    • TB AP sends to SUT
      • see above



*Distinguish the software as “Testbed controller”

Requirements for implementing the centralized “Testbed Controller”


In scope for Version 1.0

  1. Testbed Controller covers listed test cases and operates with a minimum of manual intervention
  2. Ability to verify compliance on a technical level in terms of the PEPPOL AS2 profile
  3. Ability to verify compliance on a technical level in terms of the PEPPOL AS4 profile
  4. Ability to verify compliance on a technical level in terms of the E-SENS AS4 profile
  5. Implemented as a web-based application with a self-service user interface. Self-identification based on PEPPOL test certificate based on PKI v3 is foreseen. Not open to everyone.
  6. Provide efficient and (best effort) scalable self-service Testbed on-boarding, for connectivity testing (not for Backbone AP integration)
  7. The main purpose of the Testbed from the viewpoint of the tester: to support developers and operations with manual processes
  8. The main purpose of the Testbed from the viewpoint of OpenPEPPOL: to check the maturity and verify the compliance of an onboarding entity
  9. The Testbed Controller is controlled (managed, maintained, operated and supported) by OpenPEPPOL operations.
  10. The Testbed Controller is being hosted by a 3rd party in a generic setup (provider) and implemented in a way to ensure minimal maintenance activities. SLA is mandatory between 3rd party host and OpenPEPPOL
  11. The Testbed should enable the usage of different PEPPOL compliant AP implementations (like Oxalis, ph-as2 ...) as Backbone APs. The same implementation can be available in different versions.
  12. The Testbed should produce a report with the result of executed test cases and which Backbone AP implementations were tested. It should be possible to assert the authenticity of the report.
  13. Testbed Controller implementation strategy should cater for scalability in scope and functionality
  14. A bullshit AP (naming (c) Anna-Lis) implementation must be created that sends out crappy documents without validation (for the negative tests)

Out of scope for version 1.0, in scope for a later version

  1. BIS document validation and content conformance
  2. SMP server onboarding test (SMP client is in scope)
  3. Local or regional requirements mandated by OpenPEPPOL Authorities.
  4. Having a history of reports sent out (in scope for future Implementations)
  5. Identify scenarios where a production certificate might make sense
  6. Handle specific HTTP header values appropriate (Connection header, special status codes, ....)

Out of scope forever (until management decides different)

  1. Hosting of 3rd party Backbone AP implementations on behalf of OpenPEPPOL are not considered as part of the Testbed operation

Testbed AccessPoint (TB AP)

  • TB AP is a virtual AP - equals the sum of all available, active and selected Backbone AP instances
  • AS2 specific settings
    • SMP registration is required
    • The SMP must be connected to the SMK
    • https with TLS 1.2 must be used (and enforced)
    • The MIC algorithm for AS2 must be SHA-1 (and foresee SHA-256 for the future)


Test cases covered by Testbed v1.0

The first version (1.0) of the Testbed will cover the following PEPPOL AS2 and AS4 profile use cases:

  1. System under test (SP AP) is receiving
    1. Positive tests
      1. TB AP sends a valid document to service provider (SP) AP using a valid certificate (should pass) (of reasonable size)
        1. Document type ID/process ID/no SMP lookup (URL must be provided by test user in the Test Controller, forwarded to Backbone APs)
          1. Tast for the Testbed Controller: it must be possible to identify which Backbone AP sent out which file - correlate Backbone AP and sent out file;
        2. Verification; SP AP send file back to Testbed AP (see 2. a. i.) (maybe a problem because AP software changes are required!)
        3. Handle different Content-Transfer-Encodings (binary, base64, 7bit etc.) - for the payload (AS2 specific)
        4. Support SHA-1 and SHA-256 as MIC algorithms (AS2 specific)
        5. Try to verify if all the AS2 MDN HTTP headers are according to the specs (if proxying is possible) (AS2 specific)
      2. TB AP sends a valid large (50 MB+) document to SP AP (should pass)
        1. Verification; SP AP sends file back in 2.a.ii
        2. Same verification steps as described above should be applied
      3. The SP AP implementation (not the instance under test!) needs to be on the list of CEF eDelivery compliant implementations (https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/eDelivery+AS4+conformant+solutions) (AS4 specific)
    2. Negative tests
      1. TB AP sends an invalid document (valid SBDH without a payload) to SP AP (should fail with negative MDN)
      2. TB AP sends an invalid document (No SBDH, no payload at all) to SP AP (should fail with negative MDN)
      3. TB AP sends an invalid document (No SBDH, valid payload only) to SP AP (should fail with negative MDN)
      4. TB AP sends an invalid document (Invalid SBDH, XML syntax error) to SP AP (should fail with negative MDN)
      5. TB AP sends an invalid document (Invalid SBDH, missing mandatory field) to SP AP (should fail with negative MDN)
      6. TB AP sends an invalid document (SBDH endpoint ID differs from the Payload endpoint ID/or document type mismatch) to SP AP (out of scope for V1)
      7. TB AP sends a valid document to SP AP using an invalid certificate (expired PEPPOL AP certificate) (should fail)
      8. TB AP sends a valid document to SP AP using an invalid certificate (revoked PEPPOL AP certificate) (should fail) - may cause problems  in practise; Error level warning only?
      9. TB AP sends a valid document to SP AP using an invalid certificate (with a non-PEPPOL certificate) (should fail)
      10. TB AP sends a valid document to SP AP using an invalid certificate (with a PEPPOL SMP certificate) (should fail)
      11. TB AP sends a valid document to SP AP using an old TLS version (e.g. TLS 1.1) (should be rejected by SSL handshake)
  2. System under test is sending
    1. Positive tests
      1. SP AP sends a valid document to TB AP using a valid certificate
        1. Sending more than one different document type might be an issue for testing with real-life-test systems (out of scope for V1)
        2. Handle different Content-Transfer-Encodings (binary, base64, 7bit etc.) - for the MDN (AS2 specific)
        3. Verify that all the AS2 HTTP headers have correct values (according to the AS2 specs and the PEPPOL profile) (AS2 specific)
      2. SP AP sends a valid large (50 MB+) document using a valid certificate
        1. Consider reasonable read timeout values
    2. Negative tests
      1. SP AP is sending a document type that is not supported by the TB AP (e.g. OrderResponse) (should fail with negative MDN/Receipt) (out of scope for V1)
        1. Verification: verify that the negative MDN was received
      2. Out of scope version 1: list of negative self-tests, that cannot be verified by the Testbed


In a long term scenario the centralized Testbed should be used for:

  1. Onboarding test (in TEST environment using SMK)
  2. Compliance test (in PRODUCTION environment using SML)
  3. Compliance monitoring (automated monitoring of deployed APs and SMPs)
  4. BIS document validation (automated Schematron validation of BIS profiles – Self test)


Requirements for the Testbed software

  • What about GITB
  • REST API needs to be defined


Next meeting: July 18th, 13:00 to 15:00 CEST


Attachments

  File Modified

PNG File Testbed Scenario.png

Jul 11, 2018 by Mikael Aksamit

PNG File image2018-6-13_13-39-11.png

Jul 11, 2018 by Mikael Aksamit